#NPU
HumanCompatibleAI/ppo-seals-CartPole-v0 is a PPO (Proximal Policy Optimization) agent trained with the RL Zoo and stable-baselines3 on the seals/CartPole-v0 environment. It learns to balance a pole on a cart by choosing between two discrete actions (push left / push right).
MlpPolicy)ActorCriticPolicy (MLP 64-64) with separate actor (action_net) and critic (value_net) heads[cart_pos, cart_vel, pole_angle, pole_ang_vel]This repository adapts the real trained PPO policy to run on a single Ascend NPU (npu:0). All policy parameters and every input/output tensor are explicitly placed on npu:0; inference.py loads the real weights from the local checkpoint directory and produces the real policy action (action logits, probabilities and value estimate).
| Field | Value |
|---|---|
| Route | classic-rl-policy (SB3 PPO) |
| Observation dim | 4 (cart_pos, cart_vel, pole_angle, pole_ang_vel) |
| Action dim | 2 discrete (CartPole push left/right) |
| Actor | Linear(4→64) → ReLU → Linear(64→64) → ReLU → Linear(64→2) |
| Critic | Linear(4→64) → ReLU → Linear(64→64) → ReLU → Linear(64→1) |
| Output | action logits, action probabilities, state value, deterministic action |
| Sampler | deterministic (argmax) for verification |
| Seed (training) | 705888933 |
The model is evaluated on fixed, deterministic CartPole observation batches (real observations from the environment space, no random initialization at verification time). inference.py uses a fixed 5-observation batch of float32 CartPole state vectors.
The weights are loaded from the local checkpoint directory (PPO_SEALS_LOCAL_DIR) if provided, otherwise downloaded from HuggingFace Hub (HF_ENDPOINT mirror) on first run.
npu:0), CANN 8.5.1 / Ascend Toolkitpip install -r requirements.txtpython inference.pyTo load weights from a local directory instead of downloading:
PPO_SEALS_LOCAL_DIR=/path/to/model python inference.pyAll computation runs on npu:0 (Ascend910_9362). The 5-observation batch produces:
(5, 2), dtype float32, device npu:0[0, 1, 1, 0, 1](5, 1), device npu:0| Metric | Value |
|---|---|
| Max abs diff (action logits) | 2.98e-07 |
| Max abs diff (value) | 3.05e-05 |
| Tolerance | atol=1e-4, rtol=1e-3 |
| Deterministic actions match | True |
| All outputs finite | True |
| Result | PASS |
| Metric | Value |
|---|---|
| Avg latency | 0.31 ms |
| Min / Max | 0.29 / 0.34 ms |
| p50 / p90 / p95 | 0.30 / 0.32 / 0.34 ms |
| Throughput | ~16,385 decisions/s |



replay.mp4 and rollouts.npz included in the upstream repo are demonstration artifacts and are not used for NPU inference.