This directory is a complete, reproducible Ascend NPU inference candidate for the
pinned upstream model
HumanCompatibleAI/ppo-CartPole-v1.
08b61dd5dca1ee9f1a0f2d7a0fcf7740b0af5dacppo-CartPole-v1.zip — 139331 bytes — SHA256
9bc70c310a4e48d5695193a20d481540ff2ace5ee7f7fa91283ed656c9a9460cPPO with MlpPolicy (ActorCriticPolicy).torch_npu (torch 2.9.0+cpu + torch_npu 2.9.0, CANN-matched).npu:3 (Ascend910B4-1).[0, 0, 0, 0] → action / value / log_prob.The policy/model forward runs wholly on npu:3. The CPU only orchestrates Python,
deserializes the checkpoint, and preprocesses the observation. There is no CPU offload
and no CPU fallback; any NPU initialization, placement mismatch, or execution error makes
the process exit nonzero.
| Field | Value |
|---|---|
| Model ID | HumanCompatibleAI/ppo-CartPole-v1 |
| Source | https://huggingface.co/HumanCompatibleAI/ppo-CartPole-v1 |
| Pinned revision | 08b61dd5dca1ee9f1a0f2d7a0fcf7740b0af5dac |
pipeline_tag | reinforcement-learning |
library_name | stable-baselines3 |
| NPU tags | ascend, npu, torch_npu, hardware: NPU |
| Algo / policy | PPO / MlpPolicy (ActorCriticPolicy, MLP [64, 64]) |
| Observation space | Box(-4.8…, 4.8…, (4,), float32) (CartPole-v1) |
| Action space | Discrete(2) |
| Reported reward | mean_reward = 500.00 +/- 0.00 (model-index, from upstream card) |
The upstream model card declares no license field in its YAML front matter. This was
verified directly from the Hugging Face Hub model metadata (cardData contains no license
key) on 2026-08-22. Accordingly the front matter above uses license: other to denote a
non-standard / undeclared license, and the upstream card's own tag set is preserved. The
weights are treated as license-not-declared; the Stable-Baselines3 library itself is
MIT-licensed, but that does not license the model weights. Do not assume a permissive license
for redistribution without upstream confirmation.
Tested in this container (/root/ascend-model-pipeline/jobs/rl26/ppo-cartpole-v1/ppo-cartpole-v1-npu):
| Component | Version / value |
|---|---|
| OS | Linux 5.10.0 .oe2203sp4.aarch64 (openEuler, aarch64) |
| Python | 3.11.14 |
| CANN | 8.5.1 (V100R001C25SPC002B220) |
| torch | 2.9.0+cpu (image-provided, CANN-matched — do not replace) |
| torch_npu | 2.9.0 (image-provided, CANN-matched — do not replace) |
| stable-baselines3 | 2.7.0 |
| gymnasium | 1.2.0 |
| numpy | 1.26.4 |
| cloudpickle | 3.1.2 |
| npu-smi | 25.5.1 |
| Hardware | Ascend910B4-1 (8 devices; this candidate uses physical npu:3) |
requirements.txt intentionally lists only non-torch runtime dependencies. The
CANN-matched torch / torch_npu pair is pre-installed in the image and must never be
replaced by ordinary PyPI wheels.
All pip installs below use the Aliyun PyPI mirror only:
# 1. Load the CANN environment (idempotent).
source /usr/local/Ascend/ascend-toolkit/set_env.sh
# 2. Install only the non-torch runtime dependencies from Aliyun.
pip install -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple/Equivalently, pin the mirror for the whole shell:
export PIP_INDEX_URL=https://mirrors.aliyun.com/pypi/simple/
pip install -r requirements.txt⚠️ Never install
torchortorch_npufrom PyPI. They are provided by the image as a CANN-matched pair (torch 2.9.0+cpu+torch_npu 2.9.0) and must stay untouched.
# 1. Load the CANN environment (required for NPU runtime).
source /usr/local/Ascend/ascend-toolkit/set_env.sh
# 2. Sanity-check the NPU devices.
npu-smi info
# 3. Run the deterministic NPU inference (device npu:3, 8 s hold window for npu-smi
# PID correlation; log is teed to logs/npu_inference.log).
cd /root/ascend-model-pipeline/jobs/rl26/ppo-cartpole-v1/ppo-cartpole-v1-npu
HOLD_SECONDS=8 ./run_inference.sh
# 4. Inspect the structured result.
cat logs/npu_result.jsonrun_inference.sh sources set_env.sh, keeps the physical device mapping
(ASCEND_RT_VISIBLE_DEVICES defaults to 0,1,2,3,4,5,6,7 so npu:3 == physical device 3),
and exits with the Python exit code.
Command:
cd /root/ascend-model-pipeline/jobs/rl26/ppo-cartpole-v1/ppo-cartpole-v1-npu && HOLD_SECONDS=8 ./run_inference.shExit code: 0. PID: 3156813 (correlated live with npu-smi on physical npu:3).
Actual output (logs/npu_inference.log):
2026-08-22T08:07:48.010+00:00 === ppo-CartPole-v1 NPU inference candidate ===
2026-08-22T08:07:48.010+00:00 source_revision=08b61dd5dca1ee9f1a0f2d7a0fcf7740b0af5dac route=Stable-Baselines3 PPO device=npu:3 pid=3156813
2026-08-22T08:07:49.353+00:00 NPU available: devices=8 requested=npu:3 name=Ascend910B4-1
2026-08-22T08:07:49.353+00:00 artifact ok: ppo-CartPole-v1.zip bytes=139331 sha256=9bc70c310a4e48d5695193a20d481540ff2ace5ee7f7fa91283ed656c9a9460c
2026-08-22T08:07:49.438+00:00 checkpoint loaded to npu:3 in 0.0847s (deserialize only, no forward)
2026-08-22T08:07:49.438+00:00 policy param devices: ['npu:3']
2026-08-22T08:07:49.438+00:00 policy buffer devices: []
2026-08-22T08:07:49.439+00:00 input obs=[0.0, 0.0, 0.0, 0.0] dtype=torch.float32 device=npu:3 shape=[1, 4]
2026-08-22T08:07:49.439+00:00 NPU_READY pid=3156813 device=npu:3 marker=08b61dd5 ts=2026-08-22T08:07:49.439606+00:00 hold=8.0s
2026-08-22T08:07:49.439+00:00 holding 8.0s (external wrapper can run npu-smi now)...
2026-08-22T08:07:57.983+00:00 npu-smi proc evidence: rc=0
NPU ID : 3
Chip Count : 1
Process id:3156813 Process name:python Process memory(MB):116
Chip ID : 0
2026-08-22T08:07:58.195+00:00 forward ok in 0.211901s action=[1] value=[[49.979976654052734]] log_prob=[-2.86102294921875e-06]
2026-08-22T08:07:58.196+00:00 NPU_DONE pid=3156813 device=npu:3 marker=08b61dd5 ts=2026-08-22T08:07:58.196463+00:00
2026-08-22T08:07:59.303+00:00 RESULT OK exit_code=0 total=10.186s -> logs/npu_result.json
EXIT_CODE=0Key result values (all on npu:3):
| Output | Value | Shape | dtype |
|---|---|---|---|
action | 1 | [1] | int64 |
value | 49.979976654052734 | [1, 1] | float32 |
log_prob | -2.86102294921875e-06 | [1] | float32 |
Placement evidence in logs/npu_result.json (top level): engine: torch_npu,
npu_available: true, npu_device_name: Ascend910B4-1, source_revision: 08b61dd5…,
input_device: npu:3, model_device: npu:3, output_device: npu:3, exit_code: 0.
Rendered from the real final transcript / sanitized log / result JSON (no fabricated values):
assets/agent_workflow.png — agent workflow: read-only upstream routing, pinned-revision
checkpoint verification, NPU init, NPU forward, and result JSON.assets/npu_device_call.png — live npu-smi device / process call on physical npu:3
(PID correlation during the hold window).assets/model_result.png — final model result: action / value / log_prob, shapes, devices,
versions, timing and PID.assets/modelagent_live_session.png — sanitized excerpts from the real ModelAgent review
conversation, followed by the independently rerun final PID and acceptance result.evidence/modelagent_summary.log — detailed sanitized ModelAgent interaction record and
its relationship to the fresh independent NPU acceptance evidence.logs/strict_checkpoint_audit.log — independent state-dict audit: all 12 source tensors
exactly matched the NPU-loaded policy, with no model forward executed by the audit.torch_npu casts double tensors to float32
and emits a non-fatal [W822] warning during the first forward. Outputs are correct float32.learning_rate / clip_range / lr_schedule as Python 3.8 code objects that cannot be
cloudpickled under Python 3.11. These are training-time hyperparameters (not weights) and are
replaced through SB3 custom_objects with the documented values (0.001, 0.2,
ConstantSchedule(0.001)). The real policy.pth weights are loaded unchanged.PPO warns that it is "primarily intended to run on the CPU"
when placed on a non-CPU accelerator during training; this candidate intentionally runs
inference on NPU, so that specific misleading message is filtered.npu:3. On another host with a
different ASCEND_RT_VISIBLE_DEVICES mapping, verify npu:3 still selects the intended
physical device.[0, 0, 0, 0]./work/pipeline/models/rl26/ppo-cartpole-v1/ppo-CartPole-v1.zip); it is not copied into
this delivery.