HumanCompatibleAI/ppo-Pendulum-v16a0fe838119427278252dde3bf723ed7a3266df5reinforcement-learning (PPO MlpPolicy, continuous-control)npu:0policy.pth (40382 bytes, 13 tensors, MlpPolicy 64x64 tanh){
"torch_version": "2.9.0+cpu",
"torch_npu_version": "2.9.0.post1+gitee7ba04",
"torch_npu_available": true,
"device_name": "Ascend910_9362",
"device_count": 2,
"python_version": "3.11.14",
"dtype": "float32",
"model_dir": "/opt/atomgit/adapt-npu-agent/.cache_hf/models--HumanCompatibleAI--ppo-Pendulum-v1/snapshots/6a0fe838119427278252dde3bf723ed7a3266df5",
"hf_endpoint": "https://hf-mirror.com"
}安装:pip install -r requirements.txt
默认执行:python inference.py
预期输出包含:模型/revision/route、backend torch_npu、device npu:0、dtype float32、输入摘要(8 obs)、动作摘要与同步耗时、PASS/FAIL。
{
"env": "Pendulum-v1",
"observation_dim": 3,
"observation_names": ["cos(theta)", "sin(theta)", "theta_dot"],
"observation_range": {"cos_theta": [-1,1], "sin_theta": [-1,1], "theta_dot": [-8,8]},
"action_dim": 1,
"action_range": [-2.0, 2.0],
"batch": 8,
"horizon": 1,
"coordinate_frame": "torque",
"dtype": "float32",
"revision": "6a0fe838119427278252dde3bf723ed7a3266df5",
"observations": [
[1.0, 0.0, 0.0],
[0.0, 1.0, 0.0],
[-1.0, 0.0, 0.0],
[0.0, -1.0, 2.0],
[0.707, 0.707, -3.0],
[-0.707, 0.707, 5.0],
[0.0, 1.0, -8.0],
[0.0, -1.0, 8.0]
]
}观测契约:Pendulum-v1 原生观测为 [cos(theta), sin(theta), theta_dot],theta ∈ [-π, π], theta_dot ∈ [-8, 8]。无需归一化 (normalize=False)。
{
"output_shape": [8, 1],
"output_dtype": "float32",
"output_device": "npu:0",
"cpu_actions": [-0.2673841714859009, -2.0, -1.9082978963851929, 2.0, 0.8248248100280762, -2.0, 2.0, -2.0],
"npu_actions": [-0.2673843204975128, -2.0, -1.9082978963851929, 2.0, 0.8248245120048523, -2.0, 2.0, -2.0],
"finite": true,
"bounds_ok": true,
"action_range": [-2.0, 2.0],
"horizon": 1,
"action_dim": 1,
"coordinate_frame": "torque",
"sampler": "deterministic_mean"
}动作语义:PPO 确定性均值,经 torch.clamp(..., -2, 2) 钳位到 Pendulum 扭矩边界。8 个 batch 覆盖直立/水平/倒置及极端角速度。
{
"passed": true,
"atol": 0.0001,
"rtol": 0.001,
"max_abs": 2.980232238769531e-07,
"mean_abs": 6e-08,
"tolerance": 0.0021,
"max_ref": 2.0,
"shape": [8, 1],
"elements": 8,
"finite": true,
"bounds_ok": true
}验证脚本:python scripts/validate_policy_outputs.py 比较完整动作张量,阈值 atol=1e-4, rtol=1e-3,仅比较有策略语义的连续动作,未使用环境 reward 替代。
{
"avg_ms": 0.1805,
"min_ms": 0.1499,
"max_ms": 0.2485,
"p50_ms": 0.1705,
"actions_per_s": 44311.14,
"decisions_per_s": 5538.89,
"batch": 8,
"horizon": 1,
"action_dim": 1,
"dtype": "float32",
"device": "npu:0",
"warmup": 3,
"stable_runs": 10
}计时方式:每次推理前后调用 torch.npu.synchronize(),单独报告首轮编译 (≈130 ms) 与预热后延迟。批量 8 观测同步吞吐 actions/s 与 decisions/s。



三张图片均由 xterm.js 根据本次真实日志生成。
#NPU #Ascend #Ascend910 #ReinforcementLearning #PPO #Pendulum