offline-rl / discrete DQN (SB3 训练权重适配为 NPU 离散动作预测)输入观测 (observation) — LunarLander-v2 连续状态向量:
[batch, 8] float32 (batch=4 测试)x, y, vx, vy, angle, angular velocity, leg1 contact, leg2 contactnpu:0 (浮点观测张量在 NPU 上计算)normalize=False),直接喂入 FlattenExtractor输出动作 (action/Q-values):
raw_output: Q-values 形状 [batch, 4] float32, action_dim=4 离散动作空间 (0=不操作, 1=左引擎, 2=主引擎, 3=右引擎)ActionPlan: action=argmax(q_values, dim=1) ,horizon=1,coordinate_frame=discreteLinear(8->256) -> ReLU -> Linear(256->256) -> ReLU -> Linear(256->4) , 参数约 68k (557KB policy.pth)npu:0, device_count=2, torch.npu.is_available()=True, npu-smi info OK)npu:0 (next(model.parameters()).device == npu:0)npu:0npu:0 (q_values.device == npu:0)torch.npu.synchronize() 包围每次推理/tmp/hf_cache/models--sb3--dqn-LunarLander-v2/snapshots/<rev>/dqn-LunarLander-v2/policy.pth,CPU/NPU 共用权重、观测、dtype、采样参数样例输出 (NPU, batch=4):
sample 0: q=[2.7858, 2.6612, 2.8851, 3.0249] action=3
sample 1: q=[56.5517, 55.8747, 55.8580, 55.9696] action=0
sample 2: q=[36.9818, 33.8246, 34.9999, 36.4804] action=0
sample 3: q=[78.1302, 78.7737, 77.0865, 78.6620] action=1shape (4,4), dtype float32, 全部 finite,无 NaN/Inf。
atol=1e-4, rtol=1e-3 (tolerance = 0.07887)max_abs=1.91e-05, mean_abs=5.22e-06, finite=true, passed=truevalidate_policy_outputs.py --cpu /tmp/cpu_q.json --npu /tmp/npu_q.json --atol 1e-4 --rtol 1e-3 → {"count":16,"passed":true,...}136.79 +/- 42.72 仅作参考,不替代数值一致性。0.139ms / 0.134ms / 0.153ms / 0.135ms (batch=4, float32, 含同步)torch.npu.synchronize() 计时, batch=4, obs_shape=(4,8), dtype float32, 包含预处理(无)与 argmaxpython inference.py默认从本地固定权重加载,打印 model/revision、route、backend、device、dtype、输入摘要、Q-values/动作摘要、同步耗时及 PASS/FAIL。
npu:0 即可,权重未量化,保持 float32assets/agent_workflow.png — 侦察、资源、加载、NPU 推理、一致性与性能assets/npu_device_call.png — npu-smi、NPU availability、device name、model/input/output deviceassets/model_result.png — python inference.py 真实 Q-values、设备与耗时inference.py
readme.md
requirements.txt
assets/agent_workflow.png
assets/npu_device_call.png
assets/model_result.pngtorch.npu.is_available()==False 或出现 .cpu() fallback,返回 CPU_FALLBACK 失败,不生成 PASS 图。UNSUPPORTED_ARCHITECTURE。