
model.safetensors 865MB + preprocessor/postprocessor safetensors, FP32, ~0.5B 参数observation.state [6] (MEAN_STD归一), 语言指令 task (tokenizer HuggingFaceTB/SmolVLM2-500M-Video-Instruct, max_length 48)action [1, 50, 6] (50步, 6维关节/末端控制), 经 postprocess 反归一, 有限浮点make_pre_post_processors 含 rename_observations, to_batch, smolvla_new_line, tokenizer (max 48, right padding), device_processor, normalizer (STATE MEAN_STD, VISUAL IDENTITY, ACTION MEAN_STD)workspace_npu/models/lerobot_smolvla_base (865MB safetensors, 本地加载 local_files_only=True)smolvlm_with_expert.py 中 torch_dtype="bfloat16" 改为 "float32" 以支持 NPU (searchsorted 不支持 BF16), 详细见第7节pip install -r requirements.txt
# torch==2.9.0
# torch_npu==2.9.0.post1
# lerobot[smolvla]==0.4.4
# transformers==4.44.2
# accelerate
# safetensors
# numpy==1.26.4
# Pillow
# einops
# huggingface_hubpython inference.py核心逻辑 (inference.py:1):
SmolVLAPolicy.from_pretrained(..., local_files_only=True).to("npu:0"), 验证 next(policy.parameters()).device == npu:0, vision_model/vlm/expert 均在 npu:0make_pre_post_processors 分别放至 cpu / npu:0torch.randn(1,50,32) 在 CPU 生成后 clone().to(npu:0) 保证对比公平torch.npu.synchronize() 前后, 预热 3次, 正式10次执行命令: python inference.py (在 npu:0 真实执行, 权重来自本地)
[INFO] Model: lerobot/smolvla_base rev c83c3163b8ca9b7e67c509fffd9121e66cb96205
[INFO] Route: manipulation-vla (SmolVLA, chunk_size=50, action_dim=6, n_action_steps=50, num_steps=10)
[INFO] torch_npu_available: True, device_count: 2, device_name: Ascend910_9362
[INFO] CPU policy loaded in 39.5s, device: cpu, dtype: torch.float32
[INFO] CPU chunk done 16.34s shape (1, 50, 6) device cpu
[INFO] NPU policy loaded in 53.3s, device: npu:0
[INFO] NPU input devices: observation.state npu:0, observation.images.camera1 npu:0, tokens npu:0
[INFO] Warmup done (3 runs)
[RUN 0] latency 264.65 ms, shape (1, 50, 6), device npu:0
[RUN 1] latency 264.30 ms, shape (1, 50, 6), device npu:0
...
[RESULT] Action chunk shape: (1, 50, 6), horizon 50, dim 6
[RESULT] First action (6 dims): [0.268687903881073, 0.24012495577335358, 0.23859818279743195, 0.2857072353363037, -1.084153175354004, -0.6673141121864319]
[RESULT] Last action (6 dims): [-0.1286679059267044, 0.9370821118354797, 0.5172708630561829, 0.31643199920654297, -0.7268133163452148, -0.10833033919334412]
[RESULT] Finite: True, max_abs vs CPU: 0.000006, mae: 0.000003, tolerance FP32 0.001184
[RESULT] CPU_NPU_consistency: PASS (atol=1e-4 rtol=1e-3)
[PERF] avg 263.94 ms, min 261.80, max 266.01, p50 264.30, p90 266.01
[PERF] actions/s 189.44, chunks/s 3.79, batch=1, 3x256x256->512x512, horizon 50, steps 10, dtype float32
[RESULT] PASStorch.randn(1,50,32)再搬运至NPU)、同 dtype float32、同 10步 flow、同 predict_action_chunkpython .opencode/skills/npu_adapt_robotics/scripts/validate_policy_outputs.py --cpu /tmp/cpu_smolvla.json --npu /tmp/npu_smolvla.json --atol 1e-4 --rtol 1e-3atol 5e-3 rtol 5e-2 也为 true (tolerance 0.059)lerobot/policies/smolvla/smolvlm_with_expert.py:81 的 torch_dtype="bfloat16" 改为 "float32", 因 torch.bucketize 在 NPU 上不支持 BF16 (error EZ1001 DT_BFLOAT16 not supported, aclnnSearchSorted 161002), 并确保 embed_image 使用 float32; 未移动任何关键模块到 CPU, 所有浮点输入、模型参数、输出均在 npu:0 上 (已记录 device)python .opencode/skills/npu_adapt_robotics/scripts/benchmark_policy.py --latencies-ms /tmp/latencies.json --items-per-call 50 --unit actions{"actions_per_s": 189.44, "avg_ms": 263.94, "max_ms": 266.01, "min_ms": 261.80, "p50_ms": 263.64, "p90_ms": 264.65, "p95_ms": 266.01, "samples": 10}actions/s (50 actions per chunk), chunks/s 3.79, batch=1, 3x512x512 图像, float32, 含预处理后的纯模型计时 (torch.npu.synchronize 前后).
├── inference.py # 默认 python inference.py 一键 NPU 推理 + 一致性/性能打印
├── readme.md # 本文 (含三图引用)
├── requirements.txt # 精简依赖
└── assets/
├── agent_workflow.png
├── npu_device_call.png
└── model_result.pnginference.py 默认即验证: 打印模型/revision/route/backend/device/dtype/输入摘要/动作摘要/同步耗时/PASS/FAIL# 环境检查
npu-smi info
python .opencode/skills/npu_adapt_robotics/scripts/check_npu.py
# 推理 (自动含 CPU-NPU 对比与性能)
python inference.py
# 单独一致性 (已在 inference.py 内生成 /tmp/cpu_smolvla.json /tmp/npu_smolvla.json)
python .opencode/skills/npu_adapt_robotics/scripts/validate_policy_outputs.py --cpu /tmp/cpu_smolvla.json --npu /tmp/npu_smolvla.json
python .opencode/skills/npu_adapt_robotics/scripts/benchmark_policy.py --latencies-ms /tmp/latencies.json --items-per-call 50 --unit actionshuggingface-cli download lerobot/smolvla_base 至 workspace_npu/models/lerobot_smolvla_base 或自动回落至 Hub