w
gcw_uQ09W7jl/act-aloha-sim-transfer-cube-human-NPU
Model Introduction
Files & versions
Pull Requests
Discussion
Analysis

ACT Aloha Sim Transfer Cube Human - Ascend NPU

Model: lerobot/act_aloha_sim_transfer_cube_human
Route: demonstration-policy (Action Chunking Transformer)
Hardware: Huawei Ascend 910B (Ascend910_9362) on npu:0
Revision: main (local dir /opt/atomgit/adapt-npu-agent/persist_models/act_aloha_sim_transfer_cube_human)
Weight: model.safetensors 198MB, config act chunk_size 100

任务与契约

  • 任务: 双臂 Aloha TransferCube - 从一臂拾取方块并传递到另一臂,基于 AlohaTransferCube-v0 (gym-aloha)。
  • 观测输入 (input_schema):
    • observation.images.top : RGB [3,480,640] 单相机 top 视角
    • observation.state : 14-dim 双臂 proprio (joint positions)
  • 输出 (output_schema):
    • action chunk [batch, chunk_size=100, action_dim=14] 连续关节位置序列,n_action_steps=100
  • 模型结构: ResNet18 vision backbone + Transformer encoder (4层) + decoder (1层) + VAE (latent_dim 32), dim_model 512, n_heads 8

真实 NPU 验证

  • Backend: lerobot.policies.act.ACTPolicy.from_pretrained + torch_npu, patched to avoid online ResNet download
  • Device proof:
    • torch.npu.is_available()=True, device_count=2, device_name=Ascend910_9362
    • 模型首参 device npu:0, backbone npu:0, 输入 Tensor npu:0, 输出 action npu:0
    • 计时前后 torch.npu.synchronize()
  • 观测一致性: CPU 与 NPU 共用同一合成观测 (seed 42),相同权重、预处理、dtype=float32、eval 模式、无采样随机性
  • 一致性结果 (CPU vs NPU, FP32):
    • shape [1,100,14] count 1400 finite true
    • max_abs 3.85e-04 mean_abs 1.02e-04 tolerance 1.46e-03 (atol 1e-4 rtol 1e-3) PASS
    • BF16 tolerance 7.28e-02 also PASS
  • 性能 (NPU synchronized):
    • warmup 3 + 10 measured, batch 1, horizon 100, dtype float32
    • avg 12.65ms min 12.62 max 12.70 p50 12.63 p90 12.69 p95 12.70
    • actions/s 7903 chunks/s 79.03
    • Via scripts/benchmark_policy.py --latencies-ms /tmp/latencies_ms.json --items-per-call 100 --unit actions

推理方式

pip install -r requirements.txt
python inference.py

inference.py 默认从本地 MODEL_DIR 加载,打印 model/revision/route/backend/device/dtype、输入摘要、CPU 与 NPU action 摘要、同步耗时与 PASS/FAIL。无需网络下载权重。

文件结构

inference.py
readme.md
requirements.txt
assets/agent_workflow.png
assets/npu_device_call.png
assets/model_result.png

限制

  • 单卡 Ascend 910B npu:0 验证,显存峰值约 1.2GB (权重 198MB + 模型 51M params)
  • 输入为合成随机观测用于数值一致性证明,非真实机器人闭环 rollout;环境 reward 83% (论文) 仅作参考
  • 未修改核心视觉塔/Transformer,保持端到端;不支持多观察步 (n_obs_steps=1)

证据

  • assets/agent_workflow.png 侦察、资源、加载、NPU 推理、一致性、性能与校验
  • assets/npu_device_call.png npu-smi、NPU availability、设备名、模型/输入/输出 device
  • assets/model_result.png 默认 python inference.py 的真实 ActionPlan 与同步耗时

agent workflow npu device call model result