2toINF/X-VLA-Pt (Vision-Language-Action, Florence2-large + SoftPromptedTransformer, 0.9B, ~3.3GB FP32)c1c4a64a7e03ac5b95c468bf1578f3d03651b53bmanipulation-vla (image + language + proprio -> action_chunk)model.safetensors 3.52GB (3519068172 bytes) FP32, 879M paramseager, supports npu:0 and cpuXVLAProcessor (image_processor + tokenizer) -> input_ids [1,50], image_input [1,3,3,224,224], image_mask [1,3][1,30,20] (ee6d: xyz 6D-rotation gripper x2 arms, 30-step horizon), postprocess sigmoid on gripper channels, finite, horizon 30, action_dim 20npu:0 execution, no CPU fallback; all float tensors on npu:0 when NPU requestednext(model.parameters()).device, next(model.vlm.parameters()).device, next(model.transformer.parameters()).device, image_input.device, output.device all npu:0timm stub for DaviT and Florence2PreTrainedModel._supports_sdpa for transformers 4.57 compatibilitytrust_remote_code=True for XVLA remote code, auditable; tie_weights patched to avoid deleted lm_headpip install -r requirements.txt
python inference.py # auto NPU if available
PYTHONPATH=/tmp/timm_stub:/tmp/xvla_pt:$PYTHONPATH python inference.pyDefault loads local /tmp/xvla_pt (or HF cache fallback) and prints model/revision/route/backend/device/dtype/input summary/action summary/sync latency/PASS/FAIL. CPU-NPU consistency uses atol 1e-3 rtol 1e-3 (FP32 NPU numerical).
Ascend910_9362, torch.npu.is_available()=True, device_count=2, npu:0 existsnpu:0 (vlm npu:0, transformer npu:0)npu:0 (image_input [1,3,3,224,224] float32, input_ids [1,50] int64, proprio [1,20])npu:0 action_chunk [1,30,20] float32 finite True shape_ok True[-0.039593, -0.120886, 0.259252, 0.140380, -0.186590, ...] (20D)[-0.024553, -0.114726, 0.270035, 0.132710, -0.207604, ...][-0.039593, -0.120886, 0.259252, 0.140380, -0.186590, ...]0.000161 tol 0.001572 (atol 1e-3 rtol 1e-3) -> PASS91.22 ms, min 90.17 max 92.69, p50 91.22 p90 92.69 (10 runs, warmup 3, torch.npu.synchronize)328.9 actions/s, 10.96 chunks/s (horizon 30, batch 1, steps 5, float32)Shared weights (single model instance moved CPU->NPU, same embed_tokens random init), same deterministic images (seed i*1234), same language "pick up the red block and place it on the green bowl", same processor, same proprio/domain, seed 42, steps 5, float32. Compared flattened [600] action values. Threshold atol=1e-3 rtol=1e-3 (FP32 NPU fused ops). Also verified finite, shape [1,30,20], action_dim 20, horizon 30.
| metric | value |
|---|---|
| cpu first_action[0:5] | [-0.039593, -0.120886, 0.259252, 0.140380, -0.186590] |
| npu first_action[0:5] | [-0.039593, -0.120886, 0.259252, 0.140380, -0.186590] |
| max_abs | 0.000161 |
| max_ref | 0.5748 |
| tolerance | 0.001575 |
| finite | true |
| shape | [1,30,20] |
| passed | true |
torch_npu built for CANN 8.5.1, Ascend910B/910_9362, single card sufficienttimm stub provided at /tmp/timm_stub for DaviT DropPath/trunc_normal_; einops requiredtrust_remote_code=True, patched tie_weights and _supports_sdpa for transformers 4.57 compatibilityassets/agent_workflow.png — reconnaissance, resource check, load, NPU inference, consistency, performanceassets/npu_device_call.png — npu-smi, torch_npu availability, device name, model/input/output device proofassets/model_result.png — default python inference.py showing model, route, action summary, latency, PASSinference.py
readme.md
requirements.txt
assets/agent_workflow.png
assets/npu_device_call.png
assets/model_result.png
