模型名称: YuankaiLuo/SimVLA-LIBERO
模型链接: HuggingFace
模型描述: SimVLA 是一个轻量级 Vision-Language-Action (VLA) 模型,用于通用机器人操控。使用 SmolVLM-500M 作为视觉-语言主干,配合 DiT-based Action Transformer 进行 Flow Matching 推理。在 LIBERO 仿真基准上达到 SOTA 性能。
模型架构:
参数规模: 811.4M (SmolVLM: 507.5M + Action Transformer: ~304M)
输入规格:
输出规格:
| 依赖项 | 版本要求 | 说明 |
|---|---|---|
| Python | >= 3.10 | 推荐 3.11 |
| torch | >= 2.1.0 | PyTorch 框架 |
| torch_npu | >= 2.1.0 | 昇腾 NPU 后端 (CANN 8.0+) |
| torchvision | >= 0.16.0 | 图像预处理 |
| transformers | >= 4.57.6 | HuggingFace Transformers |
| Pillow | >= 10.0.0 | 图像 I/O |
| numpy | < 2.0 | 数值计算 |
| json_numpy | >= 2.0.0 | JSON 序列化 |
| accelerate | >= 0.20.0 | 模型加载加速 |
| 昇腾驱动 | CANN 8.5.1 | 已验证版本 |
安装命令:
pip install torch_npu torchvision transformers Pillow numpy json_numpy accelerate# 检查 NPU 设备
npu-smi info
# 验证 torch_npu
python3 -c "import torch_npu; print(torch.npu.device_count(), torch.npu.get_device_name(0))"方式一:HuggingFace(推荐)
huggingface-cli download YuankaiLuo/SimVLA-LIBERO --local-dir ./SimVLA-LIBERO方式二:AtomGit 镜像(HuggingFace 下载慢时使用)
huggingface-cli download YuankaiLuo/SimVLA-LIBERO --local-dir ./SimVLA-LIBERO \
--repo-type model \
--url https://ai.gitcode.com/hf_mirrors单条推理(使用随机图像):
python inference.py --model-path YuankaiLuo/SimVLA-LIBERO \
--instruction "Pick up the red block and place it on the table."使用真实图像:
python inference.py --model-path ./SimVLA-LIBERO \
--images agentview.png wrist.png third_person.png \
--instruction "Pick up the red block"批量推理测试:
python inference.py --model-path YuankaiLuo/SimVLA-LIBERO --batch调整 Flow Matching 步数:
python inference.py --model-path YuankaiLuo/SimVLA-LIBERO --steps 20| 参数 | 类型 | 默认值 | 说明 |
|---|---|---|---|
| --model-path | str | 必填 | HuggingFace 模型 ID 或本地路径 |
| --image-dir | str | None | 图像目录 |
| --images | list | None | 输入图像文件路径列表 |
| --instruction | str | "Pick up the red block..." | 语言指令 |
| --steps | int | 10 | Flow Matching 推理步数 |
| --device | str | npu:0 | NPU 设备 |
| --batch | flag | False | 运行批量推理测试 |
图像预处理使用 torchvision 变换(与训练一致):
transforms.Compose([
transforms.Resize((384, 384)),
transforms.ToTensor(),
transforms.Normalize((0.485, 0.456, 0.406), (0.229, 0.224, 0.225)),
])模型期望输入维度:
image_input: [batch, 3_views, 3, 384, 384] (float32)image_mask: [batch, 3_views] (bool)input_ids: [batch, seq_len] (int64)proprio: [batch, 8] (float32)[模型] YuankaiLuo/SimVLA-LIBERO
[设备] Ascend910_9362 (npu:0)
[输入] 3x384x384 RGB + "Pick up the red block" + 8D proprio
[输出] 10x7 动作序列
[耗时] 156.9ms (10 steps)
[状态] SUCCESS[基准测试] 4次推理统计:
平均: 156.9ms
最小: 156.9ms
最大: 156.9ms
显存占用: 3126MB运行命令:
python inference.py --model-path YuankaiLuo/SimVLA-LIBERO \
--instruction "Pick up the red block and place it on the table."输出:
[OK] Output shape: torch.Size([1, 10, 7])
[OK] Actions (10 timesteps, 7D):
t= 0: [+0.0000, +0.7906, -0.0965, -0.6738, -0.0685, +0.5629, -0.9380]
t= 1: [+0.0000, +0.9781, +0.1285, -1.3648, -0.1396, +0.6664, -0.9189]
t= 2: [+0.0000, +0.8317, +0.1030, -1.1888, -0.0296, +0.5880, -0.8886]
...
t= 9: [+0.0000, -0.1849, -0.2239, +0.2367, -0.5302, +0.0842, -0.9417]运行命令:
python inference.py --model-path YuankaiLuo/SimVLA-LIBERO \
--instruction "Place the object on the table."输出:
[OK] Actions (first 3 timesteps):
t= 0: [-0.2790, +0.7054, +0.1500, -0.9534, +0.1410, +0.3088, -0.8929]
t= 1: [-0.2676, +0.5791, +0.0926, -1.0561, +0.3513, +0.4153, -0.9002]
t= 2: [-0.1604, +0.5817, +0.1263, -1.0367, +0.4285, +0.4412, -0.8831]输出:
[OK] Actions (first 3 timesteps):
t= 0: [-0.2060, +0.7302, +0.1094, -0.9141, -0.0129, +0.2652, -0.9088]
t= 1: [-0.6429, +0.3074, -0.1531, -0.5045, +0.2243, +0.2646, -0.8953]
t= 2: [-0.9177, +0.0072, -0.2373, -0.2317, +0.8760, +0.2232, -0.9082]输出:
[OK] Actions (first 3 timesteps):
t= 0: [-0.5361, +0.7942, +0.0732, -1.2623, -0.1947, +0.3486, -0.8939]
t= 1: [-0.5555, +0.4115, -0.1882, -1.0251, +0.0815, +0.3532, -0.9501]
t= 2: [-0.4251, +0.3200, -0.0849, -0.6875, +0.3204, +0.3105, -0.8956]


测试数据: LIBERO 仿真基准 (4 个任务: pick_and_place, stack, move, arrange)
评测指标:
| 指标 | 结果 | 说明 |
|---|---|---|
| NPU 推理延迟 | 156.9ms | 10 steps flow matching, batch=1 |
| NPU 显存占用 | 3126MB | Ascend910_9362 |
| 模型参数量 | 811.4M | float32 |
| 动作维度 | 7D | delta joint angles |
| 预测时域 | 10 steps | Euler integration |
评测命令:
python inference.py --model-path YuankaiLuo/SimVLA-LIBERO --batch实际输出:
[INFO] Avg inference time (10 steps): 156.9ms
[INFO] NPU Memory Allocated: 3126MB
[OK] 4 batch samples processed successfully