模型名称: nvidia/Alpamayo-R1-10B (Alpamayo 1)
模型链接: HuggingFace
模型描述: Alpamayo 1 是 NVIDIA 推出的 Vision-Language-Action (VLA) 模型,将 Chain-of-Causation 推理与轨迹规划相结合,用于复杂自动驾驶场景的决策。基于 Qwen3-VL-8B VLM backbone 和扩散式动作解码器。
模型架构: VLA (Vision-Language-Action) - Qwen3-VL-8B + Diffusion Expert
参数规模: ~10.5B (VLM backbone: 8.2B + Action Expert: 2.3B)
输入规格:
输出规格:
| 依赖项 | 版本要求 | 说明 |
|---|---|---|
| Python | >= 3.10 | 推荐 3.11 |
| torch | >= 2.1.0 | PyTorch 框架 |
| torch_npu | >= 2.1.0 | 昇腾 NPU 后端 |
| transformers | >= 4.50.0 | HuggingFace Transformers(需支持 Qwen3-VL) |
| accelerate | >= 0.25.0 | 模型设备映射 |
| einops | >= 0.8.0 | 张量操作 |
| hydra-core | >= 1.3.0 | 配置管理 |
| pillow | >= 10.0.0 | 图像处理 |
| numpy | >= 1.24.0 | 数值计算 |
| scipy | latest | 科学计算 |
| sentencepiece | >= 0.1.99 | Tokenizer |
| 昇腾驱动 | CANN 8.5.1 | 已验证版本 |
| NPU 硬件 | Ascend 910 | 1+ 卡(64GB HBM) |
安装命令:
# 1. 安装 PyTorch 和 torch_npu(昇腾 NPU 后端)
pip install torch torch-npu
# 2. 安装 Transformers(需支持 Qwen3-VL)
pip install transformers>=4.50.0 accelerate
# 3. 安装其他依赖
pip install einops hydra-core pillow numpy scipy sentencepiece
# 4. 安装 Alpamayo 源代码
git clone https://github.com/NVlabs/alpamayo.git
cd alpamayo
PYTHONPATH=src:$PYTHONPATH pip install -e .# 加载 CANN 环境变量(必须在运行 Python 前执行)
source /usr/local/Ascend/cann-8.5.1/set_env.sh
# 禁用 torch dynamo(避免 NPU 跨设备图编译问题)
export TORCHDYNAMO_DISABLE=1
# 检查 NPU 设备
npu-smi info
# 验证 torch_npu
python3 -c "import torch_npu; print(torch.npu.device_count(), torch.npu.get_device_name(0))"方式一:HuggingFace(推荐)
huggingface-cli download nvidia/Alpamayo-R1-10B --local-dir /path/to/Alpamayo-R1-10B方式二:AtomGit 镜像(HuggingFace 下载慢时使用)
# AtomGit 镜像地址:
# https://ai.gitcode.com/hf_mirrors/nvidia/Alpamayo-R1-10B本模型为 VLA 模型,需要在源码层面进行以下修改以在昇腾 NPU 上运行:
| 修改项 | 原始代码 | 修改后 |
|---|---|---|
| 注意力实现 | flash_attention_2 | sdpa |
| 设备映射 | .to("cuda") | .to("npu:0") |
| autocast | torch.autocast("cuda") | torch.autocast("npu") |
| cholesky dtype | 直接调用 | 先 .float() 转 FP32 |
| output_logits | True | False(避免 FP32 跨设备 cast) |
source /usr/local/Ascend/cann-8.5.1/set_env.sh
export TORCHDYNAMO_DISABLE=1
export PYTHONPATH="/path/to/alpamayo/src:${PYTHONPATH}"
# 基础推理(VLM on NPU:0, expert+diffusion on CPU)
python inference.py \
--model-path /path/to/Alpamayo-R1-10B \
--num-traj-samples 1
# 多轨迹采样
python inference.py \
--model-path /path/to/Alpamayo-R1-10B \
--num-traj-samples 6 \
--max-generation-length 256| 参数 | 类型 | 默认值 | 说明 |
|---|---|---|---|
| --model-path | str | 必填 | 模型 checkpoint 路径 |
| --dtype | str | bfloat16 | 推理精度 |
| --num-traj-samples | int | 1 | 轨迹采样数 |
| --temperature | float | 0.6 | 采样温度 |
| --top-p | float | 0.98 | Top-p 采样参数 |
| --max-generation-length | int | 128 | VLM 最大生成长度 |
| --seed | int | 42 | 随机种子 |
$ python3 inference.py --model-path /tmp/.../nvidia_Alpamayo-R1-10B
=== Alpamayo-R1-10B Inference on Ascend NPU ===
Model path: /tmp/.../nvidia_Alpamayo-R1-10B
NPU devices: 2
NPU device name: Ascend910_9362
Dtype: bfloat16
Config: VLM on NPU:0, expert+diffusion on CPU
[Step 1] Loading model...
Model loaded in 50.4s
Total parameters: 10,534,841,405
VLM backbone: Qwen3-VL-8B (SDPA attention, NPU:0)
Expert model: Qwen2.5-based (2.3B params, CPU)
Diffusion decoder: FlowMatching (CPU)
[OK] Model loaded
[Step 2] Preparing input data...
Created 16 dummy images (4 cameras x 4 frames)
Egomotion history: xyz=(1, 1, 16, 3), rot=(1, 1, 16, 3, 3)
[OK] Input data prepared
[Step 3] Running VLA inference...
Inference time: 139.70s
Predicted XYZ shape: torch.Size([1, 1, 1, 64, 3])
Predicted ROT shape: torch.Size([1, 1, 1, 64, 3, 3])
Chain-of-Causation Reasoning:
Sample 0: Keep at the center of the lane to continue driving since no critical agent needs attention.
Trajectory Prediction:
WP[0]: x=-0.2471, y=0.0000, z=-0.1963
WP[1]: x=-0.4980, y=0.0001, z=-0.1963
WP[2]: x=-0.7500, y=0.0001, z=-0.1963
WP[3]: x=-1.0078, y=0.0002, z=-0.1963
WP[4]: x=-1.2656, y=0.0003, z=-0.1963
[SUCCESS] Inference completed on Ascend NPU
Model: nvidia/Alpamayo-R1-10B
Device: Ascend910_9362 (npu:0)
Dtype: bfloat16
Load time: 50.4s
Inference time: 139.70s运行命令:
source /usr/local/Ascend/cann-8.5.1/set_env.sh
export TORCHDYNAMO_DISABLE=1
export PYTHONPATH="/path/to/alpamayo/src:${PYTHONPATH}"
python inference.py \
--model-path /path/to/Alpamayo-R1-10B \
--num-traj-samples 1 \
--max-generation-length 128输出:
[Step 1] Loading model...
Model loaded in 50.4s
VLM backbone: Qwen3-VL-8B (SDPA attention, NPU:0)
Expert model: Qwen2.5-based (2.3B params, CPU)
[OK] Model loaded
[Step 3] Running VLA inference...
Inference time: 139.70s
Predicted XYZ shape: torch.Size([1, 1, 1, 64, 3])
Predicted ROT shape: torch.Size([1, 1, 1, 64, 3, 3])
Chain-of-Causation Reasoning:
Sample 0: Keep at the center of the lane to continue driving since no critical agent needs attention.
Trajectory Preview:
WP[0]: x=-0.2471, y=0.0000, z=-0.1963
WP[1]: x=-0.4980, y=0.0001, z=-0.1963
...
[SUCCESS] Inference completed on Ascend NPU


测试数据: PhysicalAI-AV-NuRec 数据集(910 场景用于 open-loop 评估)
评测指标:
| 指标 | 结果 | Baseline | 说明 |
|---|---|---|---|
| minADE@6.4s | <模型输出> | 1.22m (PhysicalAI-AV) | 平均位移误差 |
| AlpaSim Score | <模型输出> | 0.73 (PhysicalAI-AV-NuRec) | 仿真评分 |
评测命令:
python inference.py \
--model-path /path/to/Alpamayo-R1-10B \
--num-traj-samples 6 \
--max-generation-length 256flash_attention_2(CUDA 专用),已替换为 sdpa(昇腾 NPU 原生支持)。PYTHONPATH 直接引用源码运行。.float() 转换。