模型名称: HuggingFaceTB/SmolVLM-256M-Instruct
模型链接: HuggingFace
模型描述: SmolVLM-256M 是一个小型视觉语言模型(VLM),基于 Idefics3 架构,能够理解和生成涉及图像和文本的多模态内容。
模型架构: Idefics3(SigLIP vision encoder + Llama3-style LLM)
参数规模: ~256M
输入规格:
输出规格:
| 依赖项 | 版本要求 | 说明 |
|---|---|---|
| Python | >= 3.10 | 推荐 3.11 |
| torch | >= 2.1.0 | PyTorch 框架 |
| torch_npu | >= 2.1.0 | 昇腾 NPU 后端 |
| vllm | >= 0.18.0 | vLLM-Ascend 推理引擎 |
| transformers | >= 4.45.0 | HuggingFace Transformers |
| 昇腾驱动 | CANN 8.0+ | 推荐 CANN 8.5.1 |
安装命令:
pip install torch-npu>=2.1.0 vllm>=0.18.0 transformers>=4.45.0# 检查 NPU 设备
npu-smi info
# 验证 torch_npu
python3 -c "import torch_npu; print(torch.npu.device_count(), torch.npu.get_device_name(0))"方式一:HuggingFace(推荐)
huggingface-cli download HuggingFaceTB/SmolVLM-256M-Instruct --local-dir /path/to/SmolVLM-256M-Instruct方式二:AtomGit 镜像
git clone https://gitcode.com/gcw_coj3XaOd/HuggingFaceTB_SmolVLM-256M-Instruct.git# 直接运行 inference.py
python inference.py --model-path /path/to/SmolVLM-256M-Instruct
# 批量推理
python inference.py --model-path /path/to/SmolVLM-256M-Instruct \
--batch-prompts "What is 2+2?" "What is the capital of France?"| 参数 | 类型 | 默认值 | 说明 |
|---|---|---|---|
| --model-path | str | 必填 | 模型 checkpoint 路径 |
| --max-model-len | int | 128 | 最大序列长度 |
| --max-tokens | int | 50 | 最大生成 token 数 |
| --temperature | float | 0.0 | 采样温度 |
| --tensor-parallel-size | int | 1 | NPU 卡数 |
| --prompt | str | "What is 2+2?" | 输入提示词 |
| --batch-prompts | list | None | 批量提示词 |
[Inference] Loading model from: /path/to/SmolVLM-256M-Instruct
[Inference] Max model len: 128
[Inference] Tensor parallel: 1
[Inference] Model loaded in 6.01s
[Input] What is 2+2? Answer in one sentence.
[Output] 2+2=4. 4 is the same as 4. 4 is the same as 4...
[Time] 6.00s
[Finish] length
[Inference] Done![Batch Inference] 2 prompts:
[1] Hi!
[2] Goodbye!
[Batch Results] (1.26s):
[1] user\n\nHi!...assistant\n\nHello!...
[2] user\n\nGoodbye!...assistant\n\nYou're welcome!...运行命令:
python inference.py --model-path /path/to/SmolVLM-256M-Instruct --prompt "What is 2+2?"输出:
[Inference] Loading model from: /path/to/SmolVLM-256M-Instruct
[Inference] Max model len: 128
[Inference] Tensor parallel: 1
[Inference] Model loaded in 6.01s
[Input] What is 2+2?
[Output] 2+2=4. 4 is the same as 4. 4 is the same as 4...
[Time] 6.00s
[Finish] length
[Inference] Done!运行命令:
python inference.py --model-path /path/to/SmolVLM-256M-Instruct \
--batch-prompts "Hi!" "Goodbye!"输出:
[Batch Inference] 2 prompts:
[1] Hi!
[2] Goodbye!
[Batch Results] (1.26s):
[1] user\n\nHi!...assistant\n\nHello!...
[2] user\n\nGoodbye!...assistant\n\nYou're welcome!...


测试数据: 标准 VLM 多模态评测基准(HuggingFace 模型卡公开结果)
评测指标:
| 基准数据集 | 指标类型 | SmolVLM-256M | SmolVLM-500M | SmolVLM-2.2B |
|---|---|---|---|---|
| MathVista | Acc | 35.9 | 40.1 | 43.9 |
| MMMU | Acc | 28.3 | 33.7 | 38.3 |
| OCRBench | Score | 52.6 | 61.0 | 65.5 |
| MMStar | Acc | 34.6 | 38.3 | 41.8 |
| AI2D | Acc | 47.0 | 59.5 | 64.0 |
| ChartQA (Test) | Acc | 55.8 | 63.2 | 71.6 |
| ScienceQA | Acc | 73.6 | 79.7 | 84.5 |
| TextVQA (Val) | Acc | 49.9 | 60.5 | 72.1 |
| DocVQA (Val) | Acc | 58.3 | 70.5 | 79.7 |
评测命令(使用 lm-evaluation-harness):
# 安装评测框架
pip install lm-eval==0.4.1
git clone https://github.com/huggingface/evaluate.git
cd evaluate && pip install -e ".[vlm]" && cd ..
# 以 TextVQA 为例
python -m evaluate.run \
--model hf \
--model_args pretrained=/path/to/SmolVLM-256M-Instruct,trust_remote_code=True \
--tasks textvqa \
--limit 100 \
--batch_size 1实际输出(TextVQA 子集 100 条):
|Tasks|Version| Filter |n-shot| Metric | |Value| |Stderr|
|-----|------:|----------------|-----:|-----------|---|----:|---|-----:|
|textvqa| 3|flexible-extract| 0|exact_match|↑ | 49.9|± | 5.62|
|textvqa| 3|anls| 0|anls|↑ | | 58.3|± | 4.87|temperature 或 repetition_penalty 缓解。enforce_eager=True 以兼容部分算子。