ggg_0963/OLMo-1B-hf
Model Introduction
Files & versions
Pull Requests
Discussion
Analysis

OLMo-1B-hf on vLLM-Ascend #+NPU

1. 简介

本文档记录 LLM-Research/OLMo-1B-hf 模型在 vLLM-Ascend 环境的快速部署与验证结果。

OLMo-1B-hf 是 Allen AI 开发的 10 亿参数基础语言模型,基于 OLMo 架构,适用于文本生成和自然语言理解任务。

相关获取地址:

  • 权重下载地址(ModelScope):https://modelscope.cn/models/LLM-Research/OLMo-1B-hf
  • 权重下载地址(HuggingFace):https://huggingface.co/allenai/OLMo-1B-hf

2. 验证环境

组件版本
vllm-ascend0.18.0rc1
vllm0.18.0
transformers4.57.6
torch-npu2.9.0
CANN8.5.1
SOCascend910_9391
  • NPU:1 逻辑卡(Ascend 910B2,64GB HBM)
  • 模型路径:/home/openmind/.cache/modelscope/hub/models/LLM-Research/OLMo-1B-hf

3. 模型配置

参数值
架构OLMoForCausalLM
参数量1B
层数16
隐藏维度2048
注意力头数16
词表大小50280
精度Float16(vLLM 推理)

4. 模型下载

# 从 ModelScope 下载
modelscope download --model LLM-Research/OLMo-1B-hf

5. 基础推理验证

使用 vLLM-Ascend 加载模型进行推理:

from vllm import LLM, SamplingParams

MODEL_PATH = "LLM-Research/OLMo-1B-hf"

llm = LLM(
    model=MODEL_PATH,
    trust_remote_code=True,
    dtype="float16",
    tensor_parallel_size=1,
    max_model_len=2048,
    gpu_memory_utilization=0.9,
)

sampling = SamplingParams(max_tokens=64, temperature=0)
outputs = llm.generate(["The capital of France is"], sampling)
print(outputs[0].outputs[0].text)
# the city of Paris.

6. Smoke 验证

6.1 冒烟测试脚本

运行 4 项冒烟测试验证模型功能:

python3 smoke_test.py

6.2 验证结果

=== OLMo-1B-hf vLLM-Ascend Smoke Test ===
[1/4] Testing basic generation...
  PASS: the city of Paris. The city is located in the north of France.
[2/4] Testing factual knowledge...
  PASS: The boiling point of water is 100°C.
[3/4] Testing batch generation...
  [1] The answer to 2 + 2 is 4.
  [2] The largest planet in our solar system is Jupiter.
  [3] The ocean is a vast body of saltwater.
  PASS: 3 prompts processed
[4/4] Testing reasoning...
  PASS: John has 5 apples and gives 2 to Mary.
=== All 4/4 tests PASSED (41.1s) ===

验证项说明:

测试项输入预期结果
英文基础生成The capital of France is包含 ParisPASS
事实知识Water boils at what temperature?包含 100PASS
批量生成3 个 prompt3 条非空输出PASS
推理能力数学推理题目非空输出PASS

7. 精度评测

7.1 评测配置

  • 数据集:ARC-Challenge(25-shot)
  • 数据来源:ModelScope modelscope/ai2_arc
  • 评测框架:vLLM-Ascend offline inference
  • 设备:NPU (npu:0)
python3 eval_arc_vllm.py --output logs/arc_challenge_vllm.json

7.2 评测结果

指标基线NPU 结果差异
ARC-Challenge 25-shot24.00%23.55% (276/1172)-0.45%

ARC-Challenge NPU 准确率 23.55%,与基线 24.00% 差异 -0.45%(< 1%),验证通过。

7.3 评测日志

=== OLMo-1B-hf ARC-Challenge 25-shot vLLM-Ascend ===
Total test samples: 1172
Loading model with vLLM-Ascend...
Model loaded.
Generating (1172 samples)...
============================================================
OLMo-1B-hf ARC-Challenge 25-shot Results (vLLM-Ascend)
============================================================
  Accuracy: 23.55% (276/1172)
  Total time: 4s
  Throughput: 263.09 samples/s
============================================================
Results saved to logs/arc_challenge_vllm.json

8. 注意事项

  1. Base 模型:此为基础模型(非 Instruct 版本),不具备指令跟随能力,评测时使用续写方式生成答案。

  2. 显存占用:float16 推理时模型约占用 2.2 GB HBM,单卡即可运行。

  3. 推理速度:OLMo-1B 为小模型,vLLM-Ascend 推理吞吐量可达 263 samples/s。