Beaver 奖励模型(Safe-RLHF LlamaForScore,LLaMA-7B 底座)。对文本对话给出 helpfulness 奖励打分。
本仓库提供昇腾 Ascend910 (torch_npu) 上的推理入口。
LlamaModel,hidden_size=4096,32 层,vocab_size=32001,bf16)score_head(Linear 4096→1)+ normalizer(count/mean/var buffer)model.* + score_head.weight / score_head.bias + normalizer.*LlamaForScore,inference.py 内自实现并加载分片 safetensors# CLI(文本 prompt)
python inference.py --prompt "User: Write a short poem about the sea. Assistant: The sea is vast..."
# CLI(从 .txt 读取 prompt)
python inference.py --img test_prompt_helpful.txt
# FastAPI
python inference.py --serve --port 8000
curl -X POST http://127.0.0.1:8000/predict -F "prompt=User: hello Assistant: Hello! How can I help you?"返回 JSON: reward(最后一个 token 的标量分数)、inference_ms、npu_name、exit_status 等。
import torch_npu + torch.npu.set_device(0),设备 npu:0。./model_weights/PKU-Alignment/beaver-7b-v1.0-reward/,重复运行命中本地缓存([cache-hit])。scores[0, -1, 0](Safe-RLHF end_scores 语义),不做归一化(config do_normalize=false)。torch 2.9.0 / torch_npu 2.9.0.post1 / transformers 4.57.6 / CANN 8.5.1 / Ascend910