将 gmmeyer 发布的 Dreaming Bird 神经网络世界模型在昇腾 Ascend910 (Atlas 800 A2,CANN 8.5.1)上完成 vLLM-Ascend 服务化推理适配。模型以 约 1100 万参数的 nanoGPT 风格解码器(
DreamGPT)替代 Flappy Bird 的 物理引擎:给定最近若干帧的量化状态与玩家动作,逐槽位预测下一帧状态, 循环自回归即可在"梦里"把游戏跑起来。
| 特性 | 说明 |
|---|---|
| 原始权重 | gmmeyer/gpt-bird(MIT) |
| 模型类型 | 环境世界模型(world model),decoder-only 自回归 |
| 架构 | DreamGPT(nanoGPT 风格:6 层 Transformer Block,d_model=384,6 头,GELU MLP,RMSNorm) |
| 参数量 | 约 11M(SMALL tier,small_pipes.pt) |
| 上下文 | block_size=1280 token(约 256 帧,每帧 4 个状态 token + 1 个动作 token) |
| 词表 | 232:4 特殊符 + 128 bird_y 量化槽 + 64 pipe_dx 槽 + 32 gap_y 槽 + 2 状态 + 2 动作 |
| 状态表示 | 每帧 4 token,槽序 [bird_y, pipe_dx, gap_y, status],槽位值域互不重叠 |
| 权重形态 | 官方发布 small_pipes.pt(PyTorch 权重)+ model.onnx(ONNX opset 17);本次适配以 PyTorch 权重为基准 |
| 适配路线 | 权重转 safetensors → 构建 HF 兼容 config → vLLM 运行时注册 DreamGPT → OpenAI 兼容服务化推理 |
Dreaming Bird 的训练目标不是"读文本"而是"学会物理":观察状态刻意不含速度, 模型必须从位置序列(有限差分)中自行重建速度才能飞得起来。每一帧的 4 个字段 使用互不重叠的 token id 区间量化(bird_y 128 档、pipe_dx 64 档、gap_y 32 档), 推理时对每个槽位的 logits 施加合法区间掩码,从构造上杜绝畸形帧。
| 项目 | 实测值 |
|---|---|
| 硬件 | Ascend910 双卡(本适配使用其中一张,设备逻辑号 0,Phy-ID 6) |
| OS / 架构 | Linux / aarch64 |
| CANN | 8.5.1 |
| Python | 3.11.14 |
| PyTorch | 2.9.0+cpu |
| torch_npu | 2.9.0.post1+gitee7ba04 |
| vLLM | 0.18.0(源码 editable 安装于 /vllm-workspace/vllm) |
| vllm_ascend | 0.18.0(/vllm-workspace/vllm-ascend) |
| transformers | 4.57.6(以 model_type=gpt2 加载自定义配置) |
NPU 可用性校验:
npu-smi info
python -c "import torch, torch_npu; print(torch.npu.is_available(), torch.npu.device_count())"
# 实测: True 2权重优先从 GitCode HF 镜像拉取,失败则回退 ModelScope / HuggingFace:
export HF_ENDPOINT=https://ai.gitcode.com/hf_mirrors
python -c "
from huggingface_hub import snapshot_download
snapshot_download('gmmeyer/gpt-bird', local_dir='/opt/atomgit/models/gmmeyer/gpt-bird')
"本适配实际使用的文件:small_pipes.pt(44.8MB,PyTorch SMALL 权重)、config.json
(官方词表/引擎几何配置)。model.onnx 为浏览器 WebGPU 演示用,服务化推理不依赖。
small_pipes.pt 内嵌自定义 dreaming_bird 类,需用源码包加载后转出标准格式:
# 需要先获取源码:git clone https://github.com/gmmeyer/gptbird 或下载 tarball
import torch, torch.serialization
from dreaming_bird.config import ModelConfig, TokenizerConfig, EngineConfig
torch.serialization.add_safe_globals([ModelConfig, TokenizerConfig, EngineConfig])
ckpt = torch.load('small_pipes.pt', map_location='cpu', weights_only=True)
sd = dict(ckpt['model'])
sd['lm_head.weight'] = sd['lm_head.weight'].clone() # 解开 wte/lm_head 权重共享
from safetensors.torch import save_file
save_file(sd, 'model.safetensors')模型目录最终包含:config.json(HF 兼容,architectures: ["DreamGPT"],
model_type: "gpt2",max_position_embeddings: 1280)、model.safetensors、
tokenizer.json(232 词表 WordLevel 分词器,token 串为 3 位定长 id,如 004)。
vLLM 源码只读,DreamGPT 架构通过 ModelRegistry.register_model 运行时注册
(vLLM 0.18 提供懒加载字符串形式,worker 进程按需导入):
from vllm.model_executor.models.registry import ModelRegistry
ModelRegistry.register_model("DreamGPT", "gpt_bird:DreamGPTForCausalLM")
# 模型实现见适配插件 gpt_bird.py(DreamGPTForCausalLM:Embedding + RMSNorm +
# QKVParallelLinear + GELU MLP + ParallelLMHead,权重键名与 checkpoint 一一对应)ASCEND_RT_VISIBLE_DEVICES=0 \
VLLM_ENABLE_V1_MULTIPROCESSING=0 \
PYTHONPATH=<插件目录> \
python3 <插件目录>/start_server.py \
--model <模型目录> \
--served-model-name gpt-bird \
--dtype bfloat16 \
--max-model-len 1280 \
--max-num-seqs 8 \
--gpu-memory-utilization 0.08 \
--enforce-eager \
--port 8071说明:模型权重约 21MB(bf16),
--gpu-memory-utilization给 0.08 即可;--max-model-len须与 wpe 位置嵌入长度 1280 一致;--enforce-eager关闭图编译, 适配无特殊算子的小模型以获得最短启动时间。
curl -s http://127.0.0.1:8071/v1/models
# {"object":"list","data":[{"id":"gpt-bird", ..., "max_model_len":1280, ...}]}使用内置 Flappy-Bird 物理引擎(CPU 真值)生成一段真实回放作为上下文,向服务 发起槽位约束的下一帧预测:
python3 inference.py --endpoint http://127.0.0.1:8071 --model gpt-bird \
--context-frames 6 --compare --dream 8 --seed 7Smoke 实测(seed=7,6 帧上下文 + 动作):
[2] context: 31 tokens (BOS + 6 frames x 4 + 6 actions)
prompt ids: 001 068 182 208 228 230 068 181 208 228 230 068 180 208 228 230 ...
[3] one-step next-frame prediction (slot-constrained, 4 tokens)
model output ids: [74, 177, 208, 228]
model frame : bird_y=282.0, pipe_dx=204.8, gap_y=200.0, alive=True
ground truth : bird_y=281.2, pipe_dx=204.0, gap_y=197.2, alive=True
exact token match: True latency: 57.3 ms (4 slots)等价的单步 curl(一次只取 1 个槽位 token,logit_bias 屏蔽非法 id):
curl -s http://127.0.0.1:8071/v1/completions -H 'Content-Type: application/json' \
-d '{"model":"gpt-bird","prompt":"001 068 182 208 228 230","max_tokens":1,
"temperature":0,"logit_bias":{"4":0,"5":0,"6":0,"7":0,"8":0,"9":0,"10":0,"11":0,"12":0,"13":0,"14":0,"15":0,"16":0,"17":0,"18":0,"19":0,"20":0,"21":0,"22":0,"23":0,"24":0,"25":0,"26":0,"27":0,"28":0,"29":0,"30":0,"31":0,"32":0,"33":0,"34":0,"35":0,"36":0,"37":0,"38":0,"39":0,"40":0,"41":0,"42":0,"43":0,"44":0,"45":0,"46":0,"47":0,"48":0,"49":0,"50":0,"51":0,"52":0,"53":0,"54":0,"55":0,"56":0,"57":0,"58":0,"59":0,"60":0,"61":0,"62":0,"63":0,"64":0,"65":0,"66":0,"67":0,"68":0,"69":0,"70":0,"71":0,"72":0,"73":0,"74":0,"75":0,"76":0,"77":0,"78":0,"79":0,"80":0,"81":0,"82":0,"83":0,"84":0,"85":0,"86":0,"87":0,"88":0,"89":0,"90":0,"91":0,"92":0,"93":0,"94":0,"95":0,"96":0,"97":0,"98":0,"99":0,"100":0,"101":0,"102":0,"103":0,"104":0,"105":0,"106":0,"107":0,"108":0,"109":0,"110":0,"111":0,"112":0,"113":0,"114":0,"115":0,"116":0,"117":0,"118":0,"119":0,"120":0,"121":0,"122":0,"123":0,"124":0,"125":0,"126":0,"127":0,"128":0,"129":0,"130":0,"131":0}}'Ascend910 单卡、bfloat16、--enforce-eager 实测:
| 指标 | 数值 |
|---|---|
| 模型权重加载 | 0.0209 GB(bf16) |
| KV cache 容量 | 557,696 tokens(max concurrency ≈ 435x@1280 tokens/请求) |
| 单帧预测延迟(4 槽位,槽位约束贪心) | 约 57 ms |
| 持续吞吐(单客户端串行) | 约 19.6 帧/秒(549 次预测 / 28s) |
| 并发负载下 NPU AICore | 3% - 6%(瞬时) |
| 并发负载下 NPU HBM | 约 8.3 GB / 65.5 GB(含 vLLM 运行时与 KV cache) |
| 服务进程显存 | 约 5.27 GB(VLLMEngineCore) |
说明:模型仅 11M 参数、上下文短,算力开销极低,910B 的 AICore 占用常年 处于个位数百分比,属于正常现象;主要资源被 vLLM 运行时与 KV cache 预留占据。
npu-smi info 原样输出)| NPU Name | Health | Power(W) Temp(C) Hugepages-Usage(page)|
| Chip Phy-ID | Bus-Id | AICore(%) Memory-Usage(MB) HBM-Usage(MB) |
+===========================+===============+====================================================+
| 3 Ascend910 | OK | 180.2 43 0 / 0 |
| 0 6 | 0000:0B:00.0 | 5 0 / 0 8324 / 65536 |
+------------------------------------------------------------------------------------------------+
| NPU Chip | Process id | Process name | Process memory(MB) |
+===========================+===============+====================================================+
| 3 0 | 31384 | VLLMEngineCor | 5270 |
+===========================+===============+====================================================+以 CPU 物理引擎回放为真值,逐槽位贪心生成下一帧并与真值 token 对齐。在
seed=7 / seed=42 两个回放下,模型输出的 4 个槽位 token 均与真值完全一致
(如 seed=7 输出 [74,177,208,228] == 真值 [74,177,208,228])。
| 槽位 | 模型反量化 | 物理真值 | 误差 |
|---|---|---|---|
| bird_y | 282.0 px | 281.2 px | +0.8 px(1 档内) |
| pipe_dx | 204.8 px | 204.0 px | +0.8 px(1 档内) |
| gap_y | 200.0 px | 197.2 px | +2.8 px |
| status | alive | alive | 一致 |
反量化误差源于 bird_y/gap_y 的量化粒度(512px/128 档 ≈ 4px,pipe_dx 288px/64 档 ≈ 4.5px),在单档之内,与官方公布的 one-step bird_y 98.6% exact 相符。
将模型自身输出回灌上下文连续生成 8 帧,物理行为自洽:重力作用下 bird_y 逐帧 下移(282→290→298),FLAP 后 bird_y 回跳(298→310→302),pipe_dx 匀速递减 (管道左移)。未出现越界 / 死机帧,证明模型已从位置序列中重建出速度信息。
NPU out of memory 时,先 npu-smi info 确认目标设备空闲,再
通过 ASCEND_RT_VISIBLE_DEVICES 切换到空闲卡,切勿强制抢占他人设备。ASCEND_RT_VISIBLE_DEVICES=<逻辑号> 选择设备;不要混用
ASCEND_VISIBLE_DEVICES(会干扰 torch.npu 的设备映射)。small_pipes.pt 中 wte.weight 与 lm_head.weight 为同一张量
(weight tying)。转 safetensors 前需 clone() 断开共享,否则 safetensors
序列化会报 shared-memory 错误。model_type 必须取 transformers
已知值(本适配用 "gpt2"),架构解析仍由 architectures: ["DreamGPT"] +
运行时注册决定。000~231),
prompt 需以空格分隔的 3 位 id 构造;decode 为直接拼接,客户端应通过
logprobs 或按 3 位切分还原 token id。logit_bias
逐槽位施加(非法 id 置 -100),与上游 generate_state 的 legal_mask 等价。config.json、small_pipes.pt 完整下载。