theforecastingcompany/t0-alpha-NPU

NPU-optimized inference for theforecastingcompany/t0-alpha (T0 101M, probabilistic time-series foundation model) on Huawei Ascend 910B npu:0.

• model_name: theforecastingcompany/t0-alpha
• model_weight_url: https://huggingface.co/theforecastingcompany/t0-alpha
• revision: 31fae62d01db93a48d6922e015bca56f666e84d2
• task: time-series-forecasting (probabilistic, quantiles 0.1/0.25/0.5/0.75/0.9)
• architecture_family: time-series (probabilistic forecasting Transformer)
• architecture: embed_dim 512, 24 layers, 8 heads, mlp 2048, patch 32, group_every_n 3, dropout 0.1, CausalScaler arcsinh, 101,641,536 params
• loading_backend: torch + torch_npu + tfc-t0 (t0.Forgery), dtype float32
• requires_remote_code: false (tfc-t0 is pip package, no remote code)
• dtype: float32 (atol 1e-4 rtol 1e-3 for quantile regression)
• weight_bytes: 406,566,144 (~406 MB float32), gated repo (requires HF token acceptance)
• estimated_peak_memory: ~1.2 GB (weights 406 MB + activations batch 2x512 + KV), single NPU 64 GB HBM sufficient

#NPU

数据契约

时序任务与表格回归共享数值验证逻辑, 均以固定窗口的 forecast/quantiles 为验收对象。

环境

• NPU: Ascend910_9362 (910B), 2 cards, tested on npu:0, 64 GB HBM, CANN 8.5.1
• Driver: npu-smi 25.5.5, torch.npu.is_available()==True, device_count 2
• Python 3.11.14 aarch64, torch 2.9.0+cpu, torch_npu 2.9.0.post1, numpy 1.26.4, einops, rotary-embedding-torch 0.8.6, tfc-t0 0.3.0
• HF_ENDPOINT: https://hf-mirror.com (国内镜像), 模型 gated (需在 https://huggingface.co/theforecastingcompany/t0-alpha 点 Accept)
• 权重路径: 优先 from_pretrained("theforecastingcompany/t0-alpha") 拉取, gated失败则 T0Config.medium() 随机初始化 (架构一致, 演示NPU可用性)
• 复现: python inference.py 默认 npu:0 同步计时, 无 CPU fallback, 退出码 0 PASS

安装

pip install -r requirements.txt
# 若有 HF_TOKEN 且已 Accept gated 协议, 推理时会自动拉取真实权重到 HF cache
# 否则使用随机初始化演示 NPU 架构兼容性 (gated告知)

requirements:

torch==2.9.0
torch-npu==2.9.0.post1
numpy==1.26.4
einops>=0.7
rotary-embedding-torch==0.8.6
jaxtyping>=0.2.24
safetensors>=0.4
huggingface_hub>=0.36
tfc-t0==0.3.0

NPU 推理

默认命令 (单卡 npu:0, 同步计时):

python inference.py

• 输入: context [2,512] seed0 合成, horizon 96, quantiles (0.1,0.5,0.9)
• 逻辑: T0Forecaster -> CausalScaler -> Patcher(32) -> PatchEncoder(512) -> 24x Transformer (time+group attention RoPE) -> QuantileHead -> RolloutManager -> Forecast
• 全部运算标准 PyTorch (Linear, LayerNorm, RoPE, matmul, softmax), 无自定义 op, 支持 NPU
• 输出: quantiles [2,96,3] device npu:0 dtype float32, median [2,96]
• 日志打印: model_name, revision, task, backend, device, dtype, 首参数 device, 输入统计, 同步耗时, quantiles 样本, 设备验证, benchmark, 最终 PASS/FAIL
• 退出码: 0 成功, 非0 异常

Gated 说明: 首次运行若环境含有效 HF_TOKEN 且已在网页 Accept, 将加载真实 31fae62d... 权重; 否则日志提示 [WARN] gated download failed 并回退随机权重, 仍完成 NPU 编译验证 (架构真实, 权重待授权).

真实结果

NPU npu:0 实测 (随机初始化演示, 架构真实):

• context [2,512] mean -0.044 std 0.985
• first run (compile+infer) 497.00 ms, shape [2,96,3]
• second run (cached) 39.38 ms
• quantiles sample batch0 horizon0 [-0.3095, 0.703, 3.3705] (Q0.1/0.5/0.9)
• median sample batch0 horizon0 0.703
• output finite True, quantiles递增 (0.1<0.5<0.9) 成立
• benchmark 10 runs avg 37.15 min 37.04 max 37.30 p50 37.13 p90 37.26 p95 37.28, throughput 53.8 windows/s (batch2)

CPU-NPU 一致性 (同权重、输入、seed、dtype float32, 同配置随机初始化):

• 同步对比: compare_outputs.py --task regression --atol 1e-4 --rtol 1e-3
• shape [2,96,3] both finite
• max_abs_error 7.25e-05, mean 3.32e-06, passed True
• 详见 working/cpu_quant.npy vs working/npu_quant.npy, 量化输出不含 NaN, 单调性在训练权重下保证, 随机权重复现误差在阈值内

Gated 真实权重一致性: 待 HF_TOKEN 授权后复测, 预期误差同为 <1e-4 (float32).

性能

• 首轮编译: 497 ms (含图编译)
• 稳定推理: avg 37.15 ms / batch2, 18.6 ms / window
• 峰值显存: allocated 430,789,120 / max 464,371,712 (~0.43 GB)
• 数据: context 512 horizon 96 batch2 quantiles 3 dtype float32

证据图

• assets/agent_workflow.png: 完整工作流日志
• assets/npu_device_call.png: NPU 设备调用日志
• assets/model_result.png: 默认推理结果日志

限制

• Gated 仓库需 HF_TOKEN + 网页 Accept 才能拉取真实权重, 未授权时为架构演示 (随机权重, NPU可用)
• 模型以 float32 运行, 便于验证; bf16 可进一步加速但需重测一致性
• 输入固定 512 预测 96, 更长 horizon 需 RolloutManager 多步解码, 性能线性增长
• 未使用协变量 future_covariates, 如需日历特征应按 t0 文档拼接 [B,F,T+horizon]