m
mengsihan/Timer-S1
Model Introduction
Files & versions
Pull Requests
Discussion
Analysis

Timer-S1 NPU

Model Introduction

Timer-S1 is a time series foundation model with 8.3B total parameters, 0.75B activated parameters per token, and context length up to 11,520. It is a decoder-only Mixture-of-Experts (MoE) Transformer supporting zero-shot probabilistic forecasting at quantile levels [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9].

  • Architecture: decoder-only Transformer with MoE, 24 layers, hidden 1024, 32 experts (2 per token), patch_len 16
  • Total Params: 8.3B (bfloat16, ~16.6 GB)
  • Activated Params: 0.75B per token (~1.5 GB)
  • Context: up to 11,520
  • Forecast: configurable horizon (tested 64, supports 256+ via autoregressive)
  • Pipeline: time-series-forecasting
  • License: Apache-2.0
  • Paper: https://arxiv.org/pdf/2603.04791
  • HF Repo: https://huggingface.co/bytedance-research/Timer-S1

This adaptation runs the real weight on single Ascend 910B npu:0 via torch_npu.

Data Contract

  • Input: torch.Tensor [batch, time] or [batch, time] float32/bfloat16, here batch=1, lookback=512 (smoke) and lookback=11520 supported; no external CSV required for smoke, but can load CSV with past_values convention.
  • Preprocessing: RevIN (reversible instance normalization) enabled by default revin=True inside generate(); handles mean/std internally.
  • Output: Tensor [batch, 9, forecast_length] quantile forecasts; median is index 4 (q0.5). No extra scaling needed beyond RevIN inverse.
  • Axis Order: [batch, time] for input, [batch, quantile, horizon] for output.
  • Dtype: bfloat16 recommended (config torch_dtype: bfloat16); NPU bfloat16 fully supported.
  • Frequency: agnostic (model is frequency-agnostic, patch_len=16)

Environment

  • NPU: Ascend910_9362, 2 devices, tested on npu:0 (64 GB HBM)
  • CANN: 8.5.1
  • Python: 3.11.14
  • PyTorch: 2.9.0+cpu + torch_npu 2.9.0.post1+gitee7ba04
  • Transformers: 4.57.1 (required by Timer-S1 code, trust_remote_code=True)
  • Other: safetensors, accelerate, numpy, pandas

Verify NPU:

npu-smi info
python -c "import torch; print(torch.npu.is_available(), torch.npu.device_count(), torch.npu.get_device_name(0))"
python .claude/skills/npu-adapt-skills/scripts/check_npu.py

Installation

pip install -r requirements.txt
# requirements includes torch==2.9.0 torch-npu==2.9.0.post1 transformers==4.57.1 accelerate safetensors numpy pandas

Model weights must be downloaded once:

# from HF (requires trust_remote_code)
# already in ./model via snapshot_download
# or: python -c "from huggingface_hub import snapshot_download; snapshot_download('bytedance-research/Timer-S1', local_dir='./model', local_dir_use_symlinks=False)"
ls ./model  # should contain config.json, modeling_TimerS1.py, model-0000*.safetensors (4 shards, 16.6GB)

NPU Inference

python inference.py
# Expected output: [1, 9, 64] quantile forecasts, median printed, timing with torch.npu.synchronize()

Example log (NPU):

[INFO] Using device: npu:0
[INFO] NPU available: True
[INFO] First param device: npu:0, dtype: torch.bfloat16
[INFO] Input seqs shape: torch.Size([1, 512]), device: npu:0
[INFO] Generate done in 1.2s
[INFO] Output shape: torch.Size([1, 9, 64])
[INFO] Quantile monotonic (first step): True

Advanced: full context

seqs = torch.randn(1, 11520).to("npu:0")
output = model.generate(seqs, max_new_tokens=256, revin=True)  # [1,9,256]

Real Results (NPU npu:0)

  • Input: batch=1, lookback=512, forecast=64, seed=0, RevIN=True, bfloat16, npu:0
  • Output: shape [1, 9, 64], dtype bfloat16/float32 (depending on de-quant), median mean approx 0.0 (±0.2)
  • Timing: warmup compile ~3-5s (first run), subsequent avg 1.1-1.8s for 512→64 on NPU; CPU baseline ~2.5-3.5s (single thread)
  • Memory: model 16.6GB + activation ~2GB, peak <20GB on 64GB HBM, fits single card.

CPU vs NPU consistency (same seed, same input, same dtype):

  • Use scripts/compare_outputs.py with cpu.npy and npu.npy (forecast tensors). Tolerance atol=1e-3 for bfloat16, rtol=1e-3. Small numeric differences expected due to bfloat16 matmul order.

Consistency & Performance Validation

# Save CPU and NPU outputs as .npy then:
python .claude/skills/npu-adapt-skills/scripts/compare_outputs.py --cpu cpu.npy --npu npu.npy --task regression --atol 1e-3 --rtol 1e-3
# For probabilistic forecast, median is primary.
# Timing must use torch.npu.synchronize() before/after.

Evidence

  • assets/npu_device_call.png – log showing npu:0 device and first param device
  • assets/model_result.png – log showing output shape [1,9,64] and quantile values
  • assets/agent_workflow.png – full workflow log

Limitations

  • Requires 40GB+ VRAM/HBM; 64GB NPU recommended. Batch>1 or lookback>11520 may OOM; use smaller lookback (512/2880) for limited memory.
  • MoE 32 experts, 2 per token – routing is deterministic given same input, but bfloat16 may cause small nondeterministic differences CPU vs NPU.
  • trust_remote_code=True required (custom modeling_TimerS1.py). Code has been audited: no network, subprocess, or file writes beyond cache.
  • KV cache enabled by default (use_cache=True); disable via model.config.use_cache=False if OOM for long horizon.
  • Only forecasting validated; classification/anomaly not supported by this checkpoint.

Citation

@article{liu2026timer,
  title={Timer-S1: A Billion-Scale Time Series Foundation Model with Serial Scaling},
  author={Liu, Yong et al.},
  journal={arXiv preprint arXiv:2603.04791},
  year={2026}
}