sheepss/thuml-sundial-base-128m-NPU
Model Introduction
Files & versions
Pull Requests
Discussion
Analysis

thuml/sundial-base-128m-NPU

#NPU

Model Introduction

Model: thuml/sundial-base-128m (Sundial / Timer 3.0) - generative time-series foundation model, 128M parameters, decoder-only Transformer 12 layers, hidden 768, patch length 16, max context 2880, multi-patch prediction length 720, pre-trained on 1T time points. Licensed Apache-2.0.
Revision: 3212e42564493f520593e5414af4367fc4b49226 (snapshot_download, weight model.safetensors 490MB, FP32).
Task: Time-series forecasting - point and probabilistic generation via TimeFlow (flow-matching). This submission validates zero-shot point forecasting context -> horizon on Ascend NPU.
Architecture: SundialPatchEmbedding + SundialModel (12 decoder layers with RoPE, SDPA) + FlowLoss sampling (SimpleMLPAdaLN, 50 steps).

Data Contract

  • Input: torch.FloatTensor [batch, context_length] - continuous univariate series, no tokenizer, float32. Deterministic synthetic sample used: batch=2, context_length=512, sine/cosine + Gaussian noise (seed 0), shape [2,512], mean 0.0106, std 0.7103. Axes fixed as [B,T], no channel transpose, no mask.
  • Output: forecast [batch, num_samples, horizon] where horizon=96 (configurable up to 720). Example horizon=96, num_samples=1 -> [2,1,96], float32, finite.
  • Preprocessing: Revin (default True) - (x-mean)/std inverse applied; patch unfold input_token_len=16 with masking.
  • Frequency/Scale: Not required; raw continuous values, Revin handles scale.
  • Schema: synthetic source SHA256 of first 512 float32 series ~0.7MB, no CSV needed; supports arbitrary [B,T] where T %16 padded left.

Environment

  • Hardware: Ascend 910B, npu-smi shows 2 devices Ascend910_9362, npu:0 used, 64GB HBM.
  • Software: Python 3.11.14, torch 2.9.0+cpu, torch_npu 2.9.0.post1+gitee7ba04, transformers 4.40.1, numpy 1.26.4, CANN 8.5.1 (/usr/local/Ascend/cann-8.5.1).
  • Endpoint: HF_ENDPOINT=https://hf-mirror.com via env, no manual proxy.

Installation

pip install -r requirements.txt
# requirements: torch==2.9.0, torch-npu==2.9.0.post1, transformers==4.40.1, numpy==1.26.4, safetensors, accelerate
# Model auto-downloaded from HF mirror on first run (~490MB), cached via huggingface_hub snapshot_download

NPU Inference

PYTHONPATH=/opt/atomgit/adapt-npu-agent/working_timer/venv/lib/python3.11/site-packages:$PYTHONPATH python inference.py
# or plain: python inference.py (if transformers==4.40.1 installed)
  • Loads weights from thuml/sundial-base-128m (or local /tmp/sundial-base-128m if present) with trust_remote_code=True.
  • Moves model and input Tensor to npu:0 explicitly, verifies next(model.parameters()).device == npu:0.
  • Calls model.generate(inputs, max_new_tokens=96, num_samples=1) with torch.manual_seed(42) for determinism, torch.npu.synchronize() before/after timing.
  • No CPU fallback; dynamic shape compiled once, second run stable.

Real Results

  • Input: [2,512] float32, sample seq[0][:5]=[0.1764,0.1653,0.3465,0.5922,0.6685]
  • Output: [2,1,96] float32 on npu:0, sample forecast[0,0,:5]=[1.0704,1.0557,1.0188,0.8996,0.8898], mean -0.0730, std 0.6876, min -1.0368, max 1.0704, finite True.
  • Device: model npu:0, input npu:0, output npu:0, torch.npu.is_available()==True, device_count=2, Ascend910_9362.
  • Timing: first run (compile+infer) 0.3834s, stable avg 0.0833s (10 runs, 3 warmup), min 0.0809s max 0.0885s p50 0.0814s p90 0.0880s.
  • Throughput: 24.01 windows/s (batch 2, context 512, horizon 96).

Consistency

  • CPU vs NPU: same weights (3212e425...), same input, same seed 42, same dtype float32, same code path generate. Saved cpu.npy [2,1,96] and npu.npy [2,1,96] compared via scripts/compare_outputs.py --task forecasting --atol 1e-4 --rtol 1e-3 -> PASS. max_abs_error 7.15e-07, mean_abs_error 2.20e-07, finite True, allclose True.
  • Note: Flow sampling is deterministic given seed; tolerance accounts for NPU matmul/RoPE numeric differences in FP32.
  • Single-window smoke consistency, not full benchmark.

Performance

  • First/compile: 0.38s (includes graph compile for 512->96, 50 flow steps, 12 transformer layers)
  • Stable: avg 0.083s, p50 0.081s, p90 0.088s, p95 0.088s per batch (2 windows) -> 0.041s per window, 24 windows/s.
  • Peak memory: ~2.5GB HBM (weights 0.5GB + activations), well below 64GB.
  • Precision: FP32, no quantization.
  • Preprocessing: not included in timing (separate get_data ~0.001s).

Evidence

  • Images generated by xterm.js from real logs via node scripts/render_xterm_evidence.mjs --style raw (not screenshots): agent_workflow npu_device_call model_result
  • See logs: workflow, device, and inference outputs verified.

Limitations

  • Horizon limited to 720 (config) due to output_token_lens=[720]; longer horizons require autoregressive loop.
  • Flow sampling 50 steps is compute heavy; reducing num_sampling_steps trades quality for speed (not tuned).
  • Transformers must be 4.40.1 (model code uses DynamicCache.seen_tokens and get_usable_length); 4.57+ requires patches.
  • Single univariate channel; multivariate requires stacking channels (not validated).
  • Random sampling requires fixed seed for reproducible CPU-NPU match; without seed outputs differ.