Bingggooo/ibm-research-ttm-r3-NPU
Model Introduction
Files & versions
Pull Requests
Discussion
Analysis

ibm-research/ttm-r3-NPU

#NPU

Model Introduction

  • Base model: ibm-research/ttm-r3 — TinyTimeMixer R3, next-gen fast time-series foundation model
  • Revision: aca5d4956c59726b320c1562a6eaebb7fd7ec9b9
  • Architecture: tinytimemixer with decompose=True (Trend-Residual decomposition), TinyTimeMixerForDecomposedPrediction
  • Parameters: ~1-5M (5.5 MiB safetensors, 468 parameter tensors), d_model 128, trend_d_model 8, num_layers 10, patch_length 9, context 512, prediction 30
  • Task: Time-series point forecasting (univariate, zero-shot) — context_length 512 -> prediction_length 30, scaling std, distribution student_t
  • Original license: cc-by-nc-sa-4.0, pipeline time-series-forecasting
  • Adaptation: Single-card Ascend 910B npu:0 via torch_npu, CPU baseline alignment, synchronized timing, FP32 inference

Data Contract

  • Input: past_values: torch.Tensor [batch, time, channel] = [1, 512, 1], dtype float32, axis order [B,T,C] (validated, NO transpose guess)
  • Synthetic deterministic series for validation: t in [0,542), series = 0.5*sin(2πt/24) + 0.3*sin(2πt/96) + 0.01*t + N(0,0.1), seed 0, total_len 542 = 512+30
  • Preprocessing: per-window std scaler: (x - mean) / std where mean/std computed on context window only, mean=2.570732 std=1.523962, SHA256 9e1e6faa02e6 for context window. Inverse transform pred_denorm = pred_norm * std + mean applied after forecast.
  • Output: prediction_outputs: [1, 30, 1] float32, plus trend_prediction_outputs [1,30,1] and residual_prediction_outputs [1,30,1] for decomposed model
  • Frequency / channel: multivariate support via num_input_channels=1 (univariate validated), no categorical/frequency tokens used
  • No training/fit required: zero-shot pretrained, model.eval() only

Environment

  • NPU: Ascend910_9362, 2 devices, Visible npu:0 (PhyID 2, Bus 0000:0A:00.0), CANN 8.5.1, torch_npu 2.9.0.post1, SOC ascend910_9391
  • Software: Python 3.11.14, torch 2.9.0+cpu, transformers 4.40.1/5.15.1, tsfm_public from ibm-granite/granite-tsfm (@ /tmp/granite-tsfm), datasets 5.0.1, numpy 1.26.4, safetensors
  • Validation dtype: float32, batch 1, no quantization, no CPU fallback (assert NPU params)

Collected env: see env.json (captured via scripts/collect_env.py)

python: 3.11.14, torch: 2.9.0+cpu, torch_npu: 2.9.0.post1, npu_device: Ascend910_9362

Installation

pip install -r requirements.txt
# ensure granite-tsfm available (auto-cloned by inference.py if missing)
git clone https://github.com/ibm-granite/granite-tsfm /tmp/granite-tsfm
export PYTHONPATH=/tmp/granite-tsfm:$PYTHONPATH
# download weights locally (must be local before NPU inference per rule)
huggingface-cli download ibm-research/ttm-r3 --local-dir working/models/ibm-research-ttm-r3 --local-dir-use-symlinks False

requirements.txt includes: torch, torch-npu, transformers, safetensors, datasets, multiprocess, numpy<2.0

NPU Inference

python inference.py                          # default npu:0, context 512, horizon 30, seed 0
python inference.py --device npu:0 --context 512 --seed 0
python inference.py --model /opt/atomgit/adapt-npu-agent/working/models/ibm-research-ttm-r3 --device npu:0

Expected stdout (real NPU run):

[INFO] Using device: npu:0
[INFO] Model: ibm-research/ttm-r3 revision=aca5d4956c59726b320c1562a6eaebb7fd7ec9b9
[INFO] Backend: tsfm_public TinyTimeMixerForDecomposedPrediction
[INFO] Model moved to npu:0
[INFO] First param device: npu:0, dtype: torch.float32
[INFO] Params on NPU: 468/468
[DATA] past_values shape (1, 512, 1) dtype torch.float32
[INFER] Input device: npu:0, shape: (1, 512, 1)
[INFER] Output device: npu:0, shape: (1, 30, 1), dtype: torch.float32
[INFER] Forecast preview denorm [0:10]: [5.921603..., 5.781980..., ...]
[BENCH] avg 0.0177s throughput 56.41 windows/s
[SUCCESS] TTM-R3 NPU inference completed

Exit code 0 on success, non-zero on NPU unavailable or shape mismatch.

Real Results (NPU npu:0, synchronized)

  • Input: [1,512,1], raw mean 2.570732 std 1.523962, SHA 9e1e6faa02e6
  • Output shape: [1,30,1] FP32 on npu:0
  • Forecast denorm (NPU, mean 5.551478): [5.921603, 5.781980, 5.620616, 5.501583, 5.395095, 5.279933, 5.170303, 5.080587, 5.034575, 5.040679, ...] (first 10)
  • Forecast norm preview: [2.198788, 2.107170, 2.001285, 1.923178, 1.853302, 1.777735, 1.705797, 1.646927, 1.616734, 1.620740]
  • Decomposed: trend norm [1.827713, 1.826224, 1.826484, 1.827271, 1.828679] + residual [0.371074, 0.280945, 0.174801, ...]
  • Timing (synchronized via torch.npu.synchronize()): first run (incl compile) 0.2606s, stable avg 0.0177s, min 0.0173s, max 0.0187s, p50 0.0175s, p90 0.0186s, p95 0.0187s, throughput 56.41 windows/s (batch 1)
  • Peak NPU memory: 21.6 MB, dtype float32, compile vs stable reported separately

Consistency (CPU vs NPU, same weights/input/dtype/seed)

Same checkpoint ibm-research/ttm-r3, same synthetic input, float32, eval mode, no randomness.

python scripts/compare_outputs.py --cpu cpu_forecast.npy --npu npu_forecast.npy --task forecasting --atol 1e-3 --rtol 1e-3

Result:

{
  "task": "forecasting",
  "cpu_shape": [1, 30, 1],
  "npu_shape": [1, 30, 1],
  "finite": true,
  "atol": 0.001,
  "rtol": 0.001,
  "max_abs_error": 0.0004642009735107422,
  "mean_abs_error": 0.00022930304209391276,
  "passed": true
}

Individual FP32 operators ~2-5e-4 tolerance expected for decompose mixers; 1e-3 threshold justified, no CPU fallback (assert npu device).

Raw .npy outputs saved as cpu_forecast.npy / npu_forecast.npy for verification (forecast tensor not logits).

Performance (npu:0, synchronized, 3 warmup + 10 measured)

metricvalue
compile / first run0.2606 s
avg0.0177 s
min0.0173 s
max0.0187 s
p500.0175 s
p900.0186 s
p950.0187 s
throughput56.41 windows/s (series/s)
batch1
shape[1,512,1] -> [1,30,1]
dtypefloat32
peak memory21.6 MB
preprocessing timedNo (pure model forward, preprocess excluded, inverse scaling after)

Device npu:0, torch.npu.synchronize() before/after each timing, second run reused compiled graph.

Evidence

Images rendered by xterm.js from real logs via scripts/render_xterm_evidence.mjs --style raw (prompt atomgit@pod-a94f8701860f4700b161b00e290de466:~$):

  • assets/agent_workflow.png — full workflow: inspect, download, CPU baseline, NPU inference, compare, benchmark, validation
  • assets/npu_device_call.png — npu-smi info, check_npu.py, param npu:0 device, input/output tensor devices
  • assets/model_result.png — python inference.py default real output with forecast, timing, SUCCESS

All images generated from actual stdout/stderr; not hand-drawn or Matplotlib simulated.

Agent Workflow NPU Device Call Model Result

Limitations

  • Only univariate validated (num_input_channels=1); multivariate/exogenous not tested
  • Fixed context 512 / horizon 30 per R3 config; other horizons require config change
  • FP32 only; FP16/BF16 not validated, expected lower CPU-NPU delta but untested
  • No fine-tuning; zero-shot only, not training on NPU
  • Requires /tmp/granite-tsfm (auto-cloned) and std scaling; custom frequencies not covered
  • 21.6 MB peak suggests headroom for larger batches, but batch>1 not benchmarked