Zero-shot multivariate probabilistic time-series forecasting foundation model
adapted to run on a single Ascend 910 NPU (npu:0).
Datadog/Toto-2.0-22m (21.9M params, decoder-only patched transformer with
alternating time/variate attention, u-μP scaling, quantile output head trained with
pinball loss)[0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9]toto2 module)Toto 2.0 is the current-generation Datadog time-series foundation model family. The 22m checkpoint is the efficient default size (~7x fewer parameters than Toto 1.0 at comparable quality). It predicts quantiles for multivariate series in a single forward pass using contiguous patch masking (CPM) and a block-decode KV cache. This repo verifies real inference of the released checkpoint on an Ascend NPU.
| Field | Value |
|---|---|
| Input | target tensor (batch, n_variates, time_steps) fp32 |
target_mask | bool tensor same shape (observed mask) |
series_ids | long tensor (batch, n_variates) |
| Batch | 1 |
| Variates | 4 (default) |
| Context length | 512 (default) |
| Horizon | 96 (default) |
| Output | quantiles (9, batch, n_variates, horizon) at levels 0.1..0.9 |
| Axis order | [quantile, batch, variate, time] |
| Scaling | internal causal std scaler + arcsinh, inverse-transformed to real space |
npu:0)pip install -r requirements.txt
# torch_npu + torch must be installed for your Ascend CANN environmentWeights must be downloaded first to a local model/ directory:
huggingface-cli download Datadog/Toto-2.0-22m --local-dir model
python inference.py --model_dir model --context_length 512 --horizon 96 --n_variates 4inference.py is fully self-contained (embeds the Apache-2.0 toto2 model and the
dd_unit_scaling unit-scaling layer). It loads the checkpoint from local disk, moves
the model to npu:0, and produces a quantile forecast. It exits non-zero on any
failure. The script requires the unit-scaling pip package at runtime.
(9, 1, 4, 96), all finite, quantiles monotonic[0.0436, 0.0452, 0.0370, 0.0382, 0.0389, 0.0387, 0.0395, 0.0415][-1.2693, -1.2481, -1.2563, -1.2573, -1.2551, -1.2530, -1.2477, -1.2541][1.3846, 1.3729, 1.3559, 1.3625, 1.3477, 1.3619, 1.3523, 1.3507]Same weights, identical input values, dtype fp32, seed fixed, eval mode:
(9, 1, 4, 96), all finite3.58e-06, mean abs error: 4.35e-07np.allclose(atol=1e-4, rtol=1e-3) → PASS| Metric | Value |
|---|---|
| first run (kernel compile) | 0.354 s |
| avg latency (10 runs) | 0.034 s |
| p50 / p95 | 0.034 s / 0.034 s |
| throughput | ~29.5 forecast/s |
| peak NPU memory | ~0.15 GB (of 64 GB HBM) |
assets/agent_workflow.png - end-to-end adaptation workflow log (xterm.js from real log)assets/npu_device_call.png - NPU device / compute evidence (xterm.js from real log)assets/model_result.png - default python inference.py run on npu:0 (xterm.js from real log)target /
target_mask / series_ids following the data contract above.decode_block_size controls block-decode caching for longer horizons.