#NPU
A standalone single-card Ascend 910 NPU inference adaptation of
ibm-research/patchtst-fm-r1,
the IBM Research PatchTST-FM (patched time-series transformer foundation
model) with a probabilistic forecasting head (99 quantiles). The checkpoint is
loaded through the self-contained patchtst_fm model code embedded in
inference.py and runs entirely on npu:0 — real weights, real forward pass,
no CPU fallback, no random tensors.
| Field | Value |
|---|---|
| Model | ibm-research/patchtst-fm-r1 (PatchTST-FM, model_type=patchtst_fm) |
| Architecture | PatchTSTFMForPrediction — 20 transformer layers, d_model 1024, 16 heads, patch length 16, RevIN (asinh) normalization |
| Task | Time-series probabilistic forecasting — 99 quantile levels |
| Context | 8192 hours |
| Horizon | 64 hours |
| Channels | single-channel (ETTh1 OT) |
| Weights | 257.9 M params, model.safetensors ≈ 1.03 GB, float32 |
| License | cc-by-nc-sa-4.0 (research, non-commercial) |
| Revision | main |
ETTh1.csv from the ETDataset
(mirrored at raw.gitcode.com/gh_mirrors/et/ETDataset/.../ETTh1.csv),
SHA-256 1951ef88397ffebe8c5c8b2c9afaee1c54266cb65e985ceee8f275a56cd8cee1.date, HUFL, HULL, MUFL, MULL, LUFL, LULL, OT.OT column (float32, [8192]).[end-8192:end], ground truth [end:end+64]), the window
never crosses the data boundary.inputs = List[torch.Tensor] of 1-D float32 series,
prediction_length=64, quantile_levels=None.[2, 99, 64] float32 quantile predictions (99 levels × 64 horizon),
already inverse-scaled to original units. Median = quantile level 0.50
(index 49).model_type=patchtst_fm is not part of the transformers library, so the
model code is embedded into inference.py (self-contained). Two documented
changes were applied to the original IBM modeling code to make it run on the
Ascend NPU:
torch.ones/zeros(f_i) were created on the CPU while x_in lives on the
accelerator, causing a device mismatch in torch.cat — they are now created
with device=x_in.device (equivalent tensors, same values).torch.autocast device type is now chosen as "npu" when the input
tensors are on an Ascend NPU; otherwise the original cuda/mps/cpu logic is
preserved.No operator was replaced, no weights were re-initialized, and no CPU fallback is used during the NPU forward pass.
npu:0pip install -r requirements.txt# auto-downloads weights + ETTh1.csv, then runs real inference on npu:0
python inference.py
# local model/data (no downloads)
python inference.py --model-dir /path/to/model --data /path/to/ETTh1.csv
# include a 10-run timing benchmark
python inference.py --bench
# CPU baseline for cross-checking
python inference.py --device cpuinference.py prints a JSON summary (model, backend, device, dtype, first
param device, input/output shape, window, metrics, forecast, timing) and
writes forecast.npy, ground_truth.npy, result.json. Exit code is 0 only
on success.
npu:0 (Ascend910_9362), torch.npu.is_available()=True,
device_count=2npu:0, dtype torch.float32[8192] float32 · output: [2, 99, 64] float32[6.6251, 6.6842, 6.8002, 6.9797, 7.1858][6.9640, 6.8940, 7.4570, 7.4570, 8.4420]Same weights, same input windows, same preprocessing, same dtype (float32) and
seed. Raw [2, 99, 64] quantile outputs saved to .npz and compared with
compare_outputs.py (task=quantile, atol=1e-2, rtol=1e-2):
cpu_shape [2, 99, 64] npu_shape [2, 99, 64]
finite true
max_abs_error 0.01219
mean_abs_error 0.00111
cpu_quantiles_monotonic true
npu_quantiles_monotonic true
passed trueThe model was originally trained in float32 and the inference path runs under float16 autocast inside the transformer; the small residual difference is consistent with that dtype and within the chosen tolerance.
Timed with torch.npu.synchronize() around every forward pass; first run
(compile) reported separately, then 3 warmup + 10 timed runs:
first_run_ms 387.2
avg_ms 18.1
min_ms 18.1
max_ms 18.2
p50_ms 18.2
p90_ms 18.2
p95_ms 18.2
windows_per_s 55.1
peak HBM 984 MiBThe screenshots below were rendered with xterm.js from the real terminal logs of this adaptation run (not recreated by hand or simulation):
— model/data inspection, CPU
baseline, NPU inference, consistency and benchmark workflow.
— npu-smi, NPU availability,
device name, and model/input/output tensor device placement.
— default python inference.py
real output: schema, window summary, forecasts, metrics and status.ibm-granite/granite-timeseries-patchtst-fm-r1 variant for commercial use.OT channel of ETTh1 is demonstrated; the model also
supports multivariate inputs.