This delivery adapts the EPFL-VILAB/videoflextok_d18_d18_k600
video tokenizer / reconstructor so that it runs end-to-end on a physical Ascend
NPU (torch_npu, device npu:7 — Ascend910B4-1, CANN 8.5.1) while preserving
the official VideoFlexTokFromHub architecture and strict-loading all real
pretrained tensors.
| Asset | Preview |
|---|---|
| Agent / pipeline workflow | ![]() |
| NPU-7 device-call capture | ![]() |
| Input vs reconstruction | ![]() |
| Live session (added by orchestrator) | ![]() |
License note. The README frontmatter uses
license: otherbecause the delivery mixes two licenses: the model weights are Apache-2.0 (per model card) and the official source code is Apple/EPFL Sample Code License 1.1, non-commercial. See LICENSE and ACKNOWLEDGEMENTS.md.
VideoFlexTok is a flexible-length coarse-to-fine video tokenizer. The d18/d18
K600 checkpoint uses 18 transformer encoder layers and 18 rectified-flow
decoder layers, an FSQ bottleneck with levels [8, 8, 8, 5, 5, 5], and a
causal VidTok VAE (16 channels, 4×8×8 compression). The full neural path exercised
here is:
VidTok VAE encode -> 18 encoder layers (dim 1152, 18 heads)
-> FSQ tokens (levels [8,8,8,5,5,5])
-> 18 rectified-flow decoder layers (adaLN, >= 1 genuine denoising step)
-> VidTok VAE decodevideoflextok/ package)torch.compile(flex_attention) (CUDA/Inductor-specific) is replaced with
native torch.nn.functional.scaled_dot_product_attention. The existing
BlockMask.mask_mod is materialized into a dense boolean token mask on the
NPU. self.scale is preserved; a non-null score_mod is explicitly rejected
(no silent semantic drop).torchvision dependency in utils/misc.py is removed; the
channel-wise denormalize is implemented with tensor arithmetic.npu/cuda/cpu from the tensor).diffusers is not required (only a type annotation upstream); annotations are
relaxed to Any.safetensors.torch.load_file(..., device="cpu")
and verified (741 F32 tensors, 1,057,722,429 elements, prefix
video_tokenizer., SHA-256 matches the immutable audit hash).No transformer layer is approximated, pruned, fabricated or skipped.
| Component | Version | Note |
|---|---|---|
| Container OS | — | Ascend container |
| torch | 2.9.0+cpu | provided by runtime — do NOT install/upgrade |
| torch_npu | 2.9.0 | provided by runtime — do NOT install/upgrade |
| CANN | 8.5.1 | ASCEND_HOME_PATH=/usr/local/Ascend/cann-8.5.1 |
| Python | 3.11.14 | venv |
| numpy | 1.26.4 | pinned in requirements.txt |
| safetensors | 0.8.0 | pinned |
| hydra-core / omegaconf | 1.3.5 / 2.3.1 | pinned |
| mup | 1.0.0 | installed with --no-deps |
| einops / PyYAML / tqdm / huggingface_hub | pinned | see requirements.txt |
| Physical device | npu:7 | Ascend910B4-1, 8 NPUs, ~62.4 GiB usable HBM per device |
Install command (all pip via the Aliyun mirror, never touching torch/torch_npu):
pip install -i https://mirrors.aliyun.com/pypi/simple/ \
--trusted-host mirrors.aliyun.com \
hydra-core==1.3.5 omegaconf==2.3.1
pip install -i https://mirrors.aliyun.com/pypi/simple/ \
--trusted-host mirrors.aliyun.com --no-deps mup==1.0.0The model weights stay external at the immutable cache:
/work/pipeline/models/videoflextok/videoflextok-d18-d18-k600/config.json/work/pipeline/models/videoflextok/videoflextok-d18-d18-k600/model.safetensorspython scripts/narrow_check.pypython inference.py --device npu:7 --out-dir .This runs the deterministic CC0 moving-gradient fixture [1, 3, 17, 128, 128]
normalized to [-1, 1] through the complete neural path with timesteps=1 and
guidance_scale=1.0 (honestly labeled a compatibility smoke test, not a quality
claim). Artifacts written under logs/:
logs/inference.log — full text loglogs/inference.pid — OS PID written by os.getpid()logs/npu_result.json — machine-readable result payloadlogs/npu_device_call.log — concurrent npu-smi device-call monitor capturelogs/input_fixture.npy, logs/reconst.npy — small render intermediatespython scripts/render_assets.pypython scripts/validate_delivery.pyDeterministic, self-authored CC0 moving-gradient fixture:
[1, 3, 17, 128, 128], normalized to [-1, 1]7d9d740e6fd478556f5516189cd88c35b7849a4f16c04fccd4e2492359e00756timesteps=1, guidance_scale=1.0, no normalized guidanceNeural path executed: VidTok VAE encode → 18 encoder layers → FSQ → 18 rectified-flow decoder layers (one genuine denoising step) → VidTok VAE decode.
Final command and observed output:
$ python inference.py --device npu:7 --out-dir .
...
2026-08-21 09:40:40 INFO Weight integrity verified (size + sha256)
2026-08-21 09:40:42 INFO Fixture ready: shape=(1, 3, 17, 128, 128) range=[-0.762, 0.762] dtype=bfloat16 checksum=7d9d740e6fd478556f5516189cd88c35b7849a4f16c04fccd4e2492359e00756
2026-08-21 09:40:50 INFO Tokenize done in 7.48s: 1 chunk(s), tokens shape=(1, 5, 256) (device=npu:7)
2026-08-21 09:40:50 INFO Detokenize done in 0.19s: 1 chunk(s), reconst shape=(1, 3, 17, 128, 128) (device=npu:7)
2026-08-21 09:40:50 INFO Inference finished successfully (total 38.89s)
$ echo $?
0Exit code 0. logs/npu_device_call.log contains the exact Python PID 214264
in the physical NPU-7 process table.
Recorded in logs/npu_result.json (final run, PID 214264):
| Item | Value |
|---|---|
| exit_code | 0 |
| device | npu:7 (Ascend910B4-1, 8 NPUs, 62,420 MB total HBM) |
| dtype | bfloat16 (recommended on Ascend; fp32 available via --dtype fp32) |
| strict load | 741 F32 tensors, 0 missing / 0 unexpected keys |
| weight | 4,230,998,164 bytes, SHA-256 62164b...c0347b5f, verified |
| tokenization | tokens shape [1, 5, 256] (1280 tokens, 1116 unique, range 6..63974) |
| reconstruction | shape [1, 3, 17, 128, 128], finite, range [-1.000, 1.047], mean 0.043, std 0.602 |
| MSE | 0.02442 |
| PSNR | 22.14 dB |
| timings | construction 19.86 s, strict load 0.03 s, sha256 4.23 s, tokenize 7.48 s, detokenize 0.19 s, total 38.89 s |
MSE/PSNR are labeled compatibility metrics computed on [-1,1] normalized
pixels (dynamic range 2.0) against the deterministic input. With a single
denoising step they measure that the pipeline runs and produces finite,
structured output — they are not a claim about reconstruction quality.
| Symptom | Likely cause / fix |
|---|---|
torch.npu.is_available() is False | torch_npu not loaded; confirm container torch==2.9.0+cpu + torch_npu==2.9.0, CANN 8.5.1 |
NPU device npu:7 unavailable | fail-closed; check npu-smi info that device 7 exists |
strict load has N missing/unexpected keys | fail-closed; model/checkpoint mismatch — verify the immutable model cache revision |
tensors found off the target device | fail-closed; model not fully placed on npu:7 |
ArgSort ... running on AiCpu warning | benign CANN kernel warning (int32/64 sort falls back to AiCPU) |
TypedStorage is deprecated warning | benign torch_npu internal warning |
posembs[slices] UserWarning | upstream indexing deprecation notice, harmless |
timesteps=1) on a synthetic moving
gradient. It validates placement, dtype, strict load and the full neural path;
it does not demonstrate reconstruction quality or generation fidelity.npu:7 is used (per container contract).torch / torch_npu must not be modified by pip.EPFL-VILAB/videoflextok_d18_d18_k600
model card).license: other.See LICENSE and ACKNOWLEDGEMENTS.md.
| Entity | Value |
|---|---|
| Model revision | b1e468168ba7a1869206d3fc74e369c095895722 |
| Official source revision | 4fd72ec0c0c8c05f879012cce285d5a14c81920f |
model.safetensors size | 4,230,998,164 bytes |
model.safetensors SHA-256 | 62164b971db5e226ef36a4212aef7ea256ad8accd09638c8d7bb63edc0347b5f |
| Test-input fixture SHA-256 | 7d9d740e6fd478556f5516189cd88c35b7849a4f16c04fccd4e2492359e00756 |