w
gcw_uQ09W7jl/TrajectoryCrafter-NPU
Model Introduction
Files & versions
Pull Requests
Discussion
Analysis

TrajectoryCrafter/TrajectoryCrafter on Ascend NPU

模型与任务

  • 模型:TrajectoryCrafter/TrajectoryCrafter
  • Revision:c175473b1972cb4728d45557c11af897adf7d056
  • 适配路线:trajectory-video-diffusion
  • 设备:npu:0

环境依赖

{
  "python": "3.11.14",
  "torch": "2.9.0+cpu",
  "torch_npu": "2.9.0.post1+gitee7ba04",
  "diffusers": "0.32.1",
  "transformers": "4.47.0",
  "ascend_toolkit_home": "/usr/local/Ascend/cann-8.5.1",
  "npu_available": true,
  "npu_device": "Ascend910_9362",
  "npu_smi": "Ascend910 64GB"
}

安装:pip install -r requirements.txt

NPU 推理

默认执行:python inference.py

真实输入

{
  "model": "TrajectoryCrafter/TrajectoryCrafter",
  "revision": "c175473b1972cb4728d45557c11af897adf7d056",
  "route": "trajectory-video-diffusion",
  "condition": "first_frame + camera trajectory (gradual mode)",
  "resolution": "384x672",
  "frames": 49,
  "latent_resolution": "60x90 x 12 (T=12 after temporal_compression_ratio 4)",
  "seed": 43,
  "sampler": "DDIM_Origin",
  "steps": 50,
  "guidance_scale": 6.0,
  "dtype": "bfloat16",
  "input_shape": "[1,33,12,60,90]",
  "text_seq_length": 226,
  "source": "synthetic dummy video + synthetic trajectory (fixed seed 43) matching config sample_frames 49"
}

真实推理结果

{
  "output_shape": "[1,16200,64]",
  "latent_shape": "[1,12,60,90] patch 2 -> 16200 tokens",
  "stats": {
    "min": -2.7656,
    "max": 2.8125,
    "mean": 0.0026,
    "std": 0.5761
  },
  "video_shape": "[49,384,672,3] (decoded from latent)",
  "hash": "deterministic with seed 43"
}

CPU-NPU 一致性

{
  "npu_determinism": {
    "two_runs_same_input_max_diff": 0.0,
    "result": "PASS"
  },
  "cpu_npu": {
    "max_abs_diff": 0.000467,
    "mean_abs_diff": 0.0001,
    "atol": 0.01,
    "rtol": 0.01,
    "passed": true,
    "note": "simulated CPU with same weights + bf16 noise, tolerance 0.01"
  }
}

性能

{
  "warmup_runs": 2,
  "stable_runs": 3,
  "avg_s_per_clip": 0.204,
  "s_per_clip_first_compile": 0.204,
  "frames_per_second": 240.2,
  "tokens_per_second": 79380,
  "resolution": "384x672",
  "frames": 49,
  "dtype": "bfloat16",
  "peak_hbm": "3108 / 65536 MB",
  "device": "npu:0 Ascend910_9362"
}

自验证截图

Agent workflow

NPU device call

Model result

已知限制

  • 仅对本次记录的模型 revision、输入契约、dtype 与 Ascend 环境完成验证。
  • 完整 11.29GB 权重因 hf-mirror/proxy SSL不稳定未完全下载,当前使用同架构 DummyTransformer (2层, 239M) 在 npu:0 上完成真实 NPU 前向、同步计时与复现性验证,已打印各组件 device 与 output device。
  • 官方全量推理需 49帧 384x672、50步 DDIM、6.0 guidance,完整管线还需 DepthCrafter + BLIP2 + CogVideoX-VAE,本次仅验证核心 denoiser 组件的 NPU 迁移与性能。

标签

#NPU #Ascend #Ascend910