FloodDiffusion: Tailored Diffusion Forcing for Streaming Motion Generation (TINY)
本仓库是 AlayaLab/FloodDiffusionTiny
(用户侧 id:ShandaAI/FloodDiffusionTiny)在 华为昇腾 Ascend910 NPU (torch_npu) 上的适配工程。
模型为 Latent Diffusion Forcing (WAN-tiny based) 文本到人体动作生成模型: 给定动作描述文本,输出 HumanML3D 263 维动作特征序列(或 22×3 关节点坐标)。
cuda 设备调用(grep -rin cuda --include=*.py . 零命中)。
模型与输入均运行在 npu 设备,get_device() 内置 try-except 回退 CPU。ldf_models/tools/attention.py 已改写为
torch.nn.functional.scaled_dot_product_attention(带 padding mask),为该模型的默认注意力实现。rope_params/rope_apply 由 float64/complex128
改为 float32/complex64(精度足够,规避 NPU 算子限制)。google/umt5-base 文本编码器优先加载本地缓存 ./model_weights/google/umt5-base,
避免重复下载。inference.py 双模式(CLI + FastAPI),权重优先使用本地缓存 ./model_weights/<模型全名>。pip install -i https://repo.huaweicloud.com/repository/pypi/simple/ \
lightning diffusers omegaconf ftfy numpy safetensors einops huggingface_hub transformerspython3 download_weights.py AlayaLab/FloodDiffusionTinyexport ASCEND_RT_VISIBLE_DEVICES=1
python inference.py --input "a person walking forward" --length 60 --num_denoise_steps 10输出示例(263 维 HumanML3D 特征,帧数 ≈ length × 4):
{
"model": "AlayaLab/FloodDiffusionTiny",
"device": "npu:0",
"text": "a person walking forward",
"length": 60,
"feature_shape": [237, 263],
"sample_rows": [[...], ...],
"stats": {"min": ..., "max": ..., "mean": ..., "std": ...}
}输出 22×3 关节点坐标:
python inference.py --input "a person walking forward" --length 60 --output_joints --smoothing_alpha 0.5python inference.py --serve # http://0.0.0.0:8000GET /:带文本输入框的 HTML 页面POST /predict:接收 JSON {"text": "...", "length": 60} 或 multipart 表单,返回动作序列 JSONfrom transformers import AutoModel
model = AutoModel.from_pretrained(
"ShandaAI/FloodDiffusionTiny",
trust_remote_code=True,
local_files_only=False,
)
motion = model("a person walking forward", length=60) # (~240, 263)
joints = model("a person walking forward", length=60, output_joints=True) # (~240, 22, 3)@article{cai2025flooddiffusion,
title={FloodDiffusion: Tailored Diffusion Forcing for Streaming Motion Generation},
author={Yiyi Cai, Yuhan Wu, Kunhang Li, You Zhou, Bo Zheng, Haiyang Liu},
journal={arXiv preprint arXiv:2512.03520},
year={2025}
}