_class_name: ControlNetModel, in_channels: 4, controlnet_conditioning_channel_order: rgb, cross_attention_dim: 768, block_out_channels: [320,640,1280,1280]torch.npu.is_available()==True, device_name=Ascend910_9362, torch.npu.device_count()==2, NPU op x.npu() @ x.npu() sync PASSpip install torch==2.9.0 torch_npu==2.9.0.post1 diffusers==0.40.0 transformers accelerate numpy Pillow huggingface_hub safetensors
# 或
pip install -r requirements.txt默认执行真实 NPU 推理 (ControlNet + SD1.5 FP16, npu:0):
python inference.py
python inference.py --mode validate
python inference.py --mode benchmark --steps 8
python inference.py --prompt "a beautiful landscape with mountains" --steps 8 --height 512 --width 512 --seed 0预期输出包含:
输入: 合成 inpaint 条件图 512x512, 原图 200灰度 + 中心 100,150,200 矩形, 掩码 255白底中心 0黑 (128阈值), 经 image[mask<128] = -1.0 处理, prompt a beautiful landscape with mountains, seed 0, steps 8, guidance 7.5, height 512, width 512, FP16, npu:0
control_image: shape [1,3,512,512], device npu:0, dtype float16, min -1.000 max 0.784 mean 0.335
[INFO] Model: OrderAndChaos/controlnet-inpaint-endpoint
[INFO] Task: image-generation / inpainting (ControlNet + Stable Diffusion v1.5)
[INFO] Loading backend: diffusers ControlNetModel + StableDiffusionControlNetPipeline
[INFO] Requested dtype: float16
[INFO] Device: npu:0 (Ascend910)
[INFO] torch_npu available: True
[INFO] device_name: Ascend910_9362
[INFO] device_count: 2
[INFO] torch version: 2.9.0+cpu
[INFO] torch_npu version: 2.9.0.post1+gitee7ba04
[INFO] NPU op success: True
[INFO] Verified Device: npu:0 (Ascend910_9362)
[INFO] ControlNet dir: /tmp/controlnet
[INFO] SD dir: /tmp/sd_full2
[INFO] Loading ControlNet from /tmp/controlnet with dtype=torch.float16 ...
[INFO] ControlNet loaded: ControlNetModel
[INFO] ControlNet device: npu:0
[INFO] Loading SD pipeline from /tmp/sd_full2 ...
[INFO] Scheduler: UniPCMultistepScheduler
[INFO] Pipe moved to npu:0
[INFO] unet device: npu:0, dtype: torch.float16
[INFO] vae device: npu:0, dtype: torch.float16
[INFO] text_encoder device: npu:0
[INFO] controlnet device: npu:0
[INFO] Created synthetic image (512, 512) and mask (512, 512)
[INFO] Prompt: a beautiful landscape with mountains
[INFO] Steps: 8, Guidance: 7.5, H: 512, W: 512, Seed: 0
[INFO] control_image shape: torch.Size([1, 3, 512, 512]), device: npu:0, dtype: torch.float16
[INFO] control_image stats: min -1.000 max 0.784 mean 0.335
[INFO] Inference elapsed: 0.800s
[RESULT] Output size: (512, 512), mode: RGB
[RESULT] Output stats: min 0.0000 max 1.0000 mean 0.6546 std 0.1400
[INFO] Saved output to output_npu.png
[PASS] ControlNet inpaint inference succeeded with real weights on npu:0
[PASS] Verified real NPU execution via torch.npu (npu:0) - device Ascend910_9362torch.npu.synchronize() 计时output_npu.png (可在 /tmp/test_npu/output_npu.png 查看)a beautiful landscape with mountains, 512x512, seed 0, steps 8python inference.py --mode benchmark --steps 8 (warmup 3, 正式 10 次, 每轮重新加载 pipeline 并 torch.npu.synchronize 计时, 含 VAE decode)
[PERF] Runs: 10, avg 481.0ms, min 477.4ms, max 487.7ms, p50 480.2ms, p90 487.7ms, p95 487.7ms
[PERF] Steps: 8, Throughput: 16.63 steps/s, seconds/image 0.481s
[PERF] Image size: 512x512, batch=1, dtype=FP16, device=npu:0
[INFO] NPU memory free 53.84 GiB / total 61.27 GiBtorch.npu.synchronize() 前后, 非预处理混测assets/agent_workflow.png - 下载 (ControlNet 1.4G + SD1.5 2.1G FP16)、侦察 (config/架构)、加载 (ControlNet+SD FP16 variant to npu:0)、NPU 推理 (512x512 8步)、验证 (确定性 max_diff 0.0) 与性能 (481ms avg) 完整流程assets/npu_device_call.png - npu-smi info (Ascend910_9362, 0/14, 3105MB) 与 torch.npu.is_available()/device_name/参数 device/ dtype FP16assets/model_result.png - inference.py 真实 控制图像统计、生成图像 512x512、像素 mean 0.6546、耗时 0.800s、PASSaten::_linalg_solve_ex 在 NPU 上回落 CPU (diffusers 中少见, 但不影响结果, 仅提示)Hardware: NPU, NPU, Ascend, Ascend910, ControlNet, Stable-Diffusion, diffusion, image-generation, inpainting, diffusers