ALISY/controlnet-inpaint-npu
Model Introduction
Files & versions
Pull Requests
Discussion
Analysis

controlnet-inpaint-npu

ControlNet v1.1 inpaint checkpoint — OrderAndChaos/controlnet-inpaint-endpoint (duplicated from lllyasviel/control_v11p_sd15_inpaint) — adapted to run on Ascend NPU (torch_npu, device npu:0 / npu:1). Base model: runwayml/stable-diffusion-v1-5.

itemvalue
taskimage inpainting (ControlNet v1.1, SD1.5)
adapterControlNetModel (diffusers format, float32)
baserunwayml/stable-diffusion-v1-5
runtimePython 3.11 · torch 2.9.0 · torch_npu 2.9.0.post1 · CANN 8.5.1 · diffusers 0.40.0
deviceAscend 910, torch.device('npu:0') (auto-fallback npu:1)

All compute runs on the Ascend NPU. No Nvidia-specific device code anywhere in the adaption. Weights are cached locally under model_weights/ (git-ignored) and never uploaded to the repo — only code/config/sample inputs are published.

Install

pip install -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple/

Usage

CLI mode

The inpainting task needs original image + mask + prompt, so --img accepts a JSON file or an input directory:

# via JSON descriptor
python inference.py --img sample_input/input.json

# or via a directory containing original.png + mask.png (+ optional prompt.txt)
python inference.py --img sample_input

Output: a single JSON line on stdout, e.g.

{"model": "OrderAndChaos/controlnet-inpaint-endpoint",
 "result": {"image_size": [512, 512], "num_inference_steps": 20, "guidance_scale": 7.5,
            "seed": 0, "elapsed_sec": 42.3, "output_image": "saved/abc123.png",
            "output_base64_png": "..."}}

Useful flags: --steps 20 --guidance 7.5 --width 512 --height 512 --seed 0.

FastAPI mode

uvicorn inference:app --host 0.0.0.0 --port 8000
curl -s -X POST http://127.0.0.1:8000/predict \
     -F "file=@sample_input/original.png" \
     -F "mask=@sample_input/mask.png" \
     -F "prompt=a beautiful castle on a hill, golden hour light, highly detailed"

Returns {"model": "...", "result": {"image_size": ..., "num_inference_steps": ..., "seed": ..., "elapsed_sec": ..., "output_image": "saved/<id>.png"}}.

Notes on the ControlNet inpaint condition

Following the original handler.py, the condition tensor is the RGB image normalized to [0,1] with all pixels where mask < 128 set to -1 (shape [1, 3, H, W]).

Files

  • inference.py — dual-mode NPU inference (CLI + FastAPI)
  • requirements.txt
  • sample_input/ — demo original.png + mask.png + prompt.txt + input.json
  • saved/ — generated inpaint outputs (git-ignored)
  • assets/ — deployment / verification screenshots