m
mengsihan/briaai-RMBG-2.0-NPU
Model Introduction
Files & versions
Pull Requests
Discussion
Analysis

briaai/RMBG-2.0 on Ascend NPU

1. 模型简介

briaai/RMBG-2.0 是基于 BiRefNet 的背景移除模型(BRIA AI),用于图像分割中的前景/背景二分类。模型架构为 BiRefNet,backbone 为 swin_v1_l,输入 1024x1024,输出 1024x1024 单通道 mask logits,经 sigmoid 得到前景概率。官方权重约 844MB (safetensors),总参数 220M,适用于人物、商品、通用前景分割后背景替换。

  • 模型名称: briaai/RMBG-2.0
  • 任务: image-segmentation (background removal)
  • 架构: BiRefNet (Swin v1 large + Decoder + ASPPDeformable)
  • 权重格式: safetensors + pytorch_model.bin (1.7GB together, use safetensors)
  • 本地路径: /tmp/modelscope_cache/briaai/RMBG-2___0 (ModelScope snapshot_download)
  • 加载后端: transformers AutoModelForImageSegmentation (trust_remote_code=True) + AutoImageProcessor (ViTFeatureExtractor, size 1024)

2. 验证环境

  • NPU: Ascend910_9362 (Ascend910B), 64GB HBM, 2 devices, tested on npu:0
  • CANN: 8.5.1, torch 2.9.0+cpu, torch_npu 2.9.0.post1+gitee7ba04
  • npu-smi: 25.5.5, health OK, temp 47C
  • CPU: aarch64, memory free 65GB
  • Python: 3.11, transformers 4.57.6, timm, kornia 0.8.3, PIL, numpy
npu-smi info
+-----------+-----------------+------+-------+------+--+
| NPU 5 Ascend910 OK 170W 47C 0/0 3110/65536 MB |
| No running processes after test |
torch.npu.is_available()=True, device_name=Ascend910_9362
torch_npu_version=2.9.0.post1+gitee7ba04

3. 安装依赖

pip install -r requirements.txt
# requirements.txt:
# torch==2.9.0
# torch_npu==2.9.0.post1
# transformers==4.57.6
# timm==1.0.28
# Pillow>=10.0
# numpy
# kornia==0.8.3
# kornia_rs==0.1.14
# accelerate

权重下载(已本地缓存,无需运行时下载):

from modelscope import snapshot_download
model_dir = snapshot_download('briaai/RMBG-2.0', cache_dir='/tmp/modelscope_cache')
# or use local dir /tmp/modelscope_cache/briaai/RMBG-2___0 directly

4. NPU 推理

默认即 NPU:

python inference.py
# NPU benchmark
python inference.py --mode benchmark
# CPU-NPU consistency
python inference.py --mode validate
# CPU fallback
python inference.py --device cpu

推理流程:

  1. 加载 processor (ViTFeatureExtractor, do_resize 1024, normalize mean 0.485/0.456/0.406)
  2. model.npu() 将 BiRefNet 移至 npu:0
  3. 输入 PIL Image -> pixel_values [1,3,1024,1024] -> .npu()
  4. torch.npu.synchronize() 前后计时,输出 list[4] logits,取最后一层 [1,1,1024,1024]
  5. torch.sigmoid 得概率,阈值 0.5 得二值 mask,可用于 alpha matting 背景替换

注意: torchvision::deform_conv2d 在 NPU 上 fallback 到 CPU(CANN 8.5.1 不支持),不影响正确性,性能约 12s/张。

5. 真实推理结果

合成测试图: 1024x1024 白底红圆(中心半径 256)

默认 NPU 推理 (python inference.py):

Model: briaai/RMBG-2.0
Task: image-segmentation (background removal) - BiRefNet
Model device: npu:0, Total params: 220.18M
Input pixel_values shape: [1,3,1024,1024]
Output mask logits shape: (1, 1, 1024, 1024), device: npu:0
Logits stats: min=-29.7615, max=836.1365, mean=15.4075
Prob stats: min=0.000000, max=1.000000, mean=0.197093
Center prob (512,512): 1.000000
Corner prob (0,0): 0.000000
Foreground pixels: 206637/1048576 (19.71%)
Inference latency: 12889.60 ms (synchronized, single sample)
PASS: Background removal produces correct mask (center foreground, corner background)

掩码正确分离: 圆心概率 1.0,角落 0.0,前景点占比 19.71% 符合圆形面积 (~19.6% with square bounds),验证分割任务成功。

6. CPU-NPU 一致性验证

python inference.py --mode validate
Running CPU inference...
CPU logits mean 15.4196 prob mean 0.197094
Running NPU inference...
NPU logits mean 15.4075 prob mean 0.197093
CPU prob mean: 0.197094, NPU prob mean: 0.197093
Max abs diff (prob): 0.000600
Mean abs diff (prob): 0.000000
Cosine similarity (prob): 0.999992
Max abs diff (logits): 4.272842
Mean abs diff (logits): 0.027871
PASS: CPU-NPU consistency validated (max_diff < 0.05, cos > 0.999)

概率层面几乎一致,logits 均值差异 <0.03,最大差异来自极大值区域但经 sigmoid 后压缩到 0.0006。

7. 性能测试

python inference.py --mode benchmark (warmup 3, test 5, batch1, 1024x1024, npu:0, fp32):

Iter 1: 12162.53 ms
Iter 2: 12279.52 ms
Iter 3: 12272.12 ms
Iter 4: 11340.85 ms
Iter 5: 11883.75 ms
  avg=11987.75 ms, min=11340.85 ms, max=12279.52 ms, p50=12162.53 ms, p90=12279.52 ms, p95=12279.52 ms
  NPU memory: total=61.27GB, free=50.35GB, used=10.92GB
  throughput ~0.083 images/s (单张约 12s,受 deform_conv CPU fallback 影响)

同步计时 torch.npu.synchronize() 包裹。单样本 12s,峰值显存约 6-11GB,64GB 卡可 batch=1 安全运行。

8. 自验证截图

三张图由 scripts/render_xterm_evidence.mjs --style raw 从真实日志渲染:

agent workflow npu device call model result

生成命令:

npm --prefix /opt/atomgit/.claude/skills/npu-adapt-skills/scripts install
node /opt/atomgit/.claude/skills/npu-adapt-skills/scripts/render_xterm_evidence.mjs --manifest evidence.json --output-dir briaai-RMBG-2.0-NPU/assets --style raw

9. 已知限制

  • torchvision::deform_conv2d 在当前 CANN/torch_npu 上 fallback CPU,导致端到端约 12s,若后续 CANN 支持可提速。
  • 输入固定 1024x1024(processor resize),过大图像需预先缩放,越小物体细节可能损失。
  • 仅验证合成图与随机图,未做完整 DIS5K 数据集评测;单样本一致性 ≠ 数据集精度。
  • FP32 推理,FP16 未测试;显存充裕可尝试 BF16 提速但需重新验证。
  • 仅单卡 npu:0,多卡需自行改 device。

10. 标签

#NPU #Ascend #Ascend910 #image-segmentation #background-removal #BiRefNet

hardware: NPU topics: [Hardware: NPU, NPU, Ascend, Ascend910]