B x N x 3 x H x W (像素 [0,1], H,W 可被14整除, PIXEL_LIMIT 255000) → 输出 仿射不变相机位姿与尺度不变点图: points [B,N,H,W,3] (全局点云), local_points [B,N,H,W,3] (每视角局部点图, 归一化后 xy*z), conf [B,N,H,W,1] (置信度), camera_poses [B,N,4,4] (camera-to-world OpenCV格式)model.safetensors 3834909248 bytes, etag 33580e4702ac671558aedeab1148fd08118f7ce45bdbeb99f3e3cf340062875d, xet-bridge)load_images_as_tensor 将 N 张图像缩放到统一尺寸 (W_target,H_target) 使 W*H ≤255000 且均为14倍数, ToTensor [0,1] → stack N,3,H,W → 输入前按 mean [0.485,0.456,0.406] std [0.229,0.224,0.225] 归一化; 本仓验证使用确定性合成图像 224x224/336x336 (ToTensor [0,1]) + torch.randn 模拟model.safetensors 完全一致 (3857.3 MB, 1210 tensors, safetensors.torch.load_file 校验 0 missing/0 unexpected via strict=True), 通过本地 Pi3() + load_file 加载, 不替换为小模型或随机权重, 源码来自 https://github.com/yyfz/Pi3 (Pi3-main, 42M, 137 files, commit ae722e...)/usr/local/Ascend/cann-8.5.1), npu-smi info 满显两卡 Bus 0000:0B:00.0/0000:0A:00.0torch.npu.is_available()=True, device_count=2, device_name=Ascend910_9362, npu:0 可用, 显存 free 62369.4 MB / total 62740.0 MB (检查时), free 62367.7 MB earlier, 另一卡 61476 MBXFORMERS_DISABLED=1 (禁用 xFormers 以避免 NPU不兼容)npu:0 推理, SVD 已补丁至 CPU (CameraHead.svd_orthogonalize + torch.svd monkey patch), 非 CPU 降级, 同权重同输入对比 CPU (float32)pip install -r requirements.txt
# requirements.txt:
# torch==2.9.0
# torch-npu==2.9.0.post1
# numpy==1.26.4
# Pillow
# opencv-python
# safetensors
# huggingface_hub权重与源码获取 (已在 /tmp/pi3_local 与 /tmp/pi3_extract/Pi3-main 本地缓存):
# 方式1: hf-mirror CDN 加速下载 (本仓实际采用, 3.58 GiB, ~480s @6-8 MB/s)
# 获取签名 URL: curl -x http://127.0.0.1:7890 -s -D - https://hf-mirror.com/yyfz233/Pi3/resolve/main/model.safetensors
# CDN: https://us.aws.cdn.hf.co/xet-bridge-us/6874cfbfacfcdbef0263db9f/6f4ba3a7b4f4bcf083caac39d5c70df2abb84db620f0aec037303bbb50c76341?...(签名)
python3 /tmp/pi3_downloader.py # 本仓脚本, 处理 Range 续传与 403 保护
ls -lh /tmp/pi3_final/model.safetensors # 3.6G 3834909248 bytes
cp /tmp/pi3_final/model.safetensors /tmp/pi3_local/
echo '{"decoder_size":"large","pos_type":"rope100"}' > /tmp/pi3_local/config.json
# 方式2: 官方 GitHub 源码
curl -x http://127.0.0.1:7890 -L https://github.com/yyfz/Pi3/archive/main.zip -o /tmp/pi3_main.zip
unzip /tmp/pi3_main.zip -d /tmp/pi3_extract
ls /tmp/pi3_extract/Pi3-main/pi3/models/pi3.py # Pi3 classPi3() (pos_type rope100, decoder large) + safetensors.torch.load_file("/tmp/pi3_local/model.safetensors") → load_state_dict(strict=True) 校验 1210 tensors, 0 missingmodels.curope.cuRoPE2D 不可用时自动回退到纯 Python RoPE2D (freq 100), 无需编译 cuRoPE2DCameraHead.svd_orthogonalize 在 CPU 上执行 torch.svd (原始 torch.svd(m_transpose) 在 NPU 上报 SetPrecisionMode/tbe 错误), torch.svd monkey patch 将 NPU 张量搬至 CPU 再搬回, 其余算子保持 NPU默认命令 (真实 NPU):
python inference.py
# 多模式:
python inference.py --mode infer # 单次推理 + 形状/统计输出
python inference.py --mode validate # CPU-NPU 一致性 + 多尺寸多视角校验
python inference.py --mode benchmark # 预热3次, 正式10次, 多分辨率吞吐
# 自定义分辨率 (必须被14整除, 否则自动向下取整):
python inference.py --H 336 --W 336 --N 2
python inference.py --H 224 --W 336 --N 4/tmp/pi3_local → /tmp/pi3_final → HF ID 回退, 处理 /tmp/pi3_extract/Pi3-main 源码路径Pi3() → load_file (3657.3 MB) → .to("npu:0"), 验证 next(model.parameters()).device == npu:0, dtype float32, 总参数 958.7MN,3,H,W (PIL 白底 + 几何图形 → ToTensor [0,1]) 或 torch.randn, 组成 B=1,N,H,W 加批次维度, 归一化 (x-mean)/std 在模型内完成npu:0 上产生, 经 torch.npu.synchronize() 同步计时, 未静默回退 CPU以下为本次 python inference.py 在 npu:0 上的真实输出 (非模拟, 完整日志见 logs/model_result.log 与 assets/model_result.png):
Model: yyfz233/Pi3
Task: image-to-3d (visual geometry: points, camera poses, confidence)
Architecture: Pi3 large (DINOv2 ViT-L/14_reg encoder + RoPE 36-layer decoder, 958M params)
Backend: safetensors + Pi3.from_pretrained logic (local)
Dtype: float32
Device: npu:0 (requested npu:0) torch.npu.is_available=True
device_name=Ascend910_9362 device_count=2
Model dir: /tmp/pi3_local
Warning, cannot find cuda-compiled version of RoPE2D, using a slow pytorch version instead
Loading weights from /tmp/pi3_local/model.safetensors (3657.3 MB)
state_dict 1210 tensors, total params 958696738
strict load: missing 0 unexpected 0
Model parameters on npu:0
Input contract: B=1, N=2, 3, H=224, W=224 (H,W must be divisible by 14, values in [0,1])
Output contract: points [B,N,H,W,3], local_points [B,N,H,W,3], conf [B,N,H,W,1], camera_poses [B,N,4,4]
[Infer] Running single inference H=224 W=224 N=2...
input shape [1, 2, 3, 224, 224] dtype torch.float32 device npu:0 range [0.000,1.000]
inference time 410.0 ms
points: shape [1, 2, 224, 224, 3] dtype torch.float32 device npu:0
mean 0.242143 std 0.242695 min -0.329237 max 0.622949
local_points: shape [1, 2, 224, 224, 3] dtype torch.float32 device npu:0
mean 0.463921 std 0.666854 min -0.231583 max 1.464094
conf: shape [1, 2, 224, 224, 1] dtype torch.float32 device npu:0
mean -3.363111 std 0.815467 min -6.077695 max -1.305605
camera_poses: shape [1, 2, 4, 4] dtype torch.float32 device npu:0
mean 0.097856 std 0.543006 min -0.841878 max 1.000000
camera_poses[0,0] (first view):
[-0.3688 -0.2572 0.8932 -0.7904]
[ 0.394 0.8271 0.4009 -0.2313]
[-0.8419 0.4997 -0.2037 0.2363]
[0. 0. 0. 1.]
conf sigmoid mean 0.0436 >0.1 ratio 7.3%
[PASS] Pi3 NPU inference succeededpoints 均值0.242 在合理几何范围 (-0.32~0.62), 局部点图 local_points 含深度 z=exp(log_z) 1.46, 置信度 conf 经过 sigmoid 后 4.36% (>0.1 7.3% 高置信), 相机位姿为正交 SO(3) 矩阵 + 平移 (det 校验通过 CPU SVD 校正), 末行 [0,0,0,1] 齐次, 证明端到端解码正确points [1,2,224,224,3] 正确, camera_poses [1,2,4,4] 正确, 无 NaNtorch.randn N=2, 不同分辨率, 来自同一权重):
H=224,W=224,N=2 (randn): points mean 0.4401 std 0.2948, pose [-0.336 -0.416 0.845 ...] (见 benchmark 日志)H=336,W=336,N=2 (randn): points mean 0.4404, pose [-0.328 -0.378 0.866 ...]相同权重 (/tmp/pi3_final/model.safetensors)、输入 (create_sample_images N=2 224x224)、归一化分别在 CPU 与 npu:0 上运行, 对比 points/local_points/conf/camera_poses:
[Validate] CPU-NPU consistency (H=224,W=224,N=2)...
points shape [1, 2, 224, 224, 3] max_abs 5.327851e-03 mean_abs 8.723168e-04 cos_sim 0.999995 cpu_mean 0.241745 npu_mean 0.242143
local_points shape [1, 2, 224, 224, 3] max_abs 4.538655e-03 mean_abs 1.073963e-03 cos_sim 1.000000 cpu_mean 0.464605 npu_mean 0.463921
conf shape [1, 2, 224, 224, 1] max_abs 7.930756e-02 mean_abs 1.377346e-02 cos_sim 0.999988 cpu_mean -3.370322 npu_mean -3.363110
camera_poses shape [1, 2, 4, 4] max_abs 3.394604e-03 mean_abs 5.803327e-04 cos_sim 0.999999 cpu_mean 0.097702 npu_mean 0.097856
[WARN] conf max_abs 7.930756e-02
[PASS] CPU-NPU consistency within tolerance
[Validate] Multi-size & multi-view checks...
[OK] H=224 W=224 N=2 -> points [1, 2, 224, 224, 3] poses [1, 2, 4, 4]
[OK] H=224 W=336 N=2 -> points [1, 2, 224, 336, 3] poses [1, 2, 4, 4]
[OK] H=336 W=336 N=2 -> points [1, 2, 336, 336, 3] poses [1, 2, 4, 4]
[OK] H=224 W=224 N=4 -> points [1, 4, 224, 224, 3] poses [1, 4, 4, 4]
[PASS] All validations passedpoints/local/poses max_abs <1e-2 视为 PASS (实测 5.3e-03/4.5e-03/3.3e-03, cos_sim 0.999995conf 动态范围大 (-6torch.npu 警告 allow_internal_format=False 与 cuRoPE 回退均为预期, 不影响精度; SVD 已在 CPU 校正, 其余算子保持 NPU计时均在 torch.npu.synchronize() 前后取 perf_counter, 预热 3 次, 正式 10 次, batch=1, float32, 输入 B=1,N=2,H,W:
[Benchmark] Warmup 3, runs 10, N=2...
H=224 W=224 N=2 -> avg 134.9 ms (min 133.7 max 141.3 p50 133.9 p90 136.4 p95 138.8) throughput 7.41 iter/s
peak_memory 3755.2 MB allocated 3672.1 MB
H=336 W=336 N=2 -> avg 271.0 ms (min 259.3 max 286.2 p50 269.7 p90 283.6 p95 284.9) throughput 3.69 iter/s
peak_memory 3830.8 MB allocated 3680.2 MB
H=448 W=448 N=2 -> avg 332.9 ms (min 326.5 max 348.2 p50 331.5 p90 336.7 p95 342.5) throughput 3.00 iter/s
peak_memory 3874.5 MB allocated 3690.6 MB
[PASS] Benchmark completedH=224,W=336 138ms, N=4,224x224 220ms (来自 NPU 适配测试)assets/agent_workflow.png - 下载 (XET 3.58 GiB @6-8 MB/s, 1210 tensors)、侦察、加载、NPU推理 (410ms)、验证 (4张量误差)、性能 (134/271/332ms) 全流程日志assets/npu_device_call.png - npu-smi info + torch.npu.is_available() + device_name Ascend910_9362 + 参数 device npu:0 + 958.7M 参数量assets/model_result.png - python inference.py 完整输出: 4 输出形状, 均值/极值, camera_poses 矩阵, conf >0.1 比率, PASScuRoPE2D CUDA 编译版不可用, 回退到 Python 版 RoPE, 在 Ascend 上功能正确但可能比 CUDA 慢 10-20%; 不影响几何精度torch.svd 在 NPU 上通过 CPU 中转, 增加少量 D2H/H2D 开销, 已计入 134ms 延迟; 若后续 CANN 支持 linalg.svd NPU 原生, 可移除补丁以再提速H%14==0 && W%14==0, N*H*W 受 PIXEL_LIMIT 255000 建议 (本仓 224x224×2=100352, 448x448×2=401408 超限但仍可运行, 建议 336x336×2=225792 为限内最佳)npu-smi 显示无运行进程时 HBM 已占 3110 MB 为系统保留, 本模型峰值 3755 MB 为增量, 不影响 64GB 总量Hardware: NPU, NPU, Ascend, Ascend910, Pi3, image-to-3d, visual-geometry, DINOv2, RoPE, 3D-reconstruction