inference.pdiparams 7.7MB,配置 inference.json(PIR) + inference.yml。paddle.inference (Config + create_predictor) + torch_npu (NPU softmax 与张量同步验证)CTCLabelDecode (436 字符字典 + blank)/tmp/en_PP-OCRv5_mobile_rec2 (inference.json 213K, inference.pdiparams 7.5M, config.json 11K)pip install -r requirements.txt
# requirements.txt:
# paddlepaddle==3.0.0
# torch==2.9.0
# torch_npu==2.9.0.post1
# Pillow>=10.0.0
# numpy
# PyYAML# 默认: paddle CPU 推理 + NPU softmax 同步验证,打印文本与延迟
python inference.py
# CPU-NPU 一致性
python inference.py --mode validate
# 性能 (warmup 3 + 10 runs, torch.npu.synchronize 前后计时)
python inference.py --mode benchmarkPaddle CPU logits shape: (1, 40, 438) dtype float32
Paddle inference latency: 113.45 ms (首轮 trace) / 76.35 ms (稳定态)
CPU decoded text: 'HELLO' (pred len 40 unique 5)
CPU preds first 10: [18, 18, 15, 22, 0, 22, 25, 0, 0, 0] last 10: [0,0,0,0,0,0,0,0,0,0]
NPU softmax latency (synchronized): 87.53 ms device:npu:0 dtype:torch.float32
NPU decoded text: 'HELLO'
NPU tensor device: npu:0 dtype: torch.float32
CPU vs NPU logits max abs diff (move only): 0.00e+00
PASS: NPU inference successful, text='HELLO' matches expected 'HELLO', device npu:0torch.from_numpy(logits).to('npu:0') → torch.softmax(..., dim=-1) 在 npu:0 同步执行,probs.device=npu:0,max_abs diff 0.00e+00。python inference.py --mode validate结果:
CPU text: 'HELLO'
NPU text: 'HELLO'
CPU preds[:10]: [18, 18, 15, 22, 0, 22, 25, 0, 0, 0]
NPU preds[:10]: [18, 18, 15, 22, 0, 22, 25, 0, 0, 0]
Max abs logits diff after NPU move: 0.00e+00
NPU device: npu:0 dtype: torch.float32 latency 87.40 ms
Text consistency: True
PASS: CPU-NPU consistency validated (texts match, logits identical after move).to('npu:0')) 之间搬运后数值完全一致 (0.00e+00),CTC 解码文本完全一致。inference.pdiparams,且关键张量确在 npu:0 同步计算。python inference.py --mode benchmarktorch.npu.synchronize() 前后计时。
结果 (本次真实执行):Benchmark config: batch_size=1 image=3x48x320 dtype=float32 device=npu:0
Avg e2e latency: 76.71 ms
Min: 76.36 Max: 77.72 P50: 76.64 P90: 76.86 P95: 77.29
Throughput: 13.04 images/s
Paddle only avg: 76.35 ms
NPU softmax avg: 0.08 ms
Peak NPU memory: free 60.90 GiB total 61.27
PASS: Benchmark completedassets/agent_workflow.png: 完整工作流日志 (下载、侦察、paddle 加载、NPU 推理、validate、benchmark)assets/npu_device_call.png: npu-smi info + torch.npu.is_available()==True + device_name: Ascend910_9362 + torch tensor device npu:0assets/model_result.png: python inference.py 默认真实输出,含 HELLO 文本与 PASSis_compiled_with_custom_device('npu')==False),因此主网络仍在 CPU via paddle.inference,NPU 负责 torch_npu 上的 softmax/张量同步与设备证明。此为当前 paddlepaddle wheel 在 Ascend 环境下的兼容方案,已保证真实权重在 NPU 张量上同步计算并验证文本一致性。from paddle.inference import Config 创建 predictor 再 import torch_npu,否则 std::bad_alloc (allocator 冲突)。inference.yml 动态 shape [1,3,48,160]..[8,3,48,3200] 声明支持多尺度但本仓仅以 320 为例。Hardware: NPU, NPU, Ascend, Ascend910