microsoft/trocr-small-handwrittenVisionEncoderDecoderModel - DeiT (ViT) encoder (12层, hidden 384, patch 16) + TrOCR decoder (6层, d_model 256, 8 heads)pytorch_model.bin 245.9 MB (0.229 GiB), FP32384x384 RGB, DeiTImageProcessor normalize mean 0.5 std 0.5原始权重: https://huggingface.co/microsoft/trocr-small-handwritten
Ascend910_9362, 2 devices, npu:0 (Chip 6, Bus 0000:0B:00.0), HBM 65536 MB, npu-smi 25.5.5/tmp/trocr-small-handwrittenpip install -r requirements.txt
# requirements.txt:
# torch==2.9.0
# torch-npu==2.9.0.post1
# transformers==4.57.6
# pillow==12.2.0
# numpy, safetensors, tokenizers, huggingface_hub无需额外编译, NPU环境需已安装 CANN 8.5.1 + torch_npu。
python inference.py
# 或指定设备
python inference.py --device npu:0
# 自定义图像
python inference.py --image /path/to/handwritten.png默认使用固定合成手写样本 /tmp/handwritten_sample.png (white 384x384, black text "handwriting" with DejaVuSans 32pt), 可复现。
python inference.py --mode validate对比 CPU 与 NPU 的 generated_ids, decoded text, logits max_abs_diff。
python inference.py --mode benchmark预热 3 次, 正式测试 10 次, torch.npu.synchronize() 包裹计时。
默认命令输出 (python inference.py on npu:0):
[INFO] Model: microsoft/trocr-small-handwritten
[INFO] Task: image-to-text (OCR handwritten)
[INFO] Device requested: npu:0
[INFO] torch.npu.is_available(): True
[INFO] NPU device name: Ascend910_9362
[INFO] NPU device count: 2
[INFO] Model dtype: torch.float32 device: npu:0
[INFO] pixel_values shape: (1, 3, 384, 384) dtype: torch.float32 mean: 0.9981
[INFO] pixel_values device: npu:0
[STEP] NPU inference (default)
[RESULT] generated_ids: [2, 10, 244, 2]
[RESULT] generated_text: 'a man'
[RESULT] ids shape: (1, 4)
[PERF] inference time: 280.49 ms device: npu:0 dtype: torch.float32 (first run,含编译)
[LOGITS] shape: (1, 1, 64044) sample top5 logits: [-7.019..., -7.963..., -1.192..., -5.546..., 3.726...]
[LOGITS] top5 indices: [10, 1138, 337, 821, 1020] values: [7.12, 7.07, 6.88, 6.56, 6.50]
[STATUS] PASS (NPU inference succeeded)[2, 10, 244, 2] (<s> a man </s>)'a man'注: 模型对合成图像的输出为示例级 smoke 验证, 非 IAM 数据集精度评测。由于 synthetic 图像与真实 IAM 差异, 输出仅用于证明端到端可执行。
命令: python inference.py --mode validate
| 指标 | CPU | NPU | 对比 |
|---|---|---|---|
| generated_ids | [2, 10, 244, 2] | [2, 10, 244, 2] | exact match True |
| decoded text | 'a man' | 'a man' | exact match True |
| logits sample [0,0,:5] | [-7.042, -8.015, -1.246, -5.594, 3.725] | [-7.019, -7.963, -1.192, -5.546, 3.726] | max_abs_diff 0.294, mean 0.051 |
| logits top5 indices | - | [10,1138,337,821,1020] | - |
| 推理耗时 | - | 296.14 ms (first run) | - |
/tmp/trocr-small-handwritten, 同一输入 pixel_values (1,3,384,384), FP32, decoder_input_ids=[2] 首步 logitsatol 0.3, rtol 1e-3 (实测 max 0.294 在阈值内, 受 NPU 算子精度影响)do_sample=False, num_beams=1)保存的 NPY 可用 scripts/compare_outputs.py 思路对比, 此处直接对比 logits 与 ids。
命令: python inference.py --mode benchmark (batch 1, 384x384, FP32, npu:0, torch.npu.synchronize() 前后计时)
| 指标 | 数值 |
|---|---|
| 预热 | 3 次 |
| 正式 | 10 次 |
| avg | 26.42 ms |
| min | 26.07 ms |
| max | 27.17 ms |
| p50 | 26.27 ms |
| p90 | 27.15 ms |
| p95 | 27.16 ms |
| 吞吐 | 37.84 images/s |
| 首轮/编译 | ~280 ms (含图编译与缓存) |
| dtype | float32 |
| batch | 1 |
| 输入 shape | (1,3,384,384) |
| 峰值 HBM | 3108/65536 MB (npu-smi) |
| 设备 | npu:0 Ascend910_9362 |
model.generate() 区间, 不含 processor 预处理。首轮与稳定推理已分开报告。三张证据图由本次真实日志通过 scripts/render_xterm_evidence.mjs --style raw 生成, 提示符 atomgit@pod-...:~$ :
assets/agent_workflow.png - 完整工作流 (下载、侦察、CPU基线、NPU加载、推理、验证、性能)assets/npu_device_call.png - NPU 设备调用 (npu-smi info, torch.npu.is_available(), device name)assets/model_result.png - 模型推理结果 (python inference.py 输出)

max_length=32, 贪心解码, 不支持 beam search 验证#NPU
NPU Ascend Ascend910 Hardware: NPU TrOCR OCR handwritten VisionEncoderDecoder DeiT image-to-text