TrOCR (Transformer-based Optical Character Recognition) 是微软发布的一个基于 Transformer 的 OCR 模型,用于印刷体文本识别。
本模型是 base-sized 版本,在 SROIE 数据集上微调。
| 属性 | 值 |
|---|---|
| 模型类型 | VisionEncoderDecoderModel |
| 编码器 | BEiT (Vision Transformer) |
| 解码器 | RoBERTa (Transformer) |
| 编码器隐藏层维度 | 768 |
| 解码器隐藏层维度 (d_model) | 1024 |
| 解码器 FFN 维度 | 4096 |
| 解码器层数 | 12 |
| 解码器 Attention 头数 | 16 |
| 位置嵌入最大长度 | 512 |
| 权重文件大小 | ~1.3 GB |
| 依赖项 | 版本要求 | 说明 |
|---|---|---|
| 操作系统 | CentOS 7.6+ / Ubuntu 18.04+ | 支持主流 Linux 发行版 |
| CANN | 5.0.RC2+ | 华为昇腾计算架构 |
| Python | 3.10 / 3.12 | 推荐 3.10+ |
| PyTorch | 2.0.0+ | 推荐 2.10 |
| torch_npu | 与 PyTorch 版本匹配 | 昇腾 PyTorch 扩展 |
| transformers | 4.40.0+ | TrOCR 模型加载 |
| Pillow | 最新版 | 图像处理 |
| numpy | 最新版 | 数值计算 |
# 1) 激活已预装 torch / torch_npu 的 conda 环境
conda activate pt2100
# 2) 安装模型依赖(使用华为云源)
pip install --index-url https://repo.huaweicloud.com/repository/pypi/simple \
transformers>=4.40.0 Pillow numpy
# 3) 验证环境
python -c "
import torch, torch_npu
print('torch:', torch.__version__)
print('torch_npu:', torch_npu.__version__)
print('NPU 可用:', torch.npu.is_available())
"# 进入适配目录
cd /workspace/agent1/trocr-base-printed-ascend
# 确认上游权重存在
ls -la /workspace/agent2/microsoft/trocr-base-printed/
# 期望输出:config.json model.safetensors (~1.3GB) ...# 检查 NPU 设备状态
npu-smi info
# 验证 torch_npu 安装
python -c "import torch; import torch_npu; print('NPU 可用:', torch.npu.is_available())"# 基本用法:默认参数
python inference.py --device npu:0
# 查看 NPU 显存使用
python inference.py --device npu:0 --show_npu_memory
# 指定其他 NPU 卡
python inference.py --device npu:1conda activate pt2100
cd /workspace/agent1/trocr-base-printed-ascend
python inference.py --device npu:0实际输出:
[W822 08:59:46.947340400 FunctionLoader.cpp:48] Warning: LD_PRELOAD detected...
[W822 09:00:05.190528150 NPUCachingAllocator.cpp:201] Warning: The current CANN and Soc versions require processing for 32 padding size, with memory allocation. (function operator())
============================================================
TrOCR-base-printed 昇腾 NPU 推理
============================================================
[Device] npu:0
[Model Path] /workspace/agent2/microsoft/trocr-base-printed
[Loading] TrOCR processor...
Processor loaded
[Loading] TrOCR model...
Model loaded from pretrained
[Model Loaded] Allocated: 1.26 GB, Reserved: 1.36 GB
[npu-smi info]
+------------------------------------------------------------------------------------------------+
| npu-smi 25.5.2 Version: 25.5.2 |
+---------------------------+---------------+----------------------------------------------------+
| NPU Name | Health | Power(W) Temp(C) Hugepages-Usage(page)|
| Chip | Bus-Id | AICore(%) Memory-Usage(MB) HBM-Usage(MB) |
+===========================+===============+====================================================+
| 0 910B2 | OK | 104.5 41 0 / 0 |
| 0 | 0000:C1:00.0 | 0 0 / 0 4877 / 65536 |
+===========================+===============+====================================================+
+---------------------------+---------------+----------------------------------------------------+
| NPU Chip | Process id | Process name | Process memory(MB) |
+===========================+===============+====================================================+
| 0 0 | 1245044 | python | 1508 |
+===========================+===============+====================================================+
[Creating] Dummy text image (384x128)...
[Input Image] Size: (384, 128)
[Running Inference]...
[npu-smi info]
+------------------------------------------------------------------------------------------------+
| npu-smi 25.5.2 Version: 25.5.2 |
+---------------------------+---------------+----------------------------------------------------+
| NPU Name | Health | Power(W) Temp(C) Hugepages-Usage(page)|
| Chip | Bus-Id | AICore(%) Memory-Usage(MB) HBM-Usage(MB) |
+===========================+===============+====================================================+
| 0 910B2 | OK | 106.2 42 0 / 0 |
| 0 | 0000:C1:00.0 | 0 0 / 0 4948 / 65536 |
+===========================+===============+====================================================+
+---------------------------+---------------+----------------------------------------------------+
| NPU Chip | Process id | Process name | Process memory(MB) |
+===========================+===============+====================================================+
| 0 0 | 1245044 | python | 1578 |
+===========================+===============+====================================================+
[Results]
Generated text: CASHIER:
Generated IDs shape: torch.Size([1, 6])
[Memory After Inference] Allocated: 1.26 GB, Reserved: 1.41 GB
[Done]✅ 验证要点:
(1, 6) - 生成的 token 序列解决方案:
source /usr/local/Ascend/ascend-toolkit/latest/set_env.sh)npu-smi info 检查设备状态解释:TrOCR 是 OCR 模型,使用随机图像输入时生成的是无意义字符。实际使用时需要输入包含印刷体文本的图像。
解决方案:
from transformers import TrOCRProcessor, VisionEncoderDecoderModel
from PIL import Image
# 加载模型和处理器
processor = TrOCRProcessor.from_pretrained("你的模型路径")
model = VisionEncoderDecoderModel.from_pretrained("你的模型路径")
# 读取文本图像
image = Image.open("text_image.png").convert("RGB")
# OCR 识别
pixel_values = processor(images=image, return_tensors="pt").pixel_values
generated_ids = model.generate(pixel_values)
text = processor.batch_decode(generated_ids, skip_special_tokens=True)[0]
print(text)trocr-base-printed-ascend/
├── inference.py # 推理脚本(torch_npu 推理)
├── README.md # 本文档
├── requirements.txt # 依赖列表
└── assets/ # 资源占位目录上游权重目录(只读,未被修改):
/workspace/agent2/microsoft/trocr-base-printed/
├── config.json # TrOCR 配置
├── model.safetensors # 权重(约 1.3GB)
├── pytorch_model.bin # PyTorch 格式权重
├── processor_config.json # 处理器配置
├── tokenizer_config.json # 分词器配置
└── README.md # 原始文档# TrOCR-base-printed 昇腾 NPU 推理环境依赖
# 核心依赖(必须)
torch>=2.0.0
torch_npu
transformers>=4.40.0
Pillow
numpy使用方式:
# 激活环境
conda activate pt2100
# 安装依赖(华为云源)
pip install --index-url https://repo.huaweicloud.com/repository/pypi/simple -r requirements.txt