OPUS-MT-LN-EN Ascend NPU 部署指南

opus-mt-ln-en-ascend/
├── inference.py          # 推理测试脚本
├── log.txt               # 测试日志
├── README.md             # 本文档
├── test_sample.txt       # 测试样本
├── inference_result.json # 推理结果
└── precision_result.json # 精度测试结果

docker exec -it test-modelagent bash

source /usr/local/Ascend/ascend-toolkit/set_env.sh

pip install transformers torch_npu

cd /data/ysws/agentsp/5-18-2/opus-mt-ln-en-ascend/
python3 inference.py

cd /data/ysws/agentsp/5-18-2/opus-mt-ln-en-ascend/
python3 inference.py precision_test

============================================================
OPUS-MT-LN-EN NPU Test
Output: /data/ysws/agentsp/5-18-2/opus-mt-ln-en-ascend
============================================================

============================================================
OPUS-MT-LN-EN Inference Test (NPU)
============================================================
Device: npu:0
Model: /data/ysws/agentsp/5-18-2/opus-mt-ln-en/Helsinki-NLP/opus-mt-ln-en
Loading tokenizer...
Loading model...
Loading weights: 100%|██████████| 258/258 [00:00<00:00, 12082.74it/s]
[transformers] Both `max_new_tokens` (=50) and `max_length`(=512) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)

Input text: ['Mbote, okosalaki yango?']
Input shape: torch.Size([1, 7])
Generated text: ['Will you accept it, or would you do it?']
Inference time: 1.103s

============================================================
Creating Test Sample
============================================================
Saved test sample
  1. Mbote, okosalaki yango?
  2. Nakokaki kotelemwa yo.
  3. Mabe ya bino ezalaka mingi.

============================================================
Precision Test (CPU vs NPU)
============================================================
Loading model on CPU...
Loading weights: 100%|██████████| 258/258 [00:00<00:00, 12771.51it/s]
[transformers] Both `max_new_tokens` (=50) and `max_length`(=512) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
CPU inference time: 2.486s
CPU output: ['Will you accept it, or would you do it?']
Loading model on npu:0...
Loading weights: 100%|██████████| 258/258 [00:00<00:00, 12689.44it/s]
[transformers] Both `max_new_tokens` (=50) and `max_length`(=512) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)
NPU inference time: 0.188s
NPU output: ['Will you accept it, or would you do it?']

CPU inference time: 2.486s
NPU inference time: 0.188s
Speedup: 13.20x
CPU output: ['Will you accept it, or would you do it?']
NPU output: ['Will you accept it, or would you do it?']
Output texts match: True
Status: PASS

============================================================
Test Complete!
============================================================

import torch
from transformers import MarianTokenizer, MarianMTModel

MODEL_DIR = "/data/ysws/agentsp/5-18-2/opus-mt-ln-en/Helsinki-NLP/opus-mt-ln-en"

tokenizer = MarianTokenizer.from_pretrained(MODEL_DIR)
model = MarianMTModel.from_pretrained(MODEL_DIR)
model = model.to("npu:0")
model.eval()

src_texts = ["Mbote, okosalaki yango?"]
inputs = tokenizer(src_texts, return_tensors="pt", padding=True)
inputs = {k: v.to("npu:0") for k, v in inputs.items()}

with torch.no_grad():
    outputs = model.generate(inputs['input_ids'], max_new_tokens=50)

translations = tokenizer.batch_decode(outputs, skip_special_tokens=True)
print(translations)  # ["Will you accept it, or would you do it?"]