opus-mt_tiny_spa-eng Ascend NPU 部署指南

opus-mt_tiny_spa-eng-ascend/
├── inference.py          # 推理测试脚本
├── log.txt               # 测试日志
├── README.md             # 本文档
├── test_sample.txt       # 测试样例
├── inference_result.json # 推理结果
└── precision_result.json # 精度测试结果

docker exec -it test-modelagent bash

source /usr/local/Ascend/ascend-toolkit/set_env.sh

pip install transformers torch_npu -i https://huaweimirror.com.cn/simple

cd /data/ysws/agentsp/5-17/opus-mt_tiny_spa-eng-ascend/

python3 inference.py

cd /data/ysws/agentsp/5-17/opus-mt_tiny_spa-eng-ascend/

python3 inference.py all

============================================================
OPUS-MT-TINY-SPA-ENG NPU Test
Model: Helsinki-NLP/opus-mt_tiny_spa-eng
Output: /data/ysws/agentsp/5-17/opus-mt_tiny_spa-eng-ascend
============================================================

============================================================
OPUS-MT-TINY-SPA-ENG Inference Test (NPU)
============================================================
Device: npu:0
Model: /data/ysws/agentsp/5-17/opus-mt_tiny_spa-eng/Helsinki-NLP/opus-mt_tiny_spa-eng

Loading tokenizer...
Loading model...
Loading weights: 100%|██████████| 151/151 [00:00<00:00, 5540.11it/s]

Input text: ['Hola, como estas hoy?']
Input shape: torch.Size([1, 7])
Generated text: ['Hello, like this today?']
Inference time: 0.981s

============================================================
Precision Test (CPU vs NPU)
============================================================

Using device: npu:0
Loading tokenizer...
Loading model on CPU...
Loading weights: 100%|██████████| 151/151 [00:00<00:00, 4582.91it/s]
Running inference on CPU...

Loading model on npu:0...
Loading weights: 100%|██████████| 151/151 [00:00<00:00, 4911.93it/s]
Running inference on NPU...

CPU inference time: 0.108s
NPU inference time: 0.053s
Speedup: 2.05x
CPU output: ['Hello, like this today?']
NPU output: ['Hello, like this today?']
Output texts match: True
Status: PASS

import torch
from transformers import MarianMTModel, MarianTokenizer

MODEL_DIR = "/data/ysws/agentsp/5-17/opus-mt_tiny_spa-eng/Helsinki-NLP/opus-mt_tiny_spa-eng"

tokenizer = MarianTokenizer.from_pretrained(MODEL_DIR)
model = MarianMTModel.from_pretrained(MODEL_DIR)
model = model.to("npu:0")
model.eval()

src_texts = ["Hola, como estas hoy?"]
inputs = tokenizer(src_texts, return_tensors="pt", padding=True)
inputs = {k: v.to("npu:0") for k, v in inputs.items()}

with torch.no_grad():
    outputs = model.generate(**inputs)

translations = tokenizer.batch_decode(outputs, skip_special_tokens=True)
print(translations)  # ['Hello, like this today?']

src_texts = [
    "Hola, como estas hoy?",
    "Estoy muy contento de verte.",
    "La traduccion automatica es muy util."
]

inputs = tokenizer(src_texts, return_tensors="pt", padding=True)
inputs = {k: v.to("npu:0") for k, v in inputs.items()}

with torch.no_grad():
    outputs = model.generate(**inputs)

translations = tokenizer.batch_decode(outputs, skip_special_tokens=True)
for src, tgt in zip(src_texts, translations):
    print(f"{src} -> {tgt}")

{
  "hidden_size": 512,
  "encoder_layers": 6,
  "decoder_layers": 6,
  "encoder_attention_heads": 8,
  "decoder_attention_heads": 8,
  "d_model": 512,
  "vocab_size": 58101
}