text_summarization Ascend NPU 部署指南

text_summarization-ascend/
├── inference.py          # 推理测试脚本
├── log.txt               # 测试日志
├── README.md             # 本文档
└── test_texts.txt       # 测试文本

source /usr/local/Ascend/ascend-toolkit/set_env.sh

pip install transformers torch_npu

cd /data/ysws/agentsp/5-15/text_summarization-ascend/

# 使用默认测试文本
python3 inference.py

# 使用指定设备
python3 inference.py --device npu:0

cd /data/ysws/agentsp/5-15/text_summarization-ascend/

# 运行完整精度测试
python3 inference.py --mode precision_test

summarize: Hugging Face: Revolutionizing Natural Language Processing. In the rapidly evolving field of Natural Language Processing (NLP), Hugging Face has emerged as a prominent and innovative force...

In the rapidly evolving field of Natural Language Processing (NLP), Hugging Face has emerged as a prominent and innovative force. This article will explore the story and significance of Hugging face, a company that has made remarkable contributions to NLP and AI.

============================================================
text_summarization NPU 推理测试
============================================================
Model dir: /data/ysws/agentsp/5-15/text_summarization
Output dir: /data/ysws/agentsp/5-15/text_summarization-ascend
NPU available: True
NPU device count: 8
NPU 0: Ascend910B3, total_memory=61.0GB

============================================================
Precision Test: CPU vs NPU
============================================================
Loading tokenizer...
Loading model for CPU...
Loading model for NPU...
Running inference on CPU...
Running inference on NPU...
CPU inference time: 0.271s
NPU inference time: 0.579s
Max absolute error: 4.196167e-05
Max relative error: 9.850012e-07 (0.0001%)
PASS: True (threshold: 1.0%)

============================================================
PRECISION TEST RESULT
============================================================
Relative error: 9.850012e-07
CPU time: 0.271s
NPU time: 0.579s
PASS: True
============================================================
Test Complete!
============================================================

import torch
from transformers import T5ForConditionalGeneration, T5Tokenizer

MODEL_DIR = "/data/ysws/agentsp/5-15/text_summarization"

tokenizer = T5Tokenizer.from_pretrained(MODEL_DIR, legacy=False)
model = T5ForConditionalGeneration.from_pretrained(MODEL_DIR)
model = model.to("npu:0")
model.eval()

input_text = "summarize: Your long text here..."
inputs = tokenizer(input_text, return_tensors="pt", padding=True, truncation=True, max_length=512)
inputs = {k: v.to("npu:0") for k, v in inputs.items()}

with torch.no_grad():
    outputs = model.generate(
        input_ids=inputs["input_ids"],
        attention_mask=inputs["attention_mask"],
        max_length=150,
        min_length=30,
        num_beams=4,
        early_stopping=True
    )

summary = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(f"Summary: {summary}")

from transformers import pipeline

summarizer = pipeline("summarization", model="Falconsai/text_summarization")
summary = summarizer("Your long text to summarize...", max_length=100, min_length=30, do_sample=False)
print(summary)

{
  "d_model": 512,
  "d_ff": 2048,
  "num_heads": 8,
  "num_layers": 6,
  "num_decoder_layers": 6,
  "vocab_size": 32128,
  "n_positions": 512
}