medical_summarization 是基于 T5-Large 的医学文本摘要模型，能够对医学文档、研究论文、临床笔记等医疗相关文本生成简洁连贯的摘要。该模型基于 Google 的 T5 Transformer 架构，参数量约 770M。
medical_summarization-ascend/
├── inference.py # 推理测试脚本
├── log.txt # 测试日志
├── README.md # 本文档
├── test_sample.txt # 测试文本样本
├── inference_result.json # 推理结果
└── precision_result.json # 精度测试结果docker exec -it test-modelagent bashsource /usr/local/Ascend/ascend-toolkit/set_env.sh模型文件位于 /data/ysws/agentsp/5-16/medical_summarization/ 目录下：
pip install transformers torch_npu safetensors -i https://pypi.huaweicloud.com/repository/pypi/simple/Run the inference script for medical text summarization:
cd /data/ysws/agentsp/5-16/medical_summarization-ascend/
# 使用默认测试文本
python3 inference.py
# 仅运行推理测试
python3 inference.py --mode inference运行精度对比测试，验证 NPU 计算结果与 CPU 一致性：
cd /data/ysws/agentsp/5-16/medical_summarization-ascend/
# 运行完整精度测试
python3 inference.py --mode precision_test| 参数 | 说明 | 默认值 |
|---|---|---|
--mode | 测试模式: all, inference 或 precision_test | all |
| 指标 | 实测值 | 阈值 | 状态 |
|---|---|---|---|
| Token 差异率 | 0.0000% | < 1.00% | PASS |
| CPU 推理时间 | 33.037s | - | - |
| NPU 推理时间 | 3.411s | - | - |
| 加速比 | 9.69x | > 1x | PASS |
输入文本 (医学文献摘要任务):
the need for magnetic resonance imaging ( mri ) in patients with an implanted pacemaker or implantable cardioverter - defibrillator ( icd ) is a growing clinical issue...
生成摘要:
the need for magnetic resonance imaging ( mri ) in patients with an implanted pacemaker or implantable cardioverter - defibrillator ( icd ) is a growing clinical issue. the need for magnetic resonance imaging ( mri ) in patients with an implanted pacemaker or implantable cardioverter - defibrillator ( icd ) is a growing clinical issue. it is estimated that as many as 75% of active cardiac device recipients will become indicated for mri...
结果: CPU 和 NPU 生成的摘要完全一致，Token 差异率为 0.0000%
完整测试日志保存在 log.txt
Medical Summarization NPU Test
Model: Falconsai/medical_summarization (T5-Large)
Output: /data/ysws/agentsp/5-16/medical_summarization-ascend
Device: npu:0 Loading model and tokenizer... Model loaded successfully Input text length: 732 chars Input tokens: 202 Inference time: 4.654s Summary: the need for magnetic resonance imaging ( mri ) in patients with an implanted pacemaker or implantable cardioverter - defibrillator ( icd ) is a growing clinical issue. the need for magnetic resonance imaging ( mri ) in patients with an implanted pacemaker or implantable cardioverter - defibrillator ( icd ) is a growing clinical issue. it is estimated that as many as 75% of active cardiac device recipients will become indicated for mri. magnetic resonance imaging ( mri ) in patients with an implanted pacemaker or implantable cardioverter - defibrillator ( icd ) system, an implantable defibrillator with no leads that touch the heart, has recently been demonstrated to be a safe and effective defibri
已保存至：/data/ysws/agentsp/5-16/medical_summarization-ascend/test_sample.txt
NPU 设备：npu:0 正在加载模型... 输入 tokens：202 正在 CPU 上运行... 正在 NPU 上运行... CPU 推理时间：33.037s NPU 推理时间：3.411s 加速比：9.69 倍 Token 差异：0 / 200（0.0000%） CPU 摘要：the need for magnetic resonance imaging ( mri ) in patients with an implanted pacemaker or implantable cardioverter - defibrillator ( icd ) is a growing clinical issue. the need for magnetic resonance imaging ( mri ) in patients with an implanted pacemaker or implantable cardioverter - defibrillator ( icd ) is a growing clinical issue. it is estimated that as many as 75% of active cardiac device recipients will become indicated for mri. magnetic resonance imaging ( mri ) in patients with an implanted pacemaker or implantable cardioverter - defibrillator ( icd ) system, an implantable defibrillator with no leads that touch the heart, has recently been demonstrated to be a safe and effective defibri NPU 摘要：the need for magnetic resonance imaging ( mri ) in patients with an implanted pacemaker or implantable cardioverter - defibrillator ( icd ) is a growing clinical issue. the need for magnetic resonance imaging ( mri ) in patients with an implanted pacemaker or implantable cardioverter - defibrillator ( icd ) is a growing clinical issue. it is estimated that as many as 75% of active cardiac device recipients will become indicated for mri. magnetic resonance imaging ( mri ) in patients with an implanted pacemaker or implantable cardioverter - defibrillator ( icd ) system, an implantable defibrillator with no leads that touch the heart, has recently been demonstrated to be a safe and effective defibri
输出文件：
import torch
from transformers import T5ForConditionalGeneration, AutoTokenizer
MODEL_DIR = "/data/ysws/agentsp/5-16/medical_summarization"
tokenizer = AutoTokenizer.from_pretrained(MODEL_DIR)
model = T5ForConditionalGeneration.from_pretrained(MODEL_DIR)
model = model.to("npu:0")
model.eval()
medical_text = """
the need for magnetic resonance imaging ( mri ) in patients with an implanted
pacemaker or implantable cardioverter - defibrillator ( icd ) is a growing clinical issue.
"""
inputs = tokenizer(medical_text, return_tensors="pt", max_length=512, truncation=True)
inputs = {k: v.to("npu:0") for k, v in inputs.items()}
with torch.no_grad():
outputs = model.generate(
input_ids=inputs["input_ids"],
attention_mask=inputs["attention_mask"],
max_length=200,
min_length=30,
num_beams=4,
length_penalty=2.0,
early_stopping=True
)
summary = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(f"Summary: {summary}")with torch.no_grad():
outputs = model.generate(
input_ids=inputs["input_ids"],
attention_mask=inputs["attention_mask"],
max_length=150, # 最大生成长度
min_length=50, # 最小生成长度
num_beams=5, # Beam 搜索数量
length_penalty=1.0, # 长度惩罚
no_repeat_ngram_size=3, # 避免重复
early_stopping=True
)| 组件 | 说明 |
|---|---|
| encoder | 6 层 Transformer 编码器 |
| decoder | 6 层 Transformer 解码器 |
| lm_head | 语言模型输出层 |
从 config.json 提取的关键参数:
{
"d_model": 512,
"d_ff": 2048,
"d_kv": 64,
"num_heads": 8,
"num_layers": 6,
"vocab_size": 32128,
"eos_token_id": 1,
"pad_token_id": 0,
"decoder_start_token_id": 0
}A: 检查 NPU 驱动是否正确安装，确保 CANN 环境变量已 source。T5 模型在 CPU 和 NPU 上的数值误差极小（< 0.01%），差异主要来自浮点精度表示。
A: 使用批处理可以显著提高吞吐量。另外，首次推理会有编译开销，后续推理会更快。NPU 相比 CPU 有显著加速（9.69x）。
A: 该模型在医学文本摘要任务上表现良好，但建议根据实际用例调整 max_length、min_length 和 num_beams 参数以获得最佳效果。
A: 该模型主要针对英语医学文本训练。T5 架构本身支持多语言，但最佳效果在英语医学文献上。
本项目遵循 Apache-2.0 许可证