intent_classification Ascend NPU 部署指南

intent_classification-ascend/
├── inference.py          # 推理测试脚本
├── log.txt               # 测试日志
├── README.md             # 本文档
├── test_sample.txt       # 测试样例
├── inference_result.json # 推理结果
└── precision_result.json # 精度测试结果

docker exec -it test-modelagent bash

source /usr/local/Ascend/ascend-toolkit/set_env.sh

pip install transformers torch_npu -i https://pypi.huaweicloud.com/repository/pypi/simple/

cd /data/ysws/agentsp/5-16/intent_classification-ascend/

python3 inference.py --mode inference

cd /data/ysws/agentsp/5-16/intent_classification-ascend/

python3 inference.py --mode precision_test

cd /data/ysws/agentsp/5-16/intent_classification-ascend/

python3 inference.py --mode all

============================================================
Intent Classification NPU Test
Model: Falconsai/intent_classification
Output: /data/ysws/agentsp/5-16/intent_classification-ascend
============================================================

============================================================
Intent Classification Inference Test (NPU)
============================================================
Device: npu:0
Model: /data/ysws/agentsp/5-16/intent_classification/Falconsai/intent_classification
Loading tokenizer...
Loading model...
Loading weights: 100%|██████████| 104/104 [00:00<00:00, 5028.45it/s]
Model loaded successfully
Input texts: ['I ordered from you 2 weeks ago and its still not here.', 'I need to bring in my daughter for a checkup.', 'How can I recover my password?']
Input shape: torch.Size([3, 15])
Logits shape: torch.Size([3, 15])
Predictions: [14, 14, 12]
Labels: ['appointment', 'appointment', 'recover password']
Inference time: 0.243s

Inference result saved to /data/ysws/agentsp/5-16/intent_classification-ascend/inference_result.json

============================================================
Precision Test (CPU vs NPU)
============================================================
Using device: npu:0
Loading tokenizer...
Loading model on CPU...
Loading weights: 100%|██████████| 104/104 [00:00<00:00, 4492.03it/s]
Loading model on npu:0...
Loading weights: 100%|██████████| 104/104 [00:00<00:00, 4535.75it/s]
Running inference on CPU...
Running inference on NPU...
CPU inference time: 0.220s
NPU inference time: 0.010s
Speedup: 21.40x
Max absolute error: 8.916855e-03
Max relative error: 0.0397% (threshold: 1.0%)
CPU predictions: [14]
NPU predictions: [14]
Predictions match: True
Status: PASS

Precision result saved to /data/ysws/agentsp/5-16/intent_classification-ascend/precision_result.json

============================================================
Creating Test Sample
============================================================
Saved test sample: /data/ysws/agentsp/5-16/intent_classification-ascend/test_sample.txt

============================================================
Test Complete!
============================================================

import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

MODEL_DIR = "/data/ysws/agentsp/5-16/intent_classification/Falconsai/intent_classification"

tokenizer = AutoTokenizer.from_pretrained(MODEL_DIR)
model = AutoModelForSequenceClassification.from_pretrained(MODEL_DIR)

model = model.to("npu:0").eval()

texts = ["I ordered from you 2 weeks ago and its still not here."]
inputs = tokenizer(texts, return_tensors="pt", padding=True, truncation=True, max_length=128)
inputs = {k: v.to("npu:0") for k, v in inputs.items()}

with torch.no_grad():
    outputs = model(**inputs)

logits = outputs.logits
predictions = torch.argmax(logits, dim=-1)
labels = [model.config.id2label[p.item()] for p in predictions]
print(labels)

texts = [
    "I ordered from you 2 weeks ago and its still not here.",
    "How can I recover my password?",
    "I want to cancel my subscription."
]

inputs = tokenizer(texts, return_tensors="pt", padding=True, truncation=True, max_length=128)
inputs = {k: v.to("npu:0") for k, v in inputs.items()}

with torch.no_grad():
    outputs = model(**inputs)

logits = outputs.logits
predictions = torch.argmax(logits, dim=-1)
labels = [model.config.id2label[p.item()] for p in predictions]

for text, label in zip(texts, labels):
    print(f"{text} -> {label}")

{
  "model_type": "distilbert",
  "n_layers": 6,
  "n_heads": 12,
  "dim": 768,
  "hidden_dim": 3072,
  "vocab_size": 30522,
  "max_position_embeddings": 512,
  "attention_dropout": 0.1,
  "dropout": 0.1
}