bert-emotion Ascend NPU 部署指南

bert-emotion-ascend/
├── inference.py          # 推理测试脚本
├── log.txt               # 测试日志
├── README.md             # 本文档
├── test_sample.txt       # 测试样例
├── inference_result.json # 推理结果
└── precision_result.json # 精度测试结果

docker exec -it test-modelagent bash

source /usr/local/Ascend/ascend-toolkit/set_env.sh

pip install transformers torch_npu -i https://pypi.huaweicloud.com/repository/pypi/simple/

cd /data/ysws/agentsp/5-16/bert-emotion-ascend/

python3 inference.py --mode inference

cd /data/ysws/agentsp/5-16/bert-emotion-ascend/

python3 inference.py --mode precision_test

cd /data/ysws/agentsp/5-16/bert-emotion-ascend/

python3 inference.py --mode all

============================================================
BERT-Emotion NPU Test
Model: boltuix/bert-emotion
Output: /data/ysws/agentsp/5-16/bert-emotion-ascend
============================================================

============================================================
BERT-Emotion Inference Test (NPU)
============================================================
Device: npu:0
Model: /data/ysws/agentsp/5-16/bert-emotion/boltuix/bert-emotion
Loading tokenizer...
Loading model...
Loading weights: 100%|██████████| 73/73 [00:00<00:00, 4817.25it/s]
Model loaded successfully
Input texts: ['I am so happy today! Everything is going great!', 'This is really disappointing and makes me sad.', "I can't believe it! That's amazing news!"]
Input shape: torch.Size([3, 15])
Logits shape: torch.Size([3, 13])
Predictions: [5, 0, 3]
Labels: ['happiness', 'sadness', 'surprise']
Inference time: 0.236s

Inference result saved to /data/ysws/agentsp/5-16/bert-emotion-ascend/inference_result.json

============================================================
Precision Test (CPU vs NPU)
============================================================
Using device: npu:0
Loading tokenizer...
Loading model on CPU...
Loading weights: 100%|██████████| 73/73 [00:00<00:00, 4589.30it/s]
Loading model on npu:0...
Loading weights: 100%|██████████| 73/73 [00:00<00:00, 4248.72it/s]
Running inference on CPU...
Running inference on NPU...
CPU inference time: 0.021s
NPU inference time: 0.008s
Speedup: 2.46x
Max absolute error: 7.374287e-04
Max relative error: 0.0110% (threshold: 1.0%)
CPU predictions: [5] -> happiness
NPU predictions: [5] -> happiness
Predictions match: True
Status: PASS

Precision result saved to /data/ysws/agentsp/5-16/bert-emotion-ascend/precision_result.json

============================================================
Creating Test Sample
============================================================
Saved test sample: /data/ysws/agentsp/5-16/bert-emotion-ascend/test_sample.txt

============================================================
Test Complete!
============================================================

import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

MODEL_DIR = "/data/ysws/agentsp/5-16/bert-emotion/boltuix/bert-emotion"

tokenizer = AutoTokenizer.from_pretrained(MODEL_DIR)
model = AutoModelForSequenceClassification.from_pretrained(MODEL_DIR)

model = model.to("npu:0").eval()

texts = ["I am so happy today!", "This is really disappointing."]
inputs = tokenizer(texts, return_tensors="pt", padding=True, truncation=True, max_length=128)
inputs = {k: v.to("npu:0") for k, v in inputs.items()}

with torch.no_grad():
    outputs = model(**inputs)

predictions = torch.argmax(outputs.logits, dim=-1)
labels = [model.config.id2label[p.item()] for p in predictions]
print(labels)

def detect_emotion(text):
    inputs = tokenizer(text, return_tensors="pt", padding=True, truncation=True, max_length=128)
    inputs = {k: v.to("npu:0") for k, v in inputs.items()}

    with torch.no_grad():
        outputs = model(**inputs)

    prediction = torch.argmax(outputs.logits, dim=-1).item()
    return model.config.id2label[prediction]

emotion = detect_emotion("I'm feeling really anxious about this.")
print(f"Detected emotion: {emotion}")

{
  "model_type": "bert",
  "hidden_size": 256,
  "num_hidden_layers": 4,
  "num_attention_heads": 4,
  "intermediate_size": 1024,
  "vocab_size": 30522,
  "max_position_embeddings": 512,
  "attention_probs_dropout_prob": 0.1,
  "hidden_dropout_prob": 0.1
}