冬
gcw_IDzXRVNw/granite-guardian-hap-38m-ascend
模型介绍文件和版本Pull Requests讨论分析
下载使用量0

Granite-Guardian-HAP-38M Ascend NPU 部署指南

项目简介

Granite-Guardian-HAP-38M 是 IBM 的 38M 参数文本分类模型，用于检测仇恨言论 (Hate Speech)、攻击性语言 (Abuse) 和不当内容。该模型基于 RoBERTa 架构，针对内容安全场景优化。

特性

  • 支持 Ascend NPU 推理加速
  • CPU 与 NPU 精度对比测试（预测完全一致，误差 < 1%）
  • 高效文本分类推理
  • 兼容 HuggingFace transformers
  • 支持三类分类：仇恨言论、攻击性语言、正常内容

环境要求

  • 硬件：华为 Ascend 910 系列 NPU
  • CANN：8.0.RC1 或更高版本
  • PyTorch：2.0+ 并带有 torch_npu
  • Docker：容器名称 test-modelagent
  • transformers：4.31+

目录结构

granite-guardian-hap-38m-ascend/
├── inference.py          # 推理测试脚本
├── log.txt               # 测试日志
├── README.md             # 本文档
├── test_sample.txt       # 测试样本
├── inference_result.json # 推理结果
└── precision_result.json # 精度测试结果

部署步骤

1. 进入容器

docker exec -it test-modelagent bash

2. 设置环境变量

source /usr/local/Ascend/ascend-toolkit/set_env.sh

3. 准备模型文件

模型文件位于 /data/ysws/agentsp/5-18-2/granite-guardian-hap-38m/ibm-granite/granite-guardian-hap-38m/ 目录下：

  • pytorch_model.bin / model.safetensors - 模型权重（约154MB）
  • config.json - 模型配置
  • tokenizer.json / vocab.json - 分词器文件
  • tokenizer_config.json - 分词器配置

4. 安装依赖

pip install transformers torch_npu

Usage

Method 1: Normal Inference Mode

cd /data/ysws/agentsp/5-18-2/granite-guardian-hap-38m-ascend/
python3 inference.py

方式二：精度测试模式 (CPU vs NPU)

cd /data/ysws/agentsp/5-18-2/granite-guardian-hap-38m-ascend/
python3 inference.py precision_test

测试验证

精度测试结果

指标实测值阈值状态
预测一致性True100%PASS
最大相对误差0.0204%< 1%PASS
NPU 加速比6.65x> 5xPASS

性能数据

操作耗时
CPU 推理时间0.040s
NPU 推理时间0.006s
加速比6.65x

分类结果示例

输入文本预测类别
"This is a hate speech example with racial slurs."hate_speech
"I love you guys, you are amazing!"neither
"You are such a stupid idiot."offensive_language

结果: CPU 和 NPU 预测完全一致，相对误差仅 0.0204%，远优于 1% 阈值

测试日志

完整测试日志如下：

============================================================
Granite-Guardian-HAP-38M NPU Test
Output: /data/ysws/agentsp/5-18-2/granite-guardian-hap-38m-ascend
============================================================

============================================================
Granite-Guardian-HAP-38M Inference Test (NPU)
============================================================
Device: npu:0
Model: /data/ysws/agentsp/5-18-2/granite-guardian-hap-38m/ibm-granite/granite-guardian-hap-38m

Loading tokenizer...
Loading model...
Loading weights: 100%|██████████| 73/73 [00:00<00:00, 4742.19it/s]
[W518 07:38:59.285068581 compiler_depend.ts:164] Warning: Device do not support double dtype now, dtype cast replace with float. (function operator())

Input text: ['This is a hate speech example with racial slurs.']
Input shape: torch.Size([1, 12])
Logits: tensor([[ 2.6934, -2.8224]], device='npu:0')
Predicted class: hate_speech
Inference time: 0.235s

============================================================
Precision Test (CPU vs NPU)
============================================================

Loading model on CPU...
Loading weights: 100%|██████████| 73/73 [00:00<00:00, 4694.64it/s]
Running inference on CPU...

Loading model on NPU...
Loading weights: 100%|██████████| 73/73 [00:00<00:00, 4721.35it/s]
Running inference on NPU...

CPU inference time: 0.040s
NPU inference time: 0.006s
Speedup: 6.65x
CPU prediction: hate_speech
NPU prediction: hate_speech
Predictions match: True
Max absolute error: 5.767345e-04
Max relative error: 2.043185e-04 (0.0204%)
Status: PASS

============================================================
Creating Test Sample
============================================================
Saved test sample
  1. This is a hate speech example with racial slurs.
  2. I love you guys, you are amazing!
  3. You are such a stupid idiot.

============================================================
Test Complete!

Python API 使用示例

import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

MODEL_DIR = "/data/ysws/agentsp/5-18-2/granite-guardian-hap-38m/ibm-granite/granite-guardian-hap-38m"

tokenizer = AutoTokenizer.from_pretrained(MODEL_DIR)
model = AutoModelForSequenceClassification.from_pretrained(MODEL_DIR)
model = model.to("npu:0")
model.eval()

labels = ["hate_speech", "offensive_language", "neither"]

texts = ["This is a hate speech example with racial slurs."]
inputs = tokenizer(texts, return_tensors="pt", padding=True, truncation=True, max_length=128)
inputs = {k: v.to("npu:0") for k, v in inputs.items()}

with torch.no_grad():
    outputs = model(**inputs)
    predictions = torch.argmax(outputs.logits, dim=-1)

predicted_label = labels[predictions.item()]
print(f"Predicted: {predicted_label}")  # "hate_speech"

模型结构

  • 架构类型: RoBERTa（RobertaForSequenceClassification）
  • 编码器: 4 层 Transformer
  • 隐藏层维度: 576
  • 注意力头数: 12
  • 参数量: ~38M
  • 分类类别: 3（hate_speech、offensive_language、neither）
组件说明
embeddingsRoBERTa 词嵌入（vocab_size=50265）
encoder4 层 Transformer 编码器
classifier线性分类层（576 → 3）

推理参数配置

从 config.json 提取的关键参数:

{
  "hidden_size": 576,
  "intermediate_size": 768,
  "num_attention_heads": 12,
  "num_hidden_layers": 4,
  "vocab_size": 50265,
  "problem_type": "single_label_classification",
  "torch_dtype": "float32"
}

常见问题

Q: 精度测试失败?

A: 检查 NPU 驱动是否正确安装，确保 CANN 环境变量已 source。

Q: 如何处理长文本?

A: 使用 truncation=True 和 max_length=128 参数截断超长文本。

Q: 支持哪些分类标签?

A: 模型输出 3 个类别：hate_speech (仇恨言论)、offensive_language (攻击性语言)、neither (正常内容)。

参考链接

  • 原始模型: https://huggingface.co/ibm-granite/granite-guardian-hap-38m
  • IBM Granite: https://www.ibm.com/blogs/research/2024/06/granite-guardian-open-source-content-safety-models/
  • HuggingFace Transformers: https://huggingface.co/transformers

许可证

本项目遵循 Apache-2.0 许可证