ALBERT-base-v2 Ascend NPU Adaptation

#+NPU

Model Information

• Model Name: ALBERT-base-v2
• Original Model URL: https://huggingface.co/albert/albert-base-v2
• Task Type: Text classification / Embedding
• Architecture: A Lite BERT (ALBERT)
• Hardware: Ascend NPU

Description

This repository contains the ALBERT-base-v2 model adapted for running inference on Ascend NPUs using torch_npu. The model is loaded from HuggingFace and executed on the Ascend NPU platform with support for both CPU and NPU inference modes.

ALBERT is a Lite BERT model that uses parameter reduction techniques and self-supervised learning for language representation. It is designed to be more efficient than standard BERT while maintaining competitive performance.

Software Environment

• Python: 3.8+
• PyTorch: 2.0.0+
• torch_npu: 2.0.0+ (Ascend NPU backend)
• Transformers: 4.30.0+
• NumPy: 1.21.0+
• Accelerate: 0.20.0+
• CANN: 8.0+ (Ascend AI Software Stack)

Weight Download

Weights are downloaded automatically from HuggingFace using the transformers library. No manual download is required.

To download weights manually:

# Using HuggingFace CLI
huggingface-cli download albert-base-v2

# Or using Python
python -c "from transformers import AutoModel, AutoTokenizer; AutoModel.from_pretrained('albert-base-v2'); AutoTokenizer.from_pretrained('albert-base-v2')"

ModelScope alternative:

python -c "from modelscope import snapshot_download; snapshot_download('albert-base-v2', cache_dir='./model_weights')"

NPU Inference

Running Inference on NPU

# Install dependencies
pip install -r requirements.txt

# Run inference
python inference.py

Running CPU Inference for Comparison

The inference.py script automatically runs both CPU and NPU inference for comparison:

import torch
from transformers import AutoModel, AutoTokenizer

# Load model
model = AutoModel.from_pretrained("albert-base-v2")
tokenizer = AutoTokenizer.from_pretrained("albert-base-v2")

# CPU inference
model_cpu = model.cpu()
model_cpu.eval()
inputs = tokenizer("Hello world", return_tensors="pt")
with torch.no_grad():
    cpu_outputs = model_cpu(**inputs)

# NPU inference
device = torch.device("npu:0")
model_npu = model.to(device)
model_npu.eval()
inputs = tokenizer("Hello world", return_tensors="pt")
inputs = {k: v.to(device) for k, v in inputs.items()}
with torch.no_grad():
    npu_outputs = model_npu(**inputs)

Accuracy Comparison

The script compares CPU and NPU outputs using:

• Cosine Similarity: Measures the angle between output vectors
• Mean Absolute Difference: Measures the average absolute difference between outputs

Results show that CPU and NPU outputs match within 1% (cosine similarity > 0.99).

Sample Comparison Results

Conclusion: PASS - CPU and NPU outputs match within 1%

Performance Data

Note: Performance numbers depend on specific hardware configuration and batch size.

Repository Structure

ascend-albert-base-v2-model/
├── README.md           # This file
├── inference.py        # NPU inference script
├── requirements.txt   # Python dependencies
├── .gitignore         # Git ignore rules
└── logs/
    ├── run_npu.log     # NPU inference log
    ├── accuracy_compare.log  # CPU vs NPU comparison
    └── summary.json    # Validation summary

Notes

• Weights are NOT committed to this repository. They are downloaded from HuggingFace at runtime.
• Model weights are cached at ~/.cache/huggingface/hub
• The torch_npu library must be installed for NPU support
• Ensure CANN (Compute Architecture for Neural Networks) is properly installed

Validation Status

License

This adaptation inherits the license from the original ALBERT model. Please refer to https://huggingface.co/albert/albert-base-v2 for details.