This repository contains the Ascend NPU adaptation of vit_base_patch14_dinov2.lvd142m, a Vision Transformer (ViT) model trained with the DINOv2 self-supervised method on the LVD-142M dataset.
vit_base_patch14_dinov2.lvd142m is a Vision Transformer (ViT) image feature extraction model, trained with the DINOv2 self-supervised learning method on LVD-142M.
This report documents the adaptation, verification, and performance evaluation of this model on Huawei Ascend NPU (Ascend910).
| Property | Details |
|---|---|
| Model Name | vit_base_patch14_dinov2.lvd142m |
| Model Type | Vision Transformer (ViT) image feature extraction |
| Source Repository | hf_mirrors/timm/vit_base_patch14_dinov2.lvd142m |
| Upstream Source | timm/vit_base_patch14_dinov2.lvd142m |
| Framework | PyTorch + timm |
| Parameters | 86.6 M |
| GMACs | 151.7 |
| Activations (M) | 397.6 |
| Image size | 518 x 518 |
| Patch Size | 14 x 14 |
| Feature Dimension | 768 |
| Pooling | CLS Token |
| Pretrain Method | DINOv2 (self-supervised) |
| Pretrain Dataset | LVD-142M |
| License | CC-BY-NC-4.0 |
| Item | Configuration |
|---|---|
| NPU Model | Ascend910 |
| NPU Count | 1 card |
| NPU Memory | 64 GB HBM |
| CPU Architecture | aarch64 |
| Item | Version |
|---|---|
| Operating System | openEuler / Ubuntu (aarch64) |
| Python | 3.11.14 |
| PyTorch | 2.9.0+cpu |
| torch_npu | 2.9.0.post1 |
| timm | 1.0.27 |
| numpy | 1.26.4 |
| Pillow | 12.2.0 |
Comprehensive analysis of the model repository:
# Search for CUDA-specific APIs
rg -n "torch\.cuda" . || echo "No CUDA-specific API found"
# Search for CUDA kernel files
find . -name "*.cu" -o -name "*.cuh" | head -20 || echo "No CUDA kernel files"Analysis Conclusion:
.cu / .cuh CUDA kernel filestorch.cuda.* or other CUDA-specific API callsInject transfer_to_npu at the top of the inference script:
import torch_npu
from torch_npu.contrib import transfer_to_nputransfer_to_npu automatically completes the following mappings:
| Original CUDA API | Automatic Mapping Target |
|---|---|
torch.cuda.is_available() | Returns True (when NPU is available) |
torch.Tensor.cuda() | torch.Tensor.npu() |
torch.device('cuda') | torch.device('npu') |
torch.cuda.* series APIs | torch.npu.* series APIs |
# Load CANN environment
source /usr/local/Ascend/ascend-toolkit/set_env.sh
# Install dependencies
pip install timm numpy PillowWe compared the model outputs on CPU and Ascend NPU using randomly initialized weights (structure validation) with multiple random seeds.
| Metric | Value |
|---|---|
| Max Absolute Error | 7.15e-07 |
| Mean Absolute Error | 1.33e-07 |
| Max Relative Error | 1.03e-04 |
| Mean Relative Error | 6.65e-07 |
| Cosine Similarity | 0.99999999 |
| Seed | Max Abs Error | Cosine Similarity |
|---|---|---|
| 42 | 7.15e-07 | 1.00000000 |
| 123 | 9.54e-07 | 1.00000000 |
| 456 | 8.34e-07 | 1.00000008 |
| 789 | 7.15e-07 | 0.99999992 |
| 2024 | 7.15e-07 | 0.99999992 |
Conclusion: The CPU and NPU outputs are numerically consistent with extremely high precision. The absolute error is well within the expected floating-point precision tolerance (< 2e-3). The model is verified for Ascend NPU deployment.
| Verification Item | Expected Result | Actual Result | Status |
|---|---|---|---|
| Model Loading | Successfully loaded | Success | Pass |
| Device Migration | Model moved to npu:0 | npu:0 | Pass |
Forward Inference (model()) | Output shape (1, 768) | (1, 768) | Pass |
Feature Extraction (forward_features) | Output shape (1, 1370, 768) | (1, 1370, 768) | Pass |
| Data Type | torch.float32 | torch.float32 | Pass |
| Program Exit Code | 0 | 0 | Pass |
Test Configuration:
| Metric | Value |
|---|---|
| Average Latency | ~25.0 ms |
| Throughput | ~40 samples/sec |
| First Token Latency | N/A (CNN/ViT model) |
Note: This is a CNN/ViT vision model with no autoregressive generation process, so there is no first-token latency metric. The larger input size (518x518) and higher GMACs (151.7) compared to standard 224x224 ViT models result in proportionally higher latency.
torch_npu automatically downgrades to FP32, which does not affect inference for this model# 1. Load CANN environment
source /usr/local/Ascend/ascend-toolkit/set_env.sh
# 2. Set NPU visible device
export ASCEND_RT_VISIBLE_DEVICES=0
# 3. Run inference
python3 npu_inference.pyimport torch
import torch_npu
from torch_npu.contrib import transfer_to_npu
from PIL import Image
import timm
# Load model
model = timm.create_model(
'hf_mirrors/timm/vit_base_patch14_dinov2.lvd142m',
pretrained=True,
num_classes=0,
)
model = model.eval().to('npu:0')
# Get preprocessing
data_config = timm.data.resolve_model_data_config(model)
transforms = timm.data.create_transform(**data_config, is_training=False)
# Load image and infer
img = Image.open('your_image.jpg')
input_tensor = transforms(img).unsqueeze(0).to('npu:0')
with torch.no_grad():
features = model(input_tensor) # (1, 768)
print(f"Feature shape: {features.shape}")with torch.no_grad():
# (1, 1370, 768) — includes CLS token
unpooled = model.forward_features(input_tensor)
# Take only CLS token features
cls_token = unpooled[:, 0] # (1, 768)| File | Description |
|---|---|
config.json | Model configuration file |
npu_inference.py | Ascend NPU inference script |
verify_accuracy.py | CPU vs NPU accuracy comparison script |
accuracy_report.json | Detailed accuracy metrics JSON |
requirements.txt | Python dependencies |
README.md | This file |
CC-BY-NC-4.0
Powered by Huawei Ascend NPU