luckymitty/vit_base_patch14_dinov2.lvd142m-ascend-NPU
Model Introduction
Files & versions
Pull Requests
Discussion
Analysis

vit_base_patch14_dinov2.lvd142m (Ascend NPU Adapted)

This repository contains the Ascend NPU adaptation of vit_base_patch14_dinov2.lvd142m, a Vision Transformer (ViT) model trained with the DINOv2 self-supervised method on the LVD-142M dataset.

Model Framework Device License

Model Overview

vit_base_patch14_dinov2.lvd142m is a Vision Transformer (ViT) image feature extraction model, trained with the DINOv2 self-supervised learning method on LVD-142M.

This report documents the adaptation, verification, and performance evaluation of this model on Huawei Ascend NPU (Ascend910).


Model Information

PropertyDetails
Model Namevit_base_patch14_dinov2.lvd142m
Model TypeVision Transformer (ViT) image feature extraction
Source Repositoryhf_mirrors/timm/vit_base_patch14_dinov2.lvd142m
Upstream Sourcetimm/vit_base_patch14_dinov2.lvd142m
FrameworkPyTorch + timm
Parameters86.6 M
GMACs151.7
Activations (M)397.6
Image size518 x 518
Patch Size14 x 14
Feature Dimension768
PoolingCLS Token
Pretrain MethodDINOv2 (self-supervised)
Pretrain DatasetLVD-142M
LicenseCC-BY-NC-4.0

Related Papers

  • DINOv2: Learning Robust Visual Features without Supervision — Oquab et al., 2023
  • An Image is Worth 16x16 Words — Dosovitskiy et al., 2021

Adaptation Environment

Hardware Environment

ItemConfiguration
NPU ModelAscend910
NPU Count1 card
NPU Memory64 GB HBM
CPU Architectureaarch64

Software Environment

ItemVersion
Operating SystemopenEuler / Ubuntu (aarch64)
Python3.11.14
PyTorch2.9.0+cpu
torch_npu2.9.0.post1
timm1.0.27
numpy1.26.4
Pillow12.2.0

Adaptation Process

1. Code Analysis

Comprehensive analysis of the model repository:

# Search for CUDA-specific APIs
rg -n "torch\.cuda" . || echo "No CUDA-specific API found"

# Search for CUDA kernel files
find . -name "*.cu" -o -name "*.cuh" | head -20 || echo "No CUDA kernel files"

Analysis Conclusion:

  • This model is a standard HuggingFace timm model repository with no custom Python code
  • No .cu / .cuh CUDA kernel files
  • No torch.cuda.* or other CUDA-specific API calls
  • Adaptation Type: Zero manual modification adaptation

2. Automatic Migration Injection

Inject transfer_to_npu at the top of the inference script:

import torch_npu
from torch_npu.contrib import transfer_to_npu

transfer_to_npu automatically completes the following mappings:

Original CUDA APIAutomatic Mapping Target
torch.cuda.is_available()Returns True (when NPU is available)
torch.Tensor.cuda()torch.Tensor.npu()
torch.device('cuda')torch.device('npu')
torch.cuda.* series APIstorch.npu.* series APIs

3. Dependency Installation

# Load CANN environment
source /usr/local/Ascend/ascend-toolkit/set_env.sh

# Install dependencies
pip install timm numpy Pillow

Verification Results

Accuracy Verification

We compared the model outputs on CPU and Ascend NPU using randomly initialized weights (structure validation) with multiple random seeds.

Comparison Metrics

MetricValue
Max Absolute Error7.15e-07
Mean Absolute Error1.33e-07
Max Relative Error1.03e-04
Mean Relative Error6.65e-07
Cosine Similarity0.99999999

Multi-Seed Stability

SeedMax Abs ErrorCosine Similarity
427.15e-071.00000000
1239.54e-071.00000000
4568.34e-071.00000008
7897.15e-070.99999992
20247.15e-070.99999992

Conclusion: The CPU and NPU outputs are numerically consistent with extremely high precision. The absolute error is well within the expected floating-point precision tolerance (< 2e-3). The model is verified for Ascend NPU deployment.

Functional Verification

Verification ItemExpected ResultActual ResultStatus
Model LoadingSuccessfully loadedSuccessPass
Device MigrationModel moved to npu:0npu:0Pass
Forward Inference (model())Output shape (1, 768)(1, 768)Pass
Feature Extraction (forward_features)Output shape (1, 1370, 768)(1, 1370, 768)Pass
Data Typetorch.float32torch.float32Pass
Program Exit Code00Pass

Performance Test

Test Configuration:

  • Device: Ascend910 (1 card)
  • Input Size: 518 x 518
  • Batch Size: 1
  • Warmup Runs: 3
  • Test Runs: 10
  • Precision: FP32
MetricValue
Average Latency~25.0 ms
Throughput~40 samples/sec
First Token LatencyN/A (CNN/ViT model)

Note: This is a CNN/ViT vision model with no autoregressive generation process, so there is no first-token latency metric. The larger input size (518x518) and higher GMACs (151.7) compared to standard 224x224 ViT models result in proportionally higher latency.

Precision Notes

  • Ascend910 does not support FP64 (double) precision; torch_npu automatically downgrades to FP32, which does not affect inference for this model
  • This model outputs image feature vectors suitable for downstream retrieval and clustering tasks; precision deviation is within a reasonable range

Usage Guide

Quick Start

# 1. Load CANN environment
source /usr/local/Ascend/ascend-toolkit/set_env.sh

# 2. Set NPU visible device
export ASCEND_RT_VISIBLE_DEVICES=0

# 3. Run inference
python3 npu_inference.py

Image Feature Extraction

import torch
import torch_npu
from torch_npu.contrib import transfer_to_npu
from PIL import Image
import timm

# Load model
model = timm.create_model(
    'hf_mirrors/timm/vit_base_patch14_dinov2.lvd142m',
    pretrained=True,
    num_classes=0,
)
model = model.eval().to('npu:0')

# Get preprocessing
data_config = timm.data.resolve_model_data_config(model)
transforms = timm.data.create_transform(**data_config, is_training=False)

# Load image and infer
img = Image.open('your_image.jpg')
input_tensor = transforms(img).unsqueeze(0).to('npu:0')

with torch.no_grad():
    features = model(input_tensor)  # (1, 768)

print(f"Feature shape: {features.shape}")

Get Unpooled Features

with torch.no_grad():
    # (1, 1370, 768) — includes CLS token
    unpooled = model.forward_features(input_tensor)

    # Take only CLS token features
    cls_token = unpooled[:, 0]  # (1, 768)

Files

FileDescription
config.jsonModel configuration file
npu_inference.pyAscend NPU inference script
verify_accuracy.pyCPU vs NPU accuracy comparison script
accuracy_report.jsonDetailed accuracy metrics JSON
requirements.txtPython dependencies
README.mdThis file

License

CC-BY-NC-4.0

Acknowledgments

  • Original model by timm
  • DINOv2 method by facebookresearch/dinov2
  • Ascend NPU adaptation by Ascend-SACT

Powered by Huawei Ascend NPU