luckymitty/vit_base_patch16_224.dino-ascend-NPU
Model Introduction
Files & versions
Pull Requests
Discussion
Analysis

vit_base_patch16_224.dino (Ascend NPU Adapted)

This repository contains the Ascend NPU adaptation of vit_base_patch16_224.dino, a Vision Transformer (ViT) model trained with the DINO self-supervised method.

Model Framework Device License

Model Overview

vit_base_patch16_224.dino is a Vision Transformer (ViT) image feature extraction model, trained with the DINO self-supervised learning method on ImageNet-1k.

This report documents the adaptation, verification, and performance evaluation of this model on Huawei Ascend NPU (Ascend910).


Model Information

PropertyDetails
Model Namevit_base_patch16_224.dino
Model TypeVision Transformer (ViT) image feature extraction
Source Repositoryhf_mirrors/timm/vit_base_patch16_224.dino
Upstream Sourcetimm/vit_base_patch16_224.dino
FrameworkPyTorch + timm
Parameters85.8 M
GMACs16.9
Activations (M)16.5
Image size224 x 224
Patch Size16 x 16
Feature Dimension768
PoolingCLS Token
Pretrain MethodDINO (self-supervised)
Pretrain DatasetImageNet-1k
LicenseApache 2.0

Related Papers

  • Emerging Properties in Self-Supervised Vision Transformers — Caron et al., 2021
  • An Image is Worth 16x16 Words — Dosovitskiy et al., 2021

Adaptation Environment

Hardware Environment

ItemConfiguration
NPU ModelAscend910
NPU Count1 card
NPU Memory64 GB HBM
CPU Architectureaarch64

Software Environment

ItemVersion
Operating SystemopenEuler / Ubuntu (aarch64)
Python3.11.14
PyTorch2.9.0+cpu
torch_npu2.9.0.post1
timm1.0.27
numpy1.26.4
Pillow12.2.0

Adaptation Process

1. Code Analysis

Comprehensive analysis of the model repository:

# Search for CUDA-specific APIs
rg -n "torch\.cuda" . || echo "No CUDA-specific API found"

# Search for CUDA kernel files
find . -name "*.cu" -o -name "*.cuh" | head -20 || echo "No CUDA kernel files"

Analysis Conclusion:

  • This model is a standard HuggingFace timm model repository with no custom Python code
  • No .cu / .cuh CUDA kernel files
  • No torch.cuda.* or other CUDA-specific API calls
  • Adaptation Type: Zero manual modification adaptation

2. Automatic Migration Injection

Inject transfer_to_npu at the top of the inference script:

import torch_npu
from torch_npu.contrib import transfer_to_npu

transfer_to_npu automatically completes the following mappings:

Original CUDA APIAutomatic Mapping Target
torch.cuda.is_available()Returns True (when NPU is available)
torch.Tensor.cuda()torch.Tensor.npu()
torch.device('cuda')torch.device('npu')
torch.cuda.* series APIstorch.npu.* series APIs

3. Dependency Installation

# Load CANN environment
source /usr/local/Ascend/ascend-toolkit/set_env.sh

# Install dependencies
pip install timm numpy Pillow

Verification Results

Accuracy Verification

We compared the model outputs on CPU and Ascend NPU using randomly initialized weights (structure validation) with multiple random seeds.

Comparison Metrics

MetricValue
Max Absolute Error1.16e-03
Mean Absolute Error2.40e-04
Max Relative Error1.48e-01
Mean Relative Error1.39e-03
Cosine Similarity1.00000000

Multi-Seed Stability

SeedMax Abs ErrorCosine Similarity
421.16e-031.00000000
1231.20e-030.99999992
4568.56e-040.99999992
7899.14e-040.99999992
20249.36e-040.99999992

Conclusion: The CPU and NPU outputs are numerically consistent. The absolute error is within the expected floating-point precision tolerance (< 2e-3). The model is verified for Ascend NPU deployment.

Functional Verification

Verification ItemExpected ResultActual ResultStatus
Model LoadingSuccessfully loadedSuccessPass
Device MigrationModel moved to npu:0npu:0Pass
Forward Inference (model())Output shape (1, 768)(1, 768)Pass
Feature Extraction (forward_features)Output shape (1, 197, 768)(1, 197, 768)Pass
Data Typetorch.float32torch.float32Pass
Program Exit Code00Pass

Performance Test

Test Configuration:

  • Device: Ascend910 (1 card)
  • Input Size: 224 x 224
  • Batch Size: 1
  • Warmup Runs: 3
  • Test Runs: 10
  • Precision: FP32
MetricValue
Average Latency~6.0 ms
Throughput~170 samples/sec
First Token LatencyN/A (CNN/ViT model)

Note: This is a CNN/ViT vision model with no autoregressive generation process, so there is no first-token latency metric.

Precision Notes

  • Ascend910 does not support FP64 (double) precision; torch_npu automatically downgrades to FP32, which does not affect inference for this model
  • This model outputs image feature vectors suitable for downstream retrieval and clustering tasks; precision deviation is within a reasonable range

Usage Guide

Quick Start

# 1. Load CANN environment
source /usr/local/Ascend/ascend-toolkit/set_env.sh

# 2. Set NPU visible device
export ASCEND_RT_VISIBLE_DEVICES=0

# 3. Run inference
python3 npu_inference.py

Image Feature Extraction

import torch
import torch_npu
from torch_npu.contrib import transfer_to_npu
from PIL import Image
import timm

# Load model
model = timm.create_model(
    'hf_mirrors/timm/vit_base_patch16_224.dino',
    pretrained=True,
    num_classes=0,
)
model = model.eval().to('npu:0')

# Get preprocessing
data_config = timm.data.resolve_model_data_config(model)
transforms = timm.data.create_transform(**data_config, is_training=False)

# Load image and infer
img = Image.open('your_image.jpg')
input_tensor = transforms(img).unsqueeze(0).to('npu:0')

with torch.no_grad():
    features = model(input_tensor)  # (1, 768)

print(f"Feature shape: {features.shape}")

Get Unpooled Features

with torch.no_grad():
    # (1, 197, 768) — includes CLS token
    unpooled = model.forward_features(input_tensor)

    # Take only CLS token features
    cls_token = unpooled[:, 0]  # (1, 768)

Files

FileDescription
config.jsonModel configuration file
npu_inference.pyAscend NPU inference script
verify_accuracy.pyCPU vs NPU accuracy comparison script
accuracy_report.jsonDetailed accuracy metrics JSON
requirements.txtPython dependencies
README.mdThis file

License

Apache-2.0

Acknowledgments

  • Original model by timm
  • DINO method by facebookresearch/dino
  • Ascend NPU adaptation by Ascend-SACT

Powered by Huawei Ascend NPU