HuggingFace镜像/vit_large_patch16_224.augreg_in21k_ft_in1k
模型介绍文件和版本分析
下载使用量0

vit_large_patch16_224.augreg_in21k_ft_in1k 模型卡片

这是一个 Vision Transformer (ViT) 图像分类模型。由论文作者在 JAX 框架下基于 ImageNet-21k 数据集进行预训练，并在 ImageNet-1k 数据集上（采用额外的数据增强和正则化手段）进行微调，后由 Ross Wightman 移植到 PyTorch 框架。

模型详情

  • 模型类型： 图像分类 / 特征骨干网络
  • 模型统计信息：
    • 参数数量（百万）：304.3
    • 每秒千兆次运算（GMACs）：59.7
    • 激活值数量（百万）：43.8
    • 图像尺寸：224 x 224
  • 相关论文：
    • 《How to train your ViT? Data, Augmentation, and Regularization in Vision Transformers》：https://arxiv.org/abs/2106.10270
    • 《An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale》：https://arxiv.org/abs/2010.11929v2
  • 数据集： ImageNet-1k
  • 预训练数据集： ImageNet-21k
  • 原始项目地址： https://github.com/google-research/vision_transformer

模型用途

图像分类

from urllib.request import urlopen
from PIL import Image
import timm

img = Image.open(urlopen(
    'https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/beignets-task-guide.png'
))

model = timm.create_model('vit_large_patch16_224.augreg_in21k_ft_in1k', pretrained=True)
model = model.eval()

# get model specific transforms (normalization, resize)
data_config = timm.data.resolve_model_data_config(model)
transforms = timm.data.create_transform(**data_config, is_training=False)

output = model(transforms(img).unsqueeze(0))  # unsqueeze single image into batch of 1

top5_probabilities, top5_class_indices = torch.topk(output.softmax(dim=1) * 100, k=5)

图像嵌入

from urllib.request import urlopen
from PIL import Image
import timm

img = Image.open(urlopen(
    'https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/beignets-task-guide.png'
))

model = timm.create_model(
    'vit_large_patch16_224.augreg_in21k_ft_in1k',
    pretrained=True,
    num_classes=0,  # remove classifier nn.Linear
)
model = model.eval()

# get model specific transforms (normalization, resize)
data_config = timm.data.resolve_model_data_config(model)
transforms = timm.data.create_transform(**data_config, is_training=False)

output = model(transforms(img).unsqueeze(0))  # output is (batch_size, num_features) shaped tensor

# or equivalently (without needing to set num_classes=0)

output = model.forward_features(transforms(img).unsqueeze(0))
# output is unpooled, a (1, 197, 1024) shaped tensor

output = model.forward_head(output, pre_logits=True)
# output is a (1, num_features) shaped tensor

模型对比

在 timm 的 model results 中探索此模型的数据集和运行时指标。

引用

@article{steiner2021augreg,
  title={How to train your ViT? Data, Augmentation, and Regularization in Vision Transformers},
  author={Steiner, Andreas and Kolesnikov, Alexander and and Zhai, Xiaohua and Wightman, Ross and Uszkoreit, Jakob and Beyer, Lucas},
  journal={arXiv preprint arXiv:2106.10270},
  year={2021}
}
@article{dosovitskiy2020vit,
  title={An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale},
  author={Dosovitskiy, Alexey and Beyer, Lucas and Kolesnikov, Alexander and Weissenborn, Dirk and Zhai, Xiaohua and Unterthiner, Thomas and  Dehghani, Mostafa and Minderer, Matthias and Heigold, Georg and Gelly, Sylvain and Uszkoreit, Jakob and Houlsby, Neil},
  journal={ICLR},
  year={2021}
}
@misc{rw2019timm,
  author = {Ross Wightman},
  title = {PyTorch Image Models},
  year = {2019},
  publisher = {GitHub},
  journal = {GitHub repository},
  doi = {10.5281/zenodo.4414861},
  howpublished = {\url{https://github.com/huggingface/pytorch-image-models}}
}