jsjs1258/WhereIsAI-UAE-Large-V1-NPU-model-2
Model Introduction
Files & versions
Pull Requests
Discussion
Analysis

UAE-Large-V1 NPU

WhereIsAI/UAE-Large-V1 adapted for Huawei Ascend NPU (Ascend910_9362). Embedding route (BertModel, 1024-dim, CLS pooling).

  • Model: https://huggingface.co/WhereIsAI/UAE-Large-V1
  • Revision: 9c9b2c999b3350cfb3171ed429320668e39b00b8
  • Architecture: BertModel (bert, 24 layers, 16 heads, hidden 1024, vocab 30522)
  • Pooling: CLS token (1_Pooling/config.json: pooling_mode_cls_token=true), L2 normalized
  • Device: npu:0 (verified), CPU fallback for non-NPU
  • Validation: cosine >=0.999 (actual 0.999992), max_diff 0.000457, 291 sentences/s (batch 6)

Usage

from transformers import AutoTokenizer, AutoModel
import torch, torch_npu, torch.nn.functional as F

model_dir = "/opt/atomgit/.cache/huggingface/hub/models--WhereIsAI--UAE-Large-V1/snapshots/9c9b2c999b3350cfb3171ed429320668e39b00b8"
tokenizer = AutoTokenizer.from_pretrained(model_dir)
model = AutoModel.from_pretrained(model_dir).to("npu:0").eval()
inputs = tokenizer(["Hello world", "Paris is the capital of France"], padding=True, truncation=True, return_tensors="pt")
inputs = {k: v.to("npu:0") for k,v in inputs.items()}
with torch.no_grad():
    out = model(**inputs)
    emb = F.normalize(out.last_hidden_state[:,0], p=2, dim=1)
print(emb)

Run python inference.py (requires torch_npu on NPU, else CPU).

Evidence

agent workflow npu device call model result

Hardware: Ascend910_9362, 2 devices, 60.9 GiB free /61.27 GiB, device npu:0 verified (model, input_ids, emb).