This is an Ascend NPU-adapted deployment for BAAI/bge-base-en-v1.5 (BertModel, embedding).
npu:0 with synchronized timing and CPU consistency validationModel and tensors are explicitly placed on npu:0 and verified:
next(model.parameters()).device == npu:0input_ids.device == npu:0attention_mask.device == npu:0last_hidden_state.device == npu:0pooled (CLS) device == npu:0normalized embedding device == npu:0Synchronization uses torch.npu.synchronize() before/after timing. First inference includes compilation, stable perf measured after warmup.
Validated on Ascend 910 (2x NPU, ~61 GiB free per card, torch_npu 2.9.0.post1).
from transformers import AutoTokenizer, AutoModel
import torch, torch_npu
import torch.nn.functional as F
model_dir = "./model"
tokenizer = AutoTokenizer.from_pretrained(model_dir)
model = AutoModel.from_pretrained(model_dir).to("npu:0")
model.eval()
texts = ["What is the capital of France?", "Paris is the capital of France."]
inputs = tokenizer(texts, padding=True, truncation=True, max_length=512, return_tensors="pt")
inputs = {k: v.to("npu:0") for k, v in inputs.items()}
with torch.no_grad():
last_hidden = model(**inputs).last_hidden_state
pooled = last_hidden[:, 0] # CLS pooling per 1_Pooling/config.json
embeddings = F.normalize(pooled, p=2, dim=1)
print(embeddings.cpu().numpy())
# cosine
cos = F.cosine_similarity(embeddings[0:1], embeddings[1:1]).item()See inference.py for full synchronized example with device proofs and stable perf.

inference.py - NPU inference with device proof and perfrequirements.txt - dependenciesassets/*.png - xterm.js evidence screenshots