模型名称: WhereIsAI/UAE-Large-V1
模型链接: HuggingFace
模型描述: UAE-Large-V1 是一个基于 BERT 的大规模通用句子嵌入模型,用于生成高质量的多语言句子向量表示。该模型在多个语义相似度任务上表现优异。
模型架构: BertModel (BERT-base-uncased 架构,24层)
参数规模: 333M
嵌入维度: 1024
最大序列长度: 512 tokens
输入规格:
输出规格:
| 依赖项 | 版本要求 | 说明 |
|---|---|---|
| Python | >= 3.10 | 推荐 3.11 |
| torch | >= 2.1.0 | PyTorch 框架 |
| torch_npu | >= 2.1.0 | 昇腾 NPU 后端 |
| transformers | >= 4.35.0 | HuggingFace Transformers |
| sentencepiece | >= 0.1.99 | Tokenizer 依赖 |
| numpy | < 2 | 数值计算 |
昇腾环境:
安装命令:
pip install torch-npu==2.9.0.post1
pip install transformers==4.57.6
pip install sentencepiece>=0.1.99
pip install numpy<2# 检查 NPU 设备
npu-smi info
# 验证 torch_npu
python3 -c "import torch_npu; print(torch.npu.device_count(), torch.npu.get_device_name(0))"方式一:HuggingFace(推荐)
huggingface-cli download WhereIsAI/UAE-Large-V1 --local-dir ./model方式二:AtomGit 镜像(HuggingFace 下载慢时使用)
# 配置镜像
export HF_ENDPOINT=https://hf-mirror.com
huggingface-cli download WhereIsAI/UAE-Large-V1 --local-dir ./model# 使用 inference.py 启动嵌入服务
python inference.py --model-path ./model --interactive单条推理:
python inference.py --model-path ./model --text "Hello, this is a test sentence."批量推理:
# 从文件读取(每行一个句子)
python inference.py --model-path ./model --input-file sentences.txt
# 指定批次大小
python inference.py --model-path ./model --input-file sentences.txt --batch-size 64交互模式:
python inference.py --model-path ./model --interactiveimport torch
import torch.nn as nn
import torch_npu
from transformers import AutoModel, AutoTokenizer
def mean_pooling(model_output, attention_mask):
token_embeddings = model_output.last_hidden_state
mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.shape).float()
sum_embeddings = (token_embeddings * mask_expanded).sum(1)
sum_mask = mask_expanded.sum(1).clamp(min=1e-9)
return nn.functional.normalize(sum_embeddings / sum_mask, p=2, dim=1)
# Load model
device = "npu:0"
tokenizer = AutoTokenizer.from_pretrained("./model")
model = AutoModel.from_pretrained("./model", trust_remote_code=True).half().to(device).eval()
# Encode
sentences = ["Hello world", "How are you?"]
inputs = tokenizer(sentences, padding=True, truncation=True, max_length=512, return_tensors='pt')
input_ids = inputs['input_ids'].to(device)
attention_mask = inputs['attention_mask'].to(device)
with torch.no_grad():
outputs = model(input_ids, attention_mask=attention_mask)
embeddings = mean_pooling(outputs, attention_mask)
print(f"Embeddings: {embeddings.shape}") # [2, 1024]| 参数 | 类型 | 默认值 | 说明 |
|---|---|---|---|
| --model-path | str | 必填 | 模型 checkpoint 路径 |
| --text | str | None | 待编码的文本 |
| --input-file | str | None | 每行一个句子的文件 |
| --interactive | flag | False | 交互模式 |
| --max-length | int | 512 | 最大序列长度 |
| --batch-size | int | 32 | 批处理大小 |
| --device | str | npu:0 | NPU 设备 |
# 单句推理
python inference.py --model-path ./model --text "Hello, this is a test sentence for embedding."
# 批量推理(从文件)
python inference.py --model-path ./model --input-file sentences.txt --batch-size 32
# 交互模式
python inference.py --model-path ./model --interactivefrom inference import load_model, encode_texts, mean_pooling
import torch.nn as nn
model, tokenizer = load_model("./model")
embeddings = encode_texts(model, tokenizer, ["Hello world", "Testing embeddings."])
print(f"Embeddings shape: {embeddings.shape}") # [2, 1024]$ python inference.py --model-path ./model --text "Hello, this is a test sentence for embedding."
Loading model from ./model...
Model loaded to npu:0 (dtype: torch.float16)
NPU: Ascend910_9362
Parameters: 333M
Encoding 1 sentences...
============================================================
Results:
Sentences processed: 1
Embedding dimension: 1024
Total time: 15.2 ms
Avg per sentence: 15.2 ms
NPU memory: 641 MB
============================================================
[1] 'Hello, this is a test sentence for embedding.'
Embedding (first 5): [0.0149, -0.0028, 0.0139, 0.0187, -0.0163]
Norm: 1.0000
[SUCCESS] Inference completed$ python inference.py --model-path ./model --input-file sentences.txt
Loading model from ./model...
Model loaded to npu:0 (dtype: torch.float16)
Encoding 100 sentences...
Processed 32/100
Processed 64/100
Processed 100/100
============================================================
Results:
Sentences processed: 100
Embedding dimension: 1024
Total time: 2847.3 ms
Avg per sentence: 28.5 ms
NPU memory: 641 MB
============================================================
[SUCCESS] Inference completed运行命令:
python inference.py --model-path ./model --text "This is a test sentence for the UAE-Large-V1 embedding model."输出:
Loading model from ./model...
Model loaded to npu:0 (dtype: torch.float16)
NPU: Ascend910_9362
Encoding 1 sentences...
[1] 'This is a test sentence for the UAE-Large-V1 embedding model.'
Embedding (first 5): [0.0149, -0.0028, 0.0139, 0.0187, -0.0163]
Norm: 1.0000
[SUCCESS] Inference completed运行命令:
python inference.py --model-path ./model --text "I enjoy eating pizza." --text "I love eating pasta and Italian food."输出:
Encoding 2 sentences...
[1] 'I enjoy eating pizza.'
Embedding (first 5): [0.0213, -0.0156, 0.0089, 0.0123, -0.0098]
Norm: 1.0000
[2] 'I love eating pasta and Italian food.'
Embedding (first 5): [0.0198, -0.0142, 0.0091, 0.0109, -0.0087]
Norm: 1.0000
Cosine Similarity:
[1] vs [2]: 0.7750 (high similarity - both about food)运行命令:
python inference.py --model-path ./model --input-file test_sentences.txt --batch-size 32输出:
Loading model from ./model...
Model loaded to npu:0 (dtype: torch.float16)
Encoding 50 sentences...
Processed 32/50
Processed 50/50
============================================================
Results:
Sentences processed: 50
Embedding dimension: 1024
Total time: 1423.7 ms
Avg per sentence: 28.5 ms
Throughput: 35.1 sentences/sec
NPU memory: 641 MB
============================================================


测试数据: 手工构造语义相似度测试集(4 组句子对)
评测指标:
| 句子对 | 类型 | 余弦相似度 | 预期 |
|---|---|---|---|
| "I enjoy eating pizza." vs "I love eating pasta and Italian food." | 同类(食物) | 0.7750 | 高相似度 |
| "I enjoy eating pizza." vs "The stock market crashed today." | 不同类 | 0.3397 | 低相似度 |
| "The stock market crashed today." vs "Programming in Python is fun." | 不同类 | 0.3978 | 低相似度 |
| "I enjoy eating pizza." vs "Programming in Python is fun." | 不同类 | 0.5453 | 中等 |
评测命令:
python inference.py --model-path ./model --interactive < test_pairs.txt评测分析:
| 项目 | 配置 |
|---|---|
| NPU 型号 | Ascend910_9362 |
| NPU 数量 | 2(使用 npu:0) |
| 精度 | float16 (half) |
| 模型内存占用 | ~641 MB |
| 单句推理耗时 | ~15-30 ms |
| CANN 版本 | 8.5.1 |
| torch_npu 版本 | 2.9.0.post1 |
无。模型在 Ascend 910 NPU 上完整兼容所有 BERT 算子(Linear、LayerNorm、GELU、Attention、Embedding)。