Tencent HunYuan HY-MT1.5 Multilingual Translation Model adapted for Huawei Ascend NPU (via vLLM-Ascend)
This repository provides the complete adaptation of Tencent HunYuan HY-MT1.5-1.8B — a multilingual neural machine translation model — to run on Huawei Ascend NPU (Atlas 800 A2, Ascend 910B) using the vLLM-Ascend inference engine.
| Model | Parameters | Type | Architecture | Status |
|---|---|---|---|---|
| HY-MT1.5-1.8B | 1.79B | Dense | HunYuanDenseV1ForCausalLM | ✅ |
| HY-MT1.5-7B | 7B | MoE | HunYuanMoEV1ForCausalLM | ⚠️ 下载待补全 |
| HY-MT1.5-1.8B-FP8 | 1.79B | Dense FP8 | W8A16暂不支持 | ❌ |
| HY-MT1.5-1.8B-GPTQ-Int4 | 1.79B | Quantized | 暂不支持 | ❌ |
| HY-MT1.5-1.8B-GGUF | 1.79B | GGUF | 暂不支持 | ❌ |
Benchmark results on Ascend 910B NPU (2 devices):
| Metric | Value |
|---|---|
| Peak Throughput (batch=4) | 146.4 tok/s |
| Single-batch Throughput | 71.1 tok/s |
| Time to First Token (TTFT) | 83.1 ms |
| Time per Output Token (TPOT) | 11.4 ms |
| Tokens per Second | 87.7 tok/s |
| Batch 1 Latency (p50) | 0.25 s |
| Batch 4 Latency (p50) | 1.97 s |
CPU vs NPU output comparison shows < 1% error across all test cases:

Ascend 910B NPU produces nearly identical outputs to CPU reference (cosine similarity > 0.9998, 23.4× speedup).
See evaluation/screenshots/ for visual verification.

Interactive translation on Ascend 910B NPU — 5 language pairs in under 1 second total.
pip install vllm-ascend)# Install dependencies
pip install vllm vllm-ascend transformers torch-npu
# Clone model from HuggingFace / ModelScope
pip install modelscope
python3 -c "
from modelscope.hub.snapshot_download import snapshot_download
snapshot_download('Tencent-Hunyuan/HY-MT1.5-1.8B',
cache_dir='/path/to/models/HY-MT1.5-1.8B')
"# Interactive mode
python3 inference.py --model /path/to/HY-MT1.5-1.8B --mode interactive
# Batch mode
python3 inference.py --model /path/to/HY-MT1.5-1.8B --mode batch \
--input test_inputs.jsonl --output results.json
# Benchmark mode
python3 inference.py --model /path/to/HY-MT1.5-1.8B --mode benchmark \
--output benchmark.json
# Evaluation with accuracy comparison
python3 inference.py --model /path/to/HY-MT1.5-1.8B --mode eval \
--output eval_results.json>>> Hello world
[en->zh] (0.25s): 你好,世界
>>> zh:en:你好世界
[zh->en] (0.28s): Hello world
>>> en:ja:Good morning
[en->ja] (0.31s): おはようございますhy-mt-ascend/
├── inference.py # Main inference script
├── readme.md # This file
├── evaluation/
│ ├── eval_accuracy.py # CPU vs NPU accuracy comparison
│ ├── eval_performance.py # Throughput & latency benchmark
│ ├── run_eval.sh # Full evaluation runner
│ ├── logs/
│ │ ├── accuracy.json # Accuracy evaluation results
│ │ ├── performance.json # Performance benchmark results
│ │ └── *.log # Run logs
│ └── screenshots/
│ ├── accuracy_comparison.png # Accuracy comparison screenshot
│ ├── inference_demo.png # Interactive inference demo
│ └── performance_benchmark.png # Performance results screenshot
## Model Details
HY-MT1.5 is a multilingual translation model supporting:
- **Languages**: Chinese, English, Japanese, Korean, French, German, Spanish, Portuguese, Russian, Arabic
- **Context Window**: 8192 tokens
- **Architecture**: Transformer Decoder (HunYuanDenseV1 variant)
- **Training**: Sequence-level knowledge distillation + contrastive learning
## Known Limitations
1. **CPU Inference**: Full model requires ~7GB RAM in BF16; CPU-only inference may fail on low-memory systems
2. **FP8 Models**: vLLM-Ascend does not natively support `compressed-tensors` format FP8 quantization; use the conversion script in `npu_ref/`
3. **GPTQ/GGUF**: Not supported by vLLM-Ascend
4. **Output Quality**: With temperature=0.0 (greedy), the model may occasionally repeat tokens; use temperature=0.7 with top_k=20 for best results
## License
This project is licensed under the Apache License 2.0.
The underlying model (Tencent HunYuan HY-MT1.5) is subject to Tencent's original license terms.
---
*Adapted for Huawei Ascend NPU by @m0_74196153*