ⓍTTS is a voice generation model that lets you clone voices into different languages using just a quick 6-second audio clip. This repository provides the Ascend NPU adapted version of XTTS-v2, enabling fast inference on Huawei Ascend 910 series NPUs.
This model has been verified to run on Ascend NPU with the following adaptations:
| Component | Adaptation |
|---|---|
| Main GPT Model | Runs natively on NPU |
| HiFi-GAN Decoder | Runs natively on NPU |
| Speaker Encoder | Runs on CPU (NPU lacks complex64 abs() support for STFT) |
| Audio Loading | Uses soundfile backend (avoids torchcodec/libnvrtc) |
weights_only=True default)soundfile instead of torchcodec (libnvrtc.so not available on Ascend)abs() on complex64 tensors from torch.stftEvaluated on Huawei Ascend 910B NPU:
| Language | Text Length | Audio Duration | Inference Time | RTF |
|---|---|---|---|---|
| English (short) | 74 chars | 5.84s | 2.91s | 0.50x |
| English (medium) | 97 chars | 4.68s | 2.55s | 0.55x |
| Chinese (zh-cn) | 20 chars | 4.54s | 2.31s | 0.51x |
| French | 68 chars | 4.64s | 2.41s | 0.52x |
| German | 75 chars | 5.24s | 2.60s | 0.50x |
RTF < 1.0 means the model generates audio faster than real-time (e.g., 0.5x means 2x faster than playback speed).
The NPU adaptation accuracy was validated on deterministic model components:
| Component | Metric | Value | Status |
|---|---|---|---|
| Speaker Embedding | MAE | 7.48e-05 | ✅ |
| Speaker Embedding | Cosine Similarity | 0.999998 | ✅ |
Accuracy: Speaker embedding error < 0.01%, confirming NPU numerical consistency with CPU baseline.
Note: Full audio output comparison is not meaningful for this generative model due to inherent GPT sampling randomness (temperature-based generation produces different audio each run).
# Install PyTorch NPU
pip install torch torch_npu torchaudio
# Install TTS dependencies
pip install TTS soundfile scipy
# Clone this repository
git clone https://atomgit.com/weixin_62994174/XTTS-v2.git
cd XTTS-v2# English TTS with default speaker
python inference.py --text "Hello world, this is a test." --language en
# Chinese TTS
python inference.py --text "你好,这是语音合成测试。" --language zh-cn
# Custom speaker voice cloning
python inference.py --text "Your text here" \
--speaker_wav /path/to/speaker_6sec.wav \
--language en \
--output output.wavfrom inference import load_model, synthesize
model, config = load_model(device="npu")
wav, elapsed, duration, rtf = synthesize(
model, config,
text="Hello, this is Ascend NPU text-to-speech!",
speaker_wav="models/samples/en_sample.wav",
language="en",
output_path="output.wav",
)
print(f"RTF: {rtf:.2f}x")XTTS-v2/
├── inference.py # Main NPU inference script
├── evaluation/
│ ├── evaluate.py # Performance & accuracy evaluation
│ └── accuracy_deterministic.py # Deterministic component validation
├── models/ # Model weights (from ModelScope)
│ ├── model.pth # Main XTTS-v2 checkpoint
│ ├── dvae.pth # Discrete VAE checkpoint
│ ├── config.json # Model configuration
│ ├── vocab.json # Tokenizer vocabulary
│ └── samples/ # Reference speaker samples
├── output/ # Generated audio outputs
├── smoke_test.py # Quick NPU verification test
└── README.md # This fileRun the full evaluation suite:
# Performance benchmark only
python evaluation/evaluate.py --perf-only
# Deterministic accuracy validation
python evaluation/accuracy_deterministic.pytorch.stft operation produces complex tensors; Ascend NPU does not support abs() on complex64 dtype. Speaker encoder runs on CPU as a workaround.soundfile as fallback.This model is licensed under the Coqui Public Model License (CPML).
@misc{xtts-v2,
author = {Coqui AI},
title = {XTTS-v2: Multilingual Voice Cloning},
year = {2024},
publisher = {ModelScope},
url = {https://www.modelscope.cn/models/AI-ModelScope/XTTS-v2}
}