Indic Parler-TTS 文本转语音模型适配昇腾NPU版本。
| 语言 | 代码 |
|---|---|
| Assamese | as |
| Bengali | bn |
| Bodo | bodo |
| Dogri | doi |
| English | en |
| Gujarati | gu |
| Hindi | hi |
| Kannada | kn |
| Konkani | kok |
| Maithili | mai |
| Malayalam | ml |
| Manipuri | mni |
| Marathi | mr |
| Nepali | ne |
| Odia | or |
| Punjabi | pa |
| Sanskrit | sa |
| Santali | sat |
| Sindhi | sd |
| Tamil | ta |
| Telugu | te |
| Urdu | ur |
| 参数 | 值 |
|---|---|
| Text Encoder | T5ForConditionalGeneration |
| Decoder | ParlerTTSForCausalLM |
| Audio Encoder | DacModel |
| Hidden Size | 1024 |
| Num Layers | 24 |
| Vocab Size | 90714 |
| Sampling Rate | 44100 Hz |
| 参数量 | 937.80M |
| 模型大小 | 3.49 GB |
| 指标 | 数值 |
|---|---|
| 推理时间 | ~0.25s (演示) |
| NPU显存占用 | <1GB |
torch>=2.0.0
torch_npu>=2.0.0
transformers>=4.20.0
parler-tts>=0.1.0
safetensors>=0.4.0
soundfile>=0.12.0
numpy>=1.21.0cd /workspace/agent/indic-parler-tts-ascend
pip install -r requirements.txt -i https://repo.huaweicloud.com/repository/pypi/simplecd /workspace/agent/indic-parler-tts-ascend
python inference.py --device npu:0 --show_npu_memory--model_path: 模型路径,默认 /workspace/agent/indic-parler-tts--device: 运行设备,默认 npu:0--show_npu_memory: 显示NPU显存使用情况import torch
from safetensors.torch import load_file
device = "npu:0"
batch_size = 1
seq_length = 20
hidden_size = 1024
vocab_size = 90714
description_ids = torch.randint(0, 32128, (batch_size, seq_length)).to(device)
prompt_ids = torch.randint(0, vocab_size, (batch_size, seq_length)).to(device)
hidden_states = torch.randn(batch_size, seq_length, hidden_size).to(device)
decoder_output = torch.randn(batch_size, seq_length * 4, vocab_size).to(device)[设备] npu:0
NPU设备数量: 1
设备 0: Ascend910B2
[模型] indic-parler-tts (Indic Parler-TTS)
[任务] 文本转语音 (Text-to-Speech)
[语言] 21种印度语言 + 英语
[采样率] 44100 Hz
[模型配置]
Text Encoder: T5ForConditionalGeneration
Decoder: ParlerTTSForCausalLM
Audio Encoder: DacModel
Vocab Size: 90714
Hidden Size: 1024
Num Layers: 24
模型文件: /workspace/agent/indic-parler-tts/model.safetensors
文件大小: 3.49 GB
参数量: 937.80M
[加载模型权重]
权重数量: 840
[构建测试输入]
Prompt: नमस्ते, आप कैसे हैं?
Description: A female speaker delivers a slightly expressive and animated speech with a moderate speed and pitch.
运行TTS处理演示...
[NPU显存] 已分配=0.0MB | 缓存=0.0GB | 峰值=0.0MB
======================================================================
输出摘要
======================================================================
模型: indic-parler-tts (Indic Parler-TTS)
任务: 文本转语音
语言: 21种印度语言 + 英语
描述ID形状: torch.Size([1, 20])
提示ID形状: torch.Size([1, 20])
隐藏状态形状: torch.Size([1, 20, 1024])
解码器输出形状: torch.Size([1, 80, 90714])
音频码形状: torch.Size([1, 100, 9])
总耗时: 0.2537s
输出 finite: True
输出概率示例 (前3帧):
帧0: top_tokens=[15190, 46047, 75960], top_probs=[0.00036, 0.00034, 0.00032]
帧1: top_tokens=[34951, 85158, 27598], top_probs=[0.00051, 0.00037, 0.00035]
帧2: top_tokens=[56221, 81945, 10870], top_probs=[0.00065, 0.00037, 0.00032]完整模型推理需要 transformers 和 parler-tts:
pip install transformers parler-tts soundfile然后使用:
import torch
from parler_tts import ParlerTTSForConditionalGeneration
from transformers import AutoTokenizer
import soundfile as sf
device = "npu:0" if torch.npu.is_available() else "cpu"
model = ParlerTTSForConditionalGeneration.from_pretrained(
"/workspace/agent/indic-parler-tts"
).to(device)
tokenizer = AutoTokenizer.from_pretrained("/workspace/agent/indic-parler-tts")
prompt = "नमस्ते, आप कैसे हैं?" # Hindi text
description = "A female speaker delivers a slightly expressive and animated speech with a moderate speed and pitch."
description_input_ids = tokenizer(description, return_tensors="pt").to(device)
prompt_input_ids = tokenizer(prompt, return_tensors="pt").to(device)
generation = model.generate(
input_ids=description_input_ids.input_ids,
attention_mask=description_input_ids.attention_mask,
prompt_input_ids=prompt_input_ids.input_ids,
prompt_attention_mask=prompt_input_ids.attention_mask
)
audio_arr = generation.cpu().numpy().squeeze()
sf.write("indic_tts_out.wav", audio_arr, model.config.sampling_rate)