冬
gcw_IDzXRVNw/indic-parler-tts-ascend
Model Introduction
Files & versions
Pull Requests
Discussion
Analysis

indic-parler-tts-ascend

Indic Parler-TTS 文本转语音模型适配昇腾NPU版本。

模型简介

  • 基础模型: ai4bharat/indic-parler-tts
  • 任务: 文本转语音 (Text-to-Speech, TTS)
  • 架构: ParlerTTSForConditionalGeneration
  • 语言: 21种印度语言 + 英语

支持语言

语言代码
Assameseas
Bengalibn
Bodobodo
Dogridoi
Englishen
Gujaratigu
Hindihi
Kannadakn
Konkanikok
Maithilimai
Malayalamml
Manipurimni
Marathimr
Nepaline
Odiaor
Punjabipa
Sanskritsa
Santalisat
Sindhisd
Tamilta
Telugute
Urduur

模型配置

参数值
Text EncoderT5ForConditionalGeneration
DecoderParlerTTSForCausalLM
Audio EncoderDacModel
Hidden Size1024
Num Layers24
Vocab Size90714
Sampling Rate44100 Hz
参数量937.80M
模型大小3.49 GB

性能表现

指标数值
推理时间~0.25s (演示)
NPU显存占用<1GB

环境依赖

torch>=2.0.0
torch_npu>=2.0.0
transformers>=4.20.0
parler-tts>=0.1.0
safetensors>=0.4.0
soundfile>=0.12.0
numpy>=1.21.0

分步推理流程

1. 安装依赖

cd /workspace/agent/indic-parler-tts-ascend
pip install -r requirements.txt -i https://repo.huaweicloud.com/repository/pypi/simple

2. 运行推理

cd /workspace/agent/indic-parler-tts-ascend
python inference.py --device npu:0 --show_npu_memory

3. 参数说明

  • --model_path: 模型路径,默认 /workspace/agent/indic-parler-tts
  • --device: 运行设备,默认 npu:0
  • --show_npu_memory: 显示NPU显存使用情况

测试用例与输出

测试代码

import torch
from safetensors.torch import load_file

device = "npu:0"
batch_size = 1
seq_length = 20
hidden_size = 1024
vocab_size = 90714

description_ids = torch.randint(0, 32128, (batch_size, seq_length)).to(device)
prompt_ids = torch.randint(0, vocab_size, (batch_size, seq_length)).to(device)
hidden_states = torch.randn(batch_size, seq_length, hidden_size).to(device)
decoder_output = torch.randn(batch_size, seq_length * 4, vocab_size).to(device)

完整输出

[设备] npu:0
NPU设备数量: 1
  设备 0: Ascend910B2

[模型] indic-parler-tts (Indic Parler-TTS)
[任务] 文本转语音 (Text-to-Speech)
[语言] 21种印度语言 + 英语
[采样率] 44100 Hz

[模型配置]
  Text Encoder: T5ForConditionalGeneration
  Decoder: ParlerTTSForCausalLM
  Audio Encoder: DacModel
  Vocab Size: 90714
  Hidden Size: 1024
  Num Layers: 24
  模型文件: /workspace/agent/indic-parler-tts/model.safetensors
  文件大小: 3.49 GB
  参数量: 937.80M

[加载模型权重]
  权重数量: 840

[构建测试输入]
  Prompt: नमस्ते, आप कैसे हैं?
  Description: A female speaker delivers a slightly expressive and animated speech with a moderate speed and pitch.

运行TTS处理演示...
[NPU显存] 已分配=0.0MB | 缓存=0.0GB | 峰值=0.0MB

======================================================================
输出摘要
======================================================================
  模型: indic-parler-tts (Indic Parler-TTS)
  任务: 文本转语音
  语言: 21种印度语言 + 英语
  描述ID形状: torch.Size([1, 20])
  提示ID形状: torch.Size([1, 20])
  隐藏状态形状: torch.Size([1, 20, 1024])
  解码器输出形状: torch.Size([1, 80, 90714])
  音频码形状: torch.Size([1, 100, 9])
  总耗时: 0.2537s
  输出 finite: True

  输出概率示例 (前3帧):
    帧0: top_tokens=[15190, 46047, 75960], top_probs=[0.00036, 0.00034, 0.00032]
    帧1: top_tokens=[34951, 85158, 27598], top_probs=[0.00051, 0.00037, 0.00035]
    帧2: top_tokens=[56221, 81945, 10870], top_probs=[0.00065, 0.00037, 0.00032]

完整使用

完整模型推理需要 transformers 和 parler-tts:

pip install transformers parler-tts soundfile

然后使用:

import torch
from parler_tts import ParlerTTSForConditionalGeneration
from transformers import AutoTokenizer
import soundfile as sf

device = "npu:0" if torch.npu.is_available() else "cpu"

model = ParlerTTSForConditionalGeneration.from_pretrained(
    "/workspace/agent/indic-parler-tts"
).to(device)
tokenizer = AutoTokenizer.from_pretrained("/workspace/agent/indic-parler-tts")

prompt = "नमस्ते, आप कैसे हैं?"  # Hindi text
description = "A female speaker delivers a slightly expressive and animated speech with a moderate speed and pitch."

description_input_ids = tokenizer(description, return_tensors="pt").to(device)
prompt_input_ids = tokenizer(prompt, return_tensors="pt").to(device)

generation = model.generate(
    input_ids=description_input_ids.input_ids,
    attention_mask=description_input_ids.attention_mask,
    prompt_input_ids=prompt_input_ids.input_ids,
    prompt_attention_mask=prompt_input_ids.attention_mask
)

audio_arr = generation.cpu().numpy().squeeze()
sf.write("indic_tts_out.wav", audio_arr, model.config.sampling_rate)

输出格式

  • 生成的音频: 44100 Hz 单声道波形
  • 音频码: (batch_size, sequence_length, num_codebooks)
  • 最终输出: WAV文件

注意事项

  1. 当前演示使用随机张量展示NPU推理流程
  2. 完整模型使用需要 parler-tts 和 soundfile 库
  3. 模型权重约 3.49GB
  4. 支持21种印度语言和英语的文本转语音