HuggingFace镜像/Step-Audio-TTS-3B
Model Introduction
Files & versions
Analysis

Step-Audio-TTS-3B

Step-Audio-TTS-3B represents the industry's first Text-to-Speech (TTS) model trained on a large-scale synthetic dataset utilizing the LLM-Chat paradigm. It has achieved SOTA Character Error Rate (CER) results on the SEED TTS Eval benchmark. The model supports multiple languages, a variety of emotional expressions, and diverse voice style controls. Notably, Step-Audio-TTS-3B is also the first TTS model in the industry capable of generating RAP and Humming, marking a significant advancement in the field of speech synthesis.

This repository provides the model weights for StepAudio-TTS-3B, which is a dual-codebook trained LLM (Large Language Model) for text-to-speech synthesis. Additionally, it includes a vocoder trained using the dual-codebook approach, as well as a specialized vocoder specifically optimized for humming generation. These resources collectively enable high-quality speech synthesis and humming capabilities, leveraging the advanced dual-codebook training methodology.

Performance comparison of content consistency (CER/WER) between GLM-4-Voice and MinMo.

Modeltest-zhtest-en
CER (%) ↓WER (%) ↓
GLM-4-Voice2.192.91
MinMo2.482.90
Step-Audio1.532.71

Results of TTS Models on SEED Test Sets.

  • StepAudio-TTS-3B-Single denotes dual-codebook backbone with single-codebook vocoder*
Modeltest-zhtest-en
CER (%) ↓SS ↑WER (%) ↓SS ↑
FireRedTTS1.510.6303.820.460
MaskGCT2.270.7742.620.774
CosyVoice3.630.7754.290.699
CosyVoice 21.450.8062.570.736
CosyVoice 2-S1.450.8122.380.743
Step-Audio-TTS-3B-Single1.370.8022.520.704
Step-Audio-TTS-3B1.310.7332.310.660
Step-Audio-TTS1.170.732.00.660

Performance comparison of Dual-codebook Resynthesis with Cosyvoice.

Tokentest-zhtest-en
CER (%) ↓SS ↑WER (%) ↓SS ↑
Groundtruth0.972-2.156-
CosyVoice2.8570.8494.5190.807
Step-Audio-TTS-3B2.1920.7843.5850.742

More information

For more information, please refer to our repository: Step-Audio.