模型概述及场景

Co‌syVoice是一款基于语音量化编码的语音生成大模型，能够深度融合文本理解和语音生成，实现自然流畅的语音体验。它通过离散化编码和依托大模型技术，能够精准解析并诠释各类文本内容，将其转化为宛如真人般的自然语音‌。CosyVoice2在原始1的基础上，把QWEN2模型接入CosyVoice的LLM部分，实现了推理加速

准备运行环境

硬件环境

arm服务器

设备型号	NPU配置
Atlas 800I A2 910B	1卡

软件版本

软件名	版本
CANN	8.1.RC1.alpha001
Python	3.11.6
torch	2.4.0
torch-npu	2.4.0.post2
ais-bench	0.0.2

模型规格

	链接	Commit ID
模型权重	https://modelscope.cn/models/iic/CosyVoice2-0.5B	9bd5b08
模型源码	https://github.com/FunAudioLLM/CosyVoice	fd45708
昇腾优化	https://gitcode.com/Ascend/ModelZoo-PyTorch/commits/master/ACL_PyTorch/built-in/audio/CosyVoice2	20cdf863

运行指导

使用以下命令导入镜像:

docker load -i my_ascend_cann_8_1_rc1_cosyvoice2_openeuler_v1.tar.gz

新建工作目录:

mkdir -p /home/mind/model/

将weight_light.tar.gz和CosyVoice.tar.gz放到工作目录/home/mind/model/并解压:

tar -zxvf weight_light.tar.gz
tar -zxvf CosyVoice.tar.gz

使用以下命令启动docker容器:

docker run -itd --name demo_cosy --privileged=true --device=/dev/davinci_manager:/dev/davinci_manager --device=/dev/devmm_svm:/dev/devmm_svm --device=/dev/hisi_hdc:/dev/hisi_hdc -v /usr/local/dcmi:/usr/local/dcmi -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi -v /usr/local/Ascend/driver/lib64/common:/usr/local/Ascend/driver/lib64/common -v /usr/local/Ascend/driver/lib64/driver:/usr/local/Ascend/driver/lib64/driver -v /etc/ascend_install.info:/etc/ascend_install.info -v /etc/vnpu.cfg:/etc/vnpu.cfg -v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info -v /home/mind/model:/home/mind/model ascend_cann_8_1_rc1_cosyvoice2_openeuler:v1.0 /bin/bash

进入容器:

docker exec -it demo_cosy bash

进入工作目录:

cd /home/mind/model/

此时会看到weight和CosyVoice目录，进入CosyVoice目录，新建run.sh脚本，填入以下内容：

# 指定使用NPU ID，默认为0
export ASCEND_RT_VISIBLE_DEVICES=3
export PYTHONPATH=third_party/Matcha-TTS:$PYTHONPATH
export PYTHONPATH=transformers/src:$PYTHONPATH

# 使能环境变量
source /usr/local/Ascend/ascend-toolkit/set_env.sh
# 规避找不到ttsfrd
export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH
# 规避找不到cstdint
export CPLUS_INCLUDE_PATH=/usr/local/Ascend/ascend-toolkit/8.1.RC1.alpha001/toolkit/toolchain/hcc/aarch64-target-linux-gnu/include/c++/7.3.0:${CPLUS_INCLUDE_PATH}
export CPLUS_INCLUDE_PATH=/usr/local/Ascend/ascend-toolkit/8.1.RC1.alpha001/toolkit/toolchain/hcc/aarch64-target-linux-gnu/include/c++/7.3.0/aarch64-target-linux-gnu:${CPLUS_INCLUDE_PATH}
export CPLUS_INCLUDE_PATH=/usr/local/Ascend/ascend-toolkit/8.1.RC1.alpha001/toolkit/toolchain/hcc/aarch64-target-linux-gnu/sys-include:${CPLUS_INCLUDE_PATH}

# 清理modelscope缓存
rm -rf ~/.cache/modelscope/

python3 infer.py --model_path=../weight/CosyVoice2-0.5B --stream

然后执行bash run.sh开始推理任务

常见问题