Ascend-SACT/MOSS-Transcribe-Diarize
Model Introduction
Files & versions
Pull Requests
Discussion
Analysis

1. 模型概述及场景

MOSS-Transcribe-Diarize 0.9B 是一个端到端的音频理解模型,用于长音频多说话人的转录、说话人分割(diarization)、时间戳标注以及声学事件感知。该模型支持 50 多种语言的转录与说话人分割,可对长达 90 分钟的音频进行单次推理,并支持通过自定义热词提示来处理特定领域的术语。给定一个音频或视频文件,该模型可在一次推理中生成紧凑的、区分说话人的转录文本,包含时间戳和匿名说话人标签,例如 [S01]、[S02] 等。

详细信息参见:https://www.modelscope.cn/models/openmoss/MOSS-Transcribe-Diarize

2. 准备运行环境

2.1 版本配套表

硬件版本

组件版本
硬件环境910B(2卡)
cann 驱动25.0.rc1.1

3. 运行指导

3.1 下载镜像,并启动容器

本模型运行在 vllm-ascend v0.26.0rc版本下,具体容器下载部署参见:https://github.com/vllm-project/vllm-ascend/blob/releases/v0.26.0rc/docs/source/quick_start.md

3.2 安装补丁

cd /vllm-workspace/vllm-ascend
if git apply --check "$PATCH_DIR/pa_workspace_reuse_extkey_vllm_ascend.patch"; then
    git apply "$PATCH_DIR/pa_workspace_reuse_extkey_vllm_ascend.patch" && echo "应用 pa_workspace_reuse_extkey_vllm_ascend 成功"
else
    echo "检查失败,跳过应用(补丁可能已应用或无法合并)"
    exit 1
fi

3.3 下载模型

在下载前,请先通过如下命令安装ModelScope 命令:pip install modelscope

下载模型 命令:modelscope download --model openmoss/MOSS-Transcribe-Diarize

3.4 设置环境变量

export SERVE_PORT=8010
export SERVE_NAME="MOSS-Transcribe-Diarize"

3.5 启动模型服务

找到本模型附带文件:vllm_boot.sh,执行命令

命令:./vllm_boot.sh

附-启动脚本如下:

#!/bin/bash

if [ -z "${LWS_WORKER_INDEX}" ]; then
    echo "需要设置环境变量: export LWS_WORKER_INDEX=    # 0: 表示Master节点, 1: 表示Worker节点"   
    echo "需要设置环境变量: export LWS_LEADER_ADDRESS=  # Master节点IP地址, 获取方法: hostname -i"
    echo "环境变量参考样例: export LWS_LEADER_ADDRESS=`(hostname -i)`; export LWS_WORKER_INDEX=0"
    exit 1
fi

SERVE_PORT=${PORT}                    # 服务端口,可根据情况修改
SERVE_NAME="MOSS-Transcribe-Diarize"  # 服务模型名称,可根据情况修改
source /opt/mamba/etc/profile.d/conda.sh
conda activate ascend-infer
source /usr/local/Ascend/ascend-toolkit/set_env.sh
source /usr/local/Ascend/nnal/atb/set_env.sh
export GLOO_SOCKET_IFNAME=lo

# 自动检测本机IP和网络接口
LOCAL_IP=$(hostname -I | awk '{print $1}')
NIC_NAME=$(route -n 2>/dev/null | grep '^0.0.0.0' | awk '{print $8}' | head -1)
if [ -z "$NIC_NAME" ]; then
    NIC_NAME=$(ls /sys/class/net/ | grep -v lo | head -1)
fi
export HCCL_IF_IP=${LOCAL_IP}
export GLOO_SOCKET_IFNAME=${NIC_NAME}
export TP_SOCKET_IFNAME=${NIC_NAME}
export HCCL_SOCKET_IFNAME=${NIC_NAME}
MASTER_IP="${LWS_LEADER_ADDRESS}"
MASTER_PORT=12890 

# 日志文件名
# LOG_FILE=/dev/null
LOG_FILE="/inspire/sj-ssd3/global_user/panqihan-W25083/lxl/models/MOSS-Transcribe-Diarize/hc_log/log_vllm_$(date +%Y%m%d_%H%M%S)_idx${LWS_WORKER_INDEX}_${LOCAL_IP##*.}.log"

{
    echo "================================================================================"
    echo "环境变量配置信息"
} 2>&1 | tee -a ${LOG_FILE}

{
    set -x 

    export VLLM_MAX_AUDIO_CLIP_FILESIZE_MB=600
    export VLLM_MAX_AUDIO_DECODE_DURATION_S=2400
    
    set +x   
} > >(grep 'export ' | tee -a ${LOG_FILE}) 2>&1  
 
# ========== Profiler 输出目录 ==========
PROF_OUTPUT_DIR="./torch_prof_$(date +%Y%m%d_%H%M%S)"

# ========== 公共启动参数(只需维护这一个地方)==========
COMMON_ARGS=(
    --host 0.0.0.0
    --port ${SERVE_PORT}
    --served-model-name ${SERVE_NAME}
    --trust-remote-code
    --max-model-len 131072
    --max-num-batched-tokens 65536   
    --async-scheduling 
    --additional-config '{"pa_shape_list":[1,2,4,8,16,24,32,40,48,56,64,72,80,88,96,104,112,120,128,136,144,152,160,168,176,184,192,200,208,216,224,232,240,248,256]}' 
    --compilation-config '{"cudagraph_mode": "FULL_DECODE_ONLY"}'
)

# ========== 根据角色添加特定参数 ==========
{
    echo "================================================================================"
    echo "模型环境配置信息"
    echo "log  file:  ${LOG_FILE}"
    echo "date time:  $(date)"
    echo "host addr:  ${LOCAL_IP}"
    echo "MODEL_PATH: ${MODEL_PATH}"
    echo "SERVE_PORT: ${SERVE_PORT}"
    echo "SERVE_NAME: ${SERVE_NAME}"
    echo "LWS_WORKER_INDEX: ${LWS_WORKER_INDEX}"
    echo "LWS_LEADER_ADDRESS: ${LWS_LEADER_ADDRESS}"
    echo "MASTER_IP: ${MASTER_IP}"
    echo "MASTER_PORT: ${MASTER_PORT}"
    echo "PROF_OUTPUT_DIR: ${PROF_OUTPUT_DIR}"
    echo "================================================================================"

    if [ "${LWS_WORKER_INDEX}" -eq 0 ]; then
        # Master节点
        echo "启动命令参数信息(Master)"
        echo "vllm serve ${MODEL_PATH} ${COMMON_ARGS[@]}"  | sed 's/ --/ \\\n  --/g'
        echo "================================================================================"

        vllm serve ${MODEL_PATH} "${COMMON_ARGS[@]}"
    else
        # Worker节点 - 添加额外参数
        echo "启动命令参数信息(Worker)"
        echo "vllm serve ${MODEL_PATH} ${COMMON_ARGS[*]} --headless --data-parallel-start-rank ${LWS_WORKER_INDEX}"  | sed 's/ --/ \\\n  --/g'
        echo "================================================================================"

        vllm serve ${MODEL_PATH} "${COMMON_ARGS[@]}" \
            --headless \
            --data-parallel-start-rank ${LWS_WORKER_INDEX}
    fi
} 2>&1 | tee -a ${LOG_FILE}

3.6 测试模型服务


LOCAL_IP=$(hostname -I | awk '{print $1}')
LOG_FILE="log_func_$(date +%Y%m%d_%H%M%S)_idx${LWS_WORKER_INDEX:-0}_${LOCAL_IP##*.}.log"

URL="http://localhost:${SERVE_PORT}/v1/audio/transcriptions"
AUDIO_FILE="${HOME_PATH}/models/MOSS-Transcribe-Diarize/assets/alimeeting/Eval_Ali/Eval_Ali_near/audio_dir/R8001_M8004_N_SPK8013.wav"

curl $URL \
  -F model="$SERVE_NAME" \
  -F file=@"$AUDIO_FILE" \
  -F response_format="json" \
  -F temperature="0" \
  2> >(tee -a $LOG_FILE >&2) \
  | jq '.' | tee -a $LOG_FILE