supplychain-xin1:用户可快速在 NPU 环境部署 Qwen 系列大模型及 BGE、ASR 等模型。支持容器化部署，提供环境初始化、模型拉起及资源监控工具，适配 vllm 等框架，便于多模型高效运行。【此简介由AI生成】

模型（Qwen2.5-14B & Qwen3.5-2B/4B/9B & BGE & ASR）N合1 镜像使用指南

0. 容器启动参考

使用如下命令启动容器

docker run -it -d --net=host --shm-size=20g --privileged=true \
    --name supplychain-xin1 \
    --device=/dev/davinci_manager \
    --device=/dev/hisi_hdc \
    --device=/dev/devmm_svm \
    --device=/dev/davinci0 \
    --device=/dev/davinci1 \
    --device=/dev/davinci2 \
    --device=/dev/davinci3 \
    --device=/dev/davinci4 \
    --device=/dev/davinci5 \
    --device=/dev/davinci6 \
    --device=/dev/davinci7 \
    -v /usr/local/Ascend/driver:/usr/local/Ascend/driver:ro \
    -v /usr/local/sbin:/usr/local/sbin:ro \
    -v /usr/local/sbin/npu-smi:/usr/local/sbin/npu-smi \
    -v /usr/local/Ascend/add-ons/:/usr/local/Ascend/add-ons \
    -v /home/model_weights:/workdir/model_weights \
    -v /home/llg:/home/llg \
    supplychain-xin1:v1.3 \
bash

1. 初始化环境

初始化conda环境 使用如下命令初始化conda环境

source $HOME/miniconda/etc/profile.d/conda.sh

预置conda环境列表

可以conda env list查看，结果如下

vllm-0.17.0rc1
asr-and-bge

激活conda环境

conda activate asr-and-bge

退出conda环境

conda deactivate

2. 拉起Qwen2.5-14B模型

前置条件 使用命令 conda activate vllm-0.17.0rc1初始化vllm-0.17.0rc1环境。

拉起命令

export VLLM_USE_MODELSCOPE=True
export PYTORCH_NPU_ALLOC_CONF=max_split_size_mb:256
export VLLM_USE_V1=1
export TASK_QUEUE_ENABLE=1
export CPU_AFFINITY_CONF=1
export OMP_PROC_BIND=false
export OMP_NUM_THREADS=10
export HCCL_OP_EXPANSION_MODE="AIV"
export VLLM_ASCEND_ENABLE_TOPK_OPTIMIZE=1
export VLLM_ASCEND_ENABLE_DENSE_OPTIMIZE=1
export VLLM_ASCEND_ENABLE_PREFETCH_MLP=1

vllm serve /workdir/model_weights/Qwen2.5-14B-Instruct/ \
    --served-model-name qwen2.5-14B \
    --max_model_len 2048 \
    --max-num-batched-tokens 20480 \
    --tensor-parallel-size 2 \
    --gpu-memory-utilization 0.9 \
    --port 8714 \
    --compilation-config '{"cudagraph_mode": "FULL_DECODE_ONLY"}'

3. 拉起Qwen3.5-2B模型

前置条件 使用命令 conda activate vllm-0.17.0rc1初始化vllm-0.17.0rc1环境。

拉起命令

export PYTORCH_NPU_ALLOC_CONF="expandable_segments:True"
export HCCL_OP_EXPANSION_MODE="AIV"
export HCCL_BUFFSIZE=1024
export OMP_NUM_THREADS=1
export TASK_QUEUE_ENABLE=1

vllm serve /workdir/model_weights/Qwen3.5-2B \
    --served-model-name qwen35-2b \
    --host 0.0.0.0 \
    --port 8352 \
    --data-parallel-size 1 \
    --tensor-parallel-size 1 \
    --max-model-len 71680  \
    --max-num-batched-tokens 16384 \
    --max-num-seqs 32 \
    --gpu-memory-utilization 0.9 \
    --trust-remote-code \
    --reasoning-parser qwen3 \
    --enable-auto-tool-choice \
    --tool-call-parser qwen3_coder \
    --async-scheduling \
    --allowed-local-media-path / \
    --mm_processor_cache_type="shm" \
    --skip-mm-profiling \
    --compilation-config '{"cudagraph_capture_sizes":[1,4,8,12,16,24,32,48,64,128,256], "cudagraph_mode":"FULL_DECODE_ONLY"}' \
    --speculative_config '{"method": "qwen3_5_mtp", "num_speculative_tokens": 3}'

4. 拉起Qwen3.5-4B模型

前置条件 使用命令 conda activate vllm-0.17.0rc1初始化vllm-0.17.0rc1环境。

拉起命令

export PYTORCH_NPU_ALLOC_CONF="expandable_segments:True"
export HCCL_OP_EXPANSION_MODE="AIV"
export HCCL_BUFFSIZE=1024
export OMP_NUM_THREADS=1
export TASK_QUEUE_ENABLE=1


vllm serve /workdir/model_weights/Qwen3.5-4B \
    --served-model-name qwen35-4b \
    --host 0.0.0.0 \
    --port 8354 \
    --data-parallel-size 1 \
    --tensor-parallel-size 1 \
    --max-model-len 71680  \
    --max-num-batched-tokens 16384 \
    --max-num-seqs 32 \
    --gpu-memory-utilization 0.9 \
    --trust-remote-code \
    --reasoning-parser qwen3 \
    --enable-auto-tool-choice \
    --tool-call-parser qwen3_coder \
    --async-scheduling \
    --allowed-local-media-path / \
    --mm_processor_cache_type="shm" \
    --skip-mm-profiling \
    --compilation-config '{"cudagraph_capture_sizes":[1,4,8,12,16,24,32,48,64,128,256], "cudagraph_mode":"FULL_DECODE_ONLY"}' \
    --speculative_config '{"method": "qwen3_5_mtp", "num_speculative_tokens": 3}'

5. 拉起Qwen3.5-9B模型

前置条件 使用命令 conda activate vllm-0.17.0rc1初始化vllm-0.17.0rc1环境。

拉起命令

export PYTORCH_NPU_ALLOC_CONF="expandable_segments:True"
export HCCL_OP_EXPANSION_MODE="AIV"
export HCCL_BUFFSIZE=1024
export OMP_NUM_THREADS=1
export TASK_QUEUE_ENABLE=1


vllm serve /workdir/model_weights/Qwen3.5-9B \
    --served-model-name qwen35-9b \
    --host 0.0.0.0 \
    --port 8359 \
    --data-parallel-size 1 \
    --tensor-parallel-size 1 \
    --max-model-len 71680  \
    --max-num-batched-tokens 16384 \
    --max-num-seqs 32 \
    --gpu-memory-utilization 0.9 \
    --trust-remote-code \
    --reasoning-parser qwen3 \
    --enable-auto-tool-choice \
    --tool-call-parser qwen3_coder \
    --async-scheduling \
    --allowed-local-media-path / \
    --mm_processor_cache_type="shm" \
    --skip-mm-profiling \
    --compilation-config '{"cudagraph_capture_sizes":[1,4,8,12,16,24,32,48,64,128,256], "cudagraph_mode":"FULL_DECODE_ONLY"}' \
    --speculative_config '{"method": "qwen3_5_mtp", "num_speculative_tokens": 3}'

6. 拉起 bge-large-zh-v1.5 模型

前置条件: 使用命令 conda activate asr-and-bge初始化asr-and-bge环境。

命令参考:

export TASK_QUEUE_ENABLE=2

cd /workdir/bge

taskset -c 144-150 python bge_large_serve.py & 
python bge_large_client.py

7. 拉起 ASR相关模型

前置条件: 使用命令 conda activate asr-and-bge初始化asr-and-bge环境。

7.1 fsmn_vad 模型

命令参考:

cd /workdir/fsmn_vad

python test_fsmn_prof.py

7.2 paraformer_online 模型

命令参考:

cd /workdir/paraform_online

taskset -c 144-150 python3 test_online_demo.py

7.3 ct-transformer 模型

命令参考:

cd /workdir/ct-transformer

python run_pipeline.py

7.4 seaco_paraformer 模型

命令参考:

cd /workdir/seaco_paraformer

python test_asr.py

8. 拉起资源占用工具

命令参考:

source /usr/local/Ascend/ascend-toolkit/set_env.sh
source /usr/local/Ascend/toolbox/set_env.sh

cd /workdir/tools

./run_calc.sh

docker run -it -d --net=host --shm-size=20g --privileged=true \ --name supplychain-xin1 \ --device=/dev/davinci_manager \ --device=/dev/hisi_hdc \ --device=/dev/devmm_svm \ --device=/dev/davinci0 \ --device=/dev/davinci1 \ --device=/dev/davinci2 \ --device=/dev/davinci3 \ --device=/dev/davinci4 \ --device=/dev/davinci5 \ --device=/dev/davinci6 \ --device=/dev/davinci7 \ -v /usr/local/Ascend/driver:/usr/local/Ascend/driver:ro \ -v /usr/local/sbin:/usr/local/sbin:ro \ -v /usr/local/sbin/npu-smi:/usr/local/sbin/npu-smi \ -v /usr/local/Ascend/add-ons/:/usr/local/Ascend/add-ons \ -v /home/model_weights:/workdir/model_weights \ -v /home/llg:/home/llg \ supplychain-xin1:v1.3 \ bash

export VLLM_USE_MODELSCOPE=True export PYTORCH_NPU_ALLOC_CONF=max_split_size_mb:256 export VLLM_USE_V1=1 export TASK_QUEUE_ENABLE=1 export CPU_AFFINITY_CONF=1 export OMP_PROC_BIND=false export OMP_NUM_THREADS=10 export HCCL_OP_EXPANSION_MODE="AIV" export VLLM_ASCEND_ENABLE_TOPK_OPTIMIZE=1 export VLLM_ASCEND_ENABLE_DENSE_OPTIMIZE=1 export VLLM_ASCEND_ENABLE_PREFETCH_MLP=1 vllm serve /workdir/model_weights/Qwen2.5-14B-Instruct/ \ --served-model-name qwen2.5-14B \ --max_model_len 2048 \ --max-num-batched-tokens 20480 \ --tensor-parallel-size 2 \ --gpu-memory-utilization 0.9 \ --port 8714 \ --compilation-config '{"cudagraph_mode": "FULL_DECODE_ONLY"}'

export PYTORCH_NPU_ALLOC_CONF="expandable_segments:True" export HCCL_OP_EXPANSION_MODE="AIV" export HCCL_BUFFSIZE=1024 export OMP_NUM_THREADS=1 export TASK_QUEUE_ENABLE=1 vllm serve /workdir/model_weights/Qwen3.5-2B \ --served-model-name qwen35-2b \ --host 0.0.0.0 \ --port 8352 \ --data-parallel-size 1 \ --tensor-parallel-size 1 \ --max-model-len 71680 \ --max-num-batched-tokens 16384 \ --max-num-seqs 32 \ --gpu-memory-utilization 0.9 \ --trust-remote-code \ --reasoning-parser qwen3 \ --enable-auto-tool-choice \ --tool-call-parser qwen3_coder \ --async-scheduling \ --allowed-local-media-path / \ --mm_processor_cache_type="shm" \ --skip-mm-profiling \ --compilation-config '{"cudagraph_capture_sizes":[1,4,8,12,16,24,32,48,64,128,256], "cudagraph_mode":"FULL_DECODE_ONLY"}' \ --speculative_config '{"method": "qwen3_5_mtp", "num_speculative_tokens": 3}'

export PYTORCH_NPU_ALLOC_CONF="expandable_segments:True" export HCCL_OP_EXPANSION_MODE="AIV" export HCCL_BUFFSIZE=1024 export OMP_NUM_THREADS=1 export TASK_QUEUE_ENABLE=1 vllm serve /workdir/model_weights/Qwen3.5-4B \ --served-model-name qwen35-4b \ --host 0.0.0.0 \ --port 8354 \ --data-parallel-size 1 \ --tensor-parallel-size 1 \ --max-model-len 71680 \ --max-num-batched-tokens 16384 \ --max-num-seqs 32 \ --gpu-memory-utilization 0.9 \ --trust-remote-code \ --reasoning-parser qwen3 \ --enable-auto-tool-choice \ --tool-call-parser qwen3_coder \ --async-scheduling \ --allowed-local-media-path / \ --mm_processor_cache_type="shm" \ --skip-mm-profiling \ --compilation-config '{"cudagraph_capture_sizes":[1,4,8,12,16,24,32,48,64,128,256], "cudagraph_mode":"FULL_DECODE_ONLY"}' \ --speculative_config '{"method": "qwen3_5_mtp", "num_speculative_tokens": 3}'

export PYTORCH_NPU_ALLOC_CONF="expandable_segments:True" export HCCL_OP_EXPANSION_MODE="AIV" export HCCL_BUFFSIZE=1024 export OMP_NUM_THREADS=1 export TASK_QUEUE_ENABLE=1 vllm serve /workdir/model_weights/Qwen3.5-9B \ --served-model-name qwen35-9b \ --host 0.0.0.0 \ --port 8359 \ --data-parallel-size 1 \ --tensor-parallel-size 1 \ --max-model-len 71680 \ --max-num-batched-tokens 16384 \ --max-num-seqs 32 \ --gpu-memory-utilization 0.9 \ --trust-remote-code \ --reasoning-parser qwen3 \ --enable-auto-tool-choice \ --tool-call-parser qwen3_coder \ --async-scheduling \ --allowed-local-media-path / \ --mm_processor_cache_type="shm" \ --skip-mm-profiling \ --compilation-config '{"cudagraph_capture_sizes":[1,4,8,12,16,24,32,48,64,128,256], "cudagraph_mode":"FULL_DECODE_ONLY"}' \ --speculative_config '{"method": "qwen3_5_mtp", "num_speculative_tokens": 3}'