环境准备

镜像准备

使用vllm-ascend官方镜像：

docker pull quay.io/ascend/vllm-ascend:v0.11.0rc2

更新依赖模块包

pip uninstall -y transformers accelerate
pip install transformers==4.49.0 accelerate==0.27.0 -i https://pypi.tuna.tsinghua.edu.cn/simple 
pip install qwen-vl-utils==0.0.8

下载权重

pip install modelscope

modelscope download --model xiaomi-open-source/Xiaomi-MiMo-VL-Miloco-7B --local_dir ./Xiaomi-MiMo-VL-Miloco-7B

启动服务

vllm serve ./Xiaomi-MiMo-VL-Miloco-7B --port 8106 --dtype bfloat16 --max-model-len 32768 --served-model-name Xiaomi-MiMo-VL-Miloco-7B

文本推理

-H "Content-Type: application/json" \
-d '{
"model": "Xiaomi-MiMo-VL-Miloco-7B",
"messages": [
{"role": "system", "content": " You are a helpful language and speech assistant."},
{"role": "user", "content": "你是谁."}
],
"temperature": 0.7,
"max_tokens": 2048
}'

推理结果如下：

{"id":"chatcmpl-8d9bf3b0cc2d431da20e749e2a5c4baf","object":"chat.completion","created":1766032687,"model":"Xiaomi-MiMo-VL-Miloco-7B","choices":[{"index":0,"message":{"role":"assistant","content":"你好，我是一个人工智能助手，名为“文心一言”。我是由百度开发的，旨在通过自然语言处理技术，帮助用户解答问题、提供信息、进行对话交流。我的目标是成为你身边的智能助手，无论你需要了解知识、解决问题还是进行日常聊天，我都会尽力为你提供高效、准确的支持。希望我能随时为你服务，让你的体验更加便捷和愉快！","refusal":null,"annotations":null,"audio":null,"function_call":null,"tool_calls":[],"reasoning_content":null},"logprobs":null,"finish_reason":"stop","stop_reason":null}],"service_tier":null,"system_fingerprint":null,"usage":{"prompt_tokens":25,"total_tokens":424,"completion_tokens":399,"prompt_tokens_details":null},"prompt_logprobs":null,"kv_transfer_params":null}

图片推理

curl http://127.0.0.1:8106/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "Xiaomi-MiMo-VL-Miloco-7B",
"messages": [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": [
        {"type": "image_url", "image_url": {"url": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-Omni/demo/cars.jpg"}},
        {"type": "text", "text": "请描述一下图片内容"}
    ]}
    ],
"temperature": 0.7,
"max_tokens": 2048
}'

推理结果如下： {"id":"chatcmpl-c070a0cd6348437699a827c78af2da81","object":"chat.completion","created":1766040269,"model":"Xiaomi-MiMo-VL-Miloco-7B","choices":[{"index":0,"message":{"role":"assistant","reasoning_content":null,"content":"这张图片展示了四款不同品牌和类型的豪华汽车，分为四个区域呈现： \n\n- 左上角：一辆白色的劳斯莱斯轿车，车身线条优雅大气，标志性的直瀑式进气格栅与车头“欢庆女神”立标极具辨识度，车辆停放在简洁的浅蓝色背景前，凸显奢华质感。 \n- 右上角：一辆深灰色的奔驰GLE SUV，处于动态行驶状态，背景是沙漠场景（地面为沙土与水洼，后方有山峦、蓝天白云），阳光洒下营造出冒险与活力氛围，车身细节（如进气格栅、轮毂）与环境倒影强化了越野风格。 \n- 左下角：一辆红色的法拉利Portofino M敞篷车，红色车身搭配黑色软顶敞篷，车内可见棕色调座椅，停放在明亮的室内展厅（背景为简洁的白色墙面与玻璃），展现跑车的运动激情与精致奢华。 \n- 右下角：一辆白色的保时捷911，经典的圆形大灯、流畅的车身线条与黑色车顶形成对比，停放在纯白色背景前，简洁背景突出车型标志性的设计风格与优雅姿态。 \n\n\n四款车分别代表了豪华轿车、SUV、敞篷跑车、经典跑车等不同细分领域，从设计、场景到品牌调性各有特色，共同展现了高端汽车的多元魅力。","tool_calls":[]},"logprobs":null,"finish_reason":"stop","stop_reason":null}],"usage":{"prompt_tokens":7891,"total_tokens":8408,"completion_tokens":517,"prompt_tokens_details":null},"prompt_logprobs":null,"kv_transfer_params":null}