LocateAnything 是基于 NVlabs Eagle 多模态框架构建的视觉定位（Visual Grounding）模型，能够根据自然语言描述在图像中定位目标并输出 <box> 坐标。本文记录了 LocateAnything-3B 在昇腾 910B NPU 上的适配与验证过程。
| 配套 | 版本 | 环境准备指导 |
|---|---|---|
| Python | 3.10 | - |
| CANN | 8.5.1 | - |
| vllm-ascend 镜像 | v0.18.0rc1 | - |
| torch / torch_npu | 随镜像内置 | - |
| peft | 0.19.1 | - |
| Eagle | main (783f656d) | - |
| LocateAnything-vLLM | main (bee9ce47) | - |
| LocateAnything-3B | modelscope 权重 | - |
整机：Atlas 800T A2
NPU：910B 昇腾
部署方式：单卡部署
操作系统：Ubuntu 20.04, ARM
驱动版本：25.5.2
docker pull quay.io/ascend/vllm-ascend:v0.18.0rc1export IMAGE=quay.io/ascend/vllm-ascend:v0.18.0rc1
docker run --rm \
--name locate_anything \
--privileged \
--net=host \
--shm-size=1g \
--device /dev/davinci_manager \
--device /dev/devmm_svm \
--device /dev/hisi_hdc \
-v /etc/hccn.conf:/etc/hccn.conf \
-v /usr/local/dcmi:/usr/local/dcmi \
-v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \
-v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/ \
-v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info \
-v /etc/ascend_install.info:/etc/ascend_install.info \
-v /root/.cache:/root/.cache \
-v /opt/data/models:/opt/data/models \
-it $IMAGE bashpip install peft==0.19.1git clone https://github.com/NVlabs/Eagle.git /workspace/Eagle
cd /workspace/Eagle && git checkout 783f656d && cd -
git clone https://github.com/WuNein/LocateAnything-vLLM.git /workspace/LocateAnything-vLLM
cd /workspace/LocateAnything-vLLM && git checkout bee9ce47 && cd -
git clone https://atomgit.com/Ascend-SACT/LocateAnything-3B.git /workspace/LocateAnything_Transfer
python3 -c "from modelscope import snapshot_download; snapshot_download('nv-community/LocateAnything-3B', local_dir='/workspace/LocateAnything-3B')"迁移补丁位于本仓库 patches/ 目录，分别覆盖 Eagle 与 LocateAnything-vLLM 两个仓库：
PATCH_DIR="/workspace/LocateAnything_Transfer/patches/"
cd /workspace/Eagle && git apply "$PATCH_DIR/eagle_ascend_migration.patch" && cd -
cd /workspace/LocateAnything-vLLM && git apply "$PATCH_DIR/vllm_ascend_migration.patch" && cd -补丁主要为适配昇腾推理平台，处理 transfer_to_npu 注入等。
processing_locateanything.py 中的 lmdb、decord 在纯推理场景非必需，将其改为可选导入以避免环境缺包报错：
cd /workspace/LocateAnything-3B
sed -i 's/^import lmdb$/try:\n import lmdb\nexcept (ImportError, ModuleNotFoundError):\n lmdb = None/' processing_locateanything.py
sed -i 's/^import decord$/try:\n import decord\nexcept (ImportError, ModuleNotFoundError):\n decord = None/' processing_locateanything.py运行 10 张 demo 图的定位验证脚本（manifest 模式）：
export ASCEND_RT_VISIBLE_DEVICES=0
cd /workspace/LocateAnything_Transfer/demo
python3 test_demo_native.py也支持单图测试：
python3 test_demo_native.py --image images/clean_COCO_train2014_000000576829_0.png --phrase "the bed"可以修改test_demo_native.py中的LocateAnythingWorker的各参数，进一步实现文字、多目标的定位等定位。
在 10 张 COCO demo 图上以 ground_single 任务、hybrid 生成模式进行验证，全部通过，平均 IoU 达 94.53%。首图为推理预热耗时约 6s，已排除预热后单图平均耗时约 476ms：
| # | 类别 | 目标描述 | Native IoU | 耗时(ms) |
|---|---|---|---|---|
| 1 | person | Pink shirt girl... | 0.9659 | 6045.6 |
| 2 | person | lady sitting | 0.9269 | 393.6 |
| 3 | animal | black cat under sink | 0.9748 | 469.0 |
| 6 | furniture | sheets on the bed (bottom-right) | 0.9225 | 480.5 |
| 7 | furniture | bedspread | 0.8390 | 609.7 |
| 7 | other | black-white striped animal (left) | 0.9666 | 502.7 |
| 8 | animal | butt zebra, cut in half | 0.9585 | 501.0 |
| 8 | clothing | tennis player in white shirt | 0.9852 | 509.8 |
| 9 | clothing | dude in blue shirt tie | 0.9694 | 469.3 |
| 10 | other | first case on left | 0.9445 | 346.0 |
原图
定位结果
在 RefCOCOg test split（共 5023 样本）上以 ground_single 任务、hybrid 生成模式进行完整评测，结果如下：
| 指标 | 实测 (Ascend 910B3 NPU) |
|---|---|
| F1 @ IoU=0.50 | 88.39% |
| F1 @ IoU=0.95 | 41.71% |
| Mean F1 (0.5:0.05:0.95) | 76.96% |
| 检测率 | 100% (5023/5023) |
| Mean IoU | 82.62% |
| 吞吐 | 1.48 samples/s |
| 总耗时 | 3382s (~57 分钟，含预热) |
| IoU 阈值 | 0.5 | 0.55 | 0.6 | 0.65 | 0.7 | 0.75 | 0.8 | 0.85 | 0.9 | 0.95 |
|---|---|---|---|---|---|---|---|---|---|---|
| F1 (%) | 88.39 | 87.34 | 86.06 | 84.85 | 82.94 | 81.01 | 78.02 | 73.92 | 65.34 | 41.71 |
| 项 | 值 | 说明 |
|---|---|---|
| Worker | LocateAnythingWorker | Eagle 仓库 |
| Task | ground_single | 单实例指代定位 |
| Prompt | Locate a single instance that matches the following description: [PHRASE]. | 评测模板 |
| generation_mode | hybrid | — |
| max_new_tokens | 8192 | — |
| temperature | 0.7 | — |
| 数据集 | lmms-lab/RefCOCOg test split | 5023 样本 |
评测采用
do_sample=True存在采样随机性。