Visual-ERM is a multimodal generative reward model for vision-to-code tasks.
It evaluates outputs directly in the rendered visual space and produces fine-grained, interpretable, and task-agnostic discrepancy feedback for structured visual reconstruction.
📄 Paper | 💻 GitHub | 📊 VC-RewardBench
Existing rewards for vision-to-code usually fall into two categories:
Visual-ERM addresses this by directly comparing:
and then generating structured discrepancy annotations that can be converted into reward signals or used for reflection-based refinement.
Visual-ERM is designed to judge whether a predicted result is visually equivalent to the target.
Given a pair of images, it can identify discrepancies such as:
This makes Visual-ERM useful not only as a reward model for RL, but also as a visual critic for test-time reflection and revision.
Visual-ERM is designed for structured visual reconstruction tasks, including:
Visual-space reward modeling
Evaluates predictions in rendered visual space instead of relying only on text matching or coarse embedding similarity.
Fine-grained and interpretable feedback
Produces structured discrepancy annotations rather than a single black-box score.
Task-agnostic reward supervision
A unified reward model that generalizes across multiple vision-to-code tasks.
Useful for both training and inference
Can be used as a reward model in RL and as a visual critic during test-time refinement.
We also release VisualCritic-RewardBench (VC-RewardBench), a benchmark for evaluating fine-grained image-to-image discrepancy judgment on structured visual data.
Dataset link:
https://huggingface.co/datasets/internlm/VC-RewardBench
Visual-ERM is fine-tuned from Qwen/Qwen3-VL-8B-Instruct and follows the same multimodal interface.
Visual-ERM takes as input:
The model outputs structured discrepancy annotations, which can then be:
A typical output format is:
{
"errors": [
{
"category": "structure_error",
"severity": 3,
"location": "legend area",
"description": "The legend is placed outside the plot area in the prediction."
},
{
"category": "style_error",
"severity": 2,
"location": "bar colors",
"description": "The colors differ from those in the reference image."
}
]
}For full inference scripts, RL training pipelines, evaluation code, and prompt templates, please refer to the official repository:
https://github.com/InternLM/Visual-ERM
Visual-ERM is intended for:
If you find this model useful, please consider citing:
@article{liu2026visual,
title={Visual-ERM: Reward Modeling for Visual Equivalence},
author={Liu, Ziyu and Ding, Shengyuan and Fang, Xinyu and Dai, Xuanlang and Yang, Penghui and Liang, Jianze and Wang, Jiaqi and Chen, Kai and Lin, Dahua and Zang, Yuhang},
journal={arXiv preprint arXiv:2603.13224},
year={2026}
}If you are interested in visual reward modeling, vision-to-code, or reinforcement learning for multimodal models, feel free to reach out.