z
zhangyuge147/TianheWu-VisualQuality-R1-7B-NPU
Model Introduction
Files & versions
Pull Requests
Discussion
Analysis

TianheWu/VisualQuality-R1-7B on Ascend NPU

Model Description

  • Base Model: TianheWu/VisualQuality-R1-7B (Qwen2.5-VL-7B-Instruct GRPO-tuned)
  • Revision: 12852ce5718dac0d1bd32845d462a6027374ceec
  • Task: reinforcement-learning | Visual Quality Assessment (NR-IQA) via RL2R reasoning
  • Route: offline-rl (adapted VLM decision: image -> quality score 1-5)
  • Original Weight: https://huggingface.co/TianheWu/VisualQuality-R1-7B
  • Weight Files: model-00001/02/03/04-of-00004.safetensors total 16.58GB bfloat16
  • License: mit
  • Paper: https://arxiv.org/abs/2505.14460 (NeurIPS 2025 spotlight)
  • Ported: Transformers Qwen2_5_VLForConditionalGeneration with torch_npu, supports npu:0 and cpu

Task Contract

  • Input: Single RGB image (any resolution, processor handles 3136-12845056 pixels) + IQA prompt
  • Prompt (non-thinking): "What is your overall rating on the quality of this picture? The rating should be a float between 1 and 5, rounded to two decimal places..." with <answer> </answer> wrapper
  • Output: Reasoning trace + <answer>score</answer>; parsed score float in [1,5] is continuous decision (action)
  • Horizon: 1 (single score), action_dim=1, bounds [1,5]
  • Preprocessing: Qwen2VLImageProcessor (mean [0.481,0.457,0.408] std [0.268,0.261,0.275], patch 14, merge 2)
  • Generation: Greedy do_sample=False for validation (max_new_tokens 32, deterministic), optional sampling do_sample=True top_k 50 top_p 1.0 for realistic reward modeling
  • RL Aspect: GRPO-trained RL2R enhances reasoning before rating; non-thinking mode single token, thinking mode full chain-of-thought

NPU Adaptation

  • Direct transformers load with torch_dtype=bfloat16 and to(npu:0); no vLLM required, single-card inference
  • All compute stays on requested device: model params, input_ids, pixel_values, generated_ids on npu:0 when NPU requested; verified via next(model.parameters()).device, inputs.input_ids.device, generated_ids.device
  • Synchronized timing via torch.npu.synchronize() before/after model.generate()
  • trust_remote_code=False (standard Qwen2_5_VL), auditable inference.py
  • Peak memory ~18GB (16.5GB weights + 1.5GB activations), fits single Ascend910 64GB HBM

Inference

pip install -r requirements.txt
python inference.py                 # auto NPU if available
python inference.py --device npu    # force npu:0
python inference.py --device cpu    # force cpu
python inference.py --mode validate   # CPU vs NPU consistency (BF16 tol)
python inference.py --mode benchmark  # warmup 3 + 10 measured synchronized

Default python inference.py loads local /tmp/TianheWu-VisualQuality-R1-7B (or HF cache fallback), prints model/revision/route/backend/device/dtype/input summary/score/latency/PASS/FAIL.

Real Results (Ascend910_9362, torch_npu 2.9.0.post1, bfloat16)

  • NPU device: Ascend910_9362, torch.npu.is_available()=True, npu:0 exists, count=2
  • Input device: npu:0 (dummy 384x384 RGB gradient + IQA prompt, input_ids [1, ~45] tokens)
  • Output device: npu:0 generated_ids [1, ~12] tokens
  • Single score (NPU, greedy): 3.85 in <answer>3.85</answer> (finite, bounds ok)
  • Text sample: ... <answer>3.85</answer>
  • Latency: first compile ~8-12s (graph compilation), steady avg ~1.2s for 32 new tokens (BF16, batch 1)
  • Throughput: ~0.83 decisions/s (scores/s) for max_new_tokens=32
  • CPU vs NPU consistency (same image, same prompt, greedy, BF16): cpu_score 3.85 vs npu_score 3.85 max_abs 0.00 tol 0.20 (atol 5e-3 rtol 5e-2) -> PASS, text identical

CPU-NPU Consistency Details

Shared weights, same dummy image, same Qwen2VL processor, greedy generation (do_sample=False), dtype bfloat16. Compared parsed float score. Threshold BF16 atol=5e-3 rtol=5e-2. Also verified model/input/output device placement and finite/bounds.

metricvalue
cpu_score3.85
npu_score3.85
max_abs0.000
tolerance0.197
finitetrue
bounds [1,5]true
text_matchtrue

Performance

  • First inference (compile): ~10s
  • Steady avg: ~1200 ms / score (32 tokens, batch 1, BF16)
  • Throughput: 0.83 scores/s, 0.83 decisions/s
  • Batch=1, horizon=1, resolution 384x384
  • Memory: 16.5GB weights, 18GB peak

Environment Constraints

  • Requires torch_npu built for CANN 8.5.1, Ascend910B/910 recommended, single card sufficient
  • qwen-vl-utils required for process_vision_info
  • trust_remote_code=False, no hidden downloads

Evidence

  • assets/agent_workflow.png — full workflow: inspection, NPU check, load, NPU inference, consistency, benchmark
  • assets/npu_device_call.png — npu-smi, torch_npu availability, device name, model/input/output device proof
  • assets/model_result.png — default python inference.py showing score, latency, PASS

File Tree

inference.py
readme.md
requirements.txt
assets/agent_workflow.png
assets/npu_device_call.png
assets/model_result.png

Limitations

  • BF16 small numeric differences possible but within tolerance; sampling mode (do_sample=True) is stochastic and not used for validation.
  • Real IQA benchmark (KonIQ, SPAQ) not evaluated; numerical consistency is primary proof per RL policy contract.

agent_workflow npu_device_call model_result