Eva-4B is a 4B-parameter model for detecting evasive answers in earnings call Q&A.
directintermediatefully_evasiveQwen/Qwen3-4B-Instruct-2507Eva-4B is intended for research and tooling around corporate disclosure quality and evasiveness in earnings call Q&A.
Given an earnings call Question (analyst) and Answer (management), the model predicts one of:
This taxonomy follows the Rasiah framework referenced in the paper.
The training set is constructed via a multi-model annotation framework:
| Rank | Model | Accuracy | F1-Macro |
|---|---|---|---|
| 1 | Claude Opus 4.5 | 83.9% | 0.838 |
| 2 | Gemini-3-Flash | 83.7% | 0.833 |
| 3 | GLM-4.7 | 82.6% | 0.809 |
| 4 | Eva-4B (Ours) | 81.3% | 0.807 |
| 5 | GPT-5.2 | 80.5% | 0.805 |
Note: based on the accuracy values, Eva-4B is 2nd among open-source models, after GLM-4.7 (82.6%).
| Class | F1 |
|---|---|
| direct | 0.851 |
| intermediate | 0.698 |
| fully_evasive | 0.873 |
The paper notes most errors are confusion between direct and intermediate.
The paper compares Eva-4B training labels (multi-model + judge) vs an Opus-only construction:
The paper reports the Opus-only baseline achieves lower training loss but worse generalization.
The prompt below matches prompts/evasion_rasiah_fine_tuning_minimalist.txt in this repo.
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
model_name = "FutureMa/Eva-4B"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
model_name,
torch_dtype="auto",
device_map="auto",
)
PROMPT_TEMPLATE = """You are a financial analyst. Your task is to Detect Evasive Answers in Financial Q&A
Question: {{question}}
Answer: {{answer}}
Response format:
```json
{"reason": "brief explanation under 100 characters", "label": "direct|intermediate|fully_evasive"}
```
Answer in json block content, no other text"""
question = "What are your revenue expectations for next quarter?"
answer = "We remain optimistic about our business trajectory and will continue to focus on executing our strategic priorities."
prompt = (
PROMPT_TEMPLATE
.replace("{{question}}", question)
.replace("{{answer}}", answer)
)
messages = [{"role": "user", "content": prompt}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer([text], return_tensors="pt").to(model.device)
with torch.no_grad():
output_ids = model.generate(
**inputs,
max_new_tokens=128,
temperature=0.7,
do_sample=True,
)
generated = output_ids[0][inputs["input_ids"].shape[1]:]
response = tokenizer.decode(generated, skip_special_tokens=True)
print(response)Expected output format:
{"reason": "...", "label": "direct|intermediate|fully_evasive"}Eva-4B is a research artifact and not financial advice. Outputs should be used as one signal among many and should be reviewed by humans for high-stakes decisions.
If you use this model, please cite the accompanying paper:
@misc{ma2026evasionbenchdetectingevasiveanswers,
title={EvasionBench: Detecting Evasive Answers in Financial Q&A via Multi-Model Consensus and LLM-as-Judge},
author={Shijian Ma and Yan Lin and Yi Yang},
year={2026},
eprint={2601.09142},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2601.09142},
}Paper: https://arxiv.org/abs/2601.09142
Last updated: 2026-01-12