hf_mirrors/Llama-3.1-8B-Instruct-singprobe
Model Introduction
Files & versions
Analysis

Llama-3.1-8B-Instruct-singprobe

Model Description

SingProbe is an intrinsic streaming guardrail built on meta-llama/Llama-3.1-8B-Instruct. Rather than running a separate safety model, this lightweight probe reuses the base model's hidden states during generation to score, at every token, query intent, response unsafety, and hallucination risk. It adds less than 0.5% decode-time overhead.

Base modelProbe parametersTapped layersOutputs
inclusionAI/Llama-3.1-8B-Instruct-singprobe8.13M[9, 19, 30]8 intents + unsafe + hallucination

See the technical report for methodology and complete results. Training codes are available at inclusionAI/SingProbe.

Evaluation

Higher is better for every metric. Results are averages over the benchmark suites specified below.

TaskMetricLlama-3.1-8B-Instruct-singprobeReference baseline
Query intent classification (6 benchmarks)F10.8759YuFeng-XGuard-Reason-8B: 0.8714
Response safety classification (8 benchmarks)F10.8624Qwen3Guard-Gen-8B-strict: 0.8604
Streaming safety (3 benchmarks)R-AUC / T-AUC0.9858 / 0.9291Qwen3Guard-Stream-8B-strict: 0.9640 / 0.8893
Hallucination detection (6 benchmarks)AUC0.7704DRIFT: 0.8000
Deployment characteristicResult
Benign-response false-positive rate0.03% average across 5 datasets
Decode overhead< 0.5%

Quick Start

SingProbe is supported through the SGLang integration branch or vLLM integration branch. Load the probe by its Hugging Face ID at server launch:

python -m sglang.launch_server \
  --model-path meta-llama/Llama-3.1-8B-Instruct \
  --probe-ckpt inclusionAI/Llama-3.1-8B-Instruct-singprobe \
  --port 30000

The integrations return one score dictionary per generated token (label_0–label_9). Use the exact base-model/probe pair: meta-llama/Llama-3.1-8B-Instruct with this checkpoint.

Citation

@article{singteam2026singprobe,
  title = {SingProbe Technical Report},
  author = {Sing Team},
  journal = {arXiv preprint arXiv:2608.30703},
  year = {2026},
}