A benchmark of naturally occurring hallucinations with model-generated ground truth, replacing synthetic entity-swap surrogates for detection evaluation

The problem

Today, hallucination detection methods are trained and evaluated against synthetic artifacts — entity-swapped summaries, outputs from a different model than the one under test — rather than hallucinations that actually arise during generation. This means reported detection numbers do not measure whether a method works on the failure mode that matters, and the modest predictive results (e.g., R² ≈ 0.27 on hallucination scores) may partly reflect this distribution mismatch. Detection validated only on 7B–13B open-weight models with synthetic data cannot be trusted as evidence about frontier-model behavior. Without a natural-hallucination benchmark, every detection paper inherits the same unquantified gap between "detects injected errors" and "detects real hallucinations."

Why it matters

Detection methods whose reported numbers reflect real generation-time failure, and the ability to test whether model-specific internal signatures generalize once trained on authentic hallucinations.

Ways to approach it

Prior-work checks are free with an account. Results someone already ran are shown to everyone.

1. 1

   Sample a fixed prompt set through an open-weight model (≥70B, plus one 7B–13B for comparability), run the model's own generations through established factuality verification (e.g., FactScore-style claim decomposition against retrieved evidence), and label hallucinated claims — yielding thousands of naturally occurring hallucination instances from the same model that produced them. Measure: hallucination rate, claim-type distribution, and how detection accuracy differs when training on this data versus entity-swapped synthetic data.

   Check for prior work
2. 2

   Replicate one internal-state detection method (probe-based or head-based) twice: once trained on synthetic entity-swap data, once on the natural benchmark from (1). Measure the accuracy delta on a held-out natural set; the delta quantifies the synthetic-data gap the literature currently cannot see.

   Check for prior work
3. 3

   Extend evaluation across model scales (7B, 13B, 70B) and at least one API-accessible frontier model using logprob/activation-free detectors, measuring how model-specific head and probe signatures transfer when trained on natural rather than synthetic hallucinations.

   Check for prior work

Why it might fail

If natural-hallucination labels from automated verification are too noisy to train on, the benchmark simply replaces one unreliable target with another, and any measured synthetic-vs-natural gap becomes uninterpretable.

Sub-problems

• Cross-Scale and Cross-Failure Evaluation of Internal Hallucination Probes

  Current internal hallucination detectors (linear probes, concept activations, and attention-head interventions) show low predictive variance ($R^2 \approx 0.27$) and have only been validated on 7B–13B open-source architectures during standard factual recall. As a result, it is unknown whether internal representations reliably signal hallucinations in larger models or whether their predictive power breaks down entirely on stubborn failure modes like premature convergence and multi-step logical errors. Without cross-scale and cross-failure evaluation, practitioners cannot determine if internal state monitoring is a viable safety mechanism or an artifact of small-scale benchmark setups.

• Benchmarking the Generalization of Internal Hallucination Probes Across Model Scales and Failure Modes

  Current internal interpretability methods for hallucination detection—such as linear probes on hidden states and targeted attention heads—exhibit low explained variance ($R^2 \approx 0.27$) and have only been validated on 7B–13B open-weight architectures (e.g., LLaMA, Mistral, Qwen). It remains unknown whether these internal markers persist in larger scale models (70B+) or generalize across distinct model families. Furthermore, existing probes are primarily evaluated on simple factual errors rather than stubborn, multi-step reasoning fallacies or premature-convergence hallucinations.

Evidence

Each paper's own statement of the limitation, verbatim.

• The Digital Dunning-Kruger Effect: Decoupling Hallucinations via Geometric Hidden-state Observation for Semantic TruthfulnessACL 2026

  Stubborn hallucination conditions are poorly understood and no dedicated datasets exist for premature-convergence hallucinations; the taxonomy may not capture multi-step logical fallacies

• Hallucination Detection in LLMs with Topological Divergence on Attention GraphsACL 2026

  Hallucination-aware heads are model-specific; effectiveness is only validated on open-source 7B-13B models (LLaMA, Mistral, Qwen) and may not transfer to proprietary or larger models like GPT-4 or Claude

• From Noise to Narrative: Tracing the Origins of Hallucinations in TransformersNeurIPS 2025

  Hallucination prediction R² of 0.271 is modest, explaining only ~27% of hallucination score variance, and layer 18 concepts fail to predict above chance

• Alleviating Hallucinations in Large Language Models through Multi-Model Contrastive Decoding and Dynamic Hallucination DetectionNeurIPS 2025

  Requires fine-tuning the base model with synthetic hallucination data (HaluEval plus DeepSeek-V3-generated samples), adding training overhead and dependence on synthetic pair construction

• Enhancing Training Data Attribution for Large Language Models with Fitting Error ConsiderationEMNLP 2024

  Evaluation relies entirely on synthetically induced hallucinations (entity swaps like England→China at ~2% of training data) on a single dataset (XSum); performance on naturally occurring hallucinations or other attribution targets is untested.

• Embedding and Gradient Say Wrong: A White-Box Method for Hallucination DetectionEMNLP 2024

  Since the datasets' generation processes cannot be reproduced, the hallucination texts were generated by a different open-source LLM than the one that produced the benchmark data, so detection is not run during actual generation by the same model

Nearest existing work

• Unsupervised Hallucination Detection by Inspecting Reasoning ProcessesEMNLP 2025
• HalluLens: LLM Hallucination BenchmarkACL 2025
• On the Origin of Hallucinations in Conversational Models: Is it the Datasets or the Models?NAACL 2022
• HalluGuard: Demystifying Data-Driven and Reasoning-Driven Hallucinations in LLMsICLR 2026
• Embedding and Gradient Say Wrong: A White-Box Method for Hallucination DetectionEMNLP 2024
• LLM-Check: Investigating Detection of Hallucinations in Large Language ModelsNeurIPS 2024
• The Dawn After the Dark: An Empirical Study on Factuality Hallucination in Large Language ModelsACL 2024
• ODE: Open-Set Evaluation of Hallucinations in Multimodal Large Language ModelsCVPR 2025
• Rethinking Evaluation for LLM Hallucination Detection: A Desiderata, A New RAG-based Benchmark, New InsightsACL 2026
• Teaching Language Models to Hallucinate Less with Synthetic TasksICLR 2024
• Detecting and Mitigating Hallucinations in Machine Translation: Model Internal Workings Alone Do Well, Sentence Similarity Even BetterACL 2023
• HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language ModelsEMNLP 2023
• A Token-level Reference-free Hallucination Detection Benchmark for Free-form Text GenerationACL 2022
• Whispers that Shake Foundations: Analyzing and Mitigating False Premise Hallucinations in Large Language ModelsEMNLP 2024
• Unified Hallucination Detection for Multimodal Large Language ModelsACL 2024

Related open problems