Audit Finds Hallucination Detector Rankings Flip on Small On-Device LLMs
A new arXiv audit shows that four hallucination detectors behave inconsistently on 4‑bit quantized Llama‑3.2‑3B and Qwen2.5‑3B models.
Source: Grafy Research LabWhat was evaluated and why it matters
Researchers from the Grafy Research Lab released an audit that directly tackles a gap in the literature: most hallucination‑detection work focuses on 7B‑70B research models with generous compute budgets. In practice, developers ship much smaller, 4‑bit quantized models—such as Phi‑3‑mini, Llama‑3.2‑3B‑Instruct, Qwen2.5‑3B‑Instruct and Gemma‑2‑2B‑Instruct—on‑device. According to the paper, these models sit outside the existing evidence base, leaving practitioners uncertain about which detector to trust when resources are limited.
Which detectors and models were tested
The audit examined four detectors spanning three methodological families (as described in the arXiv submission):
- Length‑normalized token‑margin calibration – a confidence‑scaling technique that adjusts scores based on token‑level margins.
- Lexical n‑gram baseline – a simple overlap measure that counts matching n‑grams between the model output and a reference.
- Semantic entropy – a sampling‑based consistency method that measures uncertainty across multiple generations.
- SelfCheckGPT‑NLI – another sampling‑based approach that uses a natural‑language‑inference model to verify statements.
The study ran these detectors on two open‑instruction‑tuned models loaded at 4‑bit NF4 precision: Llama‑3.2‑3B‑Instruct and Qwen2.5‑3B‑Instruct. All experiments were performed on a single Colab T4 GPU, making the setup reproducible for anyone with modest cloud resources.
How did the detectors perform
Performance was measured on the closed‑book TriviaQA benchmark, which provides factual questions with known answers. The key findings were:
- Cross‑model ranking transfer is poor. The Spearman correlation between detector rankings on the two models was , and Kendall’s tau‑b was . In plain terms, a detector that ranked highest on Llama‑3.2‑3B tended to rank lowest on Qwen2.5‑3B, effectively inverting the order.
- The cheap lexical baseline holds its own. Bootstrap 95 % confidence intervals showed that the lexical n‑gram baseline was statistically indistinguishable from the more expensive sampling‑based methods (semantic entropy and SelfCheckGPT‑NLI) on both models.
- AUROC varies dramatically with the evaluation set. When the standard token‑overlap‑to‑multiple‑choice labeler was applied to TruthfulQA‑Generation, all four detectors achieved near‑chance AUROC scores between and . The same detectors reached ‑ AUROC on TriviaQA under an identical pipeline, indicating that the labeler, not the detectors, was the weak link for TruthfulQA.
These results challenge the assumption that a detector proven on large, unconstrained models will automatically transfer to small, quantized, on‑device LLMs.
What does the labeler failure reveal
The authors argue that the poor AUROC on TruthfulQA‑Generation stems from a labeler failure. The token‑overlap method used to infer correctness for multiple‑choice questions often misclassifies correct answers as incorrect when the model’s phrasing diverges from the reference. This mis‑labeling drags down the apparent performance of every detector, creating the illusion of detector weakness.
By contrast, the same detectors perform well on TriviaQA, where the labeler aligns more closely with the model’s output style. The paper therefore recommends careful validation of any automatic correctness labeler before using it to benchmark hallucination detectors.
Practical takeaways for developers and researchers
- Don’t rely on published detector rankings for on‑device models. The audit shows that rankings can invert between two models of comparable size and architecture.
- Consider inexpensive lexical baselines. When compute is scarce, a simple n‑gram overlap may be as effective as sampling‑heavy methods.
- Validate your evaluation pipeline. Ensure that the correctness labeler matches the output distribution of the target model; otherwise you risk conflating labeler error with detector error.
- Treat detector choice as a fairness‑of‑error‑rates problem. The authors frame selection as allocating error rates across user groups, which is especially relevant for products that must meet regulatory or accessibility standards.
- Leverage the released harness. The authors provide code, prompts, and per‑item scores, enabling reproducibility and further experimentation.
How to explore the findings with Grafy
Grafy’s Reader app lets you upload the PDF of the audit and ask natural‑language questions about its methodology, results, or code snippets. You can also use Grafy Code to clone the publicly released harness, run the same one‑GPU evaluation on your own Colab instance, and modify prompts to test additional models. For quick sanity checks, the Chat app can compare the four detectors side‑by‑side by feeding them the same TriviaQA items and observing the confidence scores each returns. These tools let creators, developers, and researchers verify the paper’s claims without leaving the Grafy workspace.
Frequently asked questions
Which hallucination detector performed best on the small models?
The audit found no clear winner: the lexical n‑gram baseline performed statistically indistinguishably from the more complex sampling‑based detectors on both Llama‑3.2‑3B‑Instruct and Qwen2.5‑3B‑Instruct.
Why did the detectors show near‑chance AUROC on TruthfulQA‑Generation?
According to the authors, the token‑overlap labeler used for TruthfulQA mis‑labels many correct answers, producing AUROC scores between and . The detectors themselves achieve higher scores on TriviaQA, indicating the problem lies with the labeler, not the detectors.
Can I reproduce the audit using Grafy’s tools?
Yes. Upload the released code to Grafy Code, run the one‑GPU pipeline on a Colab T4 instance, and use Grafy Reader to examine the PDF and per‑item scores. This reproduces the exact experimental setup described in the paper.


