LLMs often ignore supplied codebook definitions, study finds

Large language models (LLMs) are being used more and more to label social‑science texts, yet a new audit reveals they often ignore the formal definitions supplied by researchers. The finding matters because high predictive accuracy does not guarantee that the model is applying the intended coding rules, which threatens the construct validity of many AI‑assisted measurements.

Which mapping do LLM annotators follow when definitions conflict with the original label mapping?

The authors of the study introduced a counterfactual codebook audit that deliberately swaps the written definitions among output labels while keeping the original label‑to‑category mapping intact. By forcing a conflict between the supplied definition and the dataset’s historic label assignment, they could observe which source the model relied on. Across three political‑text datasets, the majority of model predictions remained aligned with the original mapping rather than the reassigned definitions. In other words, when a definition says “X” but the label “X” historically corresponded to a different concept, the LLM tended to reproduce the historic answer instead of the new definition. This pattern held for both zero‑shot prompting and fine‑tuned variants, indicating a deep reliance on label‑level priors rather than the textual codebook.

Do fine‑tuning and retrieval improve adherence to the supplied definitions?

The paper examined two common accuracy‑boosting interventions: task‑specific fine‑tuning and retrieval‑augmented prompting. Fine‑tuning on the original annotated data consistently strengthened the preservation of the historic label mapping; it did not increase the model’s willingness to follow the reassigned definitions. Retrieval‑augmented methods—where the model is given the nearest training examples as in‑context demonstrations—produced higher overall accuracy on the more complex tasks, but the audit showed that this gain came mainly from the neighborhood labels rather than from the definitions themselves. When the retrieved demonstrations were relabeled to match the swapped definitions, the model’s predictions shifted accordingly, confirming that retrieval influences the model through the demonstrations, not through the definition text.

How does the choice of retrieval strategy affect annotation performance?

Two retrieval strategies were compared: Global Example‑RAG, which pulls the nearest examples regardless of class, and Class‑wise Example‑RAG, which selects examples separately within each category. Global retrieval outperformed the class‑wise alternative and also beat a baseline with no retrieval on the two more demanding datasets. However, the authors also ran a direct k‑nearest‑neighbor (kNN) classifier using the same retrieved neighborhood. The kNN approach recovered 74–93 % of the performance boost that the LLM achieved with prompting, as shown in Figure 1, which plots the weighted‑F1 difference between the LLM prompt and the kNN baseline. This result suggests that most of the improvement stems from the label information present in the retrieved texts rather than from any sophisticated reasoning performed by the model.

What happens when definitions and retrieved demonstrations encode conflicting mappings?

The study further explored scenarios where the supplied definitions and the labels of the retrieved demonstrations point to different category assignments. When the demonstrations were relabeled to align with the swapped definitions, the model’s outputs shifted dramatically toward the new mapping, indicating that demonstrations can override definitions when they are in direct conflict. Conversely, leaving the demonstrations unchanged while swapping only the definitions produced only modest changes in predictions. The authors conclude that definitions, label names, and demonstrations compete for influence, with demonstrations often carrying the strongest weight.

How Grafy creators, developers, and researchers can explore these findings

The audit highlights a practical challenge for anyone building AI‑assisted measurement pipelines: simply providing a well‑written codebook is insufficient if the model’s training data or in‑context examples encode a different label logic. Grafy users can experiment with these dynamics without writing custom code. The Reader app lets you upload the full paper (or its PDF) and ask follow‑up questions about the methodology, reproducing the counterfactual audit in a conversational way. For hands‑on testing, the Chat interface lets you prompt multiple LLMs with swapped definitions and see how their predictions change, mirroring the paper’s experimental setup. By combining these tools, researchers can quickly validate whether their own annotation pipelines respect the intended codebook or are unintentionally inheriting legacy label biases.

---

Source: Mishra, V., Gupta, S., Gupta, D., Tai, Y. C., & Li, Q. (2026). Definitions, Label Names, or Demonstrations? Auditing What Governs LLM Annotation. arXiv:2611.09457.

Frequently asked questions

Do LLMs automatically follow the definitions supplied in a codebook?

No. The audit shows that when definitions are swapped with the original label mapping, most LLMs continue to produce the historic labels, indicating they rely more on prior label associations than on the supplied definitions.

Can fine‑tuning make LLMs more faithful to new definitions?

Fine‑tuning on the original annotated data actually reinforces the historic mapping rather than improving adherence to new definitions. The study found no reliable increase in definition compliance after standard fine‑tuning.

Does retrieval‑augmented prompting guarantee that models respect the codebook?

Retrieval improves overall accuracy, but the gain comes mainly from the labels of the retrieved examples. When demonstrations are relabeled to match new definitions, the model follows them; otherwise, definitions have limited impact. Retrieval alone does not ensure codebook fidelity.

---

Source: Arxiv AI Recent — Definitions, Label Names, or Demonstrations? Auditing What Governs LLM Annotation