New Research Exposes Flaws in LLM Intersectional Bias Audits
Researchers have identified that many current LLM bias audits misidentify additive effects as intersectional bias, potentially leading to false conclusions.
Source: Grafy Research LabResearchers have identified a significant flaw in how large language model (LLM) audits detect intersectional bias, revealing that many current methods struggle to distinguish between simple additive effects and true intersectional interactions. This discovery suggests that much of what is currently reported as intersectional bias may actually be the predictable sum of independent identity-based biases.
What is intersectional bias in LLMs?
Intersectional bias occurs when an AI model treats individuals who hold multiple marginalized identities differently than what their individual identities would suggest. According to the paper "LLM GerryMandering: Separating Interactions from Additive Effects in LLM Audits," written by George Rahul, Ronit Jahilal, Bhaskara Hanuma Vedula, Ponnurangam Kumaraguru, and Pushkar Shukla, intersectionality implies that the experience of a subgroup is more than just the sum of its parts. For example, if a model shows a bias against women and a bias against a specific religion, intersectional bias is present only if the model's response to a woman of that religion deviates from the combined effect of those two separate biases.
Why do current audits fail?
Many existing audits use what the authors call a "single-attribute criterion." This method flags a subgroup as biased if the model's response to that subgroup exceeds the response to each identity attribute individually. However, the researchers argue that this criterion is fundamentally flawed because it cannot differentiate between an interaction and simple addition.
Figure 1 shows this distinction clearly. In the first scenario (Figure 1a), a model might attribute higher anxiety to women and higher depression to Muslim individuals. If a Muslim woman receives a score that is higher than both individual benchmarks, the single-attribute criterion flags her as an intersectional bias case. Yet, if that score is exactly the sum of the two separate biases, no true interaction has occurred. The researchers found that when testing this criterion on 150 synthetic datasets containing no actual interactions, the method flagged an average of 66.1 false subgroups per dataset, meaning it triggered a false positive in every single test case.
How does the Addis method work?
To solve this, the authors introduced a new framework called Addis (Additive Intersectional Screen). Instead of comparing a subgroup to its individual parts, Addis fits an additive model across all attributes to predict what the response should be if no interaction exists. It then calculates the Additive Residual Distance () between the observed response and this additive prediction.
By testing subgroups against a specific null hypothesis—one that preserves the effects of individual attributes but removes interaction effects—Addis can isolate true interactions. In their synthetic benchmarks, Addis was significantly more accurate, flagging only 0.09 false subgroups per dataset and successfully recovering 84% of the actual interactions planted in the data, compared to only 32% for the standard criterion.
What were the findings from the IndiPersona study?
To test the method on real-world complexities, the researchers created IndiPersona, a dataset of 10,000 synthetic Indian personas crossing six identity attributes: gender identity, age, sexual orientation, socioeconomic status, caste, and religion. They audited 13 open language models using this dataset.
The results showed that while the standard criterion flagged 7.1% of tested subgroups on average, Addis flagged only 3.0%. The study found that the interaction between gender identity and sexual orientation was the most prominent pairing across all models. Additionally, using counterfactual personas, the researchers were able to reveal a caste class interaction in almost every model, a detail that the observational test alone was unable to detect.
How can researchers apply these findings?
For researchers and developers working on AI safety and fairness, these findings highlight the necessity of using additive baselines rather than single-attribute comparisons. Relying on outdated auditing standards may lead to a misunderstanding of how models actually process identity, potentially misallocating resources toward fixing "interactions" that are actually just cumulative single-attribute biases.
If you are currently reviewing complex academic papers or technical reports regarding AI ethics and methodology, you can use Grafy Reader to perform deep research over the PDFs. By uploading these studies, you can ask specific questions about mathematical proofs or experimental setups to better understand the nuances of new auditing frameworks.
Source: Grafy Research Lab — LLM GerryMandering: Separating Interactions from Additive Effects in LLM Audits


