Algorithmic fairness

Overview

A model is treated as fair toward a group when its outcomes do not depend on a sensitive variable such as sex, race or age, in the sense the team has defined. The variable is called Z and the prediction \hat{Y}. Fairness is a property of the whole system, so it has to be measured on the data and the decision, not assumed from the model 2110.01167 §2.5.

The problem is partly a data problem. Biased training data can lead to discriminatory predictions, and the survey treats fairness as one step of the data pipeline 2112.06409 §5. The same source separates three points where a fix can go: before training (the data), during training (the loss or constraints), and after training (the decision rule) 2112.06409 §5.2.

Measures

Measures come first, because every intervention is judged against one of them. The survey states the common group criteria, with Z a binary sensitive attribute 2112.06409 §5.1:

• Demographic parity: the positive-prediction rate is equal across groups.
• Equalized odds: the rates of predicting positive are equal across groups within each true label.
• Predictive parity: among predicted positives, the share that is truly positive is equal across groups.
• Individual fairness: similar individuals get similar outcomes. This needs a task-specific distance, which the paper does not supply 2112.06409 §5.1.
• Causal fairness: uses a causal graph and counterfactual comparisons 2112.06409 §5.1.

The group criteria can be rewritten as three principles: independence (the prediction does not depend on Z), separation (independence holds given the true label) and sufficiency (the true label does not depend on Z given the prediction) 2110.01167 §2.5 2112.06409 §5.1. The survey also says the choice of metric depends on the type of the variables, on the loss used when a criterion cannot be met exactly, and on whether there is more than one sensitive variable 2110.01167 §2.5.

Checks that can hide bias. A fix can lower the one number it is judged by and leave the bias in place. In word embeddings, projecting out a single he-she direction shrinks the projection bias, but the most biased words still cluster by gender, and an RBF-kernel SVM still predicts gender with 88.88% accuracy after Hard-Debiased, against 98.25% before 1903.03862 §4. The authors argue that the projection measures a direction, not the whole structure 1903.03862 §2 1903.03862 §5. Card: Lipstick on a Pig. The result is for two embedding methods on English word lists, so it does not cover other models or languages 1903.03862 §5.

The same gap appears in images. CNNs trained on nine medical imaging datasets predict self-reported race, with one-vs-all ROC-AUC of 0.96 to 0.99 on chest X-rays 2107.10356 tab. 3. The signal survives external validation on other hospitals' data 2107.10356 tab. 4. Its strength falls under masking, lung-only inputs and low resolution, but it stays above chance, and the authors find no single cause 2107.10356 §C2 2107.10356 §C3. Their discussion calls for audits by race, sex and age 2107.10356 p. 22. For a team, this means that an audit must measure per-group results on the decision, since the inputs may still carry the group. Card: Reading Race. The race labels are self-reported and used as the only label, and the authors treat them as a social construct, not a biological one 2107.10356 p. 22.

Impossibility. Not all three principles can hold together in general. The two surveys state this with different conditions, see Contradictions. Neither gives a rule for choosing among them. The choice is a decision for the team, and it should be written down with the reason 2110.01167 §2.5.

Individual and group measures. Binns argues that the apparent conflict between individual fairness and group measures is not a conflict of principle. Both families can be defended from consistency and egalitarian justice. The real disagreement lies in the assumption a team makes about where disparities come from: unjust structure, or choices 1912.06883 §3 1912.06883 §5.3 1912.06883 tab. 2. The paper is a philosophical argument with no experiments. Its advice is to settle the purpose and the assumption first, then pick the measure 1912.06883 §6. Card: On the Apparent Conflict Between Individual and Group Fairness.

Counterfactual fairness. A causal criterion asks that a predictor's distribution for an individual stay the same when only the protected attribute is changed, with the latent exogenous variables held fixed 2206.15475 §6.1. A path-specific version counts only the effects along causal paths the team has marked as unfair 2206.15475 §6.1.1. The survey notes that the criterion needs the latent variables and the structural equations, and gives no procedure to fit or test them 2206.15475 §6.1. Card: Causal Machine Learning: A Survey and Open Problems.

For text, the causal survey of Feder et al. describes counterfactual swaps of protected attributes as a fairness check, and says that invariance criteria are rarely applied to text 2109.00725 §4.2. Swapping attribute words can also add new spurious correlations, and keyword swaps can miss labels and covariates 2109.00725 §4.1.1. Card: Causal Inference in Natural Language Processing.

Challenge sets for generated output. The criteria above assume a decision with a label. For generated text, such as machine translation, one measure uses a challenge set instead: templated sentences where the sensitive attribute is fixed by the sentence. WinoMT joins Winogender and WinoBias into 3,888 English sentences. It reads the gender of the translated entity with per-language rules and reports accuracy, plus the gap between pro- and anti-stereotypical sentences 1906.00591 §2 1906.00591 §3.1. The authors report gender bias in all four commercial and both academic systems they tested, with commercial accuracy from 28.9% to 74.1% 1906.00591 §3.2. The paper calls the set a proxy, notes it may be overfit, and uses only synthetic English 1906.00591 §4. The gap Δ_S is defined as F1 in the text, but the released code subtracts accuracies, so state which one a report uses 1906.00591 §3.2.

Templated sentences for sentiment. The same template idea appears for classifiers that output sentiment. The position paper by Poria et al. describes the Equity Evaluation Corpus as 8,400 templated sentences, built from 7 templates, 60 names, 4 emotions and 5 words 2005.00357 §3.9.2. The paper reports no experiments of its own 2005.00357 §1. It notes that few de-biasing works exist, and that the context dependence of BERT makes bias harder to measure than in static embeddings 2005.00357 §3.9.3. Card: Beneath the Tip of the Iceberg.

Generated images. Image generators are judged with their own measures. A survey of GAN evaluation measures lists fairness and bias as one group of measures, next to benchmarks and deepfake detection 2103.09396 §4. The survey runs no experiments of its own, so its fairness coverage is a pointer to other work, not a result 2103.09396 §1.2. Card: Pros and Cons of GAN Evaluation Measures: New Developments.

Variants

Each variant below names what it changes. Papers are listed at the end of the page.

Pre-processing: repair, generate or acquire data

• Repair the scores (Feldman et al.). For each group, replace each score by the average, over groups, of the scores at the same percentile. The rank order inside each group is kept 2112.06409 §5.2. The survey does not say how quantiles are interpolated or how ties are broken 2112.06409 §5.2.
• Generate data. Reweight the biased training set with a classifier that separates biased from unbiased examples, then train a generative model on the reweighted data 2112.06409 §5.2.
• Acquire data (Slice Tuner). Keep a learning curve per data slice, estimate how much data each slice needs under a budget, and acquire it in rounds 2112.06409 §5.2. The budget value is not stated 2112.06409 §5.2.

Data collection can also reduce bias at the source, through debiased sampling, debiased annotation (choice of annotators) and recording provenance 2110.01167 §3.1.1.

In-processing: constrain or reweight training

• Covariance constraint (Zafar et al.). The demographic-parity constraint is not convex, so the method uses a convex stand-in: the covariance between the sensitive attribute and each example's signed distance from the decision boundary. The survey says this does not readily extend to non-convex networks 2112.06409 §5.2.
• Adversarial debiasing. Train the classifier against an adversary that tries to recover the sensitive attribute from the classifier's output. Training can be unstable 2112.06409 §5.2.
• Adaptive reweighting (FairBatch, OmniFair). FairBatch adjusts the sensitive-group ratio within each batch based on intermediate fairness. OmniFair turns a constrained problem into an unconstrained one with a Lagrangian 2112.06409 §5.2. The survey's table lists these in-processing methods as tabular-only 2112.06409 tab. 1.

Post-processing: adjust the decision

• Randomized group-specific rules (Hardt et al.). For equalized odds, build a model that always predicts 0, one that always predicts 1, and the inverted model, then combine them with randomization per group. Pick the combination in the feasible region with the lowest expected loss 2112.06409 §5.2. The survey says this usually costs accuracy and does not quantify how much 2112.06409 §5.2.

This variant needs the group membership at decision time. The survey lists it as tabular 2112.06409 tab. 1.

Federated training: fair aggregation on the server

When data stays on the clients, the server cannot see per-group results, so the group-level methods above do not transfer directly. FairFed keeps local debiasing as a free choice per client and changes only the server's averaging step. Each round the server obtains the global equal-opportunity gap and the global accuracy through secure aggregation. Each client compares its own gap with the global one, and clients whose gap is far from the mean lose weight in the next average 2110.00857 §4.2 2110.00857 §4.3. The mechanism needs no shared per-group statistics, apart from the group-positive rates sent once at the start 2110.00857 §4.3 2110.00857 alg. 1. Card: FairFed.

Author-reported, with the paper's synthetic splits: on Adult at Dirichlet alpha = 0.1, the equal-opportunity difference moves from -0.174 under FedAvg to -0.017 with reweighting inside FairFed, and accuracy from 0.835 to 0.830 2110.00857 tab. 1. At large alpha, gains over local debiasing are small, except with FairBatch 2110.00857 §5.2.1. The paper reports the best of several grid-searched learning rates on the test metric, so the gains are upper bounds 2110.00857 §5.1.5. The experiments use logistic regression with five clients, and the paper leaves other models for future work 2110.00857 §7.

Federated training: personalize each device

Ditto uses "fair" in a different sense. Here the groups are the devices, and fairness means that accuracy is even across them, measured as the standard deviation of test accuracy over devices. The paper measures robustness as mean accuracy on benign devices under attack 2012.04221 §2. Each device keeps a personal model, pulled toward the shared global model by a penalty weighted by λ. The global model is trained by an ordinary federated solver, such as FedAvg, and the personal models stay on the device 2012.04221 §3.1 2012.04221 §3.2. At λ = 0 the models are purely local, and as λ grows the objective approaches the global one 2012.04221 §3.1. Card: Ditto.

Author-reported, on Fashion MNIST with clean data: Ditto reaches mean accuracy 0.943 (sd 0.06), against 0.909 (0.07) for the fair baseline TERM and 0.911 (0.08) for the global model 2012.04221 tab. 1. With 50% of devices sending corrupted labels (A1), TERM falls to 0.637 (0.13) while Ditto reaches 0.937 (0.07) 2012.04221 tab. 1. The paper argues that fair reweighting and robust aggregation can work against each other, and it reports Ditto as both robust and fair in its experiments 2012.04221 §1 2012.04221 fig. 5.

Each device picks its λ from a grid on its own validation data 2012.04221 app. D.2. The official code selects λ by a different rule from the one in the appendix, so the appendix rule has to be implemented from the paper if it is needed. See the Ditto card for the differences.

Clustering: proportional representation of groups

Fairness can also be stated for choosing cluster centers. The criterion proportionally representative fairness (PRF) asks that any large enough group of agents has about as many nearby centers as its size warrants. It needs no protected attribute, since groups are defined by location 2304.13917 §1. Standard k-means, k-median and k-center can violate PRF 2304.13917 §4 2304.13917 §6. The SEAR procedure always returns a PRF outcome in polynomial time. Unconstrained SEAR is a 3-approximation to proportional fairness, and the discrete version matches the best known (1+√2) guarantee 2304.13917 Lemma 2 2304.13917 Prop. 2. Card: Proportionally Representative Clustering.

The paper is theoretical and reports no experiments, so it gives no measured accuracy cost or runtime 2304.13917 §1. Its guarantees are about the output rule, not about any classifier's decisions. Use it when the decision is where to place centers, not a label.

Input conditioning: give the model the group

Instead of changing the data, the loss or the decision, the group can be given to the model as an input. In translation the speaker's gender decides some word forms, so a model with no speaker information has to guess. Vanmassenhove et al. add a speaker-gender tag to the English source of each sentence, then train the same LSTM encoder-decoder with and without the tag 1909.05088 §4.1 1909.05088 §4.2. Card: Getting Gender Right in Neural Machine Translation.

Author-reported BLEU on the general test set: the tag gives significant gains for French (+1.44), Italian (+0.29) and Danish (+0.31), a significant loss for Finnish (−0.47), and no starred change for the other six pairs 1909.05088 tab. 2. On EN–FR test sets, the tagged system gains 1.13 BLEU on male speakers and 1.22 on female speakers. The authors report the largest gain on female test sets 1909.05088 §5 1909.05088 tab. 3. The 0.09 BLEU difference between the two gains is small, and the significance tests in the table compare each system with its baseline, not the two gains with each other 1909.05088 tab. 3.

The tag is a gender label, not a measure of the criteria in Measures. Its labels are gender predictions from the MEP metadata, and the paper does not measure how many are wrong 1909.05088 §3. Each system is trained once, with no seeds or repeated runs reported 1909.05088 §5. The tag reflects the speaker, so it applies only where the speaker's attribute is known at translation time.

Prompting an LLM for a decision: choose the demonstrations

When a language model makes the decision through its prompt, the in-context examples are the lever. DecodingTrust tests fairness on the Adult income data, converted from table rows into text. The sensitive attribute is sex or race, and the paper reports accuracy plus two group gaps: a demographic parity gap and an equalized-odds gap 2306.11698 §10. The paper varies the base rate of the demonstrations and of the test set, and reports that accuracy and unfairness both move with the demonstrations' base rate. Sixteen balanced demonstrations already shift predictions toward fairness, at some cost in accuracy 2306.11698 §10. Card: DecodingTrust.

The fairness run uses a small test set of 200 samples, and its token limit caps the few-shot count at 32 2306.11698 §10 2306.11698 tab. 34. The paper's fairness results are one slice of a wider trust benchmark, so the numbers are not a general ranking of models.

Prompting an LLM as a member of a group: cultural prompting

When the question is whether a model's answers match a population, the prompt can name the population. The respondent is described as an average person born and living in a given country, and the model answers survey items in that role. No training data or weights change 2311.14096 §4.3. Card: Cultural Bias and Cultural Alignment of Large Language Models.

The paper places five GPT models on a survey-based cultural map, for 107 countries and territories 2311.14096 §4.1 2311.14096 §2. With the default respondent description, all five models sit in the same region, near English-speaking and Protestant European countries 2311.14096 §2 fig. 1. Cultural prompting lowers the mean distance to the target country for every model, from 2.42 to 1.57 for GPT-4o 2311.14096 §2. Author-reported, the share of countries that move closer ranges from 71.0% to 81.3% 2311.14096 §2.

The mean hides losses. With GPT-4o, Finland's distance grows from 0.20 to 2.43, Switzerland's from 0.45 to 2.48 and Taiwan's from 2.40 to 3.94 [arxiv:2311.14096 §2]. The authors put the share of countries that did not improve at 19 to 29% 2311.14096 §3. Measure per country, and do not rely on the average. The results rest on English prompts, ten wordings and one run per prompt at temperature 0 2311.14096 §3 2311.14096 §4.2. The paper does not say whether the answers predict behaviour in real conversations, and it did not test longer generation tasks 2311.14096 §3.

Generated documents: word choice, tone and invented detail

When a model writes a document about a person, bias can appear in three places: word choice, tone, and details the model adds beyond its input. Wan et al. test ChatGPT and Alpaca on recommendation letters, in a setting with only a name, age and occupation, and in a setting with a biography 2310.09219 §3.1. Word choice is scored per category as the odds of the category in male letters against female letters, and in the biography setting with WEAT 2310.09219 §4.2.2 2310.09219 §4.3.5. Tone is scored per letter as the share of sentences that are formal, positive or agentic, and the two genders are compared with a t-test 2310.09219 §3.2.2 2310.09219 tab. 4. Invented sentences are flagged with an NLI model that checks each sentence against the biography 2310.09219 §3.3.

Author-reported. In the name-only setting, with 120 ChatGPT letters, the odds ratio is 1.25 for masculine words and 0.85 for feminine words 2310.09219 tab. 2. In the biography setting, ChatGPT letters show male-higher gaps in positivity (t = 5.93) and agency (t = 10.47). Formality is not significant at 0.05 (t = 1.48, p = 0.07) 2310.09219 tab. 4. For Alpaca, formality (t = 3.04) and agency (t = 8.42) are significant, and positivity is not (p = 0.07) 2310.09219 tab. 4. For both chatbots the paper reports significant one-sided tests on the flagged sentences, which it calls propagation and amplification of the bias 2310.09219 tab. 6 2310.09219 §3.3.

Three things limit these numbers. The ChatGPT model is an unpinned alias 2310.09219 app. A. The agency classifier is trained and tested on ChatGPT rephrasings, so its test accuracy of about 90% does not show performance on real letters 2310.09219 app. F.2. The lexicon uses substring matching in the released code, which changes the counts biases_string_matching.py. Card: Gender bias in LLM reference letters.

Tabular benchmarks: check the gaps against a constant classifier

Fairness studies on tabular data reuse a small set of public datasets. The survey by Le Quy et al. covers 15 of them, from banking, criminal justice, health, employment and education, with size, class ratio and protected attributes for each 2110.00530 tab. 1. A dataset was included only if it was public, in a known repository, and used in at least three fairness-related papers 2110.00530 §2.1. Card: A survey on datasets for fairness-aware machine learning.

The authors train one logistic regression per dataset on a single 70/30 split, and report accuracy, balanced accuracy and the fairness gaps 2110.00530 §4.1 2110.00530 tab. 15. Ten of the 15 datasets have an absolute statistical parity below 0.1 in that table 2110.00530 tab. 15. In three rows the classifier predicts one class for everyone: Credit card clients and Diabetes predict only the negative class, and OULAD only the positive class. The gaps are 0.0 in those rows and balanced accuracy is 0.5, so the zeros come from a constant classifier and not from a fair one 2110.00530 tab. 15 2110.00530 §4.2.

The numbers rest on one split with one seed and report no variance 2110.00530 §4.1. The paper also notes that most datasets are old, and the oldest is 48 years old 2110.00530 §6. Its data are a baseline for logistic regression only, not a ranking of fairness methods 2110.00530 §4.2.

Benchmarks for stereotyped outputs

Bias in a pretrained model can be measured with a benchmark of stereotype pairs rather than a group gap. StereoSet has 16,995 context association tests in gender, profession, race and religion, over 321 target terms 2004.09456 tab. 1. Each item offers a stereotype, an anti-stereotype and an unrelated continuation. The language-modeling score lms is the share of items where the model prefers a stereotype or anti-stereotype over the unrelated option. The stereotype score ss is the share where it prefers the stereotype over the anti-stereotype, with 50 as ideal. ICAT combines the two, with 100 as ideal and 0 for a fully biased model 2004.09456 §6.2. The test split is hidden, and only 25% of target terms are public 2004.09456 §6.1. On the test split, the authors report GPT-2 (small) with the highest ICAT among the pretrained models they score, 73.0. Every model's ss is above 50, from 50.5 (RoBERTa-base) to 62.5 (the ensemble) 2004.09456 tab. 4. The paper says lms and ss are strongly correlated across models, so ICAT should be read with both sub-scores 2004.09456 §8. The code counts a tie between stereotype and anti-stereotype as an anti-stereotype pick, and the paper does not define ties [code: code/evaluation.py#L90-L94]. The masked-LM score is also computed differently from the paper's text 2004.09456 §7.1. Card: StereoSet.

Bias in generated answers can also be measured with a hand-built test set. BBQ pairs an ambiguous context, where "unknown" is the only correct answer, with a disambiguated context in which the answer is stated 2110.08193 §3. Its bias score is twice the share of non-unknown answers that follow the stereotype, minus one, and it is scaled by one minus accuracy in ambiguous contexts 2110.08193 §5. The set has 58,492 examples across nine social-bias categories and two intersectional ones, in US English only 2110.08193 §3 2110.08193 tab. 3. The best reported model, a UnifiedQA-11B run in its RACE format, reaches 77.8% accuracy 2110.08193 §6. The authors warn that a low bias score does not show a model is unbiased, because each category rests on only 25 templates 2110.08193 §7 2110.08193 §9. Card: BBQ.

Where the bias enters matters for the fix. A study of political lean shows that continued pretraining on partisan news or social-media text moves a language model's leaning, mostly by small amounts, and that these changes alter downstream hate-speech and misinformation classifiers by target group and by news source 2305.08283 §4.1 2305.08283 §4.2. Overall scores differ only slightly, so the group breakdown is the informative part 2305.08283 §4.2. The partisan checkpoints are not released, so the result can be reproduced only with a new pretraining run 2305.08283 Ethics Statement. Card: Political biases in pretraining to downstream tasks.

Language models: map the bias to a stage

A review of bias in large language models splits the problem along two axes. Intrinsic bias sits in the pretrained model and does not depend on a task. Extrinsic bias shows up in downstream task results and outputs 2411.10915 §2 2411.10915 §3. The intrinsic sources it names are the training corpus, the collection method, the language itself, and tokenisation with pretrained embeddings 2411.10915 §2.1 2411.10915 §2.4.

Evaluation is grouped by what it looks at: the data, the model, the output, or a human reviewer. Data-level checks cover group representation, stereotype detection in text and annotator agreement. Model-level checks cover fairness metrics, interpretability tools and counterfactual fairness. Output-level checks cover counterfactual tests on generated text, toxicity and sentiment, and acceptance rates. Human review needs a written protocol with a rubric and an agreement statistic 2411.10915 §4.1 2411.10915 §4.2 2411.10915 §4.3 2411.10915 §4.4.

Mitigation is grouped by when it acts 2411.10915 §5:

• Before training. Counterfactual data augmentation or substitution swaps attribute words in the corpus, for example "he" for "she" in a sentence about a professor 2411.10915 §5.1.
• During training. An equalizing loss and a declustering loss added to BERT fine-tuning, pruning of attention heads, and dropout with extra pretraining 2411.10915 §5.2.
• After training. Decoding-time methods such as Self-Debias, the projection methods SENT-DEBIAS and INLP, and causal prompting, which changes the prompt without touching weights or logits 2411.10915 §5.3.

The review's cost table rates the pre-training option as low and the in-training equalizing loss as high, and it rates Self-Debias as low because it needs only inference. The authors call these estimates approximate and give no source for them 2411.10915 tab. 5. The review runs no experiments of its own, so its evidence comes from the works it cites 2411.10915 §1. Card: Bias in Large Language Models: Origin, Evaluation, and Mitigation.

Fairness together with noisy data and robustness

Group membership and labels are often noisy, and a fix for one can hurt the other 2112.06409 §5.3:

• Fairness methods that tolerate noisy or missing group membership, such as distributionally robust optimization and adversarially reweighted learning 2112.06409 §5.3.
• FR-Train, which trains a classifier against a fairness discriminator and a robustness discriminator 2112.06409 §5.3.
• Adaptive sample selection, which trains only on samples that are both clean and fair 2112.06409 §5.3.

Removing noisy samples can worsen bias, because it can remove too many examples from an underrepresented group 2112.06409 §5.3. See also noisy labels for the label-cleaning side.

Organizational measures

Fairness is also a lifecycle task. The survey covers fairness-related documentation in datasheet and factsheet style 2110.01167 §3.5.1, auditing 2110.01167 §3.5.2, and accountability for the design and operation of the system 2110.01167 §2.7.

A risk taxonomy comes before any metric. Weidinger et al. list 21 ethical and social risks of language models in six areas. The first area covers social stereotypes and unfair discrimination, exclusionary norms, toxic language, and lower performance for some languages and social groups. For each risk the paper gives the mechanism, the harm types and where the harm starts [arxiv:2112.04359 §2.1] 2112.04359 tab. 1. It reports no experiments and offers no method; it is a checklist for risk assessment 2112.04359 §1. Card: Ethical and social risks of harm from Language Models.

Evidence that these checks are rarely done comes from a narrower field. A scoping review of 118 studies that use language models in education finds that privacy and equality are rarely addressed 2303.13379 §4.2 2303.13379 §4.3. Its results table reports no group-level metric, and the review runs no experiments of its own 2303.13379 §3.1 2303.13379 §4. Card: Practical and Ethical Challenges of Large Language Models in Education.

Explanations are not a fairness fix. An explanation can show which learned features a decision relied on, and that helps a separate bias analysis. The explanation survey states that explanations alone do not solve bias or unfair outcomes, and that other measures, such as regulation, are needed 1803.07517 §6 1803.07517 §7. It is a book chapter with no experiments, so use it for framing 1803.07517 §7.

When to use which

Implementation sketch

Every variant needs the same audit first: a per-group gap for the chosen criterion. The sketch below is a plain version of that check. The gap is our own simplification, and the survey does not define one.

# Per-group rates for a binary classifier  [arxiv:2112.06409 §5.1]
# Input: y (true labels), y_hat (predictions), z (group of each row)
# Output: gap per criterion; 0 means equal across groups

def fairness_gaps(y, y_hat, z):
    groups = unique(z)
    # Positive-prediction rate per group: demographic parity  [§5.1]
    rate = {g: mean(y_hat[z == g]) for g in groups}
    # True-positive rate per group: the separation criterion  [arxiv:2110.01167 §2.5]
    tpr = {g: mean(y_hat[(z == g) & (y == 1)]) for g in groups}
    # Precision among predicted positives: predictive parity  [§5.1]
    ppv = {g: mean(y[(z == g) & (y_hat == 1)]) for g in groups}
    return {
        "demographic_parity": spread(rate),
        "true_positive_rate": spread(tpr),
        "predictive_parity": spread(ppv),
    }

def spread(values):
    # Largest minus smallest group value  [own simplification]
    return max(values.values()) - min(values.values())

Report the gaps next to the accuracy of the same model. A small gap bought with a large accuracy loss is a decision the team should record, not a default.

Contradictions

• Scope of the impossibility result. The lifecycle survey states that independence and separation are incompatible when the sensitive variable is linked to the true outcome 2110.01167 §2.5. The data-centric survey states that no two of independence, separation and sufficiency can be fully satisfied together, with no such condition 2112.06409 §5.1. Both statements are cited as given. Check the condition in the source before relying on either.

• Fairness and noisy data. The survey reports that removing noisy samples can make bias worse 2112.06409 §5.3, and it also proposes adaptive sample selection that uses only clean and fair samples 2112.06409 §5.3. These are two different methods, not a settled answer. The paper does not compare them.

• Within one paper (gender tag). The text of Vanmassenhove et al. calls the Portuguese and Greek gains significant, but Table 2 does not star them and stars only French, Italian and Danish as gains 1909.05088 §5 1909.05088 tab. 2. The table is cited above. Their significance threshold also differs between Table 2 and Table 3 captions 1909.05088 tab. 2 1909.05088 tab. 3.

• Within one paper. FairFed's text reports percentage gains that the paper's own table does not reproduce; the table values are the ones to cite 2110.00857 §5.2.1 2110.00857 tab. 1.

• Within one paper (Ditto). Ditto's clean FEMNIST accuracy is 0.834 (0.09) in Table 1 and 0.836 (0.10) in Table 2, and its A1 50% value differs in the same way. The λ used for Table 2 is not stated, so the gap cannot be traced 2012.04221 tab. 1 2012.04221 tab. 2.

• Within one paper (tabular survey). The text of Le Quy et al. says Bank marketing has the worst statistical parity, but Table 15 gives 0.0153 for it, and the largest absolute values are for Dutch census and COMPAS 2110.00530 §4.2 2110.00530 tab. 15. The table values are the ones to cite.

• Group and individual measures. Binns argues that their apparent conflict is not one of principle, and that real conflicts come from paired variants that rest on opposite assumptions 1912.06883 §3 1912.06883 §5.3. The surveys above state that some group criteria cannot all hold together 2110.01167 §2.5 2112.06409 §5.1. The page reports both and does not reconcile them.

None of the sources compares the mitigation methods numerically. The WinoMT numbers measure bias in translation systems; they do not rank the methods above 1906.00591 §3.2.

Papers

• Trustworthy AI: From Principles to Practices: survey that places fairness, with the three group principles, in the AI lifecycle.
• StereoSet: Measuring stereotypical bias in pretrained language models: crowdsourced context association tests in four domains, scored as lms, ss and ICAT.
• Ethical and social risks of harm from Language Models: taxonomy of 21 risks in six areas, with mechanisms and origins; no method or experiments.
• Causal Machine Learning: A Survey and Open Problems: survey that defines counterfactual and path-specific fairness on a structural causal model.
• Data-centric AI survey of data collection and quality: survey with fairness measures and pre-, in- and post-processing methods.
• Evaluating Gender Bias in Machine Translation: WinoMT challenge set and an automatic gender-bias metric for translation output.
• Explanation Methods in Deep Learning: Users, Values, Concerns and Challenges: book-chapter survey of explanation needs; explanations do not solve bias on their own.
• FairFed: Enabling Group Fairness in Federated Learning: server-side aggregation that reweights clients by how close their local gap is to the global gap.
• Ditto: Fair and Robust Federated Learning Through Personalization: one personal model per device, pulled toward the global model; reports even accuracy across devices and robustness to attacks.
• Getting Gender Right in Neural Machine Translation: speaker-gender tag on the English source; small, mixed BLEU changes by language pair.
• BBQ: A Hand-Built Bias Benchmark for Question Answering: QA benchmark with ambiguous and disambiguated contexts and a bias score.
• AI and the Everything in the Whole Wide World Benchmark: position paper on construct validity; argues for scoped benchmarks and disaggregated analysis.
• From Pretraining Data to Language Models to Downstream Tasks: traces political lean from pretraining text to group-level differences in downstream classifiers.
• Practical and Ethical Challenges of Large Language Models in Education: scoping review of 118 education studies; equality and privacy are rarely addressed, with no group-level metric.
• DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models: trust benchmark with a fairness test on Adult; demonstration base rate shifts predictions.
• Lipstick on a Pig: shows that two embedding debiasing methods hide gender bias on the he-she projection while the structure remains.
• Reading Race: AI Recognises Patient's Racial Identity In Medical Images: CNNs predict self-reported race from medical images, across hospitals; the mechanism is not found.
• Proportionally Representative Clustering: defines PRF for centroid selection and gives SEAR, a polynomial-time rule; theory only.
• Bias in Large Language Models: Origin, Evaluation, and Mitigation: review that splits LLM bias into intrinsic and extrinsic and groups evaluation and mitigation by stage; no experiments of its own.
• Cultural Bias and Cultural Alignment of Large Language Models: places five GPT models on a survey-based cultural map; cultural prompting moves most, not all, countries closer.
• On the Apparent Conflict Between Individual and Group Fairness: philosophical argument that individual and group measures conflict only through unstated assumptions; no experiments.
• Pros and Cons of GAN Evaluation Measures: New Developments: survey of GAN evaluation measures, with fairness and bias as one group; no new experiments.
• Causal Inference in Natural Language Processing: survey that covers counterfactual swaps of protected attributes and the limits of counterfactual augmentation for text.
• Beneath the Tip of the Iceberg: position paper on open sentiment-analysis problems, including bias and a templated test corpus; no experiments.
• A survey on datasets for fairness-aware machine learning: 15 tabular datasets with group rates and fairness gaps for logistic regression; flags zero gaps from constant predictions.
• Gender bias in LLM reference letters, with the paper's own title in quotation marks: "Kelly is a Warm Person, Joseph is a Role Model". The paper tests gender gaps in ChatGPT and Alpaca letters in word choice, tone and invented detail.