A survey on datasets for fairness-aware machine learning

In plain words

Many methods try to make a learned model treat groups of people fairly, for example men and women or different races. To test them, researchers need datasets, and they have been picking them from scattered sources without a shared overview. This paper collects the public tabular datasets that fairness researchers use most often, from banking, criminal justice, healthcare, job promotion and schools.

For each dataset the authors describe its size, its columns, its class balance and the attributes that are normally treated as sensitive. They also draw a graph of which columns depend on which, so you can see whether the sensitive attribute is linked to the outcome, directly or through other columns. Then they train one simple classifier on each dataset and measure how accurate it is and how unequal its errors are across the groups.

The main message is that the datasets differ a lot, and that a fairness score can look good for a bad reason: a few numbers look fair only because the classifier ignores one class. The paper argues that researchers should pick datasets that match their task and should also build more benchmark datasets.

TL;DR

• A survey of 15 real-world tabular datasets used in fairness-aware ML, with a table of their size, domain, class ratio and protected attributes tab. 1.
• Per dataset: a Bayesian network over the attributes, then logistic regression on one 70/30 split, scored by accuracy, balanced accuracy, statistical parity, equalized odds and ABROCA tab. 15 §4.1.
• The paper gives no new method and no numbers to beat; its numbers are a baseline for logistic regression only. Read the zero fairness values with care (see Pitfalls).

Problem

Fairness-aware methods are usually evaluated on a few benchmark datasets. The paper says no overview of these datasets existed §1. A dataset is included only if it is public, appears in a known repository, and was used in at least three fairness-related papers §2.1. The conclusion says the fairness notion has to be chosen for each problem and domain, since none is universal §6.

Method

The paper has no training method of its own. Its analysis has three parts, applied to each dataset.

1. Dataset characterisation. Record instances, attributes (categorical, binary, numerical), class ratio, domain, protected attributes, target and collection period and place §3, tab. 1. Records are cleaned as described in each dataset subsection (for example COMPAS keeps 6,172 of 7,214 rows §3.2).
2. Bayesian network. Learn a directed graph over the attributes. Numerical attributes are discretised first §2.2. The class attribute is constrained to be a leaf node, so that the analysis asks which features determine the class §2.2, eq. 2. A direct or indirect edge from a protected attribute to the class is read as a sign that the data may be biased §2.2.
3. Logistic regression and fairness scores. Train logistic regression on a single 70% split, test on 30%, and report accuracy, balanced accuracy, group true positive and true negative rates, statistical parity, equalized odds and ABROCA §4.1, eq. 8–13, eq. 4, eq. 6, eq. 7.

Measures, as defined in the paper, for a binary label, a protected group s and its complement (non-protected group):

• Statistical parity: SP = P(ŷ=+ | non-protected) − P(ŷ=+ | protected) eq. 4. Zero means no gap; positive means the protected group receives fewer positive predictions §2.3.1.
• Equalized odds violation: the sum over the two true labels y of |P(ŷ=+ | protected, y) − P(ŷ=+ | non-protected, y)|, which lies in [0, 2] eq. 6 §2.3.2.
• ABROCA: the integral over false-positive rate from 0 to 1 of the absolute difference between the two groups' ROC curves eq. 7 §2.3.3.

Pseudocode for one dataset, as the authors' notebooks run it:

# Per-dataset analysis  [§2.2] [§4.1]
# Input:  table with label column, protected column S
# Output: BN structure, accuracy and fairness scores, ABROCA value

def analyse_dataset(table, label, protected):
    # 1. Discretise numerical columns for the network  [§2.2]
    #    bins are dataset-specific; see Hyperparameters
    bn_table = discretise(table)
    # 2. Learn a structure in which the class node is a leaf  [§2.2] [eq. 2]
    network = learn_structure(bn_table, class_node=label)
    # 3. Encode categorical columns as integer codes
    #    [code: experiments/Fair-metrics.ipynb#L45-L48]
    features = label_encode(table.drop(columns=[label]))
    # 4. One split: 30% test, fixed seed
    #    [code: experiments/Fair-metrics.ipynb#L53]
    train, test = split(features, label, test_size=0.3, seed=42)
    # 5. Fit logistic regression with library defaults
    #    [code: experiments/Fair-metrics.ipynb#L594]
    model = fit_logistic_regression(train, seed=0)
    scores = model.predict_proba(test)
    preds = model.predict(test)              # threshold 0.5 (library default)
    # 6. Accuracy and group rates  [§4.1] [eq. 8–13]
    report = group_metrics(preds, test.labels, test[protected])
    # 7. ABROCA on predicted probabilities  [eq. 7]
    report.abroca = abroca(scores, test.labels, test[protected])
    return network, report

Pseudocode for the fairness gaps, with the sign convention of the paper:

# Fairness gaps  [eq. 4] [eq. 6]
# Input:  preds, labels in {0, 1}; prot is True for the protected group

def fairness_gaps(preds, labels, prot):
    # 1. Positive-prediction rate per group  [eq. 4]
    rate_prot = mean(preds[prot])
    rate_non = mean(preds[not prot])
    statistical_parity = rate_non - rate_prot
    # 2. Per-label prediction rates per group  [eq. 5] [eq. 6]
    tpr_gap = abs(tpr(prot) - tpr(non_prot))   # labels == 1
    tnr_gap = abs(tnr(prot) - tnr(non_prot))   # labels == 0
    equalized_odds = tpr_gap + tnr_gap
    return statistical_parity, equalized_odds

Algorithm for ABROCA, as defined in eq. 7 and implemented in the authors' helper (see Paper vs code):

# ABROCA  [eq. 7] [code: compute_abroca.py#L12-L103]
# Input:  scores, labels, prot; majority group value
# Output: area between the two ROC curves over FPR in [0, 1]

def abroca(scores, labels, prot, majority):
    # 1. ROC curve for each group  [code: utils.py#L7-L15]
    fpr_a, tpr_a = roc_curve(labels[prot == majority], scores[...])
    fpr_b, tpr_b = roc_curve(labels[prot != majority], scores[...])
    # 2. Interpolate both on a 10,000-point FPR grid  [code: utils.py#L29-L41]
    grid = linspace(0, 1, 10000)
    # 3. Integrate the absolute difference over [0, 1]  [eq. 7]
    gap = abs(interp(grid, fpr_a, tpr_a) - interp(grid, fpr_b, tpr_b))
    return integrate(gap, over=grid)

The Bayesian-network structure learning is the least specified step: the paper gives the objective with a penalty term but no value for the penalty and no named algorithm §2.2, eq. 2. The COMPAS notebook uses an exact search with a constraint graph (see Paper vs code).

Hyperparameters

Results

Author-reported. Setup: 15 datasets from tab. 1, one protected attribute per dataset, logistic regression, one 70/30 split §4.1. Hardware and run time are not stated. The baseline is logistic regression itself: the authors say the numbers "can be considered as the baseline" for future work §4.2. Each dataset is listed with its protected attribute, the group distribution [s+, s−, s̄+, s̄−] in percent, and the metrics tab. 15.

Table A: predictive and fairness metrics (values as printed in tab. 15).

Table B: group rates from the same run tab. 15. TPR and TNR are per protected (prot.) and non-protected (non-prot.) group.

Reading the paper's own summary §4.2, checked against tab. 15: 10 of 15 datasets have |statistical parity| below 0.1; equalized odds is 0.0 for four datasets; balanced accuracy is worst for Credit card clients, Diabetes and OULAD; Student-Mathematics has the best balanced accuracy.

Paper vs code

The official repository is tailequy/fairness_dataset, linked from the paper §4.1, footnote. Its owner handle matches the first author, Tai Le Quy, and its README cites this paper [README.md]. It contains the cleaned datasets and two notebooks (Fair-metrics.ipynb, Compas-Final.ipynb) plus ACS PUMS data.ipynb. The repository has no LICENSE file, so no code may be quoted; references are by path and line only.

• Equalized odds. Consistent. The paper's equation eq. 6 is the sum of the TPR and TNR gaps. calculate_performance computes exactly that my_useful_functions.py. A sibling function, calculate_performanceEQOP, returns only the TPR gap my_useful_functions.py. The notebook's "Equalized odds" line calls calculate_performance, so the table follows the paper's definition. The sum of the TPR and TNR gaps matches the printed values in the Adult row (0.0187 + 0.0095 ≈ 0.0281).
• Statistical parity. Consistent. Code computes non-protected rate minus protected rate my_useful_functions.py, matching eq. 4. Code returns the signed value (no absolute value) my_useful_functions.py.
• ABROCA. The paper defines the measure as an integral over [0, 1] eq. 7. The code interpolates both ROC curves onto 10,000 points, integrates the absolute difference with scipy quad and a limit of 1000 compute_abroca.py compute_abroca.py. The notebook passes majority_protected_attr_val = 1, so the protected group is the other coded value Fair-metrics.ipynb.
• Classifier. The paper says only "logistic regression" §4.1. The code uses LogisticRegression(random_state=0) with library defaults and no feature scaling Fair-metrics.ipynb.
• Split. Consistent: 70/30, single split §4.1 and test_size=0.3 Fair-metrics.ipynb. The code fixes random_state=42; the paper does not give a seed.
• Categorical encoding. The paper does not say how categorical attributes are encoded for logistic regression. The code applies LabelEncoder to every object column Fair-metrics.ipynb, which gives alphabetical integer codes.
• Bayesian network. The paper gives eq. 2 with a penalty γ and a leaf constraint on the class, but no algorithm and no value for γ §2.2. The COMPAS notebook calls BayesianNetwork.from_samples(..., algorithm='exact') Compas-Final.ipynb and then a version with a constraint_graph Compas-Final.ipynb. The constraint graph, as read from the code, allows edges from the eight feature nodes into the class node and among the features, and none out of the class. The paper does not name the library or the score. No BN code is shipped for the other datasets; the README lists only COMPAS and ACS PUMS.
• Discretisation. The paper says each dataset's discretisation is given in its section §2.2; for COMPAS the text gives none. The code bins priors count into 0, 1 to 5 and above 5, and juvenile crime into 0 or above 0 Compas-Final.ipynb.
• Communities & Crime and OULAD. Table 1 lists both as multi-class tab. 1. The metric functions treat label 1 as positive and the notebook does not binarise OULAD's final_result Fair-metrics.ipynb. The Communities binarisation line is commented out Fair-metrics.ipynb. How these two rows were made binary for tab. 15 is not stated; the card does not know which label was used.
• Internal inconsistencies in the paper.
  • Text says Bank marketing has "the worst value" of statistical parity §4.2; tab. 15 has 0.0153 for Bank marketing. The largest absolute values are COMPAS recid. (−0.3398) and Dutch census (0.3568).
  • Text says ABROCA is best for Ricci, "followed by the OULAD and the KDD Census-Income datasets" §4.2; in tab. 15 KDD is 0.0074 and OULAD 0.0088, so KDD is the smaller of the two.
  • Student-Mathematics has 649 instances in tab. 1 and 395 in the text §3.4; Student-Portuguese has a class ratio of 1:5.49 in tab. 1 and 1:5.09 in the text.

Repository searched: the paper's footnote and the first author's account via the README link. No other official code was found; the card uses this repository.

Pitfalls

• Degenerate predictions give "fair" zeros. Credit card clients and Diabetes predict only the negative class (TPR 0, TNR 1 in both groups), and OULAD predicts only the positive class (TPR 1, TNR 0). Their statistical parity and equalized odds are 0.0 and balanced accuracy is 0.5 tab. 15. The paper calls 0.0 "the best value" §4.2; for these rows it reflects a constant classifier, not a fair one.
• Tiny test set for Ricci. Ricci has 118 rows tab. 1; a 30% test split has about 36 rows. Accuracy 1.0 and balanced accuracy 1.0 are from a very small sample, and the single split gives no variance.
• One split, one seed. All numbers come from one split with seed 42 §4.1 Fair-metrics.ipynb. Differences of around 0.01 in ABROCA or equalized odds can move with a different split, and the paper reports no variance to judge how much.
• Label encoding treats nominal codes as numbers. Logistic regression sees alphabetical integer codes for categorical columns, not one-hot features Fair-metrics.ipynb. Coefficients and therefore the group gaps depend on that arbitrary order.
• No feature scaling, default solver. Features are unscaled and the solver and iteration cap are library defaults Fair-metrics.ipynb. The notebook sets warnings.filterwarnings('ignore') Fair-metrics.ipynb, so a non-converged fit would not be reported.
• Protected group is coded 0. Group membership is set by the alphabetical code (for example Female = 0 for Adult) Fair-metrics.ipynb. Reusing the notebook on data with another encoding flips the sign of statistical parity.
• Bayesian network is a screening step, not a causal result. The paper reads a direct or indirect edge from a protected attribute to the class as a possible bias §2.2. The learned edges depend on discretisation, the penalty and the search, none of which the paper fixes.
• Sensitive-attribute choice. Each dataset is scored on one protected attribute §6. Other attributes in tab. 1 (for example age in Adult) are not evaluated.
• No run time. Neither the paper nor the notebooks report how long the network search or the fits take.
• Old data. Most datasets were collected 20 or more years ago, and the paper says the oldest is 48 years old §6. The ACS PUMS data added in §5.1 covers California 2007 to 2019 and is not scored in tab. 15.

Lineage

• Builds on: the fairness notions used here are statistical parity (Dwork et al., 2012), equalized odds (Hardt et al., 2016) and ABROCA (Gardner et al., 2019), each cited and defined in §2.3. No card for these is in the wiki.
• Compared against: none. The paper reports logistic regression only and asks future work to use these numbers as a baseline §4.2.
• Related: the Adult reconstruction and ACS PUMS datasets of Ding et al. (2021) are discussed in §5.1. The sequential and synthetic dataset issues are in §5.2 and §5.3.
• Superseded by: not stated.