A Unified Approach to Quantifying Algorithmic Unfairness: Measuring Individual & Group Unfairness via Inequality Indices

In plain words

A fairness check for a yes/no decision system usually asks whether two groups get the same treatment. This paper asks a different question: given two unfair systems, which one is more unfair, and by how much? Its answer borrows a tool from economics that measures how unequally income is spread. Each person gets a score for how well the system treated them, and the spread of those scores is the unfairness number. A person who was wrongly given the good outcome and a person who was wrongly denied it both count as treated unequally, but in different amounts.

The same number can also be split into two parts: how much the average treatment differs between groups, and how much people differ inside each group. The authors show that a method which pushes down only the between-group part can leave the inside-group part and the total larger. The measure is computed from predictions and true labels alone, so it can compare models trained in different ways.

TL;DR

Use the generalized entropy index with alpha = 2 over benefit values built from prediction and label, b = yhat - y + 1, as a single unfairness score for a binary classifier §2.2 eq. 1 eq. 2. Its decomposition into between-group and within-group terms §3.2 eq. 5 is the main tool for the trade-off argument. Reported results are on Adult and COMPAS with logistic regression, RBF-SVM and random forest §4. No official code was found, so the experiment details are as stated in the paper.

Problem

Fairness conditions such as equal false positive rates or equal acceptance rates answer a yes/no question, and they often cannot all hold at once §1. Designers need a number that ranks unfair algorithms, and that number should be comparable across models and learning methods §1. Most fair-learning methods reduce only between-group disparity §2.4, and they can leave individual or within-group unfairness high §3.2 §4.3. The paper's question is how to measure total unfairness and how it breaks down across groups §1 §2.4.

Method

Setting. A labelled training set of n people is given. Each person has a true label y in {0, 1} and a prediction yhat in {0, 1} from a classifier §2.1. A sensitive attribute, such as sex or race, defines disjoint groups §2.1.

Step 1: benefit per person. Map each (yhat, y) pair to a non-negative benefit b = yhat - y + 1 §2.2 eq. 1. This gives 2 for a false positive, 1 for a true positive or true negative, and 0 for a false negative §2.2 fig. 2. The authors note that these values reflect one choice of ordering and that the absolute values could be set differently §2.2.

Step 2: inequality index. Apply the generalized entropy index to the benefit vector, with mean benefit mu:

E^{\alpha}(b_1, \dots, b_n) = \frac{1}{n\alpha(\alpha-1)} \sum_{i=1}^{n} \left[ \left(\frac{b_i}{\mu}\right)^{\alpha} - 1 \right] eq. 2

alpha must not be 0 or 1 in this formula §2.3. The index is anonymous, population-invariant, satisfies the transfer principle and is zero when all benefits are equal, and it is scale-invariant and subgroup-decomposable §2.3. Among differentiable indices with population and scale invariance, the generalized entropy family is the only one, as cited by the authors §2.3. The paper uses alpha = 2, which equals half the squared coefficient of variation §4 eq. 2 with alpha = 2.

Step 3: decomposition. For a partition of people into groups g with n_g members and group mean mu_g, the index splits as a within-group sum plus a between-group sum §3.2 eq. 5:

• within-group term: sum over g of (n_g / n) * (mu_g / mu)^alpha * E^alpha(b^g), where b^g are the benefits of group g §3.2;
• between-group term: sum over g of n_g / (n * alpha * (alpha - 1)) * [ (mu_g / mu)^alpha - 1 ] §3.2.

Pseudocode for steps 1 to 3 (the paper's formulas, in our own code layout):

# Individual unfairness of a binary classifier  [§2.2] [eq. 1] [eq. 2]
# Input:  predictions (n,) in {0, 1}; labels (n,) in {0, 1}; alpha not 0 or 1
# Output: scalar unfairness

def individual_unfairness(predictions, labels, alpha):
    # Benefit per person: yhat - y + 1, values in {0, 1, 2}  [§2.2] [eq. 1]
    benefits = predictions - labels + 1
    n = len(benefits)
    # Mean benefit, the denominator in the index  [§2.2]
    mean_benefit = sum(benefits) / n
    # Generalized entropy over the benefit vector  [§2.3] [eq. 2]
    terms = (benefits / mean_benefit) ** alpha - 1
    return sum(terms) / (n * alpha * (alpha - 1))

# Split the index into within-group and between-group parts  [§3.2]
# Input:  benefits (n,) >= 0; groups (n,) integer labels; alpha
# Output: within-group part, between-group part (they sum to the total)

def decompose_unfairness(benefits, groups, alpha):
    n = len(benefits)
    mean_benefit = sum(benefits) / n
    within = 0.0
    between = 0.0
    for each group g in set(groups):
        members = benefits[groups == g]
        n_g = len(members)
        mu_g = sum(members) / n_g
        ratio = (mu_g / mean_benefit) ** alpha
        # Between-group term for g  [§3.2]
        between += n_g / (n * alpha * (alpha - 1)) * (ratio - 1)
        # Within-group term: weighted generalized entropy of the group  [§3.2]
        within += (n_g / n) * ratio * generalized_entropy(members, alpha)
    return within, between

Here generalized_entropy(members, alpha) is eq. 2 applied to the group's own benefits with the group's own mean §3.2. The paper states the decomposition for generalized entropy only; it does not hold for every inequality index. The Gini index is not subgroup-decomposable §2.3.

Step 4: individual-level and group-level measures in the theory. Let I(theta) be the unfairness of model theta on the training set, with benefits 1 + theta(x_i) - y_i §3.1. The paper compares two optimisation problems: minimise I_beta + I_omega under an accuracy floor L >= delta eq. 3, and minimise only I_beta under the same floor eq. 4 §3.2.

Empirical threshold rule. To turn a model's scores p_i into labels, rank people by p_i (ties broken at random) and label person i as 1 when rank(i) >= n * tau, for a threshold tau in [0, 1] §4.1. The paper's "oracle" predicts the true label and has p_i in {0, 1} §4.1.

# Threshold a score by rank  [§4.1]
# Input:  scores (n,) = p_i; tau in [0, 1]
# Output: labels (n,) in {0, 1}

def label_by_rank(scores, tau):
    n = len(scores)
    # Rank people by score, ascending; ties broken at random  [§4.1]
    ranks = rank_with_random_ties(scores)
    # Label 1 when rank(i) >= n * tau; rank base not stated  [§4.1]
    return (ranks >= n * tau).astype(int)

Experimental protocol. Data are split 70/30 into train and test, ten times, and averages are reported; hyperparameters are chosen on a further 70/30 split of the training part §4. Three classifiers are trained for accuracy: logistic regression, SVM with RBF kernel, and random forest §4.1. The Zafar et al. covariance-constrained logistic regression is trained with covariance multipliers 1.00, 0.95, ..., 0.00 §4.3.

Worked toy example (checked in this card). Ten people in three groups, with two classifiers C1 and C2 shown in fig. 1. With b = yhat - y + 1 and alpha = 2, the benefit vector of C1 has mean 1 and index 4 / 20 = 0.2. For C2, it has mean 1 and index 6 / 20 = 0.3. These match the caption fig. 1.

Hyperparameters

Results

Author-reported. Setup: Adult and COMPAS (ProPublica) data, logistic regression, RBF SVM and random forest, 70/30 split repeated ten times §4. Baseline: the paper compares classifiers against each other and against an oracle, and does not report a separate published baseline method §4.1.

The theory gives three propositions: accuracy and fairness optima coincide only in the separable case §3.1; minimising between-group unfairness alone can increase within-group and overall unfairness §3.2; and the between-group share grows as groups are refined §3.2.

Paper vs code

No official code.

Search: the arXiv metadata and the LaTeX source contain no code link [raw/src/1807.00787/inequality_indices_strip.tex]. GitHub repository searches for "generalized entropy fairness", "unified approach algorithmic unfairness inequality index", "inequality fairness" with the author surname and "individual fairness inequality indices" returned no repository from the paper's authors. The only match, youngmijin/fermge, is a third-party project with no license and is not used. No project page was linked in the paper.

Pitfalls

• The benefit values 2, 1, 0 are a choice §2.2. The paper says other values could be used and does not test their effect. Changing them changes every number.
• The index needs all benefits non-negative and at least one positive §2.2. With b = yhat - y + 1 this always holds, but a different benefit map must be shifted to satisfy it.
• alpha must differ from 0 and 1 in eq. 2 §2.3. The paper's limits at alpha = 0 and 1 are not written out, so use an explicit formula for those cases if needed: not stated.
• Only the generalized entropy family is decomposed in the paper. The Gini index is not subgroup-decomposable §2.3; do not swap indices and expect eq. 5 to hold.
• The decomposition printed in §3.2 is the basis of all between/within results. It was not checked numerically in this card; check it on a small example before relying on it.
• Which data the unfairness is computed on is not stated: training, test, or both §4.
• The threshold rule breaks ties at random and does not state the rank base or a seed §4.1. Results on tied scores can vary between runs.
• Classifier hyperparameters (LR regularisation, SVM C and gamma, RF size) are not stated §4.1. The numbers in fig. 6 depend on them.
• Propositions need extra conditions: the first assumes the hypothesis class is closed under complements §3.1; the second assumes some feature vector has 0 < P(y = 1 | x) < 0.5 §3.1. The accuracy-fairness conflict is shown in the appendix example app..
• Appendix example (adding a feature can raise unfairness of the accuracy-optimal classifier): the paper prints 1.075 and 1.10 for p = 0.9, r = 0.2, eps = 0.001. Our recomputation gives about 1.074 and 1.109. The ordering holds; the printed decimals are rounded differently app..
• The source compiles the long-version branch, so the appendix is included app.. The commented-out "conclusion" block (\iffalse) is not part of the paper.
• Figure 6 and Figure 7 values are read from plots; the text gives only the endpoints quoted in the table.
• Measuring fairness this way can rank a less-accurate model as fairer. The paper shows the fairness optimum can be arbitrarily less accurate than the accuracy optimum app.. Report both.

Lineage

• Builds on: generalized entropy and the other inequality indices from economics, with their axioms §2.3. No arXiv id for these prior works is given in this card.
• Compared against: group-fairness conditions such as statistical parity, equal opportunity and calibration §2.4, and the Zafar et al. covariance-constrained training used in §4.3.
• Related: fairness gerrymandering over intersecting groups (Kearns et al.), cited for the intersectional decomposition §2.4 §3.2.
• Superseded by: not stated.