---
kind: concept
slug: debiasing-word-embeddings
title: Debiasing word embeddings
summary: >-
  Removes gender bias from word vectors by shrinking their projection on the he-she direction. The paper tested two methods, Hard-Debiased and GN-GloVe, and found that gender stays recoverable from neighbours, clusters and a classifier. Judge a debiasing step with several checks, not the projection score alone.
tags: [word-embeddings, bias, debiasing, fairness]
papers: ["1903.03862"]
related: [algorithmic-fairness, text-embedding]
status: draft
updated_at: 2026-10-19
---

## Overview

A word embedding is a vector per word, learned from text. Earlier debiasing
methods pick one direction in the space, the he-she direction, and shrink
each word's projection on it. A word's bias is that projection, with all
vectors normalised [arxiv:1903.03862 §2]. The invariant the methods aim for is
a projection near zero for gender-neutral words.

The paper's argument is that this measure is too narrow. Stereotyped words
keep their relations to each other after the projection is removed, so a
downstream model can still learn gender from neighbourhoods [arxiv:1903.03862
§2] [arxiv:1903.03862 §5]. Its practical rule is to judge a debiased embedding
by several tests, not by the projection alone [arxiv:1903.03862 §4].

## Variants

Only two debiasing methods are tested in the paper. Both are compared with
their own biased starting point [arxiv:1903.03862 §3].

### Hard-Debiased (post-processing)

Bolukbasi et al.'s method, applied to a finished embedding. The paper compares
it with the original word2vec embedding trained on Google News
[arxiv:1903.03862 §3]. Its projection bias falls, but gender stays visible: the
k-means accuracy on the 1,000 most biased words drops from 99.9% to 92.5%, and
the SVM gender accuracy from 98.25% to 88.88% [arxiv:1903.03862 §4] [fig. 1].
Card: [Gonen and Goldberg, 2019](/papers/1903.03862).

### GN-GloVe (during training)

Zhao et al.'s method, which debiases while the embedding is trained. The paper
compares it with a standard GloVe model trained on the same corpus
[arxiv:1903.03862 §3]. Gender also stays visible after this method: the k-means
accuracy falls from 100% to 85.6%, and the SVM accuracy from 98.65% to 96.53%
[arxiv:1903.03862 §4] [fig. 1]. Card: [Gonen and Goldberg, 2019](/papers/1903.03862).

### Adversarial debiasing (not tested)

The paper mentions Zhang et al.'s adversarial method in a footnote, but does
not test it [arxiv:1903.03862 §2]. Nothing here says how it behaves.

## Checks after debiasing

The paper runs five checks on both debiased spaces. Each one can be run on
any debiased embedding, given the original embedding as a reference
[arxiv:1903.03862 §4].

1. **Clustering.** Split the most biased words with k-means and score the
   split against the original gender labels.
2. **Neighbour bias.** Count each word's male-biased neighbours, using the
   original bias as the label, and correlate that count with the original bias.
3. **Professions.** Repeat check 2 on a list of occupations.
4. **Association tests.** Run three WEAT-style tests of names against
   attribute word sets, on the debiased space only [arxiv:1903.03862 §4].
5. **Classifier.** Train an RBF-kernel SVM to predict gender from the vectors
   and report accuracy on held-out words [arxiv:1903.03862 §4].

## When to use which

The paper does not give a recommended method. It tests two and reports what
each leaves behind. The rows below follow from the constraints a team has.

| Situation | Use | Why |
|---|---|---|
| The embedding is fixed and you can only transform it | Hard-Debiased-style post-processing | This is the tested post-processing method [arxiv:1903.03862 §2] [§3] |
| You train the embedding yourself | GN-GloVe-style debiasing during training | This is the tested in-training method [arxiv:1903.03862 §2] [§3] |
| You need to know whether a debiasing step worked | Run all five checks, not only the projection | Both methods kept signal on every check [arxiv:1903.03862 §4] |
| You need a bias-free claim for a downstream system | Do not make the claim from the projection alone | The paper argues the projection hides bias rather than removing it [arxiv:1903.03862 §5] |

The card records no comparison of training cost or runtime between the two
methods, so this page cannot use one to choose between the first two rows.

## Implementation sketch

The shared core of the checks: a projection score per word, a neighbour count
that uses the original labels, and an agreement score. Variants differ in how
the debiased vectors are made, not in these checks.

```pseudocode
# Projection bias on unit vectors, positive = male-leaning  [arxiv:1903.03862 §2]
def projection_bias(w, vectors, he, she):
    return dot(vectors[w], he) - dot(vectors[w], she)

# Male-biased neighbour count; labels come from the ORIGINAL space  [§4]
def male_neighbour_count(w, debiased, bias_before, k=100):
    # The paper says "k nearest neighbours" without a value; 100 is the
    # value used in the released code (see Pitfalls)
    count = 0
    for n in nearest_by_cosine(w, debiased, k):
        if bias_before[n] > 0:
            count += 1
    return count

# Agreement between original bias and neighbour count  [arxiv:1903.03862 §4]
def neighbour_agreement(words, bias_before, count):
    return pearson_r([bias_before[w] for w in words],
                     [count[w] for w in words])

# Held-out gender accuracy of an RBF-kernel SVM  [arxiv:1903.03862 §4]
def gender_probe_accuracy(train, test, vectors):
    # Default SVM settings; use one split for the original and debiased runs
    model = fit_svm_rbf(vectors, train)
    return mean(model.predict(test_vectors) == test_labels)
```

## Contradictions

None known among the cards in this repository. The paper's argument runs
against the projection-only measure used by the methods it tests, which
treats a shrunk projection as removed bias [arxiv:1903.03862 §2]. Both
sides are cited above.

## Papers

- [Gonen and Goldberg, 2019](/papers/1903.03862): tests Hard-Debiased and GN-GloVe with five checks and shows that gender stays recoverable after each.