---
kind: concept
slug: target-encoding
title: Target encoding
summary: >-
  Replacing each level of a categorical feature with a number learned from the target, such as its mean outcome. Fitted on the rows it encodes, rare levels leak their own outcome. The study's best variant fits a random-intercept model and cross-fits the training rows. Author-reported: first for four of five learners in one lightly tuned benchmark, at much higher runtime with gradient boosting.
tags: [tabular, categorical-features, target-encoding, high-cardinality]
papers: ["2104.00629"]
related: [categorical-encoding, mixed-effects-model, gradient-boosted-decision-trees]
status: draft
updated_at: 2026-11-15
---

## Overview

Target encoding turns each level of a categorical feature into a number learned
from the outcome you want to predict. The simplest form is the mean outcome of
the level in the training data [arxiv:2104.00629 §1.2]. The number then goes
into the learner like any other numeric column.

The core risk is leakage. When a rare level's number is fitted on the same rows
that get encoded, the level's number mostly reflects its own outcome. The model
then trusts the feature too much and generalises poorly [arxiv:2104.00629 §1.2].
The two common fixes are regularisation, which pulls each level's number towards
the overall average, and cross-fitting, which encodes each training row with a
model that never saw that row [arxiv:2104.00629 §1.2, §2.7].

Only one card in this wiki covers this slug, so every statement below comes from
one benchmark study. For the wider family of encoders, see
[categorical encoding](/concepts/categorical-encoding).

## Variants

### Impact encoding (smoothed target mean)

Each level is replaced by its conditional target mean, pulled towards the
overall mean by a smoothing term. For classification the study uses one column
per class, on a logit link [arxiv:2104.00629 §2.6]. The study sets the smoothing
constant to 0.0001, which is almost no smoothing. The authors chose it so that
this variant serves as a plain target-encoding baseline, and they call it the
unregularised impact encoder in their results [arxiv:2104.00629 §2.6, §4.3].
A tuned smoothing value was not tested. The paper does not state how impact
encoding handles levels unseen at prediction time [arxiv:2104.00629 §2.6].

Without strong regularisation, impact encoding showed severe overfitting on the
two datasets with the most levels [arxiv:2104.00629 §4.2]. Card:
[Pargent et al.](/papers/2104.00629).

### GLMM encoding with cross-fitting

A random-intercept (mixed) model is fitted per feature. Each level's encoded
value is its conditional mode. Levels unseen at prediction time take the model's
fixed intercept [arxiv:2104.00629 §2.7]. The authors describe this as a
regularised version of impact encoding, with the amount of shrinkage set by the
model rather than by a tuned constant [arxiv:2104.00629 §2.7].

The study compares three settings:

- **No cross-fitting (glmm-noCV).** Training rows are encoded by the model that
  fitted them. It overfits like impact encoding on the high-cardinality datasets
  [arxiv:2104.00629 §4.2].
- **5-fold cross-fitting (glmm-5CV).** The headline variant. The prediction-time
  model uses all training rows, and each training row is encoded by the model
  that did not see its fold [arxiv:2104.00629 §2.7].
- **10-fold cross-fitting (glmm-10CV).** Often no better than 5-fold
  [arxiv:2104.00629 §4.3].

For multiclass problems the study fits one one-vs-rest model per class, so each
level gets one encoded column per class [arxiv:2104.00629 §2.7]. Card:
[Pargent et al.](/papers/2104.00629).

## When to use which

The numbers are the study's own and are author-reported on 24 OpenML datasets
with five learners, lightly tuned [arxiv:2104.00629 §3.3, §5.1].

| Situation | Use | Why |
|---|---|---|
| Accuracy matters most and a mixed-model fit per feature and fold is affordable | GLMM with 5-fold cross-fitting | Ranked first for every learner except SVM [arxiv:2104.00629 §4.3]. |
| No time to tune a smoothing constant on the whole pipeline | GLMM rather than impact | The authors say the shrinkage is set automatically [arxiv:2104.00629 §2.7]. This is their claim; the study does not test tuned impact. |
| Gradient boosting, tight runtime budget | Target-agnostic encoding such as frequency | Median runtime about 0.54 of one-hot, against 0.82 for impact and 7.21 for GLMM with 5-fold [arxiv:2104.00629 tab. 1]. |
| Cannot cross-fit | Do not use unregularised target encoding on high-cardinality features | The study found severe overfitting without regularisation or cross-fitting [arxiv:2104.00629 §4.2]. Use a target-agnostic encoder instead. |
| Impact encoding with a tuned smoothing value | Untested here | The study fixes the smoothing at 0.0001 [arxiv:2104.00629 §2.6]. Check the result yourself before relying on it. |
| Many classes | Budget for one model per class | GLMM fits one one-vs-rest model per class [arxiv:2104.00629 §2.7]. |
| Support vector machine | Check the figures before choosing | GLMM with cross-fitting did not rank first for SVM [arxiv:2104.00629 §4.3]. |

Runtime grows with cross-fitting. Median runtime relative to one-hot for
gradient boosting rose from 1.65 with no cross-fitting to 7.21 with 5 folds and
12.71 with 10 folds [arxiv:2104.00629 tab. 1]. Under the study's setup this is
the main cost of the best variant.

The study tuned the learners only lightly and ran 5-fold CV once. It also used
24 datasets, too few for the authors to link the ranking to dataset traits
[arxiv:2104.00629 §5.1, §5]. Do not assume the ranking transfers to tuned
pipelines or to your data.

## Implementation sketch

A generic cross-fitting wrapper, then a fitter for impact encoding. The wrapper
is the one the study uses with GLMM. Using it with impact encoding is an
extension the study did not test [arxiv:2104.00629 §2.6, §2.7].

```pseudocode
# Cross-fitting wrapper for any target-based level encoder  [arxiv:2104.00629 §2.7]
# Input:  levels (n,) categorical; target (n,)
#         fit_values(levels, target) -> (values per level, fallback for unseen)
#         K folds; K = 1 means no cross-fitting
def encode_train(levels, target, K, fit_values):
    # Encoder used at prediction time, fitted on all training rows  [§2.7]
    full = fit_values(levels, target)
    if K == 1:
        # Rows are encoded by the model that saw them: prone to leakage  [§4.2]
        return lookup(full, levels), full
    encoded = empty(n)
    # Out of fold: each row is encoded by a model that never saw its fold  [§2.7]
    for fold, rest in make_folds(n, K):
        values = fit_values(levels[rest], target[rest])
        encoded[fold] = lookup(values, levels[fold])
    return encoded, full
```

```pseudocode
# Impact fitter: per-level mean shrunk towards the global mean  [arxiv:2104.00629 §2.6]
# Regression shown; classification uses one column per class on a logit link
def fit_impact(levels, target, eps):
    # Overall mean, which the shrinkage pulls towards  [§2.6]
    global_mean = mean(target)
    values = {}
    for level in unique(levels):
        rows = levels == level
        # shrink() is not given as a formula in the paper; eps = 0.0001 in the study  [§2.6]
        values[level] = shrink(mean(target[rows]), count(rows), global_mean, eps)
    # Assumption: the paper does not state a fallback for unseen levels with impact  [§2.6]
    return values, global_mean
```

At prediction time, encode new rows with the full-data model and its fallback.
The ranking depends on how unseen levels are handled, and the paper does not
state that rule for impact encoding, so check it before comparing numbers
[arxiv:2104.00629 §2.6, §2.7].

## Contradictions

None known within this wiki. The study has one source, so no two papers
disagree. The released code differs from the paper in several places. It does
not include the impact encoder that its benchmark calls, so impact results
cannot be checked against code. The card lists the other differences, such as how
the high-cardinality threshold is counted. Those may change numbers reproduced
from the code [arxiv:2104.00629 §2.6]. See [Pargent et al.](/papers/2104.00629)
for details.

## Papers

- [Pargent et al.](/papers/2104.00629): compares regularised target encoding with cross-fitting against six other encoders on 24 datasets with five learners. It finds the GLMM variant with 5-fold cross-fitting ranked first for every learner except SVM (author-reported).
