---
kind: concept
slug: quantile-loss
title: Quantile loss
summary: >-
  Pinball loss for one target quantile, so a model can output a range such as P10 to P90 instead of one mean-like value. Used both to train models and to score predicted quantiles. One survey trains a Temporal Fusion Transformer with 14 losses; one review covers quantile regression, forests, boosting and networks.
tags: [loss-function, regression, forecasting, uncertainty, probabilistic, scoring-rules]
papers: ["2211.02989", "2209.08307"]
related: [time-series-forecasting, uncertainty-estimation, quantile-regression-forest, conformal-prediction, calibration, proper-scoring-rules]
status: draft
updated_at: 2026-11-15
---

## Overview

Quantile loss is the pinball loss for a target quantile $q$ between 0 and 1.
It weights the two kinds of error differently: when the target is above the
prediction, the error is weighted by $q$; when the prediction is above the
target, it is weighted by $1 - q$ [arxiv:2211.02989 §III.N]
[code: libs/utils.py#L67-L92]. At $q = 0.5$ it equals half the absolute
error, so the paper's claim that the median loss is MAE holds only up to a
factor of two [arxiv:2211.02989 §III.N].

The survey presents it as a way to predict intervals rather than point
estimates [arxiv:2211.02989 §III.N]. Its one model that uses it has three
output heads, for P10, P50 and P90. The training objective adds one pinball
term per head into a single loss [code: libs/tft_model_quantile_loss.py#L1074-L1089].

The review uses the same loss as a scoring function. It gives the tilted
absolute loss for a quantile level $\alpha$ [arxiv:2209.08307 eq. 9]
[eq. 10], and states that this loss is consistent for the $\alpha$-quantile
only in the generalised piecewise linear form [eq. 12] [§3.1]. It also gives
a rule that scores several quantiles at once [eq. 24] [§3.3.3].

## Variants

The sources use the loss in two roles. As a training objective, it shapes a
model's outputs: one head per quantile, a linear model, a forest or boosting
ensemble, or a neural network. As a scoring rule, it judges predicted
quantiles on held-out data. The first four variants below train with it; the
last is a reporting metric.

### Quantile model with one head per quantile

The Temporal Fusion Transformer gets one output head per quantile, each
trained with its own pinball term [code: libs/tft_model_quantile_loss.py#L1091-L1094].
This is the only one of the 14 losses in the survey whose heads learn different
quantiles. Card: [Jadon et al. survey](/papers/2211.02989).

### Linear quantile regression

The review lists quantile regression as linear in its parameters
[arxiv:2209.08307 §4.2]. Quantiles fitted separately can cross, so a higher
quantile can come out below a lower one. The review's remedies are sorting the
outputs, adding non-crossing constraints, or estimating several quantiles
jointly [§4.2.1] [§6.3]. Card: [Tyralis and Papacharalampous review](/papers/2209.08307).

### Quantile regression forests and boosting

The review lists quantile regression forests, which estimate quantiles from
the weighted sample of tree responses [arxiv:2209.08307 §5.3]. It also lists
boosting with quantile or distributional losses, naming gradient boosting,
XGBoost, LightGBM and NGBoost [§5.4]. The review gives no comparison of these
with the other variants. See [quantile regression forest](/concepts/quantile-regression-forest).

### Quantile and expectile neural networks

The review lists quantile and expectile networks [arxiv:2209.08307 §6.3]
among its neural methods, next to dropout-based and ensemble-based uncertainty
[§6.1] [§6.2]. It does not describe their training in enough detail to
reproduce.

### Normalised quantile loss as a reporting metric

The survey's scores are not the raw pinball loss. Each score is twice the mean
pinball error divided by the mean absolute target, computed per quantile
[code: libs/utils.py#L95-L116]. The paper defines the quantiles but not this
normalisation [arxiv:2211.02989 §IV.B]. The metric is scale-free within one
dataset, so compare across losses on the same dataset only.

## Paper formula and code form

The paper prints its quantile loss as two sums, one for under-predictions and
one for over-predictions, with weights $\gamma - 1$ and $\gamma$
[arxiv:2211.02989 §III.N]. The first weight has the wrong sign for the
standard form: the standard weight on an over-prediction is $1 - \gamma$. The
code uses the standard pinball form [code: libs/utils.py#L67-L92]. Implement
the standard form.

## Results

### Temporal Fusion Transformer on four forecasting datasets

Author-reported. Values are the quantile loss row of each results table, to
three decimals. Lower is better. Setup: one run per loss and dataset, on the
test split, with the reduced configuration described under Caveats.

| Dataset | P10 | P50 | P90 |
|---|---|---|---|
| Electricity [tab. 16] | 0.073 | 0.173 | 0.074 |
| Traffic [tab. 17] | 0.108 | 0.315 | 0.232 |
| Favorita [tab. 18] | 0.202 | 0.581 | 0.274 |
| Volatility [§IV] | 0.017 | 0.041 | 0.020 |

Read across the 14 losses in each column, the quantile row has the lowest P10
value and the lowest P90 value on all four datasets. It is not the lowest at
P50 on any dataset: MSE leads on electricity, log-cosh on traffic and
Favorita, and Huber on volatility [tab. 16] [tab. 17] [tab. 18] [§IV]. The
paper's own summary lists quantile among the better losses on all four
datasets [arxiv:2211.02989 §IV].

### Quantile regression against GAMLSS in a simulation

Author-reported. The review's only numerical result is one simulation with a
linear model and Gaussian noise [arxiv:2209.08307 eq. 33]. Quantile
regression and a Gaussian GAMLSS are fitted on the training half and scored on
the test half, averaged over 10 000 repetitions [§7]. The table gives the
quantile score; lower is better.

| Sample size | Level α | Quantile regression | GAMLSS |
|---|---|---|---|
| 100 | 0.05 | 0.12044 | 0.11460 |
| 100 | 0.50 | 0.41847 | 0.41317 |
| 100 | 0.95 | 0.11988 | 0.11426 |
| 1 000 | 0.05 | 0.10456 | 0.10391 |
| 1 000 | 0.50 | 0.40095 | 0.40027 |
| 1 000 | 0.95 | 0.10455 | 0.10389 |

Source: [tab. 2]. GAMLSS has the lower score in every row. The authors say its
advantage shrinks as the sample grows, and attribute it to the response being
Gaussian, which suits the parametric model by construction [§7]. The review
does not define the averaged quantile score or give its random seeds [§7].

## When to use which

| Situation | Use | Why |
|---|---|---|
| You need a range from one model, such as P10 to P90, with a neural network | Quantile loss, one head per quantile | Only quantile-trained heads learn distinct quantiles [code: libs/tft_model_quantile_loss.py#L1091-L1094]. The P10 and P90 rows are lowest on all four datasets, with the caveats below [tab. 16] [tab. 17] [tab. 18] [§IV]. |
| You need quantiles from a linear model | Linear quantile regression | The review lists it as linear in its parameters [arxiv:2209.08307 §4.2]. Sort or constrain the outputs, since separate fits can cross [§4.2.1]. |
| You can name the response family and trust it | Distributional model (GAMLSS), with quantiles read from the fit | In the review's Gaussian simulation, GAMLSS has lower quantile scores at both sample sizes [tab. 2]. The review warns that a wrong distribution is a stated risk for this model [§10.2]. |
| Tree or boosting ensemble with a quantile output | Quantile regression forest or quantile boosting | The review names both [§5.3] [§5.4]. The cards give no head-to-head numbers for them. |
| You compare models on predicted quantiles | Score each level with the quantile loss at that level | Coverage alone is not a proper scoring rule [§7] [§3.5]. The review gives the tilted absolute loss as the scoring function for each level [eq. 9]. |
| You need one point forecast | Not settled by this card | The survey finds the best loss varies by dataset, and gives no universal choice [§IV] [§V]. Test a few point losses on your data; the card names log-cosh, MAE and Huber as leaders on some datasets [§IV]. |
| Outliers or skewed targets | Compare quantile loss with robust point losses | The survey names outliers and skew as factors in the choice, without a controlled test [§V]. |
| Limited compute or a single output | Point losses first | A quantile model needs one output per quantile [code: libs/tft_model_quantile_loss.py#L1091-L1094]. The cards do not measure cost. |

## Caveats

- **One run per cell.** The survey reports no seeds or spread across runs
  [arxiv:2211.02989 §IV.B]. Differences of a few thousandths are not evidence
  of a ranking.
- **Only the quantile row learns quantiles.** The other 13 losses fit all three
  heads to the same target [code: libs/tft_model_mse_loss.py#L1151]. Their P10
  and P90 values measure heads trained to one target. The P10 and P90
  comparisons above are therefore quantile heads against non-quantile heads.
- **Quantile is scored with its own objective.** The reported metric uses the
  same pinball form that the quantile model trains with
  [code: libs/utils.py#L95-L116]. This favours the quantile row, and the card
  does not test a different metric.
- **Reduced configuration.** The results use 1000 training and 100 validation
  samples, 15 epochs and hidden size 16 [code: electricity_dataset_experiments/Huber_Experiment.py#L73-L75].
  The code's defaults are much larger, and the paper does not test whether the
  ranking holds at those settings [arxiv:2211.02989 §IV.B].
- **Range claim.** The survey says the quantile loss ranges from 0 to 1
  [arxiv:2211.02989 §III.N]. The pinball loss is not bounded by 1 in general,
  so the claim does not hold for the formula as written.
- **Simulation is one setting.** The review's simulation uses a Gaussian linear
  model, which favours the parametric model by construction. The authors say
  real data may reverse the ranking [arxiv:2209.08307 §7].
- **Crossing is not covered by the survey.** The review notes that separately
  fitted quantiles can cross [§4.2.1]. The survey's card does not discuss
  whether its three heads cross.

## Implementation sketch

Our own pseudocode of the standard form. The code's version sums the pinball
term over the output dimension [code: libs/utils.py#L67-L92].

```pseudocode
# Pinball loss for one quantile level q in (0, 1)
# Standard form; the paper's printed sign is reversed [§III.N]
def pinball_loss(y, y_hat, q):
    # Target above prediction: under-prediction, weighted by q
    under = maximum(y - y_hat, 0.0)
    # Prediction above target: over-prediction, weighted by 1 - q
    over = maximum(y_hat - y, 0.0)
    return q * under + (1.0 - q) * over

# Training objective for a model with one head per quantile
# [code: libs/tft_model_quantile_loss.py#L1074-L1089]
def training_loss(y, heads):
    # heads maps each level, e.g. 0.1, 0.5 and 0.9, to a prediction
    total = 0.0
    for q, y_hat in heads.items():
        # One pinball term per head, summed into one objective
        total = total + sum(pinball_loss(y, y_hat, q))
    return total
```

## Contradictions

- **Paper formula against code.** The printed formula reverses the sign of the
  first term relative to the standard pinball form [arxiv:2211.02989 §III.N].
  The code uses the standard form [code: libs/utils.py#L67-L92]. The card
  reports both sides and does not resolve which one the authors intended.
- **Range claim against formula.** The survey's 0-to-1 range claim
  [arxiv:2211.02989 §III.N] conflicts with the pinball form, which is not bounded
  by 1 in general.
- **Two cards, two comparisons.** The survey favours the quantile head on P10
  and P90 among its losses [tab. 16] [tab. 17] [tab. 18]. The review finds
  GAMLSS scores lower than quantile regression [tab. 2]. These compare
  different models, datasets and metrics, so they do not conflict. None known
  beyond that.

## Papers

- [Jadon et al. survey](/papers/2211.02989): compares 14 regression losses as
  training objectives for a Temporal Fusion Transformer on four forecasting
  datasets; the quantile loss is one of them and the only one with distinct
  quantile heads.
- [Tyralis and Papacharalampous review](/papers/2209.08307): reviews scoring
  rules and model families for probabilistic prediction, including quantile
  regression, forests, boosting and networks; its one simulation compares
  quantile regression with GAMLSS.
