Concept · 2 papers · updated 2026-11-15
Quantile loss
Pinball loss for one target quantile, so a model can output a range such as P10 to P90 instead of one mean-like value. Used both to train models and to score predicted quantiles. One survey trains a Temporal Fusion Transformer with 14 losses; one review covers quantile regression, forests, boosting and networks.
loss-functionregressionforecastinguncertaintyprobabilisticscoring-rules
Markdown7 min read
AI-compiled overview, updated 2026-11-15, not yet reviewed by a human. Every claim cites its source; check it when a detail matters.
Overview
Quantile loss is the pinball loss for a target quantile q between 0 and 1. It weights the two kinds of error differently: when the target is above the prediction, the error is weighted by q; when the prediction is above the target, it is weighted by 1 - q 2211.02989 §III.N utils.py. At q = 0.5 it equals half the absolute error, so the paper's claim that the median loss is MAE holds only up to a factor of two 2211.02989 §III.N.
The survey presents it as a way to predict intervals rather than point estimates 2211.02989 §III.N. Its one model that uses it has three output heads, for P10, P50 and P90. The training objective adds one pinball term per head into a single loss tft_model_quantile_loss.py.
The review uses the same loss as a scoring function. It gives the tilted absolute loss for a quantile level \alpha 2209.08307 eq. 9 eq. 10, and states that this loss is consistent for the \alpha-quantile only in the generalised piecewise linear form eq. 12 §3.1. It also gives a rule that scores several quantiles at once eq. 24 §3.3.3.
Variants
The sources use the loss in two roles. As a training objective, it shapes a model's outputs: one head per quantile, a linear model, a forest or boosting ensemble, or a neural network. As a scoring rule, it judges predicted quantiles on held-out data. The first four variants below train with it; the last is a reporting metric.
Quantile model with one head per quantile
The Temporal Fusion Transformer gets one output head per quantile, each trained with its own pinball term tft_model_quantile_loss.py. This is the only one of the 14 losses in the survey whose heads learn different quantiles. Card: Jadon et al. survey.
Linear quantile regression
The review lists quantile regression as linear in its parameters 2209.08307 §4.2. Quantiles fitted separately can cross, so a higher quantile can come out below a lower one. The review's remedies are sorting the outputs, adding non-crossing constraints, or estimating several quantiles jointly §4.2.1 §6.3. Card: Tyralis and Papacharalampous review.
Quantile regression forests and boosting
The review lists quantile regression forests, which estimate quantiles from the weighted sample of tree responses 2209.08307 §5.3. It also lists boosting with quantile or distributional losses, naming gradient boosting, XGBoost, LightGBM and NGBoost §5.4. The review gives no comparison of these with the other variants. See quantile regression forest.
Quantile and expectile neural networks
The review lists quantile and expectile networks 2209.08307 §6.3 among its neural methods, next to dropout-based and ensemble-based uncertainty §6.1 §6.2. It does not describe their training in enough detail to reproduce.
Normalised quantile loss as a reporting metric
The survey's scores are not the raw pinball loss. Each score is twice the mean pinball error divided by the mean absolute target, computed per quantile utils.py. The paper defines the quantiles but not this normalisation 2211.02989 §IV.B. The metric is scale-free within one dataset, so compare across losses on the same dataset only.
Paper formula and code form
The paper prints its quantile loss as two sums, one for under-predictions and one for over-predictions, with weights \gamma - 1 and \gamma 2211.02989 §III.N. The first weight has the wrong sign for the standard form: the standard weight on an over-prediction is 1 - \gamma. The code uses the standard pinball form utils.py. Implement the standard form.
Results
Temporal Fusion Transformer on four forecasting datasets
Author-reported. Values are the quantile loss row of each results table, to three decimals. Lower is better. Setup: one run per loss and dataset, on the test split, with the reduced configuration described under Caveats.
| Dataset | P10 | P50 | P90 |
|---|---|---|---|
| Electricity tab. 16 | 0.073 | 0.173 | 0.074 |
| Traffic tab. 17 | 0.108 | 0.315 | 0.232 |
| Favorita tab. 18 | 0.202 | 0.581 | 0.274 |
| Volatility §IV | 0.017 | 0.041 | 0.020 |
Read across the 14 losses in each column, the quantile row has the lowest P10 value and the lowest P90 value on all four datasets. It is not the lowest at P50 on any dataset: MSE leads on electricity, log-cosh on traffic and Favorita, and Huber on volatility tab. 16 tab. 17 tab. 18 §IV. The paper's own summary lists quantile among the better losses on all four datasets 2211.02989 §IV.
Quantile regression against GAMLSS in a simulation
Author-reported. The review's only numerical result is one simulation with a linear model and Gaussian noise 2209.08307 eq. 33. Quantile regression and a Gaussian GAMLSS are fitted on the training half and scored on the test half, averaged over 10 000 repetitions §7. The table gives the quantile score; lower is better.
| Sample size | Level α | Quantile regression | GAMLSS |
|---|---|---|---|
| 100 | 0.05 | 0.12044 | 0.11460 |
| 100 | 0.50 | 0.41847 | 0.41317 |
| 100 | 0.95 | 0.11988 | 0.11426 |
| 1 000 | 0.05 | 0.10456 | 0.10391 |
| 1 000 | 0.50 | 0.40095 | 0.40027 |
| 1 000 | 0.95 | 0.10455 | 0.10389 |
Source: tab. 2. GAMLSS has the lower score in every row. The authors say its advantage shrinks as the sample grows, and attribute it to the response being Gaussian, which suits the parametric model by construction §7. The review does not define the averaged quantile score or give its random seeds §7.
When to use which
| Situation | Use | Why |
|---|---|---|
| You need a range from one model, such as P10 to P90, with a neural network | Quantile loss, one head per quantile | Only quantile-trained heads learn distinct quantiles tft_model_quantile_loss.py. The P10 and P90 rows are lowest on all four datasets, with the caveats below tab. 16 tab. 17 tab. 18 §IV. |
| You need quantiles from a linear model | Linear quantile regression | The review lists it as linear in its parameters 2209.08307 §4.2. Sort or constrain the outputs, since separate fits can cross §4.2.1. |
| You can name the response family and trust it | Distributional model (GAMLSS), with quantiles read from the fit | In the review's Gaussian simulation, GAMLSS has lower quantile scores at both sample sizes tab. 2. The review warns that a wrong distribution is a stated risk for this model §10.2. |
| Tree or boosting ensemble with a quantile output | Quantile regression forest or quantile boosting | The review names both §5.3 §5.4. The cards give no head-to-head numbers for them. |
| You compare models on predicted quantiles | Score each level with the quantile loss at that level | Coverage alone is not a proper scoring rule §7 §3.5. The review gives the tilted absolute loss as the scoring function for each level eq. 9. |
| You need one point forecast | Not settled by this card | The survey finds the best loss varies by dataset, and gives no universal choice §IV §V. Test a few point losses on your data; the card names log-cosh, MAE and Huber as leaders on some datasets §IV. |
| Outliers or skewed targets | Compare quantile loss with robust point losses | The survey names outliers and skew as factors in the choice, without a controlled test §V. |
| Limited compute or a single output | Point losses first | A quantile model needs one output per quantile tft_model_quantile_loss.py. The cards do not measure cost. |
Caveats
- One run per cell. The survey reports no seeds or spread across runs 2211.02989 §IV.B. Differences of a few thousandths are not evidence of a ranking.
- Only the quantile row learns quantiles. The other 13 losses fit all three heads to the same target tft_model_mse_loss.py. Their P10 and P90 values measure heads trained to one target. The P10 and P90 comparisons above are therefore quantile heads against non-quantile heads.
- Quantile is scored with its own objective. The reported metric uses the same pinball form that the quantile model trains with utils.py. This favours the quantile row, and the card does not test a different metric.
- Reduced configuration. The results use 1000 training and 100 validation samples, 15 epochs and hidden size 16 Huber_Experiment.py. The code's defaults are much larger, and the paper does not test whether the ranking holds at those settings 2211.02989 §IV.B.
- Range claim. The survey says the quantile loss ranges from 0 to 1 2211.02989 §III.N. The pinball loss is not bounded by 1 in general, so the claim does not hold for the formula as written.
- Simulation is one setting. The review's simulation uses a Gaussian linear model, which favours the parametric model by construction. The authors say real data may reverse the ranking 2209.08307 §7.
- Crossing is not covered by the survey. The review notes that separately fitted quantiles can cross §4.2.1. The survey's card does not discuss whether its three heads cross.
Implementation sketch
Our own pseudocode of the standard form. The code's version sums the pinball term over the output dimension utils.py.
# Pinball loss for one quantile level q in (0, 1)
# Standard form; the paper's printed sign is reversed [§III.N]
def pinball_loss(y, y_hat, q):
# Target above prediction: under-prediction, weighted by q
under = maximum(y - y_hat, 0.0)
# Prediction above target: over-prediction, weighted by 1 - q
over = maximum(y_hat - y, 0.0)
return q * under + (1.0 - q) * over
# Training objective for a model with one head per quantile
# [code: libs/tft_model_quantile_loss.py#L1074-L1089]
def training_loss(y, heads):
# heads maps each level, e.g. 0.1, 0.5 and 0.9, to a prediction
total = 0.0
for q, y_hat in heads.items():
# One pinball term per head, summed into one objective
total = total + sum(pinball_loss(y, y_hat, q))
return totalContradictions
- Paper formula against code. The printed formula reverses the sign of the first term relative to the standard pinball form 2211.02989 §III.N. The code uses the standard form utils.py. The card reports both sides and does not resolve which one the authors intended.
- Range claim against formula. The survey's 0-to-1 range claim 2211.02989 §III.N conflicts with the pinball form, which is not bounded by 1 in general.
- Two cards, two comparisons. The survey favours the quantile head on P10 and P90 among its losses tab. 16 tab. 17 tab. 18. The review finds GAMLSS scores lower than quantile regression tab. 2. These compare different models, datasets and metrics, so they do not conflict. None known beyond that.
Papers
- Jadon et al. survey: compares 14 regression losses as training objectives for a Temporal Fusion Transformer on four forecasting datasets; the quantile loss is one of them and the only one with distinct quantile heads.
- Tyralis and Papacharalampous review: reviews scoring rules and model families for probabilistic prediction, including quantile regression, forests, boosting and networks; its one simulation compares quantile regression with GAMLSS.
Source and license
Compiled from the papers listed on this page, all published under CC BY, CC BY-SA or CC0, with citations. See each paper's page for its authors and license.