---
kind: concept
slug: extreme-value-theory
title: Extreme value theory
summary: >-
  Sampling or modelling heavy-tailed, non-negative vectors beyond the training maximum, by splitting each vector into a radius with a regularly varying tail and an angle on the simplex. One card so far: a pair of VAEs, one per part, with a known or learned tail index. Light-tailed generators fail here; results are author-reported on one synthetic setting and five river gauges.
tags: [heavy-tails, generative-models, risk-assessment]
papers: ["2306.10987"]
related: [heavy-tailed-noise, variational-autoencoder, generative-model-evaluation]
status: draft
updated_at: 2026-11-12
---

## Overview

Extreme value theory asks how to model and sample values far beyond those seen
in training, such as rare floods or financial losses. For a non-negative
vector, the approach in this paper writes the vector as a radius times an
angle. The radius is the sum of the components. The angle is the vector divided
by its radius, so it lies on the simplex [arxiv:2306.10987 §2.3]. The vector is
multivariate regularly varying when the radius has a tail index α, and the
angle, conditioned on a large radius, converges to an angular measure
[arxiv:2306.10987 §2.3].

Two generator designs fail for this purpose. A standard Gaussian VAE produces
light-tailed marginals, so it cannot go beyond the largest training values. The
result assumes piecewise-linear decoder networks, such as ReLU networks
[arxiv:2306.10987 §3.1] [app. G.1]. A deterministic map, such as a ReLU network
applied to a heavy-tailed input, places its angular measure on at most n
points, where n is the input dimension. Its extremes therefore follow only a few
directions [arxiv:2306.10987 §3.2].

Only one card in wikillm lists this concept, so this page describes that paper's
approach and nothing else.

## Variants

### Radius and angle VAEs with a known tail index

The radius is drawn by a VAE whose latent prior is inverse-gamma with shape α.
The decoder's Gamma shape and rate are built from two ReLU networks, as |g(z)|/z
and |f(z)|/z² [arxiv:2306.10987 §4.3] [eq. 11] [eq. 12]. The heavy-tail
guarantee needs the rate to be proportional to 1/z at both 0 and infinity
[arxiv:2306.10987 §4.3] [app. G.4]. A second VAE draws the angle given the
radius. Its latent prior is a standard normal, and the radius enters as 1/(1+r),
which makes angle and radius independent in the limit [arxiv:2306.10987 §4.4]
[eq. 17] [eq. 18]. The angle likelihood can be Dirichlet or a Gaussian projected
onto the simplex; the paper reports the Dirichlet on its synthetic data and the
projected Gaussian on river data, where each fits better on its own set
[arxiv:2306.10987 §6.1] [§6.2]. Card: [A VAE Approach to Sample Multivariate
Extremes](/papers/2306.10987). On synthetic data, the radius ELBO cost is 0.88
on training data and 1.12 on test data [arxiv:2306.10987 tab. 1].

### Same pair with a learned tail index

The tail index α is optimised by gradient descent through the closed-form KL
term, rather than set in advance [arxiv:2306.10987 §4.3] [§5.4]. On synthetic
data the learned α has mean 1.56 and standard deviation 0.2, against a true 1.5
[arxiv:2306.10987 §6.1] [fig. 5]. Its test cost is 1.15 against 1.12 for the
fixed-α version [arxiv:2306.10987 tab. 1]. On the Danube data the paper reports
a learned value of 4.5 [arxiv:2306.10987 §6.2]; see Contradictions.

### Comparison baselines (not endorsed for extremes)

- A standard Gaussian VAE. Its radius test cost is infinite, because its
  validation cost of 4.81 is far above its training cost of 1.21 and the test
  set takes the value infinity [arxiv:2306.10987 tab. 1].
- ParetoGAN, a Wasserstein GAN with a Pareto latent. The paper names it as a
  baseline; wikillm has no card for it, so its details are not given here
  [arxiv:2306.10987 §5.4] [§6.1].

## When to use which

| Situation | Use | Why |
|---|---|---|
| Samples beyond the training maximum are needed, and the tail index is known or fixed by you | Radius and angle VAEs with known α | A standard VAE stays light-tailed [arxiv:2306.10987 §3.1]; the known-α radius cost is 0.88 train and 1.12 test [tab. 1] |
| The tail index is not known and a fitted value is acceptable | Learned-α variant | Synthetic learned α is 1.56 against 1.5 [fig. 5]; test cost rises only from 1.12 to 1.15 [tab. 1] |
| Joint extremes across components matter | Separate angle model, as above | Heavy-tailed latent maps concentrate on few directions [arxiv:2306.10987 §3.2]; reported Wasserstein distance to test data is 5.37 for this method against 6.80 for ParetoGAN, on synthetic data [§6.1] |
| The decoder uses smooth activations | Verify before relying on the light-tail result | The paper's proof covers piecewise-linear networks only; it gives no result for smooth activations [arxiv:2306.10987 app. G.1] |
| Data are one-off extremes beyond training range (Danube, p = 0.99) | Treat outputs as uncertain | The training set has no p = 0.99 events, and the learned-α variant gives 0.2% against 0.4% in the test set for one station [arxiv:2306.10987 tab. 2] |

All numbers are author-reported. The synthetic study uses dimension 5 and
α = 1.5; the case study uses five river gauges. The paper does not state how
many runs or seeds back each number [arxiv:2306.10987 §6.1] [§6.2].

## Implementation sketch

Sampling, in the paper's order: draw the radius, then the angle given the
radius, then scale the angle [arxiv:2306.10987 §4.1].

```pseudocode
# Sample one heavy-tailed, non-negative vector  [arxiv:2306.10987 §4.1]
# Input: radius decoder, angle decoder, tail index alpha
# Output: x of shape (m,), non-negative

def sample_heavy_vector(radius_decoder, angle_decoder, alpha):
    # Draw the radius latent from InvGamma(alpha, scale 1)  [§4.3] [eq. 8]
    z_rad = sample_inv_gamma(shape=alpha, scale=1.0)
    # Draw the radius from Gamma(shape(z), rate(z))  [§4.3] [eq. 9]
    r = sample_gamma(radius_decoder.shape(z_rad),
                     radius_decoder.rate(z_rad))
    # Draw the angular latent from N(0, I_n)  [§4.4]
    z_ang = sample_normal(0, identity(n))
    # Draw the angle on the simplex, conditioned on r  [§4.4] [eq. 17-18]
    theta = angle_decoder.sample(z_ang, r)
    # Scale the angle by the radius  [§4.1]
    return r * theta
```

Notes where the variants differ or the paper is ambiguous:

- Training: the paper uses Adam, batch size 32, at most 5000 epochs, and
  learning rates 10⁻⁴ for the radius VAE and 10⁻⁵ for the angle VAE
  [arxiv:2306.10987 §5.2]. The radius is trained first, the angle second.
- Loss sign: both losses are negative ELBOs (KL minus log-likelihood). The
  printed sum in eq. 19 shows the signs differently, so check the sign when
  implementing [arxiv:2306.10987 §5.2] [eq. 19].
- KL digamma term: the appendix derivation uses ψ(α_φ(r)), the digamma of the
  posterior shape. The printed form in eq. 15 writes ψ(α) [arxiv:2306.10987
  §4.3] [app. G.5].
- Rate constant: the proof in appendix G.3 gives σ = βc/α for a scale c. A
  shorter remark in §2.2 gives a different rate convention, so use the proof
  [arxiv:2306.10987 §2.2] [app. G.3].
- Positivity: inverse-gamma parameters must be strictly positive for the KL.
  A ReLU output can be zero, so a guard such as |·| + ε or softplus is needed.
  The paper does not specify one [arxiv:2306.10987 §4.3].

## Contradictions

- **Danube tail index.** The paper reports a learned index of 4.5 [§6.2]. It
  also cites an external estimate of 3.5 ± 0.5 for summer months [§6.2]. The
  paper calls 4.5 "slightly higher" than that estimate, but 4.5 lies outside
  the stated range. The paper does not resolve this, so the Danube index is
  uncertain [arxiv:2306.10987 §6.2].
- **Independence threshold.** The radius above which radius and angle look
  independent is 1.3 for the VAE pair, 1.6 for the true synthetic data, and 2.7
  for ParetoGAN [arxiv:2306.10987 §6.1] [fig. 8]. The paper notes that the
  VAE and true-data thresholds differ; the asymptotic independence holds only
  approximately at finite radii [arxiv:2306.10987 §5.3] [app. E.3].

## Papers

- [A VAE Approach to Sample Multivariate Extremes](/papers/2306.10987): a VAE pair that samples a heavy-tailed radius and a conditional angle on the simplex, with a known or learned tail index.
