Concept · 1 paper · updated 2026-11-12
Extreme value theory
Sampling or modelling heavy-tailed, non-negative vectors beyond the training maximum, by splitting each vector into a radius with a regularly varying tail and an angle on the simplex. One card so far: a pair of VAEs, one per part, with a known or learned tail index. Light-tailed generators fail here; results are author-reported on one synthetic setting and five river gauges.
heavy-tailsgenerative-modelsrisk-assessment
Markdown5 min read
AI-compiled overview, updated 2026-11-12, not yet reviewed by a human. Every claim cites its source; check it when a detail matters.
Overview
Extreme value theory asks how to model and sample values far beyond those seen in training, such as rare floods or financial losses. For a non-negative vector, the approach in this paper writes the vector as a radius times an angle. The radius is the sum of the components. The angle is the vector divided by its radius, so it lies on the simplex 2306.10987 §2.3. The vector is multivariate regularly varying when the radius has a tail index α, and the angle, conditioned on a large radius, converges to an angular measure 2306.10987 §2.3.
Two generator designs fail for this purpose. A standard Gaussian VAE produces light-tailed marginals, so it cannot go beyond the largest training values. The result assumes piecewise-linear decoder networks, such as ReLU networks 2306.10987 §3.1 app. G.1. A deterministic map, such as a ReLU network applied to a heavy-tailed input, places its angular measure on at most n points, where n is the input dimension. Its extremes therefore follow only a few directions 2306.10987 §3.2.
Only one card in wikillm lists this concept, so this page describes that paper's approach and nothing else.
Variants
Radius and angle VAEs with a known tail index
The radius is drawn by a VAE whose latent prior is inverse-gamma with shape α. The decoder's Gamma shape and rate are built from two ReLU networks, as |g(z)|/z and |f(z)|/z² 2306.10987 §4.3 eq. 11 eq. 12. The heavy-tail guarantee needs the rate to be proportional to 1/z at both 0 and infinity 2306.10987 §4.3 app. G.4. A second VAE draws the angle given the radius. Its latent prior is a standard normal, and the radius enters as 1/(1+r), which makes angle and radius independent in the limit 2306.10987 §4.4 eq. 17 eq. 18. The angle likelihood can be Dirichlet or a Gaussian projected onto the simplex; the paper reports the Dirichlet on its synthetic data and the projected Gaussian on river data, where each fits better on its own set 2306.10987 §6.1 §6.2. Card: A VAE Approach to Sample Multivariate Extremes. On synthetic data, the radius ELBO cost is 0.88 on training data and 1.12 on test data 2306.10987 tab. 1.
Same pair with a learned tail index
The tail index α is optimised by gradient descent through the closed-form KL term, rather than set in advance 2306.10987 §4.3 §5.4. On synthetic data the learned α has mean 1.56 and standard deviation 0.2, against a true 1.5 2306.10987 §6.1 fig. 5. Its test cost is 1.15 against 1.12 for the fixed-α version 2306.10987 tab. 1. On the Danube data the paper reports a learned value of 4.5 2306.10987 §6.2; see Contradictions.
Comparison baselines (not endorsed for extremes)
- A standard Gaussian VAE. Its radius test cost is infinite, because its validation cost of 4.81 is far above its training cost of 1.21 and the test set takes the value infinity 2306.10987 tab. 1.
- ParetoGAN, a Wasserstein GAN with a Pareto latent. The paper names it as a baseline; wikillm has no card for it, so its details are not given here 2306.10987 §5.4 §6.1.
When to use which
| Situation | Use | Why |
|---|---|---|
| Samples beyond the training maximum are needed, and the tail index is known or fixed by you | Radius and angle VAEs with known α | A standard VAE stays light-tailed 2306.10987 §3.1; the known-α radius cost is 0.88 train and 1.12 test tab. 1 |
| The tail index is not known and a fitted value is acceptable | Learned-α variant | Synthetic learned α is 1.56 against 1.5 fig. 5; test cost rises only from 1.12 to 1.15 tab. 1 |
| Joint extremes across components matter | Separate angle model, as above | Heavy-tailed latent maps concentrate on few directions 2306.10987 §3.2; reported Wasserstein distance to test data is 5.37 for this method against 6.80 for ParetoGAN, on synthetic data §6.1 |
| The decoder uses smooth activations | Verify before relying on the light-tail result | The paper's proof covers piecewise-linear networks only; it gives no result for smooth activations 2306.10987 app. G.1 |
| Data are one-off extremes beyond training range (Danube, p = 0.99) | Treat outputs as uncertain | The training set has no p = 0.99 events, and the learned-α variant gives 0.2% against 0.4% in the test set for one station 2306.10987 tab. 2 |
All numbers are author-reported. The synthetic study uses dimension 5 and α = 1.5; the case study uses five river gauges. The paper does not state how many runs or seeds back each number 2306.10987 §6.1 §6.2.
Implementation sketch
Sampling, in the paper's order: draw the radius, then the angle given the radius, then scale the angle 2306.10987 §4.1.
# Sample one heavy-tailed, non-negative vector [arxiv:2306.10987 §4.1]
# Input: radius decoder, angle decoder, tail index alpha
# Output: x of shape (m,), non-negative
def sample_heavy_vector(radius_decoder, angle_decoder, alpha):
# Draw the radius latent from InvGamma(alpha, scale 1) [§4.3] [eq. 8]
z_rad = sample_inv_gamma(shape=alpha, scale=1.0)
# Draw the radius from Gamma(shape(z), rate(z)) [§4.3] [eq. 9]
r = sample_gamma(radius_decoder.shape(z_rad),
radius_decoder.rate(z_rad))
# Draw the angular latent from N(0, I_n) [§4.4]
z_ang = sample_normal(0, identity(n))
# Draw the angle on the simplex, conditioned on r [§4.4] [eq. 17-18]
theta = angle_decoder.sample(z_ang, r)
# Scale the angle by the radius [§4.1]
return r * thetaNotes where the variants differ or the paper is ambiguous:
- Training: the paper uses Adam, batch size 32, at most 5000 epochs, and learning rates 10⁻⁴ for the radius VAE and 10⁻⁵ for the angle VAE 2306.10987 §5.2. The radius is trained first, the angle second.
- Loss sign: both losses are negative ELBOs (KL minus log-likelihood). The printed sum in eq. 19 shows the signs differently, so check the sign when implementing 2306.10987 §5.2 eq. 19.
- KL digamma term: the appendix derivation uses ψ(α_φ(r)), the digamma of the posterior shape. The printed form in eq. 15 writes ψ(α) [arxiv:2306.10987 §4.3] app. G.5.
- Rate constant: the proof in appendix G.3 gives σ = βc/α for a scale c. A shorter remark in §2.2 gives a different rate convention, so use the proof 2306.10987 §2.2 app. G.3.
- Positivity: inverse-gamma parameters must be strictly positive for the KL. A ReLU output can be zero, so a guard such as |·| + ε or softplus is needed. The paper does not specify one 2306.10987 §4.3.
Contradictions
- Danube tail index. The paper reports a learned index of 4.5 §6.2. It also cites an external estimate of 3.5 ± 0.5 for summer months §6.2. The paper calls 4.5 "slightly higher" than that estimate, but 4.5 lies outside the stated range. The paper does not resolve this, so the Danube index is uncertain 2306.10987 §6.2.
- Independence threshold. The radius above which radius and angle look independent is 1.3 for the VAE pair, 1.6 for the true synthetic data, and 2.7 for ParetoGAN 2306.10987 §6.1 fig. 8. The paper notes that the VAE and true-data thresholds differ; the asymptotic independence holds only approximately at finite radii 2306.10987 §5.3 app. E.3.
Papers
- A VAE Approach to Sample Multivariate Extremes: a VAE pair that samples a heavy-tailed radius and a conditional angle on the simplex, with a known or learned tail index.
Source and license
Compiled from the papers listed on this page, all published under CC BY, CC BY-SA or CC0, with citations. See each paper's page for its authors and license.