This Time is Different: An Observability Perspective on Time Series Foundation Models

In plain words

Observability tools watch computer systems. They record many metrics over time, such as CPU load, memory use, request rates, error counts and latency. These series are often spiky, sparse and noisy, and their scales differ widely. Forecasting them helps with alerts and capacity planning, but a large company can have millions of such series, so training a separate model for each one is not practical.

Toto is one model that forecasts these series without per-series training. It works like a language model for numbers. It cuts a series into chunks and learns to predict each chunk from the chunks before it. Every chunk is rescaled using only the past, because scaling with future data would leak information. Attention runs mostly across time, with occasional attention across related metrics, which keeps the cost manageable when a group holds many metrics. The output is not a single number but a spread of likely values, built from bell-shaped curves with heavy tails, since spikes are common. The training loss also penalises large single-point misses, which the authors find helps avoid poor fits.

The model is trained on a mix of the company's own metrics, public forecasting datasets and generated series. The authors also release BOOM, a new benchmark built from observability data, to test forecasters on this kind of series.

TL;DR

• Toto is a 151M-parameter decoder-only forecaster for multivariate observability time series [abstract]. Its pretraining corpus has 2.36 trillion points, about 1.59 trillion of them unique real data rather than synthetic, and 43% of the mix is Datadog's own observability metrics §3.2.
• Author-reported: on BOOM it has the lowest MASE, CRPS and mean rank, 13.1% and 12.4% below the next-best model on MASE and CRPS §5 tab. 2. It has the best GIFT-Eval mean rank, while CRPS on that benchmark is second to one baseline tab. 3.
• The official repository is toto/ under Apache-2.0. Its training module is for fine-tuning. The pretraining configuration and corpus are not in it, so the pretraining recipe cannot be reproduced from the repo (see Paper vs code).

Problem

Observability metrics are sparse, spiky, noisy, highly nonstationary and often high-cardinality, with a large dynamic range §1 §4.3. Real systems produce millions to billions of such series, so training or fine-tuning a model for each one is infeasible, which motivates zero-shot foundation models §1. The authors find that general-purpose foundation models do not generalise well to this data §1.

Two common designs break down here. Normalising a series with statistics from the whole series leaks future information, which breaks causal next-patch training. Normalising with the first patch only is causal but ineffective when the statistics drift over time §3.1. Full attention over both time and variates scales with the square of the number of patches and the square of the number of variates app. A.2 eq. 1.

Method

Toto is a decoder-only transformer trained to predict the next patch of each variate from the patches before it §3.1 app. A.5. Its input and output parts are designed for observability data:

1. Causal patch scaling. Each series is standardised with a mean and a standard deviation computed from the past and the current patch only. Padded points get weight 0. The statistics come from a vectorised version of Welford's online algorithm, with Bessel's correction §3.1 app. A.1. Every time step in a patch uses the statistics of the patch's last step §3.1.
2. Clipping. The causal standard deviation is limited to a band around a full-series statistic, with a factor of 10^κ and κ = 10. The lower limit is 0.1 §3.1. At inference, the statistics come from the context only §3.1.
3. Patch embedding. Non-overlapping patches of 64 points, a context of 4096 points and an embedding size of 768 §3.1 app. A.6.
4. Factorised attention. Each block attends along one axis only. Eleven time-wise blocks, which are causal, are followed by one variate-wise block, which is bidirectional and uses a block-diagonal mask so that series packed into one batch do not attend to each other. The 11:1 ratio came from hyperparameter search §3.1 app. A.2.
5. Transformer details. Pre-normalisation, RMSNorm, SwiGLU feed-forward layers and rotary positions with XPOS §3.1.
6. Mixture head. Each token produces K = 24 Student-T components and a weight for each. Degrees of freedom are 2 plus a positive value, so the variance is finite §3.1 app. A.3 app. A.6.
7. Composite loss. Targets are scaled by the causal statistics. The loss mixes the negative log-likelihood under the mixture with a Cauchy-type robust loss on the mixture mean §3.1 app. A.5.
8. Sampling. At inference the model draws u samples at each timestamp, feeds each back as input, and yields n independent sample paths for prediction intervals app. A.4. For BOOM, Toto uses 256 samples and the median as the point forecast app. C.2.3.

# Toto forward pass for a batch of multivariate series  [§3.1] [app. A.2]
# Input:  series (B, M, L), padding_mask (B, M, L), id_mask (B, M, L)
# Output: mixture distribution over the next patch of each variate
def forward(series, padding_mask, id_mask):
    # Scale with statistics from the history so far  [§3.1]
    scaled, loc, scale = causal_patch_scale(series, padding_mask)
    # Cut the series into non-overlapping patches of 64 points  [§3.1]
    tokens = patch_embedding(scaled)              # (B, M, L/64, D)
    # Eleven time-wise blocks, then one variate-wise block  [§3.1]
    for layer in transformer_layers:
        tokens = layer(tokens, id_mask)           # (B, M, L/64, D)
    # Map each token to K Student-T components and mixture weights  [app. A.3]
    distribution = student_t_mixture(tokens)
    return distribution, loc, scale

# Causal patch scaling  [§3.1] [app. A.1]
# Input:  series (B, M, L), weights (B, M, L) with 0 on padding
# Output: normalised series, per-patch loc and scale
def causal_patch_scale(series, weights, patch=64, kappa=10):
    # Running weighted mean and Bessel-corrected variance  [app. A.1]
    mean_t, var_t = welford_causal(series, weights)
    # The 0.1 offset sits under the square root  [§3.1]
    scale_t = sqrt(var_t + 0.1)
    # Clip to a band around the full-series scale s  [§3.1]
    s = global_std(series, weights)
    scale_t = clip(scale_t, max(0.1, s * 10**-kappa), s * 10**kappa)
    # Each patch reuses the statistics of its last time step  [§3.1]
    mean_p = mean_t[..., patch - 1::patch]
    scale_p = scale_t[..., patch - 1::patch]
    # Repeat over the patch and normalise  [§3.1]
    mean_full = repeat(mean_p, patch)
    scale_full = repeat(scale_p, patch)
    return (series - mean_full) / scale_full, mean_full, scale_full

# Training loss on scaled targets  [§3.1] [app. A.5]
# Input:  distribution (Student-T mixture), targets (scaled), lambda, delta
# Output: scalar loss per point
def training_loss(distribution, targets, lambda_nll, delta):
    # Negative log-likelihood under the full mixture  [eq. 10]
    nll = -distribution.log_prob(targets)
    # Cauchy-type robust loss on the mixture mean, alpha = 0  [eq. 11]
    error = (targets - distribution.mean) / delta
    robust = log(0.5 * error ** 2 + 1)
    # Weighted sum; the paper tunes lambda and delta  [§3.1]
    return lambda_nll * nll + (1 - lambda_nll) * robust

Key equations. The causal mean, the causal standard deviation and the clipping band are unnumbered in §3.1. The complexity formulas are eq. 1 and eq. 2. The mixture density is eq. 3, the Student-T density is eq. 4, and the mixture parameters are eq. 5 to eq. 9. The NLL is eq. 10. The robust loss cases are eq. 11. The Cauchy case is stated in §3.1.

Hyperparameters

Values for the released pretraining run come from one table app. A.6. The text gives a few of them again with slightly different rounding, noted in the Notes column. Dropout is not stated.

Results

Author-reported. The main benchmarks are BOOM, a zero-shot benchmark of 2,807 series with 350 million points, from Datadog staging data §4.1 tab. 1; and GIFT-Eval §5 tab. 3. LSF is also reported zero-shot §5 tab. 4. Baselines are other foundation models and classical statistical models §5.

• BOOM subset. On BOOMlet, a 32-query subset, Toto is best on all three metrics: MASE 0.617, CRPS 0.519, rank 1.300 tab. 2.
• LSF. Toto is best on 8 of 12 metric columns when compared with other zero-shot methods tab. 4 §5. Fine-tuned LSF results are in app. D.3 and are not summarised here.
• Ablation. On held-out observability NLL, models trained for 75,000 steps degrade when a component is removed: causal scaling +27.3%, the single Student-T output instead of the mixture +27.2%, no robust loss +11.1%, and no variate-wise attention +1.6% app. E.
• Efficiency. Measured as single forward passes that produce one sample, on one A100 (40 GB) with context 2,048 and horizon 480. The authors report Toto has the lowest FLOPs at every variate count and beats Moirai from 150 variates on app. D.4. They also note it uses a KV cache, while the compared models do not appear to app. D.4. Sampling cost is not measured there.

Paper vs code

Official code: https://github.com/DataDog/toto at commit 9213d2c86f716c43010a0a3b2b6a0e3f440e6743 (default branch at the time of compiling, not the paper's version), Apache-2.0 LICENSE. The paper links this repository in its abstract, so no further search was needed. Its owner is Datadog, which the author footnotes name as the authors' organisation.

The repository also contains toto2/, a newer package. Its README names Toto 2.0 as the recommended release README.md. This card covers only the code under toto/, which matches the open release Toto-Open-Base-1.0 with 151M parameters README.md.

The discrepancies below were found by reading the paper and the code. Each one is a place where the paper and the code disagree, or where the paper is silent and the code decides.

1. Order inside a block. The paper puts the eleven time-wise layers before the variate-wise layer §3.1. The layer builder places the variate-wise layer first in each block when spacewise_first is true, and the Toto class defaults to true transformer.py toto.py. The released configuration is not in the repository, so which order the released weights use is not verified.
2. Where the 0.1 offset goes. The prose says a minimum of 0.1 is added to the causal standard deviation §3.1. The formula puts 0.1 under the square root, on the variance §3.1. The code follows the formula: it adds the offset to the variance before the root scaler.py, and the class default is 0.1 scaler.py.
3. The clip reference. The paper says the clip uses the full-variate variance s, but compares it with the causal standard deviation §3.1. The code takes the global standard deviation, the root of the global variance plus 0.1, and applies the factor 10^±κ to it scaler.py scaler.py.
4. Global statistics in training. The paper says that at inference the statistics come from the history only §3.1. At inference the forecaster passes the context length forecaster.py. The training path does not pass a length, so the clamp uses the whole input window backbone.py backbone.py. Training therefore sees future points in the clamp, which the paper calls relaxed causality §3.1.
5. Defaults differ by class. Toto and TotoBackbone default stabilize_with_global to true toto.py backbone.py. The scaler class defaults it to false scaler.py. The paper does not say which setting the released model uses.
6. Loss constants. The text gives δ = 0.1 and λ = 0.57 §3.1. The table gives 0.1010 and 0.5755 app. A.6. The code defaults are 0.1 and 0.575 losses.py. The paper does not say which values the released model used.
7. Training code is for fine-tuning. The only training module is TotoForFinetuning. Its defaults are a learning rate of 1e-4, warmup 200, stable 1000, decay 1000, betas (0.9, 0.999), weight decay 0.01 and epsilon 1e-7 lightning_module.py lightning_module.py. None of the pretraining values in app. A.6 appear in the repository, and no pretraining script was found.
8. Loss on scaled targets. The code scores targets after subtracting the causal location and dividing by the scale plus a small epsilon [code: toto/model/lightning_module.py#L252]. The paper does not state this step explicitly.
9. Mixture numerics. Degrees of freedom and scales are floored at the machine epsilon of the tensor dtype distribution.py. The paper calls ε a small positive constant without a value §3.1 app. A.3.
10. Learning-rate schedule. The code's warmup-stable-decay scheduler decays as one minus the square root of the decay progress [code: toto/model/scheduler.py#L44-L61]. The paper names WSD and gives step counts app. A.6, but not the decay shape.
11. Leaderboard ranks. The repository leaderboard lists the same MASE (0.617) and CRPS (0.375) for toto on BOOM, but a rank of 6.939, not 2.369 boom_leaderboard.csv tab. 2. On BOOMlet it lists rank 6.4, not 1.300 boomlet_leaderboard.csv. The rank columns are labelled differently, and the repository does not say why the values differ.
12. Inconsistent ranks in the paper. The text gives mean ranks of 2.351 and 4.278 §5. The table gives 2.369 and 4.328 tab. 2.
13. Sampling defaults. The README example uses 256 samples and a per-batch sample count of 256 README.md, which matches the paper's BOOM setting app. C.2.3.
14. KV cache. The backbone allocates a KV cache for decoding [code: toto/model/backbone.py#L137-L156]. The paper reports the cache in its efficiency comparison app. D.4.

Reviewed but no mismatch found: the Student-T mixture head and the composite loss formulas distribution.py losses.py. The pre-normalisation, RMSNorm, SwiGLU and XPOS wiring also match the paper transformer.py transformer.py transformer.py.

Pitfalls

• Causality differs between training and inference. Under the default global stabilisation, training scales with a clamp over the whole input window, while inference uses the context only forecaster.py backbone.py. An implementation that follows the paper's causal definition everywhere will not match training. One that copies the training path leaks future points into the scale.
• The offset placement changes the first steps. Following the prose adds 0.1 to the standard deviation, which gives a scale of 0.1 at the first time step. The formula and the code add it under the root, which gives a scale of about 0.316 at that step scaler.py.
• Clip reference. Comparing the causal standard deviation with a variance changes the band by a square. Use the global standard deviation, as the code does scaler.py.
• The released configuration is missing from the repo. The layer order, the scaler class and the number of mixture components used by the released weights are set in a configuration file that the repository does not contain. Check against the model card of Datadog/Toto-Open-Base-1.0 or against the weights themselves before reproducing the released model.
• Pretraining is not reproducible from this repo. The pretraining corpus is Datadog's internal telemetry, which is 43% of the mix §3.2. The public part is listed in app. B.2. The generator parameters for the synthetic series are not stated app. B.3. The variate-shuffling rule uses a probability of about 14% and batches of 32 variates app. B.4.
• Packing needs the ID mask. Several series can share one batch app. A.2. Without the block-diagonal ID mask in the variate-wise block, variates from different series attend to each other.
• The benchmark protocol shapes the comparison. All foundation models use a context of 2048 app. C.2.3. The sample counts differ across models. TimesFM and Chronos are run per variate, while Moirai and Toto are run jointly [app. C.2.3]. Statistical models are capped at 1000 points of input app. C.2.3. TabPFN was not run on BOOM because its batched inference was too slow §5.
• Aggregation differs between tables. BOOM uses a shifted geometric mean with ε = 1e-5 app. C.2.4 eq. 15. GIFT-Eval uses a plain geometric mean tab. 3. Compare numbers only within one table.
• BOOM is not independently reproducible here. The data comes from Datadog staging, and the authors say isolation from production is guaranteed at the platform level §4.1. The repository holds result CSVs, leaderboards and notebooks. This card did not confirm that the 350-million-point evaluation data can be downloaded.
• Scaling numerics. Statistics are computed in float64. On devices without float64, the code falls back to float32 with a warning, and extreme values can then overflow scaler.py.
• Efficiency numbers are one forward pass. The efficiency comparison uses synthetic series and a single sample app. D.4. Sampling 256 paths multiplies the decoding work, and the paper does not report that cost.
• Probabilistic forecasts need the sample count. Prediction intervals come from the samples, so their width depends on how many are drawn app. A.4. The sampling granularity in the code was not checked against the paper.

Lineage

• Builds on: decoder-only next-patch pretraining, a patch-based input and factorised attention, with RoPE and XPOS, RMSNorm, SwiGLU and robust losses §3.1 app. A.2. These works are cited in the paper, and none has a card in this wiki. The Student-T output follows TiDE app. A.3, see TiDE.
• Compared against: TimesFM 2.0, Moirai, Chronos-Bolt, Timer, Time-MoE, VisionTS, TabPFN-TS, TEMPO, TTM-R2, PatchTST, TFT, DLinear and classical statistical models tab. 2 tab. 3 tab. 4.
• Superseded by: not stated in this wiki's content. The code repository ships Toto 2.0 (see Paper vs code), which this card does not cover.