Tabular foundation model

Overview

A tabular foundation model is a network that was pretrained on many synthetic tables. Given a new table, it reads the labelled rows as context and predicts the unlabelled rows in one forward pass, with no gradient step on the new data 2501.02945 §2 2501.02945 §3.2.

The invariant the variants share is that the pretrained weights stay as they were released. The work is in turning the task into a table. The one card on this page does this for time series: each future or past time step becomes a row of features, and the observed values become the targets [arxiv:2501.02945 §3.1].

Variants

Time features with a pretrained regressor

Each row holds a running index, cyclic calendar features, a few automatically found seasonal periods, and any covariates whose future values are known. The width is 18 + 2k + D_z, where k is the number of seasonal periods and D_z the number of covariates 2501.02945 §3.1. The backbone is TabPFN-v2, an 11M-parameter model that was not pretrained on time series [arxiv:2501.02945 §1] 2501.02945 §3.2. Card: From Tables to Time: Extending TabPFN-v2 to Time Series Forecasting.

The same formulation handles univariate series and covariate-informed series. Lagged and autoregressive features are not used, because they need earlier predictions 2501.02945 §3.1.

When to use which

Covariates help most on pulses, ramps and random walks, and not on linear trends, in the paper's synthetic study 2501.02945 §5.3. Longer context helps up to 4096 observations; the gains level off after that [arxiv:2501.02945 app.]. Each seasonal period found adds two columns, so the paper's feature width grows with k 2501.02945 §3.1.

Implementation sketch

Language-agnostic pseudocode of the common core. The variant-specific parts are the feature blocks and the choice of point estimate.

# Forecast H steps of one series with a frozen tabular regressor
# [arxiv:2501.02945 §3.1] [arxiv:2501.02945 §3.2]
# Input:  y_hist (C,) observed values; t_all (C+H,) timestamps
#         cov_all (C+H, D_z) known-future covariates, or None
# Output: point forecast (H,), quantile bands (H, Q)

def forecast(y_hist, t_all, cov_all, k, quantiles):
    # One row per time step; index, calendar, seasonal, covariates [§3.1]
    X = stack_features(t_all, y_hist, k, cov_all)
    # First C rows are the context, last H rows are the queries [§3.2]
    X_train, X_test = X[:C], X[C:]
    y_train = y_hist
    # Pretrained regressor; no fitting on the new series [§1] [§3.2]
    dist = tabular_regressor(X_train, y_train, X_test)
    # Mean for squared error, median for absolute error [§3.2]
    point = dist.mean()
    # Quantiles for bands and probabilistic metrics [§3.2]
    bands = dist.quantiles(quantiles)
    return point, bands

Contradictions

None known. Only one card lists this concept. The paper reports its results in figures, and the card records where the released code differs from the paper, for example in the number of seasonal periods and the point estimate used by default. Those differences are about the code, not about conflicting results, and they are listed on the card's Paper vs code section.

Papers

• From Tables to Time: Extending TabPFN-v2 to Time Series Forecasting: casts forecasting as tabular regression with a pretrained TabPFN-v2 backbone, and reports zero-shot results on GIFT-Eval and fev-bench (author-reported).