wikillm

Paper · 2025-10-07 · cs.LG

TabPFN-Wide: Continued Pre-Training for Extreme Feature Counts

TabPFN-Wide continues pre-training the TabPFNv2 classifier on synthetic tables widened from a TabICL prior, so it accepts over 30,000 features. Continuous features get sparse random projections with noise, categorical ones copied and merged columns. Author-reported: matches TabPFNv2 on TabArena, and a Wide variant has the best mean AUROC on four of five TCGA omics sets.

Tabular foundation modelContinual learningSynthetic training dataAttention interpretabilitytabularfoundation-modelcontinued-pretraininghigh-dimensionalinterpretability

arXivPDFCodeMarkdown17 min read

AI-compiled by claude-haiku-5-5 on 2026-11-13, not yet reviewed by a human. Every claim cites its source; check it when a detail matters.

In plain words

Tabular foundation models such as TabPFN answer prediction questions about a table in a single pass. They learned this from many synthetic tables during pretraining. Their limit is width: the TabPFNv2 version was built for tables with at most a few hundred features, so wide data has to be cut down first. In biomedicine, a study may have a few hundred patients and tens of thousands of gene measurements. Cutting the columns throws away information, and it also makes it hard to say which measurements mattered.

The authors keep the pretrained model and train it a little longer, on synthetic tables that are made wide. Each table gets extra columns built from its original ones: continuous columns become sparse mixtures with noise added, and categorical columns copy values from other columns. Training on these tables teaches the model to handle wide inputs. The model's attention towards each input feature can then serve as a rough ranking of what it relies on. On a standard benchmark it stays close to its base model, and on the cancer gene data it beats the base model and the common baselines on most datasets.

TL;DR

TabPFN-Wide is the TabPFNv2 classifier, continued from its checkpoint on synthetic tables that are widened to 1,500, 5,000 or 8,000 maximum features. It is meant to work on more than 30,000 features without feature reduction. The paper reports that it matches TabPFNv2 on TabArena (Spearman 0.9935) and that a Wide variant gives the best mean AUROC on four of five TCGA omics datasets. The attention from the label token to each feature is used as a feature-importance score. The official code exists, but several of its settings differ from the paper, so read Paper vs code before reproducing.

Problem

Tabular data are heterogeneous tables, and the high-dimensional, low-sample-size (HDLSS) case means many more features than samples, typical in biomedicine § Methods (Problem Description). TabPFNv2 handles up to 10,000 samples and 500 features § Methods (Tabular Foundation Models for Predictive ML Tasks). Feature reduction makes it usable on wider data, but it loses information and blocks per-feature importance analysis § Methods (Problem Description). Post-hoc methods such as SHAP are costly on high-dimensional data and unstable at low sample sizes § Methods (Problem Description). The authors instead want to extend an existing model, not replace it, without losing per-feature interpretability § Introduction. Sequential training risks catastrophic forgetting of the original feature range § Methods (Tabular Foundation Models for Predictive ML Tasks). The omics data have up to 60,664 features and 247 to 440 patients per dataset app. A tab. 1.

Method

Section names are given in place of numbers because the paper's sections are unnumbered. Figure and table numbers follow the caption order in the LaTeX source and were not checked against a compiled PDF.

Overview. Three parts: a prior for synthetic wide data, continued pre-training of TabPFNv2 on that data, and attention-based feature scores § Methods (Methodology). Each prior dataset is drawn from the open-source TabICL prior, which the authors use as a stand-in because the TabPFNv2 prior is not public § Methods (A Prior for Synthetic HDLSS Data Generation). Its datasets have a moderate feature count m and are then widened to a target dimension d with d much larger than m § Methods (A Prior for Synthetic HDLSS Data Generation).

Feature widening. Each dataset is widened independently. Columns with 20 or fewer distinct values count as categorical, and the others as continuous. Let r_cat be the share of categorical columns. The number of new categorical columns is the floor of r_cat times (d − m), and the remaining new columns are continuous § Methods (A Prior for Synthetic HDLSS Data Generation).

  • Continuous features get a random sparse linear map with keep-probability p, followed by feature-dependent Gaussian noise whose scale is sigma times the standard deviation of each new feature alg. 1 § Methods (A Prior for Synthetic HDLSS Data Generation).
  • Categorical features copy values from a small set of donor columns per new feature, with the donor set size max(1, floor(p m_cat)). Each row picks one donor uniformly. The new column is then reduced to a target cardinality by merging the rarest categories into one of the most frequent ones alg. 2 § Methods (A Prior for Synthetic HDLSS Data Generation).
Pseudocode
# Continuous feature widening  [alg. 1]
# Input:  x_cont (n, m_cont); new count d_cont; keep-prob p; noise sigma
# Output: wide_cont (n, d_cont)

def widen_continuous(x_cont, d_cont, p, sigma):
    # Gaussian projection weights per pair  [alg. 1]
    weights = sample_normal(m_cont, d_cont)        # (m_cont, d_cont)
    # Bernoulli(p) mask keeps each weight with probability p  [alg. 1]
    mask = sample_bernoulli(p, m_cont, d_cont)     # (m_cont, d_cont)
    # Sparse linear projection of the features  [alg. 1]
    wide = x_cont @ (mask * weights)               # (n, d_cont)
    # Std of each new column, used to scale its noise  [alg. 1]
    column_std = std(wide, axis=0)                 # (d_cont,)
    # Noise scale: sigma times that std  [alg. 1]
    noise = sample_normal(n, d_cont) * sigma * column_std
    return wide + noise
Pseudocode
# Categorical feature widening  [alg. 2]
# Input:  x_cat (n, m_cat); new count d_cat; keep-prob p; max categories
# Output: wide_cat (n, d_cat)

def widen_categorical(x_cat, d_cat, p, k_max):
    # Donor set size for each new feature, at least one  [alg. 2]
    n_donors = max(1, floor(p * m_cat))
    wide_cat = empty(n, d_cat)
    for each new column j in range(d_cat):
        # Donor columns for this feature  [alg. 2]
        donors = sample_without_replacement(m_cat, n_donors)
        # Each row copies one donor value, drawn uniformly  [alg. 2]
        column = copy_random_donor_per_row(x_cat, donors)
        # Target cardinality K_j drawn from {3, ..., k_max}  [alg. 2]
        k_target = sample_cardinality(3, k_max)
        # Fold rarest value into a top-k value  [alg. 2]
        while count_unique(column) > k_target:
            column = merge_rarest_into_top(column, k_target)
        wide_cat[:, j] = column
    return wide_cat

Continued pre-training. The TabPFNv2 classifier checkpoint is the starting point, and all parameters are updated § Methods (Continued Pre-Training). Training uses the widened prior datasets with a cross-entropy loss. The target width d is sampled from a range that starts at 200 and ends at a chosen upper bound d_max. The paper uses three upper bounds, 1,500, 5,000 and 8,000, and trains one model for each § Methods (Continued Pre-Training). With probability 0.5 the original features are appended, and then the column order is randomly permuted. Sparsity p is drawn uniformly from [0, 0.05] and noise sigma from [0, 1] § Methods (Continued Pre-Training). Training runs for a fixed 10,000 optimisation steps, chosen because validation ROC-AUC plateaued there § Methods (Continued Pre-Training) app. J.

Pseudocode
# One continued pre-training step  [§ Methods] (Continued Pre-Training)
# Input:  model (TabPFNv2 classifier, all weights trainable); d_max
# Output: scalar cross-entropy loss

def training_step(model, prior_datasets, d_max):
    # Target width, from [200, d_max]  [§ Methods] (Continued Pre-Training)
    d_target = uniform_int(200, d_max)
    # Sparsity p, noise sigma: uniform  [§ Methods] (Continued Pre-Training)
    p = uniform(0.0, 0.05)
    sigma = uniform(0.0, 1.0)
    # Keep originals with probability 0.5  [§ Methods] (Continued Pre-Training)
    keep_original = uniform(0.0, 1.0) < 0.5
    widened = [
        widen_dataset(dataset, d_target, p, sigma, keep_original)
        for dataset in prior_datasets
    ]
    # Predict test rows from train rows  [§ Methods] (Continued Pre-Training)
    logits = model(widened)
    return cross_entropy(logits, labels_test)

The pseudocode follows the paper. The code samples the target width, p, sigma and the keep decision once per training step, shared by the whole batch, and it reads d as the number of added features. See Paper vs code.

Feature-wise interpretability. Attention maps are an intermediate of dot-product attention; the attention matrix A is softmax(Q K^T / sqrt(d_key)) and the output is A V eq. 1 § Methods (Feature-wise Interpretability via Attention Maps). The authors use only feature-wise attention and ignore sample-wise attention. The score of a feature is its attention from the label token, which is the last row of the map with the label column removed. These values are averaged over all samples, heads and layers § Methods (Feature-wise Interpretability via Attention Maps).

Pseudocode
# Score per original feature  [§ Methods] (Feature-wise Interpretability)
# Input:  maps, a list over samples, heads and layers of (tokens, tokens)
#         matrices; the last token is the label
# Output: scores (features,)

def attention_scores(maps):
    # Label attention to features  [§ Methods] (Feature-wise Interpretability)
    label_rows = [attention[-1, :-1] for attention in maps]
    # Average over all three axes  [§ Methods] (Feature-wise Interpretability)
    return mean(label_rows, axis=0)                # (features,)

Disabling TabPFNv2's feature grouping, its added distribution features and its feature removal keeps one token per input feature § Methods (Feature-wise Interpretability via Attention Maps).

Hyperparameters

Values in the table come from the paper unless the Where column says code. The paper does not state every value, so the Notes column records the gaps.

NameValueWhereNotes
Base modelTabPFNv2 classifier checkpoint, all parameters updated§ Methods (Continued Pre-Training), footnoteThe checkpoint comes from the Prior-Labs/TabPFN-v2-clf Hugging Face repository. The code asserts that the model config matches the checkpoint config train.py.
OptimiserAdamW§ Methods (Continued Pre-Training)
Learning rate1e-5§ Methods (Continued Pre-Training); train.sh
Weight decay1e-4§ Methods (Continued Pre-Training); train.sh
LR scheduleLinear warm-up, then cosine decay§ Methods (Continued Pre-Training)Warm-up length not stated. The script sets warm-up proportion to 0 and 10 cosine cycles train.sh train.py.
Gradient norm clip1.0§ Methods (Continued Pre-Training); train.sh
Batch size16; 8 for runs above 5,000 features§ Methods (Continued Pre-Training)The script passes 8 in total, split over 2 processes, so 4 per process train.sh train.py.
Training steps10,000§ Methods (Continued Pre-Training); app. JThe script sets 30,000 train.sh.
Numerical precisionNot statedThe script uses float16 with autocast train.sh train.py.
LossCross-entropy§ Methods (Continued Pre-Training)The code computes it on the test rows only train.py.
Prior classesUp to 10§ Methods (Continued Pre-Training)Matches the TabPFNv2 limit § Methods (Continued Pre-Training).
Prior samples40 to 400§ Methods (Continued Pre-Training)The script uses a maximum of 300 train.sh.
Prior features50 to 350§ Methods (Continued Pre-Training)train.sh
Prior train fractionNot stated in paperCode: 0.3 to 0.9 train.sh.
Added-feature range200 to d_max§ Methods (Continued Pre-Training)The code samples the number of added features from this range train.py.
d_max1,500; 5,000; 8,000§ Methods (Continued Pre-Training)train.sh train.sh train.sh
Sparsity pUniform in [0, 0.05]§ Methods (Continued Pre-Training)train.sh config.py
Noise sigmaUniform in [0, 1]§ Methods (Continued Pre-Training)train.sh config.py
Keep original featuresProbability 0.5, then permute columns§ Methods (Continued Pre-Training)utils.py utils.py utils.py
Categorical thresholdAt most 20 distinct values§ Methods (A Prior for Synthetic HDLSS Data Generation)The code also has a ratio test that cannot trigger for up to 1,000 samples utils.py utils.py.
Max categories K_maxNot stated in paperalg. 2Code uses 20 utils.py utils.py.
Category cardinalityDiscrete exponential over {3..K_max}§ Methods (A Prior for Synthetic HDLSS Data Generation)Code uses decay 0.08 utils.py utils.py. alg. 2 says uniform.
Cross-validation5-fold for biomedical data; TabArena 3-fold, repeated 3 or 10 times§ Methods (Datasets and Evaluation Protocol)
Hardware4 NVIDIA H100 GPUs, 320 GB combined memory§ Methods (Continued Pre-Training), footnoteThe sbatch file requests 4 GPUs training.sbatch while torchrun starts 2 processes train.sh.
Architecture (code only)192 embedding size, 12 layers, 6 heads, feed-forward factor 4, 1 feature per group, 0 thinking rowsNot in papertrain.sh train.sh train.sh train.sh

Results

Author-reported. Setup, as stated: five TCGA multi-omics datasets with early integration by concatenation, 5-fold cross-validation, AUROC and accuracy, and no ensembling so that the raw model behaviour is measured § Experiments (Datasets and Evaluation Protocol). The baselines are TabPFNv2, TabICL, RealMLP-TD, Random Forest and XGBoost § Experiments (Datasets and Evaluation Protocol). The 5k model is the main TabPFN-Wide model § Experiments (Results on real-world wide datasets).

Multi-omics AUROC. Average AUROC ± SEM over 5-fold cross-validation tab. 3. Columns are datasets; the best value in each column is marked in the paper.

ModelBRCACOADGBMLGGOV
#features18,20617,26118,61414,26014,229
TabPFN v20.733 ± 0.0290.710 ± 0.0380.794 ± 0.0540.973 ± 0.0180.836 ± 0.036
Wide (1.5k)0.951 ± 0.0170.925 ± 0.0510.941 ± 0.0170.988 ± 0.0090.987 ± 0.014
Wide (1.5k, No-Cat)0.954 ± 0.0180.933 ± 0.0470.960 ± 0.0180.986 ± 0.0220.977 ± 0.013
Wide (5k)0.971 ± 0.0120.954 ± 0.0340.962 ± 0.0210.988 ± 0.0220.981 ± 0.010
Wide (5k, No-Cat)0.973 ± 0.0120.953 ± 0.0320.961 ± 0.0190.988 ± 0.0200.982 ± 0.012
Wide (8k)0.952 ± 0.0170.959 ± 0.0270.936 ± 0.0210.987 ± 0.0210.982 ± 0.011
Wide (8k, No-Cat)0.978 ± 0.0090.956 ± 0.0300.965 ± 0.0240.987 ± 0.0210.982 ± 0.010
TabICL0.937 ± 0.0130.828 ± 0.0330.898 ± 0.0200.973 ± 0.0210.883 ± 0.033
Random Forest0.967 ± 0.0050.911 ± 0.0110.954 ± 0.0250.986 ± 0.0170.974 ± 0.011
XGBoost0.967 ± 0.0150.926 ± 0.0430.951 ± 0.0420.987 ± 0.0160.969 ± 0.020
RealMLP0.966 ± 0.0070.932 ± 0.0230.960 ± 0.0280.991 ± 0.0080.970 ± 0.018

The best value is a Wide variant in BRCA, COAD, GBM and OV. RealMLP is best in LGG tab. 3. The 5k model's values are 0.971, 0.954, 0.962, 0.988 and 0.981, against 0.733, 0.710, 0.794, 0.973 and 0.836 for TabPFNv2 tab. 3.

Multi-omics accuracy. Average accuracy ± SEM over the same datasets tab. 4.

ModelBRCACOADGBMLGGOV
#features18,20617,26118,61414,26014,229
TabPFN v20.528 ± 0.0060.669 ± 0.0090.303 ± 0.0100.887 ± 0.0480.514 ± 0.049
Wide (1.5k)0.744 ± 0.0220.835 ± 0.0170.717 ± 0.0500.919 ± 0.0200.877 ± 0.075
Wide (1.5k, No-Cat)0.797 ± 0.0250.862 ± 0.0160.771 ± 0.0700.976 ± 0.0260.852 ± 0.027
Wide (5k)0.852 ± 0.0320.858 ± 0.0170.799 ± 0.0710.964 ± 0.0090.880 ± 0.049
Wide (5k, No-Cat)0.852 ± 0.0280.873 ± 0.0110.787 ± 0.0880.960 ± 0.0000.877 ± 0.050
Wide (8k)0.759 ± 0.0170.869 ± 0.0420.737 ± 0.0590.972 ± 0.0110.866 ± 0.064
Wide (8k, No-Cat)0.864 ± 0.0290.873 ± 0.0170.816 ± 0.0630.964 ± 0.0090.887 ± 0.034
TabICL0.767 ± 0.0560.781 ± 0.0730.660 ± 0.0450.911 ± 0.0370.676 ± 0.063
Random Forest0.815 ± 0.0140.831 ± 0.0420.779 ± 0.0920.960 ± 0.0140.859 ± 0.047
XGBoost0.867 ± 0.0260.862 ± 0.0370.775 ± 0.0870.964 ± 0.0170.838 ± 0.072
RealMLP0.845 ± 0.0160.892 ± 0.0220.783 ± 0.0940.955 ± 0.0230.827 ± 0.041

Standard benchmarks. On 21 TabArena classification tasks (at most 10,000 samples and 500 features), the per-task AUROCs of TabPFN-Wide (5k) and TabPFNv2 have a Spearman rho of 0.9935. The authors read this as no sign of catastrophic forgetting § Experiments (Results on Standard Benchmarks and Widened Adaptations) fig. 4.

HDLSS benchmark of Li et al. Across 15 HDLSS datasets, TabPFN-Wide (5k) has a pairwise AUROC win rate of 92.0%. A paired Wilcoxon signed-rank test against each baseline gives the p-values in tab. 5: Random Forest 0.00427, RealMLP 0.00116, XGBoost 0.00012, TabPFNv2 0.00018, TabICL 6.10e-5 § Experiments (HDLSS Benchmark from Li et al.) fig. 4 app. H.

Needle-in-a-haystack on SNPs. Synthetic HAPNEST SNP data with polygenicity 0.001, 0.01 and 0.05 on chromosome 1 app. K. As non-causal SNPs are added, TabPFN-Wide and XGBoost degrade least, and TabICL drops toward AUROC 0.5. This is a qualitative claim from the plot § Experiments (Needle in a haystack) fig. 4 fig. 13.

Synthetic signal recovery. Binary datasets from scikit-learn's make_classification, 50 samples, feature counts 200 to 5,000, 3, 5 or 7 informative features, 5 seeds, against a Random Forest with 200 trees. The authors judge the attention ranking comparable to the Random Forest on both the top-k hit rate and the gap between signal and noise scores, shown only as plots app. F fig. 8 § Experiments (Interpretability).

Biomarkers. The top 10 genes by attention per dataset were checked against the literature by hand app. A tab. 2. In BRCA, nine genes link directly to breast cancer; for the OV dataset, six genes link directly to ovarian cancer and two link generally to cancer. The paper says the list is not exhaustive § Experiments (Interpretability).

Paper vs code

Official code at commit 150dcafe, Prior Labs License 1.1. The code is in raw/code/2510_06162, and its path is relative to that repository.

  • Projection weights: the paper draws them from N(0, 1) alg. 1. The code takes the default PyTorch initialisation of the linear layer and then applies the mask utils.py.
  • Noise scale: the paper writes N(0, sigma * sigma_j) alg. 1. The code treats this as a standard deviation, uses the population std, and replaces zero or NaN stds with 1 utils.py utils.py.
  • Category cardinality: the paper's algorithm samples K_j uniformly from {3..K_max} alg. 2, while its text says a discrete exponential § Methods (A Prior for Synthetic HDLSS Data Generation). The code uses the exponential with decay 0.08 over {3..20} utils.py utils.py.
  • K_max: not stated in the paper. The code uses 20 utils.py utils.py.
  • Merging: the algorithm merges one rare category at a time until K_j remain alg. 2. The code maps all rare categories to random frequent ones in one pass, with the same final count utils.py.
  • Target width: the paper samples the target dimension d in [200, d_max] and uses d − m as the number of new features § Methods (Continued Pre-Training) § Methods (A Prior for Synthetic HDLSS Data Generation). The code samples the number of added features in that range train.py. The total is then m plus that number utils.py.
  • Sharing across the batch: the code draws the added count, p and sigma once per training step, shared by all datasets in the batch train.py train.py. The paper does not say at what granularity these are drawn.
  • Original features: the code draws the keep decision once per batch, not per dataset utils.py. The paper does not say at what granularity.
  • Training steps: the paper uses 10,000 § Methods (Continued Pre-Training). The script sets 30,000 train.sh.
  • LR schedule: the paper says linear warm-up then cosine decay. The script sets warm-up to 0 and runs 10 cosine cycles with a scheduler imported from the external tabicl package train.sh train.py train.py.
  • Batch size: the paper gives 16 (8 above 5,000 features). The script passes 8 in total, split over 2 processes train.sh train.py. The sbatch file requests 4 GPUs training.sbatch.
  • Prior sample length: the paper gives 40 to 400 samples § Methods (Continued Pre-Training). The script's maximum is 300 train.sh.
  • Monitoring: the paper monitors two omics sets and three HAPNEST SNP sets app. J. The validation loop in the code uses only COAD and gbm with three omic combinations config.py train.py. No SNP set appears in that path.
  • Precision: the code uses float16 autocast. The paper does not mention it train.sh train.py.
  • Categorical detection: the paper says at most 20 distinct values. The code's helper also has a ratio test (0.001 of samples) that cannot trigger for up to 1,000 samples, so the effective rule matches the paper utils.py utils.py.

Pitfalls

  • Ambiguous width. The paper's "target dimension d" and the code's "number of added features" differ. Algorithm 2 and the d_cat formula use d − m as the new-feature count, so d is the total § Methods (A Prior for Synthetic HDLSS Data Generation). The code samples the added count from [200, d_max] train.py. Pick one and document it.
  • Scale of the projection. The noise scale is sigma times the std of each new column. Scaling the projection weights therefore scales a new feature and its noise together, so the signal-to-noise ratio does not depend on that scale. Whether the absolute scale matters depends on TabPFN's own preprocessing, which this card did not check.
  • Zero-variance columns. The code replaces a zero std with 1, so a constant new column still gets noise of scale sigma utils.py. The paper does not mention this.
  • Batch-level sampling. Width, sparsity, noise and the keep decision are shared across the batch in the code train.py. Per-dataset sampling would change the variance of the training signal.
  • Attention as importance. The paper's own limitations section says attention maps may be misleading on real data and that the synthetic ground-truth test is more reliable § Conclusion (Limitations and Outlook). The biomarker check is manual and "should not be considered exhaustive" app. A.
  • Attention needs fixed settings. The attention maps are only valid with a single estimator and 1 feature per group classifier.py classifier.py. Ensembling and feature grouping break the token-to-feature mapping.
  • Label attention averages over rows. The code's get_attention_to_label averages over every row of the last predict call, which includes training rows, and over all layers classifier.py. Other aggregations need a different save mode.
  • Quadratic cost. The architecture is unchanged, so attention memory grows with the number of features, which limits how wide and how many samples the model can take § Conclusion (Limitations and Outlook).
  • TabPFNv2 only. The prior only works for continued training of TabPFNv2. The TabICL attempt did not improve in training app. I fig. 11.
  • Classifier only. Only the classifier was continued; the regressor was not tested § Conclusion (Limitations and Outlook).
  • Internal inconsistencies in the paper. The Figure 2 caption gives the omics means as ± SD, while the Appendix D table gives ± SEM for the same numbers fig. 2 tab. 3. The BRCA feature count is 26,577 in the dataset table tab. 1 but 18,206 in the results table tab. 3, and the paper does not explain the difference.
  • Hidden training cost. The paper's run used 4 H100 GPUs with 320 GB combined § Methods (Continued Pre-Training). The sbatch file requests a 72-hour limit on one node training.sbatch.
  • External dependencies. The TabICL prior (tabicl.prior.dataset) and the scheduler come from the external tabicl package, not from this repository train.py. The base checkpoint is downloaded from Hugging Face, and its licence is not stated in the paper.
  • Licence obligations. The code is under the Prior Labs License 1.1. It is an Apache-2.0 derivative with an extra attribution clause, and it does not restrict commercial use LICENSE. Distributing the model or a product with it requires a copy of the licence and the text "Built with PriorLabs-TabPFN" on related pages, and any derived model that is distributed must include "TabPFN" at the start of its name LICENSE. Copying code snippets was not done, because this licence is not one of the permissive licences (MIT, Apache-2.0, BSD) that allow quotation under this wiki's rules.
  • Results not re-run. The author-reported numbers come from the paper's tables. The repository contains analysis CSVs, which this card did not compare against the tables.

Lineage

  • Builds on: the TabPFNv2 classifier checkpoint, which is the starting point and is updated in full § Methods (Continued Pre-Training). The TabICL prior supplies the synthetic datasets § Methods (A Prior for Synthetic HDLSS Data Generation). Neither has a card in this wiki.
  • Compared against: TabPFNv2, TabICL, RealMLP-TD, Random Forest and XGBoost § Methods (Datasets and Evaluation Protocol). The paper also builds on an earlier analysis of how TabPFNv2's attention changes across layers § Methods (Problem Description).
  • Superseded by: not known from the paper or the wiki.

Source and license

Adapted from “TabPFN-Wide: Continued Pre-Training for Extreme Feature Counts” by Christopher Kolberg, Jules Kreuer, Jonas Huurdeman, Sofiane Ouaari, Katharina Eggensperger, Nico Pfeifer (arXiv:2510.06162), used under CC BY 4.0. Changes: condensed and restructured for implementation, with tables, equations and algorithms taken from the paper and its code, each cited to its source. Not affiliated with or endorsed by the authors. Code citations refer to https://github.com/not-a-feature/TabPFN-Wide at commit 150dcafef60a (Prior Labs License 1.1 (Apache-2.0 derivative with an attribution clause; not non-commercial)). Any quoted code is shown under that license.