SBI -- A toolkit for simulation-based inference

In plain words

Many scientific models are simulators: you can run them with a set of settings and get an output, but you cannot write down the probability of seeing a given output. This paper presents sbi, a Python package built on PyTorch. Given a simulator, a prior over its settings, and an observed result, it learns from simulated runs which settings are consistent with the observation. The answer is a distribution over the settings, not a single best guess, so you can see how uncertain each setting is.

The package bundles several neural-network methods behind one interface. A simple mode runs a single call with defaults, and a flexible mode lets you choose the network and the training loop. The paper is mainly a description of the toolkit and its design goals, not of a new method.

TL;DR

Use this card if you need a ready-made, PyTorch-based toolkit for inference on black-box simulators with neural estimators. The paper gives the scope, method families and interface design only. It states no hyperparameters, no benchmark results and no algorithm box. For implementation details, read the code at the commit above, and check the Paper vs code section because the code is much newer than the paper.

Problem

Simulation-based inference (SBI) aims to find the parameter sets that are consistent with prior knowledge and with observed data, and to return the posterior distribution over them rather than one point estimate §1. Classic likelihood-based Bayesian methods need a computable likelihood. When a simulator has none, the paper names approximate Bayesian computation (ABC), which uses Monte-Carlo rejection, as the traditional alternative, and points to neural methods that train conditional density estimators or classifiers on simulated data instead §1. The paper's stated goal is to make these neural methods usable without custom networks or manual tuning, for scientists and engineers who are not specialists §2.

Method

The paper says sbi implements three families of neural inference algorithms §2:

1. Sequential Neural Posterior Estimation (SNPE): trains a density estimator that gives the posterior directly. The paper names SNPE-C as the implemented variant §2.
2. Sequential Neural Likelihood Estimation (SNLE): trains a density estimator of the likelihood, then samples the posterior with a method such as MCMC §2.
3. Sequential Neural Ratio Estimation (SNRE): trains a classifier that estimates density ratios, which are then used to sample the posterior, for example with MCMC §2.

The inference step returns a NeuralPosterior object. It can be sampled and, where the method allows, evaluated, through the same API as a PyTorch probability distribution §2.

Toolkit features, as the paper lists them §2:

• It accepts any simulator that can be wrapped in a Python callable §2.
• It handles input shapes automatically, standardizes inputs and handles failed simulations §2.
• It vectorizes simulations where possible and can run them in parallel with joblib §2.
• A learned summary network can reduce high-dimensional simulator output to features §2.
• A simple interface runs one function call with reasonable defaults. A flexible interface allows custom networks and hyperparameters §2.

The paper's density estimators are normalizing flows built with the nflows package §1.1. The paper states that sbi is closely integrated with PyTorch §1.1, §2.

Pseudocode for the simple interface, reconstructed from the code. The paper does not contain this algorithm.

# Simple interface: one-round, amortized inference  [§2]
#   [code: sbi/inference/trainers/base.py#L128-L173]
# Input:  simulator (theta -> x), prior with .sample() and .log_prob()
# Output: posterior p(theta | x) that can be sampled for any x

def infer_posterior(simulator, prior, method, num_simulations):
    # Draw parameters from the prior and run the simulator on them
    #   [code: sbi/utils/simulation_utils.py#L22-L61]
    theta, x = simulate_for_sbi(simulator, prior, num_simulations)
    # (N, D) parameters, (N, K) simulator outputs

    # Drop NaN and Inf outputs when the method excludes invalid x
    #   [code: sbi/utils/sbiutils.py#L553-L583]
    valid = find_valid_rows(x)
    theta, x = theta[valid], x[valid]

    # Train the neural estimator named by `method`: npe, nle, nre, etc.
    #   [code: sbi/inference/trainers/base.py#L158-L160]
    trainer = build_trainer(method, prior)
    density_estimator = trainer.append_simulations(theta, x).train()

    # Wrap the estimator as a posterior object  [§2]
    return build_posterior(density_estimator, prior)

Key equations: none are stated in the paper.

Hyperparameters

The paper states no hyperparameters §1, §2. The table below lists only the defaults that the code's simple interface sets. The code is a much later release than the paper, so treat these values as code defaults, not as paper settings.

Results

Not stated. The paper reports no benchmark or experimental results. It describes a software package and its design, and has no tables or figures §1 §2.

Paper vs code

Code cited at commit cd8f131444471749f2ce9baa8a8aaddccbde4c37. The latest release in the changelog is v0.27.0 CHANGELOG.md. The paper describes the 2020 release, so differences are expected.

• License. The paper's footnote says sbi is available under AGPLv3 p. 1. The repository's LICENSE file at this commit is the Apache License 2.0 LICENSE.txt. This card lists Apache-2.0 because it is the licence of the code cited here. The licence of the paper's 2020 release was not checked.
• Method names. The paper names SNPE-C, SNLE and SNRE §2. The code has trainers named NPE_C, NLE_A, NRE_A and NRE_C, plus NPE_A, NPE_B, NRE_B and vector-field methods FMPE and NPSE npe_c.py, nle_a.py, nre_a.py, nre_c.py. The code does not state that NPE_C is SNPE-C. The mapping here is inferred from the names. The legacy names snpe, snle and snre are still accepted, with a deprecation warning attached base.py, base.py.
• Simple interface. The paper describes a one-call interface with defaults §2. The code docstring says it runs one round and returns an amortized posterior, and it points to a separate flexible interface for multi-round inference base.py. The paper does not mention rounds.
• Parallel simulation. The paper says joblib is used §2. The code uses joblib in simulate_for_sbi, and it converts parameters to numpy arrays when num_workers is not 1 simulation_utils.py. Joblib is also used in the posterior sampling code mcmc_posterior.py.
• Failed simulations. The paper says failed simulations are handled §2. The code counts NaN and Inf rows and, when exclusion is on, drops them sbiutils.py.
• Estimators. The paper names nflows flow estimators §1.1. The code lists nflows and joblib as dependencies pyproject.toml, pyproject.toml. It also contains flow-matching estimators that the paper does not mention flowmatching_estimator.py.

Pitfalls

• No numbers in the paper. Nothing here gives sizes, learning rates, epochs or performance figures. An implementer who needs those must use the code or the method papers the paper cites §2, and must not borrow values from elsewhere.
• Version drift. The code is v0.27.0 and the paper is from 2020. Default values, method names and deprecations have changed. Pin the commit before you copy any default.
• Numpy inputs when parallel. With num_workers other than 1, the simulator receives numpy arrays. A TypeError from a simulator that expects tensors is re-raised with a hint to wrap it with process_simulator simulation_utils.py.
• Invalid-x defaults. The exclude_invalid_x default differs between method signatures base.py, base.py. If you do not pass it, check which method you call.
• Empty valid set. If every simulation is invalid, the code raises an error rather than continuing sbiutils.py.
• Amortized versus per-observation. The simple interface returns an amortized posterior that works for any x after one training run base.py. The paper does not explain what this means for accuracy.

Lineage

• Builds on: delfi, which sbi succeeds, and a fork of lfi that the codebase started from §1.1. Neural SBI methods cited in the paper include SNPE-C, SNLE and SNRE §2.
• Compared against: other packages named in the paper: elfi, carl, pyabc and ABCpy, plus hypothesis and pydelfi, which were under development at the time §1.1.
• Superseded by: not stated in this card.