Backup-Based Safety Filters: A Comparative Review of Backup CBF, Model Predictive Shielding, and gatekeeper

In plain words

A robot or car often has a controller that does its job well but cannot promise to stay safe. A safety filter sits between that controller and the motors. It lets the original command through while that is safe, and steps in when it is not. One common way to decide is to keep a known fallback manoeuvre, the backup, ready. The filter asks whether the robot could switch to the backup from where it is now and stay out of danger until it reaches a safe resting region.

Three known filters work this way: Backup CBF, Model Predictive Shielding and gatekeeper. This paper writes them in one shared notation and compares when each one hands control to the backup. Two of them check safety with the backup starting almost at once, which makes them give up the normal controller more often than needed. Gatekeeper searches for the latest switch time that is still safe, so it lets the normal controller run longer. The paper is a tutorial and review. It does not propose a new filter.

TL;DR

• Review and tutorial, not a new algorithm. It puts Backup CBF, Model Predictive Shielding (MPS) and gatekeeper under one safety-filter definition §I tab. 1.
• MPS is a special case of gatekeeper: every state MPS accepts is accepted by gatekeeper when Δt ≤ T_H §VI-A eq. 19. The relative interior of the Backup CBF inactive set lies in the gatekeeper inactive set §VI-B eq. 21.
• It names a shared weakness, "safety evaluation on backup": safety is judged through the backup maneuver, not through the nominal policy's continued safe run §V-A.
• Author-reported simulations: gatekeeper keeps the nominal controller on 87.5% of steps in reach-avoid, against 55.3% for MPS and 28.8% for Backup CBF tab. 2. In highway overtake it keeps it on 100%, against 67.5% for MPS and 60.8% for Backup CBF tab. 2.
• The public code (safe_control) differs from the paper in the search grid, the validity check, the MPS fallback and the event schedule. See "Paper vs code".

Problem

A backup-based filter needs a backup policy that keeps a terminal safe set S_0 invariant and recovers a neighbourhood of it within the backup horizon §II-A. The paper gives two reasons to compare these filters. First, the three methods are written in different notation, which hides what they share and where they differ §I. Second, Backup CBF and MPS judge safety through the backup maneuver. This can make them give up the nominal policy even when it would stay safe §V-A. MPS also tests only one switch time, one step ahead §IV-B. The paper calls the resulting conservatism a key limitation of backup-based filters §V-A §VI-C.

Method

Common abstraction. The system is control-affine, ẋ = f(x) + g(x)u eq. 1. The nominal policy π_nom is task-oriented but not necessarily safe §II-A. The safe set C is the superlevel set of h^C, and S_0 ⊆ C is the superlevel set of h^S eq. 2 eq. 3. A backup policy π_b must keep S_0 forward invariant and must have a recoverable set over horizon T_B that contains a neighbourhood of S_0 §II-A. The recoverable set of a policy π is the set of states that stay in C on [0, T] and end in S_0 at time T eq. 4. A safety filter π_sf keeps the state in C for all t ≥ 0 from every initial state in a set D ⊆ C §II-B. Its filter-inactive set is the set of states where it returns π_nom unchanged eq. 5.

Backup CBF (§III). The implicit safe set is the recoverable set of the backup policy over [0, T_B] eq. 6. The paper cites a theorem from the literature: under forward invariance of S_0, this set is controlled invariant and lies between S_0 and C §III-A. The backup is not applied directly. Instead, the filter solves a quadratic program that changes the nominal input as little as possible while keeping the barrier constraints on the backup trajectory eq. 10a. The constraints apply along the backup path for all τ in [t, t+T_B] eq. 10b, and at the terminal set eq. 10c. The continuum of path constraints is approximated by finitely many collocation points §III-B. The filter-inactive set is the set where the nominal input already satisfies all constraints §III-B.

Candidate trajectory and validity (§IV-A). For a switch time T_S ≥ 0, the candidate trajectory follows π_nom for T_S and then π_b for T_B §IV-A. The validity indicator is 1 when the candidate stays in C over [0, T_S + T_B] and is in S_0 at time T_S + T_B, and 0 otherwise §IV-A. The valid set V(T_S; T_B) collects the states with indicator 1 eq. 11. The check is local to the candidate. It does not require the backup to recover every state of C §IV-A.

MPS (§IV). The monitor runs at a fixed interval Δt. MPS tests only T_S = Δt eq. 12. It applies π_nom when the candidate is valid and π_b(x_k) otherwise eq. 13. Its filter-inactive set is V(Δt) eq. 14. The paper says this matches the original recovery check, written in its notation §IV-B. Algorithm 1 gives the steps alg. 1.

Gatekeeper (§V). Gatekeeper uses the same candidate and validity check, but treats T_S as a decision variable. It takes the largest valid T_S in [0, T_H] eq. 15. It applies π_nom when that value is positive and π_b when it is zero eq. 16. Its filter-inactive set is the union of V(T_S; T_B) over positive T_S ≤ T_H eq. 17. In practice the search runs on the grid {0, Δt, 2Δt, ..., N_H Δt} with T_H = N_H Δt eq. 18 §V-C. The search goes from the largest candidate downward and stops at the first valid one alg. 2. The paper notes that the candidates are independent, so they can be evaluated in parallel without changing the result §V-C. Between checks, a positive certificate is counted down by Δt rather than searched again alg. 2 §V-C.

Comparison (tab. 1). The paper's table summarises the three filters:

tab. 1

Theory (§VI). Theorem 1: if 0 < Δt ≤ T_H, then I_MPS ⊆ I_GK eq. 19. The proof uses that T_S = Δt is one of the switch times gatekeeper searches §VI-A. Theorem 2: the relative interior of I_BCBF inside the implicit safe set S is contained in I_GK eq. 21 §VI-B. The paper does not claim the full inclusion I_BCBF ⊆ I_GK, because boundary points may have no positive nominal hold time §VI-B. The paper also lists two limits: gatekeeper behaves close to MPS when T_H is short or the grid is coarse, and all three methods inherit conservatism from the backup policy and the model §VI-C.

Pseudocode of the two decision rules. The paper's rules, in our notation. The official code differs; see "Paper vs code".

# MPS decision at each monitor update  [alg. 1] [eq. 12] [eq. 13]
# Input:  state (x); nominal policy (pi_nom); backup policy (pi_b)
#         monitor interval (dt); backup horizon (T_B)
# Output: control input

def mps_input(state, pi_nom, pi_b, dt, T_B):
    # Candidate: one nominal step, then backup for T_B  [§IV-A]
    candidate = rollout_then_backup(state, dt, pi_nom, pi_b, T_B)
    # Valid candidate: keep the nominal input  [eq. 13]
    if is_valid(candidate, T_B):
        return pi_nom(state)
    # Invalid candidate: use the backup at the current state  [eq. 13]
    return pi_b(state)

# Gatekeeper decision at an update time  [alg. 2] [eq. 15] [eq. 16]
# Input:  state (x); nominal policy (pi_nom); backup policy (pi_b)
#         search horizon (T_H); grid step (dt); backup horizon (T_B)
# Output: control input

def gatekeeper_input(state, pi_nom, pi_b, T_H, dt, T_B):
    # Try switch times from the longest to the shortest  [eq. 18]
    for switch_time in grid_descending(T_H, dt):
        # Candidate: nominal until switch_time, then backup  [§IV-A]
        candidate = rollout_then_backup(
            state, switch_time, pi_nom, pi_b, T_B
        )
        # First valid candidate gives T_S*  [eq. 15]
        if is_valid(candidate, T_B):
            # Positive T_S*: nominal input  [eq. 16]
            if switch_time > 0:
                return pi_nom(state)
            # Zero T_S*: backup input  [eq. 16]
            return pi_b(state)
    # No valid switch time: T_S* stays 0, backup input  [alg. 2]
    return pi_b(state)

Key equations. The three that a reader needs most.

\dot{\mathbf{x}} = f(\mathbf{x}) + g(\mathbf{x})\mathbf{u} eq. 1

T_S^\star(\mathbf{x}_k) = \sup\{\, T_S \in [0, T_H] \mid \mathrm{Valid}(\mathbf{x}_k; T_S) = 1 \,\} eq. 15

\mathcal{I}_{\mathrm{MPS}} \subseteq \mathcal{I}_{\mathrm{GK}} \quad \text{for } 0 < \Delta t \le T_H eq. 19

Hyperparameters

Values are the paper's unless marked "code". Code values come from the public repository at code_commit. A code value that differs from the paper is also listed in "Paper vs code".

Results

Author-reported. Simulation only, no hardware. Setup: planar double integrator for the set picture §VII-A fig. 1. Reach-avoid: a robot in a narrow hallway with a side pocket and a moving obstacle §VII-B. Highway: a dynamic bicycle model with Fiala tire forces, five lanes, and a moving obstacle in the adjacent lane §VII-C. The baselines are Backup CBF and MPS. The paper reports no nominal-only run. Table II averages over five trials tab. 2.

The text says Backup CBF and MPS repeatedly retreat to the pocket and fail to reach the goal, and gatekeeper reaches it §VII-B fig. 3. In the highway case, the text says Backup CBF and MPS trigger a lane change although the nominal lane is clear §VII-C fig. 4. The parallel gatekeeper gives the same closed-loop behaviour with lower average compute time §VII-B tab. 2. Compute hardware is not stated.

Paper vs code

Repository checked: https://github.com/tkkim-robot/safe_control at code_commit. The paper's abstract links it, and its owner is one of the paper's authors. The repository has no license file, so no code is quoted verbatim here. The filter code is in shielding/. The examples are in examples/.

1. Search grid step. The paper searches the grid with step Δt eq. 18 §V-C. The code's default step is horizon_discount = 5 * dt gatekeeper.py, and neither example overrides it test_evade.py test_drift.py. The candidate set is therefore 5Δt apart: 0.5 s in the reach-avoid example and 0.25 s in the highway example. The code's candidates generally do not include T_S = Δt, so the inclusion in eq. 19 is not guaranteed for the code as run.
2. Search order is called a binary search. The comment says "Binary search" gatekeeper.py, but the loop steps down linearly from the longest candidate gatekeeper.py. The paper describes a sequential downward search §V-C. The comment does not match the code.
3. Validity check. The paper requires the candidate to stay in C over the whole interval and to end in S_0 §IV-A. The code's _is_candidate_valid checks collision only, at the states of the candidate gatekeeper.py. Collisions cover the environment boundary, static obstacles and moving obstacles gatekeeper.py. No check of membership in S_0 was found in shielding/. The terminal behaviour then depends on the backup and the environment, which the paper does not specify.
4. Safety margin. The paper has no margin. The code adds safety_margin to the robot radius only for moving obstacles gatekeeper.py gatekeeper.py. Boundary and static checks use the bare radius gatekeeper.py. The class default is 1.0 gatekeeper.py, and the examples set 0.5 and 0.01 test_evade.py test_drift.py.
5. MPS fallback. The paper's rule gives π_b(x_k) when the candidate is invalid eq. 13. The code keeps the previously committed trajectory and schedules the next check mps.py. It then outputs the next stored control mps.py. That control comes from a backup rollout computed one step earlier, so it is not π_b evaluated at the current state.
6. Event schedule. The paper's Algorithm 2 counts T_S* down by Δt between checks and searches again only when the certificate must be refreshed alg. 2 §V-C. The code re-checks when the event index reaches event_offset / dt gatekeeper.py. After a successful commit, the index is reset and incremented, so the next call searches again. In both examples the threshold is at most one step, so a successful commit is followed by a new search on the next step. The examples use event_offset = 0.05 s test_evade.py test_drift.py.
7. Search cost. The loop runs at most max_nominal_steps // discount_steps + 2 candidates gatekeeper.py. Each candidate includes a full backup rollout gatekeeper.py. This is the cost behind the parallel variant's speed-up §V-C.
8. Backup CBF gains. The paper leaves the class-K function unspecified §III-B. The code uses α(h) = 1.0 · h and an α_terminal of 2.0 · h backup_cbf_qp.py backup_cbf_qp.py. The backup horizon is discretised into int(T_B / dt) steps backup_cbf_qp.py.
9. Not reviewed. The parallel gatekeeper is in the separate safe_control_jax repository, which the README mentions README.md. It was not in the checkout, so its timing numbers in tab. 2 cannot be checked here.
10. No discrepancy found for the highway vehicle model. The paper's 8-state bicycle with Fiala tyre forces §VII-C matches the drifting-car model's eight states gatekeeper.py and the Fiala tyre model in robots/drifting_car.py.

Pitfalls

• Average T_S values are not comparable across scenarios.* Table II gives 6.00 s for gatekeeper in both scenarios. The reach-avoid horizon is T_H = 10 s §VII-B, and the paper does not say how the average is taken over steps. Check the averaging rule before using these numbers.
• The trial design is described two ways. Table II says "randomly sampled initial robot conditions" tab. 2. The highway text says "the same obstacle configuration" §VII-C. Five trials are reported without variance.
• The theorems have limits. Theorem 1 needs Δt ≤ T_H §VI-A. Theorem 2 gives only the relative interior, not the boundary §VI-B. All the results assume S_0 is forward invariant under the backup §III-A.
• Safety depends on the backup and the terminal set, which the paper leaves open. Validity checks the nominal-then-backup candidate only §IV-A. The paper gives no explicit S_0, h^C or h^S for its experiments. An implementer must design them.
• Gatekeeper behaves like MPS when the horizon is short or the grid is coarse §VI-C. The code's default 5Δt grid is coarser than the paper's (see "Paper vs code" item 1).
• Hidden cost. Gatekeeper rolls out a backup for every candidate and can repeat this every control step (see "Paper vs code" item 7). The paper's timing table reports this only as average milliseconds on unspecified hardware tab. 2.
• The conservatism claim is supported by simulations only. The review shows the effect on two scenarios and one double-integrator example fig. 1 fig. 3 fig. 4. It does not measure the effect across systems.

Lineage

• Builds on: Backup CBF [chen_backup_2021, "Backup Control Barrier Functions: Formulation and Comparative Study"]; Model Predictive Shielding [bastani_safe_2021, "Safe Reinforcement Learning with Nonlinear Dynamics via Model Predictive Shielding"]; gatekeeper [agrawal_gatekeeper_2023, agrawal_gatekeeper_2024]. These papers have no cards on this wiki yet, so they are not linked.
• Compared against: all three methods, in this paper.
• Superseded by: not stated.