Hamilton-Jacobi reachability

Overview

Hamilton-Jacobi (HJ) reachability is a robust optimal control formulation. It assigns each state a value that summarises whether the system can avoid a failure set when the model error is treated as an adversarial disturbance 2412.09989 §IV-A. The paper under study uses it for a quadruped with a simple planar model: a 3-state Dubins car whose modelling error enters as a disturbance bounded in position and heading 2412.09989 §V-A.1.

The invariant that the paper's filter relies on: where the value is above a calibrated level, the nominal command can pass through unchanged; below that level, the command is changed as little as possible so the gradient-based constraint holds 2412.09989 §V-B.2. The failure set itself depends on the environment (obstacles, friction, payload), which is why the paper conditions the value on a LiDAR scan 2412.09989 §III 2412.09989 §V.

Variants

Only one paper in this wiki covers the concept so far. Other variants (for example, grid-based solvers used directly at run time) are not covered here.

Learned, observation-conditioned value network with a QP filter

The value is approximated by a network that takes the relative state, the disturbance bound and a 100-ray LiDAR scan. It is an MLP with sine activations, chosen for good spatial gradients 2412.09989 §V-A.3. Training is supervised regression on both value and spatial gradient, with ground truth from a reachability solver over 1,000 random environments and a 2 s horizon 2412.09989 §V-A.2.

The learned output is calibrated with a split-conformal argument. The calibration level δ is chosen so that, with probability at least 1 − β, the fraction of samples where the network overestimates the value by more than δ stays at or below ε 2412.09989 §V-A.4 2412.09989 app. A.

At run time, the disturbance bound is estimated from a sliding window of prediction errors divided by the horizon 2412.09989 §V-B.1. The filter is a least-restrictive QP over the nominal twist (forward speed and yaw rate), with a worst-case margin from the disturbance bound and a slack variable with weight 10^3 2412.09989 §V-B.2. Card: One Filter to Deploy Them All.

The paper describes the filter as wrapping any planner that outputs twists and any low-level tracker. For an end-to-end policy, it runs the policy while the value is above δ and falls back to a planner and tracker otherwise 2412.09989 §V-B.2.

When to use which

All results come from the authors' own experiments, with 100 simulation trials per row and 10 hardware trials per row 2412.09989 §VI-D 2412.09989 §VII. No second paper in the wiki reproduces them.

Implementation sketch

Python-flavoured sketch of one control step. The value and gradient come from the learned network; δ is the calibrated level.

# Value and spatial gradient from the learned network  [arxiv:2412.09989 §V-A.3]
# Input:  relative state x, LiDAR scan, disturbance bound d
# Output: value V and gradient dV

def filter_step(x, scan, d, u_nom, delta):
    # Pass the nominal twist through above the calibrated level
    # [arxiv:2412.09989 eq. 15]
    V, dV = value_net_with_gradient(x, d, scan)
    if V > delta:
        return u_nom
    # Otherwise the closest twist to the nominal one that keeps the
    # worst-case margin, with a slack weighted 10^3  [arxiv:2412.09989 eq. 16]
    return qp_closest_safe_twist(u_nom, dV, d, slack_weight=1e3)

Check the paper's own equations before porting. The value is compared with δ in raw network units there, and the released code differs (see the contradictions below).

Contradictions

• Switching threshold. The paper's equations switch to the QP when the uncalibrated output is at most δ = 0.49 m 2412.09989 §V-B.2. The released hardware config adds a calibration offset and switches at 0.35 on the calibrated value, which the card converts to about 0.85 in raw units. The card's Paper vs code section records this. The two sources give different behaviour, so an implementation should follow one and say so.
• Disturbance window. The paper uses a 2 s window 2412.09989 §V-B.1. The released config uses 10 s. The paper names slow reaction to a change as a failure mode 2412.09989 §VIII.
• Bound formula. The paper's bound is |mean| plus or minus b standard deviations on a window of position disturbance norms 2412.09989 §V-B.1. The released code drops the absolute value and works per dimension.
• Calibration grid. The paper tabulates ε = 10^-3 with δ = 0.99 m 2412.09989 tab. II. The released calibration grid covers only ε from 10^-2 to 10^-1. The paper's table is the only source for ε = 10^-3.
• Network depth and loss scale. The paper says four hidden layers of 512 units and a loss summed over gradient components 2412.09989 §V-A.3. The card records that the released code uses a different layer count and an element-averaged loss.

Papers

• Quadruped safety filter driven by a learned reachability value (arXiv:2412.09989): learned HJ value network conditioned on LiDAR, with a calibrated level and a QP safety filter for a Unitree Go1.