---
kind: concept
slug: lyapunov-function
title: Lyapunov function
summary: >-
  A Lyapunov function is zero at the target, positive elsewhere on a region, and must decrease along closed-loop trajectories. Neural Lyapunov functions can be trained with a neural controller, and a mixed-integer solver checks the conditions on a box. One source so far, with author-reported results on a pendulum and two quadrotors, and no full region of attraction.
tags: [control, stability, verification, robotics]
papers: ["2109.14152"]
related: [branch-and-bound, control-barrier-function, hamilton-jacobi-reachability, model-predictive-control]
status: draft
updated_at: 2026-11-12
---

## Overview

A Lyapunov function V is a scalar function of the state that measures distance
from a target x*. The invariant the variants share is: V(x*) = 0, V(x) > 0 on
the region of interest for x ≠ x*, and V must decrease along the closed loop
[arxiv:2109.14152 §II]. Together these three conditions give the stability
certificate. The paper writes the decrease condition as
V(x[t+1]) − V(x[t]) ≤ −ε2·V(x[t]) with ε2 > 0 given [arxiv:2109.14152 §II].

The problem the paper addresses is that a neural-network controller usually
comes with no such certificate, while linear (LQR) and polynomial
(sum-of-squares) controllers do [arxiv:2109.14152 §I]. The conditions must hold
for every state in a region, not only on sampled states [arxiv:2109.14152 §I].

## Variants

The concept currently has one card. Its variant is a neural-network controller
and a neural-network Lyapunov function, trained together and checked by a
mixed-integer program. Card: [Lyapunov-stable neural-network control](/papers/2109.14152).

### Neural Lyapunov function with an exact MIP verifier

The dynamics are given as a leaky-ReLU network fitted to data [arxiv:2109.14152
§II] [arxiv:2109.14152 §V-C]. The Lyapunov function is a network term minus its
value at x*, plus a weighted 1-norm term |R(x − x*)|₁ [arxiv:2109.14152 eq. 5].
The 1-norm term is there because a network-only V is linear near x* and can go
negative [arxiv:2109.14152 §IV-A]. R has full column rank, so the strict
positivity condition is replaced by V(x) ≥ ε1·|R(x − x*)|₁ with 0 < ε1 < 1
[arxiv:2109.14152 §IV-A]. The paper proves this non-strict form is also
necessary [arxiv:2109.14152 app. C].

The verifier maximises the two violations over a bounded box B. If both maxima
are zero, the conditions are certified on B. Otherwise the worst state is a
counterexample [arxiv:2109.14152 §IV-A]. The network is encoded with big-M
constraints and one binary per unit [arxiv:2109.14152 §III].

Two trainers use the counterexamples. Approach 1 adds them to growing sample
sets and runs gradient descent on a hinge-style loss [arxiv:2109.14152 §IV-B]
[arxiv:2109.14152 alg. 1]. Approach 2 minimises the sum of the two MIP optima
over the parameters, and differentiates through the MIP with binaries and
active constraints fixed [arxiv:2109.14152 §IV-B] [arxiv:2109.14152 alg. 2].

### Region of attraction from the sublevel set

The verified box B is not itself a region of attraction, because it is not
invariant. The paper takes the largest sublevel set S = {x : V(x) ≤ ρ} inside
B, with ρ the minimum of V on the boundary of B [arxiv:2109.14152 §IV-C]
[arxiv:2109.14152 eq. 15]. The paper says this set can be small, and names a
joint search over V and ρ as future work [arxiv:2109.14152 §IV-C]
[arxiv:2109.14152 §VI].

## When to use which

| Situation | Use | Why |
|---|---|---|
| Linear plant, need a guarantee from a standard design | LQR | The paper's introduction names LQR as giving guarantees for linear controllers [arxiv:2109.14152 §I]. It is also the baseline the paper compares against [arxiv:2109.14152 §V-B]. |
| Nonlinear controller that must be a neural network, certificate needed on a bounded region, small system | Neural Lyapunov function with MIP verifier | Author-reported runs on an inverted pendulum and 2D quadrotor [arxiv:2109.14152 §V-A] [arxiv:2109.14152 §V-B]. |
| Need the guarantee on the true system | Re-verify on the true dynamics | The proof covers the learned model. The 3D model fit error is about 1e-4 MSE, and V can increase along true trajectories [arxiv:2109.14152 §V-C] [arxiv:2109.14152 app. D]. |
| Need a large or whole-state-space basin | Not supported by this paper | The usable region is the sublevel set, which can be small [arxiv:2109.14152 §IV-C] [arxiv:2109.14152 §VI]. |
| 3D or other large networks with many ReLU units | Budget for long solves, or shrink the box | Average 3D runs took 65.7 h with approach 2, and approach 1 timed out after 5 days [arxiv:2109.14152 tab. II]. The authors kept the verified region small to speed this up [arxiv:2109.14152 tab. II]. |
| Small system, trainer choice | Either approach; no clear winner | Pendulum: 8.4 s (approach 1) vs 224 s (approach 2). 2D: 948.3 s vs 1004.7 s. The authors call the comparison not conclusive [arxiv:2109.14152 tab. II] [arxiv:2109.14152 §V-C]. |

## Implementation sketch

Common core of the verifier-in-the-loop check, from the paper's equations, not
its code. Python-flavoured. The maxima are solved as MIPs over the box.

```pseudocode
# Target x*, eps2 > 0, 0 < eps1 < 1, box B  [arxiv:2109.14152 §II]
# Positivity gap: V below the 1-norm floor  [arxiv:2109.14152 §IV-A]
def positivity_gap(V, R, x_star, eps1):
    return max(eps1 * norm_1(R @ (x - x_star)) - V(x) for x in B)

# Decrease gap: V fails to shrink by eps2 in one step  [arxiv:2109.14152 §II]
def decrease_gap(V, step, eps2):
    return max(V(step(x)) - V(x) + eps2 * V(x) for x in B)

# Certified only if both maxima are zero  [arxiv:2109.14152 §IV-A]
# Otherwise the maximiser is a counterexample to feed back to training  [alg. 1]
def check(V, step, R, x_star, eps1, eps2):
    gp = positivity_gap(V, R, x_star, eps1)
    gd = decrease_gap(V, step, eps2)
    return gp <= 0 and gd <= 0, gp, gd
```

Controller and Lyapunov shapes from the paper: the controller is shifted so
u(x*) = u*, then clamped to the input limits [arxiv:2109.14152 eq. 7]
[arxiv:2109.14152 eq. 8]. The Lyapunov function subtracts its value at x*
[arxiv:2109.14152 eq. 5]. Because the clamp is piecewise affine, the closed
loop can be encoded as a MIP [arxiv:2109.14152 app. B].

## Contradictions

- **Paper text vs its code (one source).** The paper's Table III gives network
  sizes that differ from the demo scripts. For the pendulum, the paper's
  controller is (2, 2) and its Lyapunov network is (8, 4, 4), while the code
  uses (3, 2) and (8, 8, 6) [arxiv:2109.14152 tab. III]. The paper's dynamics
  update is written for the full state [arxiv:2109.14152 eq. 1a], but the code
  uses a second-order form for the pendulum (see the card's Paper vs code).
- **Region of attraction.** The paper defines ρ as the minimum of V on the
  boundary of B [arxiv:2109.14152 eq. 15]. The code's routine for this is not
  the same, and the demos do not call it [arxiv:2109.14152 §IV-C]. Do not
  assume the demos report the paper's region.

None else known.

## Papers

- [Lyapunov-stable neural-network control](/papers/2109.14152): trains a neural
  controller and neural Lyapunov function together, and checks the Lyapunov
  conditions with a mixed-integer program on a box.
