---
kind: concept
slug: differentiable-optimization
title: Differentiable optimization
summary: >-
  Passes gradients through a solver so a model can be trained on the regret of its decision or on a target control. Solutions of LPs and ILPs are piecewise constant in the costs, so each method swaps the zero derivative for a smoothing, a perturbation or a surrogate loss. The trade-offs are which problem classes a method supports and how many solver calls each training step costs.
tags: [optimization, decision-focused-learning, combinatorial-optimization, training, control]
papers: ["2307.13565", "2412.04659"]
related: [surrogate-gradient, bipartite-matching, decision-focused-learning, control-barrier-function, imitation-learning]
status: draft
updated_at: 2026-10-25
---

## Overview

A solver maps a cost vector to a decision. If a model predicts the costs and
the decision is scored by its regret under the true costs, the training signal
has to pass through the solver [arxiv:2307.13565 §2.2]. For linear and integer
programs that map is piecewise constant. Its derivative is zero almost
everywhere and does not exist at the breakpoints, so plain autodiff gives no
useful gradient [arxiv:2307.13565 §2.3].

Each method supplies a substitute for the missing factor in the chain rule, the
derivative of the decision with respect to the predicted costs. The substitute
is what distinguishes the methods [arxiv:2307.13565 §2.3] [arxiv:2307.13565 §3.1].
In the survey's setting the constraints are known and the costs are predicted
from features by a model [arxiv:2307.13565 §2.1].

Regret is not the only training target. A solver can also sit inside a
controller and be trained to reproduce a reference output. LiveNet does this
with a QP whose constraints are control barrier functions, trained by regression
on the controls of an MPC [arxiv:2412.04659 §4] [arxiv:2412.04659 §5.1]. The
chain-rule problem is the same. The loss decides which substitutes apply: the
regret surrogates in the survey need the true costs, and a regression loss on a
control does not [arxiv:2307.13565 §3.1.4].

## Variants

### Differentiating a convex layer

OptNet, CvxpyLayers and Fold-opt return a layer that can sit anywhere in a
network. OptNet handles convex QPs by implicit differentiation of the KKT
conditions. CvxpyLayers differentiates the self-dual embedding of a convex cone
program. Fold-opt differentiates the fixed point of an unrolled solver, and it
also covers nonconvex problems [arxiv:2307.13565 §3.1.1] [arxiv:2307.13565 tab. 1].
Card: [Decision-focused learning survey and benchmark](/papers/2307.13565).

LiveNet uses a convex QP of this kind inside a robot controller. Each robot runs
a network that outputs a reference control and two sets of penalties. A QP with
an identity quadratic term, a linear term from the reference, and one row per
barrier inequality then adjusts the control. Supervised training backpropagates
through the QP [arxiv:2412.04659 §4] [arxiv:2412.04659 §5.1]. Card:
[LiveNet](/papers/2412.04659).

### Smoothing LPs and ILPs

These methods add a term that makes the LP relaxation differentiable. QPTL
adds a quadratic term and differentiates the resulting QP. The quadratic term
is dropped at test time [arxiv:2307.13565 §3.1.2]. HSD (Intopt) adds a
log-barrier and differentiates the self-dual embedding [arxiv:2307.13565 §3.1.2].
Mipaal turns an ILP into an LP with cutting planes and then applies QPTL
[arxiv:2307.13565 tab. 1]. All three work on the relaxation, not on the integer
problem itself [arxiv:2307.13565 §5.2.3]. Card: [Decision-focused learning survey and benchmark](/papers/2307.13565).

### Perturbation and finite differences

These methods handle linear objectives. Each one estimates the derivative from
solver outputs at perturbed costs.

- Negative identity treats the solver's Jacobian as −I [arxiv:2307.13565 §3.1.3].
- DBB takes a finite difference at a costs vector nudged against the loss gradient [arxiv:2307.13565 §3.1.3].
- I-MLE takes a finite difference with added noise [arxiv:2307.13565 §3.1.3].
- DPO differentiates the perturbed optimiser [arxiv:2307.13565 §3.1.3].
- Fenchel-Young (FY) uses a perturbed Fenchel-Young loss. It does not produce a layer [arxiv:2307.13565 tab. 1].

Card: [Decision-focused learning survey and benchmark](/papers/2307.13565).

### Surrogate losses without a backward solve

These methods replace the regret with a surrogate whose gradient needs no
solver call in the backward pass. None of them yields a layer [arxiv:2307.13565 tab. 1].

- SPO uses an SPO+ subgradient, which needs two solver calls per step [arxiv:2307.13565 §3.1.4].
- NCE, the learning-to-rank losses and LODL use surrogates over a set of feasible solutions or a learned convex local model [arxiv:2307.13565 §3.1.4].
- Listwise, Pairwise, Pairwise (diff) and MAP work on a cache of feasible solutions. The cache starts from the training optima and grows by solving on a fraction p_solve of steps, 5% in the benchmark [arxiv:2307.13565 §3.1.4] [arxiv:2307.13565 §5.2].

Gradient-free methods are surveyed but not benchmarked [arxiv:2307.13565 §3.2]
[arxiv:2307.13565 §5.2]. Card: [Decision-focused learning survey and benchmark](/papers/2307.13565).

## When to use which

The benchmark covers seven problem sets and eleven methods, with results shown
as box plots and only qualitative rankings in the text [arxiv:2307.13565 §5.2].
Treat the rankings as evidence about those problems, not a guide to yours.

| Situation | Use | Why |
|---|---|---|
| Convex problem and you need a differentiable layer | Convex-layer methods (OptNet, CvxpyLayers) | They differentiate the convex problem's optimality conditions and output a layer [arxiv:2307.13565 §3.1.1] [arxiv:2307.13565 tab. 1]. |
| Learned controller that must respect hard constraints at run time, trained on a reference control rather than regret | Differentiable QP layer over the constraints (LiveNet) | The QP enforces one barrier row per other robot and obstacle, and a liveness row when headings cross [arxiv:2412.04659 §4]. The authors report no collisions and no deadlocks in 50 simulated runs per scenario, for two robots [arxiv:2412.04659 tab. 2]. Each robot solves one QP per control step [arxiv:2412.04659 §4]. The results are simulation only [arxiv:2412.04659 §6]. |
| LP or ILP where the relaxation is close to the integer optimum | Smoothing (QPTL, HSD) | QPTL does well on matching and knapsack, where the relaxation is close, and poorly on energy scheduling, where it is not [arxiv:2307.13565 §5.2.3]. |
| Quadratic constraints | Check against the prediction-focused baseline first | HSD cannot take quadratic constraints. DBB, I-MLE, FY and QPTL do worse than the baseline on portfolio, while SPO is slightly better except at the lowest degree [arxiv:2307.13565 §5.2.1] [arxiv:2307.13565 §5.2.3]. |
| Linear objective and you can afford extra solver calls per step | Perturbation methods (DBB, I-MLE, FY) | Results vary by problem: FY, I-MLE and DBB each lead on some instances [arxiv:2307.13565 §5.2.1]. Training costs roughly 100 times the prediction-focused baseline per epoch on the energy benchmark [arxiv:2307.13565 §5.2.2]. |
| Tight training budget or a large problem | Solution-cache losses (MAP, Pairwise (diff)) | They train much faster than SPO, DBB, I-MLE and FY. The paper suggests MAP for large problems [arxiv:2307.13565 §5.2.2] [arxiv:2307.13565 §5.2.3]. |
| A robust default across problems | SPO | The text describes SPO as consistently robust, though not the best in every experiment [arxiv:2307.13565 §5.2.3]. |
| Only optimal decisions are observed, not the true costs | Not a regret-based setup | Regret and the surrogates in §3.1.4 need the true costs. The subset-selection experiment trained on decisions alone uses a different task loss [arxiv:2307.13565 §3.1.4] [arxiv:2307.13565 §5.1]. |

Watch for these when you implement a method:

- Plain autodiff through a solver gives zero gradients for LPs and ILPs. The surrogate you choose is the method you run [arxiv:2307.13565 §2.3].
- Ties at zero predicted cost make every feasible solution optimal. The benchmark assumes a fixed tie-breaking rule, so check your solver's behaviour before comparing numbers [arxiv:2307.13565 §5.2].
- Relative regret is undefined when the true optimum has zero objective. The portfolio benchmark switches to absolute regret for that reason [arxiv:2307.13565 §5.2.1].
- The grid spans several orders of magnitude for learning rate, damping and μ, and the best value changes by setting. Re-tune when problem size or solver changes [arxiv:2307.13565 tab. 3].
- Runtime figures include system overhead. Only the ratios between methods are meaningful [arxiv:2307.13565 §5.2.2].
- A QP layer can fail to solve, and the paper does not say what the network should output then. The authors' code returns a fallback instead, which the LiveNet card records as breaking its guarantee [arxiv:2412.04659 §4] [arxiv:2412.04659 §6].
- Match the sign of the QP's linear term to the weights you load. LiveNet's paper writes the QP as a maximisation, while its code minimises with the opposite sign on the reference. Shipped weights expect the code's sign [arxiv:2412.04659 §4]. See the LiveNet card's Paper vs code section.
- Derive the second Lie derivative of the collision barrier yourself. The paper and code use cos(θ_i + θ_j). A derivation from the drift term gives a minus sign, and the paper does not settle the question [arxiv:2412.04659 §4.1].

## Implementation sketch

The loop is the same for every method. Only the substitute gradient changes.
The SPO and DBB substitutes are short enough to state in full.

```pseudocode
# Shared training loop for decision-focused learning  [arxiv:2307.13565 alg. 2]
# data: (features z, true costs c, optimal decision x_star)
# method_gradient: one of the substitutes below

def train(model, data, lr, epochs):
    # Repeat over the training set  [arxiv:2307.13565 alg. 2]
    for epoch in range(epochs):
        for z, c, x_star in data:
            # Predict the costs from the features  [arxiv:2307.13565 §2.2]
            c_hat = model(z)
            # Solve with the predicted costs  [arxiv:2307.13565 §2.1]
            x_hat = solve(c_hat)
            # Replace dx_hat/dc_hat, which is zero almost everywhere  [§2.3]
            g = method_gradient(c_hat, c, x_hat, x_star)
            # Backpropagate through the predictor  [arxiv:2307.13565 alg. 2]
            model.backward(g)
            model.step(lr)
```

```pseudocode
# SPO subgradient with respect to the predicted costs  [arxiv:2307.13565 §3.1.4]
# solve(c) returns an optimal decision for costs c

def spo_gradient(c_hat, c):
    # Optimal decision under the true costs  [arxiv:2307.13565 §3.1.4]
    x_star = solve(c)
    # Decision under the perturbed costs 2 * c_hat - c  [arxiv:2307.13565 §3.1.4]
    x_perturbed = solve(2 * c_hat - c)
    # Subgradient of the SPO+ surrogate  [arxiv:2307.13565 §3.1.4]
    return x_star - x_perturbed
```

```pseudocode
# Blackbox differentiation (DBB) as written in the paper  [arxiv:2307.13565 §3.1.3]
# dL_dx: gradient of the task loss at x_hat; delta: step size

def dbb_gradient(c_hat, dL_dx, delta):
    # Decision with the costs nudged against the loss gradient  [§3.1.3]
    x_nudged = solve(c_hat + delta * dL_dx)
    # Decision with the unchanged predicted costs  [arxiv:2307.13565 §3.1.3]
    x_hat = solve(c_hat)
    # The paper has no division by delta; the authors' code divides by it
    return x_nudged - x_hat
```

## Contradictions

- **Best method on Warcraft.** The text names Listwise as best at sizes 12, 18 and 30, and also names SPO as best at size 30. Check the figure before citing a winner [arxiv:2307.13565 §5.2.1] [arxiv:2307.13565 fig. 7].
- **Paper and repository.** The authors' repository differs from the paper in several places. It divides the DBB gradient by the step size [arxiv:2307.13565 §3.1.3]. It sums the pairwise losses over one representative per distinct true cost, not over every cached rival [arxiv:2307.13565 §3.1.4]. It draws the cache-growth coin once per mini-batch, not once per instance [arxiv:2307.13565 §3.1.4]. The card for this paper lists these differences in detail: [Decision-focused learning survey and benchmark](/papers/2307.13565).
- **Paper and repository, LiveNet.** The LiveNet repository differs from its paper in fifteen listed places. Among them are the QP objective sign, the direction of the liveness critical points and the batch size [arxiv:2412.04659 §4] [arxiv:2412.04659 §5.1]. The card lists them: [LiveNet](/papers/2412.04659).

## Papers

- [Decision-focused learning survey and benchmark](/papers/2307.13565): survey of gradient-based and gradient-free methods for training a predictor on decision regret, with a benchmark of eleven methods on seven problem sets [arxiv:2307.13565 §5.2].
- [LiveNet: Robust, Minimally Invasive Multi-Robot Control for Safe and Live Navigation in Constrained Environments](/papers/2412.04659): a neural controller whose QP layer adds collision and liveness barrier rows, trained by imitating an MPC; simulation results for two robots [arxiv:2412.04659 §5.2].