---
kind: concept
slug: multi-robot-navigation
title: Multi-robot navigation
summary: >-
  Several robots share one space and must reach goals without collisions or standoffs. Two variants: a reinforcement-learning policy for drones that explore an unknown room and carry a payload (simulation only, under-specified), and a neural controller that imitates an MPC and adds a control-barrier-function QP so two robots pass doorways without deadlock (simulation only, code released).
tags: [multi-robot, navigation, deadlock-avoidance, control-barrier-function, reinforcement-learning]
papers: ["2409.03930", "2412.04659"]
related: [control-barrier-function, differentiable-optimization, imitation-learning, proximal-policy-optimization, partially-observable-markov-decision-process, model-predictive-control, sim-to-real-transfer]
status: draft
updated_at: 2026-10-25
---

## Overview

Robots that share a narrow space can block each other. If both wait, neither
moves (a deadlock). If both push on, they can collide. The goal is to keep
every robot moving toward its own target with no collisions and no deadlock
[arxiv:2412.04659 §1]. The two papers on this page attack different parts of
that problem: one learns a whole exploration-and-transport behaviour for
drones, the other learns a local, decentralised controller for pairs of
robots that meet.

What the variants share is that each agent acts on its own observations at
run time. The LiveNet controller is fully decentralised and robots exchange no
messages [arxiv:2412.04659 §1] [§5.1]. The DRAL actor sees only partial
observations at deployment, while its critic used full state in training
[arxiv:2409.03930 §3.2].

## Variants

### Reinforcement learning with an asymmetric actor-critic (DRAL)

Three cable-carrying hex-rotor drones search an unknown indoor room with one
camera and carry an unknown payload to a target. The policy is trained with
PPO and Generalized Advantage Estimation. The critic gets privileged simulator
state during training, and the actor gets only observations at deployment
[arxiv:2409.03930 §3.2]. The sensor is modelled as a 50 × 50 degree field of
view with a 3 m range [arxiv:2409.03930 §4].

The reported success rates for three payload classes are 0.824, 0.857 and
0.903. The three baselines, tabular Q-learning, deep Q-learning and Sarsa,
reach 0.502 to 0.786 [arxiv:2409.03930 tab. 1]. These numbers come from a
Gazebo simulation with three runs per method [arxiv:2409.03930 §4]. Real-world
flight is claimed, but no real-world result is reported [arxiv:2409.03930 §1]
[§5]. The reward, observation and action spaces, network, and training budget
are not stated [arxiv:2409.03930 §3.2], so the method cannot be rebuilt from
this paper alone. Card: [DRAL](/papers/2409.03930).

### Imitated MPC with a CBF-QP safety layer (LiveNet)

Two robots meet in a doorway or at an intersection. A small network outputs a
reference control and two sets of penalties. A QP layer then enforces one
collision barrier row per other robot and one liveness barrier row, which
constrains only speed, so the path stays fixed [arxiv:2412.04659 §4]
[§4.2]. The network is trained by supervised learning on the controls of a
receding-horizon MPC that uses the same barrier functions [arxiv:2412.04659
§5.1]. The paper states a Theorem 1 for liveness under stated conditions
[arxiv:2412.04659 §4.2].

In simulation, the authors report no collisions and no deadlocks in 50 runs
per scenario, for the doorway and the intersection. Each step is reported
as 10 to 20 times cheaper than the optimisation baseline built for this
setting [arxiv:2412.04659 §1] [tab. 2]. In a set of 28 perturbed doorway
scenarios, LiveNet solved 25 without deadlock or collision, against 16 for
SMG-CBF [arxiv:2412.04659 §5.2]. The experiments use two agents only, and
the authors say the code is not hardened for deployment [arxiv:2412.04659
§6]. Card: [LiveNet](/papers/2412.04659).

## When to use which

| Situation | Use | Why |
|---|---|---|
| Robots must explore an unknown space and you can run many simulated episodes | RL with asymmetric actor-critic [arxiv:2409.03930 §3.2] | Only variant on this page that covers exploration. Privileged state is used in training only. You must still supply the reward, which the paper does not state. |
| Two robots meet in a narrow passage and must not deadlock | CBF-QP layer on a learned controller [arxiv:2412.04659 §4.2] | The liveness row acts on speed only, so the path changes little. In the doorway the author-reported speed change is lower than SMG-CBF's, but the path change is higher [arxiv:2412.04659 tab. 2]. |
| You have no MPC expert and cannot build the barrier functions | Neither variant as written | LiveNet's training labels come from an MPC that uses the barrier functions [arxiv:2412.04659 §5.1]. |
| Per-step compute is tight | CBF-QP layer on a learned controller | The authors report a cycle time several times lower than the optimisation baseline [arxiv:2412.04659 tab. 2]. The unit is unsettled (see Contradictions). |
| Robots need more than two agents | Neither, as shown | LiveNet is tested with two agents only [arxiv:2412.04659 §6]. DRAL's multi-drone setting has three drones with one payload [arxiv:2409.03930 §4]. |

## Implementation sketch

Each robot runs the same loop on its own observations. The learned policy
gives a nominal command. A safety layer, when a variant has one, adjusts that
command. The DRAL paper describes no safety layer [arxiv:2409.03930 §3.2].

```pseudocode
# Each robot runs this loop on its own observations; no messages are sent
# [arxiv:2412.04659 §1]
def control_step(policy, safety_qp, obs):
    # Nominal command and penalties from the learned network  [arxiv:2412.04659 §4]
    u_ref, penalties = policy(obs)
    # No safety layer is described for the RL variant  [arxiv:2409.03930 §3.2]
    if safety_qp is None:
        return u_ref
    # One collision row per other robot; a liveness row where paths cross
    # [arxiv:2412.04659 §4.1] [§4.2]
    G, h = barrier_rows(obs, penalties)
    # Smallest change to u_ref that satisfies the rows  [arxiv:2412.04659 §4]
    return safety_qp(u_ref, G, h)
```

Training differs by variant. DRAL uses PPO with a privileged-state critic
[arxiv:2409.03930 §3.2]. LiveNet uses supervised regression on MPC controls
[arxiv:2412.04659 §5.1].

## Contradictions

- **Cycle time unit.** The LiveNet table labels cycle time in seconds
  [arxiv:2412.04659 tab. 2]. The paper does not settle whether the values are
  seconds or milliseconds, so absolute compute times cannot be quoted. The
  ratio between methods is unaffected, because both sides share one unit.
- **Invasiveness claim.** The abstract's "4 to 5 times more invasive" holds
  for the doorway only. Speed change there is 0.009 for SMG-CBF against 0.002
  for LiveNet. In the intersection it is 0.012 against 0.011 [arxiv:2412.04659
  tab. 2].
- **Path change.** In the doorway, LiveNet's path deviation is 0.008 m
  against 0.001 m for SMG-CBF. LiveNet is higher on this metric
  [arxiv:2412.04659 tab. 2].
- **QP sign.** The paper writes the minimal-change layer as maximising
  (u − u_ref)² [arxiv:2412.04659 §4]. The LiveNet card records that the
  released code minimises with the opposite sign on u_ref; see its Paper vs
  code section. The two forms give different references, so implement one
  deliberately.
- **No shared baseline.** DRAL compares against Q-learning, deep Q-learning
  and Sarsa [arxiv:2409.03930 tab. 1]. LiveNet compares against MPC-CBF,
  MACBF, PIC, BarrierNet and SMG-CBF [arxiv:2412.04659 tab. 2]. No number in
  one paper can be read against a number in the other.

## Papers

- [DRAL (Mo et al., 2024)](/papers/2409.03930): PPO with an asymmetric actor-critic for three cable-carrying drones that search and carry an unknown payload; simulation results only.
- [LiveNet (Gouru et al., 2024)](/papers/2412.04659): imitated MPC with a QP layer that enforces collision and liveness barrier functions, for two robots at doorways and intersections; simulation results only.