Multi-robot navigation

Overview

Robots that share a narrow space can block each other. If both wait, neither moves (a deadlock). If both push on, they can collide. The goal is to keep every robot moving toward its own target with no collisions and no deadlock 2412.04659 §1. The two papers on this page attack different parts of that problem: one learns a whole exploration-and-transport behaviour for drones, the other learns a local, decentralised controller for pairs of robots that meet.

What the variants share is that each agent acts on its own observations at run time. The LiveNet controller is fully decentralised and robots exchange no messages 2412.04659 §1 §5.1. The DRAL actor sees only partial observations at deployment, while its critic used full state in training 2409.03930 §3.2.

Variants

Reinforcement learning with an asymmetric actor-critic (DRAL)

Three cable-carrying hex-rotor drones search an unknown indoor room with one camera and carry an unknown payload to a target. The policy is trained with PPO and Generalized Advantage Estimation. The critic gets privileged simulator state during training, and the actor gets only observations at deployment 2409.03930 §3.2. The sensor is modelled as a 50 × 50 degree field of view with a 3 m range 2409.03930 §4.

The reported success rates for three payload classes are 0.824, 0.857 and 0.903. The three baselines, tabular Q-learning, deep Q-learning and Sarsa, reach 0.502 to 0.786 2409.03930 tab. 1. These numbers come from a Gazebo simulation with three runs per method 2409.03930 §4. Real-world flight is claimed, but no real-world result is reported 2409.03930 §1 §5. The reward, observation and action spaces, network, and training budget are not stated 2409.03930 §3.2, so the method cannot be rebuilt from this paper alone. Card: DRAL.

Imitated MPC with a CBF-QP safety layer (LiveNet)

Two robots meet in a doorway or at an intersection. A small network outputs a reference control and two sets of penalties. A QP layer then enforces one collision barrier row per other robot and one liveness barrier row, which constrains only speed, so the path stays fixed 2412.04659 §4 §4.2. The network is trained by supervised learning on the controls of a receding-horizon MPC that uses the same barrier functions [arxiv:2412.04659 §5.1]. The paper states a Theorem 1 for liveness under stated conditions 2412.04659 §4.2.

In simulation, the authors report no collisions and no deadlocks in 50 runs per scenario, for the doorway and the intersection. Each step is reported as 10 to 20 times cheaper than the optimisation baseline built for this setting 2412.04659 §1 tab. 2. In a set of 28 perturbed doorway scenarios, LiveNet solved 25 without deadlock or collision, against 16 for SMG-CBF 2412.04659 §5.2. The experiments use two agents only, and the authors say the code is not hardened for deployment [arxiv:2412.04659 §6]. Card: LiveNet.

When to use which

Implementation sketch

Each robot runs the same loop on its own observations. The learned policy gives a nominal command. A safety layer, when a variant has one, adjusts that command. The DRAL paper describes no safety layer 2409.03930 §3.2.

# Each robot runs this loop on its own observations; no messages are sent
# [arxiv:2412.04659 §1]
def control_step(policy, safety_qp, obs):
    # Nominal command and penalties from the learned network  [arxiv:2412.04659 §4]
    u_ref, penalties = policy(obs)
    # No safety layer is described for the RL variant  [arxiv:2409.03930 §3.2]
    if safety_qp is None:
        return u_ref
    # One collision row per other robot; a liveness row where paths cross
    # [arxiv:2412.04659 §4.1] [§4.2]
    G, h = barrier_rows(obs, penalties)
    # Smallest change to u_ref that satisfies the rows  [arxiv:2412.04659 §4]
    return safety_qp(u_ref, G, h)

Training differs by variant. DRAL uses PPO with a privileged-state critic 2409.03930 §3.2. LiveNet uses supervised regression on MPC controls 2412.04659 §5.1.

Contradictions

• Cycle time unit. The LiveNet table labels cycle time in seconds 2412.04659 tab. 2. The paper does not settle whether the values are seconds or milliseconds, so absolute compute times cannot be quoted. The ratio between methods is unaffected, because both sides share one unit.
• Invasiveness claim. The abstract's "4 to 5 times more invasive" holds for the doorway only. Speed change there is 0.009 for SMG-CBF against 0.002 for LiveNet. In the intersection it is 0.012 against 0.011 [arxiv:2412.04659 tab. 2].
• Path change. In the doorway, LiveNet's path deviation is 0.008 m against 0.001 m for SMG-CBF. LiveNet is higher on this metric 2412.04659 tab. 2.
• QP sign. The paper writes the minimal-change layer as maximising (u − u_ref)² 2412.04659 §4. The LiveNet card records that the released code minimises with the opposite sign on u_ref; see its Paper vs code section. The two forms give different references, so implement one deliberately.
• No shared baseline. DRAL compares against Q-learning, deep Q-learning and Sarsa 2409.03930 tab. 1. LiveNet compares against MPC-CBF, MACBF, PIC, BarrierNet and SMG-CBF 2412.04659 tab. 2. No number in one paper can be read against a number in the other.

Papers

• DRAL (Mo et al., 2024): PPO with an asymmetric actor-critic for three cable-carrying drones that search and carry an unknown payload; simulation results only.
• LiveNet (Gouru et al., 2024): imitated MPC with a QP layer that enforces collision and liveness barrier functions, for two robots at doorways and intersections; simulation results only.