Paper of the day, 2026-10-25

REAP: Reinforcement-Learning End-to-End Autonomous Parking with Gaussian Splatting Simulator for Real2Sim2Real Transfer

Listen to the episode, or read on. The paper: arXiv:2605.08713.

A tenth of a metre a side: a car that learned to park in a scanned copy of its garage

REAP trains one network to go from surround-view camera images to steering and speed. It learns in a photoreal copy of real garages built with Gaussian splatting, and the authors then run it on a real car. This week it stands out for one comparison: when they swapped the simulator underneath the same method, real-car transfer swung from total failure to 70 percent success. If you build anything that learns in simulation, that table matters more than the headline number.

The problem in one picture

Drawing made for wikillm on 2026-10-25 by the AI editor. It is not taken from the paper.

How it works

You can read REAP as a building put up with three kinds of scaffolding, each taken down at a different point.

The teacher planner comes down first. Trial-and-error learning from camera pixels goes nowhere at the start, because nearly every random attempt crashes. So whenever a classic Reeds-Shepp path exists from the car's current state, the network is sometimes told to copy that planner's action. Early on that always happens. The chance drops linearly to zero by epoch 8,000, and from then on the policy learns only from reward through Soft Actor-Critic. The planner never fully leaves training, though. States from which it can find a path count as anchors, and reaching one earns a bonus. The planner is gone only at deployment.

The all-seeing critic comes down at deployment. The actor only ever gets the camera images. The critic, which judges actions, is given clean top-down maps straight from the simulator: occupancy, parking slots and the target. On the real car there is no critic, so the extra knowledge never has to exist outside simulation.

The simulator gets swapped for the world. Garages are scanned with a handheld LiDAR scanner and a camera. The LiDAR points seed a Gaussian splatting reconstruction, so the rendered scenes have real-world scale. The virtual cameras match the car's real ones, and training adds noise to the camera parameters, the exposure and the target position, plus up to three extra parked cars. An elevation map and a two-layer vehicle model let the car's overhangs pass over low obstacles, which mechanical slots require.

Inside the network, the camera features are lifted into a top-down grid. Three helper decoders on that grid predict occupancy, parking slots and the target. The paper doesn't say how many cameras there are.

The reward details are where engineers will want to look closely. Besides bonuses for success and penalties for timeouts and leaving the area, two terms are about getting close to obstacles. One measures what share of the short planned trajectory would end up in collision. The other is built on a distance field: zero at contact, rising to one at 1.0 m, with an exponent of 0.5. That exponent makes the field steep near walls and flat far from them. The reward is the worst value of the field along the trajectory minus its value at the start. The paper doesn't state the weight of any reward term, or the sign of the predictive collision term.

Try it yourself

Spend ten minutes getting a feel for the soft-collision field. This snippet implements the field as the card gives it. For simplicity it treats each state's clearance as one number; the paper takes the minimum over the car's boundary points.

Python

import numpy as np
import matplotlib.pyplot as plt

d0, tau = 1.0, 0.5

def T(dist):
    return np.maximum(0.0, (np.minimum(dist, d0) / d0) ** tau)

d = np.linspace(0, 2.1, 200)
plt.plot(d, T(d), label="soft-collision field T(d)")
plt.axvline(0.1, color="red", ls="--", label="mechanical slot clearance, about 0.1 m")
plt.xlabel("clearance to the nearest wall (m)")
plt.ylabel("field value")
plt.legend()
plt.show()

def r_sc(clearances):  # worst clearance on the path minus the start
    return min(float(T(c)) for c in clearances) - float(T(clearances[0]))

print("tight slot:", round(r_sc([1.0, 0.5, 0.1]), 3))
print("steady:    ", round(r_sc([0.5, 0.5, 0.5]), 3))

What you should see:

The plot shows the field: zero at contact, climbing fast over the first few centimetres, and flat beyond one metre. The red line marks the mechanical slot.
The field is zero at contact and climbs fast over the first few centimetres. At half a metre it is already most of the way to its cap.
Values at 1.0 m and 2.1 m are identical: beyond d0 the reward stops caring.
The tight-slot path gets a clearly negative score, and the steady path gets exactly zero.

Now look back at the drawing. To finish in the mechanical slot, the car has to end up about 0.1 m from the walls. Under this term, every successful mechanical park is penalised compared with where it started. Keep that in mind for the questions below.

What the authors report, and what they leave open

All results are reported by the authors. In simulation, each method was run on 200 random start-target pairs, and every learned method used the same perception model. REAP-SAC parks in 83.5 percent of standard-slot trials and 70.0 percent of mechanical ones, ahead of every baseline in the table. Plain SAC reaches 73.0 and 58.0, and Hybrid A* with a pre-built map reaches 67.5 and 50.0. REAP-SAC's standard-slot collision rate is 11.5 percent, against 22.0 for plain SAC. The baselines were adapted to a shared camera-based protocol, not reproduced exactly.

The ablations point to the scaffolding as the part that carries the method. Without the second, hybrid training stage, success falls to 0.0 percent and the timeout rate rises to 88.5. Removing the actor's helper decoders drops success to 23.0 percent with 71.5 percent collisions. Each reward term, taken out alone, costs several points.

The simulator table is the one to remember. A CAD-based CARLA setup scores 79.5 percent in simulation and 0.0 on the real car. A mesh simulator transfers at 45.0 percent, and the splatting simulator at 70.0. On the real vehicle, five scenarios with 20 trials each range from 55.0 percent in the mechanical slot to 85.0 percent in an open ground-level lot.

Open points the card flags:

Learning rates, the optimiser, the SAC temperature, the helper-loss weight, buffer sizes and mixing ratio, success thresholds and all reward magnitudes are not stated.
It is not stated whether the copy-the-planner decision is made per step, per episode or per batch.
The simulator table gives 82.5 percent where the main table gives 83.5, and it describes the real-car trials differently from the real-vehicle table. The paper doesn't explain either gap.
Each real trial moves a success rate by 5 points. No variance is reported, and results cover one vehicle and one seed.
No code, scans, simulator or models are released. Unseen outdoor scenes, moving obstacles and formal safety guarantees are left open by the authors.

Ask yourself

The soft-collision term penalises ending up closer to walls than you started, yet the mechanical slot leaves only about 0.1 m a side. How would you weight or reshape that term so it doesn't fight the goal in tight slots?
CARLA and the splatting simulator score almost the same in simulation but very differently on the real car. What would you measure to tell whether the gap comes from rendering realism, scene geometry or the camera noise?
The critic sees clean maps the actor will never have. Where else in your own systems could training-only information make learning faster without becoming a deployment dependency?