Paper · 2026-10-14 · cs.RO, cs.LG
Investigating Behavior Cloning from Few Demonstrations for Autonomous Driving based on Bird's-Eye View in Simulated Cities
Behaviour cloning for CARLA driving from six-channel 192x192 bird's-eye-view inputs. Each expert sample's Beta negative log-likelihood is weighted by the inverse of a kernel density estimate of its action, so rare braking and launch actions count more. Author-reported: on Town02, which was not used for training, driving score 98 ± 0.03 vs 34 ± 18 for plain BC.
Imitation learningBird's-eye-view (BEV) representationImportance weightingautonomous-drivingbehavior-cloningimitation-learningcarlabird-eye-viewkernel-densitytraffic-lights
AI-compiled by claude-haiku-5-5 on 2026-11-13, not yet reviewed by a human. Every claim cites its source; check it when a detail matters.
In plain words
Self-driving systems can learn by copying an expert. You record what the expert does in each situation, then train a network to act the same way. This paper does that in the CARLA driving simulator. The network sees a top-down map of the road around the car, not raw camera pictures.
The problem is that some of the moments that matter most are rare in the recordings. Stopping for a red light and pulling away on green happen only briefly, while cruising straight happens all the time. Plain copying therefore learns the common case well and the rare case poorly. The authors weight each training example by how rare its action is, so these moments count for more during training.
The authors report that this lets the agent finish its routes in the town it trained on, and in a town it never saw during training, where plain copying did much worse. These numbers come from the authors' own tests in simulation.
TL;DR
A behaviour-cloning policy for CARLA driving takes a six-channel 192×192 bird's-eye-view image and the car's recent state, and outputs acceleration and steering as Beta distributions. Each training sample's negative log-likelihood is multiplied by the inverse of a Gaussian kernel density estimate of its expert action §2.2 §2.3 eq. 3. The authors report higher route completion and fewer red-light infractions than plain BC on both the training town and an unseen town. No official code was found. The training details (optimiser, learning rate, epochs, kernel bandwidth) are not stated.
Problem
Behaviour cloning from expert demonstrations is the usual way to train driving policies in CARLA §1. Earlier CARLA work left traffic lights out of scope. Couto, a co-author, reported behaviour cloning results in CARLA in which signals were not part of the task [Couto2021] §1.
The authors identify two issues §1:
- Stopping at red and launching at green are underrepresented in the expert data, because they occur only at light transitions §1.
- Only eight training routes are used §1 §4.2, so a plain loss does not learn the rare actions well.
The authors argue that a mid-level input, a bird's-eye view, makes the policy's job easier, because the network does not have to learn perception from raw pixels §1 §2.1.
Method
BEV input
The policy input is a bird's-eye view centred on the vehicle. The view is a set of images of size W × H pixels at a ground resolution of φ metres per pixel. The agent's own position stays at a fixed point in the image, so the view moves with the car §2.1. φ and the physical range of the view are not stated.
The input has six channels, each 192×192 §2.1 §3.1 fig. 1 fig. 3:
- desired route;
- drivable area;
- lane boundaries;
- traffic-light state at three past time steps, −16, −9 and −1 §3.1 fig. 1.
The BEV channels come from a CARLA module that has access to the city map §3.1. They are not produced by perception on camera images in this paper. fig. 1 shows three 256×144 frontal cameras only as an illustration.
Behaviour cloning with a Beta policy
The policy is a stochastic policy π_θ(s) that outputs two actions, acceleration and steering, both in [−1, 1] §2.2 §3.2. The policy is parameterised by the shape parameters α and β of a Beta distribution for each action component §2.2. The Beta distribution has bounded support, so the policy needs no clipping or squashing of its output §3.2.
Expert actions in [−1, 1] are mapped to [0, 1] before the likelihood is computed §2.2:
- a' = (a + 1) / 2 §2.2.
The plain BC loss is the mean negative log-likelihood over the N expert samples, summed over the acceleration and steering components eq. 1.
# Plain behaviour-cloning loss, Beta policy [eq. 1] [§2.2]
# Input: states (N, ...); expert actions (N, 2) in [-1, 1]
# Output: scalar loss
def bc_loss(states, actions):
# Shape parameters of the Beta distribution, per action [§3.3]
alpha, beta = policy(states) # (N, 2) each
# Map expert actions from [-1, 1] to [0, 1], the Beta support [§2.2]
targets = (actions + 1.0) / 2.0 # (N, 2)
# Beta log-density for acceleration and steering [§2.2]
log_likelihood = beta_log_pdf(targets, alpha, beta) # (N, 2)
# Average over the N samples; sum over the two components [eq. 1]
return -log_likelihood.sum(axis=1).mean()Density weighting
The density of each expert action is estimated with a kernel density estimator. The paper uses a Gaussian kernel and writes the estimate as a one-dimensional sum over the N data points, with bandwidth h §2.3 eq. 2:
- ĝ(x) = (1 / (N h)) Σ_j K((x − x_j) / h), with K(u) = exp(−u² / 2) / √(2π) §2.3.
Each training sample is weighted by w_i = 1 / ĝ(a_i), the inverse of the estimated density at its own expert action. The weight multiplies both components of that sample's loss §2.3 eq. 3.
# Gaussian KDE of one action value, as written in the paper [eq. 2] [§2.3]
# Input: x (scalar), data (N values), bandwidth (not stated in paper)
# Output: estimated density at x
def kde_density(x, data, bandwidth):
# Standardised distance to each expert value [eq. 2]
distances = (x - data) / bandwidth # (N,)
# Gaussian kernel at each data point [§2.3]
kernel = exp(-0.5 * distances ** 2) / sqrt(2 * pi) # (N,)
# Sum over the N points, scaled by N and the bandwidth [eq. 2]
return kernel.sum() / (len(data) * bandwidth)# Weighted behaviour-cloning loss [eq. 3] [§2.3]
# Input: states (N, ...); expert actions (N, 2) in [-1, 1]
# Output: scalar loss
def weighted_bc_loss(states, actions, densities):
# densities[i]: KDE of expert action i, computed once on the data [§2.3]
# Weight is the inverse density; no normalisation is stated [eq. 3]
weights = 1.0 / densities # (N,)
alpha, beta = policy(states) # (N, 2) each
targets = (actions + 1.0) / 2.0 # (N, 2)
log_likelihood = beta_log_pdf(targets, alpha, beta) # (N, 2)
# Weight each sample, sum over components, average over N [eq. 3]
return -(weights * log_likelihood.sum(axis=1)).mean()The unweighted baseline uses the same loss with every weight set to 1 §4.2.
Agent network
The policy has two input branches §3.3 fig. 3:
- The BEV branch is a convolutional network. The text says it outputs a 256-dimensional embedding §3.3. Figure 3 shows the BEV branch flattened to 1024 values instead (see Pitfalls).
- The state branch is a multi-layer perceptron that outputs a 256-dimensional embedding §3.3 fig. 3.
The two embeddings are concatenated and passed through a further MLP. Two output heads give the Beta parameters §3.3 fig. 3. Figure 3 shows each head as a two-unit linear layer followed by Softplus, and the caption says the two heads give α and β fig. 3.
# Policy network [§3.3] [fig. 3]
# Input: bev (B, 6, 192, 192); state (B, 6)
# Output: alpha (B, 2), beta (B, 2)
def policy(bev, state):
# Six convolutions, ReLU after each; stride 2 except the last [fig. 3]
features = bev
for channels, kernel, stride in BEV_CONVS: # see Hyperparameters
features = relu(conv(features, channels, kernel, stride))
bev_embedding = flatten(features) # (B, 1024) per fig. 3
# Two dense layers of width 256 for the state [fig. 3]
state_embedding = mlp(state, [256, 256]) # (B, 256)
joint = concat(bev_embedding, state_embedding) # (B, 1280)
# Dense layers 512, 256, 256, 256 before the heads [fig. 3]
hidden = mlp(joint, [512, 256, 256, 256])
# Softplus keeps the Beta shape parameters positive [fig. 3]
alpha = softplus(linear(hidden, 2)) # (B, 2)
beta = softplus(linear(hidden, 2)) # (B, 2)
return alpha, betaExpert data and training split
The environment is Town01 from the CARLA Leaderboard, with ten predefined routes §4.1. The expert is a deterministic agent that follows a dense point trajectory with a PID controller. The agent under test sees only a sparse trajectory, which gives a general direction and is placed every 50 m or at a change of manoeuvre §4.1 fig. 4. The expert breaks at red lights and accelerates when the light turns green §4.1. The paper does not say how the sparse points are drawn into the route channel.
The ten routes were recorded at 10 Hz, giving 21,287 labelled samples. The shortest route has 1,480 samples and the average has 2,129 §4.1. The paper reports about 30 GB of uncompressed data, about 36 minutes and 8 km of driving §4.1. The policy is trained on eight of the ten routes §4.2. The paper does not say which two routes are held out.
Hyperparameters
| Name | Value | Where | Notes |
|---|---|---|---|
| BEV input | 6 channels, 192 × 192 pixels | §3.1 fig. 1 fig. 3 | Resolution φ and physical range not stated §2.1 |
| Frontal camera image size | 256 × 144 RGB | fig. 1 | Shown only for illustration; not an input |
| BEV convolutions (kernel, stride, channels) | 5×5 s2 ×8; 5×5 s2 ×16; 5×5 s2 ×32; 3×3 s2 ×64; 3×3 s2 ×128; 3×3 s1 ×256 | fig. 3 | ReLU after each; padding not stated |
| BEV spatial sizes | 94, 45, 21, 10, 4, 2 (flatten to 1024) | fig. 3 | Consistent with unpadded convolutions |
| State inputs | Text: speed, previous acceleration, previous steering. Figure: six values | §3.1 fig. 3 | Conflict, see Pitfalls |
| State MLP | 256, 256; ReLU | fig. 3 | |
| Embedding width (text) | 256 for BEV and for state | §3.3 | Conflicts with figure (1024 flattened) |
| Fusion MLP | 512, 256, 256, 256; ReLU | fig. 3 | |
| Output heads | Two 2-unit linear layers with Softplus | fig. 3 | Heads are α and β, each over (acc, steer) per caption |
| Policy distribution | Beta, (a+1)/2 mapping | §2.2 §3.2 | |
| Kernel | Gaussian | §2.3 | |
| Bandwidth h | not stated | §2.3 | |
| KDE dimensionality | not stated (eq. 2 is 1-D; actions are 2-D) | §2.3 eq. 2 | See Pitfalls |
| Sample weight | 1 / ĝ(a_i), no normalisation stated | §2.3 eq. 3 | |
| Training routes | 8 of 10 | §4.2 | |
| Expert sampling rate | 10 Hz | §4.1 | |
| Evaluation frequency | Every 8 training epochs | §4.3 fig. 5 | |
| Runs per agent | 4 random runs | §4.3 fig. 5 | |
| Optimiser, learning rate, batch size, epochs, weight decay | not stated | ||
| Seeds, hardware, CARLA version | not stated |
Results
Author-reported. Setup: CARLA simulator, Town01 (training town) and Town02 (test town, not used in training). Each town has ten routes §4.3 tab. 1. The expert row is the scripted expert §4.1. Values are mean ± spread as printed in the table; the spread is not defined in the text tab. 1.
| Agent | Town | Driving score (%) | Route completion (%) | Infraction penalty (%) | Red-light infractions (#/km) | Crossed traffic lights (#) | Source |
|---|---|---|---|---|---|---|---|
| Expert | Town01 | 100 ± 0 | 100 ± 0 | 100 ± 0 | 0 ± 0 | 4.7 ± 1.9 | tab. 1 |
| Expert | Town02 | 100 ± 0 | 100 ± 0 | 100 ± 0 | 0 ± 0 | 9.0 ± 2.16 | tab. 1 |
| BC | Town01 | 81 ± 20 | 83 ± 18 | 97 ± 5 | 0.39 ± 0.68 | 3.78 ± 1.03 | tab. 1 |
| BC | Town02 | 34 ± 18 | 53 ± 18 | 66 ± 20 | 3.08 ± 2.45 | 2.79 ± 2.27 | tab. 1 |
| Weighted BC | Town01 | 100 ± 0 | 100 ± 0 | 100 ± 0 | 0 ± 0 | 4.7 ± 1.9 | tab. 1 |
| Weighted BC | Town02 | 98 ± 0.03 | 100 ± 0 | 98 ± 0.03 | 0 ± 0 | 9.0 ± 2.16 | tab. 1 |
The training curve in fig. 5 counts incomplete routes out of ten in Town01, every eight epochs, over four runs. Both agents start at ten. The weighted agent reaches zero infractions earlier and ends at zero, while plain BC does not §4.3 fig. 5. The KDE plot fig. 6 shows the highest density for straight driving at constant acceleration with zero steering, and the lowest for the short high-acceleration period after a red light turns green §4.3 fig. 6.
Authors' summary: the weighted agent matches the expert in Town01, and performs significantly better than plain BC in Town02 §4.3. The paper names no statistical test.
Paper vs code
No official code.
The paper names no code repository, and the arXiv record has no comments field. The paper's project page, sites.google.com/view/weighted-bc, lists videos only and no code link. GitHub searches for "weighted bc carla", "carla behavior cloning", "kde weighted behavior cloning", "antonelo carla" and "couto carla" found no repository from the authors. A GitHub user search for "antonelo" found accounts with unrelated repositories and no author match.
Pitfalls
- Weight scale and boundary actions. Eq. 3 divides by the density and does not normalise the weights, and no cap is stated eq. 3. Expert braking has acceleration −1 fig. 6, which maps to 0 under a' = (a + 1) / 2 §2.2. At 0 the Beta log-density is unbounded: if α < 1 the density is infinite, and if α > 1 it is zero. The paper says nothing about clamping or epsilon, so implementers must choose.
- Dimensionality of the KDE. Eq. 2 is one-dimensional, but the action is a pair (acceleration, steering) §2.3 §3.2. The paper does not say whether the density is joint over the pair, or per component and then combined. It also does not state whether the KDE is computed over the eight training routes or all ten §2.3.
- Bandwidth. h is not stated. The density values, and so the weights, depend strongly on it.
- Shared weight across heads. Eq. 3 uses one weight per sample for both components eq. 3. The paper does not discuss whether the two components should have separate weights.
- BEV embedding size conflict. The text says the CNN outputs a 256-dimensional embedding §3.3. Figure 3 shows a flattened 1024-value vector before concatenation fig. 3. The text and figure cannot both be right, and the card does not resolve it.
- State inputs conflict. The text lists three state values: speed, previous acceleration and previous steering §3.1. Figure 3 lists six: last throttle, last steer, last brake, last gear, and the two velocity components fig. 3. Implementers should check the code, if released, before choosing.
- Softplus is only in the figure. Figure 3 shows Softplus on the output heads, and the text does not mention a positivity mapping fig. 3 §3.3.
- Throttle and brake mapping. The car has steering, throttle and brake, but the policy outputs acceleration and steering. Braking is implied by negative acceleration, so the agent cannot brake and accelerate together §3.2. The conversion from the acceleration value to throttle and brake commands is not stated.
- Evaluation overlaps training. Training uses eight routes from Town01 §4.2, and evaluation in Town01 uses all ten routes §4.3. The paper does not say which two routes are unseen. Only the Town02 numbers are on unseen data.
- Identical expert and weighted rows. The Weighted BC Town01 row equals the expert row, including the crossed-traffic-light count, and the Town02 crossed-light count is also the same as the expert's tab. 1. The paper does not comment on this.
- Spread not defined. The ± values in the table are not defined in the text. The training curve uses four runs fig. 5, but the table does not say how many runs underlie its values tab. 1. The "significantly better" claim has no named statistical test §4.3.
- Infraction penalty values. The table note gives an example penalty of 0.7 for one red light. It is an example, not the CARLA leaderboard value, and the paper does not list the per-infraction penalties tab. 1, note a.
- Missing training details. The optimiser, learning rate, batch size, number of epochs, weight decay, input normalisation, augmentation and random seeds are not stated. The Hyperparameters table lists them as not stated.
- Simulator ground truth, not perception. The BEV comes from a simulator module with map access §3.1. Results therefore say nothing about transfer to perception from real sensors, which the paper does not test.
- Accepted manuscript. The title footnote says this version is the accepted manuscript. It does not reflect post-acceptance changes, and the version of record is at the DOI given in that footnote (LNCS 15413, pp. 155–168) [title footnote].
- Figure descriptions. The BEV camera figure is illustrative and not an input to the policy fig. 1.
Lineage
- Builds on: behaviour cloning for CARLA driving, Codevilla et al. (bibliography keys
codevilla2018endtoendandcodevilla2019exploring) §1. The Beta policy for bounded actions follows Petrazzini and Antonelo (Petrazzini2021) §3.2. The KDE reference is Silverman (Silverman2018density) §2.3. - Prior work from the same group: Couto and Antonelo (
Couto2021), behaviour cloning with raw pixels and no traffic-light requirement §1. - Compared against: plain behaviour cloning with the same policy and no weights §4.3 tab. 1.
- Superseded by: not stated.
Source and license
Adapted from “Investigating Behavior Cloning from Few Demonstrations for Autonomous Driving based on Bird's-Eye View in Simulated Cities” by Eric Aislan Antonelo, Gustavo Claudio Karl Couto, Christian Möller, Pedro Henrique Fernandes (arXiv:2610.20203), used under CC0 1.0. Changes: condensed and restructured for implementation, with tables, equations and algorithms taken from the paper and its code, each cited to its source. Not affiliated with or endorsed by the authors.