PONI: Potential Functions for ObjectGoal Navigation with Interaction-free Learning

In plain words

A robot is told to find a chair in a house it has never seen. One common way to do this is to let a learning agent wander and reward it when it finds the chair. This paper argues that half of the job, deciding which unexplored spot is the most promising place to look next, can be learned without the robot moving at all. The robot only needs a map it has built so far, with the objects it has seen marked on it.

The method trains a small image-to-image network on many top-down maps of real indoor scenes. From a partly filled map, the network guesses two things for every edge between known and unknown space: how much unexplored room lies beyond it, and how close it is to the target object. Averaging those two guesses gives a score, and the spot with the highest score becomes the next goal. A classic path planner then walks the robot there, step by step.

The appeal is cost. Learning to explore by trial and error can take a lot of simulated experience. Here the expensive part is a supervised training run on maps that already exist, and the navigation itself uses no learned policy.

TL;DR

• Modular ObjectNav: a learned "where to look" network plus an analytical local planner, with no RL in the loop §3.2.
• The network predicts an area potential and an object potential at map frontiers from a partial semantic map, trained with MSE on maps built from 3D scene annotations §3.3, §3.6.
• Author-reported: best Gibson (val) and MP3D (val) SPL among the methods compared, and the best MP3D test-standard SPL on the Habitat leaderboard at submission tab. 1, tab. 2.
• Training cost is reported as up to 1,600x lower than the MP3D state of the art (THDA), measured in GPU hours §5, fig. 4.
• Relevant when you want goal-directed search with a small, cheap training run. Check the segmentation dependence and the dataset terms before use (see Pitfalls).

Problem

ObjectNav asks an agent to reach within 1.0 m of a named object category and call stop, within a 500-step budget §3.1. Most strong methods learn the whole policy from interaction, often with reinforcement learning, which needs a lot of simulated experience and compute §1, §5. Modular methods that keep a semantic map and a planner still tend to learn the long-term goal choice by RL §4.1, SemExp. The paper argues that deciding "where to look" for an unseen object is a perception problem that can be trained from passive data, given a semantic map and a goal category §1, §3.3.

Assumptions: depth and odometry are noise-free in simulation, and semantic labels come from the 3D scene annotations of Gibson and Matterport3D §4. The paper says these choices match the CVPR 2021 ObjectNav challenge setup §4.

Method

PONI has three parts: a semantic mapper, the potential function network (PFN), and an analytical local policy §3.2, fig. 2.

1. Task and episode. At each step the agent gets 640 x 480 RGB-D, odometry and a goal category. Its actions are move forward, turn left, turn right and stop §3.1.

2. Semantic mapper. Depth and pose are projected into an allocentric top-down map of size (N + 2) x M x M. Channel 1 marks obstacles, channel 2 marks explored area, and the remaining N channels mark object categories. Pixels are labelled by a segmentation network, and the mapping follows SemExp §3.4 (see Lineage).

3. Frontiers. A frontier is an edge between explored free space and unexplored space on the partial map. Potentials are defined only at frontiers, because any route to an unexplored location crosses one §3.3.

4. Area potential target. Group the unexplored free-space cells into connected components. Attach a component to a frontier if one of its pixels is an 8-neighbour of a frontier pixel. The area potential of a frontier is the summed area of its attached components, divided by the total free space of the complete map. On MP3D the paper divides by a fixed constant instead §3.3, footnote 1 (see Paper vs code).

5. Object potential target. For a frontier x and category o, the object potential is max(1 - d_g(o, x) / d_max, 0), where d_g is the geodesic distance from x to the 1.0 m success zone around the nearest instance of o. The value d_max is the distance at which the potential reaches zero, chosen on validation data §3.3, eq. 1.

6. Network. A UNet encoder with four downsampling blocks feeds two UNet decoders with four upsampling blocks each. The area decoder outputs one channel. The object decoder outputs N channels, one per category. Both end in a sigmoid §3.3.

7. Long-term goal. Combine the two potentials as U = alpha * U^a + (1 - alpha) * U^o with alpha = 0.5 §3.3, eq. 2. Set U to zero at explored locations other than frontiers, then take the maximum as the long-term goal g_t. The paper says that keeping predictions at unexplored cells makes the choice tolerant of noisy frontiers §3.3.

8. Local policy. A Fast Marching planner computes the shortest obstacle-avoiding path to g_t, and the agent follows it with deterministic actions §3.5.

9. Training data. Take a complete semantic map of a scene, apply random translations and rotations, and pick two free-space cells. Compute the shortest path between them. Reveal a square patch around each path cell, and copy the complete map only where it was revealed. This gives a partial map. The targets are computed from the complete map §3.6. A 3 m x 3 m square gives results comparable to a view-cone mask [supp. §S4, tab. S2].

10. Loss. Pixel-wise mean squared error. The area term and the object term are both averaged over frontier pixels, and the total is their sum §3.6, eq. 4, eq. 5.

11. Optimisation. Adam with learning rate 1e-3, reduced by a factor of 10 after two epochs §4.2.

Key equations as stated in the paper:

U^o_t(o_t, x) = \max\bigl(1 - d_g(o_t, x)/d_{max},\ 0\bigr) eq. 1

U_t = \alpha\, U^a_t + (1 - \alpha)\, U^o_t, \quad \alpha = 0.5 eq. 2

L_a = \frac{1}{|\mathcal{F}|}\sum_{x \in \mathcal{F}} \lVert \hat U^a(x) - U^a(x) \rVert_2^2 eq. 4

L_c = \frac{1}{|\mathcal{F}| N}\sum_{x \in \mathcal{F}}\sum_{n=1}^{N} \lVert \hat U^o(x, n) - U^o(x, n) \rVert_2^2 eq. 5

Here F is the set of frontier pixels and N the number of object categories. The total loss is L = L_a + L_c §3.6.

Pseudocode for goal selection at inference, following the paper's description. Frontier and explored masks are derived from the partial map as described in step 3:

# Long-term goal selection for one ObjectNav step  [§3.3] [eq. 2]
# Input:  semantic_map (N + 2, M, M) from the mapper  [§3.4]
#         goal (int) category index; explored (M, M) bool
# Output: long_term_goal (row, col)

def select_long_term_goal(semantic_map, goal, explored):
    # Encode the partial map and decode both potentials  [§3.3]
    area_pf, object_pfs = potential_function_network(semantic_map)
    object_pf = object_pfs[goal]  # (M, M) channel for the goal  [§3.3]
    # Average the two potentials with alpha = 0.5  [eq. 2]
    potential = 0.5 * area_pf + 0.5 * object_pf
    # Frontier: explored free cell next to an unexplored cell  [§3.3]
    frontier = compute_frontiers(semantic_map, explored)
    # Zero explored cells that are not frontiers  [§3.3]
    potential = where(explored & ~frontier, 0.0, potential)
    # Take the maximum as the long-term goal  [§3.3]
    row, col = argmax(potential)
    return (row, col)

Pseudocode for one training loss evaluation, from the paper's equations:

# Training loss for the potential function network  [§3.6] [eq. 4] [eq. 5]
# Input:  pred_area (M, M), pred_objects (N, M, M) in [0, 1]
#         target_area (M, M), target_objects (N, M, M)
#         frontier (M, M) bool, the frontier pixel set F  [§3.6]
# Output: scalar loss

def potential_loss(pred_area, pred_objects, target_area, target_objects,
                   frontier):
    # Area term: squared error averaged over frontier pixels  [eq. 4]
    loss_area = ((pred_area - target_area) ** 2)[frontier].mean()
    # Object term: squared error averaged over frontier pixels and categories  [eq. 5]
    loss_object = ((pred_objects - target_objects) ** 2)[:, frontier].mean()
    return loss_area + loss_object

Hyperparameters

Paper values come first. Code values are listed where the paper is silent or differs. Code values are defaults unless a script overrides them. Section 4.2 states the training schedule.

Results

Author-reported. Setup: Habitat simulator. Gibson uses 25 train and 5 val scenes from the Gibson tiny split. MP3D uses the standard split of 61 train, 11 val and 18 test scenes. Baselines are non-interactive (BC, Predict-theta, Predict-xy, Predict-A), end-to-end RL (DD-PPO, Red-Rabbit, THDA) and modular (FBE, ANS, SemExp) §4, §4.1. Table 1 averages 3 seeds and gives no variance. Missing entries were not reported in the cited papers tab. 1.

Validation results (tab. 1)

Source: tab. 1. Succ. in %, SPL in %, DTS in m (lower is better).

Habitat challenge, MP3D test-standard (tab. 2)

Source: tab. 2. The caption says PONI is best on 3 of 4 metrics. Success is not one of them. Results are published challenge entries as of 14 November 2021.

Ablation (tab. 3)

Source: tab. 3. The last row uses ground-truth segmentation, which the paper labels as privileged.

Training cost (fig. 4). Author-reported. Cost is measured in GPU-hours (number of GPUs times training time). The end-to-end RL costs are taken from their papers fig. 4, §5. The paper reports PONI at about 7x lower cost than SemExp on Gibson, and up to 1,600x lower than THDA on MP3D §5.

Qualitative (fig. 5). The paper shows an episode in which the area potential drives early exploration. Once the map has enough context, the object potential picks a frontier near the bedroom entrance, and the agent finds the bed fig. 5, §5.

Object potential influence. The paper measures how often the object potential, rather than the area potential, sets the goal. Each episode counts if its goal lies at least 1 m from the area-potential maximum. That share rises later in the episode [supp. §S6].

Limitations (supp. tab. S1). Replacing predicted segmentation with ground truth raises Gibson success by 14.9 percent and MP3D success by 45.4 percent. SPL divided by success falls by only 6 percent on Gibson and 19 percent on MP3D [supp. §S1, tab. S1].

Paper vs code

Official code at code_commit (the repository of the first author, MIT). The repository contains training (train.py, poni/), the evaluation code (semexp/, hlab/) and experiment scripts. The cloned code is shallow and does not include the dependencies/ submodules (Habitat and related). The paper's values are cited as §. Code values as path.

1. Object cutoff d_max. The paper does not give a value §3.3. The code uses 10.0 m by default default.py and 5.0 m in the MP3D training script train_poni.sh. The potential is clipped (cutoff - d) / cutoff at zero dataset.py, which matches eq. 1 with d_max = cutoff.

2. Area potential normalisation. The paper divides by the total free space of the complete map, and on MP3D by a fixed constant §3.3, footnote 1. The code's default divides by half the complete floor area dataset.py. The MP3D script uses a constant of 40 m^2 train_poni.sh. The code keeps only the five largest frontier contours dataset.py and clips scores to [0, 1] dataset.py. The paper says nothing about either step.

3. Area loss mask. The paper averages L_a over frontier pixels F §3.6, eq. 4. The code averages over all pixels, with no frontier mask train.py. The object loss is masked by a dilated frontier mask, normalised per sample and per category dataset.py train.py. This is close to eq. 5 but not identical.

4. Epoch count. The paper trains for 3 epochs §4.2. The code sets a total of 40,000 updates default.py. The number of epochs is then ceil(40000 / ceil(N / effective batch)) train.py. For the Gibson script the effective batch is 40 train_poni.sh default.py. With 400,000 training tuples that gives 4 epochs, not 3. This is derived from the code and the paper's numbers, not from a run. The real count depends on the dataset on disk.

5. Explored-cell masking at inference. The paper zeroes explored cells except frontiers §3.3. The code multiplies the goal channel by umap = 1 - explored model_pf.py global_agent.py. Explored cells are zeroed with no frontier exception. The eval scripts enable this with --pf_masking_opt unexplored eval_poni.sh.

6. Extra agent-proximity mask at inference. The paper has no such step. The Gibson eval script passes --mask_nearest_locations eval_poni.sh. Scores within mask_size (1.0 m) of the agent are then set to zero model_pf.py. The default is off in the SemExp argument parser arguments.py and on in the transfer config default.py. The MP3D eval scripts use the transfer config.

7. Potential weighting. The paper's alpha is 0.5 §3.3. The code blends the two maps with area_weight_coef = 0.5 model_pf.py arguments.py. The distance-based blend, dist_weight_coef (0.3) arguments.py, is off unless add_agent2loc_distance_v2 is enabled, and PONI does not use it. The paper's action-cost variant [supp. §S5, eq. 6] also does not appear in PONI.

8. Free-space channel. The network input is built from the obstacle and explored channels: free space is explored and not obstacle model_pf.py. The paper does not describe this.

9. Frontier definition. The code finds frontiers with a shifted-neighbour test on the unexplored and free maps geometry.py. The paper gives the definition in words only §3.3.

10. Architecture. The code's UNet uses batch normalisation and bilinear upsampling by default model.py model.py. Width is nsf = 32 default.py. The paper gives neither.

11. Object count. The MP3D script sets 23 model channels: 21 object categories plus floor and wall train_poni.sh train.py. The paper lists 21 object categories §4.2. The Gibson default of 17 channels is 15 objects plus floor and wall default.py.

No code found for the training schedule beyond the above.

Pitfalls

• Segmentation dependence. The stop decision uses only the semantic map, and the method has no learned mechanism for segmentation errors. Ground-truth segmentation lifts Gibson success by 14.9 percent and MP3D success by 45.4 percent [supp. §S1, tab. S1].
• Small margins on Gibson. PONI leads SemExp by 1.9 points success and 1.4 SPL on Gibson val tab. 1. Table 1 has no variance, so this difference is within what you should check with your own seeds.
• Metric mix on the challenge. On MP3D test-standard, PONI has the best SPL, SoftSPL and DTS but not the best success tab. 2.
• Compute claims. The 1,600x figure depends on reported GPU-hours and on the RL costs taken from the original papers fig. 4. It is not a controlled comparison.
• Goal period T = 1. The planner is re-invoked every step. The paper says this is tuned on validation data, and it differs from T = 25 for the prior modular methods §4.2.
• Argmax ties and empty maps. The paper does not say what happens when the potential is flat or zero everywhere. The code takes the argmax over the flattened map model_pf.py and does not say how ties are broken.
• Frontier masks at training time. Loss and target masks are dilated by about 0.15 m dataset.py. This matters for narrow corridors.
• Map scale. Map scale and crop sizes are only in the code. Using a different resolution changes the square-patch size (3 m) and the planner's cell size together dataset.py.
• Data terms. The training maps come from the Gibson and Matterport3D scene datasets and their semantic annotations. Both have their own terms of use. Gibson's semantic annotations come from 3DSceneGraph DATASETS.md. This card does not cover those datasets. Check their terms before commercial use, and do not assume the MIT code license covers them.
• Missing artefacts. The evaluation scripts expect checkpoints under pretrained_models/ and precomputed datasets under data/. The clone contains neither directory eval_poni.sh.
• Habitat dependencies. Simulation requires the Habitat submodules and the astar_pycpp fork. The .gitmodules file points them at SSH URLs .gitmodules.

Lineage

• Builds on: SemExp, arXiv:2006.13171, for the semantic mapper and the analytical local policy §3.4, §3.5. PONI replaces SemExp's learned long-term goal policy with the potential function network §3.3, §4.1. The area potential follows the exploration reward used in prior work §3.3.
• Compared against: non-interactive baselines (BC, Predict-theta, Predict-xy, Predict-A); end-to-end RL (DD-PPO, Red-Rabbit, THDA); modular methods (FBE, ANS, SemExp) §4.1.
• Superseded by: not known from this wiki.