Feudal Networks for Visual Navigation

In plain words

A robot is shown a photo of something it should find, say a particular chair in a house it has never seen, and has to walk there with only its camera and a depth sensor. Earlier systems either learned by trial and error or drew a map made of pictures linked by distances, which needs position tracking and gets large.

This work splits the job across three helpers working at different scales. The top helper keeps a small picture of where it has already been, built from features learned without labels, so it can tell when a room is well explored. The middle helper looks at the current view and points at a spot on the image to walk toward, copying how people click on a screen to guide a robot through a house. The bottom helper turns that spot into a few small turns and steps. The whole system is learned from a dataset of those human clicks rather than from reinforcement learning, and it uses no map of metric distances.

TL;DR

FeudalNav is a three-level navigator for the image-goal task. It learns without RL, graphs, odometry or a metric map §1. The high-level manager keeps a latent memory map from self-supervised features, the mid-level manager predicts a pixel waypoint from RGBD plus that map, and the worker turns the waypoint into 15° turns and 0.25 m steps §4. Author-reported, on Gibson with the NRNS protocol, both variants beat NRNS on curved-path averages in success and SPL, but Stacked FeudalNav's SPL on straight paths is lower tab. 2. Evaluation is simulation only.

Problem

Visual navigation to an image goal, without odometry and in unseen scenes, has no distance reward to train an RL agent on §1. Graph-based methods need odometry and grow in memory as nodes are added, and some predict unseen neighbours §2.1. NRNS removes RL and learns from passive videos, but still builds a topological graph from images and odometry §1, §2.1. The paper argues that a learned latent memory can replace the graph, and that a hierarchy at two spatial and temporal scales makes the task easier to learn from little data §1, §4.2.

Method

Section numbers are Arabic: §4.1 is the paper's Section IV-A, §5.2 is V-B, and so on.

1. Human navigation data §3. A human teleoperator clicks a point on the egocentric view. Each click is mapped to one of three actions: forward 0.25 m, turn left 15°, or turn right 15° §3.2 fig. 4. Right clicks mark landmarks §3.2. Recorded per step: RGBD observation, action, click, and X, Y, Z position §3.2. A trajectory ends after 500 actions or when the space is covered §3.2. The data covers Gibson and Matterport scenes in Habitat-Sim §3.1. Totals: 300 trajectories, 14,281 landmarks, 103,498 frames tab. 1. Sample frames are in fig. 2; per-trajectory histograms are in fig. 3.

2. High-level manager: latent memory map §4.1.

• Training: a modified SMoG model learns image features from positive pairs §4.1. Positives come from clusters of views of the same area. A new view joins the cluster whose SuperGlue match confidence is highest, if that confidence is above ψ, and otherwise starts a new cluster §4.1 §5.2. Positives are sampled from each cluster. Standard augmentations are not used §4.1.
• Inference: each new view is assigned by feature MSE to the nearest cluster center if the MSE is below α, otherwise it becomes a new center §4.1 §5.2.
• Map: centers are projected to 2-D with Isomap. A region around each projected center grows with the number of views assigned to it, and each cluster gets its own colour in a three-channel map §4.1 fig. 4.

3. Mid-level manager: waypoint network §4.2.

• A ResNet-18 is fine-tuned to output the pixel the agent should move toward §4.2.
• Its first layer takes seven channels: the RGBD observation (4) concatenated with the three-channel memory map §4.2.
• The stacked variant concatenates the RGBD-map inputs of the two previous frames and of the current frame along the channel axis §4.2.
• Training uses the human point-clicks as the target for each frame of the human dataset §4.2 §5.2 fig. 3 fig. 6.

4. Goal matching §4.2. For the image-goal task, SuperGlue matches the current view against the goal image. If the match confidence is above β, the mean of the matched keypoints replaces the predicted waypoint §4.2.

5. Low-level worker: actions §4.3 fig. 5.

• The image is split into a 3 × 2 grid. Bottom-left turns left, bottom-right turns right, the bottom-middle cell does nothing, top-middle moves forward, and the top corners turn and then move forward §4.3 fig. 5.
• Temporal abstraction: the worker takes t = 2 environment steps per manager waypoint. The first step uses the predicted waypoint. For the next step the waypoint is re-located by matching a crop around it with SuperGlue, and the mean of the matches is used §4.3.
• With probability ε the worker takes a random action instead §4.3 §5.2.

6. Stopping §4.3. The agent stops when the goal-match confidence is above β and the depth at the centre of the matched keypoints is at most 1 m.

# Managers: choose a waypoint from the current view  [§4.1] [§4.2]
# Input:  observation (H, W, 4) RGBD; goal_image (H, W, 3) or None
# Output: waypoint as a (row, col) pixel

def choose_waypoint(state, observation, goal_image):
    # High-level manager: add the view to the latent memory map  [§4.1]
    feature = latent_encoder(observation)
    # Join nearest cluster if MSE < alpha = 300, else start a new one  [§5.2]
    update_memory_map(state.memory_map, feature)

    # Mid-level manager: predict where to move next  [§4.2]
    manager_input = concat_channels(observation, state.memory_map)
    # (H, W, 7): RGBD plus the 3-channel map
    waypoint = waypoint_net(manager_input)

    # Goal task: a confident goal match replaces the waypoint  [§4.2]
    if goal_image is not None:
        matches = superglue(observation, goal_image)
        if match_confidence(matches) > 0.7:      # beta = 0.7  [§5.2]
            waypoint = mean(matches)
    return waypoint

# Worker: act on one waypoint for t = 2 environment steps  [§4.3] [§5.2]
# Input:  env, observation, waypoint (row, col)
# Output: observation after the steps

def run_worker(env, observation, waypoint):
    # Two environment steps per manager waypoint, re-locating it in between
    for step in range(2):
        if step > 0:
            # Re-locate the waypoint by matching a crop around it  [§4.3]
            waypoint = match_crop(observation, waypoint)
        # Epsilon = 0.1 random action; its distribution is not stated  [§5.2]
        if random() < 0.1:
            actions = [random_action()]
        else:
            actions = grid_actions(waypoint, observation.shape)
        for action in actions:
            observation = env.step(action)
    return observation

# Map a waypoint pixel to actions on a 3 x 2 grid  [fig. 5] [§4.3]
# Assumes row 0 is the top of the image; the paper does not say so.
def grid_actions(waypoint, shape):
    height, width = shape[0], shape[1]
    row, col = waypoint
    # Bottom strip: outer cells turn, middle cell does nothing  [fig. 5]
    if row > 0.8 * height:
        if col < 0.4 * width:
            return [TURN_LEFT_15]
        if col > 0.6 * width:
            return [TURN_RIGHT_15]
        return []
    # Top strip: turn first on the outer cells, then move forward  [fig. 5]
    if col < 0.4 * width:
        return [TURN_LEFT_15, FORWARD_025]
    if col > 0.6 * width:
        return [TURN_RIGHT_15, FORWARD_025]
    return [FORWARD_025]

Hyperparameters

Results

Author-reported. Setup: Gibson test environments, NRNS protocol, about 6k start-goal pairs §5.3. Baseline rows cite [hahn2021no] (NRNS) and [wijmans2019dd] (DD-PPO). The paper does not say whether baseline numbers were re-run or copied.

Dataset statistics tab. 1:

Navigation, image-goal task tab. 2. Succ = success, values in %. * uses a simulator.

Ablation on the image-goal task tab. 3. "Latent map" marks whether the high-level memory map is an input. "Mid-level input" is what the waypoint network sees. All rows use the worker.

Headline claims as written in the text §6.2: on straight paths, Stacked FeudalNav is about 7% above NRNS in success, and its SPL is lower because the paper attributes this to broader exploration. On curved paths, the text reports gains of 69% in success and 27% in SPL over NRNS. Tab. 2 gives these as Stacked FeudalNav's success (40.50 vs 23.97) and FeudalNav's SPL (15.83 vs 12.43); see Pitfalls. The paper also reports about 37k training images vs 3.5M for NRNS §6.2.

Qualitative: waypoint predictions mostly overlap the human clicks, and when they diverge they stay on navigable areas fig. 6 §6.1. In the feature study, SMoG preserves ground-truth trajectory distances better than MoCo, SwAV and ResNet-18 on one Gibson trajectory fig. 7 §7.1. Adding the memory map gives the largest gain in success tab. 3 §7.2.

Paper vs code

No official code.

Searched: (1) the LaTeX source, links.tex and the arXiv abstract page: the only links are the dataset on Hugging Face (visnavdataset/lavn) and Google Docs. (2) GitHub search via the API for "FeudalNav", "feudal navigation" and "feudal visual navigation waypoint". The only candidate, visnavdev/feudalnav (MIT, 2 commits), is not linked by the paper, has no author names in its README or profile, and does not state a link to 2402.12498, so it does not clearly belong to the authors and was not cloned.

The dataset is a separate artefact: the paper footnote links it §1. Its Hugging Face page cites another arXiv id (2402.14281) for the dataset. Its licence is not checked here, and this card does not describe the dataset.

Pitfalls

• Table 2 and Table 3 disagree on FeudalNav's straight-path average. Table 3's FeudalNav row averages 45.73 / 37.69. Its own per-difficulty columns average to 47.57 / 38.41, which is what Table 2 reports. The Table 3 figures equal the NRNS w/ noise average. Treat 47.57 / 38.41 as the consistent value tab. 2 tab. 3.
• The curved-path percentages in the text mix two models. The 69% matches Stacked FeudalNav's success (40.50 vs 23.97). The 27% matches FeudalNav's SPL (15.83 vs 12.43). The Table 2 caption and the text attribute both figures to "our method" tab. 2 §6.2.
• Straight-path SPL is lower than NRNS. FeudalNav gets 38.41 and Stacked gets 30.74, against 41.49 for NRNS without noise tab. 2. The authors call this broader exploration, which is a hypothesis, not a measured cause §6.2.
• Baseline provenance is not stated. The NRNS and BC rows cite [hahn2021no], and the RL rows use extra data and more steps tab. 2. The paper does not say whether these were re-run under the same seeds or copied.
• Unstated thresholds and features. The α = 300 MSE threshold depends on the feature scale and dimension, which are not stated, so it may not transfer to another encoder without recalibration (this is an inference) §4.1 §5.2.
• No rule for when to leave a region. The paper says map density signals when to move on, but gives no density threshold §4.1.
• Goal-match confidence is not defined. The paper says a "confidence" is compared to β but does not say how it is aggregated over keypoints §4.2.
• Stop depth is underspecified. The 1 m test uses the depth at the centre of the matched keypoints. The paper does not say how that centre is computed §4.3.
• Grid origin and boundaries. The grid is defined by a 0.4 / 0.6 column split and a 0.8 row line fig. 5. The paper does not say whether row 0 is the top of the image; the pseudocode above assumes it is.
• Action accounting. A turn plus a forward step counts as two actions towards the 500-action limit §5.1. Whether it is two environment steps or one grouped step is not stated, so the pseudocode executes both.
• Random actions. Their distribution is not stated §4.3.
• Training subset. 117 of the dataset's trajectories are used §5.2. How they were chosen, and which scenes they cover, is not stated. The test environments are the NRNS Gibson set §5.3.
• Unknown training recipe. Optimizer, learning rate, batch size, epochs, loss and any pretraining for ResNet-18 are not stated. Standard augmentations are deliberately not used for SMoG §4.1.
• Simulation only. The conclusion discusses sim-to-real and modular transfer but reports no robot experiment §8. The real-world dataset is not in the paper body.
• Latency and compute are not reported in the paper body.

Lineage

• Builds on: Feudal RL [vezhnevets2017feudal] (not in this wiki), which this work borrows the manager/worker structure from without RL §2.2.
• Compared against: NRNS [hahn2021no], DD-PPO RL [wijmans2019dd], behaviour cloning with a metric map or GRU, and the SMoG, MoCo, SwAV and ResNet-18 features in the ablation §6.2 §7.1. SuperGlue [sarlin2020superglue] is used throughout for matching.
• Superseded by: not known from this paper.