LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Models

In plain words

A household robot needs a plan before it moves, such as "pick up the potato, then open the fridge". This paper uses a large language model (GPT-3) as that planner. Instead of training a planner on thousands of human-labelled examples, it puts a handful of similar, already-solved examples into the prompt and lets the model write a plan for the new task.

A plan is only useful if it matches the room. So the planner also receives a list of the objects the robot has seen so far. If the robot gets stuck, for example because it cannot find something, the planner is asked again with the new object list and the steps already done, and it can change course.

All experiments use ALFRED, a benchmark of household tasks in simulated homes. The planner is paired with an existing controller that carries out each planned step, and it is compared with controllers that were trained on the full data set. The main message is that a few examples plus what the robot sees can come close to a fully trained system, though not in every setting.

TL;DR

• LLM-Planner writes a high-level plan, a list of (action, object) subgoals, for an ALFRED task. It uses GPT-3 with 9 in-context examples retrieved from a pool of 100 training pairs §4.1 §4.3 §5.3.
• It adds the observed objects to the prompt and re-plans when an action fails or after a step budget n. The value of n is not stated §4.4 alg. 1.
• Author-reported: 13.41% success on the unseen test split with goal instructions only, and 16.42% with step-by-step instructions, using HLSM as the controller tab. 1.
• Relevant if you need few-shot planning for simulated household tasks with a fixed action set. The paper evaluates only ALFRED, so it says nothing about real robots §5.1.

Problem

Embodied agents usually learn each task from many paired examples of language and trajectories. The paper argues this is costly and limits how many tasks an agent can learn §1. The full ALFRED training set has 21,023 pairs §5.1 §5.3.

Earlier LLM planners such as SayCan rank a list of admissible skills. That needs the agent to know the skills up front, and it calls the LLM once per candidate. The paper reports 22 calls per task for SayCan against 7 for LLM-Planner, even with SayCan given oracle knowledge of the scene §5.4. Static plans written at the start of a task also cannot adapt to what the agent finds §1 §4.4.

Assumptions:

• The task uses hierarchical planning. Once the high-level plan is fixed, the low-level planner no longer depends on the instruction §3.
• The "true few-shot" setting: there is no separate validation set. Prompt design and choices such as K are made by cross-validation on the same small training pool §3 §4.2 §5.3.

Method

Task and action space

The high-level plan is L_h = [g_0, ..., g_T]. Each subgoal g_i is a pair of a high-level action and an object §3. The action space has one Navigation action and seven ALFRED interaction actions: PickupObject, PutObject, OpenObject, CloseObject, ToggleOnObject, ToggleOffObject and SliceObject §3. The low-level planner turns each subgoal into primitive actions. In the paper it is the controller of HLSM, which is not trained on language data §3 §5.3.

Static planning

The prompt has these parts in order: a task introduction with the allowed actions, the retrieved examples, the current task description, optional step-by-step instructions, the completed plans, the visible objects, and a "Next plan:" marker where the model writes its answer §4.1 §4.2. The paper tested layout choices by cross-validation and kept the best one. The chosen layout uses plain action names, no punctuation and comma delimiters app. C tab. 3.

# Static high-level planner  [§4.1] [§4.2] [§4.3] [§5.3]
# Input:  instruction, observed objects, completed subgoals, training pool
# Output: list of (action, object) subgoals

def plan_with_llm(instruction, objects, done, train_pool, k=9):
    # Retrieve the k pool examples whose instruction embedding is nearest
    # [§4.3]; the pool is the 100 sampled training examples [§5.3]
    examples = nearest_k(instruction, train_pool, k)
    # Header, then examples, then the query ending in "Next plan:"  [§4.2]
    prompt = header(allowed_actions)
    for example in examples:
        prompt += format_example(example)
    prompt += format_query(instruction, objects, done)
    # Add +0.1 to logits of allowed action and object tokens and of the
    # observed objects  [§4.1] [§5.3]
    bias = {token: 0.1 for token in biased_tokens(objects)}
    # Greedy decoding with temperature 0  [§5.3]
    text = gpt3_complete(prompt, temperature=0, logit_bias=bias)
    return parse_subgoals(text)

Example retrieval

For each test instruction, the method takes the K training examples whose instruction embeddings are closest. The embedder is a frozen BERT-base model. The paper defines closeness as Euclidean distance between the embeddings §4.3. K is tuned under the true few-shot protocol. The final value is 9 §5.3 §5.6.

Logit biases

Logit biases raise the scores of the allowed actions and objects, and also of the objects the agent has observed. The paper uses +0.1 §4.1 §5.3. Removing the bias lowers the cross-validated HLP accuracy from 40.59 to 38.10, and removing both the bias and the retriever lowers it to 13.43 tab. 2.

Grounded re-planning

The static plan is executed step by step. The agent keeps a set of observed objects and a list of completed subgoals. When the current subgoal fails, or after n time steps, the planner is called again with the new observations §4.4 alg. 1 fig. 2. The paper says the new plan continues the completed part of the plan §4.4. Algorithm 1 instead restarts the subgoal index at 0 on the new plan alg. 1.

# Grounded re-planning loop  [alg. 1] [§4.4]
# n is the step budget between re-plans; the paper does not state it [§4.4]
# Input:  instruction, training pool, env (controller and object detector)

def run_episode(instruction, train_pool, env):
    # Objects seen so far and subgoals completed so far  [alg. 1]
    observed = set()
    done = []
    plan = plan_with_llm(instruction, observed, done, train_pool)
    index = 0
    steps = 0
    while index < len(plan):
        # Run one primitive action for the current subgoal  [alg. 1]
        env.step(low_level_plan(plan[index]))
        # Add the objects the detector reports for this frame  [alg. 1]
        observed.update(env.detect_objects())
        steps = steps + 1
        # Re-plan if the subgoal failed or the step budget is used up
        # [§4.4] [alg. 1]
        if env.subgoal_failed() or steps == n:
            plan = plan_with_llm(instruction, observed, done, train_pool)
            index = 0
            steps = 0
        # Move to the next subgoal once this one is complete  [alg. 1]
        elif env.subgoal_done(plan[index]):
            done.append(plan[index])
            index = index + 1
            steps = 0
    return done

Integration with existing agents

The integration needs two things from the agent: an object list, and a low-level planner that turns each subgoal into actions. The paper makes no other assumption about the agent §4.5. For the end-to-end results it is connected to HLSM. Only HLSM's high-level planner is replaced, and its navigation, perception and depth models are kept. Those are trained on simulator trajectories only §5.3 app. A.

Hyperparameters

Results

Author-reported. ALFRED, valid and test splits. SayCan has no test-split number because the benchmark forbids the test metadata it needs tab. 1. Hardware is not stated.

Full copy of the main results table tab. 1. Columns are success rate (SR), goal-condition success (GC) and high-level plan accuracy (HLP ACC). Where the source gives a range, the static and dynamic values are shown as "lower -- upper" as printed. HLP ACC is defined in §5.2.

Reported in the text:

• Grounded re-planning improves unseen-test SR by 1.83 points over static planning in the goal-only setting (11.58 to 13.41) §5.4 tab. 1.
• Under the same 100 examples, retrained HLSM and FILM "can barely complete any task" §5.4 tab. 1.
• The paper frames the result as comparable to HLSM trained on all data, despite using under 0.5% of the paired data §1 §5.4. 100 of 21,023 is about 0.48% (arithmetic from §5.3).

Ablation on the 100-pair pool, leave-one-out cross-validation, high-level plan accuracy tab. 2:

Scaling with training-pool size, valid unseen SR tab. 5:

The paper says HLP accuracy rises with more examples up to about 250, then flattens §5.6 fig. 3. Those larger pools were not used for the main results §5.6.

Static HLP accuracy by task type, valid unseen and valid seen tab. 4: Pick & Place 51 / 46; Stack & Place 38 / 25; Place Two 39 / 45; Examine 44.4 / 49; Heat & Place 36 / 48; Cool & Place 43 / 46; Clean & Place 48.8 / 32.

Cost: about 7 GPT-3 calls per task for LLM-Planner against 22 for SayCan §5.4.

Paper vs code

Code: official repository OSU-NLP-Group/LLM-Planner (MIT), linked from the project page and the paper's abstract. The owner is the authors' lab (The Ohio State University NLP group). Commit 64cd03d6b032c15f7018fc6bcd48beb37820c2bc was cloned for this card. The paper's experiments use text-davinci-003. The repository also ships a GPT-2-tokenizer based generator in hlp/ and a GPT-4o-mini ALFRED evaluator in e2e/.

• Object list source. Paper: detections from HLSM's perception model with confidence above 80% §5.3. The ALFRED evaluator reads the simulator's visibility flag directly, so the list is ground truth rather than detector output thor_connector.py [code: e2e/src/run_eval.py#L202]. Results from this code do not test the detector pathway.
• Retriever. Paper: frozen BERT-base-uncased, Euclidean distance §4.3. Code: paraphrase-MiniLM-L6-v2 sentence embeddings, cosine similarity hlp_planner.py hlp_planner.py.
• Logit bias. Paper: bias on allowed action and object tokens plus observed objects §4.1 §5.3. The hlp/ generator biases only the object string hlp_planner.py hlp_planner.py. The e2e/ LLM call has no logit-bias argument run_eval.py.
• Engine. Paper: text-davinci-003 §5.3. The hlp/ default is text-davinci-002 hlp_planner.py. The e2e/ config uses gpt-4o-mini config_alfred.yaml. The e2e README says the prompt was tuned for text-davinci-003 and that the GPT-4 family "performs slightly worse" README.md.
• Re-planning. Paper: on failure or after n steps §4.4 alg. 1. The e2e/ code re-plans only when an action raises an error, with at most max_retries per action and max_replanning per task [code: e2e/src/run_eval.py#L353-L370]. No step-count trigger appears in the code. The config ships with dynamic: False, which disables re-planning [code: e2e/config/config_alfred.yaml#L10-L12].
• Budgets. Paper: n not stated. Code: max_retries: 1, max_replanning: 5 config_alfred.yaml.
• Prompt text. Paper header: "Create a high-level plan ... using the allowed actions and visible objects" app. C. Code: a different header that adds "Follow the exact output format ... Always try to navigate to the object before interacting with it" and a whitelist of valid ALFRED object names [code: e2e/src/hlp_planner.py#L66-L90]. The paper does not mention the whitelist.
• Example plan split. Paper: not stated. Code: a random split point within each example's gold plan, with the first part as "Completed plans" [code: hlp/hlp_planner.py#L102] hlp_planner.py.
• Decoding. Paper: temperature 0 §5.3. Code: temperature 0 in both paths hlp_planner.py run_eval.py. Output length is 200 tokens in hlp/ hlp_planner.py and 300 in e2e/ run_eval.py. The paper does not state a limit.
• Number of examples. Paper: 9 §5.3. Code: num_in_context_examples: 9 config_alfred.yaml. No discrepancy.

Pitfalls

• Object list is the largest gap. The paper's grounding claim depends on a detector with a confidence threshold. The released ALFRED evaluator uses simulator ground truth. Numbers from that code are not the paper's numbers without a detector step thor_connector.py.
• Re-planning definition is ambiguous. The text says the new plan continues the completed part of the plan §4.4. Algorithm 1 sets the index back to the first subgoal of a freshly generated full plan alg. 1. An implementation that follows the text and the algorithm would behave differently. Choose one and test it.
• The step budget n is missing. Neither the paper nor the code gives a value for "after n time steps", and the code has no step trigger. Treat n as a free hyperparameter.
• Unsupported actions count as done. In the code, an action that is skipped as unsupported is added to the completed list with a "[SKIPPED - UNSUPPORTED]" tag run_eval.py. The paper does not describe this, and the tag then reaches the next prompt as a completed plan.
• Table 1 cells are garbled in the source. The HLSM row lists the Valid Unseen GC as 31.24 and the HLP ACC range as "31.24 -- 70.17", so the same value appears twice. The source also prints some ranges with bold markers on one end only. Check these numbers in the PDF before quoting them.
• Figure and appendix numbering was inferred. The LaTeX source has no numbers for figures and appendices. Figure numbers follow the source order (intro = 1, overview = 2, LOOCV = 3, case studies = 4, task-type success = 5). Appendix letters follow the section order (A to D). Confirm them in the PDF.
• Equation numbers do not exist. The method has no numbered equations. The only formal content is the plan and policy factorisation in §3 §3.
• Few-shot does not mean no tuning. Prompt layout, K and the bias weight were chosen by leave-one-out cross-validation on the same 100 pairs, which the paper calls "true few-shot" §3 §4.2 app. C. Re-running the selection on a new pool changes the outcome.
• Gains come with the controller. End-to-end success uses HLSM's navigation, perception and depth, and the paper says the low-level planner and detector cause much of the remaining error app. D. The planning metric (HLP ACC) is not the same as end-to-end success.
• Cost is per task, not per step. The 7 calls per task are an average for LLM-Planner §5.4. Re-planning adds calls. The paper does not report total calls or tokens for dynamic runs.
• Test metadata. Test-split numbers are only available through the ALFRED benchmark. SayCan could not be scored there tab. 1.
• Author-reported only. No independent reproduction is cited in the paper.

Lineage

• Builds on: hierarchical planning with high- and low-level planners §3; in-context learning with GPT-3 §1; dynamic example retrieval §4.3; logit biases §4.1. Sources are cited in the paper by key only, and no arXiv ids are given in the source.
• Compared against: HLSM, FILM, E.T., HiTUT, M-TRACK, LEBP, SayCan, and (SL)^3 tab. 1 app. B. (SL)^3 could not be compared under the same few-shot setting because its code was not public at submission app. B.
• Superseded by: not stated in the source; no later result is recorded in this wiki.