Task decomposition

Overview

Task decomposition splits a task into sub-tasks, solves each one, and keeps the results for later steps. This page covers seven methods and two taxonomies. In the prompt-only variant, a decomposer prompt writes a program of sub-questions, and a controller runs it 2210.02406 §3 §3.2. The motivation is that a few demonstrations of a complex task may not teach every step, so a model can fail on a step it could do alone 2210.02406 §1. In the fixed-pipeline variant, the order of prompt modules is set in advance: a text-to-SQL system links the schema, classifies the question, then generates SQL with a prompt chosen by that class 2304.11015 §4. In the staged-team variant, four prompts split one instruction into sub-tasks, pick a robot or a team for each, and write code that assigns them, which an interpreter then runs 2309.10062 §4. In the select-then-infer variant, a selection step picks facts from the context, and an inference step derives one new fact from only those facts 2205.09712 §4. In the learned variant, a high-level policy picks the next sub-task and a low-level policy carries it out, with a text memory that tracks progress 2603.03596 §3.1. Most of these pass earlier results forward in the prompt, and the learned variant rewrites a text memory 2210.02406 §3.2 2304.11015 §4 2309.10062 §4 2205.09712 §4 2603.03596 §3.2. In the routed variant, a classifier LLM sends each instruction to one module, and a prediction module writes arithmetic over detected boxes that is evaluated outside the LLM 2407.14229 §3.2 §3.3. The progress variant adds a check: the policy signals when a subtask is nearly done, and a VLM decides whether to move on or retry an earlier subtask 2601.02295 §4.2.

Variants

Each variant rests on a single card here. Other decomposition methods named in the first paper, such as least-to-most prompting and successive prompting, have no card here yet, so they are only named. The paper describes them as producing a linear sequence of sub-questions 2210.02406 §2.

Typed sub-task handlers

Each sub-question is routed by name to a handler. A handler can be a few-shot prompt, either chain-of-thought style or another decomposed prompt; a symbolic function such as a calculator or a supervised model; or a retrieval system 2210.02406 §3. A retrieval handler returns both an answer and the documents, so later steps can use the answer while a reader uses the documents 2210.02406 §3.3 fig. 6. Card: Decomposed Prompting: A Modular Approach for Solving Complex Tasks.

Hierarchical decomposition

A sub-task that still fails with many examples can be split again. The paper's example splits a word into positioned letters, then picks the k-th one 2210.02406 §3.3 fig. 4.

Recursive decomposition

The decomposer calls the same procedure on smaller inputs. For list reversal, the paper splits a list into halves, reverses each half the same way, and joins them in reverse order. It stops at a base case of fewer than 4 items 2210.02406 §3.3 alg. 1. The paper reports that the base chain-of-thought prompt does not generalise to longer lists, while the recursive version does on lengths 4 to 10 2210.02406 §4.2 fig. 8.

Fixed pipeline of prompt modules

No decomposer writes the program here. The order is fixed. For text-to-SQL, four modules run in sequence, each a prompt to the same LLM: schema linking, which lists the tables, columns, join conditions and values the question uses; classification into EASY, NON-NESTED or NESTED, which also lists the tables to join and any sub-questions; SQL generation with one few-shot prompt per class; and a zero-shot self-correction pass 2304.11015 §4 fig. 2. The class decides which generation prompt runs, so a wrong label sends the query down the wrong branch 2304.11015 §4.2. The paper gives no accuracy for the classifier on its own 2304.11015 §5.6. Removing the schema-linking module lowers CodeX Davinci's Spider dev execution accuracy from 69.9 to 65.9, and removing the classifier lowers it to 63.1 2304.11015 tab. 5. The authors report 85.3% execution accuracy on the Spider holdout test with GPT-4, using no database content 2304.11015 tab. 2. Card: DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correction.

Staged planning for a robot team

Four prompt stages turn one instruction into code for several robots. The first splits the instruction into sub-tasks, each with the skills it needs. The second writes a coalition policy, which picks one robot or a team for each sub-task. The third writes code that assigns robots to sub-tasks and runs independent ones in parallel. The fourth is an interpreter that runs that code 2309.10062 §4 fig. 2. A single robot takes a sub-task only if it has every required skill; a team works if the members' skills together cover it 2309.10062 §3. The author-reported GPT-4 success rate is 1.00 on elemental tasks and between 0.62 and 0.71 on the other three categories of a 36-task AI2-THOR benchmark 2309.10062 tab. 1. In the ablation, removing the coalition stage lowers success from 0.75 to 0.60, but that table's task scope and run count are not stated 2309.10062 tab. 3. The real-robot runs report no metrics 2309.10062 §6.2. The released code makes three model calls, not four, and the card's Paper vs code section lists this and other differences. Card: SMART-LLM: Smart Multi-Agent Robot Task Planning using Large Language Models.

Routed modules with arithmetic over detections

A classifier LLM sorts each instruction into prediction, correction or confirmation 2407.14229 §3.2. A prediction then takes one of two branches. A named object goes to a language-grounded segmenter, and the point is the heatmap's maximum §3.3. A position relative to objects goes to an open-set detector, and the LLM writes two expressions for the target's x and y from the detected boxes. The paper's own code evaluates those expressions outside the LLM; the authors say this beat having the LLM compute the number by about 10% in preliminary experiments 2407.14229 §3.3. A correction gets the current point and new boxes and proposes a new target §3.4; the human confirms the final point §3 fig. 2. The paper's main summary is that splitting the task lets smaller models match larger ones: the best overall median is 0.61 with GPT-3.5, and 0.59 with a 7B model, on the same detector and segmenter 2407.14229 §4.1 tab. 1. The table's only baseline is random point sampling, which scores 0.13 tab. 1; it has no single undivided prompt to compare against. The paper also gives no sample count per table cell 2407.14229 §4.1. Card: Words2Contact: Identifying Support Contacts from Verbal Instructions Using Foundation Models.

Select-then-infer loop

Each step runs two few-shot prompts on the same frozen LLM. The selection prompt picks a few facts from the context. The inference prompt sees only those facts, not the question, and writes one new fact, which is appended to the context 2205.09712 §4 alg. 1. The authors say that letting the model generate the selection freely invented facts, so candidates are scored by log-likelihood and the best one is kept 2205.09712 §4.1 alg. 2. The loop runs for a fixed number of steps, and a learned stop is left for future work 2205.09712 §4 §7. With a 7B model in 5-shot, the paper reports 58.75% average generative accuracy over 10 logic tasks, against 41.32% for chain-of-thought with the same model 2205.09712 §5 fig. 4a. In the paper's §6 comparison, training the two modules separately raises the prompt-based 57.93% to 78.95% 2205.09712 §6. Card: Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoning.

Learned subtask policy with a text memory

A trained high-level policy outputs the next subtask instruction and an updated language memory. A low-level policy turns the subtask into a chunk of actions 2603.03596 §3.1. The memory is a short text summary, and its training labels come from a pre-trained LLM that summarises each episode's subtask history, with a success flag per subtask. The LLM is asked to drop or compress detail, such as replacing individual object attributes with a count and a location 2603.03596 §3.2. The memory does not change until a subtask succeeds, and the paper reports that concatenating raw instructions works significantly worse than learned summaries 2603.03596 §4.1. The same paper also adds a short-horizon video memory of recent camera frames. That is a separate mechanism from the subtask split, and this page does not cover it 2603.03596 §3.3. Card: MEM: Multi-Scale Embodied Memory for Vision Language Action Models.

Subtask progress with VLM backtracking

Training labels split each demonstration into subtasks. Gripper segments set the boundaries when their count matches the subtask count; otherwise an LLM infers them from movement primitives 2601.02295 §4.1 app. B. The policy then outputs a stop flag and a progress value per step, in the same head as its actions 2601.02295 §4.1. At inference, once progress reaches 0.9, a VLM sees both camera views and the subtask list. It either moves on or sends the robot back to an earlier subtask 2601.02295 §4.2 app. G. A signal counts as confirmed when it holds on two consecutive steps, or recurs after two low steps app. D. A retry draws several action chunks from the same observation and runs the one closest on average to the others, which is Minimum Bayes Risk selection 2601.02295 §4.2 eq. 4. Each subtask gets at most three retries app. D. Card: CycleVLA: Proactive Self-Correcting Vision-Language-Action Models via Subtask Backtracking and Minimum Bayes Risk Decoding. The released code does not use the paper's MBR formula by default, and it applies the confirmation rule only to some benchmark suites. The card's Paper vs code section lists both differences.

Taxonomy only: planning with and without feedback

A survey of LLM-based agents gives no method of its own. It describes an agent with four modules (profile, memory, planning, action) and places planning in two families: planning without feedback, and planning with feedback from the environment, a human or the model itself 2308.11432 §2.1 §2.1.3. Its remark is that planning without feedback suits short tasks, while planning with feedback handles long-range tasks but needs more design 2308.11432 §2.1.3. Card: A Survey on Large Language Model based Autonomous Agents.

Taxonomy only: segmenting recorded tasks into tagged actions

A labelling scheme for robot manipulation of deformable objects, with no algorithm and no trained model 2412.20998 §4. The authors label the time instants where a robot action changes any category, and each action segment receives one tag set 2412.20998 §5.2 tab. 2. The tags cover deformation, robot motion and grasp or contact. Labelling is by hand, and the paper gives no protocol beyond these rules or any agreement score 2412.20998 §5.2. It is useful for describing how a demonstration is split into steps, not for splitting a task at run time. Card: T-DOM: A Taxonomy for Robotic Manipulation of Deformable Objects.

When to use which

Results are author-reported on each paper's own tasks and models. The prompt-variant paper gives no significance tests 2210.02406 §4. The learned-variant paper's numbers are mostly in figures, and its text gives few values 2603.03596 §4. The fixed-pipeline paper's dev-set numbers are not held out, since most development used Spider dev, so compare dev results only with other dev results 2304.11015 §5.5.2. Its cost is about $0.5 and about 60 seconds per question with GPT-4 2304.11015 §7.

For arithmetic word problems, the prompt-variant paper reports GSM8K rising from 36 to 50.6 and MultiArith from 78 to 95, when a GPT-3 sub-task extracts the answer from a chain-of-thought. The metric for these two numbers is not stated, and the card notes that the samples are small (300 and 200 questions) 2210.02406 §4.5 app. B.

Implementation sketch

The loop below is the common core of the prompt variant. Names are our own. The variants differ in what a handler does and whether it can call the controller again.

# Controller for a decomposition program  [arxiv:2210.02406 §3.2] [fig. 3]
# Input:  query; decomposer prompt; handlers keyed by sub-task name
# Output: final answer

def decompose_and_answer(query, decomposer, handlers):
    # Earlier (sub-question, answer) pairs; empty at the start  [§3.2]
    history = []
    # Decomposer names the next sub-task and writes its sub-question  [§3.2]
    step = decomposer(query, history)
    while step.name != "[EOQ]":
        # Prompt, function, retrieval, or this same controller on a smaller input  [§3]
        answer = handlers[step.name](step.sub_query, history)
        # The next decomposer call sees this pair  [§3.2]
        history.append((step.sub_query, answer))
        step = decomposer(query, history)
    # The last answer is the prediction  [§3.2]
    return history[-1][1]

The paper's decomposer may refer to earlier answers as #1, #2, and so on 2210.02406 §3.1. It gives no cap on the number of steps 2210.02406 §3.2, so a real implementation needs its own limit.

The learned variant has the same shape with three changes. The decomposer is a trained high-level policy that outputs the next subtask and rewrites a text memory, which replaces the history. The handler is a low-level policy that returns actions rather than text. The loop ends when the task is done, not at an end marker 2603.03596 §3.1 §3.2. The paper card has a control-loop sketch with these changes.

The progress variant adds a check step. When progress passes the threshold, a VLM either advances or rewinds to an earlier subtask. A rewind resamples chunks and runs the MBR choice 2601.02295 alg. 1 §4.2.

The fixed pipeline has no loop. Each module runs once, in order, and the last module's output is the answer 2304.11015 §4. The classifier's label is the only branch, and it selects which generation prompt runs.

The staged team planner has no loop either. Its four stages run once each, in order, and the code written in the third stage is the plan the interpreter runs 2309.10062 §4.

The routed variant keeps the one-pass shape but adds a branch and a loop. A classifier names the module for each instruction, and a correction re-enters the loop until the user confirms 2407.14229 §3 fig. 2. The card's pseudocode shows this loop.

Contradictions

No reported numbers conflict across the cards. The papers use different tasks, metrics and models, so their results cannot be compared directly.

Within the learned-variant paper, the post-training memory horizon is "up to 18 frames and 54 seconds" in one section and "up to 1 minute" in another 2603.03596 §3.4 §4.3. The 54-second figure implies a frame spacing that the paper does not state. The high-level policy is also written as conditioned on the current observation alone in one place and on the last K observations in another 2603.03596 §3.1 §3.2. Neither conflict is resolved here.

Within the staged-team paper, GPT-4's Complex success rate is 0.71 in the main benchmark table and 0.48 in the variability study, which runs five times on one random task per category. The paper does not explain the gap 2309.10062 tab. 1 tab. 2 §6.1. Neither number is resolved here.

Within the prompt-variant paper, its worked example says to extract the third letter after a first-letter step, while the experiments use position 3 in a different setup 2210.02406 §3.1 §4.1. Implement from the prompts, not the example.

Papers

• Decomposed Prompting: A Modular Approach for Solving Complex Tasks: a decomposer prompt writes a program of sub-questions, run by a controller with typed handlers, hierarchical and recursive decomposition.
• DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correction: a fixed pipeline of prompt modules (schema linking, classification, class-specific SQL generation, self-correction) for text-to-SQL, with no fine-tuning.
• SMART-LLM: Smart Multi-Agent Robot Task Planning using Large Language Models: four prompted stages (decomposition, coalition formation, allocation, code) that turn one instruction into a plan for a multi-robot team in AI2-THOR.
• Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoning: a frozen LLM alternates a selection prompt over the context with an inference prompt that sees only the selected facts, adding one fact per step.
• A Survey on Large Language Model based Autonomous Agents: a survey with a four-module agent framework and a split of planning into families with and without feedback; reports no experiments.
• MEM: Multi-Scale Embodied Memory for Vision Language Action Models: a high-level policy outputs subtasks and rewrites a text memory, trained with LLM-written summaries, on top of a VLA robot policy.
• CycleVLA: Proactive Self-Correcting Vision-Language-Action Models via Subtask Backtracking and Minimum Bayes Risk Decoding: a VLA with stop and progress outputs, where a VLM decides to move on or backtrack, and retries pick a chunk by MBR.
• Words2Contact: Identifying Support Contacts from Verbal Instructions Using Foundation Models: an LLM classifies each instruction and routes it to a segmentation or detection branch, with LLM arithmetic over boxes and user corrections before confirmation.
• T-DOM: A Taxonomy for Robotic Manipulation of Deformable Objects: a hand-labelled tag set that splits recorded deformable-object tasks into tagged action segments; no algorithm and no robot experiment.