wikillm

Paper · 2026-10-13 · cs.AI

Navalia: Advancing Cowork Agents through Verifiable Task Synthesis and Long-Horizon Post-Training

Navalia builds finance-report analysis tasks whose evidence, multi-step computation chain and reference answer pass independent LLM review and programmatic checks. The verified tasks train Qwen3.6-35B-A3B by ReAct-trajectory SFT, then RL with trajectory advantages shared across FIFO segments. Author-reported: OfficeQA Pro 39.60 to 62.66 and gains on all five benchmarks. No code released.

LLM agentSynthetic training dataReinforcement learning with verifiable rewards (RLVR)Policy gradientagentstool-usesynthetic-training-datareinforcement-learningfinancecontext-management

arXivPDFMarkdown16 min read

AI-compiled by claude-haiku-5-5 on 2026-11-21, not yet reviewed by a human. Every claim cites its source; check it when a detail matters.

In plain words

Teaching an AI assistant to answer questions about company financial reports needs many practice questions. Writing them by hand is slow, and asking a language model to make them up often gives questions with muddled numbers or answers that are simply wrong. This paper builds the practice questions from the numbers themselves. It pulls the figures out of each report, picks the ones a question needs, and chains simple calculations on top of them. Only questions that a second model can solve the same way, and that pass automatic checks, are kept.

The assistant is then trained in two steps. First it copies good worked examples, where it searches documents, runs calculations and writes an answer. Then it practises on the questions it sometimes gets wrong, and it is rewarded only for the final answer. Its earlier steps share in that final reward, so the search and reading steps that led to a right answer are reinforced too. The authors report large gains on a finance question set and smaller gains on office-work benchmarks. These numbers come from the authors' own runs, and the authors have not yet released the code or model weights.

TL;DR

Navalia is a pipeline for "Cowork Agents" (assistants that plan, call tools and finish multi-step work) on corporate financial reports §1. It has two parts: verifiable task synthesis, which produces questions whose supporting facts, computation chain and reference answer are recorded and checked §2, and a two-stage post-training recipe of cold-start SFT on teacher trajectories followed by RL §3. Applied to Qwen3.6-35B-A3B, the authors report gains on all five benchmarks, from 39.60 to 62.66 on OfficeQA Pro tab. 1. No code or weights are released, and the task synthesis depends on LLMs the paper does not name.

Problem

Improving cross-document evidence integration, multi-step numerical reasoning and long-horizon execution needs many training tasks that are analytically deep and verifiable §1. Expert authoring is costly and does not scale. Free LLM generation from raw documents can mix inconsistent metric definitions or reporting scopes, make arithmetic errors, or produce steps that do not depend on each other §1. Public documentation of the full data-to-post-training path for Cowork Agents was described by the authors as limited §1.

Assumptions: the tasks are built only from publicly disclosed corporate reports §1 §4.1. The evaluation uses fixed harnesses per benchmark §4.1 app. D.

Method

Notation. A task is a tuple of a question q, a reference answer y, an interactive environment C, supporting facts F+, a computation chain G, anchors a (companies, periods, metrics, scope) and operator parameters θ §2 eq. 1.

1. Dual-view representation. Each report d is mapped to two views: a Markdown document view r(d) that the agent browses with tools, and a fact view F(d) used for construction and checking §2.1 eq. 2. Each fact keeps its number and unit, its place in the source, the table around it, the period it covers, how its metric is defined, which entity and scope it applies to, and a version tag. Facts with incomplete numeric, period or unit information are dropped §2.1. The facts of all documents form one global database F eq. 3 §2.1.

2. Operator-driven design and environment. A predefined set O holds analytical operators and composite templates. Its categories are aggregation over periods, trend comparison, statistics, regression, numeric transforms and cross-company analysis §2.2. Composite templates chain compatible operators into multi-step tasks §2.2. The sampling order is: operator or template o, then anchors a conditioned on o and F, then parameters θ conditioned on o §2.2 eq. 4. The configuration must cover the required periods, peer data and every step's inputs; otherwise it is resampled §2.2. The candidate documents D_cand are joined by distractors drawn from the same companies' other reporting periods and from peers §2.2.

3. Semantic selection of supporting facts. The method does not pre-filter facts with keywords or fixed metric rules. An LLM reads every fact of every candidate document and judges whether it meets the operator, anchors and parameters, including period, metric definition, scope, unit and context §2.3. When a document's fact set is large, a within-document preselection runs first §2.3. A reasoning model then selects F+ across documents from the aggregated pool §2.3 eq. 5. Each selected fact keeps its identifier, location, period, unit, metric definition and scope §2.3. If F+ misses raw data required by the computation, or if its definitions or scopes do not match the task, the instance is discarded and resampled §2.3.

4. Executable computation chains. Simple single-step tasks (lookup, sum, rate of change, average) are answered directly by an LLM from F+ §2.4. Complex operators (regression, forecasting, statistics, numeric transforms) are split: the LLM extracts values and arranges them to the operator's input format, and a predefined numerical routine computes the result deterministically §2.4. For a linear chain of K steps the recurrence is z_1 = o_1(x_1; θ_1) and z_k = o_k(z_{k-1}, x_k; θ_k), where x_k is raw data and z_k is the step output §2.4 eq. 6. Later steps must use earlier outputs; a chain without real dependencies is excluded §2.4. Instances are dropped for incomplete data, type mismatches or failed evaluation §2.4. The final output is converted to the required unit, precision and format to become y §2.4.

5. Question generation and quality control. The question is generated from the chain and y. It must state every answer-affecting constraint (companies, periods, metrics, scope, algorithm parameters) and any unit, precision or output format §2.5. Single-step questions state the objective and scope; multi-step questions follow the chain's order and ask only for the final result §2.5. An independent review model answers the question from q and F+ alone, without the generator's reasoning, and compares its result with y §2.5. Tasks that pass are then checked programmatically: every non-final step's output must feed a later step, and the periods named in the question must match the supporting facts §2.5. Only tasks passing both stages are kept §2.5.

Pseudocode
# Build one candidate task instance  [§2] [eq. 1]
# docs: parsed reports; facts: global fact database  [eq. 3]
def build_task(operators, docs, facts):
    # Sample operator or template, then anchors, then parameters  [§2.2] [eq. 4]
    operator = sample_operator(operators)
    anchors = sample_anchors(operator, facts)
    params = sample_params(operator)
    # Resample when document coverage fails  [§2.2]
    if not covers_requirements(operator, anchors, docs):
        return None
    # Candidates plus distractors from other periods and peer companies  [§2.2]
    candidates, distractors = pick_documents(anchors, docs)
    # Read all facts of each candidate; preselect per document if large  [§2.3]
    pool = preselect_per_document(candidates, operator, anchors, params)
    # Reasoning model selects the supporting facts F+  [§2.3] [eq. 5]
    support = select_facts(operator, anchors, params, pool)
    # Resample if F+ misses raw data or mismatches definitions  [§2.3]
    if not sufficient(support, operator, anchors):
        return None
    # Chain: LLM for simple steps, numeric routine for complex  [§2.4] [eq. 6]
    chain = build_chain(operator, support, params)
    answer = evaluate_chain(chain)
    # Write the question with every answer-affecting constraint  [§2.5]
    question = write_question(chain, answer, anchors, params)
    # Independent review answers from q and F+ only; compare with answer  [§2.5]
    if not review_agrees(question, support, answer):
        return None
    # Programmatic checks: outputs used, periods match  [§2.5]
    if not programmatic_checks(chain, question, support):
        return None
    return (question, answer, chain, support, anchors, params)

6. Trajectory collection and cold-start SFT. A strong teacher model (not named) solves the verified tasks in the document environment with a lightweight ReAct harness and FIFO context management, and the complete trajectories are recorded §3.2 §1. Trajectories are kept only when the final answer is right, their tool calls are valid, and they reach a minimum turn count §3.2. They are grouped into capability pools by operator or template and sampled evenly across pools §3.2. Turn-level quality assessment masks the loss on low-quality responses but keeps them as context §3.2.

Training samples follow the FIFO scheme and advance 10 turns at a time. Each sample holds the task instructions, up to 20 earlier turns and up to 10 new turns §3.2. The loss is cross-entropy on the new turns that pass quality assessment, normalised by the number of supervised tokens. The supervised intervals do not overlap, so each response is trained in exactly one sample §3.2.

Pseudocode
# Cut one trajectory into FIFO SFT samples  [§3.2] [app. C]
# turns: ordered list of interaction turns, supervised per block
def fifo_samples(instructions, turns, history=20, stride=10):
    samples = []
    # Disjoint supervised blocks of `stride` turns  [§3.2]
    for start in range(0, len(turns), stride):
        # Visible history: up to `history` turns before the block  [§3.2]
        context = turns[max(0, start - history):start]
        # Loss only on the new turns of this block  [§3.2]
        new_turns = turns[start:start + stride]
        samples.append((instructions, context, new_turns))
    return samples

7. Long-horizon RL. The policy starts from the cold-start SFT model and uses the same harness and FIFO context §3.3. Before RL, tasks are selected by the SFT model's rollout outcomes. Tasks with both successes and failures are kept, and all-success tasks are dropped. All-failure tasks are kept only if a stronger model solves them §3.3.

For each question the policy samples G trajectories, each made of J_g segments §3.3 eq. 7. The final segment, the anchor, gives the reward R_g of the whole trajectory §3.3. Group-relative advantages are computed from anchor rewards only and then shared by all segments of the same trajectory, so earlier retrieval and analysis segments receive the task outcome §3.3. Dynamic sampling keeps only task groups whose anchor rewards vary §3.3. The loss is applied only to newly generated responses in each segment; reused history and environment observations are masked §3.3. If a single response hits the generation limit, the trajectory keeps its anchor reward for advantage and sampling statistics, but all its tokens are removed from the loss §3.3.

The rollouts are on-policy, and each rollout batch is used for one policy update §3.3 app. C. The objective is a clipped token-level surrogate with importance ratio r_u(θ) and a shared advantage Â_u, normalised by the number of valid tokens |M| §3.3 eq. 8.

Pseudocode
# Choose RL tasks from cold-start rollouts  [§3.3]
def select_rl_tasks(sft_policy, candidate_tasks, stronger_model):
    kept = []
    for task in candidate_tasks:
        outcomes = run_rollouts(sft_policy, task)
        # Keep mixed-outcome tasks; all-success tasks are dropped  [§3.3]
        if any_success(outcomes) and any_failure(outcomes):
            kept.append(task)
        # Keep all-failure tasks only if a stronger model solves them  [§3.3]
        elif all_failure(outcomes):
            if stronger_solves(stronger_model, task):
                kept.append(task)
    return kept
Pseudocode
# One RL update on a batch of tasks  [§3.3] [eq. 7] [eq. 8]
# eps_low, eps_high: clip bounds (values not stated in the paper)
def rl_update(policy, old_policy, tasks, G, eps_low, eps_high):
    # Sample G trajectories per question  [§3.3] [eq. 7]
    groups = [sample_group(policy, task, G) for task in tasks]
    # Keep only groups whose anchor rewards differ  [§3.3]
    groups = [g for g in groups if reward_variance(g) > 0]
    # Group-relative advantage from anchor rewards  [§3.3]
    # (normalisation not stated)
    advantages = group_advantages(anchor_rewards(groups))
    # Share each trajectory's advantage with all its segments  [§3.3]
    token_adv, token_logp, token_old_logp, token_mask = flatten_segments(
        groups, advantages, policy, old_policy)
    # Per-token importance ratio, new over old policy  [eq. 8]
    ratio = exp(token_logp - token_old_logp)
    clipped = clip(ratio, 1 - eps_low, 1 + eps_high)
    per_token = minimum(ratio * token_adv, clipped * token_adv)
    # Token-level normalisation over all valid tokens |M|  [§3.3] [eq. 8]
    loss = -sum(per_token * token_mask) / count(token_mask)
    return loss

Harness and context. The ReAct harness is described as lightweight and single-agent §3.1 §5. The FIFO window used in training and the RL rollouts holds at most 30 interaction turns and advances by a stride of 10 §4.1 app. C. The paper does not describe the system prompt or tool interfaces.

Hyperparameters

Settings stated in the paper; everything not listed is "not stated". The training values come from app. C; the evaluation values come from app. D.

NameValueWhereNotes
initial modelQwen3.6-35B-A3B§4.1 app. Cbase context length 262,144 tokens §4.1
SFT frameworkslimeapp. C
SFT epochs2app. C
SFT global batch size128app. C
SFT max training sequence length256K tokensapp. C
RL frameworkdressageapp. Con-policy training
RL updates per rollout batch1§3.3 app. C"single policy update"
max tokens per model response64Kapp. C
max interaction turns per trajectory200app. C
FIFO visible windowup to 30 turns§4.1 app. C
FIFO stride10 turns§3.2 §4.1 app. C
SFT history per sampleup to 20 turns§3.2
SFT new turns per sampleup to 10 turns§3.2
SFT minimum interaction turns (filter)not stated§3.2threshold not given
SFT turn-level quality criterionnot stated§3.2who judges and how is not given
learning rate, optimiser, schedulenot stated
RL group size Gnot stated§3.3
RL clip bounds eps_low, eps_highnot stated§3.3
RL learning rate, batch size, stepsnot stated
RL advantage normalisationnot stated§3.3
reward function (beyond anchor-based final reward)not stated§3.3
dynamic sampling rulekeep groups with nonzero reward variance§3.3
teacher model (SFT trajectories)not stated§3.2"a strong teacher model"
stronger model (RL task filter)not stated§3.3
reasoning model (fact selection)not stated§2.3
review model (independent check)not stated§2.5
within-document preselection thresholdnot stated§2.3"when the candidate fact set is large"
operator set size and sampling distributionnot stated§2.2categories given, counts not
SFT trajectory count (final mix)not stated§3.2Table 2 reports 6,031 for the FIFO ablation tab. 2
OfficeQA Pro evaluationFull Corpus, 133 questions, max 200 stepsapp. Dsliding window of the last 30 assistant turns
FinSearchComp evaluationT2 and T3, max 200 turnsapp. Dfull history kept, no truncation
Finance Agent evaluationPublic-77 (50 from v1.1 + 27 from v2), 489 criteriaapp. DLLM judge with thinking mode
APEX-Agents-AA evaluation452 of 480 tasks, Stirrup harness, Pass@1app. D
Workspace-Bench-Lite evaluation100 tasks, Claude Code harness, rubric pass rateapp. D
evaluation sampling settings and seedsnot statedapp. D
training-data statistics5.85 supporting documents, 208.69 distractors per task (maxima 15 and 994)fig. 2 §4.3which training set is not stated
training-trajectory size59.27 tool-use turns, 61.10 tool calls on average§4.3

Results

Author-reported throughout, on the five benchmarks listed in app. D. The baseline Qwen3.6-35B-A3B and Navalia were run by the authors under a unified setup within each benchmark §4.1 app. D. Table 1 marks published results with * (the whole row when the marker follows the model name, one cell when it follows a score). Scores marked † used a different document representation or harness and are for reference only tab. 1. Unmarked values were obtained by the authors. The paper reports single values, with no seeds, run counts or intervals for the main table tab. 1.

Setup: the baseline is the unmodified Qwen3.6-35B-A3B; comparators are frontier closed models and large open-weight models listed in tab. 1 §4.1.

ModelOfficeQA Pro (correctness)FinSearchComp (T2&T3 acc.)Finance Agent (Public-77 CB all-pass)APEX-Agents-AA (Pass@1)Workspace-Bench-Lite (rubric pass rate)Source
GPT-5.450.40----33.2652.9published (row)
Claude Opus 4.655.60----33.04--published (row)
Claude Opus 4.746.6†------66.6published (row)
Gemini 3.1 Pro Preview39.10----32.0138.7published (row)
Seed2.1 Pro70.9†------53.0†published (row)
Qwen3.7 Plus55.3961.156.9422.4255.01APEX published; rest by authors
Kimi-K367.9268.865.0237.1360.56by authors
DeepSeek-V4 Flash65.6665.258.8736.4358.08by authors
Qwen3.8 Flash Next64.0470.358.4838.8661.96by authors
GLM-5.263.4165.763.133.8959.34by authors
Kimi-K2.658.6559.154.4929.2854.96by authors
Qwen3.5-397B-A17B47.6250.451.8214.6050.51by authors
Qwen3.6-35B-A3B (baseline)39.641.743.1912.2446.2by authors
Navalia-35B-A3B62.66 (+23.06)55.80 (+14.10)57.10 (+13.91)25.44 (+13.20)59.27 (+13.07)by authors

Gains are in percentage points over the baseline tab. 1 §4.2. Navalia's scores are 62.66, 55.80, 57.10, 25.44 and 59.27 §4.2.

Generalisation. The authors describe the gains on APEX-Agents-AA and Workspace-Bench-Lite as transfer to general Cowork tasks §4.2. Those two benchmarks ran under the Stirrup and Claude Code harnesses respectively app. D §5.

Comparison with larger models. The paper states that Navalia exceeds Qwen3.5-397B-A17B on all five benchmarks, and that it is ahead of Kimi-K2.6 by 4.01 on OfficeQA Pro and by 2.61 on Finance Agent §4.2 tab. 1.

Training data and behaviour (author analysis). Each training task has on average 5.85 supporting documents and 208.69 distractors, with maxima of 15 and 994 fig. 2 §4.3. A training trajectory averages 59.27 tool turns and makes 61.10 calls §4.3. No single capability category exceeds 16.5% of the training data fig. 2. On OfficeQA Pro, accuracy rises from 39.60% to 62.66%. Tasks with every supporting document retrieved go up from 45.4% to 69.8%, and recall over supporting documents climbs from 50.8% to 74.7%. Tool-use turns rise from 39.3 to 47.8 and tool calls from 41.5 to 55.2 fig. 2 §4.3.

Context management ablation. Under a limit of 30 visible turns, a per-turn sliding window compared with FIFO gives the numbers in Table 2 tab. 2 §4.3. Samples per trajectory fall from 40.52 to 3.90 (a 90.4% reduction), and OfficeQA Pro accuracy rises from 49.96% to 51.56% tab. 2 §4.3.

Segmentation strategyTrajectoriesSamplesSamples per trajectoryOfficeQA Pro accuracy (%)
Per-turn sliding window6,826276,59840.5249.96
FIFO6,03123,5233.9051.56

Paper vs code

No official code.

The title page links https://github.com/AllSpark-Research/AgentEnv/tree/main/Navalia. I cloned that repository at commit 1deac75488f1f9ce1caf19b47e216ebc59810be8. Its Navalia folder holds only a README, which says the code and model weights are "undergoing approval for public release" README.md. The repository has no LICENSE file, and its other folders are separate benchmark projects. The paper's contributions list says it plans to release the weights and key pipeline components §1. Because no Navalia code is available, the card records no code, code_commit or code_license. I did not run a GitHub search for other repositories.

Every implementation detail above therefore comes from the paper text, and none has been checked against code. The frameworks slime, dressage and Stirrup are named in the paper app. C app. D but are not part of this repository.

Pitfalls

  • No code, data or weights. The linked repository only says release is pending. An implementer must rebuild the whole pipeline from the text.
  • Task synthesis depends on unnamed models. Fact selection, question writing, independent review, the SFT teacher and the RL "stronger model" are all unnamed §2.3 §2.5 §3.2 §3.3. The paper also gives no task count, pass rate or cost, so the scale of the synthesised data cannot be checked.
  • Advantage and reward are under-specified. The group-relative advantage formula (mean, standard deviation, epsilon) and the reward function beyond "final task reward" are not given §3.3. The clip bounds eps_low and eps_high, the group size G and the learning rate are not stated §3.3.
  • Clipping may never bind. Each rollout batch gets one policy update §3.3 app. C, and the importance ratio is defined against the old policy §3.3. If the batch is consumed in one gradient step, the ratio is 1 and the clip has no effect. The paper does not say whether the batch is split into mini-batches, so this is an inference from the stated setting.
  • Trajectory-level advantage weights by length. The advantage is shared across all segments, and the loss is averaged over valid tokens §3.3. By inference from that formula, longer trajectories carry more tokens and so more gradient weight. The paper presents token-level normalisation as avoiding segment weighting, but does not analyse this.
  • Truncated trajectories. A response that reaches the 64K-token limit keeps its anchor reward for advantages and sampling, while all its tokens are removed from the loss §3.3 app. C. Reward of truncated runs therefore still affects the group, which an implementer should not drop.
  • Turn-level masking criterion is not given. SFT masks low-quality responses but keeps them as context §3.2. Who scores quality and against what rule is not stated.
  • Evaluation context differs from training. OfficeQA Pro uses a sliding window of the last 30 assistant turns at evaluation app. D, while FinSearchComp and Finance Agent keep the full history app. D. Training used FIFO §3.2. Do not assume the training context policy at evaluation time.
  • Table 2 is not the headline setup. Table 2 gives OfficeQA Pro accuracy of 49.96% and 51.56%, but the main result is 62.66 tab. 1 tab. 2. The paper does not state the evaluation configuration behind Table 2. Quote each number with its setup.
  • Finance Agent sizes are worded ambiguously. Section 4.1 reads as if v1.1 and v2 each contribute 77 questions §4.1, while Appendix D says 50 from v1.1 and 27 from v2 make up the 77 app. D. The appendix reading is the consistent one.
  • Most reported gains are author-run. Results use single runs with no seeds or intervals tab. 1. Several baseline and comparator values were run by the authors on their own harness, and the paper says the † values use a different harness tab. 1.
  • Harness-specific claims. The transfer claims on APEX-Agents-AA and Workspace-Bench-Lite depend on the Stirrup and Claude Code harnesses app. D §5. Each benchmark is run under one harness in the paper app. D, and no same-benchmark results under a second harness are reported.
  • Limitations stated by the authors. The single-agent ReAct design has no compression, long-term memory or multi-agent component §5. The fixed window removes earlier turns from view, and the final-reward advantage does not distinguish the contribution of actions at different stages §5.
  • Benchmark sizes are small. OfficeQA Pro uses 133 questions, Public-77 uses 77 questions with 489 criteria, and Workspace-Bench-Lite uses 100 tasks app. D. The paper reports no sampling intervals, so differences of a few points cannot be judged against noise.

Lineage

  • Builds on: the ReAct paradigm (Yao et al., 2022) §1 §4.1; cold-start SFT followed by RL, as in DeepSeekMath (Shao et al., 2024) and DeepSeek-R1 (Guo et al., 2025) §1; FIFO context management attributed to the Seed 2.0 model card (Seed, 2026) §1 §4.1.
  • Compared against: the frontier and open-weight models in tab. 1, and the unmodified Qwen3.6-35B-A3B baseline §4.1.
  • Related benchmarks used for evaluation: OfficeQA Pro (Opsahl-Ong et al., 2026) app. D; FinSearchComp (Hu et al., 2026) app. D; Finance Agent Benchmark (Bigeard et al., 2025) app. D; APEX-Agents (Vidgen et al., 2026) app. D; Workspace-Bench 1.0 (Tang et al., 2026) app. D.
  • Related work on task synthesis for Cowork Agents: OfficeVerse and ClawGym, both contrasted with Navalia's use of corporate financial reports app. B.
  • Superseded by: not stated.

Source and license

Adapted from “Navalia: Advancing Cowork Agents through Verifiable Task Synthesis and Long-Horizon Post-Training” by Hengqi Liu, Ziyuan Liu, Yang Qin, Wanting Zhou, Zichuan Wang, Jiachen Liang, Xu Chu, Shaowei Chen, Zhaokai Luo, Mu Chuan (arXiv:2610.26887), used under CC BY 4.0. Changes: condensed and restructured for implementation, with tables, equations and algorithms taken from the paper and its code, each cited to its source. Not affiliated with or endorsed by the authors.