Paper · 2026-10-13 · cs.AI
Navalia: Advancing Cowork Agents through Verifiable Task Synthesis and Long-Horizon Post-Training
Navalia builds finance-report analysis tasks whose evidence, multi-step computation chain and reference answer pass independent LLM review and programmatic checks. The verified tasks train Qwen3.6-35B-A3B by ReAct-trajectory SFT, then RL with trajectory advantages shared across FIFO segments. Author-reported: OfficeQA Pro 39.60 to 62.66 and gains on all five benchmarks. No code released.
LLM agentSynthetic training dataReinforcement learning with verifiable rewards (RLVR)Policy gradientagentstool-usesynthetic-training-datareinforcement-learningfinancecontext-management
AI-compiled by claude-haiku-5-5 on 2026-11-21, not yet reviewed by a human. Every claim cites its source; check it when a detail matters.
In plain words
Teaching an AI assistant to answer questions about company financial reports needs many practice questions. Writing them by hand is slow, and asking a language model to make them up often gives questions with muddled numbers or answers that are simply wrong. This paper builds the practice questions from the numbers themselves. It pulls the figures out of each report, picks the ones a question needs, and chains simple calculations on top of them. Only questions that a second model can solve the same way, and that pass automatic checks, are kept.
The assistant is then trained in two steps. First it copies good worked examples, where it searches documents, runs calculations and writes an answer. Then it practises on the questions it sometimes gets wrong, and it is rewarded only for the final answer. Its earlier steps share in that final reward, so the search and reading steps that led to a right answer are reinforced too. The authors report large gains on a finance question set and smaller gains on office-work benchmarks. These numbers come from the authors' own runs, and the authors have not yet released the code or model weights.
TL;DR
Navalia is a pipeline for "Cowork Agents" (assistants that plan, call tools and finish multi-step work) on corporate financial reports §1. It has two parts: verifiable task synthesis, which produces questions whose supporting facts, computation chain and reference answer are recorded and checked §2, and a two-stage post-training recipe of cold-start SFT on teacher trajectories followed by RL §3. Applied to Qwen3.6-35B-A3B, the authors report gains on all five benchmarks, from 39.60 to 62.66 on OfficeQA Pro tab. 1. No code or weights are released, and the task synthesis depends on LLMs the paper does not name.
Problem
Improving cross-document evidence integration, multi-step numerical reasoning and long-horizon execution needs many training tasks that are analytically deep and verifiable §1. Expert authoring is costly and does not scale. Free LLM generation from raw documents can mix inconsistent metric definitions or reporting scopes, make arithmetic errors, or produce steps that do not depend on each other §1. Public documentation of the full data-to-post-training path for Cowork Agents was described by the authors as limited §1.
Assumptions: the tasks are built only from publicly disclosed corporate reports §1 §4.1. The evaluation uses fixed harnesses per benchmark §4.1 app. D.
Method
Notation. A task is a tuple of a question q, a reference answer y, an interactive environment C, supporting facts F+, a computation chain G, anchors a (companies, periods, metrics, scope) and operator parameters θ §2 eq. 1.
1. Dual-view representation. Each report d is mapped to two views: a Markdown document view r(d) that the agent browses with tools, and a fact view F(d) used for construction and checking §2.1 eq. 2. Each fact keeps its number and unit, its place in the source, the table around it, the period it covers, how its metric is defined, which entity and scope it applies to, and a version tag. Facts with incomplete numeric, period or unit information are dropped §2.1. The facts of all documents form one global database F eq. 3 §2.1.
2. Operator-driven design and environment. A predefined set O holds analytical operators and composite templates. Its categories are aggregation over periods, trend comparison, statistics, regression, numeric transforms and cross-company analysis §2.2. Composite templates chain compatible operators into multi-step tasks §2.2. The sampling order is: operator or template o, then anchors a conditioned on o and F, then parameters θ conditioned on o §2.2 eq. 4. The configuration must cover the required periods, peer data and every step's inputs; otherwise it is resampled §2.2. The candidate documents D_cand are joined by distractors drawn from the same companies' other reporting periods and from peers §2.2.
3. Semantic selection of supporting facts. The method does not pre-filter facts with keywords or fixed metric rules. An LLM reads every fact of every candidate document and judges whether it meets the operator, anchors and parameters, including period, metric definition, scope, unit and context §2.3. When a document's fact set is large, a within-document preselection runs first §2.3. A reasoning model then selects F+ across documents from the aggregated pool §2.3 eq. 5. Each selected fact keeps its identifier, location, period, unit, metric definition and scope §2.3. If F+ misses raw data required by the computation, or if its definitions or scopes do not match the task, the instance is discarded and resampled §2.3.
4. Executable computation chains. Simple single-step tasks (lookup, sum, rate of change, average) are answered directly by an LLM from F+ §2.4. Complex operators (regression, forecasting, statistics, numeric transforms) are split: the LLM extracts values and arranges them to the operator's input format, and a predefined numerical routine computes the result deterministically §2.4. For a linear chain of K steps the recurrence is z_1 = o_1(x_1; θ_1) and z_k = o_k(z_{k-1}, x_k; θ_k), where x_k is raw data and z_k is the step output §2.4 eq. 6. Later steps must use earlier outputs; a chain without real dependencies is excluded §2.4. Instances are dropped for incomplete data, type mismatches or failed evaluation §2.4. The final output is converted to the required unit, precision and format to become y §2.4.
5. Question generation and quality control. The question is generated from the chain and y. It must state every answer-affecting constraint (companies, periods, metrics, scope, algorithm parameters) and any unit, precision or output format §2.5. Single-step questions state the objective and scope; multi-step questions follow the chain's order and ask only for the final result §2.5. An independent review model answers the question from q and F+ alone, without the generator's reasoning, and compares its result with y §2.5. Tasks that pass are then checked programmatically: every non-final step's output must feed a later step, and the periods named in the question must match the supporting facts §2.5. Only tasks passing both stages are kept §2.5.
# Build one candidate task instance [§2] [eq. 1]
# docs: parsed reports; facts: global fact database [eq. 3]
def build_task(operators, docs, facts):
# Sample operator or template, then anchors, then parameters [§2.2] [eq. 4]
operator = sample_operator(operators)
anchors = sample_anchors(operator, facts)
params = sample_params(operator)
# Resample when document coverage fails [§2.2]
if not covers_requirements(operator, anchors, docs):
return None
# Candidates plus distractors from other periods and peer companies [§2.2]
candidates, distractors = pick_documents(anchors, docs)
# Read all facts of each candidate; preselect per document if large [§2.3]
pool = preselect_per_document(candidates, operator, anchors, params)
# Reasoning model selects the supporting facts F+ [§2.3] [eq. 5]
support = select_facts(operator, anchors, params, pool)
# Resample if F+ misses raw data or mismatches definitions [§2.3]
if not sufficient(support, operator, anchors):
return None
# Chain: LLM for simple steps, numeric routine for complex [§2.4] [eq. 6]
chain = build_chain(operator, support, params)
answer = evaluate_chain(chain)
# Write the question with every answer-affecting constraint [§2.5]
question = write_question(chain, answer, anchors, params)
# Independent review answers from q and F+ only; compare with answer [§2.5]
if not review_agrees(question, support, answer):
return None
# Programmatic checks: outputs used, periods match [§2.5]
if not programmatic_checks(chain, question, support):
return None
return (question, answer, chain, support, anchors, params)6. Trajectory collection and cold-start SFT. A strong teacher model (not named) solves the verified tasks in the document environment with a lightweight ReAct harness and FIFO context management, and the complete trajectories are recorded §3.2 §1. Trajectories are kept only when the final answer is right, their tool calls are valid, and they reach a minimum turn count §3.2. They are grouped into capability pools by operator or template and sampled evenly across pools §3.2. Turn-level quality assessment masks the loss on low-quality responses but keeps them as context §3.2.
Training samples follow the FIFO scheme and advance 10 turns at a time. Each sample holds the task instructions, up to 20 earlier turns and up to 10 new turns §3.2. The loss is cross-entropy on the new turns that pass quality assessment, normalised by the number of supervised tokens. The supervised intervals do not overlap, so each response is trained in exactly one sample §3.2.
# Cut one trajectory into FIFO SFT samples [§3.2] [app. C]
# turns: ordered list of interaction turns, supervised per block
def fifo_samples(instructions, turns, history=20, stride=10):
samples = []
# Disjoint supervised blocks of `stride` turns [§3.2]
for start in range(0, len(turns), stride):
# Visible history: up to `history` turns before the block [§3.2]
context = turns[max(0, start - history):start]
# Loss only on the new turns of this block [§3.2]
new_turns = turns[start:start + stride]
samples.append((instructions, context, new_turns))
return samples7. Long-horizon RL. The policy starts from the cold-start SFT model and uses the same harness and FIFO context §3.3. Before RL, tasks are selected by the SFT model's rollout outcomes. Tasks with both successes and failures are kept, and all-success tasks are dropped. All-failure tasks are kept only if a stronger model solves them §3.3.
For each question the policy samples G trajectories, each made of J_g segments §3.3 eq. 7. The final segment, the anchor, gives the reward R_g of the whole trajectory §3.3. Group-relative advantages are computed from anchor rewards only and then shared by all segments of the same trajectory, so earlier retrieval and analysis segments receive the task outcome §3.3. Dynamic sampling keeps only task groups whose anchor rewards vary §3.3. The loss is applied only to newly generated responses in each segment; reused history and environment observations are masked §3.3. If a single response hits the generation limit, the trajectory keeps its anchor reward for advantage and sampling statistics, but all its tokens are removed from the loss §3.3.
The rollouts are on-policy, and each rollout batch is used for one policy update §3.3 app. C. The objective is a clipped token-level surrogate with importance ratio r_u(θ) and a shared advantage Â_u, normalised by the number of valid tokens |M| §3.3 eq. 8.
# Choose RL tasks from cold-start rollouts [§3.3]
def select_rl_tasks(sft_policy, candidate_tasks, stronger_model):
kept = []
for task in candidate_tasks:
outcomes = run_rollouts(sft_policy, task)
# Keep mixed-outcome tasks; all-success tasks are dropped [§3.3]
if any_success(outcomes) and any_failure(outcomes):
kept.append(task)
# Keep all-failure tasks only if a stronger model solves them [§3.3]
elif all_failure(outcomes):
if stronger_solves(stronger_model, task):
kept.append(task)
return kept# One RL update on a batch of tasks [§3.3] [eq. 7] [eq. 8]
# eps_low, eps_high: clip bounds (values not stated in the paper)
def rl_update(policy, old_policy, tasks, G, eps_low, eps_high):
# Sample G trajectories per question [§3.3] [eq. 7]
groups = [sample_group(policy, task, G) for task in tasks]
# Keep only groups whose anchor rewards differ [§3.3]
groups = [g for g in groups if reward_variance(g) > 0]
# Group-relative advantage from anchor rewards [§3.3]
# (normalisation not stated)
advantages = group_advantages(anchor_rewards(groups))
# Share each trajectory's advantage with all its segments [§3.3]
token_adv, token_logp, token_old_logp, token_mask = flatten_segments(
groups, advantages, policy, old_policy)
# Per-token importance ratio, new over old policy [eq. 8]
ratio = exp(token_logp - token_old_logp)
clipped = clip(ratio, 1 - eps_low, 1 + eps_high)
per_token = minimum(ratio * token_adv, clipped * token_adv)
# Token-level normalisation over all valid tokens |M| [§3.3] [eq. 8]
loss = -sum(per_token * token_mask) / count(token_mask)
return lossHarness and context. The ReAct harness is described as lightweight and single-agent §3.1 §5. The FIFO window used in training and the RL rollouts holds at most 30 interaction turns and advances by a stride of 10 §4.1 app. C. The paper does not describe the system prompt or tool interfaces.
Hyperparameters
Settings stated in the paper; everything not listed is "not stated". The training values come from app. C; the evaluation values come from app. D.
| Name | Value | Where | Notes |
|---|---|---|---|
| initial model | Qwen3.6-35B-A3B | §4.1 app. C | base context length 262,144 tokens §4.1 |
| SFT framework | slime | app. C | |
| SFT epochs | 2 | app. C | |
| SFT global batch size | 128 | app. C | |
| SFT max training sequence length | 256K tokens | app. C | |
| RL framework | dressage | app. C | on-policy training |
| RL updates per rollout batch | 1 | §3.3 app. C | "single policy update" |
| max tokens per model response | 64K | app. C | |
| max interaction turns per trajectory | 200 | app. C | |
| FIFO visible window | up to 30 turns | §4.1 app. C | |
| FIFO stride | 10 turns | §3.2 §4.1 app. C | |
| SFT history per sample | up to 20 turns | §3.2 | |
| SFT new turns per sample | up to 10 turns | §3.2 | |
| SFT minimum interaction turns (filter) | not stated | §3.2 | threshold not given |
| SFT turn-level quality criterion | not stated | §3.2 | who judges and how is not given |
| learning rate, optimiser, schedule | not stated | ||
| RL group size G | not stated | §3.3 | |
| RL clip bounds eps_low, eps_high | not stated | §3.3 | |
| RL learning rate, batch size, steps | not stated | ||
| RL advantage normalisation | not stated | §3.3 | |
| reward function (beyond anchor-based final reward) | not stated | §3.3 | |
| dynamic sampling rule | keep groups with nonzero reward variance | §3.3 | |
| teacher model (SFT trajectories) | not stated | §3.2 | "a strong teacher model" |
| stronger model (RL task filter) | not stated | §3.3 | |
| reasoning model (fact selection) | not stated | §2.3 | |
| review model (independent check) | not stated | §2.5 | |
| within-document preselection threshold | not stated | §2.3 | "when the candidate fact set is large" |
| operator set size and sampling distribution | not stated | §2.2 | categories given, counts not |
| SFT trajectory count (final mix) | not stated | §3.2 | Table 2 reports 6,031 for the FIFO ablation tab. 2 |
| OfficeQA Pro evaluation | Full Corpus, 133 questions, max 200 steps | app. D | sliding window of the last 30 assistant turns |
| FinSearchComp evaluation | T2 and T3, max 200 turns | app. D | full history kept, no truncation |
| Finance Agent evaluation | Public-77 (50 from v1.1 + 27 from v2), 489 criteria | app. D | LLM judge with thinking mode |
| APEX-Agents-AA evaluation | 452 of 480 tasks, Stirrup harness, Pass@1 | app. D | |
| Workspace-Bench-Lite evaluation | 100 tasks, Claude Code harness, rubric pass rate | app. D | |
| evaluation sampling settings and seeds | not stated | app. D | |
| training-data statistics | 5.85 supporting documents, 208.69 distractors per task (maxima 15 and 994) | fig. 2 §4.3 | which training set is not stated |
| training-trajectory size | 59.27 tool-use turns, 61.10 tool calls on average | §4.3 |
Results
Author-reported throughout, on the five benchmarks listed in app. D. The baseline Qwen3.6-35B-A3B and Navalia were run by the authors under a unified setup within each benchmark §4.1 app. D. Table 1 marks published results with * (the whole row when the marker follows the model name, one cell when it follows a score). Scores marked † used a different document representation or harness and are for reference only tab. 1. Unmarked values were obtained by the authors. The paper reports single values, with no seeds, run counts or intervals for the main table tab. 1.
Setup: the baseline is the unmodified Qwen3.6-35B-A3B; comparators are frontier closed models and large open-weight models listed in tab. 1 §4.1.
| Model | OfficeQA Pro (correctness) | FinSearchComp (T2&T3 acc.) | Finance Agent (Public-77 CB all-pass) | APEX-Agents-AA (Pass@1) | Workspace-Bench-Lite (rubric pass rate) | Source |
|---|---|---|---|---|---|---|
| GPT-5.4 | 50.40 | -- | -- | 33.26 | 52.9 | published (row) |
| Claude Opus 4.6 | 55.60 | -- | -- | 33.04 | -- | published (row) |
| Claude Opus 4.7 | 46.6† | -- | -- | -- | 66.6 | published (row) |
| Gemini 3.1 Pro Preview | 39.10 | -- | -- | 32.01 | 38.7 | published (row) |
| Seed2.1 Pro | 70.9† | -- | -- | -- | 53.0† | published (row) |
| Qwen3.7 Plus | 55.39 | 61.1 | 56.94 | 22.42 | 55.01 | APEX published; rest by authors |
| Kimi-K3 | 67.92 | 68.8 | 65.02 | 37.13 | 60.56 | by authors |
| DeepSeek-V4 Flash | 65.66 | 65.2 | 58.87 | 36.43 | 58.08 | by authors |
| Qwen3.8 Flash Next | 64.04 | 70.3 | 58.48 | 38.86 | 61.96 | by authors |
| GLM-5.2 | 63.41 | 65.7 | 63.1 | 33.89 | 59.34 | by authors |
| Kimi-K2.6 | 58.65 | 59.1 | 54.49 | 29.28 | 54.96 | by authors |
| Qwen3.5-397B-A17B | 47.62 | 50.4 | 51.82 | 14.60 | 50.51 | by authors |
| Qwen3.6-35B-A3B (baseline) | 39.6 | 41.7 | 43.19 | 12.24 | 46.2 | by authors |
| Navalia-35B-A3B | 62.66 (+23.06) | 55.80 (+14.10) | 57.10 (+13.91) | 25.44 (+13.20) | 59.27 (+13.07) | by authors |
Gains are in percentage points over the baseline tab. 1 §4.2. Navalia's scores are 62.66, 55.80, 57.10, 25.44 and 59.27 §4.2.
Generalisation. The authors describe the gains on APEX-Agents-AA and Workspace-Bench-Lite as transfer to general Cowork tasks §4.2. Those two benchmarks ran under the Stirrup and Claude Code harnesses respectively app. D §5.
Comparison with larger models. The paper states that Navalia exceeds Qwen3.5-397B-A17B on all five benchmarks, and that it is ahead of Kimi-K2.6 by 4.01 on OfficeQA Pro and by 2.61 on Finance Agent §4.2 tab. 1.
Training data and behaviour (author analysis). Each training task has on average 5.85 supporting documents and 208.69 distractors, with maxima of 15 and 994 fig. 2 §4.3. A training trajectory averages 59.27 tool turns and makes 61.10 calls §4.3. No single capability category exceeds 16.5% of the training data fig. 2. On OfficeQA Pro, accuracy rises from 39.60% to 62.66%. Tasks with every supporting document retrieved go up from 45.4% to 69.8%, and recall over supporting documents climbs from 50.8% to 74.7%. Tool-use turns rise from 39.3 to 47.8 and tool calls from 41.5 to 55.2 fig. 2 §4.3.
Context management ablation. Under a limit of 30 visible turns, a per-turn sliding window compared with FIFO gives the numbers in Table 2 tab. 2 §4.3. Samples per trajectory fall from 40.52 to 3.90 (a 90.4% reduction), and OfficeQA Pro accuracy rises from 49.96% to 51.56% tab. 2 §4.3.
| Segmentation strategy | Trajectories | Samples | Samples per trajectory | OfficeQA Pro accuracy (%) |
|---|---|---|---|---|
| Per-turn sliding window | 6,826 | 276,598 | 40.52 | 49.96 |
| FIFO | 6,031 | 23,523 | 3.90 | 51.56 |
Paper vs code
No official code.
The title page links https://github.com/AllSpark-Research/AgentEnv/tree/main/Navalia.
I cloned that repository at commit 1deac75488f1f9ce1caf19b47e216ebc59810be8.
Its Navalia folder holds only a README, which says the code and model weights
are "undergoing approval for public release" README.md. The
repository has no LICENSE file, and its other folders are separate benchmark
projects. The paper's contributions list says it plans to release the weights and
key pipeline components §1. Because no Navalia code is available, the card
records no code, code_commit or code_license. I did not run a GitHub search
for other repositories.
Every implementation detail above therefore comes from the paper text, and none has been checked against code. The frameworks slime, dressage and Stirrup are named in the paper app. C app. D but are not part of this repository.
Pitfalls
- No code, data or weights. The linked repository only says release is pending. An implementer must rebuild the whole pipeline from the text.
- Task synthesis depends on unnamed models. Fact selection, question writing, independent review, the SFT teacher and the RL "stronger model" are all unnamed §2.3 §2.5 §3.2 §3.3. The paper also gives no task count, pass rate or cost, so the scale of the synthesised data cannot be checked.
- Advantage and reward are under-specified. The group-relative advantage formula (mean, standard deviation, epsilon) and the reward function beyond "final task reward" are not given §3.3. The clip bounds eps_low and eps_high, the group size G and the learning rate are not stated §3.3.
- Clipping may never bind. Each rollout batch gets one policy update §3.3 app. C, and the importance ratio is defined against the old policy §3.3. If the batch is consumed in one gradient step, the ratio is 1 and the clip has no effect. The paper does not say whether the batch is split into mini-batches, so this is an inference from the stated setting.
- Trajectory-level advantage weights by length. The advantage is shared across all segments, and the loss is averaged over valid tokens §3.3. By inference from that formula, longer trajectories carry more tokens and so more gradient weight. The paper presents token-level normalisation as avoiding segment weighting, but does not analyse this.
- Truncated trajectories. A response that reaches the 64K-token limit keeps its anchor reward for advantages and sampling, while all its tokens are removed from the loss §3.3 app. C. Reward of truncated runs therefore still affects the group, which an implementer should not drop.
- Turn-level masking criterion is not given. SFT masks low-quality responses but keeps them as context §3.2. Who scores quality and against what rule is not stated.
- Evaluation context differs from training. OfficeQA Pro uses a sliding window of the last 30 assistant turns at evaluation app. D, while FinSearchComp and Finance Agent keep the full history app. D. Training used FIFO §3.2. Do not assume the training context policy at evaluation time.
- Table 2 is not the headline setup. Table 2 gives OfficeQA Pro accuracy of 49.96% and 51.56%, but the main result is 62.66 tab. 1 tab. 2. The paper does not state the evaluation configuration behind Table 2. Quote each number with its setup.
- Finance Agent sizes are worded ambiguously. Section 4.1 reads as if v1.1 and v2 each contribute 77 questions §4.1, while Appendix D says 50 from v1.1 and 27 from v2 make up the 77 app. D. The appendix reading is the consistent one.
- Most reported gains are author-run. Results use single runs with no seeds or intervals tab. 1. Several baseline and comparator values were run by the authors on their own harness, and the paper says the † values use a different harness tab. 1.
- Harness-specific claims. The transfer claims on APEX-Agents-AA and Workspace-Bench-Lite depend on the Stirrup and Claude Code harnesses app. D §5. Each benchmark is run under one harness in the paper app. D, and no same-benchmark results under a second harness are reported.
- Limitations stated by the authors. The single-agent ReAct design has no compression, long-term memory or multi-agent component §5. The fixed window removes earlier turns from view, and the final-reward advantage does not distinguish the contribution of actions at different stages §5.
- Benchmark sizes are small. OfficeQA Pro uses 133 questions, Public-77 uses 77 questions with 489 criteria, and Workspace-Bench-Lite uses 100 tasks app. D. The paper reports no sampling intervals, so differences of a few points cannot be judged against noise.
Lineage
- Builds on: the ReAct paradigm (Yao et al., 2022) §1 §4.1; cold-start SFT followed by RL, as in DeepSeekMath (Shao et al., 2024) and DeepSeek-R1 (Guo et al., 2025) §1; FIFO context management attributed to the Seed 2.0 model card (Seed, 2026) §1 §4.1.
- Compared against: the frontier and open-weight models in tab. 1, and the unmodified Qwen3.6-35B-A3B baseline §4.1.
- Related benchmarks used for evaluation: OfficeQA Pro (Opsahl-Ong et al., 2026) app. D; FinSearchComp (Hu et al., 2026) app. D; Finance Agent Benchmark (Bigeard et al., 2025) app. D; APEX-Agents (Vidgen et al., 2026) app. D; Workspace-Bench 1.0 (Tang et al., 2026) app. D.
- Related work on task synthesis for Cowork Agents: OfficeVerse and ClawGym, both contrasted with Navalia's use of corporate financial reports app. B.
- Superseded by: not stated.
Source and license
Adapted from “Navalia: Advancing Cowork Agents through Verifiable Task Synthesis and Long-Horizon Post-Training” by Hengqi Liu, Ziyuan Liu, Yang Qin, Wanting Zhou, Zichuan Wang, Jiachen Liang, Xu Chu, Shaowei Chen, Zhaokai Luo, Mu Chuan (arXiv:2610.26887), used under CC BY 4.0. Changes: condensed and restructured for implementation, with tables, equations and algorithms taken from the paper and its code, each cited to its source. Not affiliated with or endorsed by the authors.