VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforcement Fine-Tuning

In plain words

Video chat models often cannot say when something happens in a clip, or where an object is across its frames. VideoChat-R1 starts from an existing 7B video model and trains it further by trial and error. For each question the model writes several answers. A few automatic checks score each one: does the time range it names overlap the right part of the clip, is the chosen option right, did it use the required tags. The training then favours the answers that scored better than their siblings for the same question.

The authors train on a small mix of such tasks. They report that the model locates events and follows objects much better, while keeping its general video chat ability. A second idea lets the model name the clip it relied on, then look at that clip again at a higher resolution before answering. These are the authors' results, not independently checked.

TL;DR

• VideoChat-R1 is Qwen2.5-VL-7B trained with GRPO, a critic-free policy-gradient method that scores each sampled answer against its group, using rule-based rewards §3.1 §3.3.
• Rewards: a format check eq. 2, an IoU for time ranges and boxes eq. 3, answer accuracy eq. 4, and an LLM-judged event recall for captions eq. 5 §3.2.
• Joint training covers temporal grounding, object tracking and grounded QA on 18,031 samples §3.3. Author-reported gains over the base model include +31.8 mIoU on Charades-STA and +31.2 overlap on GoT-10k tab. 1.
• The released code contains the grounding, tracking, grounded-QA and classification training. The caption-recall reward and the clue-perception inference are not in the cloned repository.

Problem

The authors say that video understanding needs spatio-temporal perception, such as localising an event in time or tracking an object across frames, which needs long-range temporal association, and that existing video models barely address these abilities §1. They argue that rule-based rewards from verifiable outputs can train this skill, and that reinforcement fine-tuning is more data-efficient than supervised fine-tuning §1 §4. In their tables, supervised fine-tuning for three epochs lowers ActivityNet mIoU from 20.6 to 17.3 compared with one epoch tab. 3.

Method

Base model and sampling. Training starts from Qwen2.5-VL-7B; captioning experiments use Qwen2-VL-7B §4. For each video and question, GRPO samples G candidate responses under the current policy. The paper does not give G; the released scripts use 8 run_grpo_video_tg.sh.

Rewards. Each response gets a score from one or more reward functions. The scores are summed per response §3.3 grpo_trainer_video_tg.py.

1. Format reward: 1 if the output matches the required think-then-answer layout, 0 otherwise §3.2 eq. 2.
2. IoU reward for temporal grounding and object tracking: overlap over union of the predicted and ground-truth intervals or boxes §3.2 eq. 3.
3. Accuracy reward for multiple-choice QA and video quality: 1 if the predicted answer equals the ground truth, 0 otherwise §3.2 eq. 4.
4. Recall reward for captions: an LLM (Qwen2.5-72B in the paper) first splits both captions into events, then judges which ground-truth events the prediction entails. The recall ratio maps to a reward, but the paper does not state the mapping §3.2 eq. 5.

Reward combinations per task §3.3:

• Temporal grounding and object tracking: R_st = format + IoU.
• Multiple-choice QA and video quality: R_qa = format + accuracy.
• Grounded QA (answer plus time evidence): format + IoU + accuracy.
• Captioning: format + recall.

Equations (as written in the paper; R_i is the reward of response i in its group).

A_i = \frac{R_i - \mathrm{mean}(\{R_j\}_{j=1}^{G})}{\mathrm{std}(\{R_j\}_{j=1}^{G})} eq. 1

R_{\mathrm{format}} = \begin{cases} 1, & \text{if output matches format}, \\ 0, & \text{if output doesn't match format}. \end{cases} eq. 2

R_{\mathrm{IoU}} = \frac{|\mathcal{I}_{\mathrm{pred}} \cap \mathcal{I}_{\mathrm{gt}}|}{|\mathcal{I}_{\mathrm{pred}} \cup \mathcal{I}_{\mathrm{gt}}|} eq. 3

R_{\mathrm{accuracy}} = \begin{cases} 0, & \text{if } A_{\mathrm{pred}} \neq A_{\mathrm{gt}}, \\ 1, & \text{if } A_{\mathrm{pred}} = A_{\mathrm{gt}}. \end{cases} eq. 4

R_{\mathrm{recall}} = \mathrm{Recall}_{\mathrm{event}}(C_{\mathrm{pred}}, C_{\mathrm{gt}}) eq. 5

In eq. 3 the sets are time intervals or detection boxes. Eq. 1 is the group-relative advantage: subtract the group mean and divide by the group standard deviation §3.1.

Objective. The paper says the objective "incorporates a KL-divergence term" that keeps the policy close to the original model §3.1; it does not write the objective out. The released trainer uses a per-token policy term weighted by the advantage, minus β times a KL estimate against the reference model. Each response's token losses are averaged over its length, then over the batch grpo_trainer_video_tg.py. See Paper vs code.

# One GRPO update for one video-question pair  [§3.1]
# [code: Videochat-R1/src/open_r1/trainer/grpo_trainer_video_tg.py#L395-L529]
# Input:  video, question, policy, reference model, reward functions
# Output: scalar loss

def grpo_loss(video, question, policy, reference, reward_funcs):
    # Sample G responses at temperature 1  [§3.1]
    # [code: Videochat-R1/src/open_r1/trainer/grpo_trainer_video_tg.py#L303-L309]
    responses = sample_group(policy, video, question, group_size=G)

    # Score every response with each reward and sum them  [§3.3]
    # [code: Videochat-R1/src/open_r1/trainer/grpo_trainer_video_tg.py#L513]
    rewards = sum(reward(responses) for reward in reward_funcs)

    # Normalise rewards inside the group  [eq. 1]
    # [code: Videochat-R1/src/open_r1/trainer/grpo_trainer_video_tg.py#L517-L522]
    advantages = (rewards - mean(rewards)) / (std(rewards) + 1e-4)

    # Per-token KL estimate against the reference model  [§3.1]
    # [code: Videochat-R1/src/open_r1/trainer/grpo_trainer_video_tg.py#L476]
    kl = exp(ref_logp - logp) - (ref_logp - logp) - 1

    # Advantage-weighted policy term minus beta times KL, token by token
    # [code: Videochat-R1/src/open_r1/trainer/grpo_trainer_video_tg.py#L526-L527]
    per_token = -(advantages * logp - beta * kl)

    # Average over each response's tokens, then over the batch
    # [code: Videochat-R1/src/open_r1/trainer/grpo_trainer_video_tg.py#L529]
    return mean(sum_tokens(per_token) / num_tokens)

Grounding IoU reward (temporal grounding; code path for object tracking differs, see Paper vs code):

# Overlap-over-union of a predicted time range  [eq. 3]
# [code: Videochat-R1/src/open_r1/grpo_tg.py#L97-L138]
# Input:  response text, ground-truth start and end in seconds

def iou_reward(response, gt_start, gt_end):
    # Read the last <answer> block; a missing or invalid range scores 0
    # [code: Videochat-R1/src/open_r1/grpo_tg.py#L73-L95]
    pred = parse_last_answer(response)
    if pred is None:
        return 0.0
    pred_start, pred_end = pred
    # Length of the overlap  [eq. 3]
    inter = max(0, min(pred_end, gt_end) - max(pred_start, gt_start))
    # Length of the union  [eq. 3]
    union = max(pred_end, gt_end) - min(pred_start, gt_start)
    # An empty union is scored as 1.0 in the released code
    # [code: Videochat-R1/src/open_r1/grpo_tg.py#L118-L121]
    return inter / union if union > 0 else 1.0

Format reward (temporal grounding) §3.2 eq. 2 grpo_tg.py:

# 1 only if the whole stripped response is think, then answer  [eq. 2]
# Input:  response text

def format_reward(response):
    # The pattern must match the entire response, not a substring
    # [code: Videochat-R1/src/open_r1/grpo_tg.py#L143-L146]
    matched = fullmatch(
        r"<think>.*?</think>\s*<answer>.*?</answer>",
        strip(response),
        DOTALL,
    )
    return 1.0 if matched else 0.0

Temporal Clue-driven Reasoning (inference; alg. 1 §3.3). The paper's Algorithm 1 is restated here in our words:

# Two-pass answer with clue-based re-watching  [alg. 1] [§3.3]
# Input:  low-resolution video, question, resolution and fps boost factors
# Output: refined answer

def clue_driven_answer(model, video_low, question, res_boost, fps_boost):
    # First pass returns an answer and the time clues behind it  [alg. 1]
    first_answer, clues = model(video_low, question)
    # Cut the clue segments out of the low-resolution video  [alg. 1]
    segments = select_segments(video_low, clues)
    # Re-encode the segments at a higher resolution and frame rate  [alg. 1]
    video_high = upsample(segments, res_boost, fps_boost)
    # Second pass gives the final answer  [alg. 1]
    return model(video_high, question)

Training data §3.3. Temporal grounding: Charades-STA training split, 5,338 samples. Object tracking: GoT-10k, 9,335 samples. Grounded QA: NExTGQA validation split, 3,358 samples. Captioning: FIBER-1k, 1,000 samples. Video quality: VidTAB quality task in a 100-shot setting, 200 samples. The joint model trains on the first three sets, 18,031 samples in total. The released joint script loads the same three sets run_grpo_video_task.sh and grpo_tasks.py.

Figures (described, not reproduced here). fig. 1 gives an overview of the training and the chat system. fig. 2 shows temporal grounding examples in which the model's time interval becomes more accurate after thinking. fig. 3 shows video QA examples where the model also gives the time period that supports its answer.

Hyperparameters

Only settings the paper or released code state are listed. "not stated" means neither source sets the value.

Results

Author-reported. Setup: Qwen2.5-VL-7B base unless noted; the authors evaluate with a fixed number of input pixels, so their baseline is slightly below the original report tab. 1. VideoMME uses its short subset only §4. "VideoChat-R1-thinking" uses the reasoning output at test time.

Main results tab. 1

• = the model overfitted and cannot answer properly tab. 1.

Co-training across tasks tab. 2. Each GRPO row is one training run; "STA" is Charades-STA, "GQA" is NExTGQA grounded QA, "GoT" is object tracking.

Temporal grounding ablation tab. 3. "Think" and "Answer" give the prompt tags used in training and at test time. Deltas in the paper are against the matching baseline row.

Table 3 values are rendered with the authors' deltas in parentheses. "N/A*" = overfitted.

Object tracking tab. 4 (8 frames as input for training and evaluation tab. 4).

Grounded and multiple-choice QA tab. 5. "Tags" lists the tags the prompt asked for: think, answer, and glue (time evidence).

Captioning and video quality tab. 6.

Recall judge ablation tab. 7. Both judges give nearly identical results §4.

Clue-perception re-watching tab. 8. Long-video results; average durations are 1010 s (VideoMME) and 473 s (LongVideoBench).

The paper notes that its training videos are all under one minute, which it offers as the reason the base model gains little on long videos without clues §4.

Paper vs code

Code is from the official repository at commit 16f03aa4bb470ade41b34bf09272c5466161cc5f. The paper's author block links the same repository, and the README says it releases the paper's code README.md.

• Objective. The paper states only that a KL term is used §3.1. The code uses an unclipped policy term weighted by the advantage, minus β times a KL estimate. Its comment says this form assumes one update per batch grpo_trainer_video_tg.py grpo_trainer_video_tg.py.
• Advantage. Eq. 1 has no epsilon eq. 1; the code divides by the sample standard deviation plus 1e-4 grpo_trainer_video_tg.py.
• Reward sum. The paper sums the reward terms per task §3.3; the code sums them with equal weight grpo_trainer_video_tg.py.
• Tracking reward. The paper defines IoU for intervals or boxes §3.2 eq. 3 but not how a sequence of boxes is scored. The code averages per-box IoU over the predicted boxes and returns 0 if the box count differs from the ground truth grpo_tasks.py grpo_tasks.py.
• Grounded-QA IoU. The paper does not say how multiple evidence intervals combine. The code merges overlapping intervals in both prediction and ground truth before computing the IoU grpo_tasks.py grpo_tasks.py.
• Format. The grounding format check requires the entire stripped response to match, with no extra text outside the tags grpo_tg.py. The grounded-QA check additionally requires a glue tag grpo_tasks.py. The paper says only that it checks format §3.2 eq. 2.
• Accuracy. Eq. 4 is an exact match eq. 4. The grounded-QA code extracts the first capital letter A to G from the answer after stripping a fixed list of phrases; long answers with no such letter score 0 grpo_tasks.py. The classification code compares strings case-insensitively grpo_cls.py.
• Recall reward. Eq. 5 is given without the mapping from recall to reward eq. 5 §3.2, and the caption training code is absent from the clone. The README points to a separate VideoChat-R1-caption release README.md; this was not checked.
• Epochs. Table 3 reports three-epoch grounding runs tab. 3. No script sets three epochs for grounding. The one file named _3e, for GQA with no thinking, sets one epoch run_grpo_video_gqa_nothink_3e.sh.
• Frames. The tracking table states 8 frames tab. 4. The released scripts set no frame count, so frame sampling falls back to the helper's defaults of 2 fps and 4 to 768 frames, if that path is used my_qwen_utils.py my_qwen_utils.py.
• Clue-perception. The clue-based inference of alg. 1 is not in the clone. Searching the Python sources for "clue" finds nothing in the Videochat-R1 directory. Δres and Δfps are not stated alg. 1.
• Prompt. The grounding prompt asks for time stamps inside <timestep> tags grpo_trainer_video_tg.py. The paper does not mention these tags, and no reward reads them.
• Joint routing. The joint trainer applies only the reward functions for the sample's task type grpo_tasks_trainer.py.
• Sample counts. 5,338 + 9,335 + 3,358 = 18,031 matches §3.3. The code loads these three sets grpo_tasks.py.

Code license. The repository has no LICENSE file for the authors' code. The only LICENSE file is inside the bundled lmms-eval fork LICENSE. Because the authors' code has no stated license, nothing is quoted from it; the card cites paths and lines only.

Pitfalls

• Format and IoU interact. The IoU reward reads the last <answer> block even when the format check fails grpo_tg.py. A malformed response can therefore keep its IoU but lose its format point, so sums are not all-or-nothing.
• Strict time parsing. The parser accepts only "X to Y" or "X and Y" with decimal numbers that start with a digit. It rejects ".5", negative numbers and start greater than end, and each rejection scores 0 grpo_tg.py. Reimplementations must match this or their rewards will differ.
• Empty union. The released code scores an empty union as IoU 1.0 grpo_tg.py. This only happens when prediction and ground truth coincide at a single point, but it is a silent choice.
• Zero-variance groups. When all G rewards are equal, the advantages are 0 and the step only applies the KL term. Dividing by the standard deviation plus 1e-4 keeps this numerically safe grpo_trainer_video_tg.py.
• Reference model. With no separate reference model, the code runs the reference pass with the adapter disabled grpo_trainer_video_tg.py. Read this when using LoRA; the paper does not discuss it.
• Answer extraction. Because the grounded-QA extractor takes the first capital A to G anywhere in the answer, a leading word such as "Answer: B" yields "A" grpo_tasks.py. Test with real outputs.
• Tracking box counts. Responses with the wrong number of boxes score 0, and an exception also gives 0 grpo_tasks.py. The prompt asks for eight boxes grpo_tasks_trainer.py.
• Baseline choice. The IoU gains in the headline and the ablations use different baselines. The +31.8 headline is against the base row without thinking (29.0). The +31.2 in tab. 3 is against the thinking-test baseline (28.1) tab. 1 tab. 3.
• Thinking helps QA more than grounding. The paper finds that reasoning output does not help temporal grounding but does help QA and VideoMME §4. Choose the output mode by task.
• Data access. The scripts read videos from internal S3 paths (p2:s3://...) run_grpo_video_tg.sh. The Charades, ActivityNet and VideoMME videos are not in the repository, and their licenses were not checked here.
• Chain-of-thought is not a clear win. The paper states that its chain-of-thought outputs were sometimes worse than direct output on perception tasks, and that the training data cannot yet teach effective video reasoning chains §4.
• Figures are described, not reproduced, per the project's figure rule.

Lineage

• Builds on: GRPO, DeepSeekMath, arXiv:2402.03300 §3.1 (not carded).
• Builds on: Qwen2.5-VL-7B, arXiv:2502.13923 §4 (not carded). Captioning uses Qwen2-VL-7B, arXiv:2409.12191 §4 (not carded).
• Related RL-for-video work named in the paper: Video-R1, arXiv:2503.21776, and TimeZero, arXiv:2503.13377 §2 (not carded). Nothing from their content is used here.
• Compared against: the Qwen2.5-VL-7B baseline and single-task SFT models tab. 1. The vision-expert rows (FlashVTG, InternVideo2-6B, SG-DETR) appear in tab. 3 as cited numbers only.
• Superseded by: not stated in the wiki.