---
kind: concept
slug: multi-view-representation
title: Multi-view representation
summary: >-
  Represent a 3D scene by several fixed rotated views, give each object a shared feature plus a per-view position code, then average the fused outputs so the result does not depend on the starting view. One card so far, for 3D visual grounding. All numbers are author-reported on Nr3D and Sr3D.
tags: [multi-view, point-cloud, 3d-visual-grounding, transformer]
papers: ["2204.02174"]
related: [attention-mechanism, bev-representation]
status: draft
updated_at: 2026-10-19
---

## Overview

A multi-view representation describes one scene from several viewpoints and
combines the per-view outputs. The reason, in the one card so far: a spatial
phrase such as "left of the bed" depends on where the viewer stands, so a
model fixed to one view can fail when the view changes [arxiv:2204.02174 §1].
The combination is order-independent, so the score for each object does not
depend on which view came first [arxiv:2204.02174 §3.4].

## Variants

Only one variant is covered so far. No other card lists this slug. For a
different view-based scene representation, see
[Bird's-eye-view (BEV) representation](/concepts/bev-representation).

### Rotated views, shared object features, averaged fusion

The scene is rotated about the vertical Z axis into N equal-angle views,
θ_j = 2π(j−1)/N [arxiv:2204.02174 §3.1]. Each candidate object's point-cloud
feature is computed once and shared by all views. A per-view position code,
made from the rotated box centre and size, is added to it
[arxiv:2204.02174 §3.2]. One transformer decoder, shared across views, fuses
each view's object features with the query's word features
[arxiv:2204.02174 §3.3]. The N outputs are averaged for each object before a
two-layer head scores it [arxiv:2204.02174 §3.4]. Training also rotates the
whole scene by one of the N angles [arxiv:2204.02174 §4.4.1]. The main model
uses N = 4 [arxiv:2204.02174 §4.2]. Card:
[Multi-View Transformer for 3D Visual Grounding](/papers/2204.02174).

## When to use which

The choices below depend on constraints. All numbers are the paper's own
ablations on Nr3D, so they do not show what happens on other tasks.

| Situation | Use | Why |
|---|---|---|
| Single view is too weak for viewpoint-dependent phrases | N views, averaged | Overall 55.1 with 4 train and 4 test views, against 46.2 with 1 and 1 [arxiv:2204.02174 tab. 5] |
| Test-time compute is tight | Train with 4 views, test with 2 | 54.3 with 4 train and 2 test, against 55.1 with 4 and 4 [arxiv:2204.02174 tab. 5] |
| Only single-view inference is possible | Still train with several views | 4 train and 1 test gives 51.6, against 46.2 for 1 and 1 [arxiv:2204.02174 tab. 5] |
| Need an order-independent combination | Average over views | Average 55.1, max 52.7, average plus max 55.2 [arxiv:2204.02174 tab. 6]. Averaging is the default |
| Latency budget per object-utterance pair | Measure on your hardware first | The paper reports 264 ms at N = 4 and 261 ms at N = 1 on one GPU, and does not say which stages it times [arxiv:2204.02174 §4.3] |

## Implementation sketch

The core of the variant. Positional encoding and aggregation are the parts
a different method would change.

```pseudocode
# Shared point feature per object, computed once  [§3.2] [eq. 5]
shape = encode_points(object_points)              # (M, d)

fused = []
for j in range(N):
    # Rotate the box centre into view j  [§3.1] [§3.2] [eq. 3] [eq. 6]
    centre_j = rotate_z(box_centre, 2 * pi * j / N)
    # Position code of each object in view j  [§3.2] [eq. 7]
    pos_j = encode_box(centre_j, box_size)        # (M, d)
    # View feature = shared feature + position code  [§3.2] [eq. 8]
    obj_j = shape + pos_j
    # Shared decoder fuses view j with the words  [§3.3] [eq. 10]
    fused.append(decoder(obj_j, word_feats))

# Order-independent aggregation: mean over views  [§3.4] [eq. 11]
agg = sum(fused) / N
scores = score_head(agg)                          # (M,)
```

## Contradictions

None known. Only one card covers this concept, so no two papers disagree.

## Reproduction notes

- The paper says the text encoder starts from the first three BERT-base
  layers [arxiv:2204.02174 §4.2]. The released code, as the card reads it,
  builds those layers randomly. Details are on the card under "Paper vs
  code".
- The paper does not say how the reported checkpoint is chosen. The card notes
  that the released code keeps the checkpoint with the best test-split
  accuracy, so the reported Nr3D and Sr3D numbers carry a selection effect of
  unknown size.
- The ScanRefer results rest on separate code that the card did not check.

## Papers

- [Multi-View Transformer for 3D Visual Grounding](/papers/2204.02174): fuses N rotated views of a point-cloud scene with a shared transformer decoder, then averages the views before scoring.