MASS: Multiplayer World Models with Authoritative Shared State

In plain words

Most video world models that simulate a game work from what one player sees. They keep their memory as a running stack of pictures. When many players share one world, every camera ends up carrying its own copy of that world, and the copies can disagree about what is where.

MASS works more like the server of a multiplayer game. It keeps one structured description of the world, a list of typed fields for every snake, every food cell and the tick counter, and a learned model moves that description forward one tick using all players' moves. A second learned model then draws the picture for any camera from that same description. The world is advanced once, and every view is drawn from the same state, so the views agree about what exists.

The authors test this on a Snake game and on several other games. Their main argument is that the structured state can be checked before any image is drawn, which lets an error be traced to the world's dynamics or to the rendering.

TL;DR

MASS separates world dynamics from view rendering. A decoder-only Transformer (the Logic Engine) predicts the next typed state of the whole world from the current state and the joint actions, and a residual U-Net (the Rendering Engine) draws each requested camera from that state §3. On a matched two-player, two-view Snake benchmark the authors report parser state recovery of 0.764 against 0.128 for the best video baseline, and zero cross-view disagreement by construction tab. 2. They also report 1,024 simulated entities for 10,000 recurrent ticks §4.5. No official code has been found, and the paper's hyperparameter coverage is partial.

Problem

Interactive video world models carry one recurrent memory per player view. In a multiplayer world, each of many views would need its own history of the same shared content, so the histories can drift apart, views can disagree about one entity, and the cost of simulating the world grows with the number of views being watched §1. Existing multiplayer models keep their shared representation visual, dense or externally maintained. The authors argue none of them uses a typed, authoritative state that serves as both the recurrent memory and the synchronisation object §1.

Method

The world is a set of typed records declared by a game schema eq. 1. The state at tick t is one record for every declared entity, so K records in total, with K the sum of the declared counts eq. 2 §3.1. In Snake, the schema has three entries: a global tick counter, one entry per snake (1,024 of them) and 4,096 food entries that each hold the food cells of one 8×8 region. One tick therefore has K = 5,121 records §3.1 app. A.1.

World State Tokenizer §3.1. Each record is turned into a short token sequence from its own fields, a neighbourhood of cells around it (taken from the current state and the actions), the action of its attached player if it has one, and its share of the exogenous input eq. 3. Numbers, coordinates, flags, optional values and lists get type tags at fixed positions, so every record decodes back to its fields without loss §3.1. Each new game supplies a schema and recorded trajectories. The model design is shared, while each game gets its own tokenizer, embeddings and weights §3.1.

Logic Engine §3.2. The world-level transition is a function of the current state, the joint actions and the exogenous input eq. 4. The implementation factorises it record by record: each record's next tokens are predicted conditioned on that record's own token sequence and on its earlier decoded tokens eq. 5. Self-attention stays inside each record's sequence, and interaction between objects enters only through the context window, which is computed from the current state before any prediction eq. 5 §3.2. A mask derived from the schema limits every output slot to the values its field may take, and a fixed selector keeps lists sorted and unique while decoding §3.2 app. A.2. The decoded records are assembled into the next state, which becomes the next recurrent input and the single state that every view reads eq. 6. Movement, collision, growth, collection, death and reward are not coded anywhere; they are learned app. A.2.

Rendering Engine §3.3. For camera i, a projection builds a tensor local to that camera. It records the occupant of every visible cell, the entity behind it, the player who holds the camera and the viewing geometry. A learned residual U-Net then turns this tensor into an RGB frame eq. 7. For a set of cameras the observation model factorises over cameras, and all of them read the same predicted state eq. 8 §3.3. Cameras can be added or moved at rollout time without changing the world transition §3.3.

Server and client logic §3.4. After each global transition the server publishes a versioned typed snapshot. The per-tick server workload is the logic cost for N active entities plus the sync cost of sending the snapshot to each of C clients eq. 9. The transition runs once; distribution is linear in the number of clients; rendering runs on each client §3.4. During an update stall, a client advances its latest authoritative version with the same Logic Engine. It supplies its own actions, uses no-op actions for other players, and leaves unknown spawns empty until the next version arrives §3.4 app. A.2.

# One learned shared-world update  [alg. 1]  [app. A.2]
# Input:  state (K records), joint_actions, exogenous, cameras
# Output: next_state, next_version, one RGB frame per camera

def step_world(state, version, joint_actions, exogenous, cameras):
    # Split the current state into its typed records  [eq. 2]
    records = records_of(state)
    decoded = []
    for record in records:
        # Read the context from the current state only  [§3.2] [app. A.2]
        context = neighborhood(record, records)
        # Tokenize the record, its context, actions and exogenous share  [eq. 3]
        tokens = tokenize(record, context, joint_actions, exogenous)
        # Decode the next record field by field under the schema mask  [eq. 5]
        decoded.append(decode_with_masks(tokens))
    # Assemble the decoded records into the next authoritative state  [eq. 6]
    next_state = assemble(decoded)
    # Advance the version counter once per tick  [alg. 1]
    next_version = version + 1
    frames = []
    for camera in cameras:
        # Project the full next state into this camera's tensor  [eq. 7]
        projection = project(next_state, camera)
        # Render one RGB frame; the state is read and never changed  [eq. 7]
        frames.append(render(projection, camera))
    return next_state, next_version, frames

# Teacher-forced token loss for one transition  [app. A.2]
# Each target token is scored given the true earlier tokens of its record

def logic_loss(records, targets):
    loss = 0.0
    for record, target_tokens in zip(records, targets):
        for j in range(len(target_tokens)):
            # Negative log-likelihood of token j given the record and its prefix
            loss -= log_prob(target_tokens[j], record, target_tokens[:j])
    return loss

Two state codecs are used. The matched Snake benchmark serialises the whole small state, with tick, 64 food coordinates and per-player body, heading, survival and death anchor, into one 421-token state block app. A.2 app. B.2. The input prefix is 447 tokens: BOS and STATE markers, the state, an ACTION marker and the joint actions, and an EXOGENOUS marker with the tick and up to eight food spawns, followed by an OUTPUT marker app. A.2. The population-scale codecs (1,024 entities) instead factorise the world into records and predict them in batches, with the spatial context described above app. A.1 app. A.2. Table A.1 lists record counts, token limits and local context for the population-scale Snake, Crate Pusher and Pac-Man codecs tab. A.1 app. A.1.

The method's equations are eq. 1 to eq. 9 in the Method section. The training loss is given in the appendix app. A.2.

Hyperparameters

Values for the matched Logic Engine, the population-scale Logic Engine and the renderers are the ones the paper states. Anything not stated is marked.

Results

Author-reported. Setup: the matched Snake benchmark (N = 2, two views, 128 ticks at 128×128, 8 test episodes), with the baselines retrained on the same data. MultiWorld is the official code, retrained on the matched corpus §4.3 app. B.1. Bold marks the best value in each row.

Matched Snake, headline comparison tab. 2:

The authors say MASS leads six of seven metrics. Its invalid rate is second-best tab. 2. Their claim that X-view disagreement is zero is guaranteed by the design, because every view decodes from one predicted state §4.5.

Rendering quality on held-out reference states, 256×256 tab. 3:

The same architecture, with per-game weights and tokenizers, renders these eight games §4.5 fig. 3. Five of the eight exceed 32 dB §4.5. Population-scale 2D and 2.5D renderers give SSIM of 0.9795 to 0.9970 on reference states app. C.2.

Client prediction during missing server updates on Snake tab. 4:

Matched Snake logic, typed carrier versus dense diagnostics tab. B.4. Fixed validation seed, 24 episodes, feedback of raw predictions. Values are percentages except latency:

Full-state exact match for the typed model is 41.7% at H = 1 and 0.0% at H = 32 and H = 128 app. B.2. Training for 100,000 updates raises H = 1 exactness to 45.8% app. B.4.

Cross-game state prediction (typed codec only) tab. C.1:

Long-horizon Snake stress test without ground-truth continuation tab. C.2: 159 reference transitions; no-op actions and empty spawns; held-out starts.

With the 100,000-update checkpoint, two initialisations produce 1,850 and 302 unique states app. C.3.

SRSC (VLM-judged semantic correctness, mean score 0–100) tab. D.1. The judge is an auxiliary diagnostic, and the sample sizes are small (see Pitfalls):

Pac-Man ends at H = 300 tab. D.1. The authors read the Renderer-GT control rows as showing that rendering does not accumulate semantic error over the horizon, and that most decay in the full pipeline comes from the learned dynamics app. D.2.

Paper vs code

No official code.

The paper's front matter links only a project page, https://alaya-lab.github.io/MASS/, and that page has no code link. I searched GitHub for the method name, for "authoritative shared state world model", and for the author names with "world model". The authors' lab org alaya-lab has a repository MASS, but it is a static project-page site (HTML, CSS and JavaScript, no model, training or data code, and no LICENSE file), so it is not treated as code. lixuan27/agentworld-demo describes itself as a reproduction and has no license; it is not from an author and was not used. No repository was cloned.

Because no code is available, the implementation details that the paper leaves open (see Pitfalls) cannot be checked against source.

Pitfalls

• Capacity and budget are not matched to the strongest baseline. The MultiWorld baseline has 5.60B trainable parameters and follows its official 35-epoch schedule. MASS's predictor has 5.66M parameters and 20,000 updates app. B.1. The comparison is useful as a representation test, not as a parameter-matched or compute-matched result.
• The headline matched benchmark is only two players. The end-to-end comparison uses N = 2 and two views, because that is MultiWorld's interface. The logic-only comparison covers N = 2, 4 and 8 app. B.1 tab. B.4.
• The matched benchmark gets spawn information from recorded data. Its exogenous channel is a compatibility payload stored from adjacent recorded states. The population-scale studies use prospective, engine-native randomness instead app. B.2. Results from the two codecs should not be combined.
• Zero X-view disagreement is by construction. Both views decode from one predicted state, so the metric confirms the design but says little about the Rendering Engine §4.5.
• Only the world transition is view-independent. The transition runs once per tick, but rendering cost grows with the number of requested views. It rises from a mean of 189.6 ms for one view to 842.4 ms for 1,024 views, and the learned CNN part rises from 4.4 ms to 379.7 ms, on one H100 app. C.4. The hardware for the matched latency in tab. B.4 is not stated.
• The 1,024-entity, 10,000-tick claim lacks a table. The abstract and introduction state it §1 §4.5. In the appendix I found only stress runs with 2, 4 and 8 snakes up to 4,096 ticks app. C.3, plus render timings and client-prediction windows for 1,024 entities app. C.4. The paper says the appendix reports per-horizon results and throughput for this case; I found no such table.
• Full-state exactness is strict. Typed predictions have zero full-state exactness at H = 32 and H = 128 on Snake, though field accuracy stays high tab. B.4 app. B.2. Tile Merger full-state exactness is zero at every horizon tab. C.1.
• Greedy decoding without repair. Evaluation feeds back every predicted token, with no state replacement or gameplay-rule repair §4.2. Selectors only enforce sort order, uniqueness and canonical padding; collisions, growth and food consumption are learned app. A.2.
• Camera paths are fixed from the reference state. Cameras are not recentred on the prediction, so every method sees the same view locations app. B.1. An implementation must do the same, or the image metrics are not comparable.
• The semantic parser gates the metrics. Semantic evaluation runs only if the frozen parser recovers at least 0.99 of the ground-truth state, which it does on the matched test set app. B.1. The parser is Snake-specific.
• Unstated training details. The renderer loss weights and the checkpoint-selection weights are not given app. A.2 app. B.1. The population-scale Logic Engine's learning rate, batch size, warmup and the sampling factor for rare events are not stated app. B.1. The matched seed value is only described as fixed app. B.4.
• The typed model is about 12 to 15 times slower per step than the dense baselines. In the matched table it takes 45.55 ms, against 3.07 to 3.94 ms tab. B.4. The server-side synchronisation cost term in the paper's cost model is not measured §3.4 eq. 9.
• The SRSC evaluation is small. It uses 20 scenes for Snake and Crate Pusher and 6 for Pac-Man tab. D.1. The judge's logic-axis score correlates with the programmatic state metric at r = 0.962 app. B.1. Renderer-GT scores for Snake fluctuate between 63.8 and 75.0 without a trend, and that gap reflects the categorical raster reference, not rendering drift app. D.2.
• Client prediction is exact only for the local avatar. The local avatar stays exact in every window with local actions only. Other visible objects drift as unknown actions accumulate, from 0.815 agreement at k = 1 to 0.429 at k = 8 tab. 4 app. D.1. Reconciliation replaces the speculative state when a new version arrives app. A.2.

Lineage

• Builds on: the authoritative-server architecture of networked games, which separates one canonical world update from per-client snapshots and presentation §2 §3.4. The paper cites Bernier and Claypool for it. Neither paper has a card here.
• Compared against:
  • MultiWorld, a coupled multi-view video model (arXiv:2604.18564); its official code was retrained on the matched corpus §4.3.
  • Internal baselines: B-PV (independent per-view video predictors), B-SL (one shared visual latent) and B-UN (dense joint-state U-Net, the "typed-carrier" ablation) §4.3 app. B.1.
  • Single-player interactive video world models named in the introduction: GameNGen (arXiv:2408.14837), Oasis, DIAMOND, Genie and WHAM §1.
  • Multiplayer generation work named in the related work: MultiGen (arXiv:2603.06679), Gamma-World, WanToFight, Solaris, WorldWeaver and others §2.
• Superseded by: not stated; no later work is cited in the paper.