Open reference record

Rocket Science

Synchronized four-player Rocket League video, keyboard actions, game events, and per-frame physics state used to train the MIRA multiplayer world model.

Creator: General Intuition and Kyutai, in collaboration with Epic Games
Access: Public listing with gated acceptance
License: CC BY-NC-SA 4.0
Commercial use: Restricted or non-commercial
Record reviewed: 2026-07-24

Scale and delivery

15,769 train matches and 62 test matches, approximately 29.2 TB

Formats: WebDataset shards, MP4, JSONL, Typed Python loader
Timebase: Single match clock with per-recording offsets

Modalities

Four synchronized video views
Keyboard actions
Events
Physics state

Observations

1280x720 H.264 video at 20 fps from four player perspectives

Actions

Nine-key multi-hot keyboard state aligned to frames

State

Ball state
Four-car state
Score
Clock
Overtime

Outcomes

Goals
Demolitions
Boost events
Match result

Rights note

The dataset is non-commercial. Its card states that commercial use of Rocket League content requires separate permission from Epic Games.

Preferred citation

Hu et al. Multiplayer Interactive World Models with Representation Autoencoders. 2026.

Need a commercially cleared collection?

Reference data can frame a requirement. A commercial program needs the right environment, signals, coverage, rights, and delivery contract.

Request access