Open reference record

NVIDIA NitroGen Action Annotations

Per-frame gamepad action annotations for public gameplay videos, intended for behavior cloning and action-to-video world-model research.

Creator: NVIDIA Corporation
Access: Open annotation archive
License: CC BY-NC 4.0
Commercial use: Restricted or non-commercial
Record reviewed: 2026-07-24

Scale and delivery

30K annotated videos and approximately 15B annotated frames

Formats: Parquet, JSON metadata, Twenty-second chunks
Timebase: Per-frame annotations aligned to referenced video

Modalities

Gamepad actions
Source-video metadata

Observations

Externally hosted public gameplay video referenced by URL

Actions

17 buttons
Two analog joystick vectors
Per-frame controller state

State

No full game-engine state supplied in the annotation release

Outcomes

Not supplied as a normalized outcome stream

Rights note

The dataset is non-commercial and provides action labels rather than a licensed copy of each source video. Source-video rights require separate review.

Preferred citation

Magne et al. NitroGen: An Open Foundation Model for Generalist Gaming Agents. 2026.

Need a commercially cleared collection?

Reference data can frame a requirement. A commercial program needs the right environment, signals, coverage, rights, and delivery contract.

Request access