Language describes the task, not the whole trajectory
A task string can express intent while images describe the current scene. The action target still needs a declared interface: pose delta, joint target, end-effector command, gripper state, discrete skill, or another embodiment-specific control.
Paraphrases, corrections, subgoals, and termination language should remain linked to the episode segment they govern. Treating one caption as supervision for an entire session can blur which instruction caused which behavior.
- Instruction, goal, and optional subgoal text
- Camera and sensor observations with calibration
- Action tokens plus physical units and control rate
- Embodiment, task, success, failure, and intervention metadata
Action tokenization must remain reversible
A VLA may predict discretized tokens, continuous action chunks, or higher-level skills. The release should preserve the original command beside any quantized or normalized representation.
Token vocabulary, bin boundaries, chunk horizon, control frequency, coordinate frame, and clipping behavior belong in the schema. Otherwise identical token IDs can imply different motion across robots or datasets.
Cross-embodiment mixtures need explicit normalization
Large VLA programs often mix tasks and robot types. Positive transfer is possible, but morphology, sensors, action spaces, control rates, and success definitions do not become equivalent because they share one loader.
Keep source-dataset identity and embodiment metadata available during training and evaluation. Hold out robots, scenes, objects, and instruction families to measure generalization rather than memorization.