Ask five robotics teams what an "episode" is and you will get five answers. For some it is a recording session. For others it is one attempt at a task. For others still it is whatever length the training pipeline happens to accept. The disagreement is not academic: it decides what a model learns, how success is measured, and whether two datasets can ever be combined.
Three levels, one timeline
We have settled on three levels of structure, all living on a single shared timeline.
An episode is one complete engagement with a task: it begins at a defined initial state and ends at a terminal outcome — success, failure, or abort. Nothing shorter than that is an episode, and nothing longer.
An attempt is a discrete try at a sub-goal inside the episode. A grasp is an attempt. A regrasp after a slip is a second attempt. Attempts can fail while the episode succeeds, and that distinction matters enormously for learning: a dataset that hides failed attempts inside successful episodes teaches a model that hesitation is part of the task.
A sub-task is a stage of the task with its own success criterion: approach, grasp, transport, place. Sub-tasks partition the episode; attempts occur within them.
Why boundaries drift
Most boundary inconsistency comes from anchoring to appearance. An annotator watches the video and places the "grasp begins" marker where it looks like grasping starts. Another annotator, or the same one a week later, sees it differently. Multiply by thousands of episodes and the training signal blurs.
The fix is to anchor boundaries to state. The gripper closes at a measurable moment. The base stops moving at a measurable moment. Contact begins when force-torque crosses a threshold. When sensors permit, propose the boundary from the signal and have the annotator confirm it. When they do not, write the guideline in terms of the closest observable trigger.
A boundary you can reproduce from telemetry is a boundary that will still be correct in a year.
Outcomes are not optional
The single most damaging defect we see in demonstration datasets is the missing outcome. An episode ends; nobody records whether it succeeded. The model is trained on it anyway. In our platform, an episode cannot leave the annotation stage without an explicit outcome on the episode and on every attempt within it. Agents flag the omission; a person fills it in.
Metadata at ingest
Task, embodiment, scene, operator, and environment conditions are trivial to capture at recording time and nearly impossible to reconstruct afterward. Treat them as required fields. They are what make condition-based evaluation possible later, and they cost almost nothing now.
Get this structure right early and you will never revisit it. Get it wrong and every downstream stage inherits the confusion.