Most annotation tooling was built for images and later extended to point clouds by projecting them onto a screen. For autonomous driving, where the data is dense and the objects are large, that mostly works. For robotics — sparse indoor sweeps, small objects, sensors mounted on a moving arm — it fails in ways that are hard to see and expensive to discover.
Failure one: annotating the projection
When a point cloud is shown as a 2D projection, annotators draw what they see, and what they see depends on the viewing angle. A cuboid that looks tight from above may be twenty centimeters off in depth. The label passes visual review because visual review shares the same blind spot.
Our Annotation Studio treats the 3D extent as the label and derives every 2D view from it. Cuboids are checked against point density on all six faces. An agent flags faces with no supporting points, which usually means the box is too large, and faces that cut through points, which means it is too small.
Failure two: ignoring the cameras
Robots carry cameras for a reason. A point cloud tells you where an object is; a camera tells you what it is. Tools that annotate LiDAR in isolation lose the second half. We fuse camera views into the point-cloud editor so the annotator sees the RGB patch under every cuboid, and agents compare the projected box against camera-based detections. Disagreement is routed to review with both views attached.
Failure three: sweeps as frames
A LiDAR sweep is not an instant; it is a rotation that takes tens of milliseconds, during which the robot and the objects may move. On a mobile manipulator, treating each sweep as a frozen frame produces smeared points and mislabeled extents. Motion compensation using odometry and joint state is applied at ingest, and the correction is recorded as provenance like any other operation.
The label is the geometry. The screen is just one view of it.
Failure four: no ground truth for the ground
Indoor robotics scenes are dominated by floor, walls, and shelving. Without ground removal and structural segmentation, annotators spend most of their time on points that will never matter to the policy. We remove ground automatically, color remaining points by height and intensity, and let agents propose structural masks the annotator confirms once per scene rather than once per sweep.
What good LiDAR annotation looks like
Tight cuboids checked on six faces. Camera views fused, not adjacent. Motion-compensated sweeps. Ground and structure handled once. Every geometric correction proposed by an agent with the measurement that justified it, confirmed by a person. It is slower to build than a projection editor. It is much faster to trust.