A cutaway of the rover’s technology
Each layer depends on the ones beneath it. The upper layers are what the rover does in the field, and the lower ones are how it learns to do it. Every layer says plainly how far along it is.
Open the live lab to drive an arm and pull grass in your browser.
Mission autonomy
A state machine runs the whole job: drive a coverage path, stop when a target is in view, pull it, check the result, and move on.
PlannedNot written yet. It depends on the layers below it.
Perception
Two depth cameras: a 90° camera scans ahead and a 60° camera looks close for arm targeting. A YOLOv8 segmentation model outlines each plant, and only the plant classes selected for pulling become targets.
PlannedCamera layout and label schema are set, with 1 target and 2 look-alike placeholder classes. No model has been trained yet.
Manipulation
A 6-joint arm with 0.5 m of reach and a two-finger gripper. It comes in from the side, closes on the stem low down and pulls. If the root holds harder than the grip can, the stem slips.
In progressThe arm is in the rover model, and the pull sequence runs on a test arm in the live lab. Arm control on the rover isn't written yet.
Mobility
Differential drive on 15 cm wheels set 35 cm apart, on a chassis 50 cm long. Turning the wheels at different speeds steers it, so it can turn in place.
In progressA scripted drive test exists for the Isaac Sim scene. Waypoint following and coverage paths are still to come.
Synthetic data
Omniverse Replicator renders plants under varied conditions, and labels them automatically: camera height 0.12 to 0.6 m, sun elevation 20° to 80°, and 20 to 80 plants per square meter.
In progressProcedural plant assets, randomizers and a label writer are in place. The plan is a 500-frame pilot, then 20,000 frames.
Simulation
Every layer is built and tested in NVIDIA Isaac Sim before any hardware exists. The rover and arm are one model, so driving and pulling share the same physics.
In progressThe rover model and field scene scripts are written. Physics values are still rough placeholders.
Field hardware
A ROS 2 workspace is set aside for moving onto a physical rover.
PlannedDormant. There is no physical hardware yet.