TRACK EVERYMOVE
Detect a cat, hold its identity, and say exactly how far it is — in meters, with an honest confidence interval — from one ordinary camera. Then see where it will be, so you frame the shot before it lands.
- detector approaches
- 2
- every estimate
- ± CI
- FOV undistort
- 120°
detector approaches
every estimate
FOV undistort
Core capabilities
Three things, done honestly.
No magic numbers. Every distance carries its uncertainty, every prediction shows its lead time, and every robot move has a hard safety stop.
Distance ± CI
Pinhole geometry and metric depth are fused into a confidence-weighted median, so you get meters with a real uncertainty band — not a single hopeful number.
Where it will be
Per-track motion is extrapolated over an adjustable lead time. A ghost box shows the landing spot and its predicted distance, so the camera is ready before the cat is.
Robot follow
An anti-oscillation controller drives the chassis to hold a safe distance and keep the cat centered — smooth, with a hard ultrasonic stop in firmware.
How it works
One frame, four stages.
Pretrained perception plus classical camera geometry — no training required to run. The same pipeline drives the overlay, the JSON, and the robot.
Detect
YOLO11 or RT-DETR finds the cat (COCO class 15) on the undistorted frame.
Track
BoT-SORT + ReID keeps a stable identity through full occlusion.
Distance
Geometry ⊕ metric depth → meters ± a conformal confidence interval.
Predict & follow
Extrapolate the next position, then drive the robot to hold its distance.
Architecture
One pipeline, end to end.
Pretrained perception plus classical camera geometry. The same chain produces the overlay, the JSON report, and the robot commands — no training required to run.
- 01
Capture
One ordinary camera, RTSP or webcam.
- 02
Undistort
120° FOV / division model rectify.
- 03
Detect
YOLO11 or RT-DETR · COCO class 15.
- 04
Track
BoT-SORT + ReID · stable IDs.
- 05
Distance
Geometry ⊕ depth → m ± CI.
- 06
Predict & follow
Extrapolate, then drive the chassis.
Two interchangeable approaches
- Detector
- YOLO11s — CNN, fast
- Tracker
- ByteTrack
- Pick when
- Latency and command smoothness matter most.
- Detector
- RT-DETR-l — transformer, NMS-free
- Tracker
- BoT-SORT + ReID
- Pick when
- Cluttered, occluded scenes where identity must survive.
Honest distance
Two estimates, one number, a real interval
Z = fy · H_real / h_px
Pinhole math on the rectified frame with per-class size priors, down-weighted as boxes drift to the distorted edges.
Depth-Anything-V2 / UniDepthV2
A metric depth model sampled inside the box, ingesting the camera K for true scale and a per-pixel confidence.
confidence-weighted median ± CI
The two are fused into one median, with a confidence interval from estimator spread plus split-conformal residuals.
The frozen metric
We only keep what moves the number.
Every change runs against a frozen eval and is kept only if the metric improves — a keep/reject loop, no moving the goalposts. These are the dimensions we score.
Distance MAE
the scored metric
Mean absolute error in meters on the hidden Go2 test set — the one number the challenge is graded on.
Real-time
frames per second
The pipeline is profiled for FPS so the overlay, the JSON, and the robot all stay live.
Track continuity
identity through occlusion
ReID keeps a cat's ID stable through full occlusion, measured as track breaks per sequence.
±1 cm
ultrasonic ground truth
An HC-SR04 confirms estimates live on the rig — “1.84 m” predicted next to “1.86 m” measured.
Training & models
Fine-tuned only when it earns its keep.
The pretrained baseline always runs — it is the guaranteed demo. A fine-tune is a stretch on top, kept only if it beats the frozen metric. Nothing trains from scratch.
Baseline always runs
The pretrained model is the guaranteed demo. One command rolls back to it instantly, so no experiment can ever block the live run.
Fine-tune a pretrained YOLO
We fine-tune yolo11s on a cat dataset with a deterministic seed and a resumable single-file harness — never from scratch, where there are no labels and worse generalization.
Kept only if the metric moves
Each candidate is scored on the same frozen eval and kept only if distance MAE improves past a tolerance band — a proxy gain alone never justifies a keep.
The fine-tune loop
- 01
Dataset
Roboflow, Open Images, or manual — registered in configs.
- 02
Prepare
Formatted into one Ultralytics data.yaml.
- 03
Fine-tune
yolo11s.pt · deterministic seed · resumable.
- 04
Evaluate
Scored on the frozen metrics, against ground truth.
- 05
Keep / reject
Beats the baseline, or it is dropped.
- 06
Promote
Winner wired in — gated and baseline-safe.
Pluggable by design
Swap the model, swap the data
- YOLO11s — CNN, fast
- RT-DETR-l — transformer, NMS-free
- Roboflow
- Open Images
- Manual / on-rig
Fixed-budget experiments sweep the training knobs and auto-apply the same keep/reject contract — as durable jobs that survive a crash and recover on restart.
Hardware
Fully wireless. One laptop brain.
The laptop talks to the camera over Wi-Fi and the robot over Bluetooth — no gateway. Connect it all from the admin view in a few clicks.
Tapo C211
EyesWi-Fi · RTSP
Live frames over the network, with optional pan/tilt.
Arduino Mega
MotionBluetooth · Serial
Drives the wheels and the camera-pan servo from the laptop brain.
HC-SR04
Ground truth±1 cm ultrasonic
Confirms the model live: “1.84 m” estimated, “1.86 m” measured.
Documentation
Go deeper.
Every claim on this page traces back to the repo — run instructions, the fact-checked design rationale, and the per-challenge architecture.
Point a camera. Know the distance.
Run the console locally, connect your camera and robot, and start a session. The pretrained baseline runs out of the box — no training, no calibration to get a first reading.