EagleGrid
An architecture study for real-time multi-sensor object tracking on power-constrained edge hardware.
A design study, not a deployed system: how you'd fuse camera and depth into stable object tracks on an edge device. Detection and tracking (YOLO, ByteTrack), stereo depth and pose, running under TensorRT on a Jetson, streaming state over WebSockets and MQTT to an AR client. Written up as a build specification with the hard parts named up front — sensor disagreement, identity over time, and latency budgets.
The question
If you had to turn several cheap sensors into a single, stable picture of moving objects — on a power-constrained edge device, in real time — how would you build it? EagleGrid is my written answer: how the pieces fit, and, more usefully, where the hard parts are.
The interesting problems in perception aren't detection. They're agreement, identity, and latency.
The shape of the system
- Sense. A camera and a depth source, time-aligned.
- Detect & track. Object detection (YOLO-family) feeding a tracker (ByteTrack) so detections become persistent tracks.
- Fuse. Reconcile the sensors into one set of objects with stable identities and estimated pose — the layer where most of the difficulty lives.
- Serve. Stream track state over WebSockets/MQTT to an AR client that overlays it, all under a latency budget tight enough to feel live.
Inference runs at the edge (TensorRT on a Jetson) because round-tripping frames to a server blows the latency budget the moment the network hiccups.
The hard parts, named up front
Sensor disagreement. When the camera and depth disagree, which do you believe, and how do you degrade gracefully when one drops out? Identity over time. Keeping a track attached to the same real object through occlusion and re-entry is the difference between a useful system and a flickering mess. Latency budgets. Every stage spends milliseconds; the design is really an argument about where to spend them.
What building it would take
A calibration rig for the stereo/depth setup, a labeled slice of data for the domain, and a honest latency harness before any of the fusion work is worth trusting. Naming that cost is part of the point: the value of a concept is a clear-eyed view of the road, not a claim that you've already driven it.