HexBug Head Tracking

Framework to track a freely and randomly moving object (“HexBug”) head from a video, within the context of the Tracking Olympiad (TRACO) course.

For the SS2023 Tracking Olympiad course at FAU, we had to propose a solution in groups of three to gain hands-on experience as AI developers. The task involved tracking the heads of all the HexBugs visible in a 10-second video, under various background conditions while maintaining a static background.

A set of videos containing the ground-truth positional information was provided at the beginning of the course. The team’s score was evaluated by applying the tracking algorithm to previously unseen or withheld videos at the end of the course and we achieved the second place between 7 groups.

Examples of our proposed tracking solution on three different previously unseen videos. The videos on the left display the detected head axis from the center of the HexBug, the videos in the center show the final predicted head position indicated by a red circle, and the videos on the right illustrate the initially detected HexBugs using YOLO-NAS, Fast-SAM and CLIP models.

Our solution is composed of four steps:

  1. HexBug detection (YOLO-NAS) followed by box filtering (ViBe) based on background subtraction and Non-Maximum Supression (NMS)
  2. Object tracking and box interpolation (CLIP-ViT + K-means)
  3. HexBug-agnostic segmentation (FastSAM)
  4. Head position estimation based on an ellipsoid representation of the HexBugs

The pipeline is illustrated in the following image.

Pipeline of our framework for HexBug head tracking.

We show two additional examples of our solution: