2.2. Video ROI Extraction and Tracking

The 96 ROIs (one ROI per well) were manually tagged on a computer using a custom cross-platform analysis program implemented with Python and OpenCV. The program offers a user-friendly GUI, and a skilled user can mark all 96 ROIs in under a minute. This method can easily be extended to a smartphone interface that allows the user to tap on ROI centers, with no loss of functionality. Automatic extraction methods are also possible using colored templates or special markings on the plate.

However, it is impractical to manually extract ROIs from each frame. A 5 second video at 60 FPS will have 300 frames, requiring over two hours of manual data entry. Likewise, automatic ROI extraction may be too expensive to compute for each frame on the smartphone. Therefore, we developed an algorithm to track ROIs from frame to frame throughout the video, given a set of initial coordinates.

We used Shi-Tomasi corner detection to obtain a set of corner points, and we computed the LucasKanade optical flow for each point. Each frame of the video, we updated the location of the ROI centers using the ten nearest corner points. Using the sparse optical flow of the image, we found the distance each corner point had traveled since the previous frame. We moved the ROI center by the mean displacement of the ten nearest corner points. Although the method worked well for short videos, the ROI centers began to drift after several hundred frames and for fast-moving video content. Therefore, we also manually updated the ROI center locations when necessary. Using OpenCV, we overlaid the video with the ROI boundaries to ensure that there was no drift and that the algorithm correctly extracted the 96 wells for all video frames.