Stage 5: Kalman filtering and Hungarian algorithm

The final part of the software introduced a three-state Kalman filter to smoothen the moving trajectories and to predict the flight pattern of the target. Based on the historical velocity, motion direction and current orientation of existing tracks, as well as the abilities of bees to make sharp turns, accelerate and decelerate, the 3D tracking module distinguished multiple targets during their convergence and kept track of the correct one. The Kalman filtering model was built with three state variables and their derivatives (3D coordinates and 3D velocity) as the state vector. The covariance matrix of the process noise Q and the covariance matrix of the observation noise R were defined based on a set of simulation results and were adjusted to provide reference values with different weights between historical data and current measurement. The whole group of discrete 3D coordinates in every video frame was fed into the filter to initialize and in return optimize the filter itself. After initialization of the whole model, new detections grabbed from the current frame were allocated to existing tracks through a methodology called the Hungarian Algorithm (Kuhn 1955). It calculated a 2D cost (distance) matrix between each newly detected object and all the existing tracks and output an allocation strategy with a minimum overall cost. This, in the first place, attached every object in the current frame to its nearest existing track based on the cost matrix. Unallocated points, due to their distance from all the existing tracks being too large, were considered as the starting points of new tracks. Existing tracks that obtained no allocation were marked as ‘frame skipped/loss’ and were paid more attention as the filtering process continued, whether their tracked targets had disappeared permanently or merely for a couple of frames. The processing upon the dual channel binarized video yielded a set of complete 3D tracks for a 5-min video. The result was visualized with a color-coded interface using VTK (The Visualization ToolKit).

Analysis based on the curvature of the tracks was conducted. This was done by calculating the change in flight orientations across the whole track, given by

Where α_i is the angle between v_i and v_i+1, the instantaneous velocities at measurement point i and i+1, respectively, N is the length of the track, and A is the vector containing the array of α across the entire track. If more than 90% of elements in A are smaller than 5°, the track is marked as a ‘straight track’, otherwise a ‘curved track’.