Rigid and non-rigid registration

Registration between two point sets is the process consisting of modifying, as a whole, the positions of the points from one set to make them as close as possible to the points in the other set. An important step of our tracking approach consists of registering all the spot detections of the sequence to a single reference frame in the middle of the sequence. This section explains why and how it is achieved.

Applying spot detection independently on each 3D stack of a sequence produced as many 3D point sets St as time frame: {St, t = 1,.., T}. Because the spot detection step was inaccurate due to the low image resolution and the high density of objects, each somata over time could be alternatively detected as one, several or no 3D point leading data association of regular spot tracking algorithm to fail. Therefore, we took advantage of the fact that in our case, unlike in regular tracking applications, all points belong to the same tissue. Consequently, registering 3D point sets as a whole to cancel movements and group detections over time by somata should help resolving data association even in case of sub- or over-detections. In our case we registered all the stacks of the sequence to a reference frame using rigid (in fact affine), then non rigid registration algorithms.

The purpose of the affine registration was to first cancel large and global shifts due to erratic movements of the fly caused for instance by the pulsatile organ or by the spatial drift. Each 3D point set S_t corresponding to each time frame t < T/2 of the first half of the sequence was registered to the point set of the next time frame S_t+1 by identifying optimal parameters of an affine homogeneous transformation matrix A_t (including rotation, translation and scaling but not shearing) that minimizes the sum of distances between the two point sets. To this aim, two random subsets of equal size 100 from S_t and S_t+1 were considered and optimally coupled using the Hungarian algorithm⁴⁸ prior minimization. The Hungarian finds the optimal 2D association between two point sets of the same size and the minimization process identifies the affine transform that minimize the sum of squared distance between the 100 couples of point thus defined. The process to align each time frame t to the time frame t + 1 thus reads:

Once all homogeneous transformation matrices obtained {At⁎, t = 1 to T/2}, all data points for t < T/2 were transformed to the reference frame t = T/2 chosen to be in the middle of the sequence. To transform any points x from an arbitrary time frame t₀ < T/2 to the reference frame, we chained the homogeneous transformations the following way:

The same principle was applied in the reverse way for all time frames t > T/2. At the end of this process, all points from all time frames were roughly aligned to the frame T/2, correcting for the coarse movements of the brain.

This first step could correct for large movements but was not able to correct for smaller, local deformations caused by the elastic property of the tissue. This finer grain correction was performed through an additional non rigid registration we performed using a straightforward application of the Coherent Point Drift (CDP) algorithm²⁷. CDP considers the two data point sets to register as realizations of close by Gaussian Mixture Models (GMM). The algorithm forces the GMM to move from one set to the other coherently as a multimodal distribution to preserve the topological structure of the point sets. Unlike with the rigid transformation, each 3D point set S_t was not registered to S_t+1 but directly to the reference 3D point set S_T/2. This is because the affine registration already corrected for the coarse long term movements. On the other hand, note that CDP could not directly correct for large shifts on its own and the prior rigid registration enabled CDP to properly correct for local deformations. Overall, aligning all 3D detections overtime enabled to form clusters which the following step could detect and use to further identify, split and complete trajectories.