2.1. Problem formulation

Let I^N denotes a group of gray scale images I^N = {I_n|n = 1, …, N}, In:Ω→R, Ω⊂Rd represents each image in the group. The proposed method applies for I_n as 2D or 3D images, but throughout the rest of the paper, we assume they are 3D images representing one phase in time in a 4D-CT dataset. The objective of GroupRegNet is to find a set of dense transformations that map the same anatomical locations between any two individual images in the group.

The optimization problem to be solved by GroupRegNet is formulated as:

where L_simi, L_smo, and L_cyc are the similarity, smoothness, and cyclic regularization losses, TtemN is a set of transformations {TtemN|n=1,…,N} that maps anatomical locations in the template to the corresponding locations in the input images, TtemN∘In, and TtemN∘IN represent the warped nth input image and all warped input images, respectively, Item=1N∑n(TtemN∘In) is the implicit template by averaging warped input images (Vandemeulebroucke et al 2011), λ₀ and λ₁ are the weights for smoothness and cyclic regularization, respectively. The cyclic regularization term will only be present if the relative motion in the image group is periodic or symmetric. The objective of the iterative optimization then becomes finding the optimal transformation Ttemn that aligns every image in the group to a template image while keeping the deformation field smooth and cyclically consistent. The inverse transformation Tntem that maps the same anatomical locations in the input image to the implicit template is determined from a fixed-point method (Chen et al 2008). The transformation mapping between the nth and mth image Tmn can be calculated using the composition of the deformation field: Tmn(x)=Ttemn(Tmtem(x)).

Figure 1 illustrates the components and data flowing of GroupRegNet. As compared to the common structure of a learning-based method VoxelMorph (Balakrishnan et al 2019), GroupRegNet uses similar components including a CNN (to be explained in the later subsections), a spatial transformer (implemented as a 3D linear interpolation), a similarity loss, a cyclic loss, and a smoothness loss. The input images are processed by the CNN to directly estimate the displacement fields. Existing methods in the literature explicitly select the reference and moving images to form a pair and then warp the moving image to the reference image. By contrast, in GroupRegNet, the input images in the group are first stacked in the channel dimension before feeding into the neural network, and the computed transformation then aims to warp the input image into the common space of the template image. It should be noted that CNN’s output is the displacement field Dtemn(x) instead of the transformation field Ttemn(x), which are related through Ttemn(x)=Dtemn(x)+x. The details of the components in this flowchart are further elaborated in the next subsections.

Flowchart of GroupRegNet. The expression (n, D, H, W) represents the number of images in the group and the spatial dimensions of the image.