Multi-modal GRU for MCI conversion prediction

Our problem can be considered as a sequential data classification. The classification objective is to predict whether an individual with MCI at baseline is converted to AD or not using sequence data, which consist of four modalities including cognitive performance, CSF, and MRI biomarkers as well as demographic information. Even though demographic data and MRI biomarkers are not longitudinal data we will consider them as length-one sequential data.

To apply a GRU-based classification algorithm to our problem, we need to design a model that can incorporate the four modalities of data. The main idea of our model is to separately build GRU feature extractors for each modality and integrate the extracted four feature vectors at the end. Our model is comprised of two training steps: (1) learning a single GRU for each modality of data, and (2) learning the integrative feature representation to make the final prediction. At the first training step, a single GRU is trained separately for each modality in which the classification objective is to predict conversion to AD from MCI. Using GRUs is essential to take longitudinal data and transform them into a fixed-size vector. This is quite similar to the approach proposed in³⁶ that maps the input sequence into fixed-length representation. In the second step, MCI conversion is predicted based on the four vectors produced from each GRU components. For merging four vectors, we select concatenation-based data integration, which is conceptually the simplest method to integrate multiple sources of data into a single vector³⁷. For the final prediction, l₁-regularized logistic regression³⁸ is used for the classification between MCI-C and MCI-NC. The overview of our proposed method is illustrated in Fig. 6.

Overview of the proposed method. Our proposed method contains multiple GRU components that accept each modality of the dataset. At the first training step (blue dashed rectangle), each GRU component takes both time series or non-time series data to produce fixed-size feature vectors. And then the vectors are concatenated to form an input for the final prediction in the second training step (red dashed rectangle).