2.3. Performance evaluation and statistical tests

In the age estimation test, the ages estimated from the test subjects (n = 30) on individual structures were compared with the subjects' actual ages by linear regression. The estimation of functional deficits in the dementia population was similarly evaluated by linear regression between the estimated ADAS.11 scores and the clinically measured ADAS.11 in the ADNI subjects (n = 90). The R² was used to evaluate the goodness-of-fit of the linear regression, and the p-value from the t-statistics was used to evaluate the significance of linear regression with False Discovery Rate (FDR) (Benjamini and Hochberg, 1995) correction. The linear correlations between volumes of each structure and age or ADAS.11 were also computed for comparison. The volumes were obtained using a multi-atlas segmentation pipeline, as described in Section 2.2.1 and the proposed atlas-weighting in Section 2.2.2.

To assess the significance of the estimated ADAS.11 among ADNI groups, a one-way analysis of variance (ANOVA) was performed among the AD/MCI/NC test subjects (n = 30 each), and the p-values from ANOVA tests were obtained and corrected by FDR for multiple ROI comparisons. The same ANOVA test was also performed on the volumetric measurements.

In addition to the regional presentation of patient attributes estimated from the MAV diagram, the regional features can also be combined to classify the disease categories. We compared the classification performance using the volumetric measurements, estimated ADAS.11 scores, and the estimated diagnostic category probabilities based on Eq. (5). In order to use the three types of categorical probabilities (AD/MC/NC) for feature selection and classification purposes, we integrated them into a single dementia probability measurement with a simple linear combination: p(Dementia | I_T , l) = p(G_AD | I_T , l) × 1.0 + p(G_MCI | I_T , l) × 0.5 + p(G_NC | I_T , l) × 0.0, for structure l of target image I_T.

To extract the most discriminative features from the high-dimensional feature vector (volumes or dementia probabilities estimated from 289 brain structures), we tested two approaches using the training data (atlases, n = 20 per group): 1) the top one or top 20 structures that showed the most prominent group difference, based on the p-values from the ANOVA tests; 2) the LASSO method (least absolute shrinkage and selection operator) (Tibshirani, 1996), which is a regression analysis method that selects the best subset of variables to enhance prediction accuracy. In our study, we performed LASSO using a regression model between the pre-determined diagnostic category (response variable) and the volumes, estimated ADAS.11 scores, or dementia probabilities from each structure (covariates). The LASSO method determines the optimal number of structures when the mean square fitting error is minimum. The regional features selected by the top one or top 20 criteria or LASSO were then fed into a linear discriminant analysis (LDA) classifier, using a leave-one-out cross validation approach on the ADNI test subjects (n = 90). The sensitivity, specificity, and overall accuracy of two-category (AD/NC) or three-category (AD/MCI/NC) classifications were evaluated. The LASSO and LDA were performed using R packages (https://cran.r-project.org/).

We evaluated the effect of scan protocol by including the protocol type (six types of protocols used in ADNI data acquisition) as another factor in addition to the estimated ADAS.11 scores, and tested its significance among the three groups, using two-way ANOVA followed by FDR correction. The protocol effect was statistically significant only in two of the 289 brain segments (left fusiform gyrus and left subcortical white matter of the inferior temporal gyrus).