2.8. Reproducibility Analysis and comparisons

The Dice similarity coefficient (DSC) and coefficient of variation (C_v) were used to compare the reproducibility of the ROIs and estimated CST streamline fibers. The DSC is a statistical measure of the spatial overlap of two distinct groups. It has a range between 0 and 1, with 0 indicating no overlap and 1 indicating perfect agreement between groups. Obtaining a DSC value greater than 0.7 indicates an excellent agreement between two raters (Zijdenbos et al. 1994a). The C_v is calculated as the ratio of the measurement standard deviation divided by its mean and multiplied by 100 to provide an intuitive estimate of variance expressed as a relative percentage. The ROIs were tested for reproducibility with three different comparison schemes:

Intra-rater / Intra-session (comparison of rater against self): Comparisons between the two manual segmentations of each ROI in the same scan acquisition by the same rater. This yielded 10 comparisons for each rater per ROI (3) yielding a total of 30 comparisons per rater.

Intra-rater / Inter-session (Comparison of rater between sessions): Comparisons of each rater-representative ROI from each scan session to all others scan sessions, 45 per ROI [(session 1 vs session 2, 1 vs 3, … or 10 scans × 9 comparisons) / 2] for a total of 135 comparisons per rater.

Inter-rater / Intra-session (Comparison of raters within sessions): Comparisons of the rater-representative ROI between raters for the same scan session. This yields 60 comparisons [(10 sessions, rater 1 vs rater 2, 1 vs 3, …, or 4 raters × 3 comparisons) / 2] resulting in 180 comparisons per rater for the 3 ROI's.

Given an expected increased variance between scan sessions and the individual rater's ability to hand-segment ROIs, we expected highest overlap in intra rater/intra session and lowest overlap in inter-rater / inter-session.

Tractography reproducibility was assessed with two different comparison schemes:

Intra-rater / Intra-session: Comparison of CST tract volume obtained with each rater's manual ROI segmentation of each ROI for each session for a total of 30 comparisons.

Each rater-representative ROI vs. the majority rules / intra-session: Comparison of CST tract volume obtained with each rater-representative ROI segmentation to the majority rules ROI for each session for a total of 30 comparisons.

The estimated CST also allows the calculation of FA and AD values along the CST path, and tractography derived measures: 1) edge weight (Hagmann et al. 2007), 2) tract count, 3) tract length, and 4) axial tract cross-sectional area. The edge weight, w(e) of edge e and is defined as

where S_i is the surface of the connecting nodes, f is a streamline that connects the nodes, E_f is set of all streamlines connecting the nodes, and l(f), is the length of streamline f. The edge weight includes all streamlines connecting the nodes regardless of their location of origin. The track count is the number of streamlines defining the CST, track average cross-sectional is the average area of the CST in axial slices, and track length is the average length of the streamline fibers in the estimated CST.