2.6.7. Movie‐viewing data analysis

Precise timing for the start of the movie during each run is required for alignment between the movie runs and feature regressors. The start time of each movie was computed by calculating cross‐correlations between the movie audio collected during each imaging run (recorded with the HD‐DOT data through one of the ADC channels). Data time courses were then shifted by the lag that maximized the correlation between the collected and known audio signals extracted from the movie file. This approach corrected for any variable stimulus delays caused by the experimenter or equipment, allowing highly timing‐sensitive analyses like feature regression to be accurately time‐locked (Vanderwal et al., 2019).

To study inter‐run synchronization, voxel‐wise correlations were calculated between the oxyhemoglobin signals across time from the two movie runs of interest (Fishell et al., 2019; Golland et al., 2007; Hasson et al., 2004). Random effects t‐statistic maps were constructed to aggregate the results across multiple pairs of runs and allow comparisons between conditions, for example across all matched movie run pairs, or across mismatched movie run pairs as a negative control (Fishell et al., 2019; Sherafati et al., 2020).

For feature regressor analysis, speech and face regressors were coded for every movie clip in the library by three independent experimenters, using a protocol adapted from previous studies (Bartels & Zeki, 2004; Fishell et al., 2019). Speech content was scored on a binary basis (present or not) in 1‐s bins across each movie clip. The salience of faces in the scene was graded on a 4‐point scale (0 = absent, 1 = present but in the background, 2 = present and salient, 3 = filling the screen or otherwise highly salient) for frames sampled at 1‐s intervals across each movie clip. Each coded regressor was averaged across the three experimenter's ratings, with a final reviewer resolving major disagreements at any time points attributable to human error. While automatic approaches for detecting speech and faces were also considered, these techniques are built upon manually coded datasets and traditionally focused on real‐world face detection leading to inaccuracies when applied to the task of animated face detection (Kim et al., 2021). The resulting consensus regressors were then convolved with a canonical hemodynamic response function (Hassanpour et al., 2014) and z‐scored. In the current study, the same hemodynamic response function was used for both adult and child analysis. Both regressors and HD‐DOT data were filtered with the same high‐pass cutoff of 0.02 and low‐pass cutoff of 0.1 prior to correlation analysis.

To conduct univariate regressor analysis (e.g., simple speech mapping) for a run of data, voxel‐wise Pearson correlations rv,f were calculated across the run between the zero‐mean‐centered oxyhemoglobin signal timecourse Sv for the vth voxel and the regressor Rf for feature f, as in previous studies (Fishell et al., 2019):

For a more refined version of parallel feature mapping, voxel‐wise feature correlations were evaluated by calculating regression model coefficients using a multivariate linear regression approach. Multiple features of interest (speech and faces) and nuisance regressors (GVTD, audio envelope, luminance, temporal derivative of luminance, temporal derivative of squared luminance, hands, and bodies) were incorporated into a design matrix, X. The data Y were modeled mathematically as a function of X and a matrix of regression coefficients, β:

where the error term e represents zero‐mean Gaussian noise. The pseudoinverse of X was then used to estimate β from the data through the method of least squares:

In all cases, results were aggregated across runs by calculating random effects t‐statistics and plotting them on an atlas surface.

To evaluate how much temporal covariance between feature regressors could drive the spatial overlap between regressor maps, we computed the correlation M between the zero‐mean‐centered speech and face maps, S and F, as well as the correlation R between the speech and face regressors, s and f, for all runs across all participants. For the ith run in our data set: