fMRI experiment I: Modeling brain responses to emotional images
This protocol is extracted from research article:
Emotion schemas are embedded in the human visual system
Sci Adv, Jul 24, 2019; DOI: 10.1126/sciadv.aaw4358

Participants. We recruited 18 healthy, right-handed individuals (10 females, Mage = 25) from the Boulder area. As there were, to our knowledge, no previous studies relating activation in convolutional neural nets to human fMRI responses to emotional images, this sample size was not determined a priori. The experimental design focused on maximizing task-related signal within participants by showing them 112 affective images. Confirmatory post hoc analysis of effect size and the variance of parameter estimates corroborated that this sample size was sufficient for reliably detecting effects and minimizing the variance of parameter estimates (e.g., predicting EmoNet outcomes from occipital lobe activity using a random sample of only nine participants produced an average effect size of d = 3.08 and 95% CI of 2.08 to 4.36; see fig. S7). Participants did not meet the Diagnostic and Statistical Manual of Mental Disorders (DSM V) criteria for any psychological disorder and were screened to ensure safety in the MR environment. All participants provided informed consent before the experiment in accordance with the University of Colorado Boulder Institutional Review Board.

Experimental paradigm. In this experiment, brain activity was measured using fMRI, while participants viewed a series of emotional images. Stimuli were selected from the IAPS and the Geneva Affective PicturE Database (GAPED) using published normative arousal ratings to have either positive or negative valence and high arousal (26, 33, 57, 58). A total of 112 images were used for this experiment.

Image presentation lasted 4 s, with a jittered intertrial interval of 3 to 8 s [average internal interval (ITI), 4 s]. The scanning session was divided into two runs lasting 7.5 min, where the images were presented in a randomized order. Stimulus presentation was controlled using code written in MATLAB using the Psychophysics toolbox. Eye position was assessed during scanning using an EyeLink 1000 system (SR Research, Ottawa, Ontario, Canada) with a sampling rate of 1 kHz. Bivariate associations between the variance of eye position (i.e., the SD of lateral and vertical position) and EmoNet predictions were computed to confirm that eye movements were not highly correlated with model output.

MRI data acquisition. Gradient-echo echo-planar imaging (EPI) blood-oxygen-level-dependent (BOLD)–fMRI was performed on a 3-T Siemens MRI scanner (Siemens Healthcare). Functional images were acquired using multiband EPI sequence: echo time (TE), 30 ms; repetition time (TR), 765 ms; flip angle, 44°; number of slices, 80; slice orientation, coronal; phase encoding = h > f; voxel size, 1.6 mm × 1.6 mm × 2.0 mm; gap between slices, 0 mm; field of view, 191 mm × 191 mm; multiband acceleration factor, 8; echo spacing, 0.72 ms; bandwidth, 1724 Hz per pixel; partial Fourier in the phase encode direction, 7/8.

Structural images were acquired using a single-shot T1 MPRAGE sequence: TE, 2.01 ms; TR, 2.4 s; flip angle, 8°; number of slices, 224; slice orientation, sagittal; voxel size, 0.8 mm isotropic; gap between slices, 0 mm; field of view, 256 mm × 256 mm; GeneRalized Autocalibrating Partial Parallel Acquisition (GRAPPA) acceleration factor, 2; echo spacing, 7.4 ms; bandwidth, 240 Hz per pixel.

MRI preprocessing. Multiband brain imaging data were preprocessed following procedures used in the Human Connectome Project (59). This approach includes distortion correction, spatial realignment based on translation (in the transverse, sagittal, and coronal planes) and rotation (roll, pitch, and yaw), spatial normalization to MNI152 space using T1 data, and smoothing using a 6-mm full width at half maximum (FWHM) Gaussian kernel.

MRI analysis. Preprocessed fMRI data were analyzed using general linear models (GLMs) with Statistical Parametric Mapping (SPM8) software (Wellcome Trust Centre for Neuroimaging, UK). Separate models were estimated for each participant that included the following: (i) a regressor for every image presented to participants, modeled as a 4-s boxcar convolved with the canonical hemodynamic response function (HRF) of SPM; (ii) 24 motion covariates from spatial realignment (i.e., translation in x, y, and z dimensions; roll, pitch, and yaw; and their first- and second-order temporal derivatives); (iii) nuisance regressors specifying outlier time points, or “spikes,” that had large deviations in whole-brain BOLD signal; and (iv) constant terms to model the mean of each imaging session.

To identify mappings between patterns of brain activity and features of EmoNet, PLS regression models were fit on data from the entire sample (n = 18) using the full set of single-trial parameter estimates (112 trials for each participant) as input and activation in the last fully connected layer of EmoNet as the output (20 different variables, one per emotion category). We also conducted additional analyses using the same approach to predict high-dimensional patterns of activation from earlier layers of EmoNet (layers conv1 to conv5, fc6, and fc7). Model generalization (indicated by the correlation between observed and predicted outcomes and mean squared error) was estimated using leave-one-subject-out cross-validation. Inference on model performance was performed using permutation testing, where model features (i.e., activation in layer fc8) were randomly shuffled on each of 10,000 iterations. Performance relative to the noise ceiling was estimated by computing the ratio of cross-validated estimates to those using resubstitution (which should yield perfect performance in a noiseless setting; see Supplementary Text).

Inference on parameter estimates from PLS was performed via bootstrap resampling with 1000 replicates, using the means and SE of the bootstrap distribution to compute P values based on a normal distribution. Bootstrap distributions were visually inspected to verify that they were approximately normal. Thresholding of maps was performed using false discovery rate (FDR) correction with a threshold of q < 0.05. To visualize all 20 models in a low-dimensional space, principal component decomposition was performed on PLS regression coefficients on every bootstrap iteration to produce a set of orthogonal components and associated coefficients comprising a unique pattern of occipital lobe voxels. Procedures for inference and thresholding were identical to those used for parameter estimates, if only they were applied to coefficients from the PCA. Brain maps in the main figures are unthresholded for display. All results reported in the main text (and supplementary figures) survive FDR correction for multiple comparisons.

Note: The content above has been extracted from a research article, so it may not display correctly.



Q&A
Please log in to submit your questions online.
Your question will be posted on the Bio-101 website. We will send your questions to the authors of this protocol and Bio-protocol community members who are experienced with this method. you will be informed using the email address associated with your Bio-protocol account.



We use cookies on this site to enhance your user experience. By using our website, you are agreeing to allow the storage of cookies on your computer.