# Also in the Article

Procedure

Kriegeskorte et al. (2008) demonstrated that RDMs allow the direct comparison of neural representations between a monkey IT and human IT, although they used radically different measurement modalities for these two species (single-cell recording for monkeys and functional resonance imaging for humans). We used RDMs to compare the characteristics of the responses in the DCNN saliency map model with those of the neural representation in V1, V4, and IT.

We computed the representational dissimilarity (RD) between all pairs of natural object surfaces (Kriegeskorte et al., 2008; Hiramatsu et al., 2011; Goda et al., 2014) based on the firing rates of V1, V4, and IT neurons recorded by Tamura et al. (2016). To compute the RDMs, we standardized the mean firing rates based on the Gaussian distribution with a mean of zero and a variance of one with respect to each neuron in the visual cortices. We computed the representational dissimilarity RDv between two natural object surfaces (#i and j) with respect to the rates of V1, V4, and IT neurons based on the correlation distance as follows:

where v represents the visual cortices (V1, V4, or IT); i and j represent the natural object surface number ($1≤i,j≤64$); n is the identity of the neuron; $fn,iv$ represents the firing rates of the neuron n in the visual cortex v when the object surface #i is presented; and $fiv¯$ represents the mean rates of the neural population of v to the object surface #i. We computed the representational dissimilarity RDv(i, j) across the population of biological neurons in the monkeys (Kiani et al., 2007; Haxby et al., 2011). The RDv(i, j) exhibited an intensity value ranging from zero to two. If the neuronal response patterns for natural object surfaces i and j were identical, the intensity of the RDv(i, j) became zero. By contrast, the RDv(i, j) increased as the level of representational dissimilarity between response patterns for two stimuli increased. We computed the RDv(i, j) with respect to all 2016 pairs of natural object surfaces, which were summarized and represented as percentiles for each element of the RDMs (Kriegeskorte et al., 2008). Each element of the RDMs represented the comparison of the response patterns across neurons induced by two stimuli. Note that each RDM was symmetric, with a diagonal of zeros.

In the same manner, we computed the representational dissimilarity RDl between all input image pairs based on the activities of model neurons in the layer of the DCNN saliency map model as follows:

where l represents the layers in the DCNN saliency map model (Fig. 1A); $an,il$ represents the activities of model neuron n in layer l of the DCNN model with respect to the object surface i; and $ail¯$ represents the mean activities of the model neuron population of layer l to the object surface i. Note that we used all model neurons from all channels of each layer in the DCNN model to compute RDl(i, j). We summarized RDv(i, j) as shown in Equation 2.

We used Pearson’s correlation coefficient to quantify the correspondence between the RDMs for the monkey V1, V4, and IT and those for each layer of the DCNN saliency map model. The correspondence rvl between visual cortices and the DCNN saliency map model is defined as follows:

where v and l represent the visual cortex (V1, V4, or IT) and the layer in the DCNN saliency map model (Fig. 1A), respectively. We computed rvl using 2016 RDM elements representing response patterns with respect to distinct pairs of natural object surfaces. $RD¯$ represents the mean intensity of these 2016 RDM elements. Because the intensity of the diagonal elements of the RDM [RD(i, i)] became zero, we removed these diagonal elements from our analysis.

To understand the characteristics of the responses in the DCNN saliency map model in greater detail, we computed the partial correlation of RDMs between the specific visual cortex and each layer of the DCNN saliency map model, which removed the effects of other visual cortices. The partial correlation is defined as follows:

where rlx·y is the magnitude of the partial correlation between the activities of model neurons from the specific l-th DCNN layer (layer l) and the neuronal firing rates of visual cortex x required for removing the effect of visual cortex y; and rlx, rxy, and rly are the correlation of RDMs between the activities of DCNN model neurons in layer l and the rates of visual cortex x, between visual cortices x and y, and between model neurons in layer l and visual cortex y, respectively.

Note: The content above has been extracted from a research article, so it may not display correctly.

Q&A