Paradigm and procedures

As described previously [13], a 2 × 2 study design was used, with factors of presentation (video vs. live) and emotion (positive vs. negative), resulting in four conditions (Fig 3). After obtaining written informed consent and the EMG electrodes were attached, the participant and model engaged in conversation for 3 minutes through the prompter. The conversation demonstrated to the participants that the prompter system transmitted live images of the model. In the behavioral experiment (50 participants), the conversation was followed by eight practice passive-viewing trials, 60 passive-viewing trials, four practice rating trials, and 16 rating trials. In the neuroimaging experiment (44 participants), there were 200 passive-viewing trials, four practice rating trials, and 16 rating trials. The practice trials gave participants an understanding of what to expect and allowed them to ask any questions before data acquisition began. The relative proportions of passive viewing (EMG) trials and rating trials aligned with our prior assumption about the effect size and signal-to-noise ratio of spontaneous facial mimicry and ratings. The EMG would have a smaller effect size and lower signal-to-noise ratio than ratings, thus requiring more repetitions per participant. However, participants might experience a loss of attention or interest if the passive viewing period persists for too long. The total testing time per session for the study’s behavioral and neuroimaging versions was less than 1 hour.

During the passive-viewing component, the participants fixated on a cross (behavioral experiment: mean inter-trial interval = 2,604 ms; range: 2,000–3,750 ms; neuroimaging experiment: mean inter-trial interval = 4,475 ms; range: 2,950–10,150 ms) until they were shown the announcement of either “Video” or “Real Person” for 1 s (1.25 s in the neuroimaging experiment). The instruction enabled a clear audience effect [59, 60, 63]. Participants did not need to de-ambiguate or infer a live vs. pre-recorded situation by relying on subtle cues such as microexpressions or environmental inconsistencies, which do not guarantee successful detection/judgment of live performances. Also, otherwise, it would have been difficult to ensure whether participants were really in mental states appropriate for live interactions. Without instructions, it would be more difficult to claim whether an effect (if any) was attributable to low-level processing rather than social cognition. During the video trials, one pre-recorded video clip was presented immediately after the instructions. During the live trials, the models performed dynamic facial expressions according to the instructions in their earphones. The screen displayed the fixation cross again after the facial expressions had been shown. In the behavioral experiment, after eight practice trials, 15 passive-viewing trials were completed per condition for a total of 60 trials, with a break after 32 trials (8 trials per condition). In the neuroimaging experiment, after eight practice trials, there were five runs of 40 trials (10 trials per condition) for a total of 200 trials. To avoid movement-associated EMG artifacts, and to detect spontaneous, automatic facial mimicry without top-down emotional processing, as in previous studies [83, 84], no rating was requested during the passive-viewing trials. For each participant, the sequence of conditions during the trial was pseudo-randomized, and the presentation sequence of the pre-recorded videos per condition was randomized. EMG data were collected during the passive-viewing trials.

After the passive-viewing trials, participants received instructions about and explanations of the subjective experiential ratings. During the subjective rating trials [76], each trial began with the described passive-viewing component; valence (pleasure-displeasure) and arousal ratings were completed sequentially after the participant had viewed dynamic facial expressions (Fig 3). Four practice rating trials were followed by 16 test-rating trials, during which no EMG data were acquired. For video conditions, the clips used in the passive viewing trials (15 per emotional condition) and rating trials (4 per emotional condition) were not repeated. After the ratings had been completed, the electrodes were removed; participants then completed the AQ and IRI questionnaires. Seventy-two participants (all participants tested in 2020 and 2021 and one participant in 2019) also completed the State-Trait Anxiety Inventory [85] and the Interaction and Audience Anxiousness Scale [86] (the resulting data were not analyzed in the present study). The participants were remunerated for their participation and then dismissed.