Evaluation Phase

We evaluated this intervention in a double-blind RCT conducted in our (A.G.L., R.E.B., D.R.H.) three sections of introductory biology (N = 917). This study was conducted in the Fall of 2021, the first semester in which students had returned to in-person classes at this university after the onset of the COVID-19 pandemic. Across these three sections, 70% of students were women, 36% were from racial/ethnic groups that are underrepresented in the biomedical fields (i.e., Black, Hispanic/Latinx, Native American/Alaska Native, or Native Hawaiian/Pacific Islander), and 29% were first-generation college students (i.e., neither parent held a 4-year college degree).

The study procedure is summarized in Figure 2. Students were randomly assigned to either an intervention or a control condition within each course section. Before completing any experimental activities, students completed a short baseline survey (for a small amount of course credit) assessing their attributions of struggle to strategy usage, reliance on memorization in high school biology, feelings of belonging in the course, uncertainty about belonging in the biological and health sciences, and confidence about performing well in the course (see the Supplemental Material for items and scale reliabilities). In both conditions, students completed the three experimental activities over the course of the semester, which were included in the course as graded assignments for a small amount of completion credit.

Process for evaluating the challenge reattribution intervention. Introductory Biology I students were randomly assigned to complete three intervention or control activities throughout the Fall 2021 term. They then reported their attributions for struggle, approach to studying, and sense of belonging. Student performance data were collected from instructors at the end of the term, and persistence data were collected the following semester. We used a conservative Bayesian analysis to assess treatment effects.

In the intervention condition, students were shown the three intervention activities described earlier. In the control condition, students were shown three activities that were similar to the intervention activities in terms of surface features (each including background on the student interviews, a 5- to 7-minute video featuring interview clips from the same group of former students, and a brief writing exercise). However, the control activities did not include the critical theme of reattributing struggle in the course to developing study strategies, and instead focused on related topics about succeeding in and beyond the course. Notably, the first control activity focused on study strategies that helped former students to learn the material. The control condition was therefore conservative, ruling out exposure to improved study strategies (rather than reattributing struggle) as a mechanism of intervention effects. The second control activity focused on how students’ academic interests develop over time, and the third control activity focused on how students adjust to college life.

After the third experimental activity and before the final exam, students completed a survey (for a small amount of course credit) that measured their attributions for struggle, approach to studying in the course, and feelings of belonging. After the term, we (A.G.L., R.E.B., D.R.H.) provided students’ letter grades for analysis, as well as records of whether students took Introductory Biology II (i.e., the second course in the introductory biology sequence) the following semester.

The end-of-semester survey was completed by approximately three-quarters of the sample (missingness = 26–27%, depending on the outcome) and there was no differential attrition for any of the outcomes (see the Supplemental Material). All items were measured on a six-point strongly disagree–strongly agree Likert-type scale, unless otherwise noted. Consistent with recommended best practices (see Lovelace and Brickman, 2013), we ensured that internal consistency (Cronbach’s alpha) was high for multi-item scales. Note that we used short scales for each outcome (one to four face-valid items), because the survey was included as a brief course assignment and therefore had strict length limitations.

Attributions for struggle were measured on a five-point not at all likely to think this–extremely likely to think this Likert-type scale (adapted from Yeager et al., 2016). Students were instructed to: “Pretend that, later today or tomorrow, you got a bad grade on a very important assignment in this class. Honestly, if that happened, how likely would you be to think these thoughts?” Two items measured attributions to strategy usage (“I can get a higher score next time if I find a better way to study,” “I will need to change the way I prepare for exams in this class if I want to get a higher score”; r = 0.70, α = 0.82).

We tested two novel measures of students’ approaches to studying that were designed specifically for this study. First, we assessed students’ perceptions that it was important to experiment with study strategies in order to be successful in the course using one item (“It is important to experiment with new study strategies to be successful in [Introductory Biology I]”). Second, we assessed students’ reliance on the strategies they had used to study for high school biology with one item (“My approach to studying in this class has been similar to how I studied for biology in high school”).

Feelings of belonging were measured both at the level of the course and the field more generally. Course belonging was measured with four items on a six-point strongly disagree–strongly agree Likert-type scale (“I feel comfortable in this class,” “I feel accepted in this class,” “I feel like I can be myself in this class,” “I feel like I belong in this class”; α = 0.89; PERTS, 2022). Uncertainty about belonging in the biological and health sciences was measured with two items (“I don’t know if I really belong in the biological and health sciences,” “Sometimes I’m not sure if I really belong in the biological and health sciences”; r = 0.82, α = 0.90; adapted from Harackiewicz et al., 2014).

Performance in the course was assessed with letter grades obtained from official transcripts (“A” = 4.00, “A−” = 3.67, “B+” = 3.33, “B” = 3.00, “B−” = 2.67, “C+” = 2.33, “C” = 2.00, “C−” = 1.67, “D+” = 1.33, “D” = 1.0, “D−” = 0.67, “F” = 0.00). Thirty students in the sample did not have course grades because they withdrew from the course. Persistence was measured by assessing whether students took the second course in the introductory biology sequence the subsequent semester (continued to next course = 1, did not continue = 0), also obtained from official records.

Correlations and descriptive statistics for all outcomes are presented in Table 1. Correlations between survey measures, including the new measures designed for this study, were consistent with the patterns that would be expected with valid measures of the theoretical constructs (see the Supplemental Material for a deeper discussion of these associations).

Correlations and descriptive statistics for outcomes in the RCT

*p < 0.050.

***p < 0.001.