Condition and outcome set calibration

The most important step in fsQCA is to transform the relevant outcome and conditions into new variables based on decision and definition rules. For this so-called calibration process, three anchors for the outcome and the investigated conditions need to be defined: full inclusion in the set of the new variable (corresponds to a membership score of 0.95), the crossover point with maximum ambiguity (membership score of 0.5), and full exclusion from the set of the new variable (non-membership score of 0.05). Based on in-depth knowledge of the cases and use of existing theory or prior research, the anchors are usually defined, making sense of variation in the data [34]. The primary stakeholders (funder and program team) together with the researchers developed a logic model for the intervention that guided the development of the decision rules described below [33] and listed in detail in Table 1. The same anchors were used for investigation of the pathways leading to both success and non-success. The first author defined the specific anchors and afterwards the members of the research team provided feedback to the anchors. Additionally, as part of the process evaluation we conducted qualitative interviews with the teachers of the intervention group of approximately 30 min length at the end of the intervention. As we used the information from the qualitative interviews only to verify the extreme cases of the quantitative questionnaire regarding the condition benefits of the intervention, we describe this methodological step briefly. With the consent of the participants, we recorded the semi-structured interviews and transcribed them verbatim, and ensured anonymity. Before we started the calibration process, we read through all the interviews to become familiar with the cases.

For calibrating the condition dosage of the intervention, we drew on arousal theories proposing an inverted U-shaped association between performance and arousal [43]. It is assumed that children aged seven to 10 years have the potential to concentrate for up to 20 min, after which a physical activity bout or relaxation activity is seen as necessary [44]. With 3 minutes of physical activity per academic hour, four academic hours per day, and 5 days a week, a total of 60 min of physical activity bouts per week should be recommended, defining the anchor for full membership. Providing physical activity breaks on average only once a day for 3 minutes was considered the anchor for non-membership.

Concerning the quality of the implementation condition, we defined the anchor for full membership when the trainers rated a teacher’s ability to conduct an active curriculum with the best quality rating twice (at the end of the first year and at the end of the program). If a teacher achieved at least twice the worst quality rating, full non-membership was stated.

We calibrated the condition relative advantage using the anchors 3.84 for full membership (i.e. allowing a minimum of skepticism regarding the benefits of the intervention, which means that one out of eight was rated 3.0) and 3.4 for full non-membership (i.e. at least five out of three items were rated 3.0). Overall, this reflects the evidence [24–26] that higher expectations regarding the benefits of the intervention would be associated with a greater enthusiasm for implementing the intervention.

We assigned the highest possible achievable value for perceived self-efficacy as the anchor for full membership with 10.0, drawing on the evidence that higher values lead to profounder implementation processes and better intervention outcomes [4, 24, 27, 28, 45]. We defined full non-membership with 6.1 as the anchor, where teachers already expressed substantial concerns about their confidence to implement the intervention properly.

For the condition knowledge about the intervention, we defined the anchors for full membership with 3.65, giving the possibility that teachers rating one item with 3.0 instead of 4.0 out of three items would still qualify for full membership. Therefore, teachers rating their knowledge as very high or almost very high qualified for full membership.

Finally, regarding the investigated outcome, we specified that a class would be fully included in the outcome set for success if 25% of the children in that specific class showed an improvement in emotional and social school experience. This low success rate was chosen in relation to the overall small improvements in the intervention group. The further anchors were defined so as to achieve similarly large groups for full non-membership as for full membership. It was defined as fully non-membership if 9% of the children showed an improvement in emotional and social school experience. The crossover point reflecting maximum ambiguity in membership was defined as 16%.

Calibration of Conditions and Outcome for Success^a

Abbreviations: BOI benefits of intervention, C Condition, O Outcome, KAI knowledge about the intervention, PAB physical activity breaks (dosage), PSE perceived self-efficacy, SCE, school-experience, QOI quality of implementation

^aThe same anchors were used for analyzing pathways of non-success. The full non-membership anchors for defining success relate to the full membership anchors for defining non-success and vice versa

~ The possible range, different values can take, is listed in Table ​Table22