Survey

Student eligibility criteria includes being enrolled in one of the pilot schools as a year 9-13 student and able to consent and complete a web-based survey. For schools with a roll <100, all students of years 9-13 are eligible to participate. For schools with a larger roll, all students in 1 class in a given subject for each year (9-13) are selected. Class selection is negotiated with the school liaison and prioritizes the inclusion of Māori students. A new class of year 9 (aged 13 years) is selected in each school in year 2, term 1 of the study.

We will attend selected classrooms during class time and invite eligible students to participate in the research by completing 5 web-based surveys over 5 school terms. Students are asked to share their understanding of what is being asked of them. During data collection sessions, there are discreet options available for students choosing not to participate. Consenting students complete the baseline survey on school digital devices during class time, with researchers available to respond should taitamariki have any questions as they complete the survey. Sandwiches are provided to all students at the end of class time. Students can exit the survey and complete it later. The remaining 4 surveys are completed independently by students in response to email and text nudges. Nudges are sent during week 5 of the subsequent 4 terms. Students are free to withdraw at any time by emailing the project. Survey data are exported into a secure server at the Auckland University of Technology.

In preparation for the trial, the survey was pilot-tested in 2 schools (based on convenience) with approximately 60 students. This provided information on the time to complete the survey and identified the technical issues. Focus groups with students who completed the survey will be conducted to assess acceptability, comprehension, and appropriateness. Survey refinements will be made as indicated. Demographic characteristics include age, year in school, gender, ethnicity, internet exposure, and mobile phone access.

Students will complete web-based surveys at baseline (school term 1, year 1), 12 weeks (term 2, year 1), 24 weeks (term 3, year 1), 36 weeks (term 4, year 1), and 47 weeks (term 1, year 2). In selecting outcomes, the team considered the following: (1) positive, strengths-based measures (rather than deficit-based); (2) instruments developed with young people (rather than adult measures modified for young people); (3) instruments developed with indigenous young people; and (4) instruments developed with taitamariki Māori. At the time of study planning, there were no validated strengths-based measures developed with taitamariki Māori. The overwhelming majority of instruments used with young people measured dating or adolescent violence and had been modified from adult instruments.

Table 3 presents the final selection of the primary and secondary outcomes. All primary and secondary outcomes are assessed in each of the 5 surveys. Our primary outcome of interest is taitamariki RSE and includes 2 measures: confidence to talk about or seek help for themselves (RSE-self) and confidence to help others (RSE-others). The self-efficacy items are modeled on the Self-efficacy to Deal with Violence Scale [48,49]. In total, 2 items address RSE-self (How confident are you that you could check if parts of your relationship are ok, if you are not sure and How confident are you that you could seek help when your boyfriend or girlfriend has done something that’s not ok), and 2 items address RSE-others (How confident are you that you could help or support a friend or whānau member if they were not sure about parts of their relationships and How confident are you that you could help or support a friend or whānau member whose boyfriend or girlfriend has done something that’s not ok).

Harmonised outcome measures.

RSE-self (2)

RSE-others (2)

0=not at all confident

3=very confident

0-6

0-6

WB (5)

0=at no time to 5=all the time

0-25

General health (1)

0=poor to 4=excellent

0-4

Connectedness-family or whānau (5)

Connectedness-friends (5)

0=not at all true

4=very true

0-20

0-20

Cybersafety–being safe (7)

Cybersafety–taking action (8)

0=definitely not

3=definitely yes

0-21

0-24

^aRSE: relationship self-efficacy.

^bWB: well-being.

The Cyber-Safety questionnaire was developed from original research on young Māori women [57]. The 15-item questionnaire begins with a scenario modified from the Coping with Cyberbullying Questionnaire [56], as follows:

Imagine that for a few weeks, you have been receiving nasty and threatening text messages. Aside from that, you found out that embarrassing pictures of you are being spread around.

Taitamariki then respond how likely they would be to use each of the 15 strategies to keep yourself safe on the internet (eg, I would talk to my friends about it, I pay attention to who has access to my data). Other secondary outcomes include well-being, general health, and connectedness (Table 3).

The baseline data (preimplementation of the intervention acquired in all schools before the transition period) will be extensively analyzed, leading to a full analytical design. In particular, exploratory factor analysis of the outcomes will lead to the creation or confirmation of subscores, making RSE, connectedness, and cybersafety bivariate outcomes.

Schools are stratified by size (small or large). Large schools are further stratified by ongoing standardized delivery of a healthy relationship program (HRP). Within each stratum, the school labels are randomly ordered using a computer-generated sequence of pseudovariates. They are then assigned in this random order to the first sequence period (year 1, term 2: 2 small schools, 1 large school with HRP, and 1 large school with no HRP), and then the second sequence period (year 1, term 3: 1 small school, 2 large schools with HRP, and 1 large school with no HRP).

The data will be kept stratified within the clusters by gender, Māori versus non-Māori ethnicity (hereafter identified as ethnicity), and year-group. The year-group (years 9-13) is defined as usual for the first 4 periods and crosses over to the next nominal year-group in the fifth period, so that each year-group defines a subcohort followed over time. The year 9 group from period 5 is identified as a separate year-group. The analysis sets consist of intention-to-treat, as-treated, and adopter (students reporting app use) sets. All primary analyses will take place in the intention-to-treat set. General effectiveness hypotheses will be tested using a linear mixed model with nested participant, year-group, and school random effects. All models will initially be fitted with the function lme from the R package nlme [58]. If the results fail to converge or otherwise display poor numerical behavior, PROC MIXED from SAS/STAT version 9.4 (SAS Institute Inc) will be used instead. 

To ensure the overlap of the intervention and control in the design, data will be collected during the transition periods and included in the analysis, assuming an intervention effect half the size of that in the posttransition periods. This approach is nonstandard but necessary in this instance and broadly plausible under the conditions of implementation and the nature of the intervention.

Māori subgroup analyses are planned. They will consist of all primary analyses limited to Māori participants. Subgroup analyses will take place in the intention-to-treat and as-treated sets. It will extend to all outcomes covered by primary analyses.

A blind review of the data will take place (before allocation unblinding) to determine whether any transformation is necessary, to settle on the final models, and to determine whether any missing covariate or outcome data require multiple imputation, and generally to finalize the statistical analysis plan. All tests will be performed at a 5% significance level against 2-sided alternatives. There are no circumstances in which unblinding is permissible.

Recruitment of 8 schools and data collection over 5 terms is judged feasible, with app implementation (the intervention) scheduled at 2 time points (terms 2 and 3, respectively, in the first year). We assumed roughly equal numbers of participants from each school. We use the method of Hussey and Hughes [59] to compute the power for different effect sizes under a model including the primary outcome (RSE), school-related random effect, and fixed effect associated with the term. The sample size computation was programmed in R version 3.x (R Foundation for Statistical Computing) by the study statistician in accordance with the analysis plan, including the specification that the intervention effect is assumed to be halved during the transition period. Other covariates may be included in the model, as decided during the blind review of the data.

Assuming an attrition of 35% (conservatively applied to all assessment time points postbaseline) and using a school-specific intraclass correlation of 0.07, evidenced in a bullying study in New Zealand schools [60], we estimate that recruiting 600 students is sufficient for detecting an effect size of 0.25 with 83% power and an effect size of 0.30 with 94% power. These correspond respectively and approximately to a change of 0.75 and 0.9 in the mean score of either component of the RSE score, based on the baseline data.