Skip to main content
JMIR Research Protocols logoLink to JMIR Research Protocols
. 2021 Dec 30;10(12):e24792. doi: 10.2196/24792

Evaluation of a Healthy Relationship Smartphone App With Indigenous Young People: Protocol for a Co-designed Stepped Wedge Randomized Trial

Jane Koziol-McLain 1,, Denise Wilson 2, Alain C Vandal 3, Moana Eruera 4, Shyamala Nada-Raja 5, Terry Dobbs 1, Michael Roguski 6, Te Wai Barbarich-Unasa 1
Editor: Gunther Eysenbach
Reviewed by: Megan O'Connell, Andrew Copas
PMCID: PMC8759014  PMID: 34967750

Abstract

Background

We co-designed a smartphone app, Harmonised, with taitamariki (young people aged 13-17 years) to promote healthy intimate partner relationships. The app also provides a pathway for friends and family, or whānau (indigenous Māori extended family networks), to learn how to offer better support to taitamariki.

Objective

The aim of our taitamariki- and Māori-centered study is to evaluate the implementation of the app in secondary schools. The study tests the effectiveness of the app in promoting taitamariki partner relationship self-efficacy (primary outcome).

Methods

We co-designed a pragmatic, randomized, stepped wedge trial (retrospectively registered on September 12, 2019) for 8 Aotearoa, New Zealand, secondary schools (years 9 through 13). The schools were randomly assigned to implement the app in 1 of the 2 school terms. A well-established evaluation framework (RE-AIM [Reach, Effectiveness, Adoption, Implementation, Maintenance]) guided the selection of mixed data collection methods. Our target sample size is 600 taitamariki enrolled across the 8 schools. Taitamariki will participate by completing 5 web-based surveys over a 15-month trial period. Taitamariki partner relationship self-efficacy (primary outcome) and well-being, general health, cybersafety management, and connectedness (secondary outcomes) will be assessed with each survey. The general effectiveness hypotheses will be tested by using a linear mixed model with nested participant, year-group, and school random effects. The primary analysis will also include testing effectiveness in the Māori subgroup.

Results

The study was funded by the New Zealand Ministry of Business, Innovation, and Employment in October 2015 and approved by the Auckland University of Technology Ethics Committee on May 3, 2017 (application number: 17/71).

Conclusions

This study will generate robust evidence evaluating the impact of introducing a healthy relationship app in secondary schools on taitamariki partner relationship self-efficacy, well-being, general health, cybersafety management, and connectedness. This taitamariki- and indigenous Māori–centered research fills an important gap in developing and testing strengths-based mobile health interventions in secondary schools.

Trial Registration

Australian New Zealand Clinical Trials Registry ACTRN12619001262190; https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=377584

International Registered Report Identifier (IRRID)

RR1-10.2196/24792

Keywords: indigenous, Māori, young people, relationships, school, mHealth, smartphone app, mobile phone

Introduction

Study Rationale

Abuse in intimate partner relationships is a human rights violation and a social and public health problem [1,2]. In Aotearoa, New Zealand, 1 in 3 women experience physical or sexual violence by a partner, with rates higher for indigenous Māori (58%) compared with New Zealand European (34%) women [3,4]. For many, their first experience of relationship abuse is during adolescence (13-18 years) [5-9]. Relationship abuse may include psychological, physical, sexual, or cyber abuse threats [10-12]. There is a small but growing body of literature examining relationship abuse prevalence, prevention, and intervention during this critical period of adolescence, when individuals transition from childhood to adulthood [13-15]. Comprehensive, accessible, innovative, and cost-effective interventions are required to prevent intimate partner abuse among young people. However, frequently, research programs appear to be done on rather than with young people, and few studies provide an indigenous lens. Given the significant health and social inequities of indigenous Māori in Aotearoa, New Zealand, we embarked on a program of co-designed research to develop and evaluate a personalized healthy relationship smartphone app using a taitamariki- (young person) and Māori-centered approach. The RE-AIM (Reach, Effectiveness, Adoption, Implementation, Maintenance) framework, including qualitative and quantitative data collection methods, was used to examine the adoption and implementation of the app in 8 pilot secondary schools in New Zealand [16,17]. In this paper, we present a trial protocol for testing the effectiveness of the app.

Taitamariki-Centered Approach

As a team, we are committed to including the voice of young people in our work, consistent with their right to express opinions freely and have these considered in any matter that affects them [18,19]. We will convene taitamariki advisory groups (TAGs) to enable young people to cocreate the app using participatory research principles [20] and cocreation processes [21,22]. Before the formal trial, Māori team members experienced in child advocacy and focus group methodology (ME, TD, and TWBU) recruited and facilitated 2 cohorts of TAGs. TAG members and their parents provided written consent to participate in this research project. The initial tuakana (older) TAG included 7 taitamariki from 1 New Zealand region. At recruitment, they ranged in age from 15 to 17 years and included both taitamatane (boys; n=4) and taitamahine (girls; n=3); 5 of the 7 taitamariki self-identified as Māori. As the app was developed and trial was planned, meetings were convened kanohi-ki-te-kanohi (face-to-face), supplemented with communication via a private Facebook group. The second teina (younger) TAG members are recruited from our pilot schools across New Zealand and include 15 taitamariki (boys, n=4; girls, n=11). Communication includes kanohi-ki-te-kanohi meetings, videoconferencing, and interaction on Instagram. TAG members participated in project branding, app development, and trial design. The project branding Harmonised was chosen by the TAG. The TAG members will participate in the implementation, interpretation of findings, and dissemination.

Māori-Centered Approach

As a team, together with our community advisors, we are committed to a Māori-centered approach. At our first team hui (meeting), we specified our purpose, kaingākau (values), tikanga (right way of doing things in a Māori worldview), and whanonga pono (principles) to guide our mahi (work). A Māori-centered approach is premised on the bicultural relationship between Māori as tangata whenua (indigenous people of the land) and tauiwi (nonindigenous people) [23]. This research privileges a Māori worldview, ensuring that processes and outcomes are beneficial for Māori, inclusive of Māori values, expectations, and needs, and cedes control to protect Māori interests.

Early in the development of this research program, we took a philosophical turn from the mainstream deficit-based approach of reducing violence to a Māori well-being and strengths-based approach of promoting healthy relationships. From that point, we problematized our processes and decisions. For example, we identified that available validated measures focused on measuring adolescent relationship abuse (or dating violence) rather than measuring healthy relationships—the core focus of the research. In addition, identified measures were predominantly developed for adults and modified for young people, often without input from young people themselves. Finally, we could identify no measures for our variables of interest that represented a Māori or indigenous youth perspective. Therefore, in many respects, we are traveling uncharted territory, balancing cross-disciplinary and cross-cultural bodies of knowledge.

Why Mobile Health?

As people are increasingly seeking health information from the internet [24,25], mobile health interventions offer promise for improved health and well-being across the life span [26,27]. There is also evidence of value in the development of mobile health interventions aligned with indigenous frameworks [28-30]. In research with taitamariki Māori in Northland New Zealand about healthy and unhealthy intimate relationships, taitamariki expressed a need for more information and more effective support from friends and whānau [31,32]. However, rather than being lectured to or given advice when they haven’t asked for it, they suggested safe, easily accessible social media and web-based tools to use privately [33]. The Harmonised app is meant to address this information need identified by taitamariki Māori. The intent is to develop a resource that would supplement, rather than replace, other healthy relationship learning in schools, whānau, or families or communities. New Zealand secondary schools have a range of digital technology devices [34] and a Ministry of Education–funded Chrome Education License [35].

Harmonised Healthy Relationship App

The Harmonised healthy relationship smartphone app was developed consistent with our taitamariki- and Māori-centered approach. Development was iterative, involving repeated cycles of taitamariki input, app development, and testing. Our software engineer and app developer worked together in applying the specifications identified by taitamariki in focus groups, usability testing, and by our TAG members (Figure 1). Taitamariki specified their desire for an interactive, private social network that allows taitamariki users to (1) create their own relationship profile and choose their relationship values, (2) learn about common relationship issues (healthy relationship information and skill resources), and (3) post issues or comments about their relationships with others. Taitamariki control whether to make posts private (sharing only with selected safe people) or public (shared anonymously). Responding to the taitamariki desire for better support from their friends and whānau, the Harmonised app supports 2 user types: (1) primary users are taitamariki, with access to all app functions, and (2) secondary users are friends, whānau, or family and community who have access to healthy relationship information resources only, unless invited as a primary user’s safe person to comment on a specific post.

Figure 1.

Figure 1

Screenshot from the Apple App Store (version 1.8.3; August 2019).

A Harmonised moderation protocol guides safe practices for all posts (posts and comments), minimizing the risk of harm to primary app users, harmful use of the app, and harm to the platform. The protocol is aligned with best practice safety and security measures from Netsafe [36], Technology Safety [37], and others [21,38]. The moderation protocol includes automated (inappropriate language block and suicidality or self-harm automated message), human (daily review by the research team with the ability to block posted comments or individual users), community-based (users’ report inappropriate content), and reputation-based moderation. An escalation pathway identifies procedures, should moderators have a safety concern. The safe and responsible use of the app is communicated to all users. The protocol is aligned to the New Zealand Harmful Digital Communications Act 2015.

Healthy relationship resources are collated by a Harmonised resource working group to provide taitamariki with the information they have requested. The topics covered include, for example, communication, consent, dealing with anger, and how to talk with your parents or children about healthy relationships and sexuality. The working group followed a protocol for resource selection, modification, and permission for use. Resources include links to videos, quizzes, brief articles, and stories from a range of open-access sources. Where to get help is a recommended resource for all users. New Zealand and Māori resources were prioritized.

Following a beta test version available in June 2017, version updates continued with bug fixes and enhancements. In some cases, changes were made in response to requests by school staff (eg, ability for persons to accept or not accept a safe person’s invitation). As a prevention intervention, there is also attention to promoting the app for all taitamariki, regardless of relationship status. For example, presenting the interactive selection of relationship values as a requirement before advancing to the app feed allows all taitamariki to participate in the healthy relationship values exercise. We also use gender-diverse language, sensitive to the minority (estimated at 3.8%) of Aotearoa young people attracted to others of the same sex or both sexes [21,39,40].

The hybrid app in English and Te Reo Māori (translation by a certified Māori language interpreter) is available across the digital ecosystem, including mobile devices (smartphones and tablets), laptops, and desktop computers. The app was first published in June 2018 in the Apple App Store, Google Play Store, and web browser by the Auckland University of Technology Enterprises Ltd. To increase privacy, the app is personal identification number code–protected to keep information safe once a user moves away from the screen. To reduce the risk of harm from nontaitamariki inappropriately posting (eg, trolling), the posting function is limited to students enrolled at participating schools and validated during app registration against email lists provided by the schools.

Methods

Trial Purpose and Hypotheses

We are investigating the adoption and implementation of the interactive, personalized healthy relationship Harmonised app in 8 pilot secondary schools in New Zealand using the well-established RE-AIM evaluation framework [16,17]. Mixed method data sources include a quantitative web-based taitamariki survey, focus groups with taitamariki and school stakeholders (persons identified by schools who are involved in their response to taitamariki health and well-being), app use data, school engagement notes, and itemized costs (Table 1). In this paper, we present the trial protocol for testing the effectiveness of the Harmonised app using a quantitative web-based taitamariki survey.

Table 1.

Harmonised evaluation data sources guided by the Reach, Effectiveness, Adoption, Implementation, and Maintenance framework.

Implementation Quantitative data sources Qualitative data sources
Engage with schools to adopt the Harmonised app
  • Number of schools agreeing to participate, number of schools invited, and proportion of invited schools agreeing to participate

Drivers of and barriers to school participation:
  • School engagement log

  • School engagement notes

  • Preimplementation school information forms

  • Preimplementation focus groups

The app is implemented in schools
  • Number of schools implementing in accordance with random assignment

  • Costs of implementation (time and money)

Understanding implementation:
  • School implementation plans

  • Implementation run sheets

  • Postimplementation stakeholder focus groups

  • Postimplementation taitamariki focus groups

The app reaches the target population
  • Number and proportion of students in participating schools and others that download and use the app:

  • App download data (Firebase reports)

  • App use assessed in taitamariki survey postimplementation

Drivers of and barriers to app access and use:
  • Open-ended query in survey reason for not downloading

  • Postimplementation focus groups with taitamariki

  • Postimplementation focus groups with stakeholders

  • Whānau interviews

The app is effective at improving relationship and well-being outcomes
  • Primary and secondary outcomes assessed in web-based survey completed by taitamariki at 5 school terms over 18 months

Impacts of the app:
  • Postimplementation focus groups with taitamariki

Maintenance of the Harmonised app beyond the implementation period
  • App use data posttrial (Firebase)

Understanding long-term impacts and app retention:
  • Postimplementation hui (meeting)

We hypothesize (primary analysis) that implementation of the app (compared with before implementation) will (1) increase taitamariki relationship self-efficacy (RSE; primary outcome) and (2) increase taitamariki well-being, general health, cybersafety management, and connectedness (secondary outcomes). We will also conduct (3) primary analyses limited to Māori participants (subgroup analyses). Other measures include school and student characteristics and taitamariki help-seeking and barriers to getting help.

Trial Design

A pragmatic, stepped wedge, cluster randomized (one-directional crossover) trial in 8 pilot secondary schools tests the effectiveness of the Harmonised app. Secondary schools in New Zealand include years 9 through 13 (5 years of high school, generally young people aged 13 to 17 years). In the stepped wedge design, the app is implemented in all 8 schools in one of 2 school terms, with the order of implementation determined at random [41-44]. There are 5 assessments (web-based surveys) per school and 2 time steps over a 15-month trial period (Table 2). Our protocol description follows the CONSORT (Consolidated Standards of Reporting Trials) stepped wedge, cluster randomized trial extension [44]. The study was registered with the Australian New Zealand Clinical Trials Registry (ACTRN12619001262190) on September 12, 2019.

Table 2.

Stepped wedge implementation design plan.

School ID Number Year and school term

Year 1 Year 2, T1a

T1a T2a T3a T4a
8 CCb CC TPc ICd IC
7 CC CC TP IC IC
6 CC CC TP IC IC
5 CC CC TP IC IC
4 CC TP IC IC IC
3 CC TP IC IC IC
2 CC TP IC IC IC
1 CC TP IC IC IC

aT1 to T4 are the terms for the implementation of the design plan.

bCC: control condition.

cTP: transition period.

dIC: intervention condition.

Eligibility for Clusters: School Selection

Overview

The New Zealand Ministry of Education website provides school characteristics, including type of school, gender, ethnicity proportion, and other characteristics [45]. The 366 state schools in New Zealand include a mix of coeducational and boys- or girls-only schools across years 9 to 13. Although there are National Certificate of Educational Achievement unit standards for healthy relationships, their integration into secondary school curricula varies. Our inclusion criteria includes secondary or composite schools that had a school roll >175 (based on average class sizes for each year) and ≥16% Māori (proportion of Māori in the population; StatsNZ). This left 194 eligible schools. In this taitamariki- and Māori-centered study, we added 6 schools to the eligibility list that had a high proportion of Māori students and had participated in earlier kaupapa Māori research in the Ngāpuhi tribal boundaries within the Northland region of New Zealand [33].

School Recruitment

From the list of eligible schools, we will purposefully invite schools until we achieve our desired sample of 8 schools. We aim to balance girls-only, boys-only, and coeducational schools. We also aim for a balance of schools that have and have not participated in specialized healthy relationship curricula such as Mates and Dates (Accident Compensation Corporation) that are being piloted [46,47]. Finally, we will consider the geographic diversity and travel feasibility.

Implementation Conditions

The intervention consists of the implementation of the Harmonised app in each participating secondary school. An app implementation plan is being developed jointly with each school following a stakeholder focus group. A one-off single-period (40-50 minutes) implementation delivery occurs within individual classrooms or during school assembly. Participating schools may elect for app implementation sessions to be school-led or researcher-led. A hard-copy training resource designed specifically for the study supports implementation and includes instructions regarding app navigation and function. Researcher-led training includes demonstrations of app functions. Students are led through the process of registration, creation of user profiles, and creation of posts. Strategies to support app adoption include the use of posters and school newsletters. The intended frequency and duration of use of the app beyond the initial guided implementation is at the students’ discretion. The tailored nature of the implementation limits the evaluation of fidelity.

Comparison Condition

The comparison period includes school terms before app implementation from the clustered one-directional crossover design. In the preimplementation comparison period, researchers engage with an identified school liaison, convene a preimplementation focus group, and confirm the school’s pathway for referral of students needing health and well-being services, including social workers, counselors, and health providers. No control period activities that affect school students will be undertaken.

Survey

Eligibility for Survey Participants: Student Selection Within Schools

Student eligibility criteria includes being enrolled in one of the pilot schools as a year 9-13 student and able to consent and complete a web-based survey. For schools with a roll <100, all students of years 9-13 are eligible to participate. For schools with a larger roll, all students in 1 class in a given subject for each year (9-13) are selected. Class selection is negotiated with the school liaison and prioritizes the inclusion of Māori students. A new class of year 9 (aged 13 years) is selected in each school in year 2, term 1 of the study.

Student Recruitment and Retention for the School Survey

We will attend selected classrooms during class time and invite eligible students to participate in the research by completing 5 web-based surveys over 5 school terms. Students are asked to share their understanding of what is being asked of them. During data collection sessions, there are discreet options available for students choosing not to participate. Consenting students complete the baseline survey on school digital devices during class time, with researchers available to respond should taitamariki have any questions as they complete the survey. Sandwiches are provided to all students at the end of class time. Students can exit the survey and complete it later. The remaining 4 surveys are completed independently by students in response to email and text nudges. Nudges are sent during week 5 of the subsequent 4 terms. Students are free to withdraw at any time by emailing the project. Survey data are exported into a secure server at the Auckland University of Technology.

Survey Development and Pilot-Testing

In preparation for the trial, the survey was pilot-tested in 2 schools (based on convenience) with approximately 60 students. This provided information on the time to complete the survey and identified the technical issues. Focus groups with students who completed the survey will be conducted to assess acceptability, comprehension, and appropriateness. Survey refinements will be made as indicated. Demographic characteristics include age, year in school, gender, ethnicity, internet exposure, and mobile phone access.

Outcomes

Students will complete web-based surveys at baseline (school term 1, year 1), 12 weeks (term 2, year 1), 24 weeks (term 3, year 1), 36 weeks (term 4, year 1), and 47 weeks (term 1, year 2). In selecting outcomes, the team considered the following: (1) positive, strengths-based measures (rather than deficit-based); (2) instruments developed with young people (rather than adult measures modified for young people); (3) instruments developed with indigenous young people; and (4) instruments developed with taitamariki Māori. At the time of study planning, there were no validated strengths-based measures developed with taitamariki Māori. The overwhelming majority of instruments used with young people measured dating or adolescent violence and had been modified from adult instruments.

Table 3 presents the final selection of the primary and secondary outcomes. All primary and secondary outcomes are assessed in each of the 5 surveys. Our primary outcome of interest is taitamariki RSE and includes 2 measures: confidence to talk about or seek help for themselves (RSE-self) and confidence to help others (RSE-others). The self-efficacy items are modeled on the Self-efficacy to Deal with Violence Scale [48,49]. In total, 2 items address RSE-self (How confident are you that you could check if parts of your relationship are ok, if you are not sure and How confident are you that you could seek help when your boyfriend or girlfriend has done something that’s not ok), and 2 items address RSE-others (How confident are you that you could help or support a friend or whānau member if they were not sure about parts of their relationships and How confident are you that you could help or support a friend or whānau member whose boyfriend or girlfriend has done something that’s not ok).

Table 3.

Harmonised outcome measures.

Outcomes Sources and modifications Scales and subscales (number of items) Response options Possible score (range)
Primary outcomes

RSEa Items modelled on Self-efficacy to Deal with Violence Scale [48,49]
  • RSE-self (2)

  • RSE-others (2)

  • 0=not at all confident

  • 3=very confident

  • 0-6

  • 0-6

Secondary outcomes

WBb World Health Organization-Five Well-Being Index [50] used in children [51,52] and in New Zealand youth [53,54]. Modified “I have felt active and vigorous” to “I have felt active and full of energy”
  • WB (5)

  • 0=at no time to 5=all the time

  • 0-25


General health Single 5-point Likert scale to rate respondent’s general health
  • General health (1)

  • 0=poor to 4=excellent

  • 0-4


Connectedness Retained 2 subscales from Hemingway Measure of Adolescent Connectedness [55] with language regionalized and negatively worded items not scored
  • Connectedness-family or whānau (5)

  • Connectedness-friends (5)

  • 0=not at all true

  • 4=very true

  • 0-20

  • 0-20


Cybersafety 15-item questionnaire began with a scenario modified from the Coping with Cyberbullying Questionnaire [56]; items are from original research with young Māori women [57]
  • Cybersafety–being safe (7)

  • Cybersafety–taking action (8)

  • 0=definitely not

  • 3=definitely yes

  • 0-21

  • 0-24

aRSE: relationship self-efficacy.

bWB: well-being.

The Cyber-Safety questionnaire was developed from original research on young Māori women [57]. The 15-item questionnaire begins with a scenario modified from the Coping with Cyberbullying Questionnaire [56], as follows:

Imagine that for a few weeks, you have been receiving nasty and threatening text messages. Aside from that, you found out that embarrassing pictures of you are being spread around.

Taitamariki then respond how likely they would be to use each of the 15 strategies to keep yourself safe on the internet (eg, I would talk to my friends about it, I pay attention to who has access to my data). Other secondary outcomes include well-being, general health, and connectedness (Table 3).

The baseline data (preimplementation of the intervention acquired in all schools before the transition period) will be extensively analyzed, leading to a full analytical design. In particular, exploratory factor analysis of the outcomes will lead to the creation or confirmation of subscores, making RSE, connectedness, and cybersafety bivariate outcomes.

Randomization

Schools are stratified by size (small or large). Large schools are further stratified by ongoing standardized delivery of a healthy relationship program (HRP). Within each stratum, the school labels are randomly ordered using a computer-generated sequence of pseudovariates. They are then assigned in this random order to the first sequence period (year 1, term 2: 2 small schools, 1 large school with HRP, and 1 large school with no HRP), and then the second sequence period (year 1, term 3: 1 small school, 2 large schools with HRP, and 1 large school with no HRP).

Analysis

The data will be kept stratified within the clusters by gender, Māori versus non-Māori ethnicity (hereafter identified as ethnicity), and year-group. The year-group (years 9-13) is defined as usual for the first 4 periods and crosses over to the next nominal year-group in the fifth period, so that each year-group defines a subcohort followed over time. The year 9 group from period 5 is identified as a separate year-group. The analysis sets consist of intention-to-treat, as-treated, and adopter (students reporting app use) sets. All primary analyses will take place in the intention-to-treat set. General effectiveness hypotheses will be tested using a linear mixed model with nested participant, year-group, and school random effects. All models will initially be fitted with the function lme from the R package nlme [58]. If the results fail to converge or otherwise display poor numerical behavior, PROC MIXED from SAS/STAT version 9.4 (SAS Institute Inc) will be used instead. 

To ensure the overlap of the intervention and control in the design, data will be collected during the transition periods and included in the analysis, assuming an intervention effect half the size of that in the posttransition periods. This approach is nonstandard but necessary in this instance and broadly plausible under the conditions of implementation and the nature of the intervention.

Māori subgroup analyses are planned. They will consist of all primary analyses limited to Māori participants. Subgroup analyses will take place in the intention-to-treat and as-treated sets. It will extend to all outcomes covered by primary analyses.

A blind review of the data will take place (before allocation unblinding) to determine whether any transformation is necessary, to settle on the final models, and to determine whether any missing covariate or outcome data require multiple imputation, and generally to finalize the statistical analysis plan. All tests will be performed at a 5% significance level against 2-sided alternatives. There are no circumstances in which unblinding is permissible.

Sample Size

Recruitment of 8 schools and data collection over 5 terms is judged feasible, with app implementation (the intervention) scheduled at 2 time points (terms 2 and 3, respectively, in the first year). We assumed roughly equal numbers of participants from each school. We use the method of Hussey and Hughes [59] to compute the power for different effect sizes under a model including the primary outcome (RSE), school-related random effect, and fixed effect associated with the term. The sample size computation was programmed in R version 3.x (R Foundation for Statistical Computing) by the study statistician in accordance with the analysis plan, including the specification that the intervention effect is assumed to be halved during the transition period. Other covariates may be included in the model, as decided during the blind review of the data.

Assuming an attrition of 35% (conservatively applied to all assessment time points postbaseline) and using a school-specific intraclass correlation of 0.07, evidenced in a bullying study in New Zealand schools [60], we estimate that recruiting 600 students is sufficient for detecting an effect size of 0.25 with 83% power and an effect size of 0.30 with 94% power. These correspond respectively and approximately to a change of 0.75 and 0.9 in the mean score of either component of the RSE score, based on the baseline data.

Ethics and Safety

The trial protocol was approved by the Auckland University of Technology Ethics Committee (application number: 17/71), approved on May 3, 2017. All schools have a pathway for students needing health and well-being services, including social workers, counselors, and health providers. Consent to participate is provided by each school’s principal and Board of Trustees. Schools follow their processes for sharing information about the study with parents and gaining parental consent (information and consent forms provided by researchers). Students choosing to participate in the survey provide consent in the introduction to the web-based survey that details confidentiality. Students were identified using a randomized code number. All communication and visits with schools are documented by the research staff and reviewed by senior investigators as contextual data and to audit trial conduct. Our Harmonised ethical research practice and data sharing protocol provides a process for accessing Harmonised data aligned with our Māori-centered approach, available on request. Guided by our tikanga, we will prioritize taitamariki, schools, and whānau to disseminate our findings.

Results

The study was funded by the New Zealand Ministry of Business, Innovation, and Employment in October 2015 and approved by the Auckland University of Technology Ethics Committee on May 3, 2017 (application number: 17/71). A total of 8 schools were recruited, and data were collected over 5 school terms.

Discussion

Principal Findings

The Harmonised trial will generate robust evidence to evaluate the impact of introducing a healthy relationship app into secondary schools. Importantly, this strengths-based taitamariki- and Māori-centered research counters the dominant adult-focused and deficit-based intimate partner violence literature. Working with taitamariki and community advisors, we have created the Harmonised brand focusing on what young people have told us about healthy partner relationships. The Harmonised app provides a safe digital network with resources for taitamariki to consider the values that are important to them for a healthy intimate partner relationship.

Conclusions

This pragmatic trial offers an opportunity and challenge to understand whether a healthy relationship digital resource can be embedded in the secondary school environment and whether the resource benefits taitamariki, particularly Māori. Secondary schools are busy places that are typically underresourced to meet all the complex needs to support students and families to flourish. Our Harmonised study guided by explicit tikanga and using mixed methods guided by the RE-AIM framework will make an important contribution to understanding drivers of and barriers to conducting research in this unique setting. Our trial measures will identify whether the introduction of the app improves taitamariki partner RSE, well-being, general health, cybersafety management, and connectedness.

Acknowledgments

The authors acknowledge the contribution of the taitamariki and community advisory group members in challenging them along the journey to be true to their tikanga. The New Zealand Ministry of Business, Innovation, and Employment funded the study. They did not have a role in the design of the study (protocol) or writing of the manuscript.

Abbreviations

CONSORT

Consolidated Standards of Reporting Trials

HRP

healthy relationship program

RE-AIM

Reach, Effectiveness, Adoption, Implementation, Maintenance

RSE

relationship self-efficacy

TAG

taitamariki advisory group

Footnotes

Authors' Contributions: JKM is the principal investigator; she conceived the study with DW, ACV, ME, SNR, and TD. ACV contributed their statistical expertise in the trial design and analysis plan. DW, ME, TD, MR, and TWBU contributed to the study of the indigenous Māori caucus. All authors contributed to study protocol refinement and the interpretation of the findings. All authors read and approved the final manuscript.

Conflicts of Interest: None declared.

References

  • 1.World Health Organization . Violence Against Women Prevalence Estimates, 2018: Global, Regional and National Prevalence Estimates for Intimate Partner Violence Against Women and Global and Regional Prevalence Estimates for Non-partner Sexual Violence Against Women. Geneva: World Health Organization; 2021. pp. 1–87. https://apps.who.int/iris/handle/10665/341337 . [Google Scholar]
  • 2.Lim SS, Vos T, Flaxman AD, Danaei G, Shibuya K, Adair-Rohani H, AlMazroa MA, Amann M, Anderson HR, Andrews KG, Aryee M, Atkinson C, Bacchus LJ, Bahalim AN, Balakrishnan K, Balmes J, Barker-Collo S, Baxter A, Bell ML, Blore JD, Blyth F, Bonner C, Borges G, Bourne R, Boussinesq M, Brauer M, Brooks P, Bruce NG, Brunekreef B, Bryan-Hancock C, Bucello C, Buchbinder R, Bull F, Burnett RT, Byers TE, Calabria B, Carapetis J, Carnahan E, Chafe Z, Charlson F, Chen JS, Chen JS, Cheng AT, Child JC, Cohen A, Colson KE, Cowie BC, Darby S, Darling S, Davis A, Degenhardt L, Dentener F, Jarlais DC, Devries K, Dherani M, Ding EL, Dorsey ER, Driscoll T, Edmond K, Ali SE, Engell RE, Erwin PJ, Fahimi S, Falder G, Farzadfar F, Ferrari A, Finucane MM, Flaxman S, Fowkes FG, Freedman G, Freeman MK, Gakidou E, Ghosh S, Giovannucci E, Gmel G, Graham K, Grainger R, Grant B, Gunnell D, Gutierrez HR, Hall W, Hoek HW, Hogan A, Hosgood HD, Hoy D, Hu H, Hubbell BJ, Hutchings SJ, Ibeanusi SE, Jacklyn GL, Jasrasaria R, Jonas JB, Kan H, Kanis JA, Kassebaum N, Kawakami N, Khang Y, Khatibzadeh S, Khoo J, Kok C, Laden F, Lalloo R, Lan Q, Lathlean T, Leasher JL, Leigh J, Li Y, Lin JK, Lipshultz SE, London S, Lozano R, Lu Y, Mak J, Malekzadeh R, Mallinger L, Marcenes W, March L, Marks R, Martin R, McGale P, McGrath J, Mehta S, Memish ZA, Mensah GA, Merriman TR, Micha R, Michaud C, Mishra V, Hanafiah KM, Mokdad AA, Morawska L, Mozaffarian D, Murphy T, Naghavi M, Neal B, Nelson PK, Nolla JM, Norman R, Olives C, Omer SB, Orchard J, Osborne R, Ostro B, Page A, Pandey KD, Parry CD, Passmore E, Patra J, Pearce N, Pelizzari PM, Petzold M, Phillips MR, Pope D, Pope CA, Powles J, Rao M, Razavi H, Rehfuess EA, Rehm JT, Ritz B, Rivara FP, Roberts T, Robinson C, Rodriguez-Portales JA, Romieu I, Room R, Rosenfeld LC, Roy A, Rushton L, Salomon JA, Sampson U, Sanchez-Riera L, Sanman E, Sapkota A, Seedat S, Shi P, Shield K, Shivakoti R, Singh GM, Sleet DA, Smith E, Smith KR, Stapelberg NJ, Steenland K, Stöckl H, Stovner LJ, Straif K, Straney L, Thurston GD, Tran JH, Van Dingenen R, van Donkelaar A, Veerman JL, Vijayakumar L, Weintraub R, Weissman MM, White RA, Whiteford H, Wiersma ST, Wilkinson JD, Williams HC, Williams W, Wilson N, Woolf AD, Yip P, Zielinski JM, Lopez AD, Murray CJ, Ezzati M. A comparative risk assessment of burden of disease and injury attributable to 67 risk factors and risk factor clusters in 21 regions, 1990–2010: a systematic analysis for the Global Burden of Disease Study 2010. Lancet. 2012 Dec 15;380(9859):2224–60. doi: 10.1016/S0140-6736(12)61766-8. http://europepmc.org/abstract/MED/23245609 .S0140-6736(12)61766-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Fanslow J, Robinson E. Violence against women in New Zealand: prevalence and health consequences. N Z Med J. 2004 Nov 26;117(1206):U1173. [PubMed] [Google Scholar]
  • 4.Fanslow J, Robinson E, Crengle S, Perese L. Juxtaposing beliefs and reality: prevalence rates of intimate partner violence and attitudes to violence and gender roles reported by New Zealand women. Violence Against Women. 2010 Jul;16(7):812–31. doi: 10.1177/1077801210373710.16/7/812 [DOI] [PubMed] [Google Scholar]
  • 5.Addressing Violence Against Women and Achieving the Millennium Development Goals. Geneva, Switzerland: Department of Gender, Women and Health Family and Community Health; 2005. [Google Scholar]
  • 6.Sears HA, Byers ES, Whelan JJ, Saint-Pierre M, Dating Violence Research Team "If it hurts you, then it is not a joke": adolescents' ideas about girls' and boys' use and experience of abusive behavior in dating relationships. J Interpers Violence. 2006 Sep;21(9):1191–207. doi: 10.1177/0886260506290423.21/9/1191 [DOI] [PubMed] [Google Scholar]
  • 7.Breiding MJ, Chen J, Black MC. Intimate partner violence in the United States - 2010. National Center for Injury Prevention and Control, Centers for Disease Control and Prevention. 2014. [2020-02-26]. https://www.cdc.gov/violenceprevention/pdf/cdc_nisvs_ipv_report_2013_v17_single_a.pdf .
  • 8.Stöckl H, March L, Pallitto C, Garcia-Moreno C, WHO Multi-country Study Team Intimate partner violence among adolescents and young women: prevalence and associated factors in nine countries: a cross-sectional study. BMC Public Health. 2014 Jul 25;14:751. doi: 10.1186/1471-2458-14-751. https://bmcpublichealth.biomedcentral.com/articles/10.1186/1471-2458-14-751 .1471-2458-14-751 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Breiding MJ, Smith SG, Basile KC, Walters ML, Chen J, Merrick MT. Prevalence and characteristics of sexual violence, stalking, and intimate partner violence victimization--national intimate partner and sexual violence survey, United States, 2011. MMWR Surveill Summ. 2014 Sep 05;63(8):1–18. https://www.cdc.gov/mmwr/preview/mmwrhtml/ss6308a1.htm .ss6308a1 [PMC free article] [PubMed] [Google Scholar]
  • 10.Zweig JM, Dank M, Lachman P, Yahner J. Technology, teen dating violence and abuse, and bullying. Urban Institute, Washington DC. 2013. [2020-06-16]. http://www.urban.org/sites/default/files/publication/23941/412891-Technology-Teen-Dating-Violence-and-Abuse-and-Bullying.PDF .
  • 11.Draucker CB, Martsolf D, Stephenson P, Risko J, Heckman T, Sheehan D, Perkins S, Washington K, Cook C, Ferguson C. Aggressive events in adolescent dating violence. Issues Ment Health Nurs. 2010 Sep;31(9):599–610. doi: 10.3109/01612841003793056. [DOI] [PubMed] [Google Scholar]
  • 12.Jackson S. Abuse in dating relationships: young people's accounts of disclosure, non-disclosure, help-seeking and prevention education. NZ J Psychol. 2002;31(2):79–86. https://www.researchgate.net/publication/279594931_Abuse_in_dating_relationships_Young_people's_accounts_of_disclosure_non-isclosure_help-seeking_and_prevention_education . [Google Scholar]
  • 13.Taquette SR, Monteiro DL. Causes and consequences of adolescent dating violence: a systematic review. J Inj Violence Res. 2019 Jul;11(2):137–47. doi: 10.5249/jivr.v11i2.1061. doi: 10.5249/jivr.v11i2.1061.1061 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Crooks CV, Jaffe P, Dunlop C, Kerry A, Exner-Cortens D. Preventing gender-based violence among adolescents and young adults: lessons from 25 years of program development and evaluation. Violence Against Women. 2019 Jan;25(1):29–55. doi: 10.1177/1077801218815778. [DOI] [PubMed] [Google Scholar]
  • 15.Patton GC, Sawyer SM, Santelli JS, Ross DA, Afifi R, Allen NB, Arora M, Azzopardi P, Baldwin W, Bonell C, Kakuma R, Kennedy E, Mahon J, McGovern T, Mokdad AH, Patel V, Petroni S, Reavley N, Taiwo K, Waldfogel J, Wickremarathne D, Barroso C, Bhutta Z, Fatusi AO, Mattoo A, Diers J, Fang J, Ferguson J, Ssewamala F, Viner RM. Our future: a Lancet commission on adolescent health and wellbeing. Lancet. 2016 Jun 11;387(10036):2423–78. doi: 10.1016/S0140-6736(16)00579-1.S0140-6736(16)00579-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Gaglio B, Shoup JA, Glasgow RE. The RE-AIM framework: a systematic review of use over time. Am J Public Health. 2013 Jun;103(6):38–46. doi: 10.2105/AJPH.2013.301299. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Gaglio B, Phillips SM, Heurtin-Roberts S, Sanchez MA, Glasgow RE. How pragmatic is it? Lessons learned using PRECIS and RE-AIM for determining pragmatic characteristics of research. Implement Sci. 2014 Aug 28;9:96. doi: 10.1186/s13012-014-0096-x. https://implementationscience.biomedcentral.com/articles/10.1186/s13012-014-0096-x .s13012-014-0096-x [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Alderson P, editor. Young Children's Rights Exploring Beliefs, Principles and Practice, 2nd Ed. London, United Kingdom: Jessica Kingsley Publishers in association with Save the Children; 2008. [Google Scholar]
  • 19.Hart RA. Children's Participation: From Tokenism to Citizenship. Florence - Italy: UNICEF International Child Development Centre; 1992. [Google Scholar]
  • 20.Wallerstein N, Duran B, Oetzel JG, Minkler M, editors. Community-Based Participatory Research for Health: Advancing Social and Health Equity, 3rd Edition. San Francisco: Jossey-Bass; 2017. [Google Scholar]
  • 21.Whitehouse SR, Lam P, Balka E, McLellan S, Deevska M, Penn D, Issenman R, Paone M. Co-creation with TickiT: designing and evaluating a clinical ehealth platform for youth. JMIR Res Protoc. 2013 Oct 18;2(2):e42. doi: 10.2196/resprot.2865. https://www.researchprotocols.org/2013/2/e42/ v2i2e42 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Witell L, Kristensson P, Gustafsson A, Löfgren M. Idea generation: customer co‐creation versus traditional market research techniques. J Serv Manag. 2011 Apr 26;22(2):140–59. doi: 10.1108/09564231111124190. [DOI] [Google Scholar]
  • 23.Hudson M, Milne M, Reynolds P, Russell K, Smith B. Te Ara Tika Guidelines for Māori Research Ethics: A Framework for Researchers Ethics Committee Members. Auckland: Health Research Council of New Zealand; 2010. [Google Scholar]
  • 24.Diaz Andrade A, Hedges MR, Karimikia H, Techatassanasoontorn A. World Internet Project: the internet in New Zealand 2017. New Zealand Work Research Institute (NZWRI) 2018. [2021-10-06]. https://workresearch.aut.ac.nz/__data/assets/pdf_file/0009/174915/WIP-2017.pdf .
  • 25.Smith P, Bell A, Miller M, Crothers C. Internet Trends in New Zealand 2007–2015. Auckland, New Zealand: Institute of Culture, Discourse & Communication, Auckland University of Technology; 2016. [Google Scholar]
  • 26.Merry SN, Stasiak K, Shepherd M, Frampton C, Fleming T, Lucassen MF. The effectiveness of SPARX, a computerised self help intervention for adolescents seeking help for depression: randomised controlled non-inferiority trial. Br Med J. 2012 Apr 18;344:e2598. doi: 10.1136/bmj.e2598. http://www.bmj.com/lookup/pmidlookup?view=long&pmid=22517917 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Stasiak K, Hatcher S, Frampton C, Merry SN. A pilot double blind randomized placebo controlled trial of a prototype computer-based cognitive behavioural therapy program for adolescents with symptoms of depression. Behav Cogn Psychother. 2014 Jul;42(4):385–401. doi: 10.1017/S1352465812001087.S1352465812001087 [DOI] [PubMed] [Google Scholar]
  • 28.Jones L, Jacklin K, O'Connell ME. Development and use of health-related technologies in Indigenous communities: critical review. J Med Internet Res. 2017 Jul 20;19(7):e256. doi: 10.2196/jmir.7520. https://www.jmir.org/2017/7/e256/ v19i7e256 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.McCallum GB, Versteegh LA, Morris PS, Mckay CC, Jacobsen NJ, White AV, D'Antoine HA, Chang AB. Mobile phones support adherence and retention of Indigenous participants in a randomised controlled trial: strategies and lessons learnt. BMC Public Health. 2014 Jun 18;14:622. doi: 10.1186/1471-2458-14-622. https://bmcpublichealth.biomedcentral.com/articles/10.1186/1471-2458-14-622 .1471-2458-14-622 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Koziol-McLain J, Vandal AC, Wilson D, Nada-Raja S, Dobbs T, McLean C, Sisk R, Eden KB, Glass NE. Efficacy of a web-based safety decision aid for women experiencing intimate partner violence: randomized controlled trial. J Med Internet Res. 2018 Jan 10;19(12):e426. doi: 10.2196/jmir.8617. https://www.jmir.org/2018/12/e426/ v19i12e426 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Eruera M, Dobbs T. Taitamariki Māori Kōrero About Intimate Partner Relationships. Whangarei, New Zealand: Amokura Family Vilence Prevention Consortium; 2010. [Google Scholar]
  • 32.Eruera M. Supporting taitamariki Māori in the development of healthy partner relationships. Doctoral Dissertation, Te Whare Wananga o Awanuiarangi, Whakatane, New Zealand. 2015. [2021-10-06]. https://tinyurl.com/rtsw2s2x .
  • 33.Eruera M. ‘Hooked up’ - Te hononga whaiaipo reducing and preventing violence in taitamariki Māori intimate partner relationships. Te Whare Wananga o Awanuiarangi, Whakatāne, New Zealand. 2015. [2021-10-06]. https://tinyurl.com/2k3dynmn .
  • 34.Johnson M, Maguire J, Wood A. Digital technologies in schools 2016-2017. Research New Zealand. 2017. [2021-10-06]. https://2020.org.nz/wp-content/uploads/2014/05/Digital-Technologies-in-Schools-2016-17-04-05-2017-FINAL.pdf .
  • 35.Google software for schools. Ministry of Education, Wellington, New Zealand. 2020. [2021-08-29]. https://www.education.govt.nz/school/digital-technology/software/google/#computers .
  • 36.Netsafe. [2021-08-29]. https://www.netsafe.org.nz/
  • 37.National Network to End Domestic Violence. Technology Safety. [2021-08-29]. https://www.techsafety.org/
  • 38.Lewis TL, Wyatt JC. Mhealth and mobile medical apps: a framework to assess risk and promote safer use. J Med Internet Res. 2014 Sep 15;16(9):e210. doi: 10.2196/jmir.3133. https://www.jmir.org/2014/9/e210/ v16i9e210 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Lucassen MF, Clark TC, Denny SJ, Fleming TM, Rossen FV, Sheridan J, Bullen P, Robinson EM. What has changed from 2001 to 2012 for sexual minority youth in New Zealand? J Paediatr Child Health. 2015 Apr 10;51(4):410–8. doi: 10.1111/jpc.12727. [DOI] [PubMed] [Google Scholar]
  • 40.Lucassen MF, Hatcher S, Fleming TM, Stasiak K, Shepherd MJ, Merry SN. A qualitative study of sexual minority young people's experiences of computerised therapy for depression. Australas Psychiatry. 2015 Jun;23(3):268–73. doi: 10.1177/1039856215579542.1039856215579542 [DOI] [PubMed] [Google Scholar]
  • 41.Brown CA, Lilford RJ. The stepped wedge trial design: a systematic review. BMC Med Res Methodol. 2006 Nov 08;6:54. doi: 10.1186/1471-2288-6-54. https://bmcmedresmethodol.biomedcentral.com/articles/10.1186/1471-2288-6-54 .1471-2288-6-54 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Mdege ND, Man M, Brown CA, Torgerson DJ. Systematic review of stepped wedge cluster randomized trials shows that design is particularly used to evaluate interventions during routine implementation. J Clin Epidemiol. 2011 Sep;64(9):936–48. doi: 10.1016/j.jclinepi.2010.12.003.S0895-4356(10)00435-X [DOI] [PubMed] [Google Scholar]
  • 43.Hargreaves JR, Prost A, Fielding KL, Copas AJ. How important is randomisation in a stepped wedge trial? Trials. 2015 Aug 16;16:359. doi: 10.1186/s13063-015-0872-1. https://trialsjournal.biomedcentral.com/articles/10.1186/s13063-015-0872-1 .10.1186/s13063-015-0872-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Hemming K, Haines TP, Chilton PJ, Girling AJ, Lilford RJ. The stepped wedge cluster randomised trial: rationale, design, analysis, and reporting. Br Med J. 2015 Feb 06;350:h391. doi: 10.1136/bmj.h391. http://www.bmj.com/lookup/pmidlookup?view=long&pmid=25662947 . [DOI] [PubMed] [Google Scholar]
  • 45.Education counts - New Zealand schools. Ministry of Education, Wellington, New Zealand. [2021-09-21]. https://www.educationcounts.govt.nz/data-services/directories/list-of-nz-schools .
  • 46.Appleton-Dyer S, Dale-Gandar L, Adams J, Ansari Z. Accident Compensation Corporation, Wellington, New Zealand. 2018. [2021-10-06]. https://www.matesanddates.co.nz/assets/resources/ACC-Mates-Dates-Survey-Summary-Handout-190416.pdf .
  • 47.Rape prevention education. Bodysafe. [2021-08-29]. https://www.bodysafe.nz/#home .
  • 48.Van Camp T, Hébert M, Guidi E, Lavoie F, Blais M, Members of the PAJ Team Teens' self-efficacy to deal with dating violence as victim, perpetrator or bystander. Int Rev Vict. 2014 Sep;20(3):289–303. doi: 10.1177/0269758014521741. http://europepmc.org/abstract/MED/26807554 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 49.Cameron CA, Byers ES, Miller SA, McKay SL, St. Pierre M, Glenn S. Dating violence prevention in New Brunswick. Muriel McQueen Fergusson Centre For Family Violence Research, University Of New Brunswick, Canada. 2007. [2021-10-06]. https://unbscholar.lib.unb.ca/islandora/object/unbscholar%3A9051/datastream/PDF/view .
  • 50.Topp CW, Østergaard SD, Søndergaard S, Bech P. The WHO-5 well-being index: a systematic review of the literature. Psychother Psychosom. 2015;84(3):167–76. doi: 10.1159/000376585. https://www.karger.com?DOI=10.1159/000376585 .000376585 [DOI] [PubMed] [Google Scholar]
  • 51.Allgaier A, Pietsch K, Frühe B, Prast E, Sigl-Glöckner J, Schulte-Körne G. Depression in pediatric care: is the WHO-five well-being index a valid screening instrument for children and adolescents? Gen Hosp Psychiatry. 2012;34(3):234–41. doi: 10.1016/j.genhosppsych.2012.01.007.S0163-8343(12)00009-6 [DOI] [PubMed] [Google Scholar]
  • 52.de Wit M, Pouwer F, Gemke RJ, van de Waal HA, Snoek FJ. Validation of the WHO-5 well-being index in adolescents with type 1 diabetes. Diabetes Care. 2007 Aug;30(8):2003–6. doi: 10.2337/dc07-0447.dc07-0447 [DOI] [PubMed] [Google Scholar]
  • 53.Fleming TM, Clark T, Denny S, Bullen P, Crengle S, Peiris-John R, Robinson E, Rossen FV, Sheridan J, Lucassen M. Stability and change in the mental health of new zealand secondary school students 2007-2012: results from the national adolescent health surveys. Aust N Z J Psychiatry. 2014 May;48(5):472–80. doi: 10.1177/0004867413514489. https://journals.sagepub.com/doi/10.1177/0004867413514489?url_ver=Z39.88-2003&rfr_id=ori:rid:crossref.org&rfr_dat=cr_pub%3dpubmed .0004867413514489 [DOI] [PubMed] [Google Scholar]
  • 54.Yallop K, McDowell H, Koziol-McLain J, Reed PW. Self-reported psychosocial wellbeing of adolescent childhood cancer survivors. Eur J Oncol Nurs. 2013 Dec;17(6):711–9. doi: 10.1016/j.ejon.2013.06.007.S1462-3889(13)00073-2 [DOI] [PubMed] [Google Scholar]
  • 55.Karcher M. Hemingway: measure of adolescent connectedness. Adolescent Connectedness. 2011. [2021-10-06]. https://adolescentconnectedness.com/
  • 56.Sticca F, Machmutow K, Stauber A, Perren S, Palladino B, Nocentini A, Menesini E, Corcoran L, Guckin C. The coping with cyberbullying questionnaire: development of a new measure. Societies. 2015 May 28;5(2):515–36. doi: 10.3390/soc5020515. [DOI] [Google Scholar]
  • 57.Wilson D, Cootes K, Mikahere-Hall A, Sherwood J, Berryman K, Jackson D. Reflecting and learning: a grounded theory on reframing deficit views of young indigenous women and safety. Health Care Women Int. 2020 Jun;41(6):690–708. doi: 10.1080/07399332.2019.1621316. [DOI] [PubMed] [Google Scholar]
  • 58.Pinheiro J, Bates D, DebRoy S, Sarkar D, R Core Team nlme: Linear and nonlinear mixed effects models. R Package Version 3.1-144. 2020. [2021-10-06]. https://cran.r-project.org/web/packages/nlme/index.html .
  • 59.Hussey MA, Hughes JP. Design and analysis of stepped wedge cluster randomized trials. Contemp Clin Trials. 2007 Feb;28(2):182–91. doi: 10.1016/j.cct.2006.05.007.S1551-7144(06)00063-2 [DOI] [PubMed] [Google Scholar]
  • 60.Denny S, Peterson ER, Stuart J, Utter J, Bullen P, Fleming T, Ameratunga S, Clark T, Milfont T. Bystander intervention, bullying, and victimization: a multilevel analysis of new zealand high schools. J School Violence. 2014 Sep 17;14(3):245–72. doi: 10.1080/15388220.2014.910470. [DOI] [Google Scholar]

Articles from JMIR Research Protocols are provided here courtesy of JMIR Publications Inc.

RESOURCES