Abstract
Background
Eliminating violence against children is a prominent policy goal, codified in the Sustainable Development Goals, and parenting programs are one approach to preventing and reducing violence. However, we know relatively little about dissemination and scale-up of parenting programs, particularly in low- and middle-income countries (LMICs). The scale-up of two parenting programs, Parenting for Lifelong Health (PLH) for Young Children and PLH for Parents and Teens, developed under Creative Commons licensing and tested in randomized trials, provides a unique opportunity to study their dissemination in 25 LMICs.
Methods
The Scale-Up of Parenting Evaluation Research (SUPER) study uses a range of methods to study the dissemination of these two programs. The study will examine (1) process and extent of dissemination and scale-up, (2) how the programs are implemented and factors associated with variation in implementation, (3) violence against children and family outcomes before and after program implementation, (4) barriers and facilitators to sustained program delivery, and (5) costs and resources needed for implementation.
Primary data collection, focused on three case study projects, will include interviews and focus groups with program facilitators, coordinators, funders, and other stakeholders, and a summary of key organizational characteristics. Program reports and budgets will be reviewed as part of relevant contextual information. Secondary data analysis of routine data collected within ongoing implementation and existing research studies will explore family enrolment and attendance, as well as family reports of parenting practices, violence against children, child behavior, and child and caregiver wellbeing before and after program participation. We will also examine data on staff sociodemographic and professional background, and their competent adherence to the program, collected as part of staff training and certification.
Discussion
This project will be the first study of its kind to draw on multiple data sources and methods to examine the dissemination and scale-up of a parenting program across multiple LMIC contexts. While this study reports on the implementation of two specific parenting programs, we anticipate that our findings will be of relevance across the field of parenting, as well as other violence prevention and social programs.
Keywords: Evidence-based practice implementation, Parenting, Violence against children, Scale-up, Dissemination
Contributions to the literature.
We describe a plan for learning from the widespread dissemination and scale-up of two parenting programs in a range of low- and middle-income country settings. Few previous studies have explored this scale of dissemination of a social program.
The study will combine primary qualitative and organizational data, as well as secondary quantitative data collected by implementing agencies and other research teams.
The results will provide insights into dissemination and implementation across the implementation stages identified in the Exploration, Preparation, Implementation and Sustainment framework.
Background
The United Nations Sustainable Development Goal 16 has set the aspiration to “end abuse, exploitation, trafficking and all forms of violence against and torture of children” (target 16.2). This is no small challenge: over a billion children each year experience some form of violence, with most of the burden in low- and middle-income countries (LMICs) [1, 2]. Parenting programs are one of the seven strategies to prevent and reduce violence against children identified in the INSPIRE guidelines led by the World Health Organization (WHO) [3–5]. Parenting programs involve group-based or individual family meetings that aim to build effective parenting practices, strengthen positive parenting, reduce harsh or violent parenting, and improve child outcomes. Delivery of parenting programs may also help address multiple global goals such as reducing preventable deaths of infants and children under age five (target 3.2), reducing substance use (target 3.5), and violence (target 16.1) among children and young people, improving child’s and caregiver’s mental health (target 3.4), and promoting education (targets 4.1) [6–9].
There is growing interest in the scale-up of parenting programs [10–12]. Scale-up can be conceptualized as reaching wider geographical areas and more people, as well as embedding delivery into lasting systems, for instance, by integrating new programs into existing service systems [13–17]. In the field of parenting programs, as in other areas, many research-informed initiatives are not taken up widely [18, 19]. When programs are indeed taken up, several questions arise. One frequent concern is whether transporting a program from one cultural context to another may reduce family engagement and program effectiveness [20]. A recent systematic review found that parenting programs may have comparable or even greater effectiveness when implemented in regions and with populations different from where they were developed, with relatively minimal content adaptation [21, 22]. Yet not all transported family programs show effects [23, 24]. Another key question is whether program effects achieved in research trials can be replicated in routine services [25, 26]. Studies of parenting programs implemented across entire areas have shown that establishing and maintaining quality delivery is a challenge if programs are delivered by overburdened volunteers or staff [27–30].
Parenting for Lifelong Health (PLH) is an initiative led by individuals from the WHO, UNICEF, and the Universities of Bangor, Cape Town, Oxford, and Stellenbosch. It aims to develop, test, and scale-up parenting programs to reduce violence against children and improve child wellbeing in LMICs. Since starting work in 2012, PLH supported a suite of low-cost, Creative Commons-licensed parenting programs, including developing PLH for Young Children (2 to 9 years), and PLH for Parents and Teens (10 to 17 years). Both programs are group-based and can be supplemented with home visits. Group sessions normally take place in community venues, such as village halls. Initially tested in randomized controlled trials in South Africa with promising results [31–33], the programs have subsequently been tested by the developers and other researchers in multiple countries (e.g., the Philippines and Thailand; [34–36]). Program manuals are freely available online [37].
These two programs have experienced rapid dissemination in over 25 LMICs in sub-Saharan Africa, Eastern Europe, Southeast Asia, and the Caribbean (Czech Republic, Democratic Republic of the Congo, Federal Democratic Republic of Ethiopia, Kingdom of Eswatini, Kingdom of Lesotho, Kingdom of Thailand, Malaysia, Montenegro, Republic of Botswana, Republic of Cameroon, Republic of Côte d’Ivoire, Republic of Haiti, Republic of India, Republic of Kenya, Republic of Malawi, Republic of Moldova, Republic of North Macedonia, Republic of South Africa, Republic of South Sudan, Republic of the Philippines, Republic of Uganda, Republic of Zambia, Republic of Zimbabwe, Romania, and United Republic of Tanzania).
Delivery has been led by non-governmental organizations working with varying numbers of families. For example, in South Sudan, delivery by Catholic Relief Services reached several hundred families in 2017–2018 [38], while Pact Tanzania worked with 16,000 families in 11 districts in 2018–2019 and plans to work with approximately 47,800 families in 2020–2021 in 8 districts of Tanzania. In several countries, including Montenegro, South Africa, the Philippines, and Thailand, governments have led delivery. In many cases, PLH programs have been integrated into packages of services, such as the existing government conditional cash transfer system in the Philippines [34]. Adaptations of the programs have been tailored for a variety of populations, such as families reunited after children exit residential care in Kenya, and families with orphans and vulnerable children as well as families of adolescent girls in the context of HIV prevention in Tanzania.
In 2020, as part of the COVID-19 response, core program components were converted into tip sheets for families with an interagency collaboration including the WHO, UNICEF, UNODC, USAID, the Centers for Disease Control and Prevention, and the Global Partnership to End Violence Against Children, as well as NGOs [39]. Focusing on parenting during lockdown and school closures, the tips have been disseminated to an estimated over 72 million families in 178 countries.
Two non-profit organizations, Clowns Without Borders South Africa (CWBSA), and the Children’s Early Intervention Trust (CEIT) in Wales provide the main support in Africa and Europe, respectively. PLH uses a cascading dissemination model in which CWBSA and CEIT (i.e., the program purveyors [40]) transfer technical capacity to implementing agencies. Agencies interested in implementing PLH typically request technical support, which includes the following:
Adapting the programs to fit the local context and culture;
Conducting an implementation readiness assessment;
Providing materials and tools for implementation, monitoring, and evaluation;
Training frontline service providers including program facilitators, coaches, trainers, and coordinators;
Assessing and certifying personnel.
PLH facilitators range from community volunteers to professional psychologists depending on the context. Facilitators deliver the program to families—caregivers only for PLH for Young Children, and caregivers and their teens for PLH for Parents and Teens—and receive ongoing supportive supervision during delivery. To ensure high-quality program delivery, facilitator certification is done via structured live or video observations and is a requirement to be eligible to be trained as a PLH coach; coaches (usually professionals) provide ongoing support to facilitators. Likewise, PLH coaches are assessed for certification, which is a requirement before being trained as a PLH trainer who is licensed to train facilitators and coaches.
The global uptake of the PLH programs provides an unprecedented opportunity to explore questions of scale-up across multiple LMICs. As a blueprint for the Scale-Up of Parenting Evaluation Research (SUPER) study, this paper will (1) describe research questions and their rationale, (2) outline study structure and methods, and (3) discuss challenges and opportunities.
Research questions
The study will use the Exploration, Preparation, Implementation and Sustainment (EPIS) framework [41, 42]. EPIS is a comprehensive model of implementation stages and their determinants. It organizes the study of implementation into the following stages: (1) exploration—the process of intervention selection; (2) preparation—setting up for implementation; (3) implementation—delivery of evidence-based practices and ongoing monitoring of the implementation process; and (4) sustainment—the process of program embedding in ongoing services. We use these stages to inform data collection, and analysis—for instance, to structure our key informant interview and focus group questions.
The SUPER study focuses on the following research questions.
(1) What is the process and extent of dissemination and scale-up of PLH programs?
Dissemination of evidence-informed practices is at the heart of implementation research. It is important to understand how and why PLH has been taken up by funders and implementers since this will inform dissemination of similar programs in future and of the same programs in new settings [18]. The dissemination approach may also influence the quality and sustainment of the program. We will map the types of organizations, contexts, populations, and service combinations where PLH has been adopted, as well as identifying several cases where it was considered but not taken up.
(2) How are the PLH programs implemented in various contexts and what are the factors associated with variation in implementation?
Program fidelity and adaptation exist in constant tension [43]. While strict adherence to protocols may ensure fidelity to the program as it was originally tested, adaptation is often considered necessary to fit new settings and populations [44]. In addition to formal systematic adaptation, adaptation often occurs during delivery in a more ad hoc, informal manner. Facilitators may change session content to address the concerns of the families with whom they work. Due to external factors, such as funding cycles or agricultural seasons, implementation agencies may adjust intervention length or facilitator recruitment processes. This cannot only lead to intervention drift [45] but also to innovations and maturation that could strengthen program outcomes [46]. During the COVID-19 pandemic, PLH adaptations have included remote training of implementation staff and remote provision of family support instead of in-person. To understand how PLH is implemented at scale, we will examine data on program delivery, adaptation, and participant engagement, and factors associated with these implementation outcomes.
(3) Are there changes in violence against children and other targeted family outcomes, following program delivery?
Previous studies of parenting programs within routine delivery have examined family outcomes using various data sources. These include examining administrative data from social services in the USA [28], data collected by teachers in Scotland [29], families completing questionnaires in England [47], and data collected by research assistants in Kenya from several hundred families [48]. These studies have found mixed results in terms of changes in family outcomes. To the best of our knowledge, no large-scale data analysis of family outcomes has been reported in LMICs during ongoing parenting program delivery. We will conduct a secondary analysis of family-level pre-post outcome data collected by implementing partners in an estimated 5–8 countries through existing monitoring systems.
(4) What are barriers and facilitators to sustainment of PLH programs?
Since the two PLH programs have been implemented for only a few years, it is important to examine the prospects and challenges for sustainment, such as via integration into existing service structures [45]. For example, factors such as community support and leadership consistency were identified as important for sustaining delivery of an early years maternal and child health intervention in South Africa [19]. We will examine from interviews and organizational records whether organizations are continuing to deliver PLH after the initial work with program purveyors is completed. We will also explore, using interviews and focus groups, the stakeholder perceptions of barriers and facilitators to continued delivery.
(5) What are the costs and resources needed for PLH delivery?
An essential first step to financial and resource sustainability is understanding the costs in practice and at scale. While it is possible to extrapolate from implementation with smaller numbers of families, costs at scale may differ [49]. There may be economies of scale, for instance, when facilitators deliver multiple rounds of the program after a single training. Therefore, we will explore the costs and resources required for PLH delivery, as well as how the PLH program delivery is currently being funded.
Study design
This study will use a mixed-methods approach. The research questions, and corresponding implementation phases and data collection methods are outlined in Table 1. The research team will collect primary data in a sub-sample of implementation settings. Data will also be collected by implementing agencies and intervention purveyors as part of the program delivery and may be shared with the research team in a de-identified or anonymized form for secondary data analysis. Although the study timeline is 2020–2024, some of the secondary data has already been collected by implementing agencies and research partners. There can be a tension between pre-specifying research plans and being responsive to dynamic implementation realities. Therefore, further methods and questions may be added, as the study evolves in collaboration with stakeholders.
Table 1.
Type of data | Data collectors | Data collection method | Study participants | EPIS phase | Research questions |
---|---|---|---|---|---|
Primary data | SUPER research team | Interviews and focus groups | Purveyor trainers and program specialists |
Preparation Implementation |
1,2,4,5 |
Donor agency staff |
Exploration Preparation Implementation Sustainment |
1,2,4 | |||
Local, regional and national policymakers |
Exploration Preparation Implementation Sustainment |
1,2,4 | |||
Other local stakeholders |
Exploration Preparation Implementation Sustainment |
1,2,3,4 | |||
Interviews and focus groups | Program coordinators, directors |
Exploration Preparation Implementation Sustainment |
1,2,3,4,5 | ||
Monitoring and evaluation team of the implementing organization |
Implementation Sustainment |
2,3,4,5 | |||
PLH facilitators, coaches, trainers |
Preparation Implementation Sustainment |
2,3,4 | |||
Surveys on organizational characteristics | Program coordinators, directors |
Implementation Sustainment |
1,2,4 | ||
Budgets, surveys, interviews on costs/resources for delivery | Program coordinators, directors |
Implementation Sustainment |
5 | ||
Primary and secondary data | SUPER research team, implementers, purveyors | Document review of other background materials | Implementing organization, local area |
Exploration Preparation Implementation Sustainment |
1,2,3,4,5 |
Secondary data | Implementers (usually collected by program facilitators and other research teams) | Family reports of parenting practices, child behavior, child and caregiver mental health | Families participating in the PLH programs |
Implementation Sustainment |
3 |
Implementers (usually collected by program facilitators) | Family enrolment, attendance, engagement, and dropout |
Implementation Sustainment |
2,3 | ||
Purveyors | Surveys on the sociodemographic and professional background of staff delivering the program | Facilitators, coaches |
Implementation Sustainment |
2 | |
Assessments of competent adherence to the program |
Implementation Sustainment |
2 | |||
Program documentation, reports | All |
Preparation Implementation |
1,2,3,4,5 |
Study sample
All organizations implementing PLH for Young Children and PLH for Parents and Teens, as well as researchers of formal studies, will be invited to contribute to the study. To allow a more in-depth understanding of our research questions, we will examine three case studies in depth, selected based on variation along the following dimensions: (1) geographical region, (2) level of government involvement in delivery, and (3) number of families reached [18, 50]. Given that the program development and most dissemination have been in the African continent, the case studies will be in Africa.
Study recruitment and ethical procedures
The study has received ethical approval from the Universities of Oxford (SPICUREC1a__20_015) and Cape Town (PSY2017-040). We have obtained country-level ethics clearance from 15 participating countries, and further, country-level ethics will be collected as the study recruits additional projects. We will seek agreement from each of the organizations involved in the project before approaching individuals within the organization. For primary data collection (Table 1), informed consent will be obtained by the research team. In respect to data shared for secondary analysis, implementation organizations and research teams contributing data to the project are responsible for ensuring that families and staff can consent or opt-out from being a part of the standard monitoring and evaluation data collection, as appropriate. Before any existing data are shared by organizations or investigators, data sharing agreements will be signed; this includes the requirement that partners will anonymize datasets before sharing them.
Data collection tools and methods
Implementation and outcome measures
Implementation staff demographic data
Partner organizations and intervention purveyors collect demographic data about PLH facilitators and coaches. Demographic data will include questions on age, gender, marital status, parental status, number and age of children, employment status, previous experience, and educational level. The questionnaire will also assess facilitator/coach self-efficacy, and attitude to corporal punishment.
Facilitator competent adherence
Facilitator competent adherence to the program can be defined as following the activities outlined in the program manual while also exercising clinical and teaching skills and judgment [51]. Competent adherence to the program will be evaluated using fidelity checklists of session activities and the PLH-Facilitator Assessment Tool (PLH-FAT) [52]. The PLH-FAT is an observation assessment tool used either live during group sessions or with video recordings. The PLH-FAT was developed by the study investigators and program developers to assess the proficiency of program delivery by facilitators as a prerequisite to certification. Seven standard behavior categories outlined in the program manuals are grouped into two scales based on (1) core activities and (2) process skills outlined in the program manuals.
Coach competent adherence
Coach competent adherence to the program will be collected using a similar observational assessment tool, the PLH-Coach Assessment Tool (PLH-CAT). This tool assesses the quality of coaching provided to facilitators based on either live observations or video recordings of coaching sessions, including an activity subscale on the discussion of highlights and challenges and similar items for process skills as in the PLH-FAT.
Family enrollment and attendance
Enrollment refers to the share of recruited caregivers (and adolescents in PLH for Parents and Teens) who attend at least one session. Attendance refers to the number of sessions, out of the total possible number, attended by an enrolled participant. For the PLH programs, an overall attendance rate is calculated by using both group sessions and home visits, as well as an additional group session rate. Enrollment and attendance data are collected using organizational recruitment lists and attendance registers.
Family outcomes
Program purveyors offer organizations implementing PLH a set of family outcome measurement tools. Organizations are encouraged to add these questionnaires to their monitoring and evaluation data collection if they already have a system in place for family data collection, for instance, on HIV risk factors in the context of sub-Saharan Africa. The measures recommended for PLH implementers focus on caregiver and child behaviors and wellbeing (Table 2). The tools were chosen because they assess the core outcomes the PLH programs target and are freely available. PLH for Young Children surveys are parent-reported, while PLH for Parents and Teens has both parent- and adolescent-report versions. Versions of different lengths (short, medium, and long) allow implementers to select the most appropriate version for their needs. Family questionnaires are usually collected by intervention facilitators at the first and last program sessions, with follow-up sessions at home, where possible, in case of missed sessions.
Table 2.
Domain | Source | Respondents | ||
---|---|---|---|---|
Parents of young children | Parents of adolescents | Adolescents | ||
Demographics | Selected details (e.g., age, gender, education) | ✓ | ✓ | ✓ |
Involved parenting | Alabama Parenting Questionnaire involved parenting subscale [53] | ✓ | ✓ | ✓ |
Parental monitoring | Alabama Parenting Questionnaire parental monitoring subscale [53] | ✓ | ✓ | ✓ |
Child behavior | Strengths and Difficulties Questionnaire [54] | ✓ | ✓ | ✓ |
Child mental health | Strengths and Difficulties Questionnaire [54] | N/A | N/A | ✓ |
Harsh discipline | ISPCAN Child Abuse Screening Tool [55, 56] | ✓ | ✓ | ✓ |
Parenting stress | Parenting Stress Scale [57] | ✓ | ✓ | N/A |
Acceptability of corporal punishment | Multiple Indicator Cluster Survey item [58] | ✓ | ✓ | N/A |
Parental depression | Centre for Epidemiological Studies-Depression [59] | ✓ | ✓ | N/A |
Risk avoidance | Risk avoidance planning scale (Developed specifically for Sinovuyo Teen intervention, based on [60] | ✓ | ✓ | ✓ |
Economic strengthening | Family financial coping scale (Developed specifically for Sinovuyo Teen intervention, based on [61–63] | ✓ | ✓ | ✓ |
Parental support of education | Parental Support for Education Scale [64] | ✓ | ✓ | ✓ |
Note: The table presents the most comprehensive version of the questionnaires (“long”). The questionnaires are available on the study’s OSF page
Organizational data.
Organizational surveys
To characterize the organizations delivering PLH programs, and explore any variation in uptake, implementation, and sustainment between them, we will distribute a short survey to these organizations to capture basic organizational characteristics, such as size and focus of activity.
Policies, protocols, and progress reports
Researchers will review locally relevant policies, protocols, and progress reports to identify the context that affects PLH implementation. Information on the implementation process will be extracted from progress reports and other routine documents, such as the implementation readiness assessment, where available. Formal requests will be sent to participating organizations to obtain permission to review relevant documents.
Costing questionnaires for organizations
We aim to collect the program set-up and implementation costs from at least the case study countries to examine how much the delivery costs in different contexts. We will review program budgets and collect data directly from the organizations directly about resources involved, based on the Global Health Consortium and J-PAL guidance [65, 66], as well as previous analyses of the PLH for Parents and Teens costs [67].
Qualitative data
Key informant interviews
Semi-structured interviews will be conducted with representatives of governments, funding agencies, implementing partners, program facilitators, coaches, and trainers. Interviews will be held in person or virtually following interview guides, aiming for approximately 4–8 interviews per implementation site or until saturation is reached.
Focus group discussions
Focus group discussions of between 6 and 12 participants will be conducted with PLH program facilitators and supervisors. The number of focus groups per site will be determined by the scale of implementation and heterogeneity of the participants. An experienced moderator will lead the discussions following a discussion guide, assisted by notetakers.
The qualitative data collection will focus on the projects selected as case studies, although it may also include additional sites. Interviews and focus groups will be conducted by experienced interviewers who either speak the same language as the participants or are assisted by an interpreter. Interviews and focus groups will be transcribed. The transcripts will be either written in or translated into English, the main language of analysis. Spot-checking will be done during the translation and transcription phases to ensure consistency and accuracy.
Fieldnotes
In addition to interviews and focus groups, fieldnotes will be compiled through stakeholder meetings, such as community of practice meetings. In such meetings, participants involved in the implementation of the programs will be solicited to give an overview of PLH programs in their countries, discuss challenges, and deliberate on possible solutions.
Analysis plans
Since this is an overarching blueprint, we provide an overview of planned analyses approaches. Where possible, data collection tools, protocols, and detailed plans for specific analyses will be published separately and listed on the project’s Open Science Framework page (https://osf.io/v597r/).
Quantitative analyses
To address Research Questions 1 and 2 (examining the dissemination and implementation of PLH programs), we will examine the attendance rates and trends among participating families, as well as variation in enrollment, attendance, and program completion. We will also summarize and examine variation in facilitator and coach sociodemographic characteristics, key organizational characteristics, and the relationship of these characteristics to other implementation outcomes, such as facilitator and coach competent adherence to the program, using regression and structural equation models. We will conduct analyses to examine family behaviors and wellbeing before and after participation in the program to address Research Question 3 (impact of PLH at scale on family-level outcomes), using regression-based models. We will also examine variation in family outcomes based on baseline family characteristics and program implementation. We also plan to compare implementation and outcomes associated with different combinations of services as PLH is often delivered alongside other programs. For Research Question 5 (costs and resources), we will summarize the average program costs and their variation.
Techniques such as hierarchical linear modeling and generalized estimating equations will be used to account for the nesting of data—for example, families within areas. Missing data will be addressed using methods such as full information maximum likelihood or multiple imputation [68], or other methods. Reporting will follow STROBE standards [69] where appropriate. Since implementing partners and PLH research teams may have used different measures to assess targeted outcomes, where necessary and possible, data harmonization strategies will be used for analyses involving data from multiple projects.
Where possible, the research team will support implementation teams to lead their own analyses and ensure that data collected in a specific country remains stored there, even if it is shared for additional analysis elsewhere. It is also important to acknowledge that any analysis of routinely collected data, even in relatively high-resource settings, poses challenges, such as with data quality, social desirability, and selection biases [70–72]. In some countries, the study team is providing support to agencies to strengthen data collection and management systems, and a monitoring and evaluation manual is in process of development to support best practices in data collection.
Qualitative and mixed-method analyses
Thematic analysis [73] will be applied to qualitative data from interviews and focus group discussions to identify patterns within and between case study sites on the dissemination, implementation, and sustainment of PLH programs. Document analysis will be two-phased, combining content and thematic analyses [74]. Relevant information on will be extracted and organized through content analysis guided by the study research questions. The information will then be analyzed thematically. Across our analyses, we will examine how findings vary along key dimensions such as scale of delivery and characteristics of implementing organizations [27, 45, 75].
Reporting will follow the COREQ standards [76]. Summaries of findings will be shared with the research team, participants, and implementing partners for member checking [77]. To support collaboration between researchers, implementers, and other stakeholders [78], we are setting up systems of advisory committees.
Where possible, we will combine multiple data sources to answer research questions. For instance, to help answer Research Question 2 (implementation and adaptation of PLH programs), analysis will draw on the qualitative interviews with facilitators and coaches, and quantitative data on facilitator adherence. We will use the FRAME framework to examine program adaptations [79]. Analyses will include (1) timing and method of adaptation; (2) whether the adaptation was planned or ad hoc; the decision making process for making adaptations; (3) the level and extent of adaptation; (4) whether the adaptation targeted program content, delivery, structure or context of implementation; and (5) how adaptation may have impacted fidelity to core components and program effectiveness.
Discussion
This project will examine nested information about the community, implementing organizations, types of programs implemented, implementation staff, and the caregivers and children who participate in the PLH programs. The use of multiple data sources and methods is designed to support learning as violence prevention programs are tested, disseminated, and scaled-up. No previous study that we are aware of has collected and examined dissemination and scale-up of a parenting program across multiple LMIC contexts. While this study addresses two specific parenting programs, we anticipate that our findings will be of relevance across the field of parenting, as well as many types of social programming, including those focused on education and violence prevention.
Supplementary information
Acknowledgements
The PLH SUPER study would not be possible without the support of wonderful colleagues at numerous organizations, including (to date):
Academic Model Providing Access to Healthcare (AMPATH) Kenya; Agency for Research and Development Initiative, South Sudan; Botswana Stepping Stones International; Catholic Relief Services and 4Children DRC and Haiti; Catholic Relief Services Headquarters, USA; Catholic Relief Services South Sudan, Zimbabwe, Côte d’Ivoire, Cameroon, and Lesotho; Emmanuel Hospital Association, India; FHI 360 Ethiopia; Health for Youth Association, Moldova; Institute for Marriage, Family and Systemic Practice (ALTERNATIVA), North Macedonia; Institute for Security Studies Africa; Justice and Violence Prevention Programme, South Africa; Manenberg People’s Centre, South Africa; Project Hope, South Africa; Ikamva Labantu, South Africa; Pact Eswatini, Tanzania, South Sudan, and Zambia; Lilongwe Catholic Health Commission (LCHC), Malawi; Schola Empirica, Czech Republic; South Sudan’s Ministry of Health; Tanzanian Ministry of Health, Community Development, Gender, Elderly and Children; The Seven Passes Initiative, South Africa; UNICEF HQ; UNICEF Montenegro; World Health Organization. This is a list of our partners in the SUPER study at the time of publishing this paper: more will be added as the study progresses.
We also thank colleagues at the Ateneo de Manila University and the University of the Philippines, Philippines; Babes-Bolyai University, Romania; Georgia State University, USA; St. Cyril and Methodius University Skopje, North Macedonia; Stellenbosch University and University of Cape Town, South Africa; Universitat Klagenfurt, Austria; University of Bremen, Germany; Queens University Belfast, Bangor University, University of Glasgow, and University of Oxford, UK. We thank all the staff at the Children’s Early Intervention Trust and Clowns Without Borders South Africa, in addition to those who are authors. Thank you also to Professor Ana Baumann, Professor George Howe, Professor J. Lawrence Aber, Dr. Thees Spreckelsen, Dr. Lesley Gittings, and Professor Aaron Reeves for advice on study design.
Abbreviations
- PLH
Parenting for Lifelong Health
- LMIC
Low- and middle-income countries
- EPIS
Exploration, Preparation, Implementation and Sustainment (framework)
- WHO
World Health Organization
Authors’ contributions
LC, JML, and CW led the development of the original study idea. All authors participated in the conceptual design of the study and/or the program implementation. YS led the conceptualization and drafting of the manuscript. CW, LC, JML, IW, FG, IW, LLN, MT, HS, and KO wrote the sections and revised the manuscript. The authors read and approved the manuscript.
Funding
The PLH SUPER study has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation program (Grant agreement No. 737476 and No. 771468), Research England, the UK Research and Innovation (UKRI) Global Challenges Research Fund (GCRF) through the UKRI GCRF Accelerating Achievement for Africa’s Adolescents Hub (Grant Ref: ES/S008101/1), the Complexity in Health Improvement Program of the Medical Research Council (MRC) UK (Grant No. MC_UU 12017/14), the National Research Foundation of South Africa (Grant No. 118571), and the Oak Foundation.
Availability of data and materials
Key study materials are available on the project Open Science Framework page: https://osf.io/v597r/
Ethics approval and consent to participate
Ethics committee approval for the SUPER project was granted by the Department of Social Policy and Intervention at the University of Oxford (SPICUREC1a_20_015) and University of Cape Town (PSY2017-040). Country-specific ethics approvals were obtained by the implementation partners in the Democratic Republic of Congo, Montenegro, Haiti, South Sudan, Tanzania, Eswatini and by research partners in the Czech Republic, India, Kenya, North Macedonia, Moldova, Romania, Philippines, and Thailand.
Consent for publication
Not applicable.
Competing interests
Cluver, Ward, Lachman, Hutchings, and Gardner were involved in the development of the PLH programs. Shenderovich, Lachman, and Wessels worked on the PLH trials in South Africa and based their doctoral work on these. The work of Cluver, Shenderovich, Wessels, Ward, Lachman, Okop, Sacolo-Gwebu, and Gardner is partly funded by the UKRI GCRF Accelerating Achievement for Africa’s Adolescents Hub. The work of Lachman has been supported by a GCRF Centre Hub Grant. The work of Cluver, Shenderovich, Wessels, Ward, Lachman, Gardner, Hutchings, and Janowski has been partly funded by grants under the European Research Council’s Horizon 2020 program. The work of Lachman, Ward, and Gardner has also been partly funded by UNICEF Thailand, the work of Hutchings and Lachman by UNICEF Montenegro, and the work of Hutchings, Lachman, Ward, and Gardner by UNICEF Philippines. Further, Lachman is the former Executive Director of Clowns Without Borders South Africa and receives income as a PLH Trainer. Ngcobo is Co-Director of Clowns Without Borders South Africa. Ward reports grants from the National Research Foundation of South Africa and the World Childhood Foundation. Janowski reports a scholarship from the University of Cape Town. Hutchings receives occasional income as a PLH Trainer.
Footnotes
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary information
Supplementary information accompanies this paper at 10.1186/s43058-020-00086-6.
References
- 1.Hillis S, Mercy J, Amobi A, Kress H. Global prevalence of past-year violence against children: a systematic review and minimum estimates. Pediatrics. 2016;137(3) Available from: 10.1542/peds.2015-4079. [DOI] [PMC free article] [PubMed]
- 2.Devries K, Knight L, Petzold M, Merrill KG, Maxwell L, Williams A, et al. Who perpetrates violence against children? A systematic analysis of age-specific and sex-specific data. BMJ Paediatrics Open. 2018;2(1):e000180. doi: 10.1136/bmjpo-2017-000180. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.World Health Organization. INSPIRE: seven strategies for ending violence against children. Luxembourg; 2016.
- 4.Gardner F, Leijten P, Melendez-Torres GJ, Landau S, Harris V, Mann J, et al. The earlier the better? Individual participant data and traditional meta-analysis of age effects of parenting interventions. Child Dev. 2019;90(1):7–19. doi: 10.1111/cdev.13138. [DOI] [PubMed] [Google Scholar]
- 5.Knerr W, Gardner F, Cluver LD. Improving positive parenting skills and reducing harsh and abusive parenting in low- and middle-income countries: a systematic review. Prev Sci. 2013;14(4):352–363. doi: 10.1007/s11121-012-0314-1. [DOI] [PubMed] [Google Scholar]
- 6.Cluver LD, Orkin FM, Campeau L, Toska E, Webb D, Carlqvist A, et al. Improving lives by accelerating progress towards the UN Sustainable Development Goals for adolescents living with HIV: a prospective cohort study. Lancet Child Adolesc Health. 2019;3(4):245–254. doi: 10.1016/S2352-4642(19)30033-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Barlow J, Smailagic N, Huband N, Roloff V, Bennett C. Group‐based parent training programmes for improving parental psychosocial health. Cochrane Database Syst Rev. 2014;(5) Art. No.: CD002020. 10.1002/14651858.CD002020.pub4. [DOI] [PMC free article] [PubMed]
- 8.Steinert JI, Cluver LD, Meinck F, Doubt J, Vollmer S. Household economic strengthening through financial and psychosocial programming: evidence from a field experiment in South Africa. J Dev Econ. 2018;134:443–466. doi: 10.1016/j.jdeveco.2018.06.016. [DOI] [Google Scholar]
- 9.Engle PL, Black MM, Behrman JR, De Mello MC, Gertler PJ, Kapiriri L, et al. Strategies to avoid the loss of developmental potential in more than 200 million children in the developing world. Child Care Health Dev. 2007;33(4):502–503. doi: 10.1016/S0140-6736(07)60112-3. [DOI] [PubMed] [Google Scholar]
- 10.Gardner F, Montgomery P, Knerr W. Transporting evidence-based parenting programs for child problem behavior (age 3–10) between countries: systematic review and meta-analysis. J Clin Child Adolescent Psychol. 2015;45:749–762. doi: 10.1080/15374416.2015.1015134. [DOI] [PubMed] [Google Scholar]
- 11.Ward C, Sanders MR, Gardner F, Mikton CR, Dawes A. Preventing child maltreatment in low- and middle-income countries. Parent support programs have the potential to buffer the effects of poverty. Child Abuse Negl. 2016;54:97–107 Available from: 10.1016/j.chiabu.2015.11.002. [DOI] [PubMed]
- 12.Bellis MA, Hughes K, Leckenby N, Hardcastle KA, Perkins C, Lowey H. Measuring mortality and the burden of adult disease associated with adverse childhood experiences in England: a national survey. J Public Health. 2015;37(3):445–454. doi: 10.1093/pubmed/fdu065. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Britto PR, Singh M, Dua T, Kaur R, Yousafzai AK. What implementation evidence matters: scaling-up nurturing interventions that promote early childhood development. Ann N Y Acad Sci. 2018;1419(1):5–16. doi: 10.1111/nyas.13720. [DOI] [PubMed] [Google Scholar]
- 14.Cavallera V, Tomlinson M, Radner J, Coetzee B, Daelmans B, Hughes R, et al. Scaling early child development: what are the barriers and enablers? Arch Dis Child. 2019;104:S43–S50. doi: 10.1136/archdischild-2018-315425. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Greenhalgh T, Papoutsi C. Spreading and scaling up innovation and improvement. BMJ. 2019;365(May):1–7. doi: 10.1136/bmj.l2068. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Goldmann L, Lundgren R, Welbourn A, Gillespie D, Bajenja E, Muvhango L, et al. On the CUSP: the politics and prospects of scaling social norms change programming. Sex Reprod Health Matters. 2019;27(2):51–63. doi: 10.1080/26410397.2019.1599654. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.World Health Organization. ExpandNet. Practical guidance for scaling up health service innovations. Geneva; 2009. Available from: https://expandnet.net/PDFs/WHO_ExpandNet_Practical_Guide_published.pdf.
- 18.Willis CD, Riley BL, Stockton L, Abramowicz A, Zummach D, Wong G, et al. Scaling up complex interventions: insights from a realist synthesis. Health Res Policy Syst. 2016;14(1):1–16 Available from: 10.1186/s12961-016-0158-4. [DOI] [PMC free article] [PubMed]
- 19.Tomlinson M, Hunt X, Rotheram-Borus MJ. Diffusing and scaling evidence-based interventions: eight lessons for early child development from the implementation of perinatal home visiting in South Africa. Ann N Y Acad Sci. 2018;1419(1):218–229. doi: 10.1111/nyas.13650. [DOI] [PubMed] [Google Scholar]
- 20.Baumann AA, Powell BJ, Kohl PL, Tabak RG, Penalba V, Proctor EK, et al. Cultural adaptation and implementation of evidence-based parent-training: a systematic review and critique of guiding evidence. Child Youth Serv Rev. 2015;53:113–20 Available from: 10.1016/j.childyouth.2015.03.025. [DOI] [PMC free article] [PubMed]
- 21.Gardner F. Parenting interventions: how well do they transport from one country to another? Firenze: Innocenti UNICEF; 2017. [Google Scholar]
- 22.Leijten P, Melendez-Torres GJ, Knerr W, Gardner F. Transported versus homegrown parenting interventions for reducing disruptive child behavior: a multilevel meta-regression study. J Am Acad Child Adolesc Psychiatry. 2016;55(7):610–7 Available from: 10.1016/j.jaac.2016.05.003. [DOI] [PubMed]
- 23.Eisner M, Nagin D, Ribeaud D, Malti T. Effects of a universal parenting program for highly adherent parents: a propensity score matching approach. Prev Sci. 2012;13(3):252–266. doi: 10.1007/s11121-011-0266-x. [DOI] [PubMed] [Google Scholar]
- 24.Fonagy P, Butler S, Cottrell D, Scott S, Pilling S, Eisler I, et al. Multisystemic therapy versus management as usual in the treatment of adolescent antisocial behaviour (START): a pragmatic, randomised controlled, superiority trial. Lancet Psychiatry. 2018;5(2):119–33 Available from: 10.1016/S2215-0366(18)30001-4. [DOI] [PMC free article] [PubMed]
- 25.Bold T, Kimenyi M, Mwabu G, Ng’ang’a A, Sandefur J. Scaling-up what works: experimental evidence on external validity in Kenyan education. Oxford: CSAE Working Paper; 2013. [Google Scholar]
- 26.Nagin DS, Sampson RJ. The real gold standard: measuring counterfactual worlds that matter most to social science and policy. Ann Rev Criminol. 2019;2(1):123–145. doi: 10.1146/annurev-criminol-011518-024838. [DOI] [Google Scholar]
- 27.Hutchings J. Introducing, researching, and disseminating the incredible years programmes in Wales. Int J Confl Violence. 2012;6(2):225–233. [Google Scholar]
- 28.Prinz RJ, Sanders MR, Shapiro CJ, Whitaker DJ, Lutzker JR. Population-based prevention of child maltreatment: the U.S. Triple P System Population Trial. Prev Sci. 2009;10(1):1–12. doi: 10.1007/s11121-009-0123-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Marryat L, Thompson L, Wilson P. No evidence of whole population mental health impact of the Triple P parenting programme: findings from a routine dataset. BMC Pediatr. 2017;17(1):1–10. doi: 10.1186/s12887-017-0800-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Little M, Berry V, Morpeth L, Blower S, Axford N, Taylor R, et al. The impact of three evidence-based programmes delivered in public systems in Birmingham, UK. Int J Confl Violence. 2012;6(2):260–272. [Google Scholar]
- 31.Ward CL, Wessels IM, Lachman JM, Hutchings J, Cluver L, Kassanjee R, et al. Parenting for Lifelong Health for Young Children: a randomized controlled trial of a parenting program in South Africa to prevent harsh parenting and child conduct problems. J Child Psychol Psychiatry Allied Discip. 2020;61(4):503–512. doi: 10.1111/jcpp.13129. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Cluver LD, Meinck F, Steinert J, Shenderovich Y, Doubt J, Herrero-Romero R, et al. Parenting for Lifelong Health: a pragmatic cluster randomised controlled trial of a non-commcialised parenting programme for adolescents and their families in South Africa. BMJ Glob Health. 2018;3(1):1–16. doi: 10.1136/bmjgh-2017-000539. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Lachman JM, Cluver LD, Ward CL, Hutchings J, Mlotshwa S, Wessels IM, et al. Randomized controlled trial of a parenting program to reduce the risk of child maltreatment in South Africa. Child Abuse Negl. 2017;72(July):338–351. doi: 10.1016/j.chiabu.2017.08.014. [DOI] [PubMed] [Google Scholar]
- 34.Alampay LP, Lachman JM, Landoy BV, Madrid BJ, Ward CL, Hutchings J, et al. Preventing child maltreatment in low-and middle-income countries: Parenting for Lifelong Health in the Philippines. In: Developmental Science and Sustainable Development Goals for Children and Youth: Springer; 2018. p. 277–93.
- 35.Gardner F, McCoy A, Tapanya S, Ward CL, Lachman JM. Testing a parenting intervention in the public health system in Thailand to prevent violence against children: randomised controlled trial. 2020. [Google Scholar]
- 36.Lachman JM, Alampay LP, Jocson R, Alinea MCD, Madrid BJ, Ward CL, et al. Integrating parent management training within a conditional cash transfer system for low-income families: results from a small-scale randomized controlled trial in the Philippines. 2020. [Google Scholar]
- 37.World Health Organization. Parenting for Lifelong Health (PLH). Available from: https://www.who.int/violence_injury_prevention/violence/child/plh/en/. Accessed 20 Oct 2020.
- 38.Janowski RK, Wessels I, Bojo S, Monday F, Maloney K, Achut V, et al. Transferability of evidence-based parenting programs to routine implementation in postconflict South Sudan. Res Soc Work Pract. 2020;30(8):858–869. doi: 10.1177/1049731520932986. [DOI] [Google Scholar]
- 39.Cluver L, Lachman JM, Sherr L, Wessels I, Krug E, Rakotomalala S, et al. Parenting in a time of COVID-19. Lancet. 2020;395(10231):e64. doi: 10.1016/S0140-6736(20)30736-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Fixsen DL, Naoom SF, Blase KA, Friedman RM, Wallace F. Implementation research: A synthesis of the literature. Tampa: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231); 2005. [Google Scholar]
- 41.Moullin JC, Dickson KS, Stadnick NA, Rabin B, Aarons GA. Systematic review of the Exploration, Preparation, Implementation, Sustainment (EPIS) framework. Implement Sci. 2019;14(1):1–16. doi: 10.1186/s13012-018-0842-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Moullin JC, Dickson KS, Stadnick NA, Albers B, Nilsen P, Broder-Fingert S, et al. Ten recommendations for using implementation frameworks in research and practice. Implement Sci Commun. 2020;1(1):1–12. doi: 10.1186/s43058-020-00018-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Castro FG, Barrera M, Martinez CR. The cultural adaptation of prevention interventions: resolving tensions between fidelity and fit. Prev Sci. 2004;5(1):41–45. doi: 10.1023/B:PREV.0000013980.12412.cd. [DOI] [PubMed] [Google Scholar]
- 44.Parra-Cardona R, Leijten P, Lachman JM, Mejía A, Baumann AA, Amador Buenabad NG, et al. Strengthening a culture of prevention in low- and middle-income countries: balancing scientific expectations and contextual Realities. Prev Sci. 2018:1–11 Available from: http://link.springer.com/10.1007/s11121-018-0935-0. [DOI] [PMC free article] [PubMed]
- 45.Lucas JE, Richter LM, Daelmans B. Care for Child Development: an intervention in support of responsive caregiving and early child development. Child Care Health Dev. 2018;44(1):41–49. doi: 10.1111/cch.12544. [DOI] [PubMed] [Google Scholar]
- 46.Tommeraas T, Ogden T. Is there a scale-up penalty? Testing behavioral change in the scaling up of parent management training in Norway. Adm Policy Ment Health Ment Health Serv Res. 2017;44(2):203–216. doi: 10.1007/s10488-015-0712-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Gray GR, Totsika V, Lindsay G. Sustained effectiveness of evidence-based parenting programs after the research trial ends. Front Psychol. 2018;9(OCT):1–11. doi: 10.3389/fpsyg.2018.02035. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48.Poulsen MN, Vandenhoudt H, Wyckoff SC, Obong’o CO, Ochura J, Njika G, et al. Evaluation of a U.S. evidence-based parenting intervention in rural western Kenya: from Parents Matter! to Families Matter! AIDS Educ Prevent. 2010;22(4):328–343. doi: 10.1521/aeap.2010.22.4.273. [DOI] [PubMed] [Google Scholar]
- 49.Remme M, Michaels-Igbokwe C, Watts C. Paper 4: Approaches to scale-up and assessing cost-effectiveness of programmes to prevent violence against women and girls. What Works to Prevent Violence against women and girls evidence reviews. 2015. [Google Scholar]
- 50.Woolcock M. Using case studies to explore the external validity of “complex” development interventions. Evaluation. 2013;19(3):229–248. doi: 10.1177/1356389013495210. [DOI] [Google Scholar]
- 51.Forgatch MS, Patterson GR, DeGarmo DS. Evaluating fidelity: predictive validity for a measure of competent adherence to the Oregon model of Parent Management Training. Behavior Ther. 2005;36(1):3–13. doi: 10.1016/S0005-7894(05)80049-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52.Lachman JM, Martin M, Booij A, Tsoanyane S, Majenga N. Parenting for Lifelong Health for Teens: facilitator assessment tool manual. Cape Town.
- 53.Frick PJ. Alabama Parenting Questionnaire: Unpublished rating scale, University of Alabama; 1991.
- 54.Goodman A, Goodman R. Strengths and difficulties questionnaire as a dimensional measure of child mental health. J Am Acad Child Adolesc Psychiatry. 2009;48(4):400–3 Available from: 10.1097/CHI.0b013e3181985068. [DOI] [PubMed]
- 55.Dunne MP, Zolotor AJ, Runyan DK, Andreva-Miller I, Choo WY, Dunne SK, et al. ISPCAN Child Abuse Screening Tools Retrospective version (ICAST-R): Delphi study and field testing in seven countries. Child Abuse Negl. 2009;33(11):815–825. doi: 10.1016/j.chiabu.2009.09.005. [DOI] [PubMed] [Google Scholar]
- 56.Meinck F, Boyes M, Cluver LD, Ward C, Schmidt P, Dunne M. Development and psychometric properties of the ISPCAN child abuse screening tool for use in trials among South African adolescents and their primary caregivers. Child Abuse Negl. 2017;82(January):45–58 Available from: 10.1016/j.chiabu.2018.05.022. [DOI] [PubMed]
- 57.Berry JO, Jones WH. The parental stress scale: initial psychometric evidence. J Soc Pers Relat. 1995;12(3):463–472. doi: 10.1177/0265407595123009. [DOI] [Google Scholar]
- 58.UNICEF . Multiple Indicator Cluster Survey. 2014. [Google Scholar]
- 59.Radloff LS. The CES-D scale: a self-report depression scale for research in the general population. Appl Psychol Meas. 1977;1(3):385–401. doi: 10.1177/014662167700100306. [DOI] [Google Scholar]
- 60.Hutchinson MK. The parent-teen sexual risk communication scale (PTSRC-III): instrument development and psychometrics. Nurs Res. 2007;56(1):1–8. doi: 10.1097/00006199-200701000-00001. [DOI] [PubMed] [Google Scholar]
- 61.Lown JM. Development and validation of a financial self-efficacy scale. J Financ Couns Plan. 2011;22(2):54–63. [Google Scholar]
- 62.Karlan D, Linden LL. Loose knots: strong versus weak commitments to save for education in Uganda. 2014. [Google Scholar]
- 63.Kast F, Meier S, Pomeranz D. Under-savers anonymous: Evidence on self-help groups and peer pressure as a savings commitment device (No. w18417). National Bureau of Economic Research. 2012. Available from: https://www.nber.org/papers/w18417.
- 64.Ceballo R, Maurizi LK, Suarez GA, Aretakis MT. Gift and sacrifice: parental involvement in latino adolescents’ education. Cult Divers Ethn Minor Psychol. 2014;20(1):116–127. doi: 10.1037/a0033472. [DOI] [PubMed] [Google Scholar]
- 65.Vassall A, Sweeney S, Kahn JG, Gomez G, Bollinger L, Marseille E, et al. Reference case for estimating the costs of Global Health Services and Interventions. 2017. [Google Scholar]
- 66.Abdul Latif Jameel Poverty Action Lab . J-PAL costing guidelines. 2020. [Google Scholar]
- 67.Redfern A, Cluver LD, Casale M, Steinert JI. Cost and cost-effectiveness of a parenting programme to prevent violence against adolescents in South Africa. BMJ Glob Health. 2019;4(3):1–9. doi: 10.1136/bmjgh-2018-001147. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 68.Schafer JL, Graham JW. Missing data: our view of the state of the art. Psychol Methods. 2002;7(2):147–177. doi: 10.1037/1082-989X.7.2.147. [DOI] [PubMed] [Google Scholar]
- 69.Vandenbroucke JP, Von Elm E, Altman DG, Gøtzsche PC, Mulrow CD, Pocock SJ, et al. Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): explanation and elaboration. PLoS Med. 2007;4(10):e297. doi: 10.1371/journal.pmed.0040297. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 70.Lindsay G, Strand S, Davis H. A comparison of the effectiveness of three parenting programmes in improving parenting skills, parent mental-well being and children’s behaviour when implemented on a large scale in community settings in 18 English local authorities: the parenting early i. BMC Public Health. 2011;11(1):962. doi: 10.1186/1471-2458-11-962. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 71.Wolpert M, Rutter H. Using flawed, uncertain, proximate and sparse (FUPS) data in the context of complexity: learning from the case of child mental health. BMC Med. 2018;16(1):1–11. doi: 10.1186/s12916-018-1079-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 72.Gugerty MK, Karlan D. The Goldilocks challenge: right-fit evidence for the social sector. New York: Oxford University Press; 2018. [Google Scholar]
- 73.Braun V, Clarke V. Using thematic analysis in psychology using thematic analysis in psychology. Qual Res Psychol. 2006;3(2):77–101. doi: 10.1191/1478088706qp063oa. [DOI] [Google Scholar]
- 74.Braun V, Clarke V. Successful qualitative research: a practical guide for beginners. London: SAGE Publications; 2013. [Google Scholar]
- 75.Aldridge WA II, Murray DW, Prinz RJ, Veazey CA. Final report and recommendations: the Triple P implementation evaluation. Chapel Hill; 2016. Available from: https://ictp.fpg.unc.edu/sites/ictp.fpg.unc.edu/files/resources/TPIEFinalReport_Jan2016_1.pdf.
- 76.Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19(6):349–357. doi: 10.1093/intqhc/mzm042. [DOI] [PubMed] [Google Scholar]
- 77.Orgill M, Gilson L, Chitha W, Michel J, Erasmus E, Marchal B, et al. A qualitative study of the dissemination and diffusion of innovations: bottom up experiences of senior managers in three health districts in South Africa. Int J Equity Health. 2019;18(1):1–15. doi: 10.1186/s12939-019-0952-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 78.Hull L, Goulding L, Khadjesari Z, Davis R, Healey A, Bakolis I, et al. Designing high-quality implementation research: development, application, feasibility and preliminary evaluation of the implementation science research development (ImpRes) tool and guide. Implement Sci. 2019;14(1):1–20. doi: 10.1186/s13012-019-0897-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 79.Stirman SW, Baumann AA, Miller CJ. The FRAME: an expanded framework for reporting adaptations and modifications to evidence-based interventions. Implement Sci. 2019;14(1):1–10. doi: 10.1186/s13012-018-0842-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
Key study materials are available on the project Open Science Framework page: https://osf.io/v597r/