Caregiver Participation Engagement in Child Mental Health Prevention Programs: a Systematic Review

Abstract

Prevention programs are a key method to reduce the prevalence and impact of mental health disorders in childhood and adolescence. Caregiver participation engagement (CPE), which includes caregiver participation in sessions as well as follow-through with homework plans, is theorized to be an important component in the effectiveness of these programs. This systematic review aims to (1) describe the terms used to operationalize CPE and the measurement of CPE in prevention programs, (2) identify factors associated with CPE, (3) examine associations between CPE and outcomes, and (4) explore the effects of strategies used to enhance CPE. Thirty-nine articles representing 27 unique projects were reviewed. Articles were included if they examined CPE in a program that focused to some extent on preventing child mental health disorders. There was heterogeneity in both the terms used to describe CPE and the measurement of CPE. The majority of projects focused on assessment of caregiver home practice. There were no clear findings regarding determinants of CPE. With regard to the impact of CPE on program outcomes, higher levels of CPE predicted greater improvements in child and caregiver outcomes, as well as caregiver-child relationship quality. Finally, a small number of studies found that motivational and behavioral strategies (e.g., reinforcement, appointment reminders) were successful in promoting CPE. This review highlights the importance of considering CPE when developing, testing, and implementing prevention programs for child mental health disorders. Increased uniformity is needed in the measurement of CPE to facilitate a better understanding of determinants of CPE. In addition, the field would benefit from further evaluating strategies to increase CPE as a method of increasing the potency of prevention programs.

Introduction

The occurrence of mental health (MH) disorders in childhood is associated with poor educational, vocational, and health outcomes, and can place children and adolescents (hereafter referred to as children) at a higher risk of experiencing additional MH issues throughout their lifetime (Currie et al., 2010; Fryers & Brugha, 2013; Jaycox et al., 2009). Compounding these risks, a large number of children with MH disorders do not receive the appropriate treatment, increasing the likelihood of long-term negative outcomes (Whitney & Peterson, 2019). Child MH prevention programs have been developed to interrupt the long-term and cascading negative individual and societal outcomes associated with child MH disorders such as poor school performance, later mental health and substance use disorders, and high treatment costs (Caldwell, 2019; Sawyer et al., 2015; Skeen et al., 2019; Wilson & Lipsey, 2007).

In general, MH prevention programs fall into three levels of care: (1) universal (or primary) interventions, which target the general population; (2) selective (or secondary) interventions, targeting those at higher than average risk of developing MH disorders; and (3) indicated (or tertiary) interventions, which target those identified at high risk with already detectable symptoms (Institute of Medicine, 1994). Meta-analytic reviews of child MH prevention programs have demonstrated their positive impact on a range of target problems (e.g., disruptive behaviors, depression) in multiple service settings (e.g., community-based and school-based programs) (e.g., Beelmann & Lösel, 2006; Gillham et al., 2006; Horowitz & Garber 2006; Wilson & Lipsey, 2007). These programs are designed to prevent child MH disorders by modifying children’s risk exposure and strengthening their coping mechanisms.

Caregiver involvement has been shown to be a critical component in treatments for child MH concerns, with meta-analyses showing that treatments that involve caregivers have larger effect sizes than treatments focused only on the individual child (Dowell & Ogles, 2010; Sun et al., 2018). Therefore, it is important to understand the role of caregiver engagement in programs designed to prevent child MH disorders.

Caregiver engagement is a broad construct, and has been conceptualized numerous ways in the child MH treatment literature (e.g., Becker et al., 2015; Piotrowska et al., 2017). Most simply, engagement can be organized into attitudinal and behavioral engagement (Haine-Schlagel & Walsh, 2015; Staudt, 2007). The attitudinal component of engagement is the perception that the benefits of a program or treatment outweigh its costs (e.g., “buy-in;” Becker et al., 2015; Staudt, 2007). The behavioral component of engagement consists of three elements: (1) seeking or initiating help by enrolling in a program or service; (2) attending the program or service; and (3) participating actively and meaningfully in the program or service, both in interactions with providers and by following through with recommendations (Haine-Schlagel & Walsh, 2015; Staudt, 2007). This third component, referred to as caregiver participation engagement (CPE; Haine-Schlagel & Walsh, 2015), includes participating in sessions by sharing opinions, asking questions, and providing a point of view on a problem or solution, as well as through active involvement in therapeutic activities such as role plays and games. CPE also includes caregiver follow-through with homework plans such as changing one’s own parenting behavior, serving as a co-provider to continue the delivery of program strategies at home, and/or supporting the child’s behavior change efforts.

Prevention scientists have emphasized the significance of CPE given the importance of caregivers interacting dynamically with program material and other participants (e.g., Berkel et al., 2011; Coatsworth et al., 2017). However, overall there is variability in the terms used both within the child MH prevention literature as well in the broader MH field. The numerous terms used to describe CPE make it difficult to characterize the landscape of CPE within child MH. There is also variability in how CPE is measured, which limits our understanding of CPE in MH prevention efforts. Although behavioral engagement is conceptualized as a complex array of components, program enrollment and attendance are most frequently utilized as a proxy for the entire construct of behavioral engagement. However, the active participation component of behavioral engagement is also important and does not conceptually overlap entirely with attendance. Some caregivers may attend frequently but fail to engage with or apply the curriculum. Alternatively, some caregivers may attend infrequently but learn the curriculum content because they actively engaged in the sessions they attended and implemented practice skills at home. Indeed, researchers have found that participation in program activities have effects above and beyond that of attendance alone (Berkel et al., 2018). Little is known about the role CPE may play in child MH prevention programs. Though preventive parent training programs for caregivers have become increasingly available, many studies note difficulties engaging caregivers (Morawska & Sanders, 2006), underscoring the need to further characterize the knowledge base regarding CPE within this service type.

Several factors spanning multiple levels (e.g., family, service context) can make it more difficult to garner CPE in prevention programs. For example, caregivers from distressed and disadvantaged families, the primary target of many prevention programs, are also often populations that can be the most difficult to engage and to have participate in intervention activities (Gardner et al., 2009; Jones et al., 2014). Related family level factors that may affect CPE include the availability of transportation, childcare, and financial resources (Garvey et al., 2006; Nix et al., 2009), as well as the perceived relevance of the program and concerns about stigma (Finan et al., 2020). More broadly, a high degree of existing stressors (e.g., limited resources, high crime rate neighborhood, interparental conflict, acculturative differences) may reduce caregivers’ capacity to enroll in, attend, and/or participate in prevention programs (Dadds & McHugh, 1992; Mauricio et al., 2018; McKay et al., 1996; Prinz & Miller, 1994; Winslow et al., 2016). Thus, the very factors that can make a prevention program important for a child may also inhibit a caregiver’s ability to engage with the program.

In terms of service context, the very nature of prevention programs can impact caregivers’ engagement and participation in such services. In contrast to MH treatment services, in which help-seeking is often a result of an obvious problem or concern and thus can be conceptualized as the foundation of caregivers’ overall behavioral engagement (Haine-Schlagel & Walsh, 2015), prevention programs aim to address disorders before they manifest. Therefore, help-seeking may be a different process in prevention programs, which in turn may influence CPE. In addition, the format of such programs may also detract from CPE. For example, prevention programs are often more didactic in nature than treatment and are often delivered in a group format, with other group members’ participation impacting CPE (Nix et al., 2009).

Given the complexities of the populations served by prevention programs and the challenges associated with perceived need and program characteristics, the prevention science field has pointed to various aspects of CPE as important factors to address when implementing prevention programs in community settings. For example, Berkel et al. (2011) identify participant responsiveness, which includes participant attendance and program satisfaction with CPE, as an important dimension of prevention program implementation. Dane and Schneider (1998) conceptualize implementation fidelity, or the degree to which a prevention program is delivered as intended, as including CPE. The Translation Science to Population Impact Framework (TSci Impact; Spoth et al., 2013) highlights the need to understand factors that influence engagement in prevention programs, and emphasizes the importance of developing implementation strategies to facilitate caregiver engagement overall, including CPE. Further understanding of the existing knowledge, including the collection of terms and forms of CPE examined and associations with other key factors, represents a key first step to such efforts.

The current study systematically reviews the literature on CPE in MH prevention programs for children (see Online Resource 1: PRISMA Checklist). Specific aims are to (1) describe the terms used to operationalize CPE and the measurement of CPE in child MH prevention programs, (2) identify factors associated with CPE, (3) examine associations between CPE and program outcomes, and (4) explore the effects of strategies used to enhance CPE. In the present review, we chose to focus only on studies where the authors explicitly state that the target of the intervention was to prevent youth mental health problems. Although other important child-focused prevention or early intervention areas such as child abuse, alcohol/drug use, and autism-specific intervention can have secondary effects on mental health, the nature of the circumstances affecting these populations are different, the consumers are different, and the providers are often different. In order to be able to appropriately synthesize findings specific to mental health prevention studies, we excluded studies in which the program/intervention’s stated target was to reduce general “risk” for poor outcomes with no mention of mental health outcomes as a focus (whether proximal or distal). The current review represents an important extension of prior work examining CPE in child MH treatment services (Haine-Schlagel & Walsh, 2015) to support comprehensive understanding of the role of CPE in the implementation and effectiveness of child MH programs overall.

Method

Search Strategy

Please contact the first author for access to the systematic review protocol. The literature search was first conducted using the electronic databases PubMed and PsycINFO (see Table 1 for search terms). No limitations were placed on publication date given the goal of characterizing the entire literature examining CPE in MH prevention programs; the last date searched was April 6, 2020. In addition, we contacted seven content experts in the field of caregiver engagement and/or prevention science to elicit additional articles to review. Four responded and sent articles that were included in the article selection process. The first author, with expertise in caregiver engagement, also submitted articles to be reviewed as part of the article selection process. A small number of articles were identified through the examination of the reference lists of articles that were screened for inclusion in this review. Duplicate articles identified through the search process were removed prior to article selection procedures. See Fig. 1 for the number of articles screened from each source.

Table 1.

Search terms and inclusion and exclusion criteria

Search terms
[parent* or mother* or father* or famil* or caregiver*] AND [engage* or involve* or participat* or “homework compliance” or “homework completion” or adhere*] AND [prevent* or intervention* or service* or program*] AND [mental health” or “behavioral health” or “parent training”] NOT [schizophrenia or pregnancy or cancer or diabetes or oncology or dementia or HIV or AIDS or obesity or inpatient or “residential treatment” or “older adult*” or “adult caregiver*” or “chronic pain” or “multiple sclerosis” or “weight management” or bulimia or anorexia or “chronic illness” or “developmental delay*” or “developmental disabilit*” or “emotional eating” or “autism” or “drug abuse*” or “alcohol abuse*” or “substance abuse*”]
Abstract screening eligibility criteria
Include
About a program that has been implemented.
Exclude
About a drug or vitamin treatment only.
About adults 18 and over, not in their role as caregivers.
About pregnant women.
If the target population is limited to families with a child with a specified physical health problems like obesity, diabetes, cancer, or traumatic brain injury.
Focuses only on autism, speech/language treatment, physical therapy, or occupational therapy.
Exclude if the article focuses on children under 18 that have a diagnosed mental health disorder.
Exclude if the article is a review article, case study, or study protocol that describes a study that has not been completed yet.
Exclude if the article is clearly not about child mental health.
Article screening inclusion criteria
The article examined caregiver participation engagement in a program, defined as specific caregiver participation behaviors in clinical interactions, homework completion or adherence to intervention strategies at home, and/or any type of ratings of how much the caregiver participated in the service beyond attendance.
The article examined single or multiple program(s).
The program examined in the article is focused at least in part on addressing child (18 and younger) mental health disorders.
The program was focused at least in part on preventing child mental health disorders (versus treatment for an already diagnosed disorder).
The article included original quantitative data analyses that asked an inferential research question that identified factors associated with CPE, examined associations between CPE and outcomes, and/or explored the effects of strategies used to enhance CPE.

Fig. 1.

PRISMA figure. Q1 caregiver participation measured, Q2 study examined a single or group of program(s), Q3 program focused at least in part on addressing child mental health disorders, Q4 pro gram was preventive in nature for child mental health disorders (versus treatment for an already diagnosed disorder), Q5 original quantitative data analyses that ask an inferential research question

Article Selection Procedures

We applied a two-step process to screen and identify included articles, applying distinct but related criteria (see Table 1). The first step in the article selection process was abstract screening. One inclusion criterion and eight exclusion criteria were developed by the authors. The first author trained research assistants to screen abstracts and 24% (n = 1125) of abstracts were double-screened to assess interrater reliability (92.8% agreement). The second step was full-text article screening. Each article was independently screened by two doctoral-level authors, and discrepancies were brought to the full group of article screeners to reach consensus. See Fig. 1 for additional information about the article screening process and reasons for exclusion.

Data Extraction

The initial coding scheme was developed a priori based on the research questions, with additional codes being generated deductively. Coder training consisted of coding one article for practice, refining the coding manual based on group discussion of coding decisions for that article, assigning home-based practice with a second article, and reviewing coding decisions for the second article as a group. Similar to the article screening process, each included article was independently coded and subsequently consensus coded by two doctoral-level authors and discrepancies were brought to the full group of coders to resolve. Data were extracted using coding manuals, which were then entered into SPSS. Double data entry and verification were completed for all data collected. A copy of the coding manuals is available from the first author. Separate coding manuals were created for the article and the CPE indicator(s); in the event that an article included more than one indicator of CPE, separate coding manuals for each indicator were completed.

The extracted data for the article-level coding included 39 data points focused on (1) publication date, (2) program context, (3) sample characteristics, (4) operational definition of CPE, and (5) empirical findings that include CPE as a variable. For our coding of empirical findings, and specifically results pertaining to the impact of CPE on program outcomes, we applied the Hoagwood et al. (2012) categorization of child and family MH treatment outcome domains, including child symptoms (emotions or behaviors often leading to a formal psychiatric diagnosis), caregiver symptoms (similar to previous but assigned to the caregiver), child functioning or impairments (capacity to adapt to home, school, and community demands), consumer-oriented perspectives (child or caregiver perceptions about, for example, satisfaction with care, family strain, beliefs about treatment), interpersonal/environmental contexts (e.g., parenting, parent/child relationship, family functioning, peer-related outcomes), and services/systems (e.g., level, type, duration, or change in use of services). The extracted data for the indicator-level coding included 13 data points focused on (1) the content of the CPE indicator, (2) psychometrics of the CPE indicator, (3) the type of indicator (e.g., survey, observational coding, post-session rating), (4) who reported on the indicator, and (5) how frequently the indicator was assessed.

Reduction of Bias

To reduce the risk of reporting bias, multiple articles from the same dataset were grouped together as a project, with the project subsequently being used as the unit of analysis. For example, when two articles from the same dataset examined the association of CPE with other factors (e.g., impact of maternal depression on CPE), the finding was only counted once to prevent inflating our understanding of the association between constructs. To reduce the risk of bias in interpretation of the results, specifically for the 22 articles examining outcomes associated with CPE and/or strategies to enhance CPE, each article was independently coded by two authors to first assess the purpose of the program (i.e., was CPE mentioned in the article as a goal of the actual program or not). For those that had CPE as an explicit goal of the program (45.5%; n = 10), measurement of CPE was coded to assess whether the measure(s) had a low risk of bias (e.g., caregiver ratings or objective measures like homework completion) or high risk of bias (e.g., provider or researcher ratings). Only five articles from three projects were determined to be at high risk of bias, indicating low levels of potential bias across articles that might impact interpretation of findings.

Analysis Plan

Analyses consisted of running descriptive statistics using IBM SPSS Statistics v. 26 (IBM, 2019) to characterize the projects and articles included in the current review. Analyses involved an overview of included projects and associated articles, and specific representation across projects and/or articles regarding the study context and service delivery, child and family characteristics, provider characteristics, terms used to represent CPE, measurement of CPE, determinants and outcomes associated with CPE, and strategies used to improve CPE. Descriptives regarding the scientific rigor of each project were also analyzed (e.g., examination of reliability and validity of CPE indicators, examination of whether power analyses were conducted).

Results

Overview of Articles

Overall, 39 articles representing 27 unique projects were included in the current review. See Online Resource 2: Article Descriptions for more details about the included articles and projects (in terms of included article references, those cited in this article can be found marked with an asterisk in the “References” section; those not cited in this article can be found in Online Resource 3: Articles in Review Not Cited in Manuscript). The publication dates ranged from 1995 to 2020, with 59% of articles published after 2010. Project focus (Institute of Medicine, 1994) was as follows: 6 universal (22%), 8 selective (30%), 10 indicated (37%), 2 targeted (both selective and indicated; 7%), and 1 not able to determine (4%). See Table 2 for details regarding the child, caregiver/family, and provider characteristics in the included projects. Most projects included both female and male caregivers, and about half of participants were Caucasian. Few projects included details regarding child or provider characteristics such as child gender or race/ethnicity or provider education or discipline.

Table 2.

Child, caregiver/family, and provider characteristics in included projects

Variable		Projects
		n	%
Child age	Early childhood (0–5 years)	20	74
	Elementary school age (5–12 years)	24	89
	Adolescence (13–17 years)	7	26
Child gender	Female and male	3	11
	Male only	2	7
	Not specified	22	81
Child race/ethnicity	Caucasian	3	11
	African American	2	7
	Asian/Pacific Islander	2	7
	Latinx/Hispanic	2	7
	Native American	1	4
	Other	2	7
	Not specified	24	89
Caregiver gender	Female and male	21	78
	Female only	1	4
	Not specified	5	19
Caregiver race/ethnicity	Caucasian or White	16	59
	Average % Caucasian or White (SD)	--	46
	African American	11	41
	Asian/Pacific Islander	6	22
	Latinx/Hispanic	13	48
	Native American	3	11
	Multiethnic/multiracial	4	15
	Other	11	41
	Not specified	8	30
Provider education level	High school	2	7
	Bachelor’s	3	11
	Current graduate student/trainee	5	19
	Master’s	2	7
	PhD/MD	1	4
	Graduate degree not specified	9	33
	Not specified	18	67
Provider discipline	Psychology	5	19
	Social work	3	11
	Counseling	1	4
	Education	3	11
	Paraprofessionals	1	4
	Other discipline	5	19
	Not specified	18	67

n = 27

Aim 1: Terms and Measurement of CPE

Numerous terms were used to describe CPE across projects, with some projects using multiple terms. Seven projects utilized the term “engagement” or some variant (e.g., parent engagement, program engagement), five used the term “participation” or a variant (e.g., quality of participation), and five used the term “homework” or a variation (e.g., homework completion, home practice success, skill generalization). Fifteen projects included an operational definition of CPE. Many projects examined how well the caregiver participated in interactions with providers, which could include behaviors as well as expressed attitudes such as openness and cooperation. Other projects focused on behaviors outside of the provider’s presence, such as completion of homework or number of clicks for online programs. Six projects used definitions informed by prior literature (Bamberger et al., 2014, Berkel et al., 2018; Giannotta et al., 2019; Nix et al., 2009; Nye et al., 1999; Schoenfelder et al., 2013). A total of 36 unique measures of CPE were used across the 27 projects. A total of five projects included two measures and two projects included three measures. See Table 3 for a full description of the types and targets of measurement tools used. The psychometric performance of most of these measures was not previously evaluated, with only five projects (19%) using a CPE measure informed by and/or associated with prior published work.

Table 3.

Measurement of caregiver participation engagement

	Projects
	n	%
Specific construct measured
Global	10	37
Specific behaviors: home practice	19	70.3
Specific behaviors: in session	12	44.4
Measurement method
Survey	10	37
Caregiver-report survey	4	14.8
Provider-report survey	6	22.2
Objective indicator	11	40.7
Post-session observational rating	10	37
Measurement frequency
Each session	17	63
At regular intervals	1	3.7
One or more specific sessions	2	7.4
Randomly selected points in time	1	3.7
At end of program	3	11.1
Other	2	7.4
Measurement psychometrics
Internal consistency	10	37
Interrater reliability	4	14.8
Validity	1	3.7
Not reported	16	59.3

n = 27

Aim 2: Factors Associated with CPE

A total of 22 projects (81.5%) examined determinants of CPE. Table 4 includes information about specific findings, with factors organized into categories based on an ecological multi-level services research framework (Haine-Schlagel & Walsh, 2015; Rodriguez et al., 2014).

Table 4.

Summary of factors associated with CPE

P#	Article authors and years	Child factors	Caregiver/family factors	Provider factors	Service factors	Study quality
Universal programs
6	Bamberger et al. (2014)		Chronic family tension* (−) Genderns Family tension during sessionsns		Intervention condition (Strengthening Families Program or SFP v. Mindfulness Based SFP)a	Lb
6	Coatsworth et al. (2017)	Internalizing symptomsns Externalizing symptomsns	College degree obtained* (+) Partnered marital status* (+) Parenting avoidance* (−) Caregiver-child negative affective relationship quality* (+) Parenting competencens Parenting hasslesns Positive parenting supportns Positive parenting involvementns Caregiver-child positive affective relationship qualityns Baseline depressive symptomsns Caregiver first session engagementns		Intervention condition* (+; MSFP greater than SFP) Attendance* (+)	Lb
12	Dumas et al. (2007)	Genderns Agens Baseline externalizing symptomsns	Partnered marital status* (+) Lower maternal education level* (+) Agens Identified race/ethnicityns Family incomens Baseline stressful life eventsns Time/scheduling demandsns Caregiver perceived relevance or trust in parenting programsns		Attendance* (+) Geographic location of study site* (+; Indianapolis greater than Harrisburg) Intervention demandsns	Lb
12	Dumas et al. (2008)		Agens Identified race/ethnicityns Marital statusns Lower maternal education level (years)ns Family incomens	Provider SES match with caregiver* (+) Overall provider match with caregiver* (+) Provider ethnic match with caregiverns Provider attitude match with caregiverns Provider values match with caregiverns	Geographic location* (+; Indianapolis greater than Harrisburg)	Lb
12	Dumas et al. (2010)		Agens Identified race/ethnicityns Marital statusns Family incomens		Intervention condition* (+; monetary incentive greater than no monetary incentive)	Lb
12	Begle and Dumas (2011)				Attendance* (+)	Lb
12	Hernandez Rodriguez et al. (2020)		Partnered marital status* (+) Agens Identified race/ethnicity		Attendance* (direction not reported)	Lb
13	Lucia and Dumas (2013)				Attendancens	L
14	Dumas et al. (2011)				Attendancens	L
Selective programs
2	Baydar et al. (2003)		Harsh, inconsistent, and supportive parenting-caregiver and observer reports* (+) Parenting-observational* (+) Depressive symptoms* (−) History of substance abuse* (+) Trait anger/aggressionns Experience of harsh or abusive parentinga			Lb
2	Reid et al. (2004)	Baseline externalizing symptoms* (+) Baseline prosocial behaviorns				Lb
4	Garvey et al. (2006)				Attendance* (+)	Lb
5	Breitenstein et al. (2010)			Adherence to protocol* (+) Competencens		Lb
5	Gross et al. (2011)				Attendance* (+) Child care discountns	L
9	Lopez et al. (2018)				Attendance* (+)	L
15	Schoenfelder et al. (2013)	Baseline externalizing symptoms* (−) Baseline internalizing symptoms (a)	Baseline depression* (−) Baseline positive parenting* (+) Baseline griefns		Attendance* (+) Intervention group* (significant percent of variance in CPE associated with which specific program group the caregiver was assigned to within the intervention condition)	Lb
16	Martinez and Eddy (2005)	Genderns Agens Birthplace (U.S. vs. other)ns				L
17	Doty et al. (2016)		Female gender* (+) College degree education* (+) No history of deployment* (+) Number of months deployed* (+)			Lb
17	Pinna et al. (2017)		Age* (−) European American identified race/ethnicity* (+) Genderns Educationns Number of childrenns Family incomens Deployment statusns	Shared military-connected backgroundns	Home practice satisfaction* (+) Participant satisfaction across intervention sessionsns Positive group experiencens	L
17	Zhang et al. (2017)		Female gender* (+) Baseline dispositional mindfulnessns		In-person attendance* (+)	Lb
23	Nicholson et al. (2002)				Time/intervention phase* (+)	Lb
24	Berkel et al. (2018)		Baseline relationship quality-caregiver report* (+) Baseline relationship quality-child reportns Baseline consistent discipline-caregiver report* (+) Baseline consistent discipline-child report* (+) Baseline conflict-caregiver reportns Baseline conflict-child reportns		Attendance* (+)	Lb
25	Day and Sanders (2018)				Intervention condition* (+; practitioner support enhanced program greater than self-directed program)
Indicated programs
7	Ogg et al. (2014)	Agens Current diagnostic statusns Baseline externalizing symptomsns	Agens Educationns		Intervention versionns (English or Spanish)	Lb
8	Davidson et al. (2017)		Baseline psychological distressa			L
11	Cunningham et al. (1995)				Intervention conditionns (Community/Group parent training v. Clinic/Individual parent training)	L
18	Orrell-Valente et al. (1999)	Baseline externalizing symptomsns	European American identified race/ethnicity* (+) Agens SESns Male in householdns Baseline stressful life eventsns Baseline parenting efficacyns Baseline satisfaction with parenthoodns	Therapeutic engagement with caregiverns Provider race match with caregiverns Provider SES match with caregiverns Relevant life events shared with caregiverns		Lb
18	Nix et al. (2009)	Identified race/ethnicityns Baseline externalizing symptomsns	Higher education level* (+) Agens Single parentns Higher occupational prestige* (+) Baseline stressful life eventsns Baseline depressive symptomsns Baseline social supportns Baseline home environmentns Neighborhood qualityns		Intervention group* (significant percent of variance in CPE associated with which specific program group the caregiver was assigned to within the intervention condition) Other group members’ quality of participation* (+)	Lbc
21	Eisenhower et al. (2016)		Baseline sociodemographic riskd ns			L
22	Morgan et al. (2018)	Female gender* (+) Agens Baseline internalizing symptomsns Baseline impairmentns Baseline child inhibitionns	Access to printer* (+) Agens Marital statusns Educationns Lives in major cityns Only childns English as primary household languagens Family financial hardshipns Weekly internet usens Baseline family strainns Baseline psychological distressns Baseline overinvolved/overprotective parentingns			Lb
26	Ristkari et al. (2019)				Implementation type* (community trial had higher CPE than research trial)
Not specified programs
27	Giannotta et al. (2019)			Understanding caregivers’ problems-caregiver report* (+) Agens Genderns Educationns Group management skills-caregiver reportns Supportive of caregivers-caregiver reportns	Intervention condition* (Comet intervention program higher CPE than Cope program, Incredible Years program, and Connect program)	L

Results may be different across articles from the same project because the samples and/or analyses may have been different for each article. The value in parentheses indicates the direction of significant effects, with (+) indicating a positive association and (−) indicating a negative association. Any indication of the baseline time point is included in this table when it is clearly articulated in the article

P# project number, L longitudinal design

^*Significant association with CPE

^ns No significant association with CPE

^aNo inferential statistical test reported

^bMultivariate analyses

^cPower analyses conducted

^dHigh school or less, low income, teen parenthood, unemployment, racial minority status, and single-caregiver status

Child Factors Associated with CPE

A total of nine articles from eight projects examined child-level factors, including child demographic factors and child MH symptoms.

Child Demographics

In general, no significant associations were found between CPE and child gender, race, or birth location (U.S. versus other). Only one of the four projects examining CPE and child gender observed a significant association, with caregivers of female children completing homework more frequently than caregivers of male children.

Child Baseline MH Symptoms

The most commonly examined child factor was child baseline MH symptoms, with seven projects exploring how baseline symptoms may contribute to CPE. Most found no significant association. Two projects identified significant links between baseline externalizing symptoms and CPE, with mixed results.

Caregiver/Family Factors Associated with CPE

Nineteen articles across 12 projects examined how caregiver/family-level factors may predict CPE. These factors included caregiver/family demographics, caregiver baseline MH symptoms, parenting behaviors, and family environment.

Caregiver/Family Demographics

Significant associations between caregiver education level, marital status, race/ethnicity, and CPE were found in at least half of the projects that examined these factors. Of six projects, three observed a significant, positive association between caregiver education and CPE. Four projects examined marital/partner status and CPE, with two noting a significant association between having a partner and higher levels of CPE. Caregiver race/ethnicity was a significant predictor of CPE in two out of the three projects that examined it, with caregivers from European American backgrounds exhibiting more CPE. The majority of projects found no significant relationship between CPE and income/socioeconomic status (examined in five projects), caregiver age (five projects), and primary household language (one project).

Caregiver Baseline MH Symptoms

Of the six projects examining caregiver MH, three observed significant associations; for example, caregiver depressive symptoms were related to less CPE and caregiver history of substance use was related to more CPE.

Parenting Behaviors

The majority of projects examining parenting behaviors and CPE (four out of five) found that greater levels of negative and positive parenting behaviors were associated with lower and higher CPE, respectively.

Family Environment

Five projects examined how different types of family environment factors, such as family tension and strain, family risk, time and scheduling demands, marital discord, number of children in the home, and military deployment status, influenced CPE. Only two projects found a significant association, such that greater family tension was associated with lower CPE, and caregivers with a history of deployment showed lower CPE than caregivers who had not been deployed. No significant associations between CPE and stressful life events (e.g., death in the family) were found.

Provider Factors Associated with CPE

Four projects examined provider-level factors, including demographics, provider-caregiver match, fidelity, and engagement skills.

Provider Demographics

One project examined the association between provider demographics (e.g., gender, race/ethnicity) and found no significant association with CPE.

Provider-Caregiver Match

Of the three projects examined the match between provider and caregiver on one or more of several variables (ethnicity, SES, attitudes, values, shared background/life events), one project observed a significant and positive association between CPE and the extent of caregiver-provider match for the variable(s) studied.

Provider Fidelity

One project explored the relationship with fidelity, noting a significant, positive association between provider adherence to protocol, but not competence, and CPE.

Provider Engagement Skills

Two projects examined links between provider therapeutic engagement skills and CPE, with only one finding a significant, positive association between better provider understanding of caregiver problems and higher CPE.

Service Factors Associated with CPE

Seventeen projects across 24 articles examined service-level factors, including caregiver attendance and program characteristics.

Attendance

The majority of projects examining attendance (seven out of ten) found that higher caregiver attendance was associated with greater CPE.

Program Characteristics

Fifteen projects examined program factors such as greater group member engagement, which intervention group a caregiver/family was a member of within the intervention condition, amount of time in the program, and number of phone consultations received. All of these factors were significantly linked to greater CPE. In contrast, the intervention condition the caregiver/family was assigned to and provision of incentives were not consistently significantly linked to CPE.

Aim 3: Associations Between CPE and Program Outcomes

CPE Predicting Program Outcomes

A total of 13 projects (33.3%) examined whether CPE was associated with improved outcomes, all employing a longitudinal design to do so (see Table 5). We applied Hoagwood et al. (2012) categorization of program outcomes to organize results. The most common outcome domains examined were child symptoms and interpersonal/environmental contexts. As child functioning, caregiver/family symptoms, consumer-oriented perspectives, and services/systems were only examined in one or two projects each, our summary will focus on the outcome domains more frequently examined.

Table 5.

Summary of associations between CPE and program outcomes based on Hoagwood et al. (2012) domains

P#	Authors and years	CPE categories measured	Child symptoms	Caregiver/family symptoms	Child functioning/impairment	Consumer-oriented perspectives	Interpersonal/environmental contexts	Services/systems	Study quality
Universal programs
12	Begle and Dumas (2011)	Global	ns		+	+	ns		L #
12	Jobe-Shields et al. (2015)	Global					ns		L #
Selective programs
2	Baydar et al. (2003)	Specific At home					+		L#
2	Reid et al. (2004)	Specific At home	+		+				L #
4	Garvey et al. (2006)	Specific	+	+					L #
15	Schoenfelder et al. (2013)	Global At home	ns	+			+		L #
17	Zhang et al. (2017)	At home					+		L #
4	Berkel et al. (2018)	At home					+		L #
Indicated programs
3	Tucker et al. (1998)	At home	ns				+		L ^
8	Davidson et al. (2017)	At home	@						L
10	Hooven et al. (2013)	Global Specific						+	L #
18	Nix et al. (2009)	Global					+		L #
20	Nye et al. (1995)	Global Specific At home	+						L #
20	Nye et al. (1999)	Global Specific At home	+						L #
22	Morgan et al. (2018)	At home	+						L # ^
Not specified programs
27	Giannotta et al. (2019)	At home	ns			+	+		#

P# project number, + significant positive association with CPE found such that increased CPE was associated with more positive outcomes, ns no significant associations with CPE demonstrated, @ no inferential statistical test reported, L longitudinal design,

^#multivariate analyses,

^{^}power analyses conducted,

Global overall ratings of CPE beyond attendance, Specific specific CPE behaviors in clinical interactions, At home homework completion or adherence to program strategies at home

Child Symptoms

Nine projects explored the influence of CPE on child MH symptom outcomes, with five noting a significant change in symptoms. Overall, higher levels of CPE prospectively predicted greater improvements in child internalizing and externalizing symptoms, and were associated with greater improvement in child coping and prosocial competence following the program.

Interpersonal/Environmental Contexts

Seven of the eight projects that examined interpersonal/environmental contexts found that higher CPE predicted more positive outcomes related to parenting behaviors such as caregiver-child relationship quality and parenting skills, as well as caregivers’ dispositional mindfulness.

CPE Mediating or Moderating Program Outcomes

As shown in Tables 6 and 7, CPE was examined as a moderator or mediator of program outcomes in three and two projects, respectively. Results suggested that CPE moderated the effect of key child (e.g., prior foster care placements; DeGarmo et al., 2009) and family (e.g., caregiver ethnicity, family resources; Nye et al., 1995; Zhang et al., 2017) determinants on program outcomes. In terms of mediation, results suggested that CPE mediated the impact of caregiver-level predictors (e.g., positive parenting, caregiver depression, caregiver education, and occupation) on program outcomes (Nix et al., 2009; Schoenfelder et al., 2013).

Table 6.

Summary of CPE as a moderator of baseline determinants of program outcomes

Article	P#	Determinant domain			Study quality
		Child	Family	Provider
DeGarmo et al. (2009)	1	Genderns Agens Prior number of placements* (for high level of prior placements, CPE associated with better outcomes)	Ethnicity* (for Hispanic caregivers, CPE associated with better outcomes) Agens Caregiver-child ethnic matchns Moodns Kinship caregiverns	Provider ethnic match with caregiverns	L #
Nye et al. (1995)	20		Family resources* (higher CPE suppressed negative association between family resources and program outcomes) Agens Psychopathologyns		L #
Zhang et al. (2017)	17	Gender* (for mothers, CPE associated with better outcomes)			L #

P# project number,

^*significant association with CPE,

ns no significant association with CPE, L longitudinal design,

^#multivariate analyses

Table 7.

Summary of CPE as a mediator of determinants of program outcomes

Article	P#	Determinanta (all in family domain)	Study quality
Nix et al. (2009)	18	Education* (+) Occupation* (+)	L # ^
Schoenfelder et al. (2013)	15	Depression* (−) Positive parenting* (+)	L #

^aAll determinants were from the caregiver/family domain

P# project number,

^*significant association with CPE,

L longitudinal design,

^#multivariate analyses,

^{^}power analyses conducted

Aim 4: Strategies to Improve CPE

Four projects examined strategies to improve CPE in a child MH prevention program, all noting significant associations between CPE improvement strategies (e.g., motivational strategies, reinforcement strategies) and program outcomes (see Table 8).

Table 8.

Summary of associations between CPE improvement strategies and outcomes

Article	Project #	CPE improvement strategya	CPE categories measured	Significant results?	Fidelity measured?
Universal
Dumas et al. (2010)	12	Motivational, not otherwise specified (NOS)	Global Specific	+	No
Hernandez Rodriguez et al. (2020)	12	Provider reinforcement Motivational, NOS	Global	+	Yes
Selective
Day and Sanders (2018)	25	Appointment reminders Motivational, not otherwise specified	At home	+	No
Doty et al. (2016)	17	Provider reinforcement	At home	+	No
Indicated
Jones et al. (2014)	19	Provider reinforcement Appt reminders Modeling Crisis management Accessibility promotion	At home	+	Yes

^aCoded post hoc based on Lindsey et al. (2014)

+ significant results, Global broad measures of caregiver participation in the program, Specific specific caregiver in-session participation behaviors, At home behaviors pertaining to caregiver home practice

Discussion

We conducted a systematic review of the literature on CPE in child MH prevention programs. The review aimed to (1) describe the terms used to operationalize CPE and measurement of CPE in prevention programs, (2) identify factors associated with CPE, (3) examine associations between CPE and program outcomes, and (4) explore the effects of strategies used to enhance CPE. From an initial pool of 5171 articles, we found a total of 39 published articles generated from 27 projects that examined CPE in child MH prevention programs.

Overall, results underlined heterogeneity in both the operationalization and the measurement of CPE. Numerous terms were used to describe CPE across projects. Projects used a variety of CPE measures; the psychometric performance of the majority of these tools was not previously evaluated or was unreported. The most frequent metric was caregiver home practice, followed by global measures of CPE and specific in-session participatory behaviors.

The most frequently examined determinants of CPE were those at the service (e.g., intra-group engagement, caregiver attendance) and caregiver/family levels (e.g., caregiver education level, marital status). However, given the variability in measurement and methodology across studies, we did not find clear determinants of CPE.

With regard to the impact of CPE on program outcomes, higher levels of CPE predicted greater improvements in child symptoms (e.g., internalizing and externalizing symptoms, coping, and prosocial behavior) and interpersonal/environmental contexts (e.g., caregiver-child relationship, parenting skills). This is consistent with the literature examining the impact of CPE on MH treatment (Haine-Schlagel & Walsh, 2015). Finally, in terms of strategies to improve CPE, the small collection of existing projects highlight the utility of motivational and behavioral strategies (e.g., reinforcement, appointment reminders) in promoting CPE. Again, these results mirror those in the treatment literature (Becker et al., 2015; Haine-Schlagel & Walsh, 2015).

Our results indicate that researchers are increasingly interested in CPE and are seeking to conduct objective, concrete measurement of the construct. Of the 39 published articles, more than half were published in 2010 or later, suggesting that understanding and improving CPE in child MH prevention is of growing importance to the field. Although few projects used measures of CPE with strong psychometric properties, the majority of studies represented in the review aimed to use objective and survey-based measurement methods to quantify CPE. Studies’ operational definitions of CPE variables were typically specific and represented distinct caregiver behaviors both within and outside interactions with providers. Many projects examined how well the caregiver participated in interactions with providers, which could include behaviors as well as expressed attitudes such as openness and cooperation. Other projects focused on behaviors outside of the provider’s presence, such as completion of homework or number of clicks for online programs.

In addition, multiple projects elected to use more than one measure of CPE. For example, Nye et al. (1995) measured both mothers’ investment in the program through participatory behaviors (both within-session participation and between-session home practice) and overall completion of the program protocol. Even within the projects that only examined at-home CPE behaviors, researchers measured home practice engagement in multiple ways, capturing a combination of home practice frequency, fidelity to the assigned home practice, and efficacy or competence of home practice of program skills (e.g., Berkel et al., 2018; Schoenfelder et al., 2013).

The variety of ways in which researchers have operationalized and measured CPE speak to the complexity of the construct and the importance of its many components in child prevention services. Broadly, we found that numerous CPE behaviors were associated with significant changes in child symptoms and/or effective parenting. As studies continue to examine the role of CPE in the future, unified definitions and measurement will allow for specificity in associations and expansion of evidence-based strategies for increasing CPE.

Our review highlighted gaps in the literature regarding CPE. For one, existing studies do not provide enough information to inform community implementation of programs. Indeed, one of the great challenges facing prevention science is large-scale dissemination and scaling up of appropriate preventative and promotive interventions (Botvin, 2004; Gottfredson et al., 2015). However, few studies included in the review provided details regarding child or provider characteristics such as child gender, child race/ethnicity, or provider education. Such details are important for understanding the context of CPE and its correlates within a given study, which in turn inform community implementation efforts (Spoth et al., 2013). Further, there was limited examination of prevention programs within the context of community service settings such as educational settings, where the majority of children receive services. Given that schools are often at the frontline of providing MH prevention services to children and families (Bosworth, 2015; U.S. Public Health Service, 1999), the landscape of CPE within school settings and strategies to increase caregiver engagement needs further investigation. It is also important for future research to focus on identifying patterns of contextual factors associated with CPE, in particular for disadvantaged families who may need increased and/or tailored supports for CPE (Hackworth et al., 2018). It is also important to note a limitation of this review, namely, that incomplete retrieval of identified research may have occurred.

Despite the recent increase in attention to CPE in child prevention services, it is disappointing to see overall limited efforts to study and improve CPE. We believe that greater attention to CPE is warranted, both in identifying determinants of and strategies for increasing CPE. In addition, there are unanswered questions about the role of CPE in program outcomes and its link to maintenance or long-term preventative effects. Insights from the science of behavior change would suggest that at-home practice of program skills is more likely to translate into sustained changes in parenting and caregiver-child interactions, which in turn translate to longer-term maintenance effects. For example, Schoenfelder et al. (2013) found that CPE during the program mediated the durability of positive parenting post-intervention, but more research is needed to replicate these findings. Further, it is not unusual to find sleeper effects in prevention research wherein the effects of the prevention program do not appear or appear stronger later in time (e.g., Prado et al., 2007; Rapee, 2013). Long-term follow-up of CPE and its associations to outcomes is necessary to understand the potential role of CPE in prevention services. Similarly, CPE is routinely examined as a static variable measured at one point in time or averaged across the program period. One exception is Bamberger et al. (2014), who used growth curve modeling to examine CPE over the course of a 7-week intervention. Future examinations that focus on trajectories of CPE (similar to attendance trajectories) and variability in CPE over time may provide valuable information on how to increase CPE to levels that are both attainable (from the program side) and sustainable (from the caregiver side).

Among the 27 projects identified, four explicitly tested motivational and behavioral strategies for increasing CPE, including motivational interviewing, accessibility promotion, and appointment reminders. These strategies strongly mirror strategies to increase CPE found in the treatment literature (Haine-Schlagel & Walsh, 2015). In addition, researchers have identified several barriers to caregiver engagement more broadly and potential solutions for increasing caregiver engagement in prevention services through qualitative studies (Cunningham et al., 2019; Finan et al., 2020). Though their inquiry often focused on attendance, insights may be applicable to engagement and participatory behaviors broadly. For example, Finan et al. (2020) found that the term “prevention” was not easily understood or compelling to lay caregivers, thus restricting caregiver interest and eventual participation. Researchers also found that balance needed to be achieved between program duration that is not burdensome and yet simultaneously long enough such that caregivers perceive benefits to participation.

Increasingly, novel modalities are being utilized for prevention program delivery and dissemination, which may help maintain the balance of providing high quality and sufficient dosage of prevention services while simultaneously keeping caregiver burden low. Technology-assisted delivery with asynchronous scheduling is increasingly popular for its potential to allow caregivers to fit prevention services into their already-demanding lives (Clarke et al., 2015). In fact, numerous studies show that computerized delivery is effective, acceptable to caregivers, and feasible to implement (e.g., Dadds et al., 2019; Irvine et al., 2015; Spencer et al., 2019). One of the main benefits of computerized delivery may be the potential to increase dosage of program content on demand (Dadds et al., 2019). Infusion of a computerized or social media component alone is not sufficient (Epstein et al., 2019). Rather, digital assistance that increases dosage and provides numerous exemplars to increase exposure and facilitates generalization of skills for caregivers appear particularly effective (Dadds et al., 2019; Irvine et al., 2015; Palmer et al., 2019). As more and more prevention programs move to online or computer-assisted platforms, explicit links between digital components and CPE need to be examined. These may inform the development of evidence-based digitally delivered strategies for increasing CPE.

Although the current review represents one of the first systematically characterizing CPE in MH prevention, several limitations should be noted. Almost all projects in the review were conducted within the context of a research study, limiting the generalizability of findings to broader populations and settings without academic research infrastructure or partnership. Further, this review is limited by focus and extraction on factors included in projects as they were reported or specified; it is possible that further characteristics, strategies, and associations exist but were not reported or specified in the original studies. This review was limited to studies published in English. Finally, it is also possible that further projects that qualitatively examined CPE in preventative MH programs exist but were not captured during data extraction in the current review.

In sum, the current systematic review highlights increasing focus on CPE within child MH prevention in recent years, and the important role that CPE plays in both the immediate and lasting effects of prevention programs. And yet, questions remain regarding strategies to increase CPE within prevention efforts. With the increasing adoption of digital/computerized programs and technology-assisted programs, the importance of CPE in child MH prevention is simultaneously magnified and tenuous.