Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Jan 1.
Published in final edited form as: J Child Fam Stud. 2019 Sep 20;29(1):71–81. doi: 10.1007/s10826-019-01575-6

Direct-to-Consumer Marketing for Parent-Child Interaction Therapy: Impact of Language and Messenger

Miya L Barnett 1, Natalie A Bernal 1, Berta Erika Luis Sanchez 1
PMCID: PMC7728243  NIHMSID: NIHMS1540435  PMID: 33311965

Abstract

Objectives:

Parent-Child Interaction Therapy (PCIT) is an effective therapy to treat early onset disruptive behavior problems and child physical maltreatment. In order to support the successful implementation and sustainment of PCIT, strategies are needed to recruit parents into care, especially for racial and ethnic minority families, who often have lower rates of access and utilization of mental health services.

Methods:

This study investigated the impact of direct-to-consumer advertisements on parents’ attitudes towards PCIT. Advertisements were delivered in Spanish and English, with either a parent testimony or therapist explaining the treatment. Participants were parents of children between the ages of 2 and 7, who were recruited through Amazon Mechanical Turk. Participants (N = 204) were 38.2% female and 49.5% Spanish speaking.

Results:

There were no main effects for language or messenger related to PCIT Help-Seeking Intentions, Attitudes, or Stigmatization. However, there was an interaction effect for language and messenger for PCIT Help-Seeking Intentions. Specifically, Spanish-speaking participants had higher intentions when the messenger was a therapist rather than a parent, and had lower intentions than English-speaking parents when the messenger was a parent.

Conclusions:

These findings are promising for direct-to-consumer advertising strategies that may help recruit more Spanish-speaking families into PCIT, which could help address disparities in access to mental health services.

Keywords: Parent-Child Interaction Therapy, direct-to-consumer marketing, evidence-based treatment, mental health disparities, parental engagement


Parent-Child Interaction Therapy (PCIT) is an evidence-based treatment (EBT) that has been shown to be effective at treating early-onset disruptive behavior disorders in children between the ages of 2 and 7 (Lieneman, Brabson, Highlander, Wallace, & McNeil, 2017; McCabe, Yeh, Lau, & Argote, 2012; Niec, Barnett, Prewett, & Shanley Chatham, 2016; Schuhmann, Foote, Eyberg, Boggs, & Algina, 1998; Thomas, Abell, Webb, Avdagic, & Zimmer-Gembeck, 2017) and reducing recidivism in parents with substantiated physical abuse (Chaffin et al., 2004). Core features of PCIT include its emphasis on the parent-child relationship and the use of in vivo feedback (i.e., coaching) to develop parenting skills, which are both treatment components that are associated with larger effect sizes in parent training programs (Kaminski, Valle, Filene, & Boyle, 2008). In order to improve mental health services for young children, states and counties have invested in scaling up Parent-Child Interaction Therapy (PCIT) into publicly-funded community settings (Beveridge et al., 2015; Timmer et al., 2016). Accumulating evidence suggests that PCIT is effective when delivered in community settings, but challenges remain with successfully engaging families in care (Lanier et al., 2011; Pearl et al., 2012; Stokes, Wallace, & McNeil, 2018; Timmer et al., 2016). Similar to other efforts to implement EBTs in community settings, challenges with initial engagement can pose a major barrier to successful uptake and sustainment of PCIT (Beveridge et al., 2015; Regan et al., 2017; Timmer et al., 2016). Initial engagement, which includes recruitment, enrollment, and initial attendance, refers to the process in which the parents intend and commit to participate in a parenting intervention (Gonzalez, Morawska, & Haslam, 2018). Steady recruitment and engagement of appropriate clients is needed for therapists to learn how to deliver PCIT with fidelity and get certified within a timely manner (Beveridge et al., 2015; Scudder & Herschell, 2015; Timmer et al., 2016). Though the majority of dissemination and implementation efforts have focused on increasing the supply of EBTs by training clinicians (i.e., “push strategies”), there is a need for innovative “pull” strategies, which are consumer-facing strategies focused on increasing the demand for and engagement in EBTs such as PCIT (Barnett, Lau, & Miranda, 2018; Becker, 2015; Gallo, Comer, Barlow, Clarke, & Antony, 2015). One suggested strategy has been to use direct-to-consumer (DTC) marketing of EBTs to generate client interest in these interventions (Becker, 2015; Friedberg & Bayar, 2017; Gallo, Comer, & Barlow, 2013; Santucci, McHugh, & Barlow, 2012).

DTC marketing are materials, such as video advertisements, designed to increase demand for products or services(Friedberg & Bayar, 2017). Typically, DTC has predominately been used by pharmaceutical companies. It has been shown to increase demand for psychotropic medications. with criticism that this may lead consumers to see medication as the only treatment option (Donohue, Cevasco, & Rosenthal, 2007; Friedberg & Bayar, 2017). To identify the potential of DTC advertising for psychological treatments, Gallo and colleagues (2015) conducted a randomized control trial of advertisements for medication and psychological treatments. Participants that saw advertisement for psychological treatments reported more intentions to seek therapy than the other advertising conditions (e.g., medication campaign, neutral campaign). As this study focused on advertising psychological treatment in general, as opposed to EBTs, the authors identifed the need for future research to focus on marketing specific treatments. Friedberg & Bayar (2017) made several recommendations regarding DTC advertising of EBTs. First, they suggested that advertisements be piloted, to allow for the analysis of the impact of the message and improve the precision of the messages before large-scale marketing. Second, they stated that “DTC marketing efforts must be educational and foster increased mental health literacy (p. 622).” Finally, they identified that a goal of DTC should be to ameliorate mental health service disparities and promote accountability and improved care across sectors. Therefore, marketing should reach all segments of the population and advertisements may need to be tailored for different cultural contexts.

Specific to parenting interventions, a recent systematic review of strategies to enhance initial engagement in parenting interventions found limited experimental research in this area, despite widespread use of different strategies (Gonzalez et al., 2018). Regarding DTC marketing, video-testimonials and online messages have been tested to see what framing of messages and types of messengers increase recruitment into parenting interventions (Morawska, Nitschke, & Burrows, 2011; Salari & Backman, 2016). Morawska and colleagues (2011) conducted a study to identify if testimonials impacted parent perceptions of the parenting intervention, Triple P. In the study, they tested expert (i.e., therapist) and peer (i.e., parent) testimonials about Triple P against no testimonials. They did not find a significant impact of the testimonials on parental perceptions of Triple P, though there was a trend for more positive perceptions after hearing an expert testimony. This study was an important first step into studying the role of messengers in recruitment into parenting interventions, but was limited by the participant sample, which was predominately White and had previous knowledge of Triple P. The authors identified that more research was needed to establish the effectiveness of testimonials as engagement strategies. First, they recognized the need to conduct research with a larger and more diverse sample. They also suggested that researchers manipulate elements of the testimonial “to determine what particular elements are the most effective and in what circumstances” (p. 96). Important components they identified included levels of expertise, characteristics of the audience, and characteristics of the testimonial provider.

Though PCIT has demonstrated effectiveness with Spanish-speaking Mexican American and Puerto Rican families (Matos, Bauermeister, & Bernal, 2009; McCabe & Yeh, 2009), specific DTC marketing may be needed to promote access to this EBT, given documented disparities in mental health service access for Spanish-speaking, Latinx immigrants. Research has demonstrated that individuals with limited English proficiency are less likely to access mental health services (Kim et al., 2011; Sentell, Shumway, & Snowden, 2007), with recent estimates showing that only 4% of undocumented Latinx immigrants access behavioral health services (Ortega et al., 2018). Further, Latinx parents are less likely to enroll in parent training programs when they are offered than non-Latinx, White parents (Baker, Arnold, & Meagher, 2011). To promote engagement, specific strategies have been developed for Latinx parents. In one community-based participatory research study, Latinx parents expressed interest in learning about parenting programs from an expert, such as a teacher, and identified that it would be helpful to hear testimonies from a parent who had participated in a program previously (Winslow et al., 2016). Based on this community input, a suite of engagement materials was developed, which included brochures with testimonials from parents, endorsements from their children’s teachers through informal conversations, and 20-minute phone calls from group leaders. Parents with children with more behavioral concerns were more likely to initiate and attend parenting group sessions if they received the engagement intervention rather than the control condition, which was a brochure and non-engagement survey interview. The engagement intervention appeared to be especially effective for highly acculturated parents. Though this engagement intervention showed benefits for parental engagement, it was not possible to disentangle which of the strategies was most impactful (Gonzalez et al., 2018; Winslow et al., 2016). For example, it was not clear if the parent testimonies or teacher endorsements both led to initial engagement in the program, or if one of the strategies was adequate to have this impact.

The use of cultural similarities (i.e., language, values) has been suggested as a tailoring strategy to improve health communication and outreach to groups that may experience more barriers to access health services (Kreuter & McClure, 2004). One similarity that may be leveraged is the source or messenger providing the information. Though the majority of research has found that expert sources are more persuasive than individuals lacking expertise, another important factor is the similarity of the source to the audience. This rationale has led to a number of health communication interventions that use messengers, who share similar backgrounds (i.e., race, health status) with the target population, to promote behavior change, such as HIV-testing (Kalichman & Coley, 1995; Kreuter & McClure, 2004). Others have highlighted the potential of having outreach for parenting interventions conducted by near peers to Latinx parents, such as parents who have successfully completed treatment or natural helpers in the community, to decrease stigma and promote engagement in care (Barnett et al., 2016; Calzada et al., 2005; Gustafson, Atkins, & Rusch, 2018). The engagement intervention developed by Winslow and colleagues (2016) and strategies to leverage near peers to promote parent involvement are promising (Barnett et al., 2016; Gustafson et al., 2018) but still require a significant time and resource investment to engage families. DTC advertisements have the potential to easily and quickly disseminate messages about PCIT to reach more parents. Therefore, additional research on how marketing materials can be tailored to increase access and utilization for traditionally underserved groups, such as Spanish-speaking parents, is warranted. Specifically, it may be beneficial to test if DTC messages delivered by similar peers improves attitudes towards seeking services.

When developing and testing DTC marketing materials it is important to identify the impact on variables linked to theoretical models of parental help-seeking. The Theory of Planned Behavior posits that the best predictor of an individual performing a behavior is the individual’s intentions to do so, which are influenced by the individual’s attitudes, subjective societal norms, and perceived behavioral control (Ajzen, 1985). In fact, one study, which investigated the Theory of Planned Behavior, found that parental intentions to attend parenting groups were predicted by their positive attitudes towards the groups and perceptions that others valued the group (Thornton & Calam, 2011). The researchers recommended that recruitment campaigns for parenting interventions focus on promoting positive attitudes amongst parents. Societal norms have been conceptualized as the mental health stigma that parents have towards seeking services, which may need to be addressed to increase help-seeking behaviors (Turner, 2012; Turner, Jensen-Doss, & Heffer, 2015; Turner & Liew, 2010). Mental health stigma has been associated with lower help seeking intentions for Latinx parents, but not non-Latinx, White or African American parents (Turner et al., 2015).

To improve recruitment into PCIT, especially for Spanish-speaking parents, the current study developed and tested DTC adverstisements in English and Spanish, using either a parent or therapist testimonial. This study followed recommendations to evaluate the relative effectiveness of culturally tailored health communication strategies (Kreuter & McClure, 2004), by comparing how the message source (i.e., peer or expert) impacted attitudes, stigma, and help-seeking intentions for parents who saw the advertisements in Spanish or English. The advertisements were pilot tested using Amazon’s Mechanical Turk (MTurk), which crowdsources workers, who can be recruited to participate in research studies. Accumulating evidence suggests that MTurk can be successfully used to gather high-quality data quickly and at a low cost for studies related to clinical psychology and family processes (Chandler & Shapiro, 2016; Schleider & Weisz, 2015). Based on research suggesting that ethnic minority parents may be less likely to seek parenting services (Baker et al., 2011) and may find the concept of EBTs less appealing (Becker et al., 2018), our first hypothesis predicted there would be differences in attitudes, stigma, and help seeking intentions for parents in the Spanish and English advertisement conditions. We also predicted a significant main effect of messenger. Given that most past research supports higher attitudes for expert sources (Kreuter & McClure, 2004), we predicted better perceptions of PCIT for the therapist testimonial. Finally, based on efforts to leverage similar peers to improve engagement for Spanish-speaking parents (Barnett et al., 2018; Gustafson et al., 2018), we explored if there was an interaction between language and messenger on parental attitudes towards PCIT. We hypothesized that parents in the Spanish advertisement condition would have improved attitudes, lower stigmatization, and higher help seeking intentions when the testimonial was provided by a parent.

Method

Participants

A total of 204 participants were recruited through Amazon’s MTurk under the restrictions that they were residents of the United States and were parents (set through MTurk’s built in settings). Participants were 61.8% male, 59.8% Latinx, 14.9% Black/ African American, 5.0% Asian/ Pacific Islander, and 26.5% non-Latinx, White. On average, participants were 32.61 (SD = 7.14) years old. Participants born outside of the United States (n = 172), reported living in the United States an average of 17.53 (SD = 8.48) years. Of the participants who self-identified as Latinx (n = 122, 59.8%) 40.2% were Mexican (n = 49), 21.3% Caribbean (n = 26), 9.8% Central American (n = 12), 19.7.8% South American (n = 24), and 9.0% from other Latinx or Spanish origin (n = 11). Participants’ reported annual income included <$30,000 (n = 37, 18.14%), $30,000 to $59,999 (n = 79, 38.73%), $60,000 to $79,999 (n = 39, 19.12%), and $80,000 or more (n = 49, 24.02%). See Table 1 for a comparison of demographics for participants who completed the study in English and in Spanish.

Table 1.

Differences between participants who took the survey in English vs. Spanish.

Spanish (n = 101) English (n = 103)

% n % n t or χ2
Gender χ2(1) = 0.01
 Male 61.4 62 62.1 64
 Female 38.6 39 37.9 39
Ethnicity χ2(1) = 97.65**
 Not Latinx 5.9 6 73.8 76
 Latinx 94.1 95 26.2 27
Race χ2(5) = 23.73**
 White 59.4 60 62.1 64
 African-American 6.9 7 22.3 23
 American Indian 9.9 10 1.9 2
 Asian-American 3.0 3 6.8 7
 Multiracial 8.9 9 3.9 4
 Not Listed 10.9 11 1.9 2
Immigration Status χ2(1) = 18.46**
 Born outside of the US 26.7 27 4.9 5
 Born in the US 73.3 74 95.1 98
Education χ2(3) = 12.57*
 High School/GED or less 28.7 29 19.4 20
 Associates Degree/Technical School 20.8 21 21.4 22
 Bachelor’s Degree 30.7 31 51.5 53
 Graduate/Professional School 19.8 20 7.8 8
Parent Relationship to the child χ2(3) = 9.88*
 Biological Parent 74.3 75 88.3 91
 Step-Parent 5.9 6 6.8 7
 Adoptive Parent 14.9 15 2.9 3
 Foster Parent 2.0 2 1.0 1
Income χ2(3) = 1.91
 Less than $30,000 17.8 18 18.4 19
 $30,000 to $59,999 35.6 36 41.7 43
 $60,000 to $79,999 22.8 23 15.5 16
 $80,000 or more 23.8 24 24.3 25

Note.

*

= p < .05

***

= p < .001.

Procedure

Based on recommendations for completing research with parents on MTurk, participants were first asked three screener questions to determine if they were eligible for the study (Schleider & Weisz, 2015). Screener questions were designed so that it would be difficult for participants to guess eligibility requirements, to limit the possibility that individuals gained access to the study through false answers. Participants were asked about their native language, if they were parents, and if they had a child between the ages of 0–1 year, 2–7 years, or 8 and older. Only parents who endorsed that they had a child between the ages of 2 and 7 years old were eligible to participate in the study. Additionally, individuals who participated in the study in Spanish responded to four additional screening questions regarding the language 1) they read in, 2) spoke at home, 3) thought in, and 4) spoke with friends to include fluent Spanish speakers in the sample. For each language question, they chose between 1) only English, 2) more English than Spanish, 3) both equally, 4) more Spanish than English, or 5) only Spanish. In order to obtain a sample consisting of fluent Spanish speakers, participants who selected, “only English or “more English than Spanish” for any of these items were not eligible for the study. Non-eligible participants were taken to the end of the survey link and told they were not eligible to continue in the study. A total of 650 individuals completed the screener (Spanish: n = 401; English; n = 249), of which 204 were eligible to participate in the study (Spanish: n = 101; English; n = 103). The exclusion rate was greater for Spanish-speaking parents given that they had to meet the child age and language requirements.

The eligible participants completed a waiver of consent and then filled out a demographic questionnaire. They were then assigned to an advertisement condition using the Randomizer setting in Qualtrics, which evenly and randomly distributed messenger conditions (peer or expert) to participants. Participants were not allowed to advance in the survey until the advertisement stopped playing. After the advertisement, participants completed a questionnaire regarding their attitudes toward PCIT. At the end of the study participants were thanked for their participation, provided with the PCIT International website link, and compensated $1 for their participation. The Institutional Review Board at [masked for review] approved all procedures of this study.

Advertisement Conditions.

Advertisements were developed by the research team after reviewing the literature on recommendations for DTC advertisements (e.g, (Friedberg & Bayar, 2017; Gallo et al., 2013; Santucci et al., 2012) and strategies for improving framing of parenting programs for Latinx parents (e.g., McCabe & Yeh, 2009; Niec et al., 2014; Winslow et al., 2016). After developing scripts, the actress, a bilingual, Latina woman with a background in Chicana feminist theater, reviewed the materials and made recommendations to improve the ease of readability and comprehension. The advertisements used the same actress, who recorded the advertisement in Spanish and English, and served in the role as the parent and therapist. This resulted in four advertisement versions: peer-English, peer-Spanish, expert-English, and expert-Spanish. Participants in each advertisement condition only watched the corresponding advertisement.

All advertisements were designed to be the same length (approximately 90 seconds) and include similar messages. The therapist and parent advertisement conditions both described PCIT as an evidence-based therapy that teaches parents skills to manage difficult behaviors in young children. Beyond the testimonies provided by the therapist or parent, the advertisement portrayed how live coaching occurs through the two-way mirror using bug-in- the ear technology. The advertisements with a parent included more personal framing of why the mother sought treatment (e.g., “My son started to misbehave and not listen to me”), whereas the therapist videos spoke broadly about how the treatment is for children with problematic behaviors. Each script normalized feelings of hesitation at starting therapy, but the therapist spoke more generally about the experience, while the parent spoke in the first person. For example, in the therapist version the actress said, “When you first begin to seek PCIT, it’s normal to feel hesitant,” whereas the parent version said, “At first, I was hesitant.” Both versions also described how the parent is with their child, “every step of the way.” For the expert condition, an actress playing the role of a therapist described PCIT and was shown welcoming the family in the treatment room and providing live coaching. The same father and child were present in all advertisement conditions. However, since both conditions show the family entering the treatment room and receiving coaching, a different actress portrayed the mother or therapist in the background shots depending on the role played by the actress giving the testimonial (i.e., during the parent testimonial the other actress was the therapist in background shots).

Measures

Demographics Questionnaire.

The demographic variables included age, gender, ethnicity, race, highest level of education, and approximate household income. Participants were also asked about their relationship with their child (e.g., biological, step-parent, adoptive, foster, other). Given the interest in Latinx immigrant populations, participants were asked about their country of origin and generational status. For participants born abroad, they were asked about their length of time living in the United States.

Parent Attitudes Toward PCIT.

Participants’ attitudes about PCIT were measured using an adaptation of the Parental Attitudes Toward Psychological Services Inventory (Turner, 2012). The PATPSI consists of 26 Likert- type items, with a scale from 0 (strongly disagree) to 5 (strongly agree). A confirmatory factor analysis supported three factors for the PATSPI (Turner, 2012): Help-Seeking Intentions (the extent to which they are willing and able to seek professional psychological help), Help-Seeking Attitudes (recognition that psychological problems exist and being open to the possibility of seeking professional help), and Stigmatization (extent to which they are concerned about what others might think if they knew the parent was seeking professional help). We used a composite mean score to measure each subscale, with higher scores representing more positive attitudes, higher likelihood of seeking services, and more stigma toward services. The PATSPI has shown good internal and test-retest reliability (Turner, 2012). In the current study, items were adapted to be specific to seeking PCIT as opposed to services in general. The measure was translated by a bilingual researcher and reviewed for comprehension by a monolingual Spanish speaker. See Table 2 for examples of the adapted items and original items. Cronbach’s alpha coefficients indicated good reliability for the three subscales: Help-Seeking Intentions (α = .84), Stigmatization (α = .93), and Help- Seeking Attitudes (α = .88)

Table 2.

Examples of Adapted Items from the PATPSI

Original Item Adapted Item
Help-Seeking Intentions 1. I would want to get professional help if my child were worried or upset for a long period of time. 1. I would want to seek out PCIT if my child were disobedient and disrespectful for a long period of time.
2. If I believed my child was having a mental breakdown, my first decision would be to get professional help. 2. If I believed my child was having behavioral difficulties, my first decision would be to seek out PCIT.

Stigmatization 3. I would not want to take my child to a professional because what people might think. 3. I would not want to take my child to PCIT because of what people might think.
4. I would be uncomfortable seeking professional help for my child because people (friends, family, coworkers, etc.) might find out about it. 4. I would be uncomfortable seeking out PCIT for my child because people (friends, family, coworkers, etc.) might find out about it.

Help-Seeking Attitudes 5. Seeking professional help is a sign of weakness. 5. Seeking out PCIT is a sign of weakness.
6. People should workout their own problems instead of getting professional help. 6. People should work out their own problems instead of using PCIT.

Data Analysis

Data was first analyzed for missing data. The proportion of missing data across variables of interest (race/ethnicity and PCIT help-seeking intentions, stigma and attitudes) ranged from 0.5 – 1.5%, which are below the 5% “cutoff” utilized in the literature (Dong & Peng, 2013; Schafer, 1999). Given these small proportions and that PCIT help-seeking intentions, stigma and attitudes yield an average composite score, we did not conduct further missing data analyses. To determine potential differences in demographic variables between participants who took the survey in English vs. Spanish, chi-square and t-tests were conducted. Further, given potential differences in attitudes towards and readiness to engage in PCIT between mothers and fathers (Niec, Barnett, Gering, Triemstra, & Solomon, 2015; Tiano, Grate, & McNeil, 2013), t-tests were conducted to see if there were sex differences on the three scales of the adapted PATPSI, to determine if sex should be included as a covariate in the model.

A 2 × 2 between subjects ANOVA (Language: Spanish or English) x (Video Messenger: Parent or Therapist) was used to determine the impact of language and messenger on participant’s PCIT related Help-Seeking Intentions, Help-Seeking Attitudes, and Stigmatization.

Results

Demographic Differences in Attitudes

Comparisons between participants who completed the study in Spanish and English revealed differences in race, ethnicity, generational status, and parent relationship to the child. Table 1 shows these demographic differences. To test if there were sex differences on PCIT Attitudes, t-tests were conducted between male and female participants. No significant differences were identified for Help-Seeking Intentions [t(1, 202) = −0.92, p = .358], Help-Seeking Attitudes [t(1, 202) = −1.86, p = .065], and Stigmatization [t(1, 202) = 0.46, p = .643], therefore sex was not entered into the model.

Impact of Language and Messenger on PCIT Attitudes

For PCIT Help-Seeking Intentions, no main effect of language was found [F(1, 200) = 2.71, p = .10] such that participants’ PCIT Help-Seeking Intentions were not different for those who took the survey in Spanish (M = 3.19, SD = 1.04) as compared to those who took the survey in English (M = 3.40, SD = 1.05). Additionally, the main effect of video messenger on PCIT Help-Seeking Intentions was not significant [F(1, 200) =.53, p = .47.], such that participants’ PCIT Help-Seeking Intentions were not different when the video messenger was a parent (M = 3.24, SD = 1.17) as compared to when the video messenger was a therapist (M = 3.33, SD = .93). However, there was a significant interaction between the language and messenger on PCIT Help-Seeking Intentions, [F(1,200) = 6.35, p = .013]. Post-hoc simple effects analyses were conducted to see if the impact of messenger differed for parents in the Spanish as compared to the English condition. There was a statistically significant difference between messengers for participants in the Spanish condition, with higher help seeking intentions when the messenger was a therapist than a parent, F(1,200) = 5.162, p = .024. Parents in the English advertisement condition did not have a significant difference in help seeking intentions between messengers, [F(1,200) = 1.64, p = .202 ]. Furthermore, participants in the Spanish condition reported significantly lower help seeking intentions when the messenger was a parent in comparison to participants in the English condition, [F(1,200) = 7.96, p = .005]. There was no difference between language conditions when the messenger was a therapist, [F(1,200)= .42, p = .519]. Figure 1 shows the interaction.

Figure 1.

Figure 1.

PCIT Help-Seeking Intentions by video messenger and language.

For the PCIT Stigmatization (Figure 2) and Attitudes (Figure 3) subscales, no main effects of language, or video messenger were found, ps > .05 (Table 3). The language X video messenger interaction term was not statistically significant for either scale.

Figure 2.

Figure 2.

PCIT Stigmatization by video messenger and language.

Figure 3.

Figure 3.

PCIT Attitudes by video messenger and language.

Table 3.

Main and Interaction Effects of Language and Messenger on Parental Attitudes towards PCIT

Effects F df p
Help-Seeking Intentions
   Language 2.71 1, 200 .101
   Messenger 0.53 1, 200 .468
   Language * Messenger 6.35 1, 200 .013
Stigmatization
   Language 0.57 1, 200 .451
   Messenger 0.29 1, 200 .589
   Language * Messenger 0.56 1, 200 .457
Attitudes
   Language 0.42 1, 200 .516
   Messenger 0.25 1, 200 .616
   Language * Messenger 0.01 1, 200 .943

Note. PCIT = Parent-Child Interaction Therapy.

Discussion

This study developed and tested DTC advertising for PCIT, an evidence-based parenting intervention for children with early onset conduct problems or histories of physical maltreatment. Contrary to our hypothesis, language did not influence parental perceptions of PCIT, with Spanish-speaking parents reporting similar attitudes, stigmatization, and help-seeking intentions towards PCIT as English-speaking parents. Furthermore, there was not a main effect for messenger on any of the outcomes. However, messenger type moderated the relationship between language and parental intentions to seek PCIT in an unexpected direction. Spanish-speaking participants reported a greater likelihood towards seeking PCIT services when the advertisement was delivered by a PCIT therapist rather than a parent. Further, Spanish-speaking parents reported being less likely than English-speaking parents to seek PCIT when the messenger was a parent. For parents in the English condition, there were no significant differences based on messenger. Overall, these results suggest that when developing DTC advertising, it may be important to tailor who delivers the message to increase initial treatment engagement across groups.

The finding that Spanish-speaking parents held similar attitudes and help-seeking intentions towards seeking PCIT as English-speaking parents was encouraging given previously documented disparities in acceptability of mental health services and help-seeking behaviors (Turner et al., 2016; Baker et al., 2011). These positive findings might have been related to the fact that the advertisements were available in Spanish and English and provided messages related to Latino values of strong familial relationships (Calzada, 2010). That is to say, DTC may be an effective strategy to reach Spanish-speaking parents if it is delivered in a culturally relevant way. Nevertheless, given well documented disparities in access and utilization for Spanish-speaking families, it is important to continue to investigate which additional factors prevent Spanish-speaking, Latinx families from enrolling in services and how they may still impact initial engagement even if parents report equally positive attitudes towards receiving the intervention.

We had hypothesized that Spanish-speaking participants’ perceptions of PCIT would improve if they viewed the video presented by a parent rather than a therapist based on suggestions that near peer messengers can decrease stigma (Barnett et al., 2018; Calzada et al., 2005). This prediction was not supported in the current study. In fact, the results suggested that Spanish-speaking parents reported being more likely to seek PCIT when the video messenger was a therapist than when it was delivered by a parent. This may indicate the importance of considering specific cultural values that may be relevant to further explore Latinx parents’ attitudes toward PCIT. For instance, the cultural value of respeto, which highlights the importance of obedience and respect for parents, elders and those in higher authority positions and/or social status, has been commonly embraced among Latinx parents and would indicate that parents may be more likely to attend to messages delivered by someone with more training (Calzada, Fernandez, & Cortes, 2010; Gonzalez-Ramos, Zayas, & Cohen, 1998; Winslow et al., 2016). It is important to note that language was used as a proxy for acculturation, and future research should use a robust measure of this construct to fully understand if DTC marketing efforts differ across levels of acculturation as was found in past research on parental engagement strategies (Winslow et al., 2016).

Strengths, Limitations, and Future Directions

The current study has a number of strengths, making it an important first step to understand the efficacy of DTC advertisement to promote PCIT interest among parents. Though we specifically focused on PCIT, findings could be applicable to a range of evidence-based parenting interventions, which could improve efforts to improve initial engagement in these interventions. Consistent with past research using MTurk (Schleider & Weisz, 2015a), the current study recruited a significant number of fathers. This strength is of particular interest as fathers entail an underrepresented group in parenting research and may be more challenging to engage in parenting programs (Fabiano & Caserta, 2018; Niec et al., 2015). The advertisements in this study were not tailored for fathers, and only showed a mother or female therapist delivering messages about PCIT. Though our results did not find significant differences in attitudes or help seeking intentions for PCIT between mothers and fathers, it still may be beneficial for future research to focus on strategies that tailor DTC advertisement for PCIT and other evidence-based parenting programs to increase father involvement. For example, it may be helpful to have fathers or male therapists provide testimonies about PCIT. The inclusion of Spanish-speaking parents was a significant strength of this study, making it one of the first to investigate how to tailor advertisements to address disparities for monolingual or bilingual Spanish-speaking parents.

Nevertherless, It is important to consider the limitations of this study when interpreting the results. First, this study only compared different video advertisements, as opposed to comparing different media forms of advertisement. Therefore, we are not able to state if videos compare differently to other marketing strategies, which might be less costly to produce, such as brochures. Though participants were not able to progress in the study until the advertisement was completed, we did not test how well they attended to the information, so we are not able to state how their comprehension of the material impacted the outcomes. Another significant limitation of this study is that it only measured attitudes and intentions to seek PCIT, without measuring whether parents would actually seek out PCIT services. It is also important to note that the wording on the original PATPSI measure was modified to reflect emphasis on PCIT attitudes in the current study. While this modified version was shown to have good internal consistency in the current sample, additional research is needed to examine the reliability and validity of this measure. Though this is similar to past studies on DTC advertisements, an important next step is to evaluate if DTC marketing actually changes parent help seeking behaviors (Becker et al., 2018; Gallo et al., 2015). This is especially important, as past research has shown that even though parents with higher intentions are more likely to enroll in parenting programs, very few parents actually do so (Thornton & Calam, 2011). Another limitation to the generalizability of the findings is that child behavior problems were not measured, so it was not clear if parents in our study would actually need PCIT services. Future studies may be interested in exploring the role of child behavior problems in the effectiveness of DTC marketing, especially given that the severity of child behavior problems has been linked to actual enrollment in services (Thornton & Calam, 2011).

Increased parent engagement is necessary to improve the implementation, sustainment, and public health impact of PCIT and other EBTs. In line with this need, the development and dissemination of informational videos and testimonials about EBTs is currently growing and are available online from a number of intervention developers, including the parenting interventions, Incredible Years (http://www.incredibleyears.com/about/video-testimonals/) and Triple P (https://www.triplep.net/glo-en/home/). Further large systems of care, such as the United States Veteran’s Administration (https://www.ptsd.va.gov/apps/aboutface/), and the state of Hawaii (Okamura et al., 2018) have developed websites with videos to increase awareness about evidence-based services. Additional research is warranted on how these strategies improve access and utilization for EBTs, especially for those who have been traditionally underserved. If DTC marketing is successful at increasing initial engagement in EBTs, such as PCIT, community mental health agencies may benefit from having these advertising materials, such as videos, readily available to include on their websites and social media accounts.

Acknowledgments

Funding: This study was funded by the McNair Scholars Program awarded to NB. Effort to write this manuscript was supported by the National Institute of Mental Health (K01MH110608) awarded to MLB. Additionally, the preparation of this article was supported in part by the Implementation Research Institute (IRI) at the Brown School, Washington University in St. Louis, through an award from the National Institute of Mental Health (R25 MH080916). MLB is a fellow of IRI.

Footnotes

Publisher's Disclaimer: This Author Accepted Manuscript is a PDF file of an unedited peer-reviewed manuscript that has been accepted for publication but has not been copyedited or corrected. The official version of record that is published in the journal is kept up to date and so may therefore differ from this version.

Ethical approval: All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. The protocol for this study was approved by the Institutional Review Board at the University of California, Santa Barbara.

Informed consent: Informed consent was obtained from all individual participants included in the study.

References

  1. Ajzen I (1985). From intentions to actions: A Theory of Planned Behavior In Action Control (pp. 11–39). Berlin, Heidelberg: Springer Berlin Heidelberg; 10.1007/978-3-642-69746-3_2 [DOI] [Google Scholar]
  2. Baker CN, Arnold DH, & Meagher S (2011). Enrollment and attendance in a parent training prevention program for conduct problems. Prevention Science, 12(2), 126–138. 10.1007/s11121-010-0187-0 [DOI] [PubMed] [Google Scholar]
  3. Barnett ML, Davis EM, Callejas LM, White JV, Acevedo-Polakovich ID, Niec LN, & Jent JF (2016). The development and evaluation of a natural helpers’ training program to increase the engagement of urban, Latina/o families in parent-child interaction therapy. Children and Youth Services Review, 65, 17–25. 10.1016/j.childyouth.2016.03.016 [DOI] [Google Scholar]
  4. Barnett ML, Lau AS, & Miranda J (2018). Lay health worker involvement in evidence-based treatment delivery: A conceptual model to address disparities in care. Annual Review of Clinical Psychology, 14(1), 185–208. 10.1146/annurev-clinpsy-050817-084825 [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Becker SJ (2015). Evaluating whether direct-to-consumer marketing can increase demand for evidence-based practice among parents of adolescents with substance use disorders: rationale and protocol. Addiction Science & Clinical Practice, 10(1), 4 10.1186/s13722-015-0028-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Becker SJ, Weeks BJ, Escobar KI, Moreno O, DeMarco CR, & Gresko SA (2018). Impressions of “evidence-based practice”: A direct-to-consumer survey of caregivers concerned about adolescent substance use. Evidence-Based Practice in Child and Adolescent Mental Health, 3(2), 70–80. 10.1080/23794925.2018.1429228 [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Beveridge RM, Fowles TR, Masse JJ, McGoron L, Smith MA, Parrish BP, … Widdoes N (2015). State-wide dissemination and implementation of parent-child interaction therapy (PCIT): Application of theory. Children and Youth Services Review, 48, 38–48. 10.1016/j.childyouth.2014.11.013 [DOI] [Google Scholar]
  8. Calzada EJ (2010). Bringing culture into parent training with Latinos. Cognitive and Behavioral Practice, 17(2), 167–175. 10.1016/j.cbpra.2010.01.003 [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Calzada EJ, Caldwell MB, Brotman LM, Brown EJ, Wallace SA, McQuaid JH, … O’Neal CR (2005). Training community members to serve as paraprofessionals in an evidence-based, prevention program for parents of preschoolers. Journal of Child and Family Studies. 10.1007/s10826-005-6851-5 [DOI]
  10. Calzada EJ, Fernandez Y, & Cortes DE (2010). Incorporating the cultural value of respeto into a framework of Latino parenting. Cultural Diversity & Ethnic Minority Psychology, 16(1), 77–86. 10.1037/a0016071 [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Chaffin M, Silovsky JF, Funderburk B, Valle LA, Brestan EV, Balachova T, … Bonner BL (2004). Parent-child interaction therapy with physically abusive parents: Efficacy for reducing future abuse reports. Journal of Consulting and Clinical Psychology, 72(3), 500–510. 10.1037/0022-006X.72.3.500 [DOI] [PubMed] [Google Scholar]
  12. Chandler J, & Shapiro D (2016). Conducting clinical research using crowdsourced convenience samples. Annual Review of Clinical Psychology, 12(1), 53–81. 10.1146/annurev-clinpsy-021815-093623 [DOI] [PubMed] [Google Scholar]
  13. Donohue JM, Cevasco M, & Rosenthal MB (2007). A decade of direct-to-consumer advertising of prescription drugs. New England Journal of Medicine, 357(7), 673–681. 10.1056/NEJMsa070502 [DOI] [PubMed] [Google Scholar]
  14. Fabiano GA, & Caserta A (2018). Future directions in father inclusion, engagement, retention, and positive outcomes in child and adolescent research. Journal of Clinical Child and Adolescent Psychology, 47(5), 847–862. 10.1080/15374416.2018.1485106 [DOI] [PubMed] [Google Scholar]
  15. Friedberg RD, & Bayar H (2017). If it works for pills, can it work for skills? direct-to-consumer social marketing of evidence-based psychological treatments. Psychiatric Services, 68(6), 621–623. 10.1176/appi.ps.201600153 [DOI] [PubMed] [Google Scholar]
  16. Gallo KP, Comer JS, & Barlow DH (2013). Direct-to-consumer marketing of psychological treatments for anxiety disorders. Journal of Anxiety Disorders, 27(8), 793–801. 10.1016/j.janxdis.2013.03.005 [DOI] [PubMed] [Google Scholar]
  17. Gallo KP, Comer JS, Barlow DH, Clarke RN, & Antony MM (2015). Direct-to-consumer marketing of psychological treatments: A randomized controlled trial. Journal of Consulting and Clinical Psychology, 83(5), 994–8. 10.1037/a0039470 [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Gonzalez-Ramos G, Zayas LH, & Cohen EV (1998). Child-rearing values of low-income, urban Puerto Rican mothers of preschool children. Professional Psychology: Research and Practice, 29(4), 377–382. 10.1037/0735-7028.29.4.377 [DOI] [Google Scholar]
  19. Gonzalez C, Morawska A, & Haslam DM (2018). Enhancing initial parental engagement in interventions for parents of young children: A systematic review of experimental studies. Clinical Child and Family Psychology Review, 21(3), 415–432. 10.1007/s10567-018-0259-4 [DOI] [PubMed] [Google Scholar]
  20. Gustafson EL, Atkins M, & Rusch D (2018). Community health workers and social proximity: Implementation of a parenting program in urban poverty. American Journal of Community Psychology, 62(3–4), 449–463. 10.1002/ajcp.12274 [DOI] [PubMed] [Google Scholar]
  21. Kalichman SC, & Coley B (1995). Context framing to enhance HIV-antibody-testing messages targeted to African American women. Health Psychology, 14(3), 247–254. 10.1037/0278-6133.14.3.247 [DOI] [PubMed] [Google Scholar]
  22. Kaminski JW, Valle LA, Filene JH, & Boyle CL (2008). A meta-analytic review of components associated with parent training program effectiveness. Journal of Abnormal Child Psychology, 36(4), 567–589. 10.1007/s10802-007-9201-9 [DOI] [PubMed] [Google Scholar]
  23. Kim G, Aguado Loi CX, Chiriboga DA, Jang Y, Parmelee P, & Allen RS (2011). Limited English proficiency as a barrier to mental health service use: A study of Latino and Asian immigrants with psychiatric disorders. Journal of Psychiatric Research, 45(1), 104–110. 10.1016/J.JPSYCHIRES.2010.04.031 [DOI] [PubMed] [Google Scholar]
  24. Kreuter MW, & McClure SM (2004). The role of culture in health communication. Annual Review of Public Health, 25(1), 439–455. 10.1146/annurev.publhealth.25.101802.123000 [DOI] [PubMed] [Google Scholar]
  25. Lanier P, Kohl PL, Benz J, Swinger D, Moussette P, & Drake B (2011). Parent-child interaction therapy in a community setting: Examining outcomes, attrition, and treatment setting. Research on Social Work Practice, 21(6), 689–698. 10.1177/1049731511406551 [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Lieneman CC, Brabson LA, Highlander A, Wallace NM, & McNeil CB (2017). Parent-Child Interaction Therapy: current perspectives. Psychology Research and Behavior Management, 10, 239–256. 10.2147/PRBM.S91200 [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Matos M, Bauermeister JJ, & Bernal G (2009). Parent-Child Interaction Therapy for Puerto Rican preschool children with ADHD and behavior problems: A pilot efficacy study. Family Process, 48(2), 232–252. 10.1111/j.1545-5300.2009.01279.x [DOI] [PubMed] [Google Scholar]
  28. McCabe K, & Yeh M (2009). Parent-child interaction therapy for Mexican Americans: A randomized clinical trial. Journal of Clinical Child and Adolescent Psychology, 38(5), 753–759. 10.1080/15374410903103544 [DOI] [PubMed] [Google Scholar]
  29. McCabe K, Yeh M, Lau A, & Argote CB (2012). Parent-Child Interaction Therapy for Mexican Americans: Results of a Pilot Randomized Clinical Trial at Follow-up. Behavior Therapy, 43(3), 606–618. 10.1016/J.BETH.2011.11.001 [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Morawska A, Nitschke F, & Burrows S (2011). Do testimonials improve parental perceptions and participation in parenting programmes? results of two studies. Journal of Child Health Care, 15(2), 85–98. 10.1177/1367493510397625 [DOI] [PubMed] [Google Scholar]
  31. Niec LN, Acevedo-Polakovich ID, Abbenante-Honold E, Christian AS, Barnett ML, Aguilar G, & Peer SO (2014). Working together to solve disparities: Latina/o parents’ contributions to the adaptation of a preventive intervention for childhood conduct problems. Psychological Services, 11(4), 410–420. 10.1037/a0036200 [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Niec LN, Barnett ML, Gering CL, Triemstra K, & Solomon DT (2015). Differences in mothers’ and fathers’ readiness for change in parent training. Child and Family Behavior Therapy, 37(3), 224–235. 10.1080/07317107.2015.1071980 [DOI] [Google Scholar]
  33. Niec LN, Barnett ML, Prewett MS, & Shanley Chatham JR (2016). Group Parent-Child Interaction Therapy: A randomized control trial for the treatment of conduct problems in young children. Journal of Consulting and Clinical Psychology, 84(8), 682–698. 10.1037/a0040218 [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Okamura KH, Orimoto TE, Mah AC, Slavin LA, Rocco S, Shimabukuro SK, … Nakamura BJ (2018). Insights in public health: The Help Your Keiki website: Increasing youth and caregiver awareness of youth psychosocial Mental Health Treatment. Hawai’i Journal of Medicine & Public Health : A Journal of Asia Pacific Medicine & Public Health, 77(8), 203–207. [PMC free article] [PubMed] [Google Scholar]
  35. Ortega AN, McKenna RM, Kemmick Pintor J, Langellier BA, Roby DH, Pourat N, … Wallace SP (2018). Health care access and physical and behavioral health among undocumented Latinos in California. Medical Care, 56(11), 919–926. 10.1097/MLR.0000000000000985 [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Pearl E, Thieken L, Olafson E, Boat B, Connelly L, Barnes J, & Putnam F (2012). Effectiveness of community dissemination of parent-child interaction therapy. Psychological Trauma: Theory, Research, Practice, and Policy, 4(2), 204–213. 10.1037/a0022948 [DOI] [Google Scholar]
  37. Regan J, Lau AS, Barnett M, Stadnick N, Hamilton A, Pesanti K, … Brookman-Frazee L (2017). Agency responses to a system-driven implementation of multiple evidence-based practices in children’s mental health services. BMC Health Services Research, 17(1). 10.1186/s12913-017-2613-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Salari R, & Backman A (2016). Direct marketing of parenting programs: comparing a promotion-focused and a prevention-focused strategy. The European Journal of Public Health, 27(3), ckw149 10.1093/eurpub/ckw149 [DOI] [PubMed] [Google Scholar]
  39. Santucci L, McHugh R, & Barlow D (2012). Direct-to-consumer marketing of evidence-based psychological interventions: Introduction. Behavior Therapy, 43(2), 231–235. [DOI] [PubMed] [Google Scholar]
  40. Schleider JL, & Weisz JR (2015a). Using Mechanical Turk to study family processes and youth mental health: A test of feasibility. Journal of Child and Family Studies, 24(11), 3235–3246. 10.1007/s10826-015-0126-6 [DOI] [Google Scholar]
  41. Schleider JL, & Weisz JR (2015b). Using Mechanical Turk to Study Family Processes and Youth Mental Health: A Test of Feasibility. Journal of Child and Family Studies, 24(11), 3235–3246. 10.1007/s10826-015-0126-6 [DOI] [Google Scholar]
  42. Schuhmann EM, Foote RC, Eyberg SM, Boggs SR, & Algina J (1998). Efficacy of Parent-Child Interaction Therapy: Interim report of a randomized trial with short-term maintenance. Journal of Clinical Child Psychology, 27(1), 34–45. 10.1207/s15374424jccp2701_4 [DOI] [PubMed] [Google Scholar]
  43. Scudder AT, & Herschell AD (2015). Building an evidence-base for the training of evidence-based treatments in community settings: Use of an expert-informed approach. Children and Youth Services Review, 55, 84–92. 10.1016/j.childyouth.2015.05.003 [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Sentell T, Shumway M, & Snowden L (2007). Access to mental health treatment by English language proficiency and race/ethnicity. Journal of General Internal Medicine, 22(SUPPL. 2), 289–293. 10.1007/s11606-007-0345-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Stokes JO, Wallace NM, & McNeil CB (2018, October 9). Effectiveness of community-delivered Parent- Child Interaction Therapy compared to usual care Child and Family Behavior Therapy, pp. 1–27. Routledge; 10.1080/07317107.2018.1522232 [DOI] [Google Scholar]
  46. Thomas R, Abell B, Webb HJ, Avdagic E, & Zimmer-Gembeck MJ (2017). Parent-Child Interaction Therapy: A meta-analysis. Pediatrics, 140(3), e20170352 10.1542/peds.2017-0352 [DOI] [PubMed] [Google Scholar]
  47. Thornton S, & Calam R (2011). Predicting intention to attend and actual attendance at a universal parent-training programme: A comparison of social cognition models. Clinical Child Psychology and Psychiatry, 16(3), 365–383. 10.1177/1359104510366278 [DOI] [PubMed] [Google Scholar]
  48. Tiano JD, Grate RM, & McNeil CB (2013). Comparison of mothers’ and fathers’ opinions of Parent–Child Interaction Therapy. Child & Family Behavior Therapy, 35(2), 110–131. 10.1080/07317107.2013.789358 [DOI] [Google Scholar]
  49. Timmer SG, Urquiza AJ, Boys DK, Forte LA, Quick-Abdullah D, Chan S, & Gould W (2016). Filling potholes on the implementation highway: Evaluating the implementation of Parent-Child Interaction Therapy in Los Angeles County. Child Abuse and Neglect, 53, 40–50. 10.1016/j.chiabu.2015.11.011 [DOI] [PubMed] [Google Scholar]
  50. Turner EA (2012). The Parental Attitudes Toward Psychological Services Inventory: Adaptation and development of an attitude Scale. Community Mental Health Journal, 48(4), 436–449. 10.1007/s10597-011-9432-7 [DOI] [PubMed] [Google Scholar]
  51. Turner EA, Jensen-Doss A, & Heffer RW (2015). Ethnicity as a moderator of how parents’ attitudes and perceived stigma influence intentions to seek child mental health services. 10.1037/cdp0000047 [DOI] [PubMed]
  52. Turner EA, & Liew J (2010). Children’s adjustment and child mental health service use: The role of parents’ attitudes and personal service use in an upper middle class sample. Community Mental Health Journal, 46(3), 231–240. 10.1007/s10597-009-9221-8 [DOI] [PubMed] [Google Scholar]
  53. Turner EA, Cheng H-L, Llamas JD, Tran AGTT, Hill KX, Fretts JM, & Mercado A (2016). Factors impacting the current trends in the use of outpatient psychiatric treatment among diverse ethnic groups. Current Psychiatry Reviews, 12(2), 199–220. 10.2174/1573400512666160216234524 [DOI] [Google Scholar]
  54. Winslow EB, Poloskov E, Begay R, Tein J-Y, Sandler I, & Wolchik S (2016). A randomized trial of methods to engage Mexican American parents into a school-based parenting intervention. Journal of Consulting and Clinical Psychology, 84(12), 1094–1107. 10.1037/ccp0000140 [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES