Impressions of “Evidence-Based Practice”: A Direct-to-Consumer Survey of Caregivers Concerned about Adolescent Substance Use

Abstract

National behavioral health organizations have recently started using direct-to-consumer (DTC) marketing strategies as a means of promoting increased utilization of evidence-based practice (EBP). Such strategies often encourage patients and caregivers to proactively seek out EBP, based on the assumptions that patients and caregivers understand the concept and view it favorably. We conducted a DTC marketing survey of caregivers concerned about their adolescents’ substance use in order to explore how these caregivers define, value, and prefer to describe the EBP concept. We also examined whether caregiver perceptions of EBP vary by socio-demographic (race/ethnicity, income per capital, education level) and clinical (adolescent’s history of therapy) characteristics. A total of 411 caregivers (86% women, 88% Non-Hispanic White) of adolescents age 12 to 19 (M age = 16.1, SD = 1.8, 82% Non-Hispanic White) completed an online survey. Caregivers answered a series of questions evaluating assumed definitions of EBP, underlying EBP principles, the appeal of EBP, and alternate terms to describe EBP. Chi-square analyses and multivariate logistic regressions were used to examine which variables were associated with the greatest likelihood of response selection. Results indicated that most parents defined EBP correctly, valued EBP principles, and found EBP appealing. However, caregivers from racial/ethnic minority groups, with lower income per capita, and lower education were more likely to define EBP incorrectly and have negative impressions of the concept. Education level was the strongest and most consistent predictor of caregiver perceptions. Clinical implications for the development of targeted, accessible marketing messages are discussed.

Adolescent substance use (SU) remains a serious and persistent public health concern in the United States. Among adolescents, SU is related to negative long-term outcomes including mental health problems, school drop-out, legal problems, conflict with family and peers, and unintended pregnancy (The National Center on Addiction and Substance Abuse, 2011). Of greater concern, SU is associated with leading causes of death in this cohort: accidents, suicide, and violent behaviors (Mattson, Cai, & Woodward, 2015; Molcho, Walsh, Donnelly, de Matos, & Pickett, 2015). Given these deleterious consequences, it is a public health imperative to intervene with adolescents at risk of SU problems. In particular, leading national organizations have called for efforts to increase the utilization of evidence-based psychological treatments, most commonly referred to as evidence-based practice (EBP), in this population (Koob, 2015; National Institute on Drug Abuse, 2015).

Prior attempts to increase EBP utilization within the behavioral health field have historically been directed towards treatment providers in community settings (see Tabak, Khoong, Chambers, & Brownson, 2012). These strategies have aimed to educate and train providers in EBP based on the assumption that the greatest barriers to treatment utilization are at the provider-level (e.g., training, knowledge, competency; Becker, 2015; Gallo, Comer, & Barlow, 2013). Although addressing provider-level barriers is crucial to increase the supply of EBP in community settings, such strategies do not address patient demand for the services provided. Data from the 2016 National Survey on Drug Use and Health (Center for Behavioral Health Statistics and Quality, 2016) indicated that less than 1 in 10 adolescents with a diagnosable SU disorder received specialty treatment in the past year. When asked why they did not seek treatment, most adolescents reported that their problems could be handled without treatment or they did not know how to get help (Center for Behavioral Health Statistics and Quality, 2016). These data highlight the need for strategies to address patient-level barriers such as knowledge and awareness of treatment options. One promising and largely untapped patient-directed strategy that has been gaining attention in the treatment literature is direct-to-consumer (DTC) marketing (Becker, 2015; Gallo et al., 2013; Szymanski, 2012). Within the context of behavioral treatment, the goal of DTC marketing is to shape patient treatment-seeking behavior in order to increase the utilization of treatment in general and EBP in particular (Friedberg & Bayar, 2017; Gallo, Comer, Barlow, Clarke, & Antony, 2015).

Over the past few years, a few pioneering organizations including the American Psychological Association’s (APA) Society of Clinical Child and Adolescent Psychology (Division 53) and the National Institute on Drug Abuse have created DTC marketing and health education materials for caregivers of adolescents at risk of SU (see http://www.effectivechildtherapy.com and https://teens.drugabuse.gov/parents). Both of these websites encourage caregivers to proactively seek out behavioral treatments designated as “evidence-based.” For instance, the APA Division 53 website states “it is important to ask about the type of treatment that a [behavioral] health care provider will offer,” and counsels caregivers to seek out “treatments with scientific evidence supporting them… called evidence-based treatments.” Although intuitively appealing, such recommendations to seek out “evidence-based” treatment are predicated on two assumptions that require empirical testing: 1) caregivers understand the concept of “evidence-based”; and 2) caregivers view the concept favorably.

Unfortunately, prior research suggests that many health care consumers have inaccurate and negative impressions of the term “evidence-based.” Carman et al. (2010) surveyed over 1,500 general health consumers (defined as adults with health insurance) and concluded that people found terms such as “evidence,” “quality guidelines,” and “quality standards” unfamiliar and confusing. Furthermore, many consumers had concerns that “evidence-based” treatment would be inflexible, restrict personal choice, and limit the clinicians’ ability to tailor care to individual patients. In a qualitative study of 77 patients with mental health disorders, Tanenbaum (2008) similarly uncovered patient misgivings that evidence-based treatment would be formulaic and insensitive to patient preferences.

More recently, our team (Becker et al, 2016a) conducted a qualitative study of 29 caregivers and 25 adolescents to explore whether these findings held with potential consumers of adolescent SU treatment. Our study revealed that only two caregivers had ever heard the term EBP, only one caregiver was able to define the term correctly, and most caregivers perceived EBP as rigid and unappealing. Moreover, many of the caregivers and adolescents had incorrect assumptions about EBP principles. For instance, several participants assumed that the term “evidence-based” referred to treatment based on legal evidence, which had especially negative connotations for adolescents involved with the criminal justice system. Qualitative feedback also indicated that caregivers were responsible for driving decision-making about treatment (Becker et al, 2016b) suggesting that efforts to disseminate information about treatment should target caregivers directly.

To expand upon our prior qualitative study, we conducted a DTC marketing survey of caregivers concerned about their adolescents’ SU. While our qualitative research explored the breadth and diversity of caregiver perceptions, this survey examined the depth of caregiver perspectives in a larger, more representative sample. Use of a larger sample also enabled us to examine whether caregiver impressions of EBP varied as a function of socio-demographic characteristics (i.e., race/ethnicity, education, income per capita) and clinical characteristics (i.e., treatment experience) that have been shown to influence consumer reactions to DTC marketing (Deshpande, Menon, Perri, & Zinkhan, 2004; Soneji, Ambrose, Lee, Sargent, & Tanski, 2014).

The primary goal of this survey was to inform future DTC marketing for adolescent SU treatment by examining: a) assumed EBP definitions among caregivers concerned about their adolescents’ SU, b) whether caregivers valued EBP treatment principles, c) how appealing caregivers viewed EBP, and d) what terms caregivers preferred to describe EBP. A secondary goal was to examine whether caregiver perceptions varied as a function of socio-demographic and clinical variables: race/ethnicity, education, income, and adolescent’s treatment history. Due to the dearth of prior research, this survey was designed to be information-generating and exploratory in nature. We did conjecture, however, that caregivers with no history of therapy, lower education levels, lower income levels, and from racial/ethnic minority groups would have more negative and inaccurate impressions of EBP.

Methods

Sampling Strategy and Procedures

Advertisements for the online survey were shared between April 2015 and March 2016 via various channels including: emails to caregivers across six Rhode Island high schools, postings in private Facebook groups for caregivers of youth with behavioral health concerns, posters and flyers throughout the community, and emails on listservs of behavioral health providers working with adolescents. Advertisements indicated that caregivers were needed for a survey about impressions of adolescent SU treatment, and contained a link to an online screener. To reach a broad audience, all advertisement content was written at a middle-school grade level. According to the Flesch-Kinkaid readability statistics in Microsoft word, emails to local schools were at an 8.9 – 9.1 grade level, Facebook posts were all at a 6.7 grade level, posters/flyers were at a 6.7 – 7.7 level, and listserv emails were all at an 8.3 grade level.

Survey inclusion criteria required respondents to be the legal guardian of an adolescent aged 12 to 19; reside in the United States; and report elevated concern about their adolescent’s SU (i.e., rating of 4+ on a 5-point Likert scale ranging from “not at all” to “extremely” concerned). We focused on caregivers’ subjective concern and not an objective indicator of severity, based on our and other’s work suggesting that subjective impressions of health drive treatment-seeking behavior (see Becker et al., 2016b; Hunt, 1993). Online screeners contained multiple safeguards: non-essential questions to mask eligibility criteria, captcha verification, IP address check, use of cookies to prevent duplicate attempts, survey tagging to prevent search engine indexing, and a multiple choice question asking where the participant learned about the survey that included fake responses.

Caregivers deemed eligible were emailed one of two survey versions. Caregivers whose adolescents did not have a history of therapy for mental health or SU completed a shorter version containing questions about their impressions of EBP (focus of this study), preferred experiences selecting a provider, attributes of an ideal therapist, and family demographics/clinical characteristics. Caregivers whose adolescents did have a therapy history completed a longer version, which contained the aforementioned questions along with questions about their most recent therapy experience. Caregivers earned either a $10 or $20 Amazon gift card, depending which survey version they completed. The full survey was written at a 6.3 grade level. Survey procedures were deemed exempt by the University institutional review board.

Survey Instrument

The focus of this analysis was a series of seven questions (Table 1) exploring caregiver perceptions of EBP. For each item, response options were based directly on the caregiver feedback reported in our prior qualitative study (Becker et al, 2016a). Because we were interested in impressions of psychological treatment specifically, questions used the term “evidence-based therapy” instead of the more generic EBP term. Items on average required a 7.1 grade level, with a range from 5.2 to 10.2. Of note, the term “evidence-based therapy” tested at over a 12^th grade level, so two items explicitly evaluating this term required a higher reading level. Simply replacing the term “evidence-based therapy” with “therapy” would decrease the overall reading difficulty of the focal items to a 6.1 grade level, with a range from 5.2 to 6.7. Readability statistics of each item are elaborated below.

Table 1.

Survey Questions

Theme	Specific Questions about Evidence-Based Therapy
Assumed EBP Definition	1. When you hear the phrase “evidence-based therapy,” which of the following explanations sounds most accurate?
	Therapy based on proof that your teen is using
	Therapy based on research studies comparing different therapy approaches (correct definition)
	Therapy based on your teen’s personal history
	Therapy based on the therapist’s personal experience
	Therapy based on genetic testing
EBP Principles	2. Which of the following therapists would you prefer?
	Uses approaches that are proven to work
	Uses a different approach with every teenager
	3. Which of the following is more important to you from your teen’s therapist?
	The process of therapy
	The outcome of therapy
	4. Which of the following is more important to you from your teen’s therapist?
	Your teen likes the therapist
	Therapist uses an approach that has been found to work with teens with similar problems
EBP Appeal	5. How appealing is the phrase “evidence-based therapy” to you?
	1= Not at All Appealing
	5= Very Appealing
Preferred EBP Terms	6. Which of the following descriptions of therapy do you like the most?
	Proven therapy
	Successful therapy
	Therapy that works
	Better therapy
	Effective therapy
	7. Which of the following descriptions of therapy do you prefer?
	Individualized
	Varied
	Flexible
	Tailored
	Not rigid

Definitions of Evidence-Based Therapy.

To evaluate assumed definitions of “evidence-based therapy,” caregivers selected which of five definitions sounded most accurate. We examined the proportion of caregivers selecting the correct definition, which was evidence based on research studies. This item tested as requiring a 9.2 grade level as written. Replacing the term “evidence-based therapy” with “therapy” would decrease the reading difficulty of this item to a 6.5 grade level.

Exploration of Evidence-Based Principles.

A set of three items, each of which contained two response options, explored whether caregivers valued principles underlying evidence-based therapy. The first item explored whether caregivers would prefer to have their adolescent’s therapist use an approach that has been proven to work or an idiosyncratic/varied approach, while the second item examined whether caregivers placed greater value on the outcome or process of therapy. The final item asked caregivers whether they would be more confident in a therapist who their adolescent liked or who used an approach that had been shown to work with adolescents with similar problems. Each of these items required a 6.7 grade level.

Appeal of Evidence-Based Therapy.

Caregivers were asked to rate how appealing they found the phrase “evidence-based therapy” on a Likert scale from 1 to 5, with 1 being “not at all appealing”, and 5 being “very appealing”. Responses were recoded into two categories: “not appealing or indifferent” (i.e., responses of 1–3) versus “appealing” (i.e., responses of 4–5). This item tested as requiring a 10.2 grade level as written. Replacing “evidence-based therapy” with “therapy” would drop the reading difficulty of this item to a 5.2 grade level.

Preferred terms to describe EBP.

Caregivers were asked two questions about which terms to describe EBP they preferred. The first asked caregivers to select among four terms to describe “evidence-based”, while the second asked caregivers to select among five adjectives to describe patient-centric care. These items were written at a 5.2 and 6.5 grade level, respectively. We evaluated descriptions of patient-centric care due to our own qualitative research (Becker et al, 2016a) and other work (Carman et al., 2010; Tanenbaum, 2008) demonstrating that health care consumers have misgivings about the rigidity of EBP.

Potential moderators.

We also examined caregiver responses to questions about race/ethnicity, education level, and income per capita, as well as their adolescent’s history of therapy. Race/ethnicity was categorized as either Non-Hispanic White or minority, while education level was split as Associates and lower vs. Bachelor’s and higher. Income per capita was calculated based on caregiver estimates of overall household income divided by the number of people living in the home, and then dichotomized using a median split (< $25,000 vs. $25,000+). Adolescent therapy history was categorized as any vs. no history.

Analysis Plan

Before analyzing results, we examined bivariate associations among the potential moderators. Using the phi coefficient for binary variables, there were significant, but small associations among all four moderators (ɸ’s <.18, p’s <.038), with the exception of income and education, which had a moderate association (ɸ =.45, p <.001). None of these associations was large enough to preclude multi-variate analyses.

To address the first objective, we examined the overall distribution of responses for each of the seven items. For the second objective, we used chi-square analyses to examine whether the response distributions varied as a function of caregiver race/ethnicity, income per capita, education level, and teen’s therapy history. If responses differed by more than one variable, multivariate logistic regression was used to determine the relative importance of variables in predicting the likelihood of response selection.

Results

Sample Characteristics

A total of 844 individuals completed the screener, of which 499 (59%) were eligible and 350 (41%) were excluded. The most common reason for exclusion was lack of concern about SU (n =121). Another 11 caregivers were excluded because their adolescent was not aged 12–19. Other reasons pertained to the following safeguards: IP address did not match where participant reported taking the survey (n =106), IP or email address indicated duplicate attempt (n =60), “false” response was selected to validity check question about where the advertisement had been posted (n =45).

A total of 411 caregivers (82% of eligible) completed the survey. Of the completers, almost half (45%) were recruited via schools, with the remainder recruited via provider listservs (40%) and private parent Facebook groups (15%). Sixty-two percent of caregivers completed the short version and 38% completed the long version. Median length of time to survey completion was 25 and 35 minutes for the short and long versions, respectively.

Sample characteristics are summarized in Table 2. Caregivers were primarily biological parents and female (i.e., mothers). Most caregivers and adolescents were Non-Hispanic White (88% caregivers, 82% adolescents), with representation of Hispanic, African-American, and Asian Americans. Eighty percent of caregivers were based in the Northeast. Mean age of adolescents was 16.1 (SD =1.8) years. Although all caregivers had elevated concern about SU, only 39% of adolescents had current SU problems, based on caregiver responses to a brief problem inventory embed in the survey (e.g., Global Appraisal of Individual Needs – Short Screener; McDonnell et al., 2009). By contrast, there were high rates of externalizing problems (66%), internalizing problems (51%) and legal problems (25%) in the sample. Caregivers also reported high rates of family history of mental health (46%) and SU (38%) problems. Thus, this was a sample of caregivers of adolescents at risk of SU and other behavioral health disorders, but not a sample experiencing acute SU problems.

Table 2.

Parent and Adolescent Socio-Demographics (n=411)

Characteristic	N	%
Parent Attributes
Gender
Female	354	86%
Male	57	14%
Relationship to Teen
Biological Parent	374	91%
Adoptive/Foster/Step Parent
Race/Ethnicity
Non-Hispanic Caucasian	362	88%
Hispanic	17	4%
African-American or Black	11	3%
Asian-American or Asian	13	3%
Biracial or Other	8	2%
Age
25–34	11	3%
35–44	129	39%
45–54	228	56%
55–64	38	9%
65 or older	5	1%
Marital Status
Married/Domestic Partner	301	73%
Divorced/Separated/Widowed	86	21%
Single/Never Married	24	6%
Income Per Capita
$25,000 or more	195	47%
Less than $25,000	216	53%
Region	327	80%
Northeast	28	7%
Midwest	31	7%
South	25	6%
West
Adolescent Attributes
Gender
Female	209	51%
Male	202	49%
Age (Mean, Standard Deviation)	16.1	1.8
Race/Ethnicity
Non-Hispanic Caucasian	342	83%
Hispanic	23	6%
African-American or Black	13	3%
Asian-American or Asian	8	2%
Biracial or Other	25	6%
Current Clinical Concerns
Any Substance Use Problems	159	39%
Any Externalizing Problems	272	66%
Any Internalizing Problems	210	51%
Any Legal Problems	104	25%
History of Treatment
Any History of Therapy	158	38%
No History of Therapy	253	62%
Family History of Clinical Concerns
Any Mental Health	187	46%
Any Substance Use	155	38%

Note. Adolescent attributes are based on parent report. Due to rounding, percentages may not add to 100%

Assumed Definitions of Evidence-Based Therapy

Responses to the seven survey items and results of the multivariate regressions are depicted in Tables 3 and 4, respectively. For the item addressing assumed definitions, 80% of caregivers selected the accurate definition. However, those from racial/ethnic minority groups [χ²(1) =5.11, p =.02], with lower education [χ²(1) =26.50, p <.001], and with lower income per capita [χ²(1) =4.055, p =.04] had significantly lower rates of selecting the correct definition. When controlling for these variables via multivariate logistic regression, education level was the only variable that remained significant: caregivers with a bachelor’s degree had 3.5 times greater odds of selecting an accurate definition, relative to caregivers without a degree.

Table 3.

Survey Responses by Race/Ethnicity, Education Level, Income per Capita, and Adolescent History of Therapy (n = 411)

	Race/Ethnicity		Education Level		Income Per Capita		Adolescent Therapy History
	Non-Hispanic, White	Minority	Less than college	BA and Above	<$25,000	$25,000+	yes	No	Overall
EBP Definition
Correct definition	81.2%*	67.3%	65.0%***	86.8%	75.1%*	83.2%	82.9%	77.5%	79.6%
Incorrect definition	18.7%*	32.6%	35.0%***	13.2%	24.9%*	16.8%	17.0%	22.5%	20.4%
EBP Principles: Item 1
Proven approach	59.1%*	40.8%	33.6%***	68.4%	44.0%**	67.8%	53.8%	58.9%	56.9%
Varied approach	40.9%*	59.2%	66.4%***	31.6%	56.0%**	32.2%	46.2%	41.1%	43.1%
EBP Principles: Item 2
Process of therapy	38.1%	34.7%	46.7%*	33.1%	44.6%*	31.3%	46.2%*	32.4%	37.7%
Outcome of therapy	61.9%	65.3%	53.3%*	66.9%	55.4%*	68.7%	53.8%*	67.6%	62.3%
EBP Principles: Item 3
Teen likes the therapist	45.0%	46.9%	47.4%	44.1%	49.2%	41.6%	50.0%	42.3%	45.3%
Therapist uses proven approach	55.0%	53.1%	52.6%	55.9%	50.8%	58.4%	50.0%	57.7%	54.7%
Appeal of EBP
Appealing	51.9%*	36.7%	48.2%	51.1%	46.6%	53.3%	53.8%	47.8%	49.9%
Not Appealing or Indifferent	48.1%*	63.3%	51.8%	48.9%	53.4%	46.7%	46.2%	52.2%	50.1%
Preferred EBP Terms: Item 1
Effective Therapy	47.0%	57.1%	42.3%	51.1%	47.7%	48.1%	52.5%	45.5%	48.2%
Proven Therapy	14.4%*	2.0%	13.1%	12.9%	11.9%	14.0%	10.8%	14.2%	12.9%
Successful Therapy	17.7%	20.4%	25.5%**	14.3%	18.7%	17.8%	17.1%	18.6%	18.0%
Therapy That Works	21.0%	20.4%	19.0%	21.7%	21.8%	20.1%	19.6%	21.7%	20.9%
Preferred EBP Terms: Item 2
Flexible	8.6%	14.3%	11.7%	8.1%	14.0%**	5.1%	8.2%	9.9%	9.2%
Individualized	68.2%	57.1%	66.4%	66.9%	63.7%	69.2%	63.9%	68.8%	66.9%
Varied	1.9%	6.1%	4.4%	1.5%	3.1%	1.9%	3.2%	2.0%	2.4%
Tailored	20.2%	20.4%	16.1%	22.4%	17.6%	22.9%	22.8%	18.6%	20.2%
Not Rigid	1.1%	2.0%	1.5%	1.1%	1.6%	0.9%	1.9%	0.8%	1.2%

Note.

^*p < .05

^**p < .01

^***p < .001

Table 4.

Multivariate Logistic Regressions Predicting Specific Survey Responses (n = 411)

	Selected Correct EBP Definition				Valued Proven Approach				Valued Therapy Outcome
Variable	B	SE	OR	95% CI	B	SE	OR	95% CI	B	SE	OR	95% CI
Constant	0.23	0.33	1.26		−1.24	0.35	0.29		0.28	0.20	1.3
White Non-Hispanic	0.52	0.35	1.68	.84, 3.34	0.54	0.35	1.72	.88, 3.39	-	-	-	-
Income per capita	−0.12	0.29	0.85	.50, 1.58	0.51*	0.23	1.66	1.05, 2.61	0.33	0.23	1.4	0.88, 2.20
Education level	1.25*	0.30	3.50	1.97, 6.24	1.17*	0.25	3.21	1.98, 5.20	0.44	0.24	1.6	0.97, 2.50
History of prior therapy	-	-	-	-	-	-	-	-	−0.57*	0.21	0.6	0.37, 0.86

Note: B = unstandardized regression coefficient; SE = standard error of regression coefficient; OR = odds ratio; CI = confidence interval.

^*p < .05

Principles Underlying Evidence-Based Therapy

Responses to the first item exploring evidence-based principles indicated that most caregivers (57%) preferred a therapist that uses a proven approach over a therapist that uses an idiosyncratic/varied approach (43%). Consistent with hypotheses, caregivers from racial/ethnic minorities [χ²(1) =5.88, p =.02], with lower education [χ²(1) =44.84, p <.001], and with lower income per capita [χ²(1) = 23.22, p < .001] had higher rates of preferring a therapist who uses an idiosyncratic approach. Multivariate logistic regression revealed that education level and income per capita were significant predictors: those with higher education and higher income had 3.2 and 1.7 times the odds, respectively, of preferring a proven approach.

For the second item, most caregivers (62%) valued the outcome versus the process of therapy, though those with lower education [χ²(1) =7.16, p =.01], lower income per capita [χ²(1) =7.80, p =.01] and whose teen had a therapy history [χ²(1) =7.86, p =.01], were more likely to value the process of therapy. In the multivariate logistic regression, only adolescents’ therapy history remained significant: a history of therapy was associated with .57 times lower odds of valuing the outcome of therapy.

Finally, the third item revealed that caregivers were fairly evenly split between valuing the teen liking the therapist (45%) versus the therapist using a proven approach (55%), and this did not vary by the putative mediators.

Appeal of Evidence-Based Therapy

There was virtually an even split between those that found evidence-based therapy “appealing” (51%) versus those who found it “not appealing or neutral” (49%). The only variable associated with appeal was race/ethnicity, with Non-Hispanic Whites demonstrating significantly higher rates of finding evidence-based therapy appealing than racial/ethnic minorities, χ²(1) = 3.98, p = .046.

Alternate Terms to Describe Evidence-Based Therapy

When asked which term they preferred to describe evidence-based therapy, caregivers selected “effective therapy” most often (48%). However, Non-Hispanic White caregivers selected “proven therapy” significantly more often than racial/ethnic minorities [χ²(1) =5.88, p =0.02] and lower education caregivers selected “successful therapy” more often than those with a bachelor’s [χ²(1) =7.71, p =0.006]. For descriptions of patient-centric care, most caregivers (66.9%) preferred “individualized,” though lower income caregivers were more likely to select “flexible” than those with higher income per capita, χ²(1) =9.49, p =0.002.

Discussion

This survey examined how caregivers perceived various aspects of EBP including its definition, underlying principles, appeal, and alternate terms. Although the survey was designed for caregivers concerned about their adolescents’ SU, the final sample did not have high rates of SU problems. In fact, caregivers reported that their adolescents had higher rates of internalizing and externalizing problems than of substance use problems. Furthermore, the focal survey questions asked about impressions of evidence-based therapy in general and did not focus specifically on therapy for substance use. For these reasons, the present results are likely to be generalizable to caregivers of youth with a myriad of other behavioral health concerns.

In contrast to prior studies suggesting that health care consumers have negative impressions of “evidence-based” care (Becker et al., 2016a; Carman et al., 2010; Tannenbaum, 2008), the current survey found that most caregivers defined EBP correctly (80%), valued EBP principles (55–62%), and found the EBP concept appealing (51%). However, consistent with hypotheses, we found that caregiver impressions were significantly moderated by socio-demographic factors. Caregivers from racial/ethnic minority groups, lower education, and lower income per capita were generally less likely to define EBP correctly and value EBP principles. These results are concerning in light of the fact that adolescents and adults from these disadvantaged groups are the least likely to access behavioral health treatment (Costello, He, Sampson, Kessler, & Merikangas, 2014; Wells, Klap, Koike, & Sherbourne, 2001). Our results essentially suggest that the very individuals we would most want to target via DTC marketing are those most likely to perceive EBP negatively.

Of the socio-demographic variables, education was the most consistent and strongest predictor of EBP impressions. Caregivers who did not have a bachelor’s degree had 3.5 times greater odds of defining EBP incorrectly and had 3.2 times greater odds of valuing an idiosyncratic approach, even when controlling for income and race/ethnicity. In the general United States population, over 67% of adults do not have a bachelor’s degree (U.S. Census Bureau, 2016). When coupled with the current results, these data underscore the need to ensure that DTC marketing materials are written at an accessible comprehension level. Ensuring accessibility is especially important given the well-documented association between education and health literacy. Data from the Department of Health and Human Services (2008) indicate that 30% of those with a bachelor’s degree have proficient levels of health literacy, while only 3% of those without a degree have proficient levels. Yet many websites, including the current APA Division 53 webpage (http://www.effectivechildtherapy.org/content/what-evidence-based-practice), use language requiring at least a 12^th grade-level according to Flesch-Kinkaid readability statistics generated by Microsoft Word. This discrepancy suggests that current DTC marketing efforts would likely benefit from use of simpler language when describing EBP. As a specific recommendation, organizations and individual clinicians seeking to disseminate information to caregivers from historically disadvantaged groups should consider replacing the term “evidence-based” with simpler terms. The term “evidence-based therapy” requires over a 12^th grade reading level and it is possible that reading difficulties might have contributed to negative impressions of the term among less educated parents.

Income per capita was also associated with caregiver impressions. In isolation, income per capita was associated with assumed EBP definitions, preferences for varied versus proven therapy, valuing the outcome versus process of therapy, and preferring the term “flexible” to describe therapy. However, most of these associations were no longer significant when controlling for education, and one association that remained significant (i.e., lower income families valuing a varied approach) was not as strong as the association between education and EBP impressions. These results suggest that the relationship between income and EBP perceptions is largely accounted for by education level. We would therefore expect efforts to increase the readability of marketing materials to simultaneously improve efforts to engage lower income families.

Associations between race/ethnicity and EBP impressions are also worth noting. Similar to income, race/ethnicity was associated with several EBP perceptions on a bivariate level, but when controlling for other socio-demographic variables, only two unique associations remained significant. First, race/ethnicity was the only variable associated with the perceived appeal of EBP, with racial/ethnic minorities being significantly less likely to view the concept as appealing. Second, minorities were less likely to prefer the term “proven” to describe EBP. Such findings suggest that DTC marketing efforts to engage caregivers from racial/ethnic minority groups should potentially avoid the use of the terms “evidence-based” and “proven therapy” altogether. Instead, it might be preferable to use terms such as “effective” and “individualized” to describe EBP, as all caregivers seemed to respond positively to these terms.

Finally, the adolescent’s history of therapy was not as important a predictor of caregiver impressions as other socio-demographic variables. There was only one significant association: those caregivers whose adolescents had previously been in therapy were significantly more likely to value the process of therapy than those without a history of therapy. This relationship remained significant when controlling for caregiver education and income level. When interpreting this finding, it is important to keep in mind that the process and outcome of therapy are not independent constructs. Process factors (such as the quality of the therapeutic relationship) are well-established, reliable predictors of treatment outcome that account for a moderate amount, but not all, of the variance (Ardito & Rabellino, 2011; Norcross & Wampold, 2011). By contrast, treatment outcome is rarely theorized to predict the quality of the therapeutic relationship. One possible reason for our finding is that caregivers with a history of therapy might have had greater awareness of the importance of the therapeutic process in influencing the ultimate outcome of treatment.

Limitations

Results of this DTC marketing survey must be interpreted within the context of several limitations. First, as with all internet surveys, the sample was a convenience sample that might not representative the full population of caregivers concerned about their adolescent’s SU. We attempted to mitigate against this limitation by following the best practices in the Checklist for Reporting Results of Internet E-Surveys (CHERRIES) guide for online surveys (Eysenbach, 2004), but response bias cannot be ruled out. Second, the survey items were developed specifically for this study based on prior qualitative research (Becker et al., 2016a) and have not been psychometrically validated. Future work should seek to validate tools to assess caregivers’ impressions of EBP, in order to promote evaluation of this outcome across multiple investigations. Finally, the sample was predominantly Non-Hispanic White, so we were unable to examine heterogeneity across and within specific racial/ethnic groups. Future studies should consider examining variables that capture this between- and within-group variability (e.g., acculturation level, perceived discrimination, racial/ethnic identity salience and pride) in order to better understand the significant associations found in this study.

Conclusions and Recommendations

The current findings advance knowledge about how to market behavioral treatments by examining caregiver impressions of EBP. Specifically, this study addressed two key questions: 1) How do caregivers define, value, and prefer to describe the EBP concept?; and 2) Do caregivers perceptions vary by key socio-demographic and clinical characteristics? Overall, our findings suggest that most caregivers understand and value EBP, but that impressions of EBP were more negative among caregivers from racial/ethnic minority groups, those with lower income per capita, and those with less education.

Our findings have concrete implications for national and professional organizations seeking to disseminate knowledge about EBP, as well as for individual clinicians aiming to share knowledge and recruit new patients. Specifically, our results suggest that organizations and practitioners should exercise caution when using terms that are popular in the research literature such as “evidence-based therapy” in their DTC marketing efforts towards caregivers. Likewise, it should not be assumed that caregivers will universally value EBP principles such as the use of proven methods and a focus on outcome assessment. Our data suggest that historically disadvantaged caregivers from racial/ethnic minorities, lower income, and lower education are less likely to value these concepts and would therefore benefit from customized marketing strategies. Just as “no size fits all for” behavioral treatments, our survey suggests that no size fits all for marketing these treatments. In particular, DTC marketing efforts towards disadvantaged caregivers should consider defining key principles underlying EBP; emphasizing the therapeutic process; and replacing the term “evidence-based” with simpler terms such as “effective.” Marketing materials could also emphasize the modifiable nature of EBP, by using terms such as “individualized” and “flexible.” A priority for further research is to evaluate if customized DTC strategies, building on the type of feedback gathered here, can serve to address the treatment gap and promote the utilization of EBP among disadvantaged caregivers.

The current study focused on considerations around the language used to describe EBP. It is important to note that language represents only a small piece of the DTC marketing puzzle. Other critically important questions pertain to how, where, when, and from whom caregivers prefer to receive information. Learning how caregivers prefer to receive information would require having caregivers prioritize among possible channels such as websites, brochures, billboards, television, and radio. Likewise, discovering where and when caregivers want to gain knowledge would require asking caregivers about their ideal settings (e.g., primary care offices, schools, mental health clinics) and timing (e.g., start of summer vacation, start of school year, etc). Finally, learning from whom caregivers prefer to receive information would require asking about trusted sources such as primary care providers, school counselors, family members, friends, and other parents of adolescents. Such information is currently being gathered by our research team and is of paramount importance to inform how large national or state organizations, as well as individual clinicians, could most effectively engage in the dissemination of EBP knowledge.