Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 May 12.
Published in final edited form as: Am Psychol. 2020 Nov;75(8):1038–1051. doi: 10.1037/amp0000635

User-Informed Marketing versus Standard Description to Drive Demand for Evidence-Based Therapy: A Randomized Controlled Trial

Sara J Becker 1, Sarah A Helseth 1, Tonya L Tavares 1, Daniel Squires 1, Melissa A Clark 2, Valarie Zeithaml 3, Anthony Spirito 4
PMCID: PMC8115027  NIHMSID: NIHMS1699200  PMID: 33252943

Abstract

Direct-to-consumer (DTC) marketing represents a vital strategy to disseminate evidence-based therapies (EBTs). This 3-phase research program, informed by the Marketing Mix, developed and evaluated user-informed DTC materials for parents concerned about adolescent substance use (SU). Phases 1 and 2 consisted of qualitative interviews (n=29 parents) and a quantitative survey (n=411), respectively, to elicit parents’ preferred terms and strategies to disseminate EBT. Building upon prior phases, the current study (Phase 3) developed a user-informed infographic (128 words, 7th-grade level) focused on SU therapy. Parents were randomly assigned to view the user-informed infographic (n=75) or a standard EBT description (n=77) from the American Psychological Association (529 words, 12th-grade level). Logistic regressions examined the effect of marketing condition on parent-reported behavioral intentions and actual requests for EBT information, controlling for correlates of parent preferences in Phase 2 (parent education-level; adolescent internalizing, externalizing, legal, and SU problems). Counter to hypotheses, condition did not have a main effect on either outcome. However, there was a significant interaction between condition and adolescent SU problems: among parents whose adolescents had SU problems, the user-informed infographic predicted 3.7-times higher odds of requesting EBT information than the standard description. Additionally, parents whose adolescents had legal problems were more likely to request EBT information than parents whose adolescents did not. The infographic was 4-times shorter and written at 5-grade levels lower, thereby providing a highly disseminable alternative. Findings highlight the value of specificity in DTC marketing, while advancing methods to create tailored marketing materials and communicate knowledge about psychological science.

Keywords: dissemination, direct-to-consumer, marketing, user-informed, adolescent

Substance use (SU) during adolescence is associated with pernicious problems across multiple life areas, such as academic failure, family conflict, legal involvement, sexually transmitted infections, and even death (National Center on Addiction and Substance Abuse at Columbia University, 2011). Fortunately, a recent comprehensive systematic review (Hogue et al., 2018) identified several therapy models (e.g., cognitive behavioral therapy, family therapy, motivational interviewing) that have demonstrated effectiveness in reducing adolescent SU across multiple clinical trials. Researchers, clinicians, legislators, and public health officials alike have recognized the urgency of promoting the use of these research-tested therapy models, commonly defined as evidence-based therapies (EBTs), in usual care settings (Koob, 2014; National Institute on Drug Abuse, 2015).

Over the past two decades, implementation science efforts to increase EBT utilization for adolescent SU disorders have predominantly targeted barriers at the organizational and provider levels, by employing strategies to increase organizational systems’ and providers’ knowledge, skills, and willingness to deliver EBTs (Becker, 2015). Example initiatives have ranged from offering free or low-cost trainings in EBTs (Edmunds et al., 2013) to incentivizing EBT delivery (Conrad & Perry, 2009), to legally mandating their delivery (Nakamura et al., 2011). These approaches explicitly aim to increase the supply of EBTs in community settings, and are most likely to engage those who are actively seeking SU services or who are already connected to the healthcare system. Critically, such approaches fail to account for patient-level barriers to seeking treatment that might affect the demand for adolescent SU services (see Gallo, Comer, Barlow, Clarke, & Antony, 2015). Data from the 2018 National Household Survey on Drug Use and Health (Substance Abuse and Mental Health Services Administration [SAMHSA], 2019) indicated that fewer than 1 in 10 adolescents who met criteria for an SU disorder received any specialty SU care. When adolescents in-need of treatment were asked why they did not seek SU care, most replied that their problems could be handled without treatment or they did not know how to find a provider (SAMHSA, 2019). Such data highlight the need for comprehensive dissemination and implementation strategies that explicitly target patient-level barriers to treatment utilization, such as patient knowledge, awareness, and motivation to seek EBTs.

Driving Demand for EBTs: Direct-to-Consumer Marketing

One promising, yet underutilized, approach to drive demand for EBTs is direct-to-consumer (DTC) marketing (Becker, 2015; Friedberg & Bayar, 2017; Santucci et al., 2012). Within the behavioral healthcare system, there is a long history of DTC marketing to promote pharmacological treatment (Donohue, Cevasco, & Rosenthal, 2007; Schwartz & Woloshin, 2019). A recent systematic review (Becker & Midoun, 2016) found moderate strength of evidence that DTC marketing has been associated with increased requests for medication by patients and increased prescribing by providers, affirming that DTC marketing has affected both patient and provider behavior.

By comparison, research on applying DTC marketing strategies to disseminate psychological EBTs is considerably less established. Clinical researchers have used DTC approaches to broadly promote behavioral therapies (Brecht et al., 2017; Gallo et al., 2015) and encourage parents to pursue EBTs for youth (Okamura et al., 2018), and have found that such efforts increased interest in behavioral EBTs. In recent years, researchers have also examined the effectiveness of DTC marketing campaigns to target patients with specific mental health disorders, such as obsessive compulsive disorder (Szymanski, 2012) or social anxiety disorder (Schofield et al., 2020), and to market specific EBTs, including the Triple P parenting program (Sanders & Kirby, 2012). A key tenet across these studies has been the critical need to employ a user-driven, bottom-up approach to build a comprehensive understanding of the target population’s DTC marketing preferences.

A leading marketing framework for guiding data collection about consumer preferences is the Marketing Mix, which describes “4 Ps” needed to formulate a successful DTC marketing strategy: Product (or service), Price, Place, and Promotion (see Zeithaml et al., 2012, p. 25). Product pertains to the service attributes that consumers value most. Promotion explores how and from whom consumers prefer to receive information. Place explores where consumers prefer to receive a service, while Price pertains to the tangible and intangible costs consumers will tolerate to obtain the service. As our team has previously delineated (see Becker, Helseth, et al., 2018, for an in-depth discussion), the goal of data collection using the Marketing Mix framework is to “put the right service in the right place at the right time and at the right price.”

We recently completed the first programmatic DTC marketing research study in the behavioral health field guided by the Marketing Mix. Our 3-phase study had several distinct objectives: a) explore the DTC preferences of potential consumers of adolescent SU therapy, b) develop user-informed marketing materials focused on SU therapy, and c) test whether user-informed marketing materials outperform the standard descriptions of EBT used by national organizations. All phases focused on parents of adolescents concerned about adolescent SU, based on research indicating that parents serve as vital gatekeepers to youth behavioral healthcare by driving decisions about treatment selection (Becker, Midoun, et al., 2016; Nock & Kazdin, 2005). Phases 1 and 2 have been previously reported, and are briefly summarized here to demonstrate the sequential, accumulating nature of data collection. This manuscript presents the results of Phase 3 in depth, and then discusses implications of the full 3-phase investigation.

Phase 1: Formative Research

Phase 1 consisted of formative research with parents concerned about adolescent SU to understand preferences for information about EBTs (Becker, Midoun, et al., 2016; Becker, Spirito, et al., 2016). In this phase, 29 parents participated in structured interviews or focus group discussions. Parents were recruited from across the adolescent SU treatment continuum, including from community settings, outpatient programs, intensive outpatient programs, and residential programs. Parent interviews and discussions were audio recorded and transcribed to enable thematic analysis.

Using the Marketing Mix framework, questions focused primarily on perceptions of the Product and Promotion dimensions. Parents were asked their impressions of EBT, which terms they would prefer to describe EBT, and how and from whom they would prefer to receive information. In the Product dimension, one unanticipated finding was that parents had extremely low familiarity with the concept of EBT (Becker, Midoun, et al., 2016): of the 29 parents, only two had ever heard of the term and only one defined it correctly. Parents also exhibited incorrect assumptions about what EBT meant and held negative views about the concept. Negative perceptions were especially salient among parents whose adolescents had a history of juvenile justice involvement, many of whom thought the concept “evidence-based” pertained to legal evidence that could be used against them. Parents also desired more basic information about how many adolescents experienced problems related to SU (e.g., basic prevalence information), and any warning signs or strategies they could use to prevent SU (e.g., risk and protective factors, effective treatments).

In the Promotion dimension, parents identified several individuals from whom they would value information about EBT, including their pediatrician or primary care doctor, school counselor or other school official, friend, family member, or another parent whose teen had received therapy. Parents also reported that they would want to access information via multiple channels including websites, brochures, radio or TV ads, billboards, and social media.

Phase 2: Survey Research

Building upon Phase 1, the goal of Phase 2 was to explore the findings in a larger sample of parents concerned about their adolescents’ SU (Becker, Helseth, et al., 2018; Becker, Weeks, et al., 2018). Parents were recruited for an online survey via advertisements shared on parent listservs in local high schools, on private parent Facebook groups, and on listservs of behavioral health providers. The screener contained multiple safeguards to prevent bots or inattentive responding, consistent with online screening best practices (see Teitcher et al., 2015). Screening was completed by 845 parents, 499 of which were eligible, and 411 of which completed the survey. Survey questions solicited impressions of EBT, views of underlying EBT principles, and preferred terms to describe EBT (Becker, Weeks, et al., 2018). Additionally, items were designed to examine parent preferences and actual parent behaviors across the 4 Ps of the Marketing Mix framework (Becker, Helseth, et al., 2018). Response options were based directly on quotes generated in the formative research phase.

In contrast to Phase 1, results of Phase 2 indicated that parents were generally able to define EBT correctly and valued underlying EBT principles (Becker, Weeks, et al., 2018). However, responses varied as a function of parent socio-demographic characteristics, such that parents with lower education, lower income per capita, or from racial/ethnic minority groups were more likely to define EBT incorrectly, dislike EBT principles, and have negative impressions of the concept. Parent education level consistently emerged as the most important correlate in multivariate analyses (Becker, Helseth, et al., 2018). In subsequent work (Crane et al., 2020), we found that parents who reported their adolescents currently had problems with SU, internalizing distress (e.g., symptoms of suicidality, depression, anxiety, or posttraumatic stress), or legal problems (e.g., involvement in the juvenile justice system or illegal activity), had distinct preferences for terms used to describe EBT, regardless of socio-demographics. In multivariate analysis, legal problems emerged as the most significant correlate (Crane et al., 2020).

Regarding Promotion, we found that parents most valued information from pediatricians and via websites, brochures, or social media (Becker, Helseth, et al., 2018). Parents’ specific preferences for marketing channels and their actual use of channels consistently varied by their education level. For instance, parents with higher education were more likely to both prefer and actually obtain information from brochures. Notably, parents’ preferences for marketing channels did not vary as a function of their adolescents’ behavioral health concerns, but parents’ actual use of marketing channels did (Helseth et al., 2020). As an example, parents reporting that their adolescents had externalizing problems (e.g., symptoms of inattention, difficulty following directions, aggression) were more likely to report using a website to find information, whereas those reporting that their adolescents had legal problems were more likely to report using a brochure.

Phase 3: Present Study

The final phase was designed to evaluate whether results from Phases 1 and 2 pertaining to the Product and Promotion dimensions of the Marketing Mix could be used to develop effective user-informed DTC marketing materials for SU therapy. Phase 3 consisted of a two-group randomized trial, during which parents viewed either the user-informed materials or a standard EBT description used by leading national organizations. Our primary aim was to compare the effects of user-informed marketing versus a standard description of EBT on parents’ behavioral intentions and information-seeking behavior. Consistent with the Theory of Planned Behavior (TPB; Ajzen, 1991), and with recent work by Chang et al. (2019) applying the TPB to measure parents’ engagement in EBT, we focused on parents’ behavioral intentions and information-seeking as key proxies for the actual behavior of seeking EBT for their adolescents.

We hypothesized that parents randomized to review the user-informed materials would report higher behavioral intentions and exhibit higher information-seeking than those viewing the standard description. For both of the primary outcomes, we controlled for the parent socio-demographic (i.e., education level) and adolescent behavioral health variables (i.e., legal, internalizing, externalizing, SU problems) associated with parent preferences in Phase 2.

Our secondary aim was to examine whether the effects of marketing condition on the primary outcomes (i.e., behavioral intentions, information-seeking) varied as a function of the covariates in the main effect analyses. Because the infographic was tailored specifically for SU therapy and designed to be accessible, we predicted that it would have the largest effects among those parents whose teens had current SU problems and those with lower education.

Methods

Recruitment Strategies and Procedures

Parents were recruited for the online randomized trial via advertisements emailed to local high schools in Rhode Island, posted to listservs of behavioral health providers, and displayed in behavioral health treatment centers. To be eligible, participants had to meet three criteria: a) parent or legal guardian of an adolescent age 12-19 years; b) currently concerned about the adolescent’s SU (based on score of ≥ 4 on a 5-point Likert scale measuring parental concern); and c) report that their adolescent had a history of therapy for SU or mental health. Reflecting our goal to drive demand for EBT, we relied upon parents’ subjective concern about SU rather than objective SU severity measures. This decision was based on prior research indicating that subjective impressions of behavioral health predict treatment-seeking behavior more than objective symptoms (see Hunt & McKenna, 1993). In Phase 2 (Becker, Helseth, et al., 2018) significant differences emerged between parents whose teens had a history of therapy and those who did not, in terms of their preferences for marketing language and channels. Therefore, we focused on parents whose teens had a history of therapy in this initial trial.

Similar to our Phase 2 procedures (Becker, Helseth, et al., 2018; Becker, Weeks, et al., 2018), participants had to complete an online screener to qualify for the study. In addition to meeting inclusion criteria, participants had to pass multiple electronic safeguards, including IP address confirmation, captcha verification, survey tagging to prevent search engine indexing, extraneous questions to mask eligibility criteria, cookies to prevent duplicate entries, and items with attention checks to prevent automated or inattentive responding. Because no identifying information was collected, the study was deemed exempt by the Brown University Institutional Review Board.

Parents who qualified were asked questions about their adolescents’ therapy history, current behavioral health concerns, and basic socio-demographics. Parents were then randomly assigned to marketing condition using the randomizer block feature in Qualtrics survey software. After viewing marketing content, parents were asked a standard set of questions. Participants received a $20 Amazon gift card upon completion of the survey.

Participants

Figure 1 depicts Phase 3 recruitment, screening, and randomization. A total of 389 individuals completed the screener, of which 171 (44%) were eligible and 218 (56%) were excluded. The most common reason for exclusion was no prior history of treatment (n = 91). In addition, multiple screeners were excluded due to the following safeguards: IP address did not match where participant reported taking the survey (n = 87), “false” response selected to one of the validity check questions (n = 16), IP or e-mail address indicated duplicate attempt (n = 12). Other reasons for exclusion included lack of concern about adolescent SU (n = 8) and age outside of the inclusion range (n = 4). A total of 152 caregivers (89% of eligible) completed the survey; median time to completion was 18 minutes and 19 seconds.

Figure 1.

Figure 1.

Open in a new tab

Phase 3 Participant Flow.

Participant characteristics are summarized in Table 1. Parents were primarily (85%) female (i.e., mothers), whereas their adolescents were evenly split between male (49%) and female (51%). The sample was predominantly non-Hispanic White (88% parents, 80% adolescents). Mean age of parents was 45.0 (SD = 7.7) years, and mean age of their adolescents was 16.1 (SD = 1.9) years. When asked about their adolescents’ behavioral health problems over the past year, 90% of parents reported externalizing problems, 88% reported internalizing problems, 47% reported legal problems, and 34% reported SU problems. The average number of behavioral health problems reported was 2.6 (SD = 1.0), with a range from 0 to 4.

Table 1.

Parent Demographics, Adolescent Demographics and Clinical Characteristics, Process Check Variables, and Outcome Variables by Marketing Condition

Infographic
(n = 75)
M±SD or N (%)		 Standard
(n = 77)
M±SD or N (%)		 p
Parent Variables: Socio-Demographics
Biological Sex .13
 Female 67 (89%) 62 (81%)
 Male 8 (11%) 15 (20%)
Age (range 25-65 yrs.) 45 ± 7.8 45 ± 7.7 .47
Race/Ethnicity .42
 Non-Hispanic White 70 (93%) 68 (88%)
 Racial or ethnic minority 5 (7%) 9 (12%)
Per Capita Income (range $0-$150,000) $25,622 ± 22,571 $25,429 ± 16,600 .95
Education .08
 Associate’s or less 32 (43%) 22 (29%)
 Bachelor’s or higher 43 (57%) 54 (71%)
Adolescent Variables: Socio-Demographics and Clinical Characteristics
Biological Sex .42
 Female 41 (55%) 37 (48%)
 Male 34 (45%) 40 (52%)
Age (range 12-19 yrs.) 16.3 ± 1.9 15.8 ± 2.0 .13
Race/Ethnicity .78
 Non-Hispanic White 59 (79%) 62 (81%)
 Racial or ethnic minority 16 (21%) 15 (19%)
Behavioral Health Problems in Past Year
 Externalizing 66 (88%) 71 (92%) .38
 Internalizing 66 (88%) 68 (88%) .95
 Legal 34 (45%) 37 (48%) .73
 Substance Use 25 (33%) 26 (34%) .95
Process Check Variables
 Correct Definition of EBT 64 (87%) 72 (97%) .15
 Easy to Understand 3.8 ± 1.1 3.9 ± 0.9 .67
 Informative 4.2 ± 0.8 4.3 ± 0.8 .34
Outcome Variables
 Behavioral Intentions (Likely) 48 (64%) 53 (69%) .53
 Actual Information Seeking (Yes) 36 (48%) 34 (44%) .64
Open in a new tab

Note. No significant group differences were found using t-tests or χ2. EBT = evidence-based therapy. Process check variables were scored on 5-point Likert scales, with the exception of the EBT definition, which was scored as correct definition versus incorrect definition.

Marketing Conditions

Figure 2 depicts the two marketing conditions. The user-informed infographic (Figure 2a) was designed to reflect parent preferences from the Product and Promotion dimensions of the Marketing Mix. In alignment with feedback in the Promotion dimension (e.g., parents most preferred information via websites, brochures, and social media), we created an infographic via Adobe Photoshop that could be easily disseminated via multiple channels. Reflecting feedback about the Product dimension (e.g., what information parents most wanted about the product/service), the EBT definition was written using terms and phrases preferred by parents, including “effective,” “some therapies work better than others,” and “every teen is different.” Additionally, in response to parents’ requests for information about adolescent SU, the infographic included data on SU prevalence as well as common risk and protective factors. The infographic had a total of 129 words and required a 7.7 grade level, according to the Flesch-Kincaid Grade Level index embedded within Microsoft Word (Kincaid et al., 1981). The content focused specifically on EBT, was 64 words, and was written at a 5.7 grade level.

Figure 2.

Figure 2.

Open in a new tab

User-Informed Infographic and Standard Description of Evidence-Based Practice

The standard description (Figure 2b) was a screenshot of the definition of EBT presented on the “for parents” portion of the American Psychological Association’s (APA) Division of Child and Adolescent Psychology (Division 53) website. Of note, the image was carefully cropped so that it did not contain any visual indicators that it was taken from a website. At the time of this study, this was the only public definition of evidence-based therapy we could find from a national psychology association that was written specifically for parents (though excellent resources have since been disseminated at the state-level, see: Okamura et al., 2018). The standard description contained 524 words and required a 12.3 grade level based on Flesch-Kincaid readability statistics (Kincaid et al., 1981).

Measures

All measures were parent-reported questions completed as part of the online survey.

Primary Outcomes.

There were two primary outcomes, a measure of self-reported behavioral intentions and an indicator of information-seeking. The Behavioral Intentions Scale, a 5-item scale developed by Zeithaml and colleagues (Zeithaml et al., 1996) and used extensively in the field of services marketing (see Baker & Crompton, 2000; Kuruuzum & Koksal, 2010), measured the extent to which individuals were likely to: say positive things about EBT to others; recommend EBT to individuals in need of therapy; encourage friends and family to seek out EBT; consider EBT for their future therapy needs; and actively seek out EBT. Response options ranged from 1 = not at all likely to 5 = very likely. Item ratings were averaged to create a behavioral intentions score, which was rounded to the nearest whole number and then split to differentiate between those likely (score of 4-5) versus those unlikely or neutral (score of 1-3) to seek out EBT. Internal consistency was excellent (α = .97) in the current sample.

The final survey item was the indicator of information-seeking. Parents were asked, “Would you like us to email you information about evidence-based therapy after this survey?,” to which they could reply yes or no. Parents who responded affirmatively were emailed an information pamphlet upon survey completion.

Covariates and Putative Moderators.

Covariates and potential moderators of outcome were those parent-reported variables that were consistently associated with parent preferences, impressions, or actual information seeking behavior in Phase 2 (Becker, Helseth, et al., 2018; Becker, Weeks, et al., 2018). The only socio-demographic variable was parent education level (dichotomized no degree vs. bachelor’s or higher), because this variable was consistently more important in multivariate analyses than adolescent race/ethnicity or income per capita as a correlate of parent preferences.

Adolescent behavioral health variables included internalizing, externalizing, legal, and SU problems. The Global Appraisal of Individual Needs (GAIN)-Short Screener (GAIN-SS; Dennis, Chan, & Funk, 2006) measured the presence of externalizing (5-items; e.g., inattention, aggression, difficulty following directions); internalizing (5-items; e.g., suicidality, depression, anxiety, trauma); and legal (5-items; e.g., property crime, interpersonal violence, drug-related crime) problems in the past year. GAIN-SS items have shown exceptional sensitivity (90%) and specificity (92%) for correctly identifying individuals with a psychiatric disorder (Dennis et al., 2006). Internal consistency of the subscales was adequate in the current sample (αs = .71-.73).

Parent-report of adolescents SU symptoms was measured using the 11-item SU disorder symptom scale from the full length GAIN (Dennis et al., 2008). Each item queried the presence of a specific symptom related to SU disorders over the past year. The scale demonstrated excellent internal consistency (α = .99) in the current sample.

Quality of Marketing Materials.

In addition to the focal outcomes and covariates, we included several items as a process check to evaluate whether the quality of marketing materials was similar across conditions. After viewing marketing content, parents answered a multiple choice question asking them to select the correct definition of EBT, using identical response options to those used in Phase 2 (see Becker, Helseth, et al., 2018; Becker, Weeks, et al., 2018). The correct definition was “therapy based on evidence from research trials evaluating different treatments.” Incorrect definitions pertained to evidence from the teen’s medical history, the therapist’s prior experience, or legal evidence. Parents were also asked to rate the marketing content on how clear, easy to understand, and informative it was on 5-point Likert scales rating from 1 = not at all to 5 = very.

Analytic Plan

Preliminary analyses consisted of testing bivariate associations among the main predictor variables: parent education level, adolescent SU, externalizing, internalizing, and legal problems. Because our secondary aim was to inform marketing to subgroups of parents, all variables were dichotomized. Consistent with our Phase 2 analyses (see Becker, Helseth, et al., 2018; Becker, Weeks, et al., 2018), behavioral health variables were dichotomized to reflect the presence or absence of past-year problems in each domain, with the latter serving as the reference group.

Next, we compared parents in the two marketing conditions to assess whether randomization was effective in controlling between-group differences. Using t-tests and chi-square, we assessed differences between the two conditions in parent and adolescent socio-demographic variables as well as in adolescent behavioral health concerns. We also examined whether parents’ ability to define EBT and ratings of advertising quality differed by condition: variables that significantly differed were included as covariates in the analysis.

The primary aim was tested via separate logistic regression models examining self-reported behavioral intentions (via the Behavioral Intentions Scale) and actual information-seeking (via the indicator of requesting EBT information). Both models assessed the effect of marketing condition on outcome, controlling for parental education and adolescent behavioral health (internalizing, externalizing, legal, SU) problems. For the secondary aim, we layered in interaction terms between condition and each of the predictor variables.

The study was powered to detect main effects and, as such, main effect analyses used a standard p < .05 criterion. Reflecting the information-generating nature of the secondary aim, exploratory analyses relied upon both significance testing and effect sizes to denote association strength. We flagged interaction effects of p < .10 with moderate to large effect sizes (odds ratios ≥ 3.47; Chen, Cohen, & Chen, 2010) as important signals worthy of future study. Significant interactions were interpreted following guidance from the UCLA Statistical Consulting Group (“Deciphering Interactions in Logistic Regression”) to calculate the relative odds associated with membership in specific groups, by combining log odds ratios for the condition term and relevant behavioral health variables. All analyses report odds ratios and 95% confidence intervals.

Because rates of internalizing and externalizing problems were so high (88-90%), there was limited variability to detect their effects. Moreover, some of the parameter estimates became unstable when including these variables as predictors and putative mediators. Analyses were replicated removing the main and interactive effects of internalizing and externalizing problems. Results were identical with regards to the direction of effects and statistical significance, and neither variable had any significant effects on outcomes. We therefore report the results of the more parsimonious models controlling only for education, legal problems, and SU problems.

Results

Preliminary Analyses

Bivariate associations among the predictor variables using phi coefficients revealed mixed patterns of significance. Parent education was not associated with any of the adolescent behavioral health variables. Associations among the four specific types of adolescent behavioral health problems were either insignificant (e.g., internalizing problems were not associated with SU or legal problems) or significant with small effect sizes (ϕ’s = .22 - .28, p’s < .01).

Table 1 presents participant demographics, clinical characteristics, process check variables, and study outcome variables by marketing condition. No significant differences were found between conditions on any of the variables. Thus, only the significant correlates from Phase 2 (i.e., education, behavioral health variables) were included as covariates. Process check variables indicated that both conditions were perceived as clear, informative, and easy to understand. Outcome variables indicated that both conditions were associated with modest self-reported behavioral intentions (64-69%) and actual information-seeking behavior (44-48%).

Primary Aim: Main Effect of Condition on Behavioral Intentions and Information Seeking

Results of the primary outcome models examining self-reported behavioral intentions and actual information-seeking are presented in the top row of Table 2. In the model predicting self-reported behavioral intentions, condition did not have a significant effect and there were not main effects of education, legal, or SU problems.

Table 2.

Logistic Regressions Predicting Self-Reported Behavioral Intentions and Actual Information Seeking about Evidence-Based Practice (n = 152)

Self-Reported Behavioral Intentions Actual Information Seeking
Predictor B(SE) Odds Ratio [95% CI] B(SE) Odds Ratio [95% CI]
Models 1 and 2: Main Effects Only
Constant 0.69 (0.41) 1.99 −0.89 (0.42) 0.42
Condition −0.18 (0.35) 0.84 [0.42, 1.60] 0.18 (0.35) 1.20 [0.60, 2.39]
Parent Education 0.23 (0.36) 1.25 [0.61, 2.56] −0.17 (0.37) 0.84 [0.41, 1.74]
Adolescent Substance Use (SU) Problems 0.25 (0.39) 1.28 [0.60, 2.75] 0.30 (0.39) 1.34 [0.63, 2.86]
Adolescent Legal Problems −0.31 (0.36) 0.73 [0.36, 1.49] 1.39 (0.36)*** 4.02 [1.98, 8.17]
Models 3 and 4: Main Effects and Moderators
Constant 0.72 (0.56) 2.04 −0.65 (0.55) 0.52
Condition −0.26 (0.71) 0.77 [0.19 – 3.12] −0.34 (0.73) 0.71 [0.17 – 3.00]
Parent Education 0.59 (0.55) 1.80 [0.61 – 5.33] −0.48 (0.56) 0.62 [0.21 – 1.85]
Adolescent Substance Use (SU) Problems −1.07 (0.60) 0.34 [0.11 – 1.10] −0.72 (0.66) 0.49 [0.14 – 1.76]
Adolescent Legal Problems 0.65 (0.64) 1.91 [0.55 – 6.67] † 2.00 (0.62)*** 7.37 [2.19 – 24.8]
Condition*Parent Education −0.72 (0.74) 0.49 [0.11 – 2.07] 0.53 (0.75) 1.69 [0.39 – 7.42]
Condition*Adolescent SU Problems −0.42 (0.83) 0.66 [0.13 – 3.32] 1.66 (0.84)* 5.28 [1.01 – 27.9]
Condition*Adolescent Legal Problems 1.35 (0.77) 3.87 [0.85 – 17.6] † −0.77 (0.80) 0.47 [0.10 – 2.22]
Open in a new tab
†

p < .10

*

p < .05

**

p < .001

In the model predicting actual information-seeking, condition again did not a have significant effect and there were no significant effects of education or SU problems. There was one significant main effect of legal problems, such that parents who reported their adolescent had a history of legal problems had significantly higher odds of requesting EBT information at the end of the survey. Specifically, parents who reported their adolescent had past year legal problems had higher odds of requesting EBT information than parents who did not report legal problems, with a moderate effect size (p < .001, odds ratio = 4.02, 95% Cl [1.98. - 8.17]).

Secondary Aim: Interactive Effects of Condition and Putative Moderators

Results of the exploratory models examining moderators of self-reported behavioral intentions and actual information-seeking are presented in the bottom row of Table 2. In the full model predicting self-reported behavioral intentions, once again we found no significant main effects of condition, education, legal, or SU problems. Using the p < .10 criterion, there was one interaction between condition and legal problems with a moderate effect size (p = .08, odds ratio = 3.87, 95% CI [0.85 – 17.6]). The interaction was interpreted by combining log-odds ratios to derive specific group combinations. Among those parents whose adolescents had legal problems, those who viewed the user-informed infographic had 2.97-times (small effect size) greater odds of reporting behavioral intentions to seek EBT (logit [IP(Legal=1) = −0.26 + 1.35]) than those who viewed the standard description. Even more notably, among those in the user-informed infographic condition, those whose adolescents had legal problems had 7.39-times (very large effect size) greater odds of reporting behavioral intentions (logit [IP(Condition=l) = 0.65 + 1.35]) than parents of adolescents without legal problems.

In the full model predicting actual information-seeking behavior, condition again did not have a significant effect and there were not any significant main effects of education or SU problems. Similar to the initial model, there was a significant, very large main effect such that parents whose adolescents had legal problems in the past year had higher odds of requesting information about EBT (p < .001, odds ratio = 7.37, 95% CI [2.19 – 24.8]). There was also a significant, small interaction effect between condition and SU problems (p < .05, odds ratio = 1.66, 95% CI [1.01 – 27.9]). The interaction was interpreted by combining log-odds ratios to derive various group combinations. Within the group with SU problems, those viewing the user-informed infographic had 3.74 times (moderate effect size) greater odds of requesting EBT information (logit [IP(SU problems=l) = −0.34 + 1.66]) than those viewing the standard description. Likewise, among those viewing the infographic, parents whose adolescents had SU problems had 2.56 times (small effect size) greater odds of requesting EBT information (logit [IP(Condition=1) = −0.72 + 1.66]) than parents whose adolescents did not have SU problems.

Discussion

The current trial was the third phase of programmatic research testing whether user-informed marketing materials, reflecting parents’ preferences from the Product and Promotion dimensions of the Marketing Mix, could outperform standard descriptions of EBT. Although the trial explicitly recruited parents concerned about their adolescents’ SU, less than half the sample reported that their adolescents had SU problems. By contrast, parents reported extremely high rates of internalizing and externalizing problems (88-90%). Similarly high rates of mental health problems were found in Phase 2 (Becker, Helseth, et al., 2018; Becker, Midoun, et al., 2016). Thus, the present results should be viewed as encompassing parents concerned about adolescent SU and a myriad of co-occurring mental health concerns, and not solely parents of adolescents with acute SU problems.

Counter to our primary hypotheses and at odds with some prior studies (e.g., Brecht et al., 2017; Gallo et al., 2015), we found no significant main effects of marketing condition on either behavioral intentions or actual information-seeking. Our failure to detect main effects might reflect an unanticipated mismatch between the infographic content, which explicitly focused on SU therapy, and the sample, less than 40% of whom reported their adolescent had SU problems. It is also important to note that both marketing conditions performed reasonably well. Prior studies, including our own Phase 2 survey, have found that only 50-80% of survey respondents can accurately identify the definition of EBT in the absence of additional information (Becker, Weeks, et al., 2018; Schofield et al., 2020). In the current trial, 89% of parents correctly defined EBT after viewing marketing content. Parents also rated both conditions as clear, easy to understand, and informative, and both conditions were associated with modest rates of self-reported behavioral intentions (64-69%) and actual information-seeking behavior (44-48%). These data offer positive news that both marketing conditions had beneficial effects.

Results directly supported the secondary hypothesis that the effect of marketing condition would vary as a function of variables that predicted parent preferences in Phase 2. A significant interaction was found between condition and SU problems: among parents whose adolescents had SU problems, the user-informed infographic was associated with more than 3-times greater odds of requesting EBT information than the standard description. This finding was highly encouraging, given that the infographic was designed to drive demand for SU therapy. Among parents the marketing content was explicitly targeting, i.e. those whose adolescents had SU problems, the infographic was significantly more effective in promoting requests for EBT information. Considering that the infographic was 4-times shorter and written 5-grade levels lower than the standard description (and created in Adobe Photoshop by a research assistant with no artistic or graphic design training), it could represent a highly disseminable alternative to build awareness of EBT among parents concerned about adolescent SU problems.

One noteworthy finding was that, regardless of condition, parents of adolescents with legal problems were significantly more likely to request information about EBT than parents of adolescents without legal problems. There was also an interaction between condition and legal problems on behavioral intentions worthy of further exploration. When probing the interaction, a very large effect was found among parents in the user-informed infographic condition: within this group, parents whose adolescents had legal problems were over 7-times more likely to report behavioral intentions to seek EBT than parents whose adolescents did not have legal problems. Whereas the Phase 1 findings (Becker, Spirito, et al., 2016) suggested that parents of adolescents with legal problems often defined EBT inaccurately and had negative impressions of the concept, the current findings indicated that such parents were significantly more likely to report behavioral intentions to seek out EBT and exhibit information-seeking behavior than parents of adolescents without such problems. Of note, Phase 1 solicited parents’ general impressions of EBT, while the current phase measured parents’ behavioral intentions and actual behavior after viewing an accurate EBT definition. Combined, results of this multi-phase study suggest that parents of justice-involved youth might be especially likely to benefit from accurate EBT information (regardless of the marketing style). Such data are valuable given that the juvenile justice system represents the primary channel through which adolescents are referred to SU treatment (SAMHSA, 2017). Pragmatic outreach strategies to provide parents of justice-involved adolescents with accurate information about EBTs could include training court intake staff or probation officers, to ensure that they are informed brokers of EBT.

Limitations

Findings must be considered within the context of limitations, which highlight potential directions for new research. First, our study accessed an online sample of convenience, such that results may only generalize to families with internet access, or to parents already engaged with one of our referral sources (e.g., schools, parent Facebook groups, behavioral health providers). The convenience sample was predominantly non-Hispanic White. Further efforts should assess marketing preferences among more ethnically and racially diverse parents as well as parents who might not have internet access. Second, our user-informed infographic was developed in response to parent feedback and tailored specifically for SU therapy. Consequently, it contained basic information about adolescent SU and risk factors that was not in the standard description. It is not possible to determine which aspect of the tailoring (e.g., the additional content, grade-level, graphics, etc.) drove the observed effects. Third, we randomly assigned participants to marketing conditions to strengthen control of our research design. While not a limitation per se, participants did not interact with both marketing conditions preventing any contrasting of their relative preferences. Given that parents rated both conditions as clear, easy to understand, and informative, future work would benefit from directly comparing parent preferences across conditions. Fourth, despite being screened for concerns about adolescent SU, parents reported that their adolescents had very high rates of internalizing (88%) and externalizing (90%) problems. This limited our ability to evaluate the effects of marketing condition on parents whose adolescents did not experience internalizing or externalizing problems. Finally, given the sparse literature in this area, our behavioral intentions measure was from the field of services marketing and our information-seeking measure was developed for this study. Development of valid measures to assess parent preferences and responses to DTC behavioral health materials remains a critical task for future investigations; standardized measurement of DTC constructs would support the harmonization of results across research studies and could enable healthcare organizations and clinicians to more carefully measure local efforts to engage families in EBT.

Conclusions

This study represented an important step towards advancing the dissemination of EBT, by engaging directly with parents concerned about adolescent SU to drive demand for therapy. A user-informed infographic, developed using both formative and survey research, was associated with greater odds of requesting EBT information than a standard description of therapy, specifically among the target population of parents whose adolescents had SU problems. Results highlight new directions for research and marketing strategies. First, it is important to note that the two different marketing approaches – one of which promoted EBT generally and one of which promoted EBT specifically for adolescent substance use – were both viewed favorably by parents (e.g., rated as easy to understand, informative, and clear), and associated with high rates of defining EBT correctly and modest rates of seeking out EBT information. These data suggest that existing approaches to marketing EBT more generally are likely to have value in increasing knowledge and awareness among potential consumers. Second, this trial found that tailoring DTC marketing materials towards parents of adolescents with specific problems increased information-seeking behavior among those parents directly targeted. Consistent with conventional wisdom in marketing and the widespread practice of customer segmentation (see Martin, 2011), specificity mattered. It is well established that investment in marketing of a specific medication increases requests for medications in that pharmaceutical class, but not in other classes of medication (Donohue et al., 2007). Just as we would not expect DTC marketing for antidepressants to affect requests for stimulants, we should not expect DTC marketing for SU therapy to affect information-seeking by consumers concerned about other behavioral health conditions. The current results suggest that general marketing about EBT likely has value in increasing knowledge and awareness, whereas tailored marketing about EBT for specific conditions is likely needed to affect treatment-seeking behavior. Overall, this multi-phase research adds to a growing body of literature (Friedberg & Bayar, 2017; Gallo et al., 2015) indicating that DTC marketing, particularly when designed carefully for a target consumer, represents a valuable component of a comprehensive dissemination and implementation strategy.

Public Health Significance Statement:

In this 3-phase study, we developed and tested direct-to-consumer (DTC) marketing materials about therapy for adolescent substance use. Parents who viewed our user-informed infographic were more likely than parents who viewed a standard description to request additional information, but only if their teens currently had substance use problems. Findings suggest that marketing is effective in influencing consumer behavior when carefully tailored to a target population.

Acknowledgments

Funding for this study was provided by NIDA K23 DA31743 awarded to Dr. Sara Becker. An NIAAA T32 Fellowship (T32 AA007459; PI: Monti) covered the time of Dr. Sarah Helseth.

Footnotes

Full disclosure of interests: The authors report no relevant competing interest

References

  1. Ajzen I (1991). The theory of planned behavior. Organizational Behavior and Human Decision Processes, 50(2), 179–211. 10.1016/0749-5978(91)90020-T [DOI] [Google Scholar]
  2. Baker DA, & Crompton JL (2000). Quality, satisfaction and behavioral intentions. Annals of Tourism Research, 27(3), 785–804. 10.1016/S0160-7383(99)00108-5 [DOI] [Google Scholar]
  3. Becker SJ (2015). Direct-to-consumer marketing: A complementary approach to traditional dissemination and implementation efforts for mental health and substance abuse interventions. Clinical Psychology: Science and Practice, 22(1), 85–100. 10.1111/cpsp.12086 [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Becker SJ, Helseth SA, Frank HE, Escobar KI, & Weeks BJ (2018). Parent preferences and experiences with psychological treatment: Results from a direct-to-consumer survey using the marketing mix framework. Professional Psychology: Research and Practice, 49(2), 167–176. 10.1037/pro0000186 [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Becker SJ, & Midoun MM (2016). Effects of Direct-To-Consumer Advertising on Patient Prescription Requests and Physician Prescribing. The Journal of Clinical Psychiatry, 77(February), e1293–e1300. 10.4088/JCP.15r10325 [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Becker SJ, Midoun MM, Zeithaml VA, Clark MA, & Spirito A (2016). Dimensions of treatment quality most valued by adolescent substance users and their caregivers. Professional Psychology: Research and Practice, 47(2), 120–129. 10.1037/pro0000066 [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Becker SJ, Spirito A, & Vanmali R (2016). Perceptions of “Evidence-Based Practice” among the Consumers of Adolescent Substance Use Treatment. Health Education Journal, 75(3), 358–369. 10.1177/0017896915581061 [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Becker SJ, Weeks BJ, Escobar KI, Moreno O, DeMarco CR, & Gresko SA (2018). Impressions of “evidence-based practice”: A direct-to-consumer survey of caregivers concerned about adolescent substance use. Evidence-Based Practice in Child and Adolescent Mental Health, 3(2), 70–80. 10.1080/23794925.2018.1429228 [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Brecht K, Swift JK, Worrall JM, & Parkin SR (2017). A randomized controlled test of direct-to-consumer marketing using the american psychological association psychotherapy works videos. Professional Psychology: Research and Practice, 48(6), 421–428. 10.1037/pro0000167 [DOI] [Google Scholar]
  10. Chang JP, Orimoto TE, Burgess A, Choy SKJ, & Nakamura BJ (2019). The theory of planned behavior applied to consumer engagement in evidence-based services. Journal of Child and Family Studies, 28, 2963–2976. 10.1007/s10826-019-01472-y [DOI] [Google Scholar]
  11. Chen H, Cohen P, & Chen S (2010). How big is a big odds ratio? Interpreting the magnitudes of odds ratios in epidemiological studies. Communications in Statistics - Simulation and Computation, 39(4), 860–864. 10.1080/03610911003650383 [DOI] [Google Scholar]
  12. Conrad DA, & Perry L (2009). Quality-based financial incentives in health care: Can we improve quality by paying for it? Annual Review of Public Health, 30(1), 357–371. 10.1146/annurev.publhealth.031308.100243 [DOI] [PubMed] [Google Scholar]
  13. Crane ME, Helseth SA, Scott K, & Becker SJ (2020). Adolescent behavioral health problems are associated with parent perceptions of evidence-based therapy and preferences when seeking therapeutic support. Professional Psychology: Research and Practice. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Dennis ML, Chan Y-F, & Funk RR (2006). Development and validation of the GAIN Short Screener (GSS) for internalizing, externalizing and substance use disorders and crime/violence problems among adolescents and adults. The American Journal on Addictions, 15 Suppl 1(SUPPL. 1), 80–91. 10.1080/10550490601006055 [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Dennis ML, White M, Titus JC, & Unsicker J (2008). Global Appraisal of Individual Needs: Administration Guide for the GAIN and Related Measures (Version 5). Chestnut Health Systems. [Google Scholar]
  16. Donohue JM, Cevasco M, & Rosenthal MB (2007). A decade of direct-to-consumer advertising of prescription drugs. New England Journal of Medicine, 357(7), 673–681. 10.1056/NEJMsa070502 [DOI] [PubMed] [Google Scholar]
  17. Edmunds JM, Beidas RS, & Kendall PC (2013). Dissemination and implementation of evidence-based practices: Training and consultation as implementation strategies. Clinical Psychology: Science and Practice, 20(2), 152–165. 10.1111/cpsp.12031 [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Friedberg RD, & Bayar H (2017). If it works for pills, can it work for skills? Direct-to-consumer social marketing of evidence-based psychological treatments. Psychiatric Services, 68(6), 621–623. 10.1176/appi.ps.201600153 [DOI] [PubMed] [Google Scholar]
  19. Gallo KP, Comer JS, Barlow DH, Clarke RN, & Antony MM (2015). Direct-to-consumer marketing of psychological treatments: A randomized controlled trial. Journal of Consulting and Clinical Psychology, 83(5), 994–998. 10.1037/a0039470 [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Helseth SA, Escobar KI, Clark MA, Spirito A, & Becker SJ (2020). Marketing therapy to parents concerned about adolescent substance use: Association of adolescent problems and parent preferences for direct-to-consumer marketing. Professional Psychology: Research and Practice, 51(1), 68–76. 10.1037/pro0000255 [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Hogue A, Henderson CE, Becker SJ, & Knight DK (2018). Evidence base on outpatient behavioral treatments for adolescent substance use, 2014-2017: Outcomes, treatment delivery, and promising horizons. Journal of Clinical Child and Adolescent Psychology, 47(4), 499–526. 10.1080/15374416.2018.1466307 [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Hunt SM, & McKenna SP (1993). Measuring quality of life in psychiatry. In Walker SR & Rosser RM (Eds.), Quality of life assessment: Key issues in the 1990s (pp. 343–354). Springer. [Google Scholar]
  23. Kincaid JP, Aagard JA, O’Hara JW, & Cottrell LK (1981). Computer readability editing system. IEEE Transactions on Professional Communication, 24(1), 38–42. 10.1109/TPC.1981.6447821 [DOI] [Google Scholar]
  24. Koob G (2014). NIAAA: Current and future priorities. Presentation at Friends of NIAAA Coalition, Bethesda, MD. [Google Scholar]
  25. Kuruuzum A, & Koksal CD (2010). The impact of service quality on behavioral intention in hospitality industry. International Journal of Business and Management Studies, 2(1), 9–15. [Google Scholar]
  26. Martin G (2011). The Importance Of Marketing Segmentation. American Journal of Business Education, 4(6), 15. 10.19030/ajbe.v4i6.4359 [DOI] [Google Scholar]
  27. Nakamura BJ, Chorpita BF, Hirsch M, Daleiden E, Slavin L, Amundson MJ, Rocco S, Mueller C, Osiecki S, Southam-Gerow MA, Stern K, & Vorsino WM (2011). Large-scale implementation of evidence-based treatments for children 10 years later: Hawaii’s evidence-based services initiative in children’s mental health. Clinical Psychology: Science and Practice, 18(1), 24–35. 10.1111/j.1468-2850.2010.01231.x [DOI] [Google Scholar]
  28. National Center on Addiction and Substance Abuse at Columbia University. (2011). Adolescent substance use: America’s #1 public health problem. [Google Scholar]
  29. National Institute on Drug Abuse. (2015). 2016-2020 NIDA strategic plan. Bethesda, MD [Google Scholar]
  30. Nock MK, & Kazdin AE (2005). Randomized controlled trial of a brief intervention for increasing participation in parent management training. Journal of Consulting and Clinical Psychology, 73(5), 872–879. 10.1037/0022-006X.73.5.872 [DOI] [PubMed] [Google Scholar]
  31. Okamura KH, Orimoto TE, Mah AC, Slavin LA, Rocco S, Shimabukuro SK, Michels MS, & Nakamura BJ (2018). The Help Your Keiki website: Increasing youth and caregiver awareness of youth psychosocial mental health treatment. Hawai’i Journal of Medicine & Public Health, 77(8), 203–207. [PMC free article] [PubMed] [Google Scholar]
  32. Sanders MR, & Kirby JN (2012). Consumer Engagement and the Development, Evaluation, and Dissemination of Evidence-Based Parenting Programs. Behavior Therapy, 43(2), 236–250. 10.1016/j.beth.2011.01.005 [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Santucci LC, McHugh RK, & Barlow DH (2012). Direct-to-Consumer Marketing of Evidence-Based Psychological Interventions. Introduction. Behavior Therapy, 43(2), 231–235. 10.1016/j.beth.2011.07.003 [DOI] [PubMed] [Google Scholar]
  34. Schofield CA, Ponzini GX, & Becker SJ (2020). Evaluating approaches to marketing cognitive behavioral therapy: does evidence matter to consumers? Cognitive Behaviour Therapy, 1–13. 10.1080/16506073.2019.1682654 [DOI] [PubMed] [Google Scholar]
  35. Schwartz LM, & Woloshin S (2019). Medical marketing in the United States, 1997-2016. Journal of the American Medical Association, 321(1), 80–96. 10.1001/jama.2018.19320 [DOI] [PubMed] [Google Scholar]
  36. Substance Abuse and Mental Health Services Administration. (2017). Treatment Episode Data Set (TEDS): 2005-2015. National Admissions To Substance Abuse Treatment Services. [PubMed] [Google Scholar]
  37. Substance Abuse and Mental Health Services Administration. (2019). Key Substance Use and Mental Health Indicators in the United States: Results from the 2018 National Survey on Drug Use and Health, www.samhsa.gov/
  38. Szymanski J (2012). Using Direct-to-Consumer Marketing Strategies With Obsessive-Compulsive Disorder in the Nonprofit Sector. Behavior Therapy, 43(2), 251–256. 10.1016/j.beth.2011.05.005 [DOI] [PubMed] [Google Scholar]
  39. Teitcher JEF, Bockting WO, Bauermeister JA, Hoefer CJ, Miner MH, & Klitzman RL (2015). Detecting, preventing, and responding to “fraudsters” in internet research: Ethics and tradeoffs. The Journal of Law, Medicine & Ethics, 43(1), 116–133. 10.1111/jlme.12200 [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. UCLA Statistical Consulting Group, (n.d.). Deciphering interactions in logistic regression. Retrieved August 10, 2019, from https://stats.idre.ucla.edu/stata/seminars/deciphering-interactions-in-logistic-regression/
  41. Zeithaml VA, Berry LL, & Parasuraman A (1996). The behavioral consequences of service quality. Journal of Marketing, 60(2), 31. 10.2307/1251929 [DOI] [Google Scholar]
  42. Zeithaml VA, Bitner MJ, & Gremler DD (2012). Services Marketing: Integrating Customer Focus across the Firm (Vol. 6). McGraw Hill Education. [Google Scholar]

RESOURCES