Pilot Evaluation of the Medication Integration Protocol for Adolescents with ADHD in Behavioral Care: Treatment Fidelity and Medication Uptake

Abstract

Although Attention-Deficit/Hyperactivity Disorder (ADHD) is prevalent among adolescents in outpatient behavioral care, one of the few evidence-based treatment options, stimulant medication, is significantly underutilized. The Medication Integration Protocol (MIP) is a family-based intervention designed to help behavior therapists assume a lead role in educating clients about ADHD in adolescents, promoting family-centered decisions about medication initiation, and integrating medication management activities within behavioral treatment planning. This pilot study evaluated treatment fidelity and medication utilization for inner-city teens receiving MIP (n = 14) compared to a matched Historical Control (HC) group (n = 21) in a community clinic. Observational analyses revealed that in comparison to HC, MIP demonstrated basic protocol fidelity with regard to adherence to the MIP protocol, therapeutic alliance with the adolescent, and clinical focus on ADHD in session. MIP showed greater psychiatric evaluation completion and ADHD medication initiation than HC. Next steps in the ongoing development of MIP are outlined.

This article describes the pilot evaluation of the Medication Integration Protocol (MIP), a new clinical resource for behavior therapists treating adolescents diagnosed with Attention-Deficit/Hyperactivity Disorder (ADHD) in outpatient care. MIP is a structured protocol designed to help behavior therapists assume a leadership role in educating teens and caregivers about ADHD symptoms and related characteristics, assisting families in making patient-centered decisions about medication initiation, and (when indicated) integrating medication management activities with ongoing behavioral treatment planning. The full conceptual and procedural model for MIP is detailed elsewhere (Hogue, Bobek, Tau, & Levin, 2014).

Prevalence of ADHD among Adolescents

Adolescents who meet diagnostic criteria for ADHD present well-documented behavioral deficits in attention, self-regulation, and social competence; academic deficits that include inconsistent attendance, poor grades, disruptive classroom behavior, and time management problems; a higher incidence of learning difficulties; and higher rates of executive functioning deficits in cognitive flexibility, working memory, and processing speed (Barkley, 2006). Recent national prevalence data gathered by the National Survey of Children’s Health (Visser et al., 2014) indicate that 14% of children aged 11-17 have received an ADHD diagnosis at some point in their lives, confirming its status as the most prevalent behavioral disorder among adolescents (Merikangas et al., 2011). ADHD is also highly prevalent among adolescents enrolled in behavioral services, affecting between 20-50% of those in outpatient mental health and substance use treatment (Turner, Muck, Muck, Stephens, & Sukumar, 2004) as well as those receiving behavioral services in nonspecialty settings such as school mental health, juvenile justice, and child welfare (Jones, Foster, & CPPRG, 2009). These data suggest that the outpatient behavioral care system (SAMHSA, 2007; 2009) treats between 750,000 and 1.5 million teenagers with ADHD on a yearly basis. Note that these are conservative estimates of clinical prevalence given that ADHD is underdiagnosed in adolescent samples (Sibley et al., 2012) and frequently co-occurs undetected with other chronic psychiatric disorders that are the primary reasons for clinical referral (see Merikangas et al., 2011).

Medication is a First-Line Treatment for ADHD in Adolescents

Medication is an evidence-based option for treating ADHD in adolescents (American Academy of Pediatrics, 2011) and is considered an essential component of effective treatment planning for this population (Obioha & Adesman, 2014). Stimulant medications in particular have strong research support for adolescents. Rapid-acting stimulants such as methylphenidate (Ritalin) and extended-release stimulants such as OROS-MPH (Concerta) have proven consistently effective in reducing ADHD symptoms and improving social and academic functioning (Sibley, Kuriyan, Evans, Waxmonsky, & Smith, 2014). Altogether, available data indicate that ADHD medication has significant effects in multiple domains for adolescents.

For adolescents with ADHD in outpatient care, integrating medication services with behavioral interventions to deliver a combined intervention package is widely considered the optimal treatment approach (American Academy of Pediatrics, 2011; Obioha & Adesman, 2014), for two main reasons. First, combined treatments have generated the strongest empirical support for treating ADHD among children (Smith, Barkley, & Shapiro, 2006). The most rigorous longitudinal study to date found that the superiority of combined ADHD treatment over monotherapies during childhood years dissipated in adolescence, which could be attributed to medication desistance, stronger maintenance effects for behavioral interventions, or both (Molina et al., 2009). Second, although parenting and classroom behavioral management interventions are consistently effective for ADHD in childhood, they have proven to be developmentally mismatched for teens given that the older group is monitored by adults less closely, motivated by more diverse and less accessible rewards, and encounter numerous teachers in various classrooms during the school day (Evans, Owens, & Bunford, 2014). And whereas school-based organizational training systems have shown promise for older youth, organizational training has not been well tested in routine clinical settings (Evans et al., 2014; Sibley et al., 2014). Thus evidence-based behavioral interventions for adolescents are in short supply for clinicians working in outpatient settings, further elevating the importance of medication services for effective ADHD treatment planning.

ADHD Medication is Underutilized among Adolescents in Clinical Care

Legitimate concerns have been raised by both researchers and the media (see Hinshaw & Scheffler, 2014; Smih & Farah, 2011) that ADHD medications are increasingly overprescribed for sub-diagnostic or incorrectly labeled youth, primarily to boost test and grade performance. However, among adolescents who meet full diagnostic criteria for ADHD, medications remain indisputably underutilized: Just over half of teens who qualify for ADHD medications actually receive them, compared with greater than two-thirds of younger children (Visser et al., 2014). This gap in quality care is even more pronounced among ethnic minorities (Yeh et al., 2014), with Hispanic and African American teens having significantly lower rates of ADHD medication prescriptions than their majority peers (Visser et al., 2014). Also, as mentioned ADHD is routinely under-diagnosed among teens who enter specialty care, which forecloses appropriate medication services for many youth with the most complex and severe behavioral needs.

The widespread fragmentation of pharmacological versus behavioral services in routine care (Institute of Medicine, 2006) creates additional barriers to ADHD medication uptake. Behavior therapists are often not trained to access pharmacological services when ADHD symptoms constitute a secondary diagnosis or emerge after treatment begins (Sobell & Sobell, 2007). Also, few evidence-based resources exist to guide clinicians in merging medication interventions into behavioral treatment planning for teens (see Bukstein & Cornelius, 2006). As a result, families in community-based services are often poorly informed about choices of available ADHD medications and the risk and benefits of each (Yeh et al., 2014), especially whenever behavior therapists themselves have insufficient knowledge about ADHD or harbor biases against psychiatric medication (Murphy, 2005).

Finally, even among those families who receive competent medication consultation, acceptance and compliance are extremely difficult to achieve. ADHD medication adherence declines precipitously from childhood through adolescence (Sanchez, Crismon, Barner, Bettinger, & Wilson, 2005), with up to 90% of teens either refusing or desisting medication by the end of high school (Sibley et al., 2014). This age-based decline is likely due a combination of several factors: increased adolescent sensitivity to medication stigma (Walker, Coleman, Lee, Squire, & Friesen, 2008), increased adolescent misperceptions or disregard for medication effects (Pelham, Gnagy, et al., 2013; Pelham, Smith, et al., 2013), and decreased parental monitoring and influence on daily adolescent self-care (Smith, Waschbusch, Willoughby, & Evans, 2000). Moreover, caregivers of children and teens alike much prefer behavioral intervention to medication as a primary treatment option (Smith et al., 2000).

Marshalling Clinical Strategies for Behavior Therapists to Integrate Pharmacological Interventions into Outpatient Behavioral Care for Adolescents with ADHD

MIP has two patient-centered clinical aims: (a) assist adolescents and caregivers in making informed decisions about ADHD medication acceptance and (b) for those that select medication, support decisions about ongoing participation and compliance with medication regimens. To achieve these aims, MIP features two evidence-based clinical strategies that advance ADHD medication integration efforts; these strategies comprise the foundation of the five MIP Tasks described in the Methods section.

Family ADHD psychoeducation provides consumer-friendly information about symptoms, prevalence and etiology, course of the disorder, impacts on multiple domains of functioning, co-occurring problems, and individual differences associated with ADHD in adolescents (Robin, 2014). It sometimes includes collaborative clinical assessment of the individual profile of ADHD-related characteristics for the given teen (Hogue, Bobek, et al., 2014). ADHD psychoeducation has been shown to enhance medication and behavioral treatment effects (Lincoln, Wilhelm, & Nestoriuc, 2007) and improve treatment adherence (Vieta, 2005) and medication compliance (Cummings & Fristad, 2007) for clients with a variety of mental health problems. And family-focused psychoeducation models (i.e., conjoint educational sessions with parents and youths) have proven especially helpful for youths who are prescribed psychiatric medication (Fristad, 2006), including medications for ADHD (Lopez, Toprac, Crimson, Boemer, & Baumgartner, 2005).

Family-based medication decision-making interventions, in which family attitudes about medication are systematically processed in the context of collaborative benefit-cost decisions about treating ADHD, are essential for facilitating family-centered decisions about medication use among teens (Davis, Claudius, Palinkas, Wong, & Leslie, 2012). Medication selection is directly related to caregiver beliefs about the causes of behavioral problems (Yeh et al., 2014), and beliefs about ADHD in particular are mutable over time when families engage in active decision-making with providers (Davis et al., 2012). Family factors play a lead role in predicting medication acceptance among teens (Smith et al., 2000) as well as safety and polypharmacy issues associated with prescribing medications to at-risk youth (Kollins, 2007). As mentioned previously, the primary reason for ADHD medication desistance among adolescents is lack of teen motivation to continue (Sibley et al., 2014). Several empirically supported family therapy models contain interventions specifically designed to engage teens meaningfully in defining treatment goals and participating in treatment processes (e.g., Liddle, 1995); such interventions are ideal candidates for boosting the effectiveness of medication decision-making (Robin, 2014).

Piloting the Medication Integration Protocol: Study Design and Hypotheses

This pilot study of the Medication Integration Protocol used a quasi-experimental, interrupted time series design that tracked target data before and after introducing a programmatic change within a single clinical site, a design appropriate for piloting the feasibility and immediate impact of new interventions in field settings (Cook & Campbell, 1979). The study followed the basic template of the stage model of behavioral treatment that proceeds from protocol development to provider training and fidelity tool development, culminating in a small pilot study (Rounsaville, Carroll, & Onken, 2001). MIP was integrated into existing treatment services at a partner clinical site and delivered by site therapists who were newly trained and monitored in MIP. MIP study cases (n = 14) were then compared to matched Historical Control (HC) cases (n = 21; matching criteria presented in Method section) that had been treated at the partner site prior to the introduction of MIP. Study aims were to explore MIP fidelity and acceptability in routine care and collect pilot data regarding MIP impact on its primary target: psychiatric services utilization.

The first hypothesis, pertaining to treatment fidelity, was that MIP therapists would show greater clinical focus on ADHD-related issues and greater utilization of MIP-specific interventions than HC therapists; treatment conditions were not expected to differ in use of core family therapy techniques, focus on general clinical issues (e.g., family relations, peer relations), and working alliance with the adolescent. The second hypothesis, pertaining to psychiatric service utilization, was that MIP cases would demonstrate greater participation in psychiatric evaluation, medication acceptance, and medication use than HC cases. Note that due to the quasi-experimental nature of the research design, study hypotheses were considered to be preliminary tests of MIP impact on routine care.

Method

The study was conducted under approval by the governing Institutional Review Board. There were no differences between study conditions in consenting procedures or incentives to participate in research interviews; also, no incentives were provided to participate in treatment.

Sample Characteristics and Group Comparability

The sample consisted of 35 adolescents across the two study conditions: 14 MIP cases and 21 HC cases. Proportional rates for pre-treatment demographic, diagnostic, and other clinical characteristics for the entire sample and separately for each condition are reported in Table 1. Participants included both males (57%) and females (43%) and averaged 15.4 years of age (SD = 1.5). Self-reported ethnicities were Hispanic (71%), African American (26%), and multiracial (3%). Households were primarily headed by a single caregiver (87%); 60% of caregivers had graduated high school and 49% worked full- or part-time; 34% earned less than $15,000 per year; and 3% reported a history of child welfare involvement. Adolescents were referred from schools (60%), the treatment site (26%), and other sources (14%).

Table 1.

Pre-Treatment Demographic and Clinical Characteristics

		Study Condition
	Total Sample (N = 35)	MIP (n = 14)	HC (n = 21)
Demographic Characteristics
Female Gender	43%	50%	38%
Age in years (M/SD)	15.4 (1.5)	15.3 (1.3)	15.5 (1.6)
Ethnicity
Hispanic	71%	50%	86%
African American	26%	43%	14%
Multiracial	3%	7%	0%
Family Composition
Single parent	87%	93%	86%
Two parent	13%	7%	14%
Caregiver graduated high school	60%	79%	48%
Caregiver employed	49%	50%	48%
Household income < $15K	34%	43%	29%
History of child welfare system involvement	3%	0%	5%
Referral source***
School	60%	36%	76%
Family Service Agency	9%	0%	14%
Treatment Site	26%	64%	0%
Other	5%	0%	10%
DSM-IV Diagnoses
Oppositional Defiant Disorder	86%	93%	81%
Conduct Disorder	63%	64%	62%
Depression (MDD and/or Dysthymia)	40%	29%	48%
Substance Use Disorder (any type)	37%	36%	38%
Generalized Anxiety Disorder	11%	14%	10%
Posttraumatic Stress Disorder	9%	0%	14%
More than one diagnosis	97%	93%	100%
Clinical Characteristics
IEP	37%	43%	33%
Educational Intervention	49%	64%	38%
Academic Problems**	60%	86%	43%
School Attendance Problems	46%	50%	43%
Disciplinary Problems*	34%	60%	19%
Picked-Up by Police	20%	21%	19%
On Probation or Parole	11%	14%	10%
Outpatient Mental Health Treatment	17%	29%	10%

MDD = Major Depressive Disorder; IEP = Individualized Education Program.

Note. Significant condition differences indicated as

^*p < .05,

^**p < .01,

^***p < .001

Structured diagnostic data were collected during the confidential research baseline interview and were not made available to the clinical site (which did not conduct its own structured diagnostic interviews with clients). As seen in Table 1, all participants carried a pre-treatment diagnosis of ADHD based on the Diagnostic and Statistical Manual of Mental Disorders, 4^th Edition (DSM-IV; American Psychiatric Association, 2000); In addition to ADHD, 34 (97%) of the adolescents were diagnosed with at least one other psychiatric disorder. Overall DSM-IV diagnosis rates were: Oppositional Defiant Disorder (ODD) = 86%, Conduct Disorder (CD) = 63%, Mood Disorder or Dysthymia = 40%, Substance Use Disorder (SUD) = 37% (94% cannabis use, 3% cannabis and alcohol, and 3% alcohol only), Generalized Anxiety Disorder = 11%, and Post-traumatic Stress Disorder = 9%. No condition differences were found in diagnosis rates. Not surprising for a sample of youth diagnosed with ADHD, highest rates of psychosocial problems were related to past-year school issues: academic problems (60%), receiving educational intervention (49%), school attendance problems (46%), and having an Individualized Education Program (37%).

To examine whether the group matching process (detailed below in the description of the HC condition) created equivalent study conditions, chi-square and mean comparison tests were used to detect group differences on all sample characteristics; see Table 1. The MIP condition was significantly more likely to contain adolescents referred directly from the treatment site (χ²(1) = 19.13, p < .001). This finding is in keeping with the differences in the HC versus MIP referral streams (described below): HC cases were recruited from the community and assigned to the treatment site; in contrast, 64% of MIP cases were recruited directly from the treatment site itself. Regarding clinical characteristics, MIP was more likely than HC to report past year academic problems (χ²(1) = 6.43, p < .01) and disciplinary problems (χ²(1) = 5.41, p < .05).

Study Enrollment and Assessment Procedures: MIP Condition

Study eligibility criteria.

Inclusion criteria for MIP enrollment were: Adolescent was age 13-17; primary caregiver was willing to participate in treatment; adolescent met DSM-IV criteria for ADHD (with or without onset prior to age 7); adolescent met DSM-IV criteria for ODD, CD, or SUD; adolescent did not have a current ADHD medication prescription; adolescent was not currently enrolled in any behavioral treatment. Exclusion criteria were: Bipolar Disorder, mental retardation, Pervasive Developmental Disorder, medical or psychiatric illness requiring hospitalization, current psychotic features, or current suicidality.

Participant recruitment and research interviews.

New study participants enrolled in the MIP condition were recruited from one of two referral streams. For both streams, conduct problems (ODD, CD, and SUD), but not ADHD, were the primary reasons for referral. The first stream was identical to the one used in the previous randomized trial to recruit the larger sample from which HC cases were selected (Hogue, Dauber, Henderson, et al., 2014): a referral network of high schools and community-based agencies serving inner-city youth and families. Network partners provided study referrals to research staff during site visits and also by phone. The second referral stream was specific to the MIP condition and drawn directly from clients entering the existing referral process at the treatment site. In this stream, research staff was alerted by clinic staff whenever an adolescent was referred; staff then contacted the primary caregiver by phone and sought verbal consent to administer an ADHD screening instrument (Swanson, Nolan, & Pelham, 1992). Eligible families were offered the opportunity to participate in the study and, for those agreeing, scheduled for a research assessment prior to enrolling in treatment. As in the HC condition, assessments were conducted primarily in the home but also in other locations upon request; caregivers and teens were consented and interviewed separately.

Study Enrollment Rates

There were 41 families referred to the MIP condition who were contacted for phone screenings. Of these, 31 (76%) consented to complete the phone screen. Of those completing screens, 26 (84%) met study eligibility requirements. Of the 26 eligible cases, 20 (77%) completed a pretreatment assessment and were scheduled for treatment intake. Of this group, 14 (70%) completed at least one clinical intake interview at the treatment site.

Treatment Site, Conditions, Therapists, and MIP Fidelity Monitoring

Treatment site.

Both study conditions were treated by front-line staff at a community-based mental health clinic that featured family therapy as the routine standard of care and provided on-site psychiatric evaluation, consultation, and psychopharmacology services based on clinical need. Previous observational treatment fidelity analyses (Hogue, Dauber, Lichvar, et al., 2015) confirmed that site therapists adhered to core principles and techniques of ecological family therapy when treating adolescents with behavior problems. Per clinic flow and billing demands, all sessions at the treatment site were scheduled for approximately 50 minutes apiece.

Medication Integration Protocol (MIP) condition.

MIP (Hogue & Bobek, 2013), a family-based protocol designed to integrate medication services into behavioral treatment planning for adolescents with ADHD (see Hogue, Bobek, et al., 2014), contains five clinical Tasks that are modular and flexibly delivered. Task 1: ADHD Assessment and Medication Consult. Therapist consults with prescriber to confirm an ADHD diagnosis and adolescent eligibility for medication. In behavioral sessions, therapist helps family understand the psychiatric evaluation and its results. Task 2: ADHD Psychoeducation and Client Acceptance. Therapist and family review educational materials that prompt interactive discussions about key issues pertaining to ADHD: prevalence rates and behavioral symptoms; common impacts on socioemotional development and school performance; and ADHD neurobiology and its relation to executive functioning and learning problems. Therapist promotes acceptance of the condition, defuses moral attributions, and establishes practical expectations for change. Therapist and family review a checklist of personality and social characteristics associated with ADHD, along with checklists of common ADHD-related impairments in three domains (family, school, peer), to anchor psychoeducation and identify family-endorsed treatment goals. Therapist educates family about benefits, expected course, and potential side effects of ADHD medications; details the trial-and-error approach to dosing; and summarizes other key factors that inform decision-making about medication initiation. Task 3: ADHD Symptoms and Family Relations: Therapist engages teen as active participant in therapeutic activities; links ADHD-related characteristics to family, school, and peer functioning; alters negative attributions about ADHD-related behavior by emphasizing unrecognized or mislabeled causes (“relabeling”); redefines adolescent referral problems as family problems with family solutions (“reframing”); assesses home environment characteristics that support or impede school and treatment success; and assesses caregiver and teen readiness to make therapeutic changes. Task 4: ADHD Medication and Family Decision-Making: Therapist and family discuss ongoing consultations with prescriber about medication issues. Therapist helps family understand the potential benefits of medication in home, school, and peer contexts. Therapist raises issues regarding stigma, side effects, titration, and substance misuse (including potential medication diversion). Family accepts, refuses, defers, or is declared ineligible (due to chaotic or unsupervised home setting) for medication. Task 5: Medication Management and Integration Planning: Therapist plays lead role in formulating a case coordination framework for medication management tailored to each family, with therapist and prescriber working in integrated fashion to support compliance and monitor benefits and side effects. Prescriber implements titration schedule and arranges ongoing medication management visits; therapist makes routine inquiries about medication issues during behavioral sessions.

Historical Control (HC) condition.

Archived cases from a previous randomized trial of routine outpatient care for adolescent behavior problems (Hogue, Dauber, Henderson, et al., 2014) were selected to serve as Historical Control cases. To select cases that most closely resembled those enrolled in MIP, the following matching criteria were used: (1) Case assigned to the same treatment site that later treated MIP cases; (2) Adolescent diagnosed with ADHD. A total of 21 cases in the previous trial met all criteria and were included in the current study.

Study therapists.

MIP therapists (n = 3) ranged in age from 29 to 32 years; all were female, two were Hispanic American, and one was European American. They averaged 3.0 years (SD = 1.7) of postgraduate therapy experience. HC therapists (n = 8) ranged in age from 31 to 59 years; 3 were female, 7 were Hispanic American and 1 was European American. They averaged 8.2 years (SD = 4.3) of postgraduate therapy experience.

MIP fidelity monitoring.

MIP therapists were trained during two 2-hour sessions by study authors (AH and MB) using a manualized protocol that described the clinical background, conceptual framework, and specific treatment techniques associated with each of the five MIP tasks, along with corresponding psychoeducation slides and clinical assessment tools (Hogue & Bobek, 2013). Therapists then participated in weekly fidelity monitoring sessions to maintain adherence to the MIP protocol and integrate MIP interventions within the overall behavioral treatment plan for each case. Fidelity monitoring procedures utilized a self-report MIP fidelity checklist (presented in Hogue, Bobek, et al., 2014) and videorecordings of each session.

MIP Fidelity Evaluation Procedures

MIP fidelity measures.

Three interrelated dimensions of treatment fidelity were measured: treatment adherence, clinical focus, and therapeutic alliance. Treatment adherence and clinical focus were assessed with the Inventory of Therapy Techniques—Observational Version—MIP (ITT-O-MIP). The ITT-O-MIP contains three sections: (a) FT Adherence Scale: 8 items capturing delivery of core family therapy (FT) interventions for adolescent conduct and substance use problems; (b) Clinical Foci: 6 items capturing important domains of adolescent developmental functioning discussed in the given session: drug use, peer interactions, family functioning, prosocial involvement, systems involvement, and ADHD-related focus; and (c) MIP Tasks: 5 items capturing delivery of MIP interventions: Psychiatric Assessment and Medication Consult, ADHD Psychoeducation and Client Acceptance, ADHD Symptoms and Family Relations, ADHD Medication and Family Decision-Making, Medication Management and Integration Planning. Ratings for the FT Adherence Scale and Clinical Foci items are based on a 5-point Likert-type scale: 1 = Not at all, 2 = A little bit, 3 = Moderately, 4 = Considerably, 5 = Extensively. Ratings for MIP Task items are estimates of the total number of minutes devoted to the given task during the session. FT Adherence Scale items and Clinical Foci items were drawn from the Therapist Behavior Rating Scale (Hogue et al., 1998), a macroanalytic tool designed to identify core treatment techniques and foci prescribed by FT for adolescent behavior problems.

Working alliance with the adolescent was measured using the Adolescent Therapeutic Alliance Scale (ATAS; Faw, Hogue, Johnson, Diamond, & Liddle, 2005), a 14-item scale that measures the bond (strength of collaborative relationship), task (specific activities aimed at facilitating change), and goal (areas of functioning targeted for change) components of alliance. ATAS items are rated on 6-point Likert-type scale ranging from 0 (none) to 5 (a great deal).

Selection of MIP and HC sessions.

Videorecordings of treatment sessions were available for 21 cases (12 MIP, 9 HC). From these a total of 119 sessions (80 MIP, 39 HC) were selected for coding, the largest number that could be coded with existing study resources. For each case, two sessions were randomly selected from available recordings from the Early phase of treatment (session 2-5), and one session was randomly selected from the Later phase (session 6 or greater); overall, 65% of selected sessions were from Early phase, 35% from Later phase.

Observational fidelity raters and rating procedures.

Observational raters (n = 9) were undergraduate research assistants from a small liberal arts college in New York City. The group contained 4 European American females, 3 European American males, 1 Hispanic female, and 1 Hispanic male; average rater age was 21 years. None had prior coding experience. Raters were trained during weekly meetings over one month using review of the ITT-O-MIP and ATAS coding manuals and related exercises to increase understanding of scale items. The group then met weekly for supportive training and monitoring of rater drift until coding was completed. Two raters were assigned to code each session and were randomly paired with one another across sessions (excepting Spanish language recordings).

Pre-Treatment Research Measures and Psychiatric Utilization Logs

The Comprehensive Addiction Severity Index for Adolescents (Meyers, McLellan, Jaeger, & Pettinati, 1995) collects demographic information on high-risk adolescents and their families and also assesses risk factors in education (academics, attendance, behavior, attitudes), family relations (history of drug use/criminality, abuse/neglect), and legal involvement (illegal activities, juvenile justice). The Mini International Neuropsychiatric Interview (Sheehan et al., 1998) is a structured diagnostic interview that assesses DSM diagnoses in adolescent and adult populations. It has strong convergent validity with the SCID (Sheehan et al., 1998) and CIDI (Lecrubier et al., 1997). Psychiatric Service Logs for each client documented (1) whether an on-site psychiatric evaluation was provided based on perceived clinical need and (2) the number of days for which ADHD medication and other medications were prescribed during treatment.

Results

MIP Fidelity

One-way random ICCs (Shrout & Fleiss, 1979) were calculated across study conditions for all scale scores and individual items; according to Cicchetti’s (1994) criteria for classifying the utility of ICC magnitudes are: below .40 is poor, .40–.59 is fair, .60–.74 is good, and .75–1.00 is excellent. For the FT Adherence scale score, ICC = .76. For the Clinical Foci items, Drug ICC = .87, Peer = .66, Family = .75, Prosocial Involvement = .68, Systems Involvement = .88, ADHD = .89. The five MIP Task items ranged .53 to .88. For the ATAS scale score, ICC = .89.

Mean between-condition differences on all fidelity measures are depicted in Table 2. Independent sample t-tests were conducted to examine group differences. HC therapists utilized FT techniques to a statistically greater extent than MIP (t(117) = 4.91, p < .001). There were several differences on Clinical Foci items: HC placed statistically stronger emphasis on Peer (t(117) = 2.10, p < .01) and Family (t(117) = 2.94, p < .01) issues, whereas MIP showed greater Prosocial (t(117) = −3.75, p < .001) and ADHD (t(63) = −5.57, p < .001) focus. MIP sessions evidenced statistically greater utilization of each MIP task: Assessment and Medication Consult (t(63) = −4.60, p < .001); ADHD Psychoeducation and Client Acceptance (t(63) = −3.80, p < .001); ADHD Symptoms and Family Relations (t(117) = −3.34, p < .001); ADHD Medication and Family Decision-Making (t(63) = −3.86, p < .001); Medication Management and Integration Planning (t(63) = −3.34, p < .001). No difference was found for strength of adolescent alliance.

Table 2.

Fidelity Scales: Session-Level Mean Scores and Between-Group Comparisons

	Mean (SD)
Item	HC n = 39	MIP n = 80	t-value
FT Adherence Mean Scale	2.1 (.48)	1.7 (.42)	4.91***
Clinical Foci Items
Drug Focus	1.5 (.87)	1.5 (.87)	0.08
Peer Focus	2.1 (.81)	1.7 (.49)	2.10**
Family Focus	3.9 (1.0)	3.2 (1.5)	2.94**
Prosocial Focus	2.5 (.79)	3.2 (1.1)	−3.75***
Systems Focus	1.7 (74)	1.6 (1.1)	0.37
ADHD Focus	1.0 (.14)	2.4 (1.16)	−5.57***
MIP Tasks
MIP Task 1	1.0 (0)	1.9 (.96)	−4.60***
MIP Task 2	1.0 (.10)	2.0 (1.2)	−3.80***
MIP Task 3	1.1 (.32)	1.6 (.74)	−3.34***
MIP Task 4	1.0 (0)	1.8 (1.1)	−3.86***
MIP Task 5	1.0 (0)	1.5 (.77)	−3.34***
Alliance
ATAS	2.2 (.61)	2.4 (.80)	−0.91

Note. Indicates significance at:

^**p<.01,

^***p<.001

These mean between-condition differences are modest in terms of absolute values. For example, the averaged ADHD Focus score increases 1.4 notches, from a flat 1.0 (“Not at all”) for HC to a 2.4 (midway between “A little bit” and “Moderately”) for MIP. However, note that mean scale scores averaged across randomly selected sessions, while highly informative, are not absolute barometers of strong adherence to manualized interventions, for three reasons (Hogue et al., 2008) : (1) For any given group of sessions, a therapist might receive a relatively high score on a given item in one or a few sessions but a low score for the remaining sessions, producing a low mean score that masks extensive but session-limited use of that technique; (2) For multi-item scales such as FT, it is not typical or clinically advisable to extensively implement any given technique(s) for a given approach across every single session; (3) For individually tailored protocols such as MIP there are no specific prescriptions for intervention doses in given sessions, making it difficult to localize where and when substantial doses did or should occur. Similarly, the averaged MIP Task items, which were measured in minutes devoted during sessions, increase only about one minute per task. However this increase represents about 5 total minutes per session, which translates into a full 50-minute session devoted to implementing MIP Tasks for every 10 treatment sessions, arguably a substantive amount given that ADHD was not a primary reason for referral for any participant. Comprehensive fidelity analyses in this vein—for example, dose effects (strength of adherence in those sessions with any MIP delivery) and phase effects (timing and sequencing of MIP delivery)—are of high interest but beyond the scope of the current study.

Psychiatric Service Utilization

Medication acceptance rates are also reported in Table 3. MIP (100%) was more likely than HC (48%) to complete a psychiatric evaluation (χ²(1) = 10.70, p < .001), to be prescribed any medication (57% vs. 5%) (χ²(1) = 12.07, p < .001), and to be prescribed an ADHD medication (43% vs. 5%) (χ²(1) = 7.62, p < .01). Because only one HC case was prescribed an ADHD medication, it was not possible to calculate between-group differences in number of ADHD medication days. Within MIP, 8 of the 14 cases did not start an ADHD medication regimen. The reasons this subgroup did not start medication were: family decided that ADHD medication was not a good fit for them (4 cases), adolescent was instead prescribed a different medication for a co-occurring disorder (2 cases), prescribing psychiatrist did not believe medication was a good fit for the family (1 case), and family did not remain in treatment long enough to complete the decision-making process (1 case). Within HC, the single case receiving ADHD medication was already on medication upon entry into treatment.

Table 3.

Medication Acceptance Rates: Sample Totals and Between-Group Comparisons

	Total (N = 35)	HC (n = 21)	MIP (n = 14)
Completed Psychiatric Evaluation***	69%	48%	100%
Prescribed any Medication***	26%	5%	57%
Prescribed ADHD Medication**	20%	5%	43%
# Days on ADHD Medication	86.4 (49.9)	105 (0)	83.3 (54.0)
Prescribed Non-ADHD MedicationΨ	6%	0%	14%
# Days on Non-ADHD Medication	30 (0)	0	30 (0)

Note. Significant condition differences indicated as

^Ψp < .10,

^*p < .05,

^**p < .01,

^***p < .001

Discussion

Results from this pilot evaluation of the Medication Integration Protocol offer promising but still preliminary support for its implementation feasibility and service utilization impacts in routine clinical settings. Community therapists trained in MIP demonstrated basic fidelity to the structured procedures of the protocol, compared to HC therapists. MIP cases (n = 14) showed greater psychiatric evaluation completion and ADHD medication acceptance than matched Historical Control cases (n = 21) recently treated in the same clinic; ADHD medication duration could not be reliably compared due to the near absence of HC cases receiving medication.

The main finding from this pilot evaluation is that MIP outperformed HC in promoting psychiatric services. The fundamental aim of MIP is to improve the integration of pharmacological interventions into behavioral treatment. The preliminary findings of this study show that MIP cases were more likely than HC cases to complete a psychiatric evaluation and be prescribed an ADHD medication. Although these non-experimental data are not conclusive, it appears that the protocol may have encouraged these community therapists to seek and obtain psychiatric consultation that culminated in initiation of medication for many clients. This clinical focus runs counter to often strong preferences for avoiding ADHD medication that commonly exist among both treatment-naïve (Waschbusch et al., 2011) and treatment-experienced (Johnston, Hommersen, & Seipp, 2008) families, preferences especially pronounced among ethnic minority clients (Krain, Kendall, & Power, 2005; Pelham, Smith, et al., 2013). These pilot results fall in line with other recent calls for increasing ADHD service involvement among adolescents via psychoeducational interventions that target ADHD stigma and medication receptivity (Bussing, Zima, Mason, Porter, & Garvan, 2011; Davis et al., 2012).

Establishing treatment fidelity is also critical in the treatment development process (Rounsaville et al., 2001). Observational fidelity analyses confirmed the basic integrity of MIP implementation by means of discriminability from the HC condition. MIP sessions evidenced modest but perhaps meaningful delivery of MIP Task interventions, as well as general clinical focus on ADHD content. Contrary to expectations, HC outpaced MIP in delivery of core family therapy techniques. Even so, MIP sessions featured a moderate level of family therapy techniques and also clinical focus on family issues, in keeping with the family-based foundation of the protocol. MIP therapists showed equivalence with HC in forming a working alliance with the adolescent.

Study Limitations

Despite the group comparability observed on most demographic and clinical variables, the quasi-experimental study design necessarily limits causal inferences that can be attributed to between-group comparisons on service utilization. MIP cases were more likely to have been referred from the partner treatment clinic and thus to have sought treatment on their own. In contrast, HC cases typically were not actively seeking treatment prior to contact with research staff. Also, because MIP cases were drawn primarily from a clinic referral stream, it is quite possible that they possessed greater commitment, resources, familiarity with the treatment system, and/or motivation to pass through clinical referral gates than the majority of HC cases. It is likewise impossible to rule out that unmeasured pre-treatment group differences, rather than MIP treatment effects, were primarily responsible for psychiatric service utilization effects. By study design, all participants had at least one co-occurring conduct or substance use disorder; thus results may not generalize fully to adolescents with ADHD only. Also, the availability of in-house psychiatric services to all participants limits the full generalizability of results to outpatient treatment facilities with similar access to prescriber resources. MIP fidelity was assessed only via comparison to HC clinicians with no exposure to MIP interventions, which can be considered a “straw man” comparison; a stronger test of fidelity would involve comparing the performance of MIP therapists to objective fidelity benchmarks, which have yet to be established for this new protocol.

There were analytic limitations as well. It was not possible to account for Therapist effects due to the thin spread of therapists across cases. Similarly, the study did not contain a sufficiently large mean number of clients per therapist to reliably estimate therapist clustering effects; thus utilization analyses were biased by non-independence among clients within a given therapist. Finally, this pilot study was not designed or powered to test whether increased medication uptake translated into improved clinical outcomes in the form of client behavioral symptoms and/or quality of life indicators; the study team is currently conducting a prospective randomized effectiveness trial of MIP to investigate protocol impacts on long-term outcomes.

Next Steps in MIP Development

Assisting adolescents with ADHD and their families in making informed decisions about medication options within a continuum of outpatient behavioral health services is an emerging area of clinical and research focus (Bussing et al., 2011; Davis et al., 2012). Results from this pilot evaluation of MIP offer initial “proof-of-concept” evidence that MIP is feasible, and potentially effective, in promoting medication interventions within routine behavioral care. The next step in protocol development is to evaluate MIP effectiveness using a randomized controlled design. This work is now underway, supported by a grant from the Patient Centered Outcomes Research Institute (CER-1403-13704). The randomized study will test the additive value of MIP when combined with evidence-based academic interventions for adolescents with ADHD treated by community behavioral therapists. The impact of integrated care (MIP plus academic interventions) versus behavioral care only (academic interventions) will be examined not only for service utilization effects (treatment attendance, medication uptake) but also clinical outcomes (ADHD symptoms, co-occurring behavioral symptoms, quality of life) along with differential client outcomes based on ADHD medication use. Adolescent and caregiver attitudes about ADHD medication and acceptance of MIP-related interventions will be assessed in quantitative and qualitative analyses. This rigorous examination of MIP will provide more reliable evidence about its fidelity and ultimate utility for addressing the complex treatment needs of adolescents with ADHD enrolled in outpatient behavioral services.