Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2025 Jul 2.
Published in final edited form as: Contemp Clin Trials. 2025 Apr 27;154:107928. doi: 10.1016/j.cct.2025.107928

Workforce and systems change to treat adolescent substance use disorder within integrated pediatric primary care: A cluster-randomized stepped-wedge trial

Leslie A Hulvershorn a,b, Matthew Aalsma a,c, Trey V Dellucci a,b, Ashlyn Burns a,b, Brigid R Marriott a,b, Bernice Pescosolido d,e, Harold D Green Jr d,f, Lisa Saldana g, Jason Chapman g, Patrick Monahan h, Sarah E Wiehe c, Edward J Miech i, Zachary W Adams a,b,*
PMCID: PMC12215800  NIHMSID: NIHMS2078831  PMID: 40300712

Abstract

Background:

While the overdose crisis has impacted all ages, overdose-related deaths among adolescents have been increasing more rapidly than any other age group, doubling between 2019 and 2020. Identifying and treating substance use disorders (SUDs) among adolescents is critical to preventing adolescent overdose deaths. While evidence-based interventions for adolescents with SUDs exist, they remain underutilized. Implementing SUD interventions in primary care settings through integrated behavioral health (IBH) is one approach for increasing access to evidence-based SUD services for adolescents.

Methods:

This is a Hybrid Type 2, cluster-randomized, stepped-wedge trial comparing SUD IBH to standard primary care treatment. In our open cohort stepped-wedge design, primary care clinics will be randomly designated to one of three cohorts. We will use a mixed-methods approach to evaluate both implementation and effectiveness outcomes, with a focus on assessing the impact of IBH on primary care provider behaviors around SUD interventions. All cohorts will complete baseline surveys during the control condition and then every 6 months. At each time point, we will also collect and analyze patient administrative data to assess patient engagement and outcomes. In addition, we will conduct qualitative interviews at pre-, mid-, and post-implementation during sustainment of the intervention.

Conclusion:

Addressing the overdose crisis is a national priority. IBH has the potential to reduce overdose rates by enhancing primary care provider willingness to deliver SUD services for adolescents.

Keywords: Substance use disorder, Overdose, Adolescents, Integrated behavioral health

1. Background

Curbing unintentional drug overdose is a national public health priority [1]. Sharp increases in overdoses have been observed over the past decade, with over 100,000 drug overdose deaths occurring annually since 2021 [2]. Rising overdose rates are mainly due to unpredictable fentanyl amounts in illicitly manufactured pills, which are generally inexpensive and widely available [3,4]. Overdose-related deaths among adolescents aged 10–19 have risen faster than any other age group, doubling between 2019 and 2020 [5]. Adolescents are particularly vulnerable to unintentional fentanyl use due to low knowledge around fentanyl presence in illicit drugs [6], lethality [7,8], and attraction to the “marketing” of colorful pills [9]. Furthermore, fentanyl presence is not limited to opioids; any illicit substance (e.g., counterfeit pharmaceuticals, methamphetamine) may contain fatal amounts of fentanyl, placing all adolescents who use substances at risk of unintentional overdose [3,4,9,10]. While adolescents with severe substance use disorder (SUD) are 3–5 times more likely to experience an overdose [1116], adolescents with mild SUD are also at risk of overdose [3,4,9,10]. Therefore, identifying and treating adolescents with any substance use, as well as those with any SUD, – including adolescents with an opioid use disorder (OUD) [17] – is critical to reducing substance use behaviors and subsequently preventing adolescent overdose deaths.

Without routine screening, SUDs may go unrecognized and untreated [18,19]. Validated SUD screening tools for adolescents exist, and behavioral and pharmaceutical treatments have demonstrated effectiveness in adolescents with SUDs [20]. The Screening-Brief Intervention-Referral to Treatment (SBIRT) model is a national public health effort to provide services to people who have or are at risk for a SUD comprising three steps: (1) universally screen adolescents for SUDs, (2) provide brief interventions for adolescents with mild SUDs, and (3) provide referrals to specialized treatment services for adolescents with more severe SUDs [21]. Brief behavioral interventions (e.g., motivational interviewing; MI) can help adolescents understand reasons for using substances, identify negative consequences of substance use, and work towards goals related to substance use [22]. Brief interventions can be more effective when combined with medications (e.g., buprenorphine, naltrexone), which can reduce acute cravings and help manage withdrawal symptoms as one decreases or abstains from using substances [20,23]. Despite promising potential, however, SBIRT and other evidence-based practices for treating adolescent SUDs remain underutilized, largely due to behavioral health workforce shortages, stigma around substance use, and implementation challenges [2426].

To overcome workforce challenges, primary care providers (PCPs) have been encouraged to adopt SBIRT [27]. Since up to 80–90 % of adolescents routinely attend primary care visits [28], primary care is familiar and convenient setting for families, providing an important opportunity to identify, diagnose, and treat adolescent SUDs [2931]. While screening by PCPs helps to identify adolescents who may benefit from interventions [25], implementation of substance use screening and interventions within primary care has been poor [30,32] due to complex barriers. First, PCPs have limited time per patient visit, and even brief screenings can be time consuming [33,34]. Second, some PCPs worry about their ability to manage positive screening results – a concern compounded by a lack of referral resources in most communities [35]. Third, stigma is a major barrier. PCPs’ attitudes towards SUDs are generally negative [36,37], with many providers believing that addiction is the fault of the individual or that using pharmaceutical interventions is akin to replacing one substance with another [35], and few PCPs are comfortable with prescribing medications for SUD to adolescents [38,39]. Furthermore, while it has become routine practice for PCPs to treat other behavioral health conditions (e.g., depression, attention-deficit/hyperactivity disorder), many PCPs view SUD treatment as outside their scope of practice.

Considering these challenges, tailored implementation strategies that address stigma and increase supports for PCPs are needed to improve the delivery and sustainability of SUD services [26]. In response, we developed a multifaceted integrated behavioral health (IBH) intervention (i.e., assessments/behavioral interventions/stigma reduction activities) to change the cultural and procedural norms of primary care settings regarding adolescent SUD treatment. The purpose of this study is to assess the impact of an IBH model on PCP behaviors surrounding adolescent SUD interventions and related youth service engagement outcomes. We will conduct a Hybrid Type 2, effectiveness-implementation, cluster-randomized stepped-wedge trial to compare the effects of the IBH model on PCP willingness to engage in adolescent SUD treatment. We hypothesize that, compared to standard primary care practices, the availability of support for assessment, behavioral interventions, and care management for SUD screening and treatment, coupled with formal stigma reduction activities, will motivate PCPs to engage in 1) SUD screening; 2) SUD referral; and 3) SUD treatment. We also will assess barriers and facilitators of implementing the IBH model. Findings from this study may inform efforts to enhance the workforce capacity and willingness to deliver SUD services for adolescents, thereby increasing access to care.

2. Materials and methods

2.1. Study design

This is a Hybrid Type 2, cluster-randomized, stepped-wedge trial comparing SUD IBH to standard primary care treatment. We will use an open cohort stepped-wedge design, where primary care sites will be randomly designated to one of three cohorts. We will perform stratified randomization based on the number of staff members per site (small vs large) to ensure that practice sizes are distributed across cohorts. Cohorts will transition from the pre-implementation phase to the SUD IBH phase every six months until all cohorts are receiving the intervention.

The stepped-wedge design was selected to maximize internal validity while operating within pragmatic constraints [4042]. Given that this is a pragmatic study within a real-world health system, the use of a stepped-wedge trial offers ethical, logistical, and statistical benefits over a conventional two condition trial. Ethically, access to SUD treatment for youth is currently limited and the SUD IBH interventions have significant potential to improve care delivery and patient outcomes; therefore, it could be considered unethical to withhold access to these interventions for the full length of the trial. By using a stepped-wedge approach, all primary care sites will ultimately receive access to the SUD IBH. Logistically, the stepped-wedge design allows us to roll out the interventions in steps to cohorts. Statistically, the cluster-randomized stepped-wedge trial is less sensitive than the parallel-arm cluster-randomized trial to the impact of a positive within-cluster intra-class correlation coefficient (ICC); consequently, the stepped-wedge design is much more powerful than the parallel-arm design for cluster-randomized trails where there are a small number of clusters, such as less than 20 clusters, which describes our scenario. Additionally, in contrast to a simple pre-post design, the stepped-wedge design allows trialists to control for potentially confounding temporal trends by adjusting for calendar time and using within- and between-cluster information. This study was approved by [MASKED] and is registered with Clinicaltrials.gov, trial number [MASKED].

2.2. Using integrated behavioral health to treat SUDs in primary care

System-level changes are needed in primary care settings to promote effective, sustainable integration of SUD care. Our multifaceted IBH model is designed to support PCPs in providing SUD treatment through four bundled components: 1) systematic screening; 2) clinical consultations; 3) provision of brief behavioral interventions and 4) stigma reduction efforts. We will integrate these interventions into primary care settings and assess their implementation and effectiveness. Specifically, we will:

  1. implement a brief assessment tool for adolescent substance use at new patient and well-child visits for individuals 12–17 years old;

  2. measure the integration of an addiction consultation model for healthcare providers designed to answer questions from clinicians related to substance use assessment, treatment, and referrals;

  3. test the feasibility and efficacy of a MI-based intervention provided by bachelor’s level interventionists who are integrated in the primary care setting;

  4. test the feasibility and efficacy of a network-based engagement intervention to improve PCPs’ willingness and comfort in treating adolescent SUDs.

These bundled interventions will build on existing infrastructure established by statewide programs, namely addiction-focused provider-to-provider consultation services [43] and an established adolescent SUD-focused treatment clinic. Next, we describe the bundled interventions in the context of this existing infrastructure.

2.2.1. Universal SUD screening

Adolescents with SUDs are unlikely to receive evidence-based services if they are not identified as needing treatment. Hence, our IBH model will implement systematic, universal screening using standardized protocols to identify adolescents with (1) no substance use or SUD symptoms for whom ongoing monitoring but no intervention is indicated, (2) mild SUD symptoms who may benefit from brief interventions, and (3) more severe symptoms who may need referral to more intensive services. We will develop a computer decision support system (CDSS), including automated distribution and prompting within the electronic health record (EHR). Two specific tools will be used. First, the Kiddie Computerized Adaptive Test for Substance Use Disorder Expanded (K-CAT-SUD-E) will be electronically administered to patients prior to well-child visits [44]. The K-CAT-SUD-E is an adaptive SUD severity scale that yields substance-specific SUD diagnostic data. If this screener is not completed prior to the appointment, it may also be administered directly by the PCP. Alternatively, PCPs may use the Brief Screener for Tobacco, Alcohol, and Other Drugs (BSTAD), an ultra-brief screener that asks youth about their recent patterns of substance use, to classify youth as being at elevated risk for SUDs or not per published cutoffs [45]. The BSTAD will be programmed into the EHR, and patients will complete the screener using a web-enabled device or as a paper form. If an item on the BSTAD is endorsed, this will trigger the fuller assessment using the K-CAT-SUD-E. [44] Screening results will prompt an automated flag to indicate a need for follow-up by the PCP or our team.

2.2.2. SUD consultations and care coordination

When indicated by screening results or clinician judgement, additional clinical assessments may be conducted and medication may be started in consultation with the Indiana Adolescent Addiction Access (AAA) program, a statewide provider-to-provider consultation line we launched in 2021 [43]. With consultation support from on-call specialists, like other pediatric mental healthcare access programs, PCPs can help fill the gap in behavioral health specialist services by delivering treatment themselves, including pharmaceutical interventions when appropriate. PCPs and other healthcare providers can call AAA and receive immediate access to advice from an expert clinician (i.e., adolescent psychiatrist, addiction psychologist, etc.), connections to additional resources (i.e., naloxone, education around fentanyl), and referrals for timely, direct patient care. This program will serve as an initial entry point for PCPs seeking support on providing SUD treatment. Based on AAA utilization data to date, we expect calls will often focus on medication management (e.g., nicotine replacement therapy, buprenorphine), and that utilizing the call line may help mitigate PCPs’ reluctance to prescribe treatments for adolescent SUDs. When needed, PCPs may also refer adolescents to our Adolescent Dual Diagnosis Clinic, an outpatient assessment and treatment program that specializes in treating SUDs and co-occurring psychiatric disorders via in-person or virtual sessions, facilitating our ability to provide treatment to a broad geographic area.

2.2.3. Provision of brief interventions

Given the shortage of SUD-focused clinicians, particularly for adolescents, [46] it is difficult to meet the demand for services. While behavioral health therapists and counselors may be intuitive providers for brief interventions, licensed therapists are not readily available in many areas, particularly in rural communities. However, since most brief interventions are highly structured and manual-standardized, staff with limited behavioral healthcare backgrounds can successfully be trained to deliver brief interventions with supervision [e.g., [4749]]. Using interventionists trained at the bachelor’s level, rather than requiring master’s or doctoral level training and licensure, provides a scalable approach to service provision [47]. By embedding bachelor’s level interventionists with specialized SUD training and supervision into primary care clinics and shifting this task to them, we will expand the workforce and make it possible for clinics to treat more adolescents while preserving time of PCPs and SUD specialists for the management of more complex and severe conditions. This is especially valuable since most adolescents with SUDs have disorders that are mild in severity. We will train these interventionists to deliver Teen Intervene (TI), a brief, three-session intervention shown to be effective in reducing adolescent substance use [50]. TI interventionists help adolescents identify their reasons for using substances, examine the impact of substances on their lives, and learn how to make healthier choices. While TI does not explicitly include overdose education, the risk of overdose and education around fentanyl will frequently be covered in these sessions. TI is ideal for trained bachelor-level staff to deliver because of its brevity, highly structured protocol, and relevance to commonly used substances. Under the supervision of licensed clinicians, the interventionists also serve in a triage role to identify and refer youth in need of more intensive treatment than TI.

2.2.4. Promoting engagement among healthcare professionals

We speculate that part of the resistance PCPs display towards SUD treatment is due to discomfort in treating youth with SUDs [32]. Thus, as part of this IBH model, in addition to the patient-facing services we will implement, we will also implement an adapted version of Game Changers directed towards health professionals working in primary care clinics. Game Changers is an individual-level peer advocacy intervention that draws on evidence-based network-driven interventions and social influence theories to increase certain behaviors [5154]. We have adapted Game Changers for the primary care setting to promote a cultural shift towards SUD treatment. Instead of the original 8, one-hour sessions [55,56], given the limited time of PCPs, we will implement a 2-session model (one hour each or less) with an optional third ‘booster’ session, based on input from early adopters. In these sessions, we will address coping with fears and personal reservations, stigma reduction, sharing of experience, management of risk, knowledge transfer, mutual support, and advocacy skills. We will recruit PCPs and office staff (RNs, MAs, administrative staff, etc.) identified as “champions” within their clinics to participate. Game changers will be delivered in brief group-based workshops, with occasional brief (15 min) check-in sessions throughout the implementation period. We anticipate that champions within the practice will first spread encouraging views of adolescent SUD care throughout the practice and then engage in similar advocacy with colleagues in other practices over time.

2.3. Study setting

In 2023, the State of Indiana and Riley Children’s Foundation partnered to support a large-scale expansion of a pediatric IBH program to at least 18 of IU Health’s primary care sites over 4+ years. The number of sites has increased to 28, and leadership at all sites has been made aware of the potential for expanded SUD services built upon the IBH infrastructure. The IBH program used across sites will follow the same model, all sites will have the general pediatric IBH program in place for at least 6 months prior to initiating SUD services, and the rollout of SUD services will follow the same cohort assignments.

Additionally, since 2011, the Adolescent Dual Diagnosis Clinic in the Riley Children’s Health Outpatient Child and Adolescent Psychiatry Clinic has delivered ENCOMPASS, an integrative SUD treatment model [57]. The ENCOMPASS model entails a 12-to 16-week course of outpatient treatment that weaves together comprehensive diagnostic assessment, MI, cognitive behavioral therapy, contingency management, pharmacotherapy, and family therapy, all managed via a structured, measurement-based care approach. The Riley ENCOMPASS team includes psychiatrists, psychologists, clinical social workers, nurse practitioners, care coordinators, and graduate and postgraduate learners. Here, the Adolescent Dual Diagnoses Clinic will be the main service for treatment of adolescents with moderate to severe SUDs that are deemed appropriate for outpatient management.

2.4. Data collection and key outcomes

We will collect quantitative and qualitative data to evaluate both implementation and effectiveness outcomes. Given our hypotheses that the IBH initiative will motivate PCPs to engage in 1) SUD screening; 2) SUD referral; and 3) SUD treatment, our primary outcomes of interest assess the impact of IBH on primary care provider behaviors (e.g., screening, referral, and prescribing behaviors) around integrated SUD treatment. All cohorts will complete baseline surveys during the control condition and then every 6 months until all cohorts are receiving the intervention, and then every year during the extended intervention periods when all cohorts are receiving the intervention. We will continue to assess outcomes for at least 18 months after the last cohort enters the implementation phase. We will obtain consent prior to any human subjects activity with each potential study participant from the research sites. All potential participants will receive information about the nature, risks, and benefits of the research so that they can make an informed decision to consent to participate or decline involvement in the study. Before completing any surveys or interviews, participants will sign a consent form. At each time point, we will also collect and analyze patient administrative data from the EHR system and related data sources (described in Table 1). We will conduct qualitative interviews with PCPs and leaders from each clinic at pre-, mid-, and post-implementation during sustainment of the intervention.

Table 1.

Outcome Measures and Data Source.

DOMAIN MEASURE DEFINITION DATA SOURCE(S)
Surveys SIC Tracker EHR State AAA Consult Line

Implementation Outcomes Primary Care Team Dynamics Survey [61] 31-items; dynamics w/i PCC teams X
Implementation Climate Scale [62] 18-items; org. Climate’s support of adopting new innovations X
Attribution Questionnaire – SUD [63] 24-items; PCC team adolescent SUD stigma X
Attitudes of MOUD Treatment [64] 15-items; attitudes towards MOUD treatment for adolescents w/ OUD X
Systems Usability Scale [65] 10-items; EHR usability X
Implementation Process Stages of Implement. Completion (SIC) Fidelity to implementation process completed for PCC by researchers X
Primary Effectiveness Screening Rates Percent of youth screened for SU X
Referral Rates Percent of youth referred to SUD treatment X
Brief SUD Treatment Percent of indicated youth who received brief SUD X
Engagement treatment (e.g., Teen Intervene)
Comprehensive SUD Treatment Percent of indicated youth who received comprehensive X
Engagement SUD treatment (e.g., ENCOMPASS)
MOUD Prescriptions Prescription fills of MOUD to adolescents X
P2P Consultation Calls Number of P2P Consultation Calls X
Secondary Nonfatal overdose EMS and ED records indicating OD X X
Effectiveness Fatal overdose Accidental OD death, vital records X X

Notes. Surveys are repeated every 6 months prior to implementation and every year after; EHR = electronic health record from Indiana University Health (IUH); To capture more complete overdose data than would be available in the IUH EHR, we will link IUH EHR data to statewide emergency medical services (EMS) and emergency departments (EDs). Data will be provided from the Indiana Adolescent Addiction Access (AAA) consultation line.

2.4.1. Implementation outcomes

We will use the Exploration, Preparation, Implementation, Sustainment (EPIS) framework to guide and evaluate the implementation of the SUD IBH program [58]. EPIS is a widely used framework that proposes an interplay between multiple contextual levels and factors theorized to influence each phase of the implementation process. We will collect implementation data from PCPs, supporting team members, and key leadership from all participating sites on organizational readiness for change, implementation climate, collaboration and dynamics within the primary care team, adolescent SUD stigma, and attitudes towards MOUD for adolescents with OUD. For quantitative data, an estimated 9 primary care team members and at least 1 system leader per site will complete baseline surveys (Table 1). Surveys will be repeated every 6 months prior to implementation and every year thereafter for use in the stepped-wedge analysis. We will collect qualitative data through semi-structured, 1-h interviews conducted with 2 participants per site (1 PCP, 1 system leader). Interviews will be digitally recorded and transcribed. Additionally, we will use the Stages of Implementation Completion (SIC) assessment tool to measure implementation process fidelity and assess implementation success [59]. The SIC is an eight-stage tool of implementation process – specifically, implementation activity completion and milestones that map onto the EPIS frameworks’ phases of implementation (pre-implementation [i.e., exploration and planning], implementation, and sustainment). The SIC has demonstrated reliable and valid measurement of the proportion and duration of implementation activity completion [60].

2.4.2. Effectiveness outcomes

We will collect effectiveness outcomes data reflecting provider behavior, patient treatment engagement, and patient outcomes. These data will be derived primarily from electronic health records, with key metric reports generated biannually. More specifically, we will assess PCP behavior by measuring rates of SUD screening as the percent of patients aged 12–17 with documented K-CAT-SUD-E results; referral rates to a behavioral health provider as the percent of patients aged 12–17 with a K-CAT-SUD-E indicative of likely SUD who are referred to SUD treatment; the overall number and percent of patients who receive a prescription medication for opioid use disorder; and the number of AAA consultation calls from participating sites. To assess patient engagement in services, we will measure engagement rates as the percent of patients aged 12–17 with mild SUD based on K-CAT-SUD-E scores and/or other diagnostic information who engage in at least one brief SUD treatment session with a bachelor’s trained interventionist (i.e., engagement in brief treatment); and the percent of patients aged 12–17 with moderate to severe SUD who engage in at least one session of outpatient SUD treatment through ENCOMPASS (i.e., engagement in comprehensive treatment). Finally, we will examine nonfatal and fatal overdose rates among patients in participating practices, stratified by substance type.

2.5. Analyses

2.5.1. Qualitative interview analysis

Following established methodology, we will analyze interviews using an inductive, interpretive approach based on grounded theory [66,67]. A trajectory approach will assess how implementation changes across the study period [68,69]. Through an iterative, consensus-building process, we will review the transcripts to identify themes; independently read transcripts (open coding) and discuss findings; coding transcripts independently; and meet to reach consensus (focused coding across individuals and time; i.e., to identify patterns or themes within the sample and how those patterns may change or evolve). If new or inconsistent data emerge, we will refine our coding scheme as needed.

2.5.2. Quantitative data analysis

We will use the standard analysis for a cluster-randomized stepped-wedge design that preserves internal validity through an intervention effectiveness test from a generalized linear mixed effects model (GLMM) that adjusts for potential confounding of temporal trends by using within- and between-cluster information. Calendar time will be specified as a fixed effect to control potentially confounding temporal trends. The linear link and normal error distribution will be used for continuous outcomes and a logit link and binomial error distribution will be used for dichotomous outcomes. If the linearity assumption for continuous outcomes is violated, we will explore polynomial terms, transformations, or nonlinear models, as needed.

The standard two-condition stepped-wedge comparison will be a GLMM test of usual care (control condition) versus intervention (implementation condition). In a secondary analysis, we will use indicator variables in the GLMM to explore whether the changes observed between control and intervention periods are maintained during the extended intervention periods. Random effects will be used in GLMM to account for potential within-site correlation of patient outcomes and within-participant (patients or providers) correlation of repeated follow-up measurements.

2.5.3. Mixed methods analysis

Utilizing NIH guidelines for mixed-methods research and prior work [7072], we will collect the quantitative and qualitative data simultaneously (QUAN + QUAL) with the functions of convergence and complementarity [7375]. Specifically, we will triangulate the quantitative and qualitative alliance data to assess for data convergence and use complementarity to assess implementation processes. Implementation outcomes will be compared across all sites. As measured by the Stages of Implementation Completion (SIC), primary care teams will be examined for quality of implementation process completion related to the final period achieved. Unlike many other implementations measured by the SIC where the decision to continue a program is optional for organizations, the chance for PCC sites to discontinue is unlikely. Instead, long term sustainment of outcomes will be measured by the maintenance of fidelity following achievement of competency.

Lastly, we will use Coincidence Analysis (CNA) to further integrate and analyze the qualitative and quantitative data together and identify difference-making conditions for both implementation and effectiveness outcomes. CNA is a leading-edge method to analyze how conditions directly link to outcomes and provides a numerical, case-oriented approach that uses applied set theory and Boolean algebra to examine multifactorial causality (i.e., when several conditions together have a joint effect on an outcome). Using CNA, researchers can identify what makes a difference, for whom and under what conditions. Each site will be categorized as having either high or low SUD treatment propensity (outcome) based on the proportion of patients with SUD receiving ENCOMPASS for more severe SUD and for youth with an OUD that receive MOUD in the last 6 months of the implementation period. Applying CNA will allow us to identify the key difference-makers that account for high versus low SUD treatment propensity at the site level.

2.5.4. Power analysis

Power was calculated with PASS software. The PASS stepped-wedge power calculation was available only for the cross-sectional design (assumes different people in each period), but results are conservative for our open cohort design where some persons supply repeated measures. Power was calculated based on a sample of 18 sites (i.e., clusters) in a custom stepped-wedge cluster-randomized design with 6 time periods (including the baseline), 5 steps, and 6 clusters (i.e., sites) per three cohorts. A cohort switches from control to treatment at each step. With an average of 36 participants per cluster, and an average of 6 participants per cluster per time period (for a total sample size of 648 participants), this design will achieve sufficient power for comparing intervention versus control conditions. Specifically, we will achieve 87 % power for comparing continuous outcomes using the model-based two-sided Wald Z-Test to detect a ½ SD difference between means, and 80 % power for comparing dichotomous outcomes using the model-based two-sided Wald Z-Test to detect a difference between proportions of 0.50 versus 0.72, assuming significance level of 0.05. These calculations are conservative for several reasons. First, we anticipate 6–10 providers to provide survey interview data (but assumed 6 persons) per site per time period. Second, we assumed a moderate ICC of 0.10 instead of a small ICC of 0.01. Third, the stepped-wedge design and analysis is relatively insensitive to ICC magnitude. Fourth, we anticipate enrolling 28 sites instead of 18 sites. Power will be even greater for administrative data where the anticipated sample size is larger. We estimate administrative data will be available for 39,658 unduplicated patients aged 12–17 during the data collection period based on the unduplicated number of patients seen at the 18 identified clinic sites between 2022 and 2024. Furthermore, an estimated 17 % of patients in pediatric primary care report substance use. In the National Survey on Drug Use and Health (NSDUH), 8.5 % of youth aged 12–17 self-reported one or more DSM-5 SUDs, of which 19.7 % of them reported prescription opioid use and 3.5 % reported misuse [17]. Thus, the power calculation above using 0.50 baseline prevalence for dichotomous outcomes is another conservative element in those calculations, given that proportions near 0 or 1 are more precise and require smaller sample sizes to achieve the same power. Therefore, power will exceed 99 % to detect the effect sizes described above using the administrative data.

3. Discussion

This Hybrid Type 2, effectiveness-implementation, cluster-randomized stepped-wedge trial will provide insight into the impact of an IBH model on PCP behaviors around adolescent SUD treatment. By implementing universal screening, improving coordination of care, shifting treatment provision to brief interventionists, and addressing stigma, we aim to foster systems-level changes that have the potential to re-shape the primary care process and, ultimately, prevent overdoses among adolescents. This study targets key individual-, organizational-, system-, and macro-level dynamics that underpin barriers to wide dissemination of adolescent integrated care-based SUD treatment. If successful, several components of the SUD IBH model (e.g., CDSS tools; stigma reduction intervention) can be standardized and packaged for broad dissemination. Our findings will yield valuable information about scalable strategies that health systems nationwide could adopt to improve care for adolescents with SUDs.

Importantly, our study is subject to several limitations and potential challenges, including intervention contamination, intervention burden, and generalizability. It is possible that sites where the IBH model has been initiated may communicate with sites that are still in the pre-implementation phase, which may dilute the benefit of IBH versus standard treatment if providers at other sites change their behavior based on what they learn from peers. To assess contamination and spillover effects, we will monitor key outcomes. It is also possible that the requirements of the IBH initiative may be too burdensome for some sites, resulting in drop out. We have built strong relationships with system leadership through our pilot work, which may mitigate potential drop out. Finally, our health system is uniquely situated to carry out the proposed study, whereas other health systems may not have an IBH approach in place at all. To aid generalizability, we include multiple sites across a spectrum of counties including rural, suburban, and urban sites.

4. Conclusions

Addressing the overdose crisis is a national priority. Currently, pediatric primary care practices largely ignore adolescent SUDs. IBH models have the potential to reduce overdose rates by enhancing PCP willingness to deliver SUD services for adolescents. Empirical evaluation of IBH will yield insights into the effectiveness and implementation of this model, informing future adaptations and scalability.

Acknowledgements

The authors are grateful to advisory board members, project staff, and students who have assisted with preparations for the study described here. We also acknowledge implementation partners at Indiana University Health and Riley Children’s Health.

Funding

This research was supported by the National Institutes of Health through the NIH HEAL Initiative under award numbers R61DA059948 and R33DA059948.

Footnotes

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

CRediT authorship contribution statement

Leslie A. Hulvershorn: Writing – review & editing, Supervision, Project administration, Methodology, Funding acquisition, Conceptualization. Matthew Aalsma: Writing – review & editing, Supervision, Project administration, Funding acquisition, Conceptualization. Trey V. Dellucci: Writing – original draft. Ashlyn Burns: Writing – original draft. Brigid R. Marriott: Writing – review & editing, Methodology, Conceptualization. Bernice Pescosolido: Writing – review & editing, Methodology, Conceptualization. Harold D. Green: Writing – review & editing, Methodology, Conceptualization. Lisa Saldana: Methodology, Conceptualization. Jason Chapman: Methodology, Conceptualization. Patrick Monahan: Writing – review & editing, Methodology, Conceptualization. Sarah E. Wiehe: Writing – review & editing, Methodology, Conceptualization. Edward J. Miech: Writing – review & editing, Methodology, Conceptualization. Zachary W. Adams: Writing – review & editing, Supervision, Project administration, Methodology, Funding acquisition, Conceptualization.

Data availability

No data was used for the research described in the article.

References

  • [1].Substance Abuse and Mental Health Services Administration, Strategic Plan 2023–2026. https://www.samhsa.gov/sites/default/files/samhsa-strategic-plan.pdf, 2024.
  • [2].Center for Disease Control and Prevention, Provisional Drug Overdose Death Counts [Online]. https://www.cdc.gov/nchs/nvss/vsrr/drug-overdose-data.htm, 2025.
  • [3].Babu K, What is Fentanyl and Why Is It Behind the Deadly Surge in US Drug Overdoses. https://www.umassmed.edu/news/news-archives/2022/05/what-is-fentanyl-and-why-is-it-behind-the-deadly-surge-in-us-drug-overdoses/, 2024. [Google Scholar]
  • [4].Friedman J, Godvin M, Shover CL, Gone JP, Hansen H, Schriger DL, Trends in drug overdose deaths among US adolescents, January 2010 to June 2021, Jama 327 (14) (2022) 1398–1400, 10.1001/jama.2022.2847. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [5].Hermans SP, Samiec J, Golec A, Trimble C, Teater J, Hall OT, Years of life lost to unintentional drug overdose rapidly rising in the adolescent population, 2016–2020, J. Adolesc. Health 72 (3) (2023) 397–403, 10.1016/j.jadohealth.2022.07.004. [DOI] [PubMed] [Google Scholar]
  • [6].Friedman J, Hadland SE, The overdose crisis among US adolescents, N. Engl. J. Med. 390 (2) (2024) 97–100, 10.1056/nejmp2312084. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [7].Ciccarone D, “the rise of illicit fentanyls, stimulants and the fourth wave of the opioid overdose crisis,” (in eng), Curr. Opin. Psychiatry 34 (4) (2021) 344–350, 10.1097/yco.0000000000000717. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [8].Green TC, Gilbert M, Counterfeit medications and fentanyl, JAMA Intern. Med. 176 (10) (2016) 1555–1557, 10.1001/jamainternmed.2016.4310. [DOI] [PubMed] [Google Scholar]
  • [9].Tarentino III FA. “Trafficker-Quantities of Rainbow Fentanyl Arrive in New York.” United States Drug Enforcement Adminstration. https://www.dea.gov/press-releases/2022/10/04/trafficker-quantities-rainbow-fentanyl-arrive-new-york (accessed Octobe 29, 2024. [Google Scholar]
  • [10].Center for Disease Control and Prevention, Most Reported Substance Use Among Adolescents Held Steady in 2022. https://nida.nih.gov/news-events/news-releases/2022/12/most-reported-substance-use-among-adolescents-held-steady-in-2022, 2024.
  • [11].Yule AM, et al. , Risk factors for overdose in treatment-seeking youth with substance use disorders, J. Clin. Psychiatry 79 (3) (2018) 18431, 10.4088/jcp.17m11678. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [12].Ochoa KC, Davidson PJ, Evans JL, Hahn JA, Page-Shafer K, Moss AR, Heroin overdose among young injection drug users in San Francisco, Drug Alcohol Depend. 80 (3) (2005) 297–302, 10.1016/j.drugalcdep.2005.04.012. [DOI] [PubMed] [Google Scholar]
  • [13].Sherman SG, Cheng Y, Kral AH, Prevalence and correlates of opiate overdose among young injection drug users in a large US city, Drug Alcohol Depend. 88 (2–3) (2007) 182–187, 10.1016/j.drugalcdep.2006.10.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [14].Werb D, Kerr T, Lai C, Montaner J, Wood E, Nonfatal overdose among a cohort of street-involved youth, J. Adolesc. Health 42 (3) (2008) 303–306, 10.1016/j.jadohealth.2007.09.021. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [15].Silva K, Schrager SM, Kecojevic A, Lankenau SE, Factors associated with history of non-fatal overdose among young nonmedical users of prescription drugs, Drug Alcohol Depend. 128 (1–2) (2013) 104–110, 10.1016/j.drugalcdep.2012.08.014. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [16].Calvo M, et al. , Young people who use drugs engaged in harm reduction programs in new York City: overdose and other risks, Drug Alcohol Depend. 178 (2017) 106–114, 10.1016/j.drugalcdep.2017.04.032. [DOI] [PubMed] [Google Scholar]
  • [17].Substance Abuse and Mental Health Services Administration, “Key Substance Use and Mental Health Indicators in the United States: Results From the 2022 National Survey on Drug Use and Health (HHS Publication No.PEP23–07-01–006, NSDUH Series H-58),” Center for Behavioral Health Statistics and Quality, Substance Abuse and Mental Health Services Administration, 2023. [Online]. Available: https://www.samhsa.gov/data/sites/default/files/reports/rpt42731/2022-nsduh-nnr.pdf. [Google Scholar]
  • [18].Grant BF, et al. , “epidemiology of DSM-5 alcohol use disorder: results from the National Epidemiologic Survey on alcohol and related conditions III,” (in eng), JAMA Psychiatry 72 (8) (2015) 757–766, 10.1001/jamapsychiatry.2015.0584. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [19].Tai B, Wu LT, Clark HW, “electronic health records: essential tools in integrating substance abuse treatment with primary care,” (in eng), Subst. Abus. Rehabil. 3 (2012) 1–8, 10.2147/sar.S22575. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [20].E. National Academies of Sciences, Medicine, Health, D. Medicine, P. Board on Health Sciences, and D. Committee on Medication-Assisted Treatment for Opioid Use, The National Academies Collection: Reports funded by National Institutes of Health, in: Mancher M, Leshner AI (Eds.), Medications for Opioid Use Disorder Save Lives, Washington (DC), National Academies Press (US) Copyright 2019 by the National Academy of Sciences, 2019. All rights reserved. [PubMed] [Google Scholar]
  • [21].National Institute on Alcohol Abuse Alcoholism, Alcohol Screening and Brief Intervention for Youth: A Practitioner’s Guide, National Institute on Alcohol Abuse and Alcoholism, US Department of Health, 2019. [Google Scholar]
  • [22].Steele DW et al. , “Brief behavioral interventions for substance use in adolescents: a meta-analysis,” (in eng), Pediatrics, 146, 2020, doi: 10.1542/peds.2020-0351. [DOI] [PubMed] [Google Scholar]
  • [23].Hadland SE, Burr WH, Zoucha K, Somberg CA, Camenga DR, Treating adolescent opioid use disorder in primary care, JAMA Pediatr. 178 (4) (2024) 414–416, 10.1001/jamapediatrics.2023.6493. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [24].Aldridge A, Linford R, Bray J, “substance use outcomes of patients served by a large US implementation of screening, brief intervention and referral to treatment (SBIRT),” (in eng), Addiction 112 (Suppl. 2) (2017) 43–53, 10.1111/add.13651. [DOI] [PubMed] [Google Scholar]
  • [25].Monico LB, et al. , “a comparison of screening practices for adolescents in primary care after implementation of screening, brief intervention, and referral to treatment,” (in eng), J. Adolesc. Health 65 (1) (2019) 46–50, 10.1016/j.jadohealth.2018.12.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [26].Thoele K, et al. , Strategies to promote the implementation of screening, brief intervention, and referral to treatment (SBIRT) in healthcare settings: a scoping review (in eng), Subst. Abuse Treat. Prev. Policy 16 (1) (2021) 42, 10.1186/s13011-021-00380-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [27].Substance Abuse and Mental Health Services Administration, Screening, Brief Intervention, and Referral to Treatment (SBIRT). https://www.samhsa.gov/sbirt, 2024.
  • [28].U. S. D. o. H. a. H. Services, QuickStats: Percentage of Children Aged <18 Years Who Received a Well-Child Checkup in the Past 12 Months, by Age Group and Year — National Health Interview Survey, United States, 2008 and 2018, MMWR Morb. Mortal Wkly. Rep. (2020), 10.15585/mmwr.mm6908a5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [29].Levy S, Fuller A, Kelly S, Lunstead J, Weitzman ER, Straus JH, A phone consultation call line to support SBIRT in pediatric primary care, Front. Psychol. 13 (2022) 882486, 10.3389/fpsyt.2022.882486. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [30].Hadland SE, et al. , Medication for adolescents and young adults with opioid use disorder, J. Adolesc. Health 68 (3) (2021) 632, 10.1016/j.jadohealth.2020.12.129. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [31].Ryan SA, Gonzalez PK, Patrick SW, Quigley J, Siqueira L, Walker LR, Medication-assisted treatment of adolescents with opioid use disorders, Pediatrics 138 (3) (2016). [DOI] [PubMed] [Google Scholar]
  • [32].Aalsma MC, et al. , Clinician willingness to prescribe medications for opioid use disorder to adolescents in Indiana, JAMA Netw. Open 7 (9) (2024) e2435416, 10.1001/jamanetworkopen.2024.35416. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [33].WHO ASSIST Working Group, The alcohol, smoking and substance involvement screening test (ASSIST): development, reliability and feasibility, Addiction 97 (9) (2002) 1183–1194, 10.1046/j.1360-0443.2002.00185.x. [DOI] [PubMed] [Google Scholar]
  • [34].Yudko E, Lozhkina O, Fouts A, A comprehensive review of the psychometric properties of the drug abuse screening test, J. Subst. Abus. Treat. 32 (2) (2007) 189–198, 10.1016/j.jsat.2006.08.002. [DOI] [PubMed] [Google Scholar]
  • [35].McNeely J, et al. , Barriers and facilitators affecting the implementation of substance use screening in primary care clinics: a qualitative study of patients, providers, and staff, Addict. Sci. Clin. Pract. 13 (1) (2018) 8, 10.1186/s13722-018-0110-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [36].Kennedy-Hendricks A, et al. , Primary care physicians’ perspectives on the prescription opioid epidemic, Drug Alcohol Depend. 165 (2016) 61–70, 10.1016/j.drugalcdep.2016.05.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [37].van Boekel LC, Brouwers EP, van Weeghel J, Garretsen HF, “comparing stigmatising attitudes towards people with substance use disorders between the general public, GPs, mental health and addiction specialists and clients,” (in eng), Int. J. Soc. Psychiatry 61 (6) (2015) 539–549, 10.1177/0020764014562051. [DOI] [PubMed] [Google Scholar]
  • [38].Bagley SM, Hadland SE, Carney BL, Saitz R, Addressing stigma in medication treatment of adolescents with opioid use disorder, J. Addict. Med. 11 (6) (2017) 415–416, 10.1097/adm.0000000000000348. [DOI] [PubMed] [Google Scholar]
  • [39].Rosenblatt RA, Andrilla CH, Catlin M, Larson EH, Geographic and specialty distribution of US physicians trained to treat opioid use disorder, Ann. Fam. Med. 13 (1) (2015) 23–26, 10.1370/afm.1735. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [40].Hemming K, Taljaard M, Reflection on modern methods: when is a stepped-wedge cluster randomized trial a good study design choice? Int. J. Epidemiol. 49 (3) (2020) 1043–1052, 10.1093/ije/dyaa077. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [41].Hemming K, Lilford R, Girling AJ, Stepped-wedge cluster randomised controlled trials: a generic framework including parallel and multiple-level designs, Stat. Med. 34 (2) (2015) 181–196, 10.1002/sim.6325. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [42].Hussey MA, Hughes JP, Design and analysis of stepped wedge cluster randomized trials, Contemp. Clin. Trials 28 (2) (2007) 182–191, 10.1016/j.cct.2006.05.007. [DOI] [PubMed] [Google Scholar]
  • [43].Adams ZW, et al. , A statewide consultation helpline for rapid linkage to services for youths with opioid use disorder and other substance use, Psychiatr. Serv. (2024) 20230289, 10.1176/appi.ps.20230289. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [44].Adams ZW, Hulvershorn LA, Smoker MP, Marriott BR, Aalsma MC, Gibbons RD, Initial validation of a computerized adaptive test for substance use disorder identification in adolescents, Subst. Use Misuse 59 (6) (2024) 867–873, 10.1080/10826084.2024.2305801. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [45].Kelly SM, Gryczynski J, Mitchell SG, Kirk A, O’Grady KE, Schwartz RP, Validity of brief screening instrument for adolescent tobacco, alcohol, and drug use, Pediatrics 133 (5) (2014) 819–826, 10.1542/peds.2013-2346d. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [46].Hadland SE, Wood E, Levy S, How the pediatric workforce can address the opioid crisis, Lancet (London, England) 388 (10051) (2016) 1260, 10.1016/s0140-6736(16)31573-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [47].Brown RL, et al. , A team approach to systematic behavioral screening and intervention, Am. J. Manag. Care 20 (4) (2014) e113. [PMC free article] [PubMed] [Google Scholar]
  • [48].Maslowsky J, Whelan Capell J, Moberg DP, Brown RL, Universal school-based implementation of screening brief intervention and referral to treatment to reduce and prevent alcohol, marijuana, tobacco, and other drug use: process and feasibility, Substan. Abuse 11 (2017) 1178221817746668, 10.1177/1178221817746668. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [49].Krupski A, et al. , Testing the effects of brief intervention in primary care for problem drug use in a randomized controlled trial: rationale, design, and methods, Addict. Sci. Clin. Pract. 7 (2012) 1–10, 10.1186/1940-0640-7-27. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [50].Winters KC, Lee S, Botzet A, Fahnhorst T, Nicholson A, One-year outcomes and mediators of a brief intervention for drug abusing adolescents, Psychol. Addict. Behav. 28 (2) (2014) 464, 10.1037/a0035041. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [51].Hunter RF, et al. , Social network interventions for health behaviours and outcomes: a systematic review and meta-analysis, PLoS Med. 16 (9) (2019) e1002890, 10.1371/journal.pmed.1002890. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [52].Latkin CA, Knowlton AR, Social network assessments and interventions for health behavior change: a critical review, Behav. Med. 41 (3) (2015) 90–97, 10.1080/08964289.2015.1034645. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [53].Valente TW, Network interventions, Science 337 (6090) (2012) 49–53. [DOI] [PubMed] [Google Scholar]
  • [54].Gesell SB, Barkin SL, Valente TW, Social network diagnostics: a tool for monitoring group interventions, Implement. Sci. 8 (2013) 1–12, 10.1186/1748-5908-8-116. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [55].Wagner GJ, et al. , Social network-based group intervention to promote HIV prevention in Uganda: study protocol for a cluster randomized controlled trial of game changers, Trials 23 (1) (2022) 233, 10.1186/s13063-022-06186-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [56].Wanyenze RK, et al. , Social network-based group intervention to promote uptake of cervical cancer screening in Uganda: study protocol for a pilot randomized controlled trial, Pilot Feasib. Stud. 8 (1) (2022) 247, 10.1186/s40814-022-01211-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [57].Riggs P, encompass: an integrated treatment intervention for adolescents with co-occurring psychiatric and substance use disorders, in: Scientific Proceedings of the American Academy of Child and Adolescent Psychiatry 61st Annual Meeting (AACAP). San Diego, 2014. [Google Scholar]
  • [58].Moullin JC, Dickson KS, Stadnick NA, Rabin B, Aarons GA, Systematic review of the exploration, preparation, implementation, sustainment (EPIS) framework, Implement. Sci. 14 (2019) 1–16. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [59].Saldana L, The stages of implementation completion for evidence-based practice: protocol for a mixed methods study, Implement. Sci. 9 (2014) 1–11, 10.1186/1748-5908-9-43. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [60].Brown CH, Chamberlain P, Saldana L, Padgett C, Wang W, Cruden G, Evaluation of two implementation strategies in 51 child county public service systems in two states: results of a cluster randomized head-to-head implementation trial, Implement. Sci. 9 (2014) 1–15, 10.1186/s13012-014-0134-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [61].Song H, et al. , Development and validation of the primary care team dynamics survey, Health Serv. Res. 50 (3) (2015) 897–921, 10.1111/1475-6773.12257. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [62].Ehrhart MG, Aarons GA, Farahnak LR, Assessing the organizational context for EBP implementation: the development and validity testing of the implementation climate scale (ICS), Implement. Sci. 9 (2014) 1–11, 10.1186/s13012-014-0157-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [63].Johnson-Kwochka A, Aalsma MC, Monahan PO, Salyers MP, Development and examination of the attribution questionnaire-substance use disorder (AQ-SUD) to measure public stigma towards adolescents experiencing substance use disorders, Drug Alcohol Depend. 221 (2021) 108600, 10.1016/j.drugalcdep.2021.108600. [DOI] [PubMed] [Google Scholar]
  • [64].Knudsen HK, Ducharme LJ, Roman PM, Link T, Buprenorphine diffusion: the attitudes of substance abuse treatment counselors, J. Subst. Abus. Treat. 29 (2) (2005) 95–106, 10.1016/j.jsat.2005.05.002. [DOI] [PubMed] [Google Scholar]
  • [65].Lewis JR, Sauro J, The factor structure of the system usability scale, in: Human Centered Design: First International Conference, HCD 2009, Held as Part of HCI International 2009, San Diego, CA, USA, July 19–24, 2009 Proceedings 1, Springer, 2009, pp. 94–103, 10.1007/978-3-642-02806-9_12. [DOI] [Google Scholar]
  • [66].Charmaz K, Constructing Grounded Theory: A Practical Guide Through Qualitative Analysis, Sage, 2006. [Google Scholar]
  • [67].Miles MB, Qualitative Data Analysis: An Expanded Sourcebook, Thousand Oaks, 1994. [Google Scholar]
  • [68].Grossoehme D, Lipstein E, Analyzing longitudinal qualitative data: the application of trajectory and recurrent cross-sectional approaches, BMC Res. Notes 9 (2016) 1–5, 10.1186/s13104-016-1954-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [69].Saldana J, Longitudinal Qualitative Research: Analyzing Change through Time, AltaMira Press, 2003. [Google Scholar]
  • [70].Creswell JW, Klassen AC, Plano Clark VL, Smith KC, “Best Practices for Mixed Methods Research in the Health Sciences,” Bethesda (Maryland) 2013, National Institutes of Health, 2011, pp. 541–545, 10.1037/e566732013-001. [DOI] [Google Scholar]
  • [71].Aalsma MC, Brown JR, Holloway ED, Ott MA, Connection to mental health care upon community reentry for detained youth: a qualitative study, BMC Public Health 14 (2014) 1–8, 10.1186/1471-2458-14-117. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [72].Salyers MP, Hood BJ, Schwartz K, Alexander AO, Aalsma MC, The experience, impact, and management of professional burnout among probation officers in juvenile justice settings, J. Offender Rehabil. 54 (3) (2015) 175–193, 10.1080/10509674.2015.1009967. [DOI] [Google Scholar]
  • [73].Onwuegbuzie A , A framework for analyzing data in mixed methods research, in: Handbook of Mixed Methods in Social & Behavioral Research, Sage, 2003. [Google Scholar]
  • [74].Bryman A, Integrating quantitative and qualitative research: how is it done? Qual. Res. 6 (1) (2006) 97–113, 10.1177/1468794106058877. [DOI] [Google Scholar]
  • [75].Tashakkori A, Creswell JW, The New Era of Mixed Methods vol. 1, Sage Publications, 2007, pp. 3–7. [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Data Availability Statement

No data was used for the research described in the article.

RESOURCES