Substance Use Screening, Brief Intervention, and Referral to Treatment in Pediatric Practice: A Quality Improvement Project in the Maryland Adolescent and Young Adult Health Collaborative Improvement and Innovation Network

Abstract

The American Academy of Pediatrics recommends pediatric providers routinely screen for, assess, and treat substance use and substance use disorders among adolescents, a process called “Screening, Brief Intervention, and Referral to Treatment,” or “SBIRT.” Because there are limited data on how Maryland pediatric practices have adopted SBIRT, a quality improvement initiative was developed within the Maryland Adolescent and Young Adult Health Collaborative Improvement and Innovation Network using a “Plan/Do/Study/Act” approach. A 2-part provider training was conducted regarding screening and motivational interviewing, and the “CRAFFT” screening tool was integrated into the practice’s electronic medical record. Results from evaluation demonstrated significant improvements in provider knowledge, attitudes, and screening behavior. The association between substance use and sexual behavior suggests a need for further expansion of this model with inclusion of sexual health screening. Overall, this study demonstrates that SBIRT implementation into a general pediatric practice is highly feasible, acceptable, and shows preliminary effectiveness.

Introduction

Alcohol and other drug use among adolescents is a major public health concern and is a leading cause of morbidity and mortality in this age group.¹ Adolescent substance use is prevalent, with nearly two thirds of high school students reporting having ever drank alcohol and one third reporting having used marijuana.¹ In addition to direct causes of death such as overdose, substance use contributes to a significant portion of unintentional injury deaths among adolescents.^2,3

The American Academy of Pediatrics (AAP) has recommended that pediatric providers routinely screen for, assess, and treat substance use and substance use disorders (SUD) among adolescents, a process called substance use “Screening, Brief Intervention, and Referral to Treatment,” or “SBIRT.”⁴ These recommendations were published in a 2011 policy statement,⁵ followed by updated guidelines in 2016 consisting of a paired clinical report⁴ and policy statement.⁶ Together, these documents provide instructions and guidance for developing SBIRT skills and implementation of SBIRT into pediatric practice.

SBIRT begins by screening for substance use with a validated tool such as “Screening to Brief Intervention” (S2BI) or “Brief Screener for Tobacco, Alcohol, and Other Drugs”; tools such as the “CRAFFT” can be used to further assess youth substance use and associated problematic behaviors.⁴ While these tools have been validated for use with adolescents and the scores correlate with the likelihood of having a diagnosable SUD, clinicians should perform further evaluation to identify individuals who meet Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) criteria⁷ for an SUD. Youth who screen positive should receive a “brief intervention” that consists of motivational interviewing and tailored advice “encouraging healthy choices so that the risk behaviors are prevented, reduced, or stopped.”⁴ For youth with significant substance use who meet DSM-5 criteria⁷ for a severe SUD, their brief intervention should also include a referral to treatment for their SUD.⁴

SBIRT is a flexible model that allows for screening via multiple modalities including by the clinician or via a self-administered computer-based screen.⁸ It has successfully been adapted for multiple settings ranging from the pediatric primary care practice, to community health centers, to school based clinics.^9–13 Unfortunately, despite the AAP recommendations and flexibility of the model, SBIRT implementation and utilization remains low among pediatric providers.^14,15 Common barriers to widespread implementation of SBIRT include provider lack of training and knowledge of validated screening tools, as well as organizational and efficiency concerns regarding workflow and electronic medical record (EMR) integration.^4,14 Quality improvement (QI) projects offer an opportunity for pediatric practices to learn the skills of SBIRT and tailor the model to fit their clinical needs and unique workflow by using one of the available implementation guides.^16–19

Methods

Development of the QI Project

This project was conducted as part of the Adolescent and Young Adult Health Collaborative Improvement and Innovation Network (AYAH-CoIIN), a national multistate initiative designed to improve adolescent preventive health services, sponsored by the Maternal and Child Health Bureau (MCHB) of the Health Resources and Services Administration (HRSA). The Maryland AYAH-CoIIN recognized that there are limited data on how pediatric practices in Maryland have integrated evidence-based practice models regarding substance use. Thus, this QI project was designed to evaluate the feasibility, acceptability, and effectiveness of an SBIRT training and implementation intervention to improve provider knowledge, confidence, adoption of a validated screening tool for adolescent substance use, delivery of a brief intervention, and screening for associated behaviors such as sexual activity. This study was approved by the Institutional Review Board of Johns Hopkins School of Medicine as well as the Maryland Department of Health.

Study Setting and Population

The Maryland AYAH-CoIIN chose to conduct this QI initiative at a pediatric group practice in a mid-sized metropolitan area with higher than national rates of substance use. The practice serves a diverse population of approximately 12,000 patients, and it has a high Medicaid payer mix. All 9 multidisciplinary pediatric providers employed by the clinic (physicians, nurse practitioners, and physician assistants) participated in the QI initiative. The practice used an electronic medical record (EMR) system, with an integrated online patient/family screening assessment website.

Study Design

A “Plan/Do/Study/Act” approach to QI was used for this project.²⁰ The planning stage began with a needs assessment of the pediatric practice to evaluate current practices, available resources, and provider preferences. Prior to the intervention, no standardized screening tool for substance use was being used, though the practice was using an online screening system for other standardized pediatric screening tools. The providers voiced interest in learning how to better engage in an open discussion with adolescents about their substance use, and in what to do if an adolescent screened positive for substance use. To maintain productivity, they wanted to learn to efficiently perform an intervention and connect adolescents and families to community resources.

The QI team used the evidence-based “CRAFFT” screening tool, which includes the “S2BI.” This tool was integrated into the practice’s EMR system to screen for substance use at all well adolescent visits for adolescents aged 13 to 20 years. Adolescents would be prompted to complete the CRAFFT screen via the online screening website prior to their well adolescent visit, or while waiting to be roomed on the day of their visit; if the adolescent did not complete the CRAFFT online, then medical staff would be prompted to administer the screen to the adolescents during the clinical visit. The information technology manager for the practice then built templates that could be pulled into providers’ progress notes that would describe the varying levels of brief intervention and referral to treatment based on the patient’s CRAFFT score.

The intervention began with a Phase 1 training for all providers, occurring during a 1-hour timeslot often utilized for practice staff meetings. The training provided an overview of adolescent substance use, as well as how to use the CRAFFT screening tool as part of SBIRT as described in the AAP Policy Statement⁵ and Clinical Report on SBIRT,⁴ and Massachusetts Department of Health SBIRT provider training manuals.^18,19 The training explained management of positive screens with a brief intervention, navigation of referral resources, and how to integrate this SBIRT process into a busy clinical practice. Providers were given a basic flowchart for how to respond to different levels of substance use, and they were introduced to the EMR tools described above.

As Phase 1 was underway, feedback was sought from the providers regarding overall feasibility and acceptability of adding SBIRT to their clinical practice, as well as specific successes and difficulties they were encountering. Logistical questions arose regarding billing and documentation. Providers were asked what they hoped to gain from future trainings, and they voiced a desire to continue to learn how to better navigate a positive screen and engage in change-generating talk with their patients. Performance measures were analyzed and used to guide development on the Phase 2 training.

A Phase 2 training was then held approximately 3 months later, again during a 1-hour timeslot utilized for practice staff meetings. This training shared and reflected on providers’ Phase 1 performance and answered the troubleshooting logistical questions that had arisen. Discussions were had regarding how to sustain this new workflow over time. The remainder of the session con- sisted of motivational interviewing training adapted from Miller and Rollnick’s Motivational Interviewing book,²¹ including examples of conversations with youth, mnemonics, and group participation exercises.

As Phase 2 was underway, again feedback was sought from providers regarding their successes and frustrations with the process. Overall feasibility and acceptability of the project were assessed through discussions with providers. Additional technical assistance was provided on an as-needed basis.

Data Collection

The metrics to evaluate the project’s effectiveness were designed to address 3 components: knowledge, attitudes, and behaviors. To assess knowledge and attitudes, a provider survey was developed and administered to all practice providers at 3 time points: prior to the Phase 1 training, after Phase 1, and after Phase 2. The survey included quizstyle questions on facts related to adolescent substance use and SBIRT designed to measure the providers’ knowledge across the project timeline. The survey also included Likert-type scale questions about their comfort with and confidence in their skills screening for substance use, and performing a brief intervention and motivational interviewing.

To assess clinician behaviors, a retrospective chart review was performed on a sample of 40 consecutive charts from well adolescent visits during a randomly selected week in each of 3 periods: prior to the intervention, at the end of Phase 1, and at the end of Phase 2. Process measures were chosen to quantify how often providers were screening for substance use via any method, screening for substance use with a validated tool such as the CRAFFT, performing brief interventions, and referring for treatment/follow-up. Related behaviors such as screening for sexual risk were also documented. Patient sociodemographic data were collected, as well as clinical characteristics including patients’ substance use and sexual activity history.

Statistical Analysis

For the provider survey, descriptive statistics were performed to calculate the means and standard deviations of the knowledge and confidence scores over time. Paired t tests were used to test for change in mean knowledge and confidence score from Pre to Post Phase 2. Box and whisker plots were generated to demonstrate the central tendencies and variation for the knowledge and confidence scores over time.

For the chart review, descriptive statistics were performed to estimate the distribution of patient characteristics included in the sample of chart reviews. Chi-square tests were used to compare physician screening and intervention behaviors between the 3 phases by comparing the number of patients who were screened for substance use (with any tool, and with validated tool), received a brief intervention, found to be using substances, screened for sexual activity, found to be sexually active, and screened for sexually transmitted infections. Simple linear regression was performed to calculate the percentage increase in provider screening behavior between phases. Simple logistic regression was performed to provide an odds ratio for sexual activity based on adolescent substance use.

Analyses were conducted using STATA IC Version 15 (StataCorp, College Station, TX). All statistical tests were 2-sided and considered significant at P < .05.

Results

Provider Knowledge and Attitudes

Provider surveys were collected from all 9 providers during each of the 3 study phases. Table 1 displays the mean knowledge and confidence scores, along with standard deviation, across the 3 study phases. Providers’ knowledge score significantly improved by a mean of 35.3 points (95% confidence interval [CI] = 17.7–52.9) from baseline to after Phase 2. Providers’ confidence score also significantly improved by a mean of 22.7 points (95% CI = 11.4–34.2) from baseline to after Phase 2. Figure 1 shows a box and whisker plot of the average knowledge score over study phases, and Figure 2 shows a box and whisker plot of the average confidence score over study phases.

Table 1.

Mean Knowledge and Confidence Scores (Scale 0–100) Over the 3 Phases.

	Pre, Mean (SD)	Post Phase 1, Mean (SD)	Post Phase 2, Mean (SD)
Total N per phase	9	9	9
Knowledgea	48.1 (23.6)	63.9 (22.8)	83.4 (18.6)
Confidencea	58.3 (15.4)	78.3 (14.6)	81.1 (8.2)

^aPaired t test P < .01 Pre to Post Phase 2.

Figure 1.

Average knowledge score over training phases.

Figure 2.

Average confidence score over training phases.

Provider Behavior and Patient Characteristics

Chart reviews were performed on 40 charts per each of the 3 phases, for a total of 120 patient charts. Table 2 displays characteristics of patients in the sample. The median age was 15 (interquartile range [IQR] = 14–17), most patients were white (71.7%), male (50.8%), and privately insured (50.0%).

Table 2.

Characteristics of Patients Included in the Chart Review.

	n (%)
Total, N	120
Age (median, IQR)	15 (14–17)
Race
White	86 (71.7%)
Black	14 (11.7%)
Other	20 (16.7%)
Sex
Male	61 (50.8%)
Female	59 (49.2%)
Insurance type
Medicaid	58 (48.3%)
Private	60 (50.0%)
Other	2 (1.7%)

Abbreviation: IQR, interquartile range.

Table 3 displays the change in provider substance use screening and intervention behavior over the 3 study phases, as well as the number of patients who reported using substances. The use of any screening method for substance use significantly increased by 15% (95% CI = 4.91% to 25.1%), from 85% at baseline to 100% after Phase 2. The use of a validated tool (such as the CRAFFT) to screen for substance use significantly increased by 97.5% (95% CI = 84.7% to 110%), from 0% at baseline to 97.5% after Phase 2. There was a nonsignificant trend toward increased reported substance use over the phases with 33% of individuals reporting substance use after Phase 2. The most commonly used substances were marijuana (n = 13) and alcohol (n = 11); less commonly used substances were tobacco, opioids, and benzodiazepines. There was also a nonsignificant trend toward increased provider provision of a brief intervention over the study phases, occurring in 25% of the time after Phase 2.

Table 3.

Provider Substance Use Screening and Intervention Behavior Over Study Phases, as Well as Number of Patients Reporting Substance Use.

Measure	Pre, n (%)	Post Phase 1, n (%)	Post Phase 2, n (%)	χ2 P
Total N per phase	40	40	40
Substance use screening
With any method	34 (85%)	39 (98%)	40 (100%)	.009
With validated tool	0 (0%)	27 (68%)	39 (98%)	<.001
Patients with positive screen/using substances	8 (20%)	10 (25%)	13 (33%)	.44
Brief intervention provided	6 (15%)	2 (5%)	10 (25%)	.043

Table 4 displays the change in provider sexual activity screening and intervention behavior over the 3 study phases, as well as the number of patients who reported being sexually active. There was no change over study phases in the rate of screening for sexual activity, screening for sexually transmitted infections, or the number of youth who reported being sexually active (P > .05 for all). Logistic regression demonstrated that youth who reported using substances were 4.37 (95% CI = 1.69–11.27) times more likely to report being sexually active than youth not using substances.

Table 4.

Provider Sexual Activity Screening and Intervention Behavior Over Study Phases, as Well as Number of Patients Reporting Sexual Activity.

Measure	Pre, n (%)	Post Phase 1, n (%)	Post Phase 2, n (%)	χ2 P
Sexual activity screening	31 (78%)	29 (73%)	35 (88%)	.24
Sexually active patients	6 (19%)	12 (41%)	10 (29%)	.17
STI screening among sexually active patients	4 (67%)	9 (69%)	6 (60%)	.90

Abbreviation: STI, sexually transmitted infection.

Discussion

In this study of one pediatric practice’s experience with an SBIRT QI initiative, implementation of this model was achievable and resulted in high provider satisfaction. Results from the evaluation metrics demonstrated significant improvements in all 3 domains measured: knowledge, attitudes, and screening behavior. This study contributes several key findings.

First, this study demonstrated the feasibility and acceptability of implementing SBIRT into a community pediatric practice guided by previously published SBIRT implementation guides. The intervention required a relatively small time commitment from providers in the practice, the ease of which was facilitated by conducting the training intervention during routine staff meetings.

Because the screening tool was integrated into the electronic health record, providers were easily able to adjust their clinical flow without burdensome extra tasks or documentation. The successful uptake of SBIRT by providers was, in part, due to the support for the project by practice leadership, who engaged providers by showing how the change would improve their clinical practice.

Second, this study showed that an initial needs assessment was critical in designing a tailored intervention that providers would find useful. Similar to what the literature demonstrates, pediatric providers in this study cited their discomfort in discussing substance use and responding to a positive screen as key barriers to performing regular substance use screening. The intervention was thus designed to include substantial training in not only how to screen for substance use, but also what follow-up questions to ask, as well as how to engage in motivational interviewing. Because of this, providers’ confidence in their skills significantly improved after just 1 training session. The second training session further enhanced their knowledge, while their confidence remained high.

Third, this project demonstrated a link between substance use and sexual health, with youth using substances being more than 4 times more likely to report sexual activity than youth who were substance-free. This association between substance use and sexual activity has been well documented in the literature,^22–24 including among adolescents.^25–27 Despite this association, there are limited programs, to our knowledge, that have focused on this link by combining SBIRT and sexual health screening.^28–30 The New York City Health Department was able to integrate SBIRT services into their public sexually transmitted diseases clinics throughout the city with the support of both federal and state funding; they demonstrated successful SBIRT implementation as well as improved outcomes in both sexual health and substance use behaviors.^28–30 There remains a need for the development of smaller scale programs integrating the two that can be used in general pediatric practice, as this study demonstrates that standard SBIRT training does not lead to changes in provider sexual health screening behavior.

There were several limitations of this study. First, the SBIRT initiative was limited to one pediatric practice in a mid-size metropolitan area, and thus may have limited generalizability. Second, the metrics assessed were largely process measures, as measuring longer term out- comes for patients was beyond the scope and capability of this project; thus, the study was unable to demonstrate whether increased substance use screening led to changes in patient health outcomes. Last, there was no patient or parent input or feedback in this project, and feedback could be helpful in assessing patient satisfaction with their clinical care.

Conclusion

This study demonstrates that SBIRT implementation into a general pediatric practice is highly feasible, acceptable, and shows preliminary effectiveness. A successful intervention is dependent on support from practice leadership that leads to buy-in among other providers. Additionally, the intervention was streamlined by integrating it into preexisting platforms for provider communication and service delivery, such as holding training sessions during routine staff meetings and incorporating screening tools into the electronic medical record. Given the association between substance use and sexual behavior, further expansion of this model with a module focused on sexual health screening and this link is warranted. Overall, this study models how a general pediatric practice can successfully and efficiently incorporate substance use SBIRT into routine clinical practice.