Impact of a Multi-Component SBIRT Training Curriculum on a Medical Residency Program

Abstract

Background

Substance-related disorders are a growing problem in the United States. The patient-provider setting can serve as a crucial environment to detect and prevent at-risk substance use. Screening, brief intervention, and referral to treatment (SBIRT) is an integrated approach to deliver early intervention and treatment services for persons who have or are at-risk for substance related disorders. SBIRT training components can include online modules, in-person instruction, practical experience, and clinical skills assessment. This paper will evaluate the impact of multiple modes of training on acquisition of SBIRT skills as observed in a clinical skills assessment.

Methods

Residents were part of an SBIRT training program, from 2009 through 2013, consisting of lecture, role play, online modules, patient encounters, and clinical skills assessment (CSA). Differences were assessed across satisfactory and unsatisfactory CSA performance.

Results

70% of the residents satisfactorily completed CSA. Demographics, type of components completed, and number of components completed were similar among residents that demonstrated satisfactory clinical skills compared to those that did not. All components of the training program were accepted equally across specialties and resident matriculation cohorts.

Conclusion

The authors conclude that the components employed in SBIRT training do not have to be numerous, or of a particular mode of training, in order to see observable demonstration of SBIRT skills among medical residents. Thus residency educators who have limited time or resources may utilize as few as one mode of training to effectually disseminate SBIRT skills among healthcare providers. As SBIRT continues to evolve as a promising tool to address at-risk substance-related disorders it is critical to train medical residents and other health professionals.

INTRODUCTION

Excessive substance use contributes to significant morbidity and mortality among patients in primary care settings. Increased substance use has been associated with both decreased patient access to preventive healthcare and compliance with prescribed medical treatment.^1,2 These patients visit the emergency department secondary to their co-morbidities and complications of substance use. The economic burden is staggering considering that excessive alcohol use alone cost a median of $2.9 billion in the United States in 2006, with the District of Columbia incurring the highest per-person cost ($1,662).³

Screening, Brief Intervention and Referral to Treatment (SBIRT) is an evidence-based public health approach that addresses at-risk substance use.⁴ Studies have shown that SBIRT is feasible^4–6 and effective in lowering the use of alcohol and other drugs, which in turn improves health outcomes.^7–10 Numerous agencies under the Department of Health and Human Services are driving the effort to make SBIRT an established practice in health care. The United States Preventive Services Task Force (USPSTF) recommends the use of SBIRT in identifying individuals over 18 years-old who are at-risk for substance abuse.¹¹ The National Institute on Alcoholism and Alcohol Abuse (NIAAA) set low-risk drinking guidelines to measure substance use, and establish limits to prevent morbidity.¹² Additionally, disseminating SBIRT into the health care delivery system is the Substance Abuse and Mental Health Administration’s (SAMHSA) mission through various agencies including medical residency programs.¹³

Some studies have observed a lack of preparedness among healthcare providers in identifying patients with health and social functioning impairments from alcohol and other drug misuse. Findings from a US Survey of primary care physicians noted 90% of providers fail to diagnose substance abuse in adults and 40% miss it in teens.¹⁴ These providers felt ill-equipped to assess patients for substance abuse, and experienced difficulty discussing the subject matter compared to other chronic conditions. Similarly, deficient knowledge of SBIRT, negative attitudes and uneasiness discussing substance abuse with patients has been documented in other studies.^15–17 To address this deficiency, 17 institutions received funding from SAMHSA to incorporate SBIRT training into medical education curriculla.^18–23

SAMHSA’s medical residency SBIRT training program allowed medical residents at grantee institutions to receive training in screening, brief intervention, motivational interventions, and increased awareness of referral to treatment mechanisms.²³ Grantee institutions developed training curricula tailored to meet the core SBIRT competencies and skills. Training components used among the SAMHSA grantees, include some or all of the following components with motivational interviewing techniques: didactics, online and web-based content, clinical skills assessment with a standardized patient, and patient encounters within medical specialty settings.⁵

SBIRT training varied by medical residency program and staff trained.⁵ In similar programs, physicians, counselors, undergraduate medical students, and non-medical staff were all targeted for training and all self-reported changes in knowledge, attitudes, and skills after intervention of as little as an hour of lecture on SBIRT or motivational interviewing.^24–32 Yet self-reported increases in skills did not always translate into a successful demonstration of acquired skills.³¹ Observation of demonstrated skills provides a more accurate assessment of SBIRT skills. Therefore, this paper will explore how multiple modes of training impact the SBIRT skills demonstrated by residents.

METHODS

“Know the R.I.S.K.” SBIRT Training Program

“Know the R.I.S.K” is a mnemonic used to reinforce SBIRT principles (1) Raise the issue of substance use, (2) Inform the patient about healthy behaviors, (3) Screen for substance abuse, and (4) Know how to offer brief intervention and referral to treatment. Funded by SAMHSA, the curriculum was designed and inaugurated in 2009 to train residents to provide culturally-competent evidence-based SBIRT for patients who have or are at-risk for substance use disorders at an urban hospital. Training components, shown in Table 1, followed protocol utilized in other studies^22,33: (1) an hour lecture, (2) role play facilitated by SBIRT faculty, (3) self-directed learning modules, (4) documented patient encounters, and (5) a clinical skills assessment (CSA) with a standardized patient (SP).

Table 1.

Know the R.I.S.K. Curriculum

Study component	Component Items	Timing
Lecture/Didactics	Background information presented on: Diagnostic criteria for substance use disorders Universal screening and validated screening tools Motivational interviewing (MI) and MI practice skills Brief intervention and physician skills Cultural competency discussed linguistic barriers, common stereotypes, patient response to the integration of culture, social norms, and values in health care delivery Mechanisms to refer patients for treatment Completed pre-test pencil and paper measures of knowledge, skills, and attitudes towards SBIRT as well as patient centeredness, a surrogate measure for cultural competency	An hour lecture during new resident orientation
Role Play/Clinical Skills Development	Small groups of 8:1 participant/facilitator ratio Practicing screening, brief intervention, motivational interviewing skills, assessing readiness to change, and the development of a prescription for change	An hour after lecture/didactics during new resident orientation
Online Modules	Six online modules: Referral to treatment module, types of treatment programs/facilities and local/national referral hotlines Cultural considerations module SBIRT Tool box module, use of validated screening tools Readiness to change module, further instruction on MI Screening and brief intervention module Substance Abuse module, DSM 5 criteria for substance use disorders	Self-directed training after new resident orientation Approximately 4 hours to complete online
Patient Encounters	Up to 5 encounters with actual patients in clinical setting Observed by attending physician in resident’s specialty area	Self-directed training after new resident orientation
Clinical Skills Assessment	Encounter with standardized patient Core SBIRT techniques – screening, assessing readiness to change, etc. 1 General performance skills – professionalism, acknowledging cultural concerns, patient care, etc Observed by SBIRT faculty instructors Completed post-test pencil and paper measures of knowledge. skills, and attitudes towards SBIRT	Approximately 1 hour long Completed following prior 4 components in a Clinical Skills Center

¹The skills related to SBIRT CSA provide learner assessment and documentation of ACGME competencies fulfilled.

Participants were residents/fellows from primary care and specialty medical residency programs who provided written consent. Data was collected between June 2009 and July 2013, yielding a total of four cohorts. Residents who successfully completed all five components received an SBIRT training certificate. Study consent, protocol, and instruments were all IRB-approved.

Clinical Skills Assessment (CSA) Session

The outcome of interest was a successful demonstration of SBIRT skills as indicated by performance on the CSA. CSA typically occurred at least two months following lecture and/or role play. The session consisted of a brief orientation to the Clinical Skills Center, after which residents proceed to their SBIRT encounter with an SP, while being observed remotely by faculty. Residents were evaluated using a modified BNI-ART interview evaluation form (please see supplement) covering core SBIRT techniques (worth 28 points) and general performance skills (worth 12 points). Participants who obtained a score of at least 70% (28 points) met requirements for successful completion of CSA. Both the SP and the SBIRT faculty member would need to be in agreement for a resident to obtain a satisfactory or unsatisfactory score. Intra-class correlation, as a measure of inter-rater reliability, was 0.82.

Data Analysis

CSA was compared, using univariate analysis, across resident demographic characteristics. CSA was also compared across the other study components (lecture, role play, patient encounters, and online instruction). Significance was established at p<0.05.

RESULTS

Training Outcome: Influence of Each Component on CSA Performance

Demographics and training components were contrasted against satisfactory and unsatisfactory CSA performance in Table 2. The two CSA groups were similar across both demographics and component completion. Both CSA groups had fairly equal proportions of male and female residents, internal medicine was the most represented of the diverse array of specialties. A vast majority of the residents were of African descent, and the highest proportions of residents were recruited from the first matriculation cohort. In terms of completed training components, lectures, which are usually accompanied by role play sessions, were completed by over 93% of the residents. This was because these two components were incorporated into resident orientation after the first matriculation cohort. The self-directed components, online modules and patient encounters, had lower completion rates.

Table 2.

Characteristics of Medical Residents (N=269) across Clinical Skills Assessment (CSA) Performance, 2009–2013

	CSA Performance (N=269)		p
	Unsatisfactory n=81 (30.11%)	Satisfactory n=188 (69.89%)
Age1 - mean (SD)	30.6 (4.4)	30.8 (4.7)	0.79
Gender2 - n (%)			0.45
Male	44 (54.3)	94 (50.0)
Female	36 (44.4)	94 (50.0)
Ethnicity2 - n (%)			0.65
White	1 (1.2)	6 (3.2)
Black	45 (55.6)	113 (60.1)
Latino	1 (1.2)	5 (2.7)
Asian	26 (32.1)	45 (23.9)
Other	3 (3.7)	10 (5.3)
Multi-ethnic	2 (2.5)	5 (2.7)
Specialty2 - n (%)			0.72
Family Medicine	7 (8.6)	23 (12.2)
Internal Medicine	32 (39.5)	74 (39.4)
Surgery	14 (17.3)	30 (16.0)
OB\GYN	5 (6.2)	19 (10.1)
Psychiatry\Neurology	13 (16.1)	25 (13.3)
Other3	10 (12.4)	16 (8.5)
Matriculation Cohort – n (%)			0.22
First (2009–2010)	38 (46.9)	103 (54.8)
Second (2010–2011)	27 (33.3)	42 (22.3)
Third (2011–2012)	14 (17.3)	33 (17.6)
Fourth (2012–2013)	2 (2.5)	10 (5.3)
Training Components Completed- n (%)
Lecture	76 (98.8)	181 (96.3)	0.37
Role Play	72 (88.9)	178 (94.7)	0.09
Online Modules	52 (64.2)	135 (71.8)	0.21
Patient Encounters	14 (17.3)	42 (22.3)	0.35
Followed Recommended Order of Components - n (%)			0.76
No	68 (30.5)	155 (69.5)
Yes	13 (28.3)	33 (71.7)

¹SD indicates standard deviation

²n_Gender=268; n_Ethnicity=262, n_Specialty=268

³Other includes dentistry, orthopedic surgery and pathology

Training Outcome: Influence of Completed Components on CSA Performance

When examining the type of training component completed, there were no significant differences. From Table 2, over 93% of the residents completed lecture and/or role play. There was not a large enough sample of residents who did not complete lecture and/or role play to compare to those that completed. Thus, an examination of the influence of number of components completed could only be done by looking at two (online modules and patient encounters) of the four components (lecture, role play, online modules, and patient encounters). Authors found that completing the online modules component (with or without completion of patient encounters) was equally as effective in achieving a satisfactory CSA as completing the patient encounter alone Additionally, the order of completion of patient encounters and/or online modules did not influence CSA performance (p=0.76).

DISCUSSION

This study contributes to the growing body of knowledge regarding integrating SBIRT training in postgraduate medical residency programs. Particularly it highlights that a uniform curriculum can be used in an interdisciplinary minority-serving training setting for residents in primary and non-primary care, as previous studies focused on primary care specialties only.^12,28,29 Approximately 70% of the current sample demonstrated skills necessary for a brief intervention. This is similar to Wilk & Jenson²⁶ findings which reported that 74% of residents in an internal medicine clinic provided an alcohol intervention to an unannounced SP following alcohol training. The current study did not utilize a pre-post SP design. As such, an assessment of brief intervention skills proficiency prior to the educational intervention was not available and may have played a role in the success rate. Satterfield and colleagues²⁹ found only moderate improvements in brief intervention skills using SPs for both pre- and post-intervention assessments. To minimize subjectivity, authors utilized dual independent evaluations of each CSA session, by SP and SBIRT faculty instructors, and recommend third-party observations such as supervised patient encounters, or patient feedback, when CSAs are not feasible. As SBIRT is adopted as a clinical tool, perhaps graduate medical programs will utilize formal evaluations that are already established in medical education such as Clinical Skills Evaluations (CSE)s. ³⁴

Marshall et al showed an increase in knowledge, experiences, and readiness to adopt SBIRT in clinical practice among trained medical residents³³, and thus an increase in CSA performance among those who completed all components compared to those who did not is expected. However, authors found that no single training component, or completion of specific number of components of the curriculum, impacted residents’ performance. While this may be the first study to examine multi-modal components in SBIRT training, it is important to note that lectures, web-based training, observations, and small group skills development have all been shown to improve learning outcomes.^28,35,36 Even though the program was designed to be completed in a specific order – lecture/role-play, self-directed learning, patient practice, CSA – residents may not have completed all components, nor in correct order. Given the various learning modalities of the curriculum presented in this study, it is possible that residents gravitated towards completing components in an order that best suited their learning needs. It is important to consider the individual learner and perhaps develop individualized educational plans to suit different learning styles. Also, Scott and colleagues²² showed that having GME work with a training program reinforced and encouraged residents to complete training. Institutional buy-in may have fortified residents’ resolve to display adequate clinical skills.

There are a number of limitations that should be considered. CSA performance may be different among those who did not consent to participate in the study, or participants may have been motivated to perform well as new residents starting a new program. Also, the CSA format was such that residents could review materials just prior to the examination, and the results may not reflect internalization of the material over time with continuous practice of SBIRT skills. Patient satisfaction or reductions in substance use were not measured in this study. Connecting SBIRT residency training to patient outcomes could serve as reinforcement and continued promotion of SBIRT skills. No control group was used as all residents went through the training program as part of orientation or grand rounds. Authors also did not follow up with medical residents after CSA. Some studies found that years in practice increase likelihood of SBIRT skill proficiency.³⁰ Additionally, this study may have been limited by the authors’ definition of successful training as successful completion of CSA. Perhaps a different definition, such as patient report of SBIRT encounters with physicians, would be more accurate as it demonstrates SBIRT skills in a real-world setting. Despite these limitations, the findings of this study support existing studies of multi-disciplinary graduate medical education. No one specialty outperformed another. The SBIRT training program was accepted equally across multiple primary and non-primary care specialties, multiple ethnicities. All this achieved in a level-I trauma center serving an urban community. The study also demonstrated that even with limited work hours, and exhausted federal funding mechanisms, applying any one of the components can result in observed increases in SBIRT skills.

Future Steps

Future studies may include an evaluation of the involvement of SBIRT champions and their reinforcement of material learned by each medical specialty. While reasons for the unsuccessful completion of CSA were largely unknown, involvement of champions and reinforcement of learned skills may in part address the needs of the residents who did not successfully demonstrate SBIRT skills in CSA. An analysis of differences in performance on individual CSA items is planned as a follow-up to this manuscript. Further steps may also include examining SBIRT efficacy with problem drug use. This has been of particular interest with mixed evidence on the efficacy of implementing SBIRT, with substances other than alcohol, in different settings. ^9,37–40 Additional next steps include adding more on-demand training tools such as apps or virtual programs for providers, following up on patients’ perception of physicians’ skills, following up with residents after training or residency graduation, comparing results to a control group of physicians who did not receive SBIRT training, and examining the effect of reinforcing SBIRT training.