Assessing the relationship between medical residents’ perceived barriers to SBIRT implementation and their documentation of SBIRT in clinical practice

Introduction

Of the 17.9 million adult Americans who have an alcohol use disorder, most (89%) are unaware of their condition.¹ The Centers for Disease Control and Prevention (CDC) lists more than 30 specific immediate and long-term health risks associated with excessive alcohol use,² suggesting that identifying and addressing alcohol use is an important component of primary medical care. Screening, brief intervention, and referral to treatment (SBIRT) is an integrated and validated approach to identifying risky or problematic alcohol use and providing services to those who need them.³

In 2009, Indiana University School of Medicine (IUSM) reached a cooperative agreement with the Substance Abuse and Mental Health Services Administration (SAMHSA) to begin training its medical residents in SBIRT. The training curriculum focuses on delivery of SBIRT services using motivational interviewing techniques (MI). In addition to developing this curriculum and integrating it into the first year of medical residency at IUSM, we developed a protocol for implementing SBIRT services at an internal medicine continuity clinic. This clinic is part of a larger safety-net health system in Indianapolis at which some residents are assigned a small panel of patients during their internal medicine residency. At the time of the study, the continuity clinic was classified as a community health centre, though it since has received federally qualified health center (FQHC) designation. Medical conditions presenting in the clinic vary, with a predominance of adult chronic diseases such as hypertension, heart failure, diabetes, and chronic obstructive lung disease. At each visit, all adult patients were screened for alcohol, drug, and tobacco use by a medical assistant (MA) using the AUDIT-C tool and single drug and tobacco screening questions. These data were given to care providers, both medical residents and staff physicians, to enable delivery of targeted SBIRT services. At the conclusion of each clinical encounter, providers were then asked to document the components of SBIRT that they completed by responding to four yes/no statements, such as ‘I referred the patient to a substance abuse counselor.’

However, across 9954 initial physician visits with unique patients that occurred between 1/31/2011 and 6/29/2012 (hereafter ‘the study period’), we observed a high frequency of incomplete SBIRT encounter forms (83.8%), meaning that the physicians generally were not documenting whether they completed SBIRT. While this was seen as a cause for concern, logically, we could not conflate incomplete forms with a failure to provide SBIRT services. We therefore were moved conceptually to separate two distinct behaviours: actually performing SBIRT, and documenting that SBIRT was completed. In order for SBIRT to be an integrated part of primary medical care, all associated processes and results need to be recorded. Since we have access to evaluative data collected during their SBIRT training that assesses residents’ anticipated barriers to utilizing SBIRT and MI in their medical practice, we examined whether medical residents’ self-reported barriers to performing SBIRT or using MI in their clinical practice predicted appropriate SBIRT documentation in their clinical practice.

Methods

A total of 80 physicians saw patients in the selected continuity clinic during the study period. Of those, 30 were medical residents who had received IUSM’s SBIRT/MI training, and 50 were either medical residents or staff physicians who had not received that training. Because elicitation of barriers (the primary predictor variables) was conducted as part of the SBIRT/ MI training, we excluded from our sample all visits conducted by the 50 physicians who had not attended the training. The eligible medical residents completed 2100 initial physician visits with unique patients during the study period (a subset of the 9954 total visits).

Data collection

At each physician visit, an SBIRT form containing the alcohol, drug, and tobacco questions, as well as four physician documentation questions, accompanied the patient’s chart. Expectations for filling out this form were conveyed via training for both MAs and residents. Because the variable of interest in this study was whether a form was completed during the patient encounter, a completed encounter form (meaning the resident indicated whether he/she completed one or more components of SBIRT) was coded as ‘0’ and an incomplete encounter form (meaning the resident left this section blank) was coded as ‘1.’ We also documented whether the physician received a form without any screening data (error at the MA level). We dichotomously coded forms as ‘missing screening data’ ‘0’ or ‘containing screening data’ ‘1.’ The MAs and residents were aware that their forms would be reviewed for accuracy and completion by their supervisor.

In order to assess barriers to SBIRT implementation, at the completion of residents’ SBIRT/MI training, as part of the project’s evaluation process, they were asked, ‘Please identify the greatest barriers to implementing motivational interviewing in your own practice with patients’ and ‘Please identify the greatest barriers to implementing SBIRT in your own practice with patients.’ Open-ended responses to these questions fit into four distinct categories of barriers that conceptually correspond to previously identified clusters of barriers:⁴ time available in the clinic, characteristics of the patient (i.e., resistance), characteristics of the system (i.e., patient flow issues), and characteristics of the resident him/ herself (i.e., lack of ability). Each resident’s response was categorized as belonging to one or more of the four categories. Although residents were not limited as to the structure of their responses, only one resident identified barriers across more than two categories. Residents were not asked to rank responses by importance and so all responses were given equal weight. Two PhD-level researchers coded the open-ended responses separately, yielding a 97% concordance.

Because each patient encounter involved a single resident, we were able to determine which perceived barriers to MI/ SBIRT were ‘present’ at each encounter. We used chi-square testing to determine whether the completion of a form at a given encounter was independent of each of the stated barriers to MI/SBIRT. Then, we used binary logistic regression to calculate adjusted odds ratios to determine likelihood of failure to complete the encounter form based on the presence of a given barrier and whether the screening data was complete at the time of the encounter.

Results

Across the 2100 patient visits, the encounter form was completed 410 times (19.6%) and was left incomplete 1690 times (80.4%). Further, the form was passed from the MA to the physician without any screening data recorded 852 times (40.6%). A total of 2044 patient visits were with a resident who previously had identified ‘time’ as a barrier to implementing SBIRT/MI, 290 visits were with a resident who had identified ‘patient characteristics’ as a barrier, 292 visits were with a resident who had identified ‘systemic concerns’ as a barrier, and 133 visits were with a resident who had identified his/her own characteristics as a barrier (see Table 1a).

Table 1a.

Perceived barriers to SBIRT and completion of encounter forms (frequencies and tests of independence) (n = 2100).

Presence of barriers in encounters	Completed encounter form	Did not complete encounter form	χ2
Time – Present	398	1646
Time – Not present	12	44	0.13
Patient characteristics – Present	98	192
Patient characteristics – Not present	312	1498	43.60***
System characteristics – Present	106	186
System characteristics – Not present	304	1504	60.76***
Residents’ characteristics – Present	20	113
Residents’ characteristics – Not present	390	1577	1.82

The null hypothesis that completion of the form was independent of each of the residents’ perceived barriers was rejected for patient characteristics (χ² = 43.60, P < .001) and systemic characteristics (χ² = 60.76, P < .001), but was upheld for time and residents’ characteristics (See Table 1a).

The results of the binary logistic regression model, which tested five predictor variables simultaneously, showed that two of the four physician-level barriers predicted completing the encounter form (adjusted odds ratios). Patient encounters with residents identifying patient characteristics (AOR = 3.47, P < .001) or systemic characteristics (AOR = 3.57, P < .001) as barriers were more likely than those not identifying those barriers to produce completed encounter forms. Further, encounters where the resident was provided with a completed screening form were more likely (AOR = 14.42, P < .001) to produce completed encounter forms. However, patient encounters with residents identifying time or residents’ characteristics as barriers were not more or less likely to produce completed forms (see Table 1b).

Table 1b.

Binary logistic regression – adjusted odds ratios for patient encounters (n = 2100).

Predictor variables (0 = Barrier present; 1 = Barrier absent)	Adjusted odds ratio	Confidence interval
Missing screening values on form	14.42***	9.47–21.95
Time	1.42	0.66–3.06
Patient characteristics	3.47***	2.48–4.84
System characteristics	3.57***	2.62–4.86
Residents’ characteristics	0.24	0.08–2.35

Note:

^*P < 0.05,

^**P < 0.01,

^***P < 0.001.

Implications of the findings

The existence of practitioner/physician-specific barriers to the use of SBIRT techniques in primary care repeatedly has been documented in the literature.⁵ Given this knowledge, our post-training evaluation for medical residents asked them to identify expected barriers (open-ended) rather than simply asking them to state whether barriers were present or not. Interestingly, our open-ended elicitation produced sets of barriers that match categories from the existing literature.^5,6 To our knowledge, however, no studies have explored linkages between these barriers and the completion of systemic processes related to SBIRT.

Our finding that physicians rarely document what happened during an ‘SBIRT encounter’ if they are not provided with screening results is unsurprising. However, it may seem counterintuitive that our data suggest that residents who do perceive patient- or system-level barriers are more likely to appropriately document their use/non-use of SBIRT services in a given visit. However, an inverse relationship between presence of barriers and performance of desired SBIRT-related behaviour is not without precedent in the literature,⁷ though it never has been thoroughly researched or explained.

Health behaviour theories (i.e., Health Belief Model or Theory of Planned Behavior) suggest that the intention to perform a specific behavior can be modified by the existence of barriers to performing that behaviour,⁸ and so one might reasonably expect, all things being equal, that barriers to one behaviour (performing SBIRT) would not affect intentions to perform another behaviour (documenting whether SBIRT was completed and/or the results of a screening). This theoretical orientation holds true for two of the elicited barrier types (time and residents’ characteristics). The results for these variables are non-significant, suggesting that there is no relationship. This further emphasizes that the potential causes of our significant findings should be investigated further. It might be the case that there is an unmeasured, shared characteristic among physicians who anticipate that patient- or system-level characteristics will pose barriers to SBIRT. For example, these individuals may be detail-oriented and more highly attuned to external factors that might affect their work; contemplation of potential problems may be indicative of taking the process seriously. It may also be the case that this and other studies examining barriers to SBIRT suffer from an unidentified methodological flaw.

In future research, we also suggest that program evaluators for SBIRT programs in medical residencies investigate not only potential barriers to performing SBIRT but also barriers to documenting what has happened during an SBIRT encounter. Significant amounts of missing data make it difficult to draw conclusions not only about the efficacy of SBIRT but also of the implementation processes that link it to the setting where it is embedded (i.e., a primary care clinic). In sites that do not currently bill for SBIRT, complete data are important for cost analyses and for evaluating patient outcomes. If and when barriers to documentation of SBIRT processes are elicited from those being trained to deliver services, training programs, such as those in medical residency programs, should acknowledge and address those barriers as a part of the training process to reduce the extent to which physician visits become a ‘black box’ regarding behavioural health processes.

Limitations

This study had several limitations that may have affected the results and interpretation. First, although we studied a large number of consultations, they were completed by a small number of residents who were not randomly selected – the results should not be generalized outside of those residents without replication with a larger, randomly selected pool of physicians. Second, the time that elapsed between residents’ face-to-face training and their patient consultations varied (<1 to 9 months) and we cannot be certain of the durability of the elicited barriers or other resident-level factors between the initial training and the consultations. Third, it is possible that a particular MA with a low form completion rate consistently worked a shift pattern matching a particular resident (and his or her associated barriers); at the same time, however, because we present adjusted odds ratios (AOR) that include completion of the form at the MA level, and because those results are in concordance with the chi-square output, we do not believe this was a significant source of bias.