Organizational Attributes and Screening and Brief Intervention in Primary Care

Abstract

Overconsumption of alcohol is well known to lead to numerous health and social problems. Prevalence studies of United States adults found that 20% of patients meet criteria for an alcohol use disorder. Routine screening for alcohol use is recommended in primary care settings, yet little is known about the organizational factors that are related to successful implementation of screening, brief intervention (SBI) and treatment in these settings. The purpose of this study was to evaluate organizational attributes in primary care practices that participated in a practice-based research network trial to implement alcohol SBI. The Survey of Organizational Attributes in Primary Care (SOAPC) has reliably measured four factors: communication, decision-making, stress/chaos and history of change. This 21-item instrument was administered to 178 practice members at the baseline of this trial, to evaluate for relationship of organizational attributes to implementation of alcohol SBI and treatment. No significant relationships were found correlating alcohol screening, identification of high-risk drinkers and brief intervention, to the factors measured in the SOAPC instrument. These results highlight the challenges related to the use of organizational survey instruments in explaining or predicting variations in clinical improvement. Comprehensive mixed methods approaches may be more effective in evaluations of implementation of SBI and treatment.

1. Introduction

Recent data from the 2010 National Health Interview Survey (NHIS) showed that 64% of adults, age 18 and older in the United States (US) currently drink alcohol, regularly or infrequently (Schiller, Lucas, Ward, & Peregoy, 2012). Prevalence studies in primary care have found 20% of patients meet criteria for “at-risk” drinking or alcohol use disorders (Whitlock, Polen, Green, Orleans, & Klein, 2004). It is well recognized that numerous health and social problems are associated with overconsumption of alcohol (Fiellin, Reid, & O’Connor, 2000). The US Preventive Services Task Force (USPSTF) and the National Institute on Alcohol Abuse and Alcoholism (NIAAA) recommended routine screening for alcohol use in primary care, and brief interventions for identified high-risk drinkers (National Institute on Alcohol Abuse and Alcoholism, 2005; “Screening and Behavioral Counseling Interventions in Primary Care To Reduce Alcohol Misuse: Recommendation Statement,” 2004). Previous studies have shown that such screening and brief interventions (SBI) in primary care can result in significant reductions in consumption and problems related to drinking (Kaner et al., 2007).

Primary care practices may miss opportunities to add alcohol screening to routine visits in the absence of organized practice approaches. Few studies have examined the organizational and provider characteristics related to alcohol screening in primary care. Qualitative analyses from our previous SBI research found that organizational factors, as well as provider and patient characteristics influenced the consistency of SBI (Miller et al., 2006; Rose et al., 2008). Delegation of alcohol screening to nursing staff, with clinician follow-up for positive screens was found to be the most important organizational issue leading to more in-depth screening and brief intervention. Consistent screening, brief intervention and referral required the development of new skills and competencies by staff, with support from clinicians and managers to integrate the screening process into the workflow. Provider issues that negatively affected SBI included competing priorities, lack of enthusiasm, and reluctance to institute practice changes. Initially, some clinicians expressed concerns about offending patients, and about impacting insurance reimbursement by recording alcohol diagnoses, where behavioral health was carved out of primary care through contracts (Rose et al., 2008). Through the use of the PPRNet-Translating Research into Practice (PPRNet-TRIP) model, we were able to identify these issues through qualitative data obtained at practice site visits, and network meetings for sharing of best practices (Miller et al., 2006; Rose et al., 2008).

Qualitative studies on the implementation of change in primary care practice have shown that stakeholders need clear vision and goals regarding the focus of change, leadership/management support, team input in decision-making, enhanced communication, motivation and willingness to experiment, resources for change, recognition of competing demands both in the practice setting and externally, management of stress and ambiguity, and learning from previous efforts at implementing change (Cohen et al., 2004; Nemeth, Feifer, Stuart, & Ornstein, 2008). The identification of these concepts has been useful in developing implementation research measurement tools, as well as providing guiding frameworks to approach translating research into practice. Newer quantitative tools to assess organizational attributes in primary care performance improvement have the potential to add to the qualitative database in this area. The Survey of Organizational Attributes in Primary Care (SOAPC) was developed to measure practice resources for implementing change in primary care (Ohman-Strickland et al., 2007). The purpose of this paper is to present an evaluation of organizational attributes in a primary care group randomized study to improve alcohol SBI and medication management.

2. Methods

2.1 Study Design

The Implementation of Alcohol Screening, Intervention and Treatment in Primary Care (AM-TRIP) study was designed as a crossover group-randomized study in 19 primary care practices that are members of a practice-based research network, PPRNet. PPRNet members submit quarterly practice data from their electronic health records (EHRs) and receive quarterly reports on their performance. The intervention was comprised of an introductory site visit to each enrolled practice to disseminate the NIAAA guidelines, EHR tools, and reports on the outcomes of alcohol SBI and use of medications for alcohol use disorder (AUD). The early intervention (EI) group received practice site visits and attended a network meeting during the first year. The delayed intervention (DI) group served as control during that phase of the study. At one year, the DI group attended a network meeting with the EI group, and received practice site visits while the EI group did not. This trial design enabled our team to evaluate the sustainability of the intervention on study outcomes. The detailed methods and results of the primary aims of this study, including qualitative data are reported elsewhere (Ornstein et al.). The present report focuses on a secondary aim of the (AM-TRIP) project, which evaluated whether organizational variables of primary care practice are related to the implementation of alcohol SBI. The Institutional Review Board at the Medical University of South Carolina approved this study.

2.2 Measures

The Survey of Organizational Attributes in Primary Care (SOAPC), recently validated in 51 practices, was selected to evaluate the primary care practices’ organizational characteristics (Ohman-Strickland et al., 2007). The SOAPC is a 21-item questionnaire that reliably measures 4 stable and internally consistent factors. These include Communication (Cronbach’s α=0.81), Decision-making (α=0.88), Stress/Chaos (α=0.85) and History of change (α=0.73). The first three factors measure components of the resources for change concept described by Cohen et al (Cohen et al., 2004). The Communication factor measures whether all members of the practices are able to work through problems as a team. High scores on Decision-making indicate the practice has a participatory approach to decision-making. High scores on Stress/Chaos indicate that employees feel overwhelmed by the workload. History of change is helpful in assessing whether previous experience with making practice change influences the success of interventions. We hypothesized that scores of high Communication, high Decision-making, lower Stress/Chaos and higher scores on History of change would predict positive changes in alcohol SBI.

2.3 Data Collection

The SOAPC survey was administered at baseline to EI and DI practices, during face- to-face meetings of all clinicians and staff attending the introductory site visits. The outcome measures based on NIAAA guidelines for implementation of alcohol screening were collected from EHR data, included the following: whether the patient was asked “Do you sometimes drink beer, wine or other alcoholic beverages?” and if positive the single at-risk drinking question “How many times in the past year have you had five or more drinks in a day (for men)/ four or more drinks in a day (for women). Those responding positive to the at-risk question were eligible for the brief intervention. Upon further clinician assessment of the positive screen, if AUD was diagnosed, the presence of a prescription for alcohol pharmacotherapy was evaluated.

2.4 Analysis

Generalized linear mixed models (GLMMs) in SAS v9.3 (Cary, NC) were used to analyze the SOAPC measures and the outcome measures. A total of 16 GLMMs were created, in which the patient was treated as the unit of analysis. Each of the AM-TRIP study outcomes of screening for alcohol use, screening for high-risk drinking, brief intervention and alcohol medication use were modeled as a function of practice treatment group (EI vs. DI) and practice SOAPC domain score (averaged across all survey responders). As the domain scores were highly correlated with one another, each of the four domains was entered into a separate model.

3. Results

Nineteen practices completed a total of 178 surveys at baseline. Survey respondents included 100% of the participants of the introductory site visit, ranging from 2 respondents to 26 respondents depending on practice size. Table 1 presents demographics of responders to the survey. Table 2 summarizes the results. None of the SOAPC domain scores were significantly associated with the probability of being screened for alcohol use, being screened for high risk drinking, or being provided a brief intervention. Overall, higher levels of stress/chaos tended to be associated with lower performance on these study outcomes; but these associations were not statistically significant. We did not present the model on alcohol medication use in the table, which was not estimable due to low numbers of prescribed medications, however the expected proportion of medication prescriptions for alcohol treatment is not known.

Table 1. Responder (N=178) Characteristics of the SOAPC Survey.

Characteristic	Frequency: N (%)
Position
Physician	48 (29.1%)
Nurse Practitioner or Physician	10 (6.1%)
Assistant
RN/LPN/LVN	36 (21.8%)
Medical Assistant/Technician	35 (21.2%)
Other (e.g. Clerical Staff)	36 (21.8%)
Gender
Male	144 (83.7%)
Female	28 (16.3%)
Age Group
<30	45 (26.6%)
31-40	43 (25.4%)
41-50	42 (24.9%)
51-60	32 (18.9%)
>60	7 (4.1%)
Work Experience in Practice
<1 year	39 (27.9%)
≥1 year	101 (72.1%)

RN: Registered Nurse; LPN: Licensed Practical Nurse; LVN: Licensed Vocational Nurse

Note: The individual frequencies do not add up to the total sample size (n=178) due to missing values for the specific respondent characteristic listed.

Table 2. Baseline SOAPC Measures as Predictors of AM-TRIP Outcomes.

	AM-TRIP Study Outcome
SOAP-C Domain	Screening for alcohol use	95% Confidence Interval
	OR	LCL	UCL	P-Value
Communication	1.15	0.89	1.49	0.277
Decision- making	1.08	0.95	1.23	0.230
Stress & chaos	0.90	0.72	1.12	0.325
History of change	1.07	0.67	1.72	0.771
	Screening for high risk drinking	95% Confidence Interval
Communication	0.99	0.60	1.62	0.965
Decision- making	1.01	0.79	1.30	0.907
Stress & chaos	0.70	0.48	1.04	0.074
History of change	0.68	0.29	1.62	0.388
	Brief intervention	95% Confidence Interval
Communication	1.32	0.85	2.05	0.217
Decision- making	1.17	0.95	1.44	0.153
Stress & chaos	0.69	0.47	1.01	0.059
History of change	1.78	0.92	3.47	0.088

Legend: OR=odds ratio associated with a 1-unit change in the SOAPC domain score; LCL=lower 95% confidence limit; UCL=upper 95% confidence limit.

4. Discussion

Limited quantitative research is available on organizational attributes as predictors of clinical improvement in primary care. As clinical improvement initiatives are undertaken to incorporate evidence-based practices such as the USPSTF or NIAAA guidelines on alcohol SBI, an important question to be answered is why do some practices succeed in their implementation, and others fail? Some of the control practices in our previous study exceeded the efforts of intervention practices, related to technical expertise in using their electronic health record, and leadership and motivation of providers and staff (Rose et al., 2008). Babor and colleagues found that complex provider and organizational characteristics influenced implementation of a particular model for SBI (Babor, Higgins-Biddle, Dauser, Higgins, & Burleson, 2005). Our present study evaluated the use of an organizational survey to attempt to understand what characteristics predict higher performance in implementing SBI. Unexpectedly, no organizational attributes predicted implementation of SBI. This result is surprising considering that practices with higher levels of resources for change would be expected to implement an alcohol screening process that was primarily delegated to the nursing staff.

Research linking the organizational attributes of primary care settings and patient level outcomes is minimal. This type of research is complex, in busy clinical settings with clinicians and managers that are dealing with the need to translate research into daily practice in coherent and clear ways through planned implementation and evaluation. Practices need to adapt with the complexities of staff and clinician turnover, changes in practice ownership, technological changes, patient demographic shifts, and demand for services. Ongoing educational development must be planned and evaluated to ensure that all team members implement consistent processes to achieve improvement. Implementing evidence-based medicine requires evidence-based management. Shortell suggested that evaluations of change assess four levels of interaction with the larger system, the organization, the group/team level and the individual rather than evaluating impacts of each of these levels independently (Shortell, 2004). Multilevel approaches, such as Shortell proposed are more consistent with current theories in contrast to the use of the SOAPC instrument. The Consolidated Framework for Implementation Research (CFIR) developed by Damshroder, merged multiple constructs for analyzing implementation of evidence-based practices in healthcare settings (Damschroder et al., 2009). Employing CFIR, using a mixed methods evaluation approach would support developing a rich understanding of context and characteristics of interventions for alcohol SBI.

Limitations of our study should be noted. The research was conducted in only 19 practices with 178 responders. Although our study did not identify any significant relationships between organizational attributes and study outcomes, it is possible that such associations do truly exist, but that they are subtler than those that were detectable given the available statistical power. Also, the data were collected at one point in time and do not measure change over time. As the data involves measurement at the individual level, it should be noted that self-report data are subject to biases. Lastly, this brief report on the secondary aim of the AM-TRIP study, does not include qualitative evaluation data from the intervention thus, it does not capture the full context of alcohol SBI implementation in primary care practices.

A key strength of this research is that we used a validated survey instrument to measure organizational attributes and had the participation of all staff and clinicians. Having used an organizational survey to evaluate how practice characteristics effect the adoption of alcohol SBI is important in advancing the science of implementation research. Mixed method research that includes components of the CFIR will inform our future studies regarding implementation of alcohol SBI, and will provide direction to other researchers and program developers for future studies in this area.

4.1 Conclusions

This research suggests that quantitative measures alone are not suitable in correlating organizational culture and climate with clinical outcomes. Qualitative evaluations may provide more specific context to explain variations in the adoption of guidelines and clinical behaviors that can reliably produce improvements in clinical outcomes. Further research, using both quantitative and qualitative methodology and a more robust evaluation framework, with larger and more varied samples of primary care practices may inform more accurately on organizational factors as they relate to SBI.

Highlights.

• Organizational attributes may impact alcohol screening and brief intervention.

• Practice members implementing screening and brief intervention were surveyed

• A previously validated survey tool was used with 178 participants in 19 practices.

• No significant predictors were found correlating alcohol outcomes with organizational factors.

• Robust mixed methods evaluations are needed to measure these complex relationships.