Organizational Readiness Assessment in Acute and Long-Term Care has Important Implications for Antibiotic Stewardship for Asymptomatic Bacteriuria

Abstract

Background:

Prior to implementing an antibiotic stewardship intervention for asymptomatic bacteriuria (ASB), we assessed institutional barriers to change using the Organizational Readiness to Change Assessment (ORCA).

Methods:

Surveys were self-administered on paper in inpatient medicine and long-term care units at 4 Veterans Affairs facilities. Participants included providers, nurses, and pharmacists. The survey included seven subscales: evidence (perceived strength of evidence) and six context subscales (favorability of organizational context). Responses were scored on a 5-point Likert-type scale.

Results:

104 surveys were completed (response rate =69.3%). Overall, the evidence subscale had the highest score; the resources subscale (mean 2.8) was significantly lower than other subscales (P < 0.001). Scores for budget and staffing resources were lower than scores for training and facility resources (P < 0.001 for both). Pharmacists had lower scores than providers for the staff culture subscale (P = 0.04). The site with the lowest scores for resources (mean 2.4) also had lower scores for leadership and lower pharmacist effort devoted to stewardship.

Conclusions:

Although healthcare professionals endorsed the evidence about non-treatment of ASB, perceived barriers to antibiotic stewardship included inadequate resources and leadership support. These findings provide targets for tailoring the stewardship intervention to maximize success.

Background

Asymptomatic bacteriuria (ASB) is defined as the presence of bacteria in the urine, in the absence of clinical signs and symptoms attributable to UTI.^1,2 Re-releases of guidelines on ASB in 2019 from both the Infectious Diseases Society of America and the United States Preventive Services Task Force recommend against screening for and treatment of ASB in non-pregnant adults.^2,3 Overtreatment of this highly prevalent condition contributes to increased healthcare costs, adverse drug effects, emergence of antibiotic resistance, and risk of Clostridioides difficile infection.⁴ Despite the preponderance of evidence that the risks of ASB treatment outweigh the benefits for most populations, guideline-discordant treatment remains common.^5,6

Implementation of guideline-concordant practices requires preparation, motivation, and investment of time and resources. Organizational readiness for change is the “extent to which organizational members are psychologically and behaviorally prepared to implement organizational change.” ⁷ Readiness is thought to be a precursor to successful change implementation; inadequate readiness may lead to failure or unsustainability of the project.⁷ The Organizational Readiness to Change Assessment (ORCA) was developed and validated through the Veterans Affairs Ischemic Heart Disease Quality Enhancement Research Initiative (QUERI) program.⁸ This survey instrument evaluates site readiness for implementation and identifies barriers to change using three major scales that are based on the core elements of the Promoting Action on Research Implementation in Health Services (PARIHS) framework. As a diagnostic tool, the ORCA can be used prior to an intervention to detect potential obstacles and to tailor the intervention based on available resources. The ORCA has been used for quality improvement projects in diverse fields including intensive care, lung cancer screening, and mental health services.^8,10 Most recently, several instruments for measuring organizational readiness for change have been used prior to antibiotic stewardship interventions in primary care clinics¹¹ and nursing homes.¹² In the former study, “high readiness” primary care practices trended toward greater improvements in antibiotic prescribing between years. Although assessment of organizational readiness is not necessary to initiate antibiotic stewardship interventions, understanding baseline barriers can help explain subsequent successes and failures, thus leading to generalizable lessons about how to implement antibiotic stewardship.

We previously implemented a successful antibiotic stewardship intervention at a large Veterans Affairs (VA) medical center.¹³ This “Kicking Catheter-Associated UTI (CAUTI) Campaign” used an evidence-based algorithm and case-based audit and feedback to train clinicians in guideline-concordant management of ASB. The current “Less is More” dissemination project aims to implement a scalable version of the Kicking CAUTI Campaign across four geographically diverse VA facilities, with four control sites.¹⁴ Prior to implementing the Less is More program, we used the ORCA to assess context, culture, and barriers at each intervention site. While the prior studies of organizational readiness for antibiotic stewardship focused on general differences between sites¹¹ or providers,¹² our study comprehensively evaluated differences in readiness for change between subscales, multiple healthcare provider types, and study sites. Additionally, our study targeted overuse of antibiotics in both acute and long-term care settings, where antibiotic stewardship for UTI is especially needed.^6,15 We hypothesized that measurements of readiness to change, including assessment of the evidence and cultural context of the organization, would provide actionable information to guide site-specific implementation of our antibiotic stewardship program for ASB. By understanding differences between subscales, provider types, and sites, we could better tailor support to providers and sites.

Methods

Design

We collected baseline ORCA surveys from healthcare personnel at four VA medical centers including Ann Arbor, Greater Los Angeles, Miami, and Minneapolis. Inpatient medicine units and long-term care units were included at each facility. We chose these units because they have a high prevalence of patients with ASB and our previous intervention demonstrated successful improvement in antibiotic prescribing for ASB in these clinical settings.¹³ Emergency departments, outpatient units, spinal cord units, and non-medicine acute care units (spinal cord, psychiatry, neurology, and surgery) were excluded.

Participants

Surveys were distributed to prescribing providers (residents, staff physicians, physician assistants, and nurse practitioners), nurses (registered nurses, licensed vocational nurses, and licensed practical nurses), clinical nurse assistants (CNA), infection preventionists, and quality managers. Survey distribution included efforts to reach evening and night-shift employees, as well as a representative sample of different types of providers. Demographic information was collected from participants, including number of years in practice.

Survey instruments

The survey was adapted from the original ORCA, which consists of three primary scales.⁸ The evidence scale evaluates the strength of the research evidence as perceived by members of the practice team. The context scale measures the favorability of the organizational context to support quality-improvement changes in general, including leadership culture; staff culture; resource support; ability to evaluate change; and communication of leadership. The staff culture assesses personal responsibility for assessing outcomes, cooperation to improve care, willingness to innovate, and receptivity to change. The facilitation scale assesses the capacity for implementation of the specific evidence-based practice (in this case the antibiotic stewardship intervention), including team member roles and details of the implementation plan. The ORCA instrument used in the present study is presented in the Supplemental Materials. The survey included the scales of evidence assessment (1 subscale) and context assessment (6 subscales). Each subscale consisted of 3-4 questions, for a total of 26 items. The facilitation scale was excluded because the items in this scale are most meaningful to assess implementation capacity after sites have been selected and are early in their implementation planning phase. The evidence assessment included 1 open-ended question that addressed indications for urine culture and sources of guidance for decision-making. All other items were answered on a 5-point Likert-type scale, with 1 meaning very weak or strongly disagree and 5 meaning very strong or strongly agree.

In addition to the ORCA survey, a site-elements questionnaire was administered to the lead co-investigator at each site. This survey included questions on available resources, such as whether any antimicrobial stewardship program was already in place, presence of an infectious-disease trained physician, and involvement of pharmacists trained in antimicrobial stewardship. We also measured full time equivalent (FTE) staff available for stewardship and leadership support at each site.

Survey distribution

Paper surveys were distributed by research assistants between January 31, 2018, and December 20, 2018. Surveys were left in staff break rooms with instructions, distributed at conferences, distributed by nurse leaders to other nurses, or handed to individual participants. Response rates were calculated as the proportion of distributed surveys that were returned. All study activities were approved by the Baylor College of Medicine Institutional Review Board and by the institutional review boards of all participating sites.

Survey analyses

Data were checked for normality. Descriptive statistics (percent, mean, and standard deviation) were used to summarize responses to the survey items. Mean Likert scores were calculated for each subscale. A repeated measures ANOVA analysis was used to compare the mean scores of seven subscales, followed by post-hoc tests with Bonferroni corrections for multiple comparisons. Depending on data distribution, one-way ANOVA or Kruskal-Wallis tests were used to compare the scores for each subscale between healthcare professional types and between the sites, followed by post-hoc tests. Pairwise comparisons were performed using Dunn’s procedure with Bonferroni corrections for multiple comparisons. Due to significant differences for the resources subscale in the overall survey and between sites, additional post-hoc tests were performed for the items within the resources subscale.

Scores were compared between providers, pharmacists, and nurses. For the purposes of comparisons, the “provider” category included residents, fellows, staff physicians, physician assistant, and nurse practitioners. The “nurse” category included nurses, clinical nurse assistants, infection preventionists, and quality managers. To maintain confidentiality, sites were de-identified for publication of the survey results.

Results

Survey response rate

In total, 104 ORCA surveys were completed, with 40.3% of surveys coming from providers, 40.3% from pharmacists, and 19.2% from nurses. The 42 provider respondents included 17 (40%) Infectious Diseases (ID) physicians, 12 (29%) staff physicians, 5 nurse practitioners, 4 resident physicians, and 4 physician assistants. The overall response rate was 104/150 (69.3%), with 14/33 (42.4%) in Site 1, 31/53 (58.4%) in Site 2, 34/39 (87.1%) in Site 3, and 25/25 (100%) in Site 4. For 91 respondents, the median number of years of practice was 10 (range 1-44).

Evidence Assessment: Participants’ Indications for Urine Culture

Of 104 respondents, 74 (71.2%) provided an answer to the open-ended question on indications to send a urine culture or start antimicrobials for patients with an indwelling urinary catheter (Fig. 1). Of those who provided a response, 64/74 (86.5%) indicated that they would send a urine culture or start antimicrobials based on the patient’s signs and symptoms, including specific evidence-based symptoms (fever, dysuria, urgency, etc.) and general responses of “signs” and “symptoms.” For example, one respondent stated: “Urine culture should only be sent on patients with urinary specific symptoms.” Laboratory data, including pyuria, peripheral leukocytosis, or culture results, were listed as an indication by 16/74 (21.6%) of respondents. Misleading symptoms, such as cloudy urine or foul smell, were specifically referenced as indications for culture and/or treatment by 8/74 (10.8%) of respondents, for example “May send a culture if urine is very cloudy.” In this open-ended question, other listed indications for sending urine cultures and treating bacteriuria included urologic procedures (10.8%) and pregnancy (5.4%).

Fig. 1.

Listed indications to send a urine culture or start antimicrobials for a patient with an indwelling urinary catheter (n=74).

Of 104 respondents, 46 (44.2%) provided an answer to the open-ended question on the sources of guidance for making diagnostic and treatment decisions (Fig. 2). Of those who provided a response, 25/46 (54.3%) listed the IDSA guidelines as a source of guidance. Other common sources of guidance included UpToDate (21.7%) and the ID team, such as the ID pharmacist or consulting provider (15.2%). Nine (19.6%) respondents stated that they would defer management decisions to a supervisor (nurse or attending provider), “As a nurse I would clinically evaluate the patient (urinary symptom, temp, malaise) and notify the providers for possible orders.”

Fig. 2.

Listed sources of guidance for making diagnostic and treatment decisions (n=46).

Evidence and Context Scales of the ORCA

Summarized results from all sites

For all sites combined, there was a significant difference in mean Likert scores of the seven subscales (P < 0.0001), with the evidence subscale having the highest score (Table 1). The resources subscale was significantly lower than the other six subscales (P < 0.001). We also found a significant difference between the items within the resources subscale (P < 0.0001) (Table 2). ORCA scores for budget and staffing resources were significantly lower than scores for training and facility resources (P < 0.001 for both comparisons).

Table 1.

ORCA scores for all subscales by provider type

subscale	Overall, Mean (SD) n=104	Providers, Mean (SD) n=42*	Nurses, Mean (SD) n=20	Pharmacists, Mean (SD) n=42	P-value**
Evidence Assessment	4.0 (0.9)	4.1 (0.9)	3.7 (1.0)	4.0 (0.7)	0.18
Leader Culture	3.5 (9.0)	3.7 (0.7)	3.6 (1.1)	3.3 (0.8)	0.07
Staff Culture	3.8 (0.8)	4.0 (0.5)	3.5 (1.0)	3.7 (0.8)	0.02
Leadership Behavior	3.2 (0.9)	4.0 (0.5)	3.5 (1.0)	3.7 (0.8)	0.07
Measurement	3.2 (0.9)	3.4 (0.8)	3.1 (0.9)	3.1 (0.9)	0.14
Opinion Leaders	3.7 (0.7)	3.8 (0.7)	3.7 (0.8)	3.7 (0.6)	0.64
General Resources	2.8 (0.9)	2.8 (0.8)	2.9 (1.1)	2.4 (0.9)	0.55

Note. ORCA, Organizational Readiness to Change Assessment; SD, standard deviation.

^*Includes 17 Infectious Diseases physicians, 12 staff physicians, 5 nurse practitioners, 4 resident physicians, and 4 physician assistants.

^**P values refer to Kruskal-Wallis tests

Table 2.

ORCA scores for items of resources subscale by site and overall

subscale	Budget, Mean (SD)	Training, Mean (SD)	Facilities, Mean (SD)	Staffing, Mean (SD)	Overall, Mean (SD)
All Sites	2.6 (0.1)	3.0 (0.1)	3.0 (0.1)	2.4 (0.1)	2.8 (0.9)
Site 1 (n=14)	2.5 (1.1)	2.7 (1.1)	3.3 (0.8)	2.6 (0.8)	2.8 (0.7)
Site 2 (n=31)	2.8 (1.3)	3.0 (1.1)	3.2 (1.0)	2.6 (1.0)	2.9 (0.9)
Site 3 (n=34)	2.2 (0.9)	2.8 (1.1)	2.5 (1.1)	1.9 (1.0)	2.4 (0.9)
Site 4 (n=25)	3.0 (0.8)	3.4 (1.0)	3.3 (0.8)	2.8 (0.9)	3.2 (0.8)

Note. ORCA, Organizational Readiness to Change Assessment; SD, standard deviation.

Inter-professional differences in ORCA scores

The mean ORCA scores for staff culture were significantly different between healthcare professionals (P = 0.02). Pharmacists had significantly lower scores than providers for the staff culture subscale (P = 0.04). Itemized analyses revealed significantly lower scores for pharmacists than providers for all items of the staff culture subscale (personal responsibility, cooperation to improve care, willingness to innovate), except for receptiveness to change. For all other subscales, scores were similar between professional types (Table 1).

Inter-site differences in ORCA scores

Comparing subscales between sites, ORCA scores were significantly different for leadership behavior, measurement, and general resources (Figure 3). Site 4 had significantly higher scores for leadership behavior, measurement, and general resources. Site 3 had significantly lower scores (mean 2.4) than sites 2 and 4 for the resources subscale (Table 2). Comparing items within the general resources subscale for site 3, the scores for staffing (mean 1.9) were significantly lower than scores for training and facilities at site 3 (P < 0.001 and P = 0.004, respectively; Table 2).

Fig. 3.

Mean ORCA scores by site.

Note. ORCA, Organizational Readiness to Change Assessment; P Values refer to Kruskal Wallis test or one-way ANOVA comparing the mean scores between sites for each subscale.

§Pairwise comparisons of the leadership behavior subscale showed significant differences between site 2 and 4, and between site 3 and 4

*Pairwise comparisons of the measurement subscale showed significant differences between sites 3 and 4

¶Pairwise comparisons of the general resources responses showed significant differences between sites 2 and 3, and between sites 3 and 4

** Attitudes and behavior of opinion leaders that support practice change

Site Elements Questionnaire

Overall, the four site co-investigators reported similar resources for antibiotic stewardship on the Site Elements Questionnaire (Table 3). All sites except site 2 reported a well-functioning antimicrobial stewardship program, supported by resources such as pharmacists trained in antimicrobial stewardship. However, there were differences between sites in resource allocation. For example, site 3 reported 0.5 pharmacist FTE devoted to antibiotic stewardship, while all other sites reported 1.0 pharmacist FTE. Although all sites indicated that physicians have fully embraced antimicrobial stewardship, only 50% reported that senior leadership fully supported stewardship. All sites reported observing common drivers of inappropriate treatment of ASB, including the belief by physicians that asymptomatic bacteriuria should be treated. Most sites also observed that nurses, patients, and/or families request urine cultures or antimicrobials.

Table 3.

Site Elements Questionnaire

Item	Site 1	Site 2	Site 3	Site 4	%
Does your facility have a well-functioning antimicrobial stewardship program?	√	-	√	√	75
Does your facility have an infectious-disease trained physician?	√	√	√	√	100
Do you have full-time infection control practitioner in your facility?	√	√	√	√	100
Does your facility have a pharmacist trained in antimicrobial stewardship?	√	√	√	√	100
Does your facility produce and distribute an antibiogram that reports bacterial resistance profiles?	√	√	√	√	100
Do you have a committed physician champion for antimicrobial stewardship?	√	-	√	√	75
Physician FTE devoted to antibiotic stewardship	0.5	0.5	0.25	0.25	NA
Do you have a committed pharmacy champion for antimicrobial stewardship?	√	√	√	√	100
Pharmacist FTE devoted to antibiotic stewardship	1.0	1.0	0.5	1.0	NA
Have physicians fully embraced antimicrobial stewardship?	√	√	√	√	100
Has senior leadership fully supported antimicrobial stewardship?		√		√	50
Does your facility have any restricted antimicrobial agents, for which approval is required before prescription?	√	√	√	√	100
Do any of the pharmacists at your facility provide advice to providers about antimicrobial choice?	√		√	√	75
If yes, in which units:  • ICU  • Acute Care  • Community living centers	√ √ √		√ √ √	√ √ √	75 75 75
Does your facility offer any computer-based decision support about choosing antimicrobials?		√	√		50
The following questions relate to screening for and treatment of asymptomatic bacteriuria (ASB). Have you observed any of the following common drivers of inappropriate treatment of ASB?
• Nurses request urine culture or antimicrobials to treat urine	√	√	√		75
• Physicians believe asymptomatic bacteriuria should be treated	√	√	√	√	100
• Patient and/or family request urine culture or antimicrobials for a patient with ASB	√	√	√		75
• Urine cultures are performed in the emergency department without an appropriate indication	√	√	√	√	100

Note. FTE, full time equivalent; ICU, intensive care unit.

Discussion

At all intervention sites, the perceived strength of the evidence base for management of ASB was high. Despite buy-in to the evidence, respondents perceived limitations in the financial and staffing resources that are necessary to support practice changes in antibiotic stewardship. Compared to providers, pharmacists perceived the organizational culture of staff members to be less conducive to change. Comparing implementation sites, site 4 had higher organizational readiness in terms of leadership behavior, measurement, and resources. In contrast, respondents at site 3 perceived significantly limited general resources, particularly for staffing. Of note, site 3 also had the lowest FTE for pharmacists devoted to antibiotic stewardship.

Prior work showed that endorsing an evidence base does not always translate to evidence-based behavior in practice.^16,17 The open-ended question for evidence assesment revealed several interesting patterns. While over half of respondents listed the IDSA guidelines to aid in decisions, Up To Date and the ID team (pharmacist and consulting team) were also commonly referenced resources. Concordant with the guidelines, most respondents listed signs and symptoms as an indication to send a urine culture or start antimicrobials. However, nurses and CNAs frequently specified misleading symptoms as an indication to send cultures or start antibiotics, as has been reported in other studies, including our own.^17–21

Pharmacists reported significantly lower scores for multiple components of staff culture, including personal responsibility for improving outcomes, cooperation to improve patient care, and willingness to innovate. One hypothesis for this difference in perception of staff culture is that within the hierarchy of healthcare professionals, pharmacists feel less empowered to innovate and implement change. The CDC and the Joint Commission recommend that antibiotic stewardship programs appoint a single pharmacist leader to co-lead the program and work to improve antibiotic use.^22,23 Despite the critical role of pharmacists in antibiotic stewardship, qualitative studies revealed that pharmacists perceive multiple barriers to executing antimicrobial stewardship programs, including lack of on-site pharmacist availability, inadequate communication between pharmacists and prescribing providers, and push back from house-staff^24,25 These organizational issues may contribute to a staff culture that lacks cooperation and innovation to improve patient care.

Our finding that the general resources subscale had the lowest ORCA scores is in alignment with other previously published ORCA results.^8,10 Although the original PARIHS framework did not include resources, this subscale was included in the ORCA based on research that “slack” (abundant) resources facilitate implementation.²⁶ In our study, lower scores for general resources suggest that many facilities may require additional staffing and financial resources to successfully implement antimicrobial stewardship initiatives. Specifically healthcare professionals at site 3 perceived a lack of staffing support, consistent with the Site Elements Questionnaire in which site 3 had the lowest pharmacist FTE (0.5) and physician FTE (0.25) devoted to antibiotic stewardship.

We found that one site (site 4) had higher organizational readiness in terms of leadership (behavior and measurement) and resources. Effective change likely requires both. As discussed by Weiner et al, organizational readiness, which focuses on perceived efficacy to change, is more complex than organizational capacity, a measure of raw resource potential.⁷ Human and financial resources are necessary, but they may not be sufficient to produce change without clear leadership.²⁷ In other words, an organization may have abundant resources but lack the impetus, motivation, and leadership to mobilize those resources to implement change.^7,27 Through the identification of specific barriers to implementation, such as lack of support or resources, discrete strategies for implementation can be tailored and modified. Our support to the sites with lower leadership scores will include identification and training of local champions who are dedicated to supporting and implementing the intervention, as suggested by the Expert Recommendations for Implementing Change (ERIC) project.²⁸ Local leaders will be recruited, designated, and prepared for the change effort. Early adopters of the stewardship intervention will be identified to learn from their experiences with practice innovation. For sites with low ORCA scores for general resources, our targeted implementation strategies can include analyzing local needs, accessing new funding, and developing resource sharing agreements.²⁸

Our study has some limitations. The ORCA is one of at least 18 instruments that have been used to assess organizational readiness for change, and there is no current gold standard^29,30 We selected the ORCA based on its validation and utility in diverse healthcare settings.^8–11,27,31 The ORCA does not specifically explore intervention-specific capacity ata given site capacity was assessed in our study through the separate Site Elements Questionnaire. The current survey results apply to a specific subset of VA facilities and may not be generalizable to other VA facilities or non-VA, non-teaching hospitals. The goal of this study was not to provide a comprehensive assessment of organizational readiness, but rather to identify site-specific barriers to change that would enable targeted implementation of our antibiotic stewardship program. The sample size at each site was small, ranging from 14-34 per site. Although the overall response rate was 69%, the response rate ranged from 42-100% between sites. Respondents at sites with lower response rates may not be representative of all providers at that site. The distribution of healthcare professional types also varied between sites, with pharmacists representing 16.1-56.0% of respondents. However, this variation in job distribution does not explain inter-site differences in ORCA scores: sites 3 and 4 had the same percentage of pharmacist respondents (56% for both), but they had significantly different ORCA scores for three subscales. A strength of our study is that we targeted a wide array of VA professionals. The variation we found in perceptions across professional types that would be involved in the intervention speaks to the value of this level of data collection. Finally, for the open-ended question on indications to send urine culture, many respondents listed only “signs” and “symptoms,” and we were unable to distinguish between evidence-based symptoms of UTI (fever, dysuria, etc.) and misleading symptoms (foul smell, cloudy urine, change in urine color, etc.).

Conclusions

Evaluating organizational readiness for change is a critical prerequisite for tailored implementation of evidence-based practices. Our study demonstrated the utility of the ORCA for identifying barriers to change prior to implementation of an antibiotic stewardship intervention. Although healthcare professionals had strong buy-in to the evidence for Management of ASB, their perceived barriers to antibiotic stewardship included inadequate resources and lack of leadership support. These findings provide targets for tailoring the intervention to maximize success of the Less is More campaign. Given the national mandate for antibiotic stewardship programs, it is crucial that healthcare organizations invest the human and financial resources that are needed to decrease overtreatment of ASB.

Highlights.

• The ORCA survey measures organizational readiness to implement best practices.

• We used the ORCA to measure barriers to an antibiotic stewardship intervention.

• Providers had strong buy-in to the evidence for managing asymptomatic bacteriuria.

• Inadequate resources and support were cited as barriers to stewardship.