Virtual learning collaboratives to improve urine culturing and antibiotic prescribing in long-term care: controlled before-and-after study

Abstract

Background

Urine culturing practices are highly variable in long-term care and contribute to overprescribing of antibiotics for presumed urinary tract infections. The purpose of this study was to evaluate the use of virtual learning collaboratives to support long-term care homes in implementing a quality improvement programme focused on reducing unnecessary urine culturing and antibiotic overprescribing.

Methods

Over a 4-month period (May 2018–August 2018), 45 long-term care homes were self-selected from five regions to participate in virtual learning collaborative sessions, which provided an orientation to a quality improvement programme and guidance for implementation. A process evaluation complemented the use of a controlled before-and-after study with a propensity score matched control group (n=127) and a difference-in-difference analysis. Primary outcomes included rates of urine cultures performed and urinary antibiotic prescriptions. Secondary outcomes included rates of emergency department visits, hospital admission and mortality. An 18-month baseline period was compared with a 16-month postimplementation period with the use of administrative data sources.

Results

Rates of urine culturing and urinary antibiotic prescriptions per 1000 resident days decreased significantly more among long-term care homes that participated in learning collaboratives compared with matched controls (differential reductions of 19% and 13%, respectively, p<0.0001). There was no statistically significant changes to rates of emergency department visits, hospital admissions or mortality. These outcomes were observed with moderate adherence to the programme model.

Conclusions

Rates of urine culturing and urinary antibiotic prescriptions declined among long-term care homes that participated in a virtual learning collaborative to support implementation of a quality improvement programme. The results of this study have refined a model to scale this programme in long-term care.

Background

Phased implementation efforts are designed to extend the benefits of evidence-based interventions to impact system-level outcomes. As there can be a drop in effectiveness when interventions move from controlled research protocols into real-world settings,¹ there is a need to closely examine what supports contribute to successful implementation and how they can be replicated at scale. Intermediary organisations that scale evidence-based interventions are faced with the challenge of finding ways to package interventions and provide implementation support to extend benefits beyond a pilot stage.

The Urinary Tract Infection (UTI) Programme includes a focus on de-implementing low value practices that contribute to antibiotic overprescribing.² In long-term care, prescribing is often driven by positive urine culture results in the absence of indicated signs and symptoms of a UTI, also known as asymptomatic bacteriuria.³ It has been estimated that a substantial proportion of antibiotics prescribed in long-term care are unnecessary and the most common reason for this is the treatment of asymptomatic bacteriuria.^4–8 Ultimately, efforts to reduce inappropriate use of antibiotics can reduce antibiotic-related harms and improve resident health outcomes.^{4 9–11} A systematic review of antimicrobial stewardship strategies in long-term care found that UTIs were the most common area targeted to improve the quality of antibiotic use.¹² However, the review emphasised the need to assess lower resource-intensive sustainable interventions in the long-term care setting.

Previous studies of the UTI Programme and similar multimodal programmes have described experts delivering on-site support.^{13 14} These types of supports present an opportunity to build the capacity of organisations to effectively implement practice change; however, tailored onsite visits may not be feasible beyond a pilot stage due to resource intensity. To help scale the UTI Programme in Ontario, Canada, virtual learning collaboratives were selected as a means to maintain support for implementation while also bringing together multiple long-term care homes (LTCHs) to facilitate shared learning.

A learning collaborative has been defined as the facilitation of groups of providers or organisations and fostering a collaborative learning environment to improve implementation of a clinical innovation.¹⁵ Originally described by the Institute for Healthcare Improvement with a focus on quality improvement initiatives in healthcare systems,¹⁶ this strategy brings together teams, practitioners or organisations with common components including didactic learning sessions combined with group discussion and skill building activities.^{17 18} While learning collaboratives have been widely used, they have not been well studied,¹⁷ and as a result, there has been a call for studies to verify the effectiveness of learning collaboratives for improving implementation.¹⁸ There is also a need for studies examining virtual sessions, as the majority of previous studies on this strategy have described the use of face-to-face meetings.^16–18 Public health measures in response to coronavirus disease 2019 (COVID-19) have placed restrictions on in-person gatherings and access to LTCHs. To continue efforts to support the implementation of best practices, it is timely to examine the effectiveness of virtual options that can increase access to external supports and create connections between professionals working in this setting.

Considering past results demonstrating the effectiveness of multimodal interventions to improve practices in this area^{13 14} and what is known about the importance of providing implementation support, we postulated the following hypotheses: (i) participation in learning collaborative sessions would help maintain fidelity to the programme model and result in practice change and (ii) level of participation in the sessions and programme would be associated with better outcomes.

Accordingly, this study set out to evaluate the implementation and outcomes of a virtual learning collaborative to build programme and implementation-specific capacity among stakeholders in long-term care.

Methods

Reporting for this study followed the Transparent Reporting of Evaluations with Non-randomised Designs (TREND) statement (see online supplemental file 1).¹⁹

Design and setting

A quasi-experimental study design (longitudinal controlled before-and-after study) was used to assess the relationship between participating in the learning collaborative and subsequent impacts on practice change. Intervention facilities were self-selected and compared with matched control facilities that were passively exposed to provincial dissemination of electronic programme materials.

Participants

All LTCHs in Ontario were eligible for this intervention unless they had previously participated in a pilot study of the programme. Provincial administrative data sources provided the opportunity to establish the control group retrospectively.

A recruitment email was distributed in April 2018 to over 1600 contacts representing 620 LTCHs in Ontario. The email included electronic versions of programme materials. Contacts were prompted to contact the project team to further discuss interest and eligibility to receive additional implementation support.

Intervention

We described this intervention using the template for intervention description and replication checklist (see online supplemental file 2).²⁰

The UTI Programme draws attention to specific practices that contribute to antibiotic overprescribing. This includes two practices that need to be de-implemented: discontinuing the use of dipsticks to diagnose UTIs^{21 22} and discontinuing routine annual or admission screening, if residents do not have indicated clinical signs and symptoms.²³ This also includes three practices that need to be followed: obtaining urine cultures only when residents have indicated clinical signs and symptoms of a UTI^{24 25}; promoting proper collection and storage of urine cultures and prescribing antibiotics only when specified criteria have been met and reassessing therapy when urine culture and sensitivity results are received.^{25 26}

The UTI Programme also includes recommended strategies and resources to support the five practice changes described above.^{2 13} Materials include a clinical decision tool, designed for use with medically stable non-catheterised residents.²⁷ The programme promotes strategies that can be used early during the change process to build readiness for change.²⁸ Education and distribution of resources to frontline staff is a foundational strategy. Additional strategies are recommended to target known barriers to practice change including: coaching support; providing information to residents and families; improving documentation and communication of resident symptoms; monitoring progress with practice changes and delivering reminders. At this time of this study, all LTCHs in Ontario had access to electronic programme materials through Public Health Ontario’s (PHO) public facing website.²⁹

PHO made a commitment to scale this quality improvement programme within the province through the delivery of support for implementation. Virtual learning collaboratives were selected to promote and support LTCHs with implementation. PHO is an intermediary organisation that provides scientific evidence and expert guidance to support partners in government, public health and healthcare. PHO was well positioned to promote and support implementation of this programme with internal expertise in antimicrobial stewardship and the presence of five Regional Infection Prevention an Control (IPAC) Support Teams situated across the province. These teams support a diverse range of stakeholders in the healthcare setting including long-term care.

All LTCHs interested in receiving implementation support selected an implementation lead from their home who participated in a readiness discussion with staff from the Regional IPAC Support Team to confirm the need for the programme, consider timing and available human resources and to discuss next steps (eg, securing senior leadership support).

PHO hosted five separate online learning collaboratives between May 2018 and August 2018, bringing together groups of LTCHs from the same geographic region. Regional learning collaboratives included three sessions that were 1–1.5 hours in length and were facilitated by staff from the Regional IPAC Support Team. A lead from each LTCH, typically a director or associate director of care with infection control responsibilities was asked to participate in the sessions.

Session 1 included a presentation on the UTI Programme and an overview of the importance of implementation planning and forming an implementation team. There was also a facilitated discussion around steps implementation leads were taking at their LTCH to establish readiness for the programme and plan for implementation. Implementation leads shared who they would be including on an implementation team. Session 2 included a facilitated discussion to identify anticipated barriers that would be most important at their LTCH. Facilitators then provided guidance on which recommended strategies would help overcome these types of barriers. The final session included a facilitated discussion on the status and details of action plans for each LTCH. There was also a focused discussion on planning for sustainability and approaches to monitoring practice changes. For additional details and materials, see online supplemental file 2.

Process evaluation

We measured participation in the learning collaborative through attendance tracking to document whether or not at least one representative from each LTCH was present. Programme withdrawal was also documented. Withdrawal was defined as situations where a LTCH decided not to implement the programme after attending a learning collaborative session or where the implementation lead from a LTCH only attended one session with no further contact.

The 45 implementation leads from each LTCH who attended the learning collaborative sessions were asked to provide feedback on the learning collaborative and to report on how they implemented the programme. Implementation leads were directors of care, associate directors of care or registered nurses with infection control responsibilities. Online surveys were used to collect this information. Following the delivery of the three learning collaborative sessions, an online survey was administered in October 2018 (online supplemental file 3). A second follow-up survey was administered 7 months after the learning collaborative sessions (March 2019) to learn how each LTCH implemented the programme (online supplemental file 3). Invitations to complete the survey and two reminders were sent to implementation leads by email from the Regional IPAC Support Team who delivered the learning collaborative sessions.

Outcomes

Primary outcomes included the number of urine cultures performed and number of urinary antibiotic prescriptions per 1000 resident days. These outcomes were selected as the programme ultimately aims to reduce antibiotic prescribing for asymptomatic bacteriuria and places a strong focus on reducing unnecessary urine culturing. For the purposes of this study, urinary antibiotics were defined as: ciprofloxacin, norfloxacin, nitrofurantoin, fosfomycin or trimethoprim with or without sulfonamides/sulfamethoxazole. A secondary outcome was total antibiotic prescriptions per 1000 resident days to detect whether there was a shift from one class of antibiotics to another. We incorporated additional balancing measures to monitor any signal of undertreatment of UTIs including rates of emergency department visits, hospital admissions and mortality.¹⁴

Data sources

Data to support the outcome assessment was available from existing administrative databases at ICES (formerly, the Institute for Clinical Evaluative Sciences). ICES is an independent, non-profit institute whose legal status under the Ontario’s health information privacy law allows it to collect and analyse healthcare demographic data, without consent, for health system evaluation and improvement. These datasets are linked using unique encoded identifiers and analysed at ICES (see online supplemental file 4 for all databases used in this study). ICES databases go through routine data quality assessments based on a data quality framework.³⁰ The source of antibiotic prescribing data was from the Ontario Drug Benefit (ODB) database, which collects antibiotic dispensing data for patients over the age of 65 years and/or residents of LTCHs. Compared with manual chart review, ODB is associated with a 99.3% accuracy in determining drug dispensing.³¹

Sample size

We determined that a minimum of 32 LTCHs would be required to detect a statistically significant difference between treatment and control groups with 80% power and 5% significance for rates of urine cultures performed. This estimate was based on the following assumptions: an event rate difference of −1.0 per 1000 resident days, control event rate of 3.2 per 1000 resident days, an average home bed size of 160¹³ and an assumption that the coefficient of variation was 0.25. Using similar methods but with an event rate difference of −0.6 and control event rate of 1.5 per 1000 resident days, we determined that 22 LTCHs would be required for rates of urinary antibiotic prescriptions.¹³

Assignment method

We used propensity score matching to establish controls for this study using 1:3 nearest neighbour matching with a calliper size of 0.2. A logistic regression model was used to calculate propensity scores for all LTCHs in Ontario that met eligibility criteria. Scores were calculated based on predicted probability of being in the intervention group based on baseline rates of urine culturing and total antibiotic prescriptions in addition to bed size. LTCHs were eligible for inclusion in the control group if they had not previously participated in a pilot of the programme; had not closed during the study period or did not have counts of zero urine cultures over the full baseline period.

Analyses

The balance of baseline facility and resident level characteristics between intervention and control LTCHs was assessed with standardised differences.³² A threshold of ≤0.1 is recommended to consider differences between groups.³²

Outcome evaluation

We defined an 18-month baseline period (November 2016–April 2018), a 4-month implementation planning phase (wash-in period) (May 2018–August 2018) and a 16-month postmplementation phase (September 2018–December 2019). In the study protocol, the point of intervention was defined as the start of the postimplementation phase. It was expected that LTCHs would need time to participate in the learning collaborative and plan for implementation before seeing changes in practice.

A difference-in-difference analysis was used to examine changes in outcome variables.^{33 34} This estimate is the difference in average outcome in the intervention homes before and after implementation minus the difference in average outcome in the control homes before and after implementation. This method requires that baseline temporal trends are equivalent with the idea that trends in both groups would have continued to be similar in the absence of receiving support to implement this programme. A Poisson regression model was used with the unit of analysis as monthly outcome data for each facility over the 38-month study period, and random-intercepts corresponding to each home. Resulting effect estimates from the difference-in-difference analysis were used to calculate the relative difference in the decline.³⁴

The primary analysis focused on LTCHs that were recruited irrespective of whether they withdrew during the study period (an intention-to-treat analysis). There was no missing data for the outcome assessment. We conducted additional sensitivity analyses to examine how level of engagement with the programme could be diluting the intervention effect. We examined outcomes excluding LTCHs that withdrew (per-protocol analysis). The analysis was also replicated focusing only on the LTCHs that withdrew. We also separately examined intervention effects among those LTCHs that attended all three learning collaborative sessions. All analyses were performed using a commercial statistical software (Statistical Analysis System for Windows, V.9.4 using the SAS Enterprise Guide V.7.15, SAS Institute, Cary, North Carolina, USA).

Results

Recruitment

Following the recruitment notice, of the 102 LTCHs that responded, 45 agreed to attend the learning collaborative after participating in a readiness assessment and 57 LTCHs declined enrolment (figure 1). Twenty-five LTCHs that declined enrolment expressed interest in a future wave. The remaining 32 LTCHs did not enrol due to either: competing priorities, decided to implement without the learning collaborative or there was no response following an initial email of interest. LTCHs that declined enrollment were eligible for the control group.

Figure 1.

Description of sample, recruitment and allocation results.

Process evaluation

Of the 45 LTCHs that signed up for the programme, 13 withdrew. Only 16 LTCHs (36%) attended all three sessions. Eighty-four per cent of LTCHs attended the first session, 64% attended session two and 49% attended session three. Among LTCHs that submitted the first survey (n=27/45), 89% (n=24/27) indicated they felt more prepared for implementation after participating in the learning collaborative. Thirty-seven per cent (n=10/27) felt that LTCHs would need additional support after the sessions. While the majority agreed (n=20/27) that the peer support component was valuable, 26% disagreed or felt neutral about this component (online supplemental file 2).

There were 17 comments shared by implementation leads who had attended the learning collaborative sessions (see online supplemental file 3). Positive feedback was received on the quality of programme resources, facilitation support and the peer support component. Recommendations for improvement focused on timing of the sessions considering LTCHs may need to implement the programme at a different pace. For example, one participant noted: “The issue was that our home rolled out the programme fairly quickly then the collaborative was still discussing steps we had already completed”. Participants also noted that additional interactions between PHO staff and LTCH staff may have been beneficial to increase engagement and buy-in.

Eleven implementation strategies are described in the UTI Programme. Table 1 includes a summary of implementation strategies that were reported as adopted by implementation leads representing the 32 LTCHs that participated in the learning collaborative and implemented the programme (ie, did not withdraw). Twenty-nine of the 30 LTCHs that submitted the final survey reported using at least one recommended strategy and on average reported using seven strategies. Six LTCHs reported using all of the 11 recommended strategies. The most common types of strategies selected were educational including providing education to staff (84%), coaching (78%) and delivering education to families and residents (78%) (table 1). There was lower uptake of strategies designed to build readiness for implementation and notably use of implementation teams (47%) (table 1).

Table 1.

Adoption of recommended implementation strategies among participating long-term care homes (n=32)

Implementation strategy description	Complete n (%)	Incomplete* n (%)	No response† n (%)
Readiness strategies
Consensus: ensuring agreement on the practices among prescribers	24 (75)	5 (16)	3 (9)
Policies and procedures: ensuring policies and procedures are aligned with five practice changes	23 (72)	6 (19)	3 (9)
Champion: selecting a champion and involving them in implementation	21 (66)	8 (25)	3 (9)
Local opinion leaders: finding way for prescribers to be involved in delivering strategies	19 (59)	10 (31)	3 (9)
Implementation team: having at least three people that are part of an implementation team	15 (47)	14 (44)	3 (9)
Education and support
Delivering education to staff	27 (84)	1 (3)	4 (13)
Coaching: ensuring there is someone to help with questions/concerns after education	25 (78)	3 (9)	4 (13)
Providing information/education to residents and families	25 (78)	3 (9)	4 (13)
Monitoring and communication
Resident symptoms: addressing improvement in how resident symptoms are documented and communicated	26 (81)	1 (3)	5 (16)
Reminders: looking for opportunities to remind staff about the practice changes	23 (72)	4 (13)	5 (16)
Process surveillance: checking records of urine cultures and antibiotics prescribed for UTIs to see how they are aligned with the programme algorithm	22 (69)	5 (16)	5 (16)

*Incomplete refers to LTCHs that indicated they did not complete this strategy.

†No response refers to LTCHs that did not complete the survey or question. Proportions reflect the total number of participating LTCHs and not the total number of LTCHs that responded to the survey.

LTCHs, long-term care homes; UTI, urinary tract infection.

Matching results

A total of 43 intervention LTCHs were matched to 127 controls (figure 1). Two intervention facilities could not be matched due to their very large bed size. The resulting sample provided diversity in facility-level characteristics including a mix of urban and rural locations and variation in size (table 2). Resident-level characteristics including demographics, functional status and device use were similar between intervention LTCHs and matched controls (table 2). Baseline rates for the primary outcomes were also similar between the two groups; however, there was a small difference in the baseline rate of urine cultures performed (table 2).

Table 2.

Baseline characteristics of residents and long-term care homes (LTCHs)

Characteristics	Matched intervention LTCHs (n=43)	Matched control LTCHs (n=127)	Standardised differences*
Bed size†, mean (SD)	128.3 (70.8)	136.2 (76.8)	−0.11
Region
Urban (%)	33 (77)	106 (84)	−0.17
Rural (%)	10 (23)	21 (17)
Mean rate at baseline per 1000 resident days
Urine culturing† (IQR)	4.0 (2.5–5.5)	3.4 (2.2–4.9)	0.15
Total antibiotic prescriptions† (IQR)	4.2 (3.7–5.1)	4.4 (3.5–5.2)	0.05
Urinary antibiotic prescriptions (IQR)	1.5 (1.2–2.0)	1.4 (1.1–1.8)	0.08
Hospital admissions (IQR)	1.7 (1.4–2.3)	2.0 (1.6–2.5)	−0.40
No. unique residents	7960	24 827
Mean resident age (SD)	85.17 (8.1)	84.79 (8)	0.05
Female proportion (%)	5515 (69)	17 023 (69)	0.02
Comorbidities, n (%):
Residents with severe cognitive impairment‡	2857 (36)	8818 (36)	0.01
Dementia and Alzheimer’s disease	5423 (68)	16 713 (67)	0.02
Paralysis	14 (0.2)	110 (0.4)	−0.05
Parkinson’s disease	550 (7)	1649 (7)	0.01
Obstructive lung disease	1510 (19)	4475 (18)	0.02
Cancer	768 (10)	2812 (11)	−0.06
Gastrointestinal disease	2546 (32)	7055 (28)	0.08
Liver disease	94 (1)	365 (2)	−0.03
Diabetes mellitus	2229 (28)	7173 (29)	−0.02
Renal disease	775 (10)	2815 (11)	−0.05
Atherosclerotic heart disease	1367 (17)	4172 (17)	0.01
Congestive heart failure	1156 (15)	3210 (13)	0.05
Peripheral vascular disease	555 (7)	1521 (6)	0.03
Stroke	2055 (26)	6392 (26)	0.002
Functional status, n (%)
Bladder incontinence	7337 (92)	22 412 (90)	0.07
Bowel incontinence	6371 (80)	19 513 (79)	0.04
Devices, n (%)
Indwelling catheter	582 (7)	1526 (6)	0.05
Other urinary catheter	40 (0.5)	212 (0.9)	−0.04
Feeding tube	56 (0.7)	241 (1.0)	−0.03

*A threshold of ≤0.1 is recommended to consider differences between groups.

†Variables used to match intervention to control LTCHs.

‡Cognitive Performance Scale ≥4.

Urine cultures

Figure 2A graphs 3-month moving averages for rates of urine cultures performed per 1000 resident days over the study period for matched intervention and control LTCHs. Rates of urine cultures performed per 1000 resident days started to decline early during the implementation planning phase when learning collaboratives were hosted by PHO. For the full cohort of LTCHs recruited (intention-to-treat analysis) (n=45), between the 18-month baseline period and the 16-month postimplementation period, the difference-in-difference analysis showed that the change in the rate of urine cultures performed was 19% lower [(1–exp (−0.21)] among LTCHs that participated in the learning collaboratives than the control group (p<0.0001) (table 3).

Figure 2.

Rates of urine culturing, urinary antibiotic prescriptions and total antibiotic prescriptions per 1000 resident days (RD) for intervention and matched control long-term care homes (LTCHs), November 2016–December 2019.

Table 3.

Baseline and postimplementation rates in the intervention and control facilities and difference-in-difference estimates for the intention-to-treat analysis

Outcome	Control (n=127)		Intervention (n=43)		Difference-in-difference
	Before mean (SD)	After mean (SD)	Before mean (SD)	After mean (SD)	Estimate†	SE	95% CI
Urine culturing	3.8 (2.2)	3.4 (2.0)	4.1 (1.9)	3.1 (2.1)	−0.21**	0.02	−0.24 to to 0.17
Urinary antibiotics	1.5 (0.7)	1.0 (0.5)	1.6 (0.7)	0.9 (0.5)	−0.13**	0.03	−0.19 to to 0.07
Total antibiotics	4.4 (1.1)	3.6 (1.2)	4.4 (1.2)	3.5 (1.1)	−0.05*	0.02	−0.08 to to 0.01
Acute care admission	2.1 (0.8)	2.0 (0.7)	1.8 (0.6)	1.8 (0.7)	0.01	0.02	−0.03 to 0.06
Mortality	1.8 (0.6)	1.9 (0.7)	1.9 (0.4)	2.1 (0.6)	0.01	0.04	−0.06 to 0.09

*P<0.01; **p<0.0001.

†From the difference-in-difference estimate (Poisson random-effects model). Before=baseline period from November 2016–April 2018. After=postimplementation period from September 2018–December 2019.

Prescriptions

Figure 2 also graphs 3-month moving averages for urinary antibiotic prescriptions (figure 2C) and total antibiotic prescriptions (figure 2E) per 1000 resident days over the study period for matched intervention and control LTCHs. There was a steady decline in rates of urinary antibiotic prescriptions in both groups during the study period; however, the decline was more pronounced for LTCHs in the intervention group. For the full cohort of LTCHs recruited (intention-to-treat analysis) (n=45), the change in the rate of urinary antibiotics prescribed was 13% lower [(1–exp (−0.13)] among LTCHs that participated in the learning collaborative than the control group (p<0.0001) (table 3). The change in the rate of total antibiotic prescriptions was 5% lower [(1–exp (−0.05)] among intervention LTCHs (p<0.01) (table 3).

Secondary outcomes

There were no signs of undertreatment of UTIs with no significant changes in rates of mortality or rates of emergency department visits and hospital admissions (table 3).

Sensitivity analysis

For the per-protocol analysis that excluded LTCHs that withdrew (did not go on to implement the programme) (n=30), reductions were even more substantial (figure 2B, D, F). The change in the rate of urine cultures performed was 28% lower [1–exp (−0.33)] among LTCHs that participated in the learning collaborative and went on to implement the programme compared with LTCHs in the control group (p<0.0001) (online supplemental file 5). The change in rate of urinary antibiotics prescribed was 20% lower [1–exp (−0.22)] (p<0.0001) and 6% lower [1–exp (−0.06)] for total antibiotic prescriptions (p<0.01) (online supplemental file 5). LTCHs that withdrew during the study period (n=13), did not see significant change in primary and secondary outcomes (online supplemental file 4). Finally, restricting the sample to LTCHs that attended all scheduled learning collaboratives (n=16) produced similar outcomes as the per-protocol analysis for rates of urine culturing and urinary antibiotic prescriptions (differential reductions of 34% and 19%, respectively, p<0.0001) (online supplemental file 5).

Discussion

The results of the difference-in-difference analysis provided support to the study hypothesis that participation in the learning collaborative sessions would result in practice changes targeted by the UTI Programme. There was a statistically significant decline in both rates of urine cultures performed and urinary antibiotics prescribed between the baseline and postimplementation period among LTCHs that participated in the intervention, after accounting for declines in the control group. The rate of total antibiotic prescriptions also declined during the study period. These findings are further supported by a per-protocol analysis that showed more substantial changes among LTCHs that did not withdraw. The results also support the hypothesis that the uptake of evidence-based practices to reduce antibiotic overprescribing for asymptomatic bacteriuria would not result in signs of undertreatment including changes in rates of emergency department visits, hospital admissions and mortality.^{14 35}

To some extent, this study demonstrates an association between outcomes and the level of participation in the sessions and programme. There were no significant changes in primary outcomes among LTCHs that did not go on to implement the programme; however, LTCHs reduced rates of urine culturing and urinary antibiotic use through variable adoption of recommended strategies. This aligns with an important assumption made in the development of this programme where LTCHs may encounter different barriers to practice change and thus may require different types of implementation strategies.² A sensitivity analysis showed that those attending all scheduled sessions did not necessarily have better outcomes. This further supports the idea that different LTCHs may require tailored support, dependent on their needs.

Overall trends show declines in both primary outcomes over the observation period for both intervention and control groups. It is important to note that an email promoting the UTI Programme went out to all LTCHs in Ontario during the recruitment phase for this intervention with links to programme materials. Incorporating the difference-in-difference analysis was an important component of the study design to measure the added value of learning collaborative sessions in the context of many other regional and national antimicrobial stewardship campaigns that were launched during the study period.

Implications

Results are consistent with studies on multimodal improvement programmes in this area.^{13 14} However, in previous studies, external experts came on-site to support implementation. The findings from this study have demonstrated the value of virtual sessions with peer organisations for knowledge exchange and implementation support in long-term care. The fact that these sessions were virtual made the scale-up of this model more feasible. This study aligns with promising findings from a few emerging studies looking at the value of learning collaboratives for supporting practice change across disciplines and areas of clinical practice improvement.^36–40

Based on the results of this study, a refined virtual implementation support model for the scale-up of this programme was developed including more tailored support for LTCHs. Engaging LTCHs in multisession learning collaboratives could be better facilitated through shorter sessions; more flexible scheduling; encouraging homes to plan ahead for staff transitions and providing additional resources for antibiotic stewardship.

Generalisability

Sustained outcomes were achieved with virtual learning collaboratives that were delivered without added travel or additional funding and that relied on existing resources. Outcomes were also observed with moderate adherence to recommended implementation strategies. Considering generalisability of the findings, a large number of LTCHs with diverse facility-level characteristics (eg, urban and rural locations and variation in size) were involved in this study. However, the recruitment process, documentation of challenges to implementation and reasons for having to withdraw during the study period demonstrate the importance of using strategies that strengthen readiness for implementation in order to replicate outcomes of this study.

Limitations

These results need to be interpreted in light of a few limitations. Use of the clinical decision-making tool and process surveillance forms were not assessed which limited opportunities to examine more precisely changes in the proportion of urine cultures performed and antibiotics prescribed in the absence of indicated signs and symptoms of a UTI. Efforts were made to capture antibiotic prescriptions commonly used to treat UTIs; however, some prescriptions meant for other indications may have been included. While the outcomes were assessed with knowledge of LTCH’s group allocation, the use of administrative data sources for the primary and secondary outcomes were objective.

The overall impact of the programme was modest and the sensitivity analyses demonstrated that impact could have been enhanced if fewer LTCHs had to withdraw from the programme. There are a number of challenges to implementing antibiotic stewardship in LTCH considering available resources in this area.

This study used a non-randomised study design for operational reasons and while efforts were made to match intervention facilities and compare across a number of facility and resident-level characteristics, there may be additional unmeasured differences between intervention and control facilities. Notably, there could be differences in baseline readiness and other contextual factors that contributed to LTCHs ability to improve practices in this area. However, it is important to point out that the UTI Programme was designed for LTCHs that need to build readiness and overcome barriers to practice change. LTCHs with more resources, experience with implementing quality improvement initiatives and with fewer barriers may be more likely to implement practice changes independently from the learning collaboratives. To further investigate the contribution of virtual learning collaboratives in improving antibiotic use in long-term care, future research could potentially randomise LTCHs to either learning collaboratives or a less intensive support strategy or randomise the start of learning collaboratives to different time intervals.⁴¹

While efforts were made to measure components of implementation quality for both the learning collaborative sessions and UTI Programme through attendance tracking and surveys, additional interviews with more staff from each LTCH could provide more information about how well the learning collaborative and UTI Programme were implemented.

Conclusion

Rates of urine culturing and urinary antibiotic prescriptions declined among LTCHs that participated in a virtual learning collaborative to support implementation of a quality improvement programme. There was positive feedback on the sessions in addition to the intervention package. However, it was evident that not all LTCHs required the same level of support and faced challenges attending multiple group sessions. The scale-up of this programme would benefit from more tailored support and guidance on building readiness for practice change.

Data availability statement

Data are available on reasonable request. Data may be obtained from a third party and are not publicly available. Survey data for the process measures is available on reasonable request. The dataset from this study is held securely in coded form at ICES. While data sharing agreements prohibit ICES from making the dataset publicly available, access may be granted to those who meet prespecified criteria for confidential access, available at www.ices.on.ca/DAS. The full dataset creation plan and underlying analytic code are available from the authors on request, understanding that the computer programs may rely on coding templates or macros that are unique to ICES and are therefore either inaccessible or may require modification.

Ethics statements

Patient consent for publication

Not required.