Effectiveness of quality improvement collaboratives in improving clinical processes and patient outcomes in stroke care

Abstract

Stroke remains one of the leading causes of death worldwide. In order to tackle the negative impacts of stroke, a high standard of clinical practice and a commitment to continuous quality improvement is needed across the stroke care pathway. One approach to quality improvement is the formation and implementation of quality improvement collaboratives (QIC’s). However, there are several barriers to the implementation of a QIC for stroke care which may impact on their success. This article critically appraises a systematic review which assessed the effectiveness of QIC’s for driving improvements in stroke care and explored the barriers to implementing a QIC’s to improve care.

Introduction

Stroke remains one of the leading causes of death worldwide with approximately 5.5 million deaths attributed annually to strokes (Collaborators 2021). In the United Kingdom, the estimated annual aggregated societal costs associated with stroke are approximately 26 billion pounds (Patel et al. 2020). The largest proportion of these expenses are driven by costs associated with patient survival following a stroke (e.g., unpaid care and lost productivity) due to a greater number of stroke survivors compared to new incidence of stroke (Feigin et al. 2022; Patel et al. 2020). In order to tackle the negative impacts of stroke, a high standard of clinical practice and a commitment to continuous quality improvement is needed across the stroke care pathway (Langhorne et al. 2020).

One approach to quality improvement (QI) is the formation and implementation of quality improvement collaboratives (QIC’s) (Devi et al. 2021). QIC’s are designed to support improvement in a specific area (e.g., stroke care) and typically involve bringing together different departments or organisations to engage in collaborative QI activities (Zamboni et al. 2020). Previous research has reported improvements in clinical knowledge, problem solving skills, and teamworking from QIC’s, but has yet to identify if contextual factors or clinical population impact on their success (Zamboni et al. 2020). Stakeholder involvement, particularly relating to patient populations, has been widely accepted as an important component for improving clinical practice (e.g., patient feedback, public adviser involvement), however no review of QICs has examined the extent to which patients have been involved (Sjølie et al. 2020).

The systematic review by Lowther et al, assessed the effectiveness of QIC’s for driving improvements in stroke care and used the Consolidated Framework for Implementation Research (CFIR) (Damschroder et al. 2009) to explore the facilitators and barriers to using a QIC to improve care (Lowther et al. 2021). The review also sought to consider the extent to which QIC’s in stroke care have engaged patients and carers, and considered health inequalities to improve clinical practice (Lowther et al. 2021).

This commentary aims to critically appraise the methods used within the review by Lowther et al, and expand upon the findings in the context of clinical practice (Lowther et al. 2021).

Methods

The systematic review undertook comprehensive literature searches of five databases (MEDLINE, CINAHL, EMBASE, PsycINFO, Cochrane Library) from date of inception through to June 2020 (Lowther et al. 2021). Database searches were supplemented by citation chaining and scoping searches of grey literature (Lowther et al. 2021). Only studies which described the implementation of a QIC in an adult stroke care setting, reported primary effect measures and/or perspectives of participating multidisciplinary stroke teams, and published in English, were included. Title, abstract, and full paper screening were undertaken by two reviewers independently. Data extraction was undertaken by a single reviewer and verified by a second reviewer. The Mixed Methods Appraisal Tool (MMAT) (Hong et al. 2018) was used to assess the quality of the included studies.

In the review, a vote counting method was used to assess effectiveness by categorising study outcomes according to the effect (positive, negative or no change) measured after the QIC (compared with baseline or control group). Multiple subgroup analyses were carried out on the factors of publication year, country, study setting, number of improvement areas, duration of QIC, number of learning sessions, and quality assessment judgement. The CFIR was used to categorise themes of facilitators and barriers (Damschroder et al. 2009); this synthesis was divided equally between two reviewers and reviewed together.

Results

A total of 815 citations were screened, of which 20 records were included in the review. Of the 20 studies, four were randomised controlled trials, four were cross-sectional studies, three were interrupted time series studies, four were before-and-after studies, and two were qualitative studies. A total of 17 studies were judged by the MMAT (Hong et al. 2018) to be of medium or high quality. The three studies judged to be of low quality included one cross sectional study, one interrupted time series study, and one before-and-after study. Studies were largely conducted in the United States of America and focused on improving urgent and/or acute stroke care. QIC’s took place in secondary care settings (e.g., hospitals), pre-hospital care environments (e.g., emergency services), and primary care settings (e.g., general practice). A range of organisations participated in the QIC’s, which included QI experts, doctors, nurses, health professionals, and practice managers (Lowther et al. 2021).

The studies included in the review assessed the effectiveness of the QIC interventions using three outcome types: clinical processes in stroke care (e.g., door-to-needle times, blood glucose testing, discharge prescriptions), patient outcomes (e.g., mortality, quality of life) and other outcomes (e.g., use of QI methods, perceptions of the QIC intervention, staff engagement). The review identified 14 studies whereby QIC’s resulted in a positive directional change in the clinical processes of stroke care (ranging from 50% to 100% of reported process outcomes). Of these 14 studies, five reported a positive directional change in 100% of the clinical process outcomes. Each of the 14 studies were critically appraised, of which 11 were scored as moderate to high quality with 3 being scored as low quality. The findings relating to patient outcomes showed that QIC’s were less effective at improving stroke patient outcomes. Just three studies reported a positive directional change in 100% of patient outcomes, two reported a positive directional change in less than half of their patient outcomes, and two reported no change. The review also found little evidence of effectiveness for QIC’s in improving other outcomes such as staff engagement and use of QI methods in clinical practice. Only two studies reported a positive directional change in one or more outcomes related to staff engagement and use of QI methods. Five studies reported no change in these outcomes. Notably, the evidence of effectiveness may be limited because of low methodological quality of some studies and heterogeneity of study design (Lowther et al. 2021).

Having synthesised the effectiveness of QIC’s, the review used the CFIR (Damschroder et al. 2009) to identify facilitators and barriers to implementing a QIC intervention to improve stroke care. Facilitators to QIC adoption included; positive staff perceptions of improvement, collaborative networks, well-organised stroke services, staff retention, shared goals, positive baseline staff performance, positive feedback mechanisms, access to teaching from experts, engaging senior leadership, sufficient resources, stroke services with less knowledge of QI, accessibility to learning sessions, positive perceptions of QIC, high levels of staff motivation, external partnerships, consistency in adopting QIC approaches, structured project approaches, monthly monitoring, and encouraging team dynamics (Lowther et al. 2021). Barriers to implementing a QIC included; a lack of external support, complexity of the intervention, changes to stroke services, staff turnover, lack of an innovative culture, perceived lack of incentives, unsupportive leadership teams, insufficient resources, limited access to patient data, lack of staff motivation, low levels of engagement, delays in employing QIC approaches, and perceptions that QI was unsustainable (Lowther et al. 2021).

The review highlighted that QIC’s rarely involve patients and carers at any stage of intervention, and rarely consider the context of health inequalities. No study included in the review conducted a service evaluation to assess if patient or carer experience had improved as a result of a QIC intervention (Lowther et al. 2021).

Commentary

The review met 10 of 11 criteria of the Joanna Briggs Critical Appraisal Checklist for Systematic Reviews and Evidence Synthesis (Tricco et al. 2022). The one criterion lacking related to the need for a detailed assessment of the likelihood and magnitude of the impact of publication bias (e.g., a funnel plot). However, due to the method of analysis, a funnel plot assessment may have not been appropriate due to the wide variation in study design and outcome measures. This wide variation in study design and outcome measures meant that the review could only synthesise the results using a vote counting method to demonstrate the direction of effect (e.g., positive directional change after the QIC). As a consequence, the review was limited in that it did not provide any information about the magnitude of the effect (Lowther et al. 2021). In addition, whilst clearly reported in the review, the methodological differences of the included studies (e.g., experimental, observational) were not accounted for in the synthesis. Limitations aside, the review provides a comprehensive narrative synthesis of findings relating to the 20 studies that explored the effectiveness of QIC’s in stroke care (Lowther et al. 2021).

The review highlights several key factors that may influence the success of QI initiatives and interventions in stroke care (Lowther et al. 2021). Firstly, the scope of the quality improvement initiative is an important factor to success with large-scale projects involving more complex change (or where the longer-term impact is on patient outcome), are less likely to be successful. In view of this, QIC’s may be preferred to other interventions because they provide shorter-term support with evidence of modest improvement within clinical processes in stroke care. However, QIC’s may not result in a positive directional change in outcomes related to stroke patient outcomes, staff engagement and use of QI methods in stroke care. As such, QICs may not be the most effective intervention aimed at improving patient focused outcomes.

The review identified several barriers and facilitators to implementing a QIC intervention to improve stroke care. One of the key barriers to success was a lack of access to appropriate data. For a QIC intervention to be effective, QI practitioners need to ensure that the measures of quality improvement have been identified and the sources of data are precise. A key facilitator to the success of a QIC was identified as having appropriate engagement from informatics teams and local information systems experts which should be established at the start of a QI initiative. With these collaborations in place, initiatives should ensure data sets are comparable across the QIC. Utilising opportunities to network and communicate with other departments and organisations may also facilitate the development of QI methods within an organisation, and build the capacity of staff (e.g., skills and competencies) to improve clinical practice (Zamboni et al. 2020). Development of organisational capacity (specifically for QI) should be established early within the initiative to ensure longer term success. Similarly, from the outset, organisational leadership should engage with QI and adopt practises endorsed by QIC’s to facilitate system level improvement in stroke care.

In addition to organisational changes, the formation of long-term regional QI support within established clinical networks may support the spread and adoption of improvement initiatives and take into consideration local population needs and health inequalities. Whilst this multidisciplinary collaborative approach for stroke care may make QIC’s more challenging to execute, it is a clinical area that may benefit from the collaboration and co-operation of teams being part of a complex clinical care pathway (Langhorne et al. 2020).

Future research in this area would benefit from a greater focus on improving system-level changes (e.g., reorganising stroke services) for larger impact, given that most QIC’s included in the review focused on improving smaller-scale clinical processes (e.g., screening rates, treatment rates). The lack of evidence from high quality randomised controlled trials prevents clinicians from establishing to what extent a QIC intervention is effective at improving system-level processes. Future investigation could specifically concentrate efforts on assessing if QIC’s could be effective in achieving larger scale changes and attempt to explore the lack of impact on patient outcomes despite the theoretical process improvement.

A limitation of the review was that it did not directly explore the influence of the facilitators and barriers on QIC effectiveness. Further research in the form of experimental studies should assess outcomes over a longer period to clarify if sustained change is achievable and if patient-related outcomes can be influenced; along with a wider exploration of the association of the facilitators and barriers with effectiveness to ascertain which factors are most important.

As none of the QIC’s included in the review involved stroke patients and carers, further research is also needed to explore how their experience and knowledge of stroke care could contribute and be evaluated as part of the implementation of a QIC (Lowther et al. 2021). This research could maximise opportunities to improve aspects of care that patients and carers value as important, but also identify the influence of improvements on patient and carer experience. The review identified a lack of consideration of health inequalities despite being a relevant issue in stroke care, and so future evaluations of QIC’s could consider whether there are any disproportionate impacts on patients as a result of the changes implemented (Lowther et al. 2021).

CPD Reflective Questions

• What are the main strengths and weaknesses of the systematic review?

• What barriers need to be overcome before implementing a QIC to improve stroke care?

• What are the key facilitators that have the potential to improve the success of QIC implementation in stroke care?

Key Points.

• There is some evidence that QIC’s improve clinical processes in stroke care, but there is less evidence that they improve patient outcomes.

• There are a wide range of facilitators and barriers to implementing QICs in stroke care.

• Future research is needed to explore the sustainability of improvements in stroke care once a QIC has concluded.

• QIC’s rarely involve patients and carers at any stage of intervention, and rarely consider the context of health inequalities.

Table 1. Summary of the key facilitators and barriers.

CFIR Domain	Facilitators	Barriers
Intervention characteristics	Care processes located in a specific geographical area or dedicated team	Complexity of the intervention or clinical population
Outer setting	Collaborative networks	Lack of external support Undermined inter-organisational collaboration
Inner setting	Well-organised stroke services Staff retention Shared goals Positive baseline staff performance Positive feedback mechanisms Access to teaching from experts Engaging senior leadership Sufficient resources Stroke services with less knowledge of QI methods Accessible learning sessions	Changes to stroke services Staff turnover Lacking an innovative culture Conflicting priorities Perceived lack of incentives Unsupportive leadership teams Insufficient resources Limited access to patient data
Individual characteristics	Positive perceptions of QIC High levels of staff motivation	Lack of staff motivation
Process	External partnerships Consistency in adopting QIC approaches Structured project approaches Monthly monitoring data Encouraging team dynamics	Low levels of engagement Delays in employing QIC approaches Perceived unsustainability