Scoping review study to identify how communities in the USA, Australia, New Zealand and Canada use quality improvement (QI) approaches to address community health and well-being

Abstract

Introduction

Both US and global communities lag on key health indicators. There has been limited progress in building capacity to improve health beyond the healthcare field. Yet, communities also need to engage in health improvement initiatives. A substantial body of literature describes standards and core components for quality improvement (QI) approaches in clinical settings. This study aims to determine how communities in the USA, Australia, New Zealand and Canada use QI approaches for health and well-being improvement and how such approaches compare to those in clinical settings.

Methods and analysis

We developed a study protocol based on scoping review framework by Arksey and O’Malley, methodological advancements for scoping studies (Levac et al) and other published protocols. We developed research questions in an iterative process and used the Population, Intervention, Comparison, Outcomes strategy to determine eligibility criteria. Electronic databases deemed appropriate (Web of Science, Scopus, and Proquest Health Management) will be searched for studies that meet inclusion criteria. References of included studies will be included when relevant. Two reviewers will independently screen all abstracts and full-text studies for inclusion. A third reviewer will adjudicate disagreements that arise. An instrument will be developed to extract data from included studies. Quantitative and qualitative results will be reported.

Ethics and dissemination

We developed this protocol to systematically conduct a scoping review of how US communities use QI approaches to address community health and well-being. Results will benefit multiple stakeholders by informing how to better support, design and evaluate community well-being improvement interventions. Results will be distributed through peer-reviewed journals, conferences, presentations and a public health graduate course.

Strengths and limitations of this study.

• While much is known about how clinical settings use quality improvement (QI) approaches, this scoping review will explore how QI approaches have been used to drive health and well-being improvements in community settings and how the approaches differ between these settings.

• Study results will inform a wide array of stakeholders including funders, researchers, practitioners and community members on how to better support, design and evaluate community well-being improvement interventions.

• No formal meta-analysis or analysis of the quality of the results will be done because the goal of this scoping review is to give an overview of what evidence exists and identify gaps.

• The consultation stage will not be performed, which limits stakeholder input on the other stages of the scoping review.

Introduction

Background

In our increasingly complex world, improving health and well-being for all has never been more important. WHO has defined health in terms of well-being for the past 70 years,¹ and the third Sustainable Development Goal (SDG) to ensure healthy lives and promote well-being for all has reinforced this idea.² In the USA, a foundational principle of the Healthy People 2030 framework is that promoting the nation’s health and well-being is a shared responsibility across national, state, tribal and community levels.³ Foundations such as the Robert Wood Johnson Foundation (RWJF) have adopted the creation of healthier communities as part of their action framework for creating a culture of health.⁴

Despite these major organisations’ focus on community well-being, progress in building capacity to improve health beyond the healthcare field has been limited. Both US and global communities lag on key indicators of health. The 2017 Global Burden of Disease study reported that most countries will fail to meet the SDG targets of reducing deaths from non-communicable diseases. The chance of any country reducing overweight in children aged 2–4 years is <5%.⁵ The US lags behind other wealthy countries in life expectancy, infant mortality and obesity.³ To address these issues, communities need to actively engage in health and well-being improvement initiatives and have the tools and capabilities to implement these improvement initiatives in a systematic way.

Study rationale and conceptual framework

This study explores the extent to which communities have used quality improvement (QI) methods to address well-being in their settings. We define a community as a group of people with diverse characteristics who are linked by social ties, share common perspectives and engage in joint action in geographical locations or settings.⁶ We adopt the definition by Batalden and Davidoff⁷ that QI is a systematic approach to improve outcomes and systems by building the capability of communities to identify, prioritise and develop solutions to local systems problems. There are several popular QI methods such as the Institute for Healthcare Improvement’s Model for Improvement, Lean and Six Sigma, but in essence, each method focuses on mapping care delivery processes and systems, measuring the quality problem using data, identifying root causes for the problem, developing and implementing change strategies to address the problem and measuring the impact of the change.⁸ While the extent to which these methods can be causally attributed to improved outcomes in healthcare settings is still debated,⁹ there is a broad belief that if implemented rigorously, they can strengthen the system of care provision.^{10 11}

The extent to which community organisations or coalitions engaged in improving community well-being recognise or use these methods is unknown. The US Centres for Disease Control and Prevention’s definition of community health improvement¹² and a Community Health Improvement Process (CHIP) proposed by the Institute of Medicine (IOM) Committee on Using Performance Monitoring to Improve Community Health¹³ emphasise community needs assessments and the selection and implementation of health interventions. However, these definitions do not focus on the common elements of QI methods in healthcare such as process maps, outcome and process measurements or small tests of change that are instrumental in developing locally appropriate solutions for changing care delivery systems.¹⁴

While health systems are inherently similar because of the nature of healthcare delivery, we contend that the notion of community and extension community coalition structures is heavily dependent on the culture, economic markets and sociopolitical sphere of the countries in which they are found. Australia, New Zealand, Canada and the USA have relatively similar national contexts. These four nations are high-income countries that are part of the Anglosphere, have liberal market economies (which can be contrasted to continental Europe’s more coordinated market economies), and experience health disparities between their White/Caucasian racial majority and their minority including indigenous populations.¹⁵

Communities are more diverse and complex than health systems. It stands to reason that community improvements are more likely to be adopted and sustained if QI methods used in healthcare settings can be used to develop locally tested community solutions or adapt evidence-based practices.¹⁶ This review aims to examine the extent to which QI methods are used in community health improvement work.

Methods and analysis

Protocol design

We selected a scoping review to answer our research questions because this method is best suited for identifying gaps in the research knowledge base,¹⁷ clarifying key concepts¹⁸ and reporting on what evidence exists rather than grading its quality.¹⁹

To design our scoping review protocol, we used scoping review framework by Arksey and O’Malley²⁰; methodological advancements for scoping studies by Levac et al ²¹ and other published scoping reviews protocols.^22–25 Our protocol consists of six stages: (1) identifying the research question; (2) identifying relevant studies; (3) selecting studies; (4) charting the data; (5) collating, summarising and reporting the results and (6) consulting with relevant stakeholders.

Stage 1: identifying the research question

We developed the research questions through an iterative process that involved identifying the research gap and refining our definitions. Research questions were informed by queries raised by graduate students during a QI course taught by one of the authors (RR). Students noted that hospital settings where most QI literature is set have well-defined and documented protocols, organisational structures with clearly identified roles and responsibilities, full-time staff paid to deliver service and buildings and infrastructure where care is provided.^{9 26} They questioned the applicability of these methods in community settings where coalition members may be loose affiliates who work part-time or volunteer, and no designated location where health improvement activities take place exists. This review seeks to answer three main questions:

1. How has community health improvement been defined?

2. What QI approaches have been used for community health improvement?

3. How are these approaches similar or different from those that have been implemented in the clinical setting (healthcare improvement)?

Stage 2: identifying relevant studies

Databases

The electronic databases of published literature identified as most appropriate for this study in consultation with a public health librarian are: Scopus, Web of Science and Proquest Health Management. These databases will be searched to identify relevant studies, and bibliographies of included studies will also be searched and included when relevant.

Search strategy

Search terms were developed in collaboration with a public health librarian and include three primary categories of terms: community and organisation, QI and well-being (box 1). These terms will be searched under topic or abstract, title and keyword, depending on the database. We will limit our search to English language and to peer-reviewed studies indexed as taking place in the USA, Australia, New Zealand, Canada or ‘undefined’ country. We will include studies from the year 2000 or later because the use of systematic QI methods to improve health was limited prior to that time.

Box 1. Search terms.

Community and organisation search terms

“Health department” OR (community AND Coalition*) OR (community And Partner*) OR Communit* OR (community AND Organization*) OR population*

AND

Quality improvement search terms

“Quality improvement” OR “improvement science” OR “continuous improvement” OR “improvement methods” OR “improvement tool” OR “quality initiative” OR “quality tool” OR “quality methods” OR “driver diagram*” OR “value stream mapping” OR “model for improvement” OR “process improvement” OR (lean AND (quality OR management)) OR “six sigma” OR “lean six sigma” OR PDSA OR “plan do study act” OR “plan-do-study-act” OR “plan do check act” OR “plan-do-check-act” OR PDCA

AND

Health and well-being search terms

Health* OR WellbeingWell-being OR well-being OR “well being” OR Prevent* OR academic* OR education* OR justice OR “social determinant” OR equit* OR housing

Exclude

Genetic OR biopsy OR surgery OR “long-term care” OR “nursing homes” OR perioperative OR postoperative OR radiology OR “patient safety” OR “patient protection” OR “patient care” OR “patient discharge” OR “length of stay” OR “medical errors” OR “intensive care unit” OR “patient-reported outcomes” Or PROs OR “enhanced recovery pathways” OR ERPs OR “medical education” OR “postgraduate training” OR “clinical clerkship” OR “hospital administration” OR microbiome OR gene OR engineering

Eligibility criteria

We used the Population, Intervention, Comparison, Outcomes strategy to develop eligibility criteria for review.²⁷ Inclusion and exclusion criteria and their operational definitions are listed in box 2.

Box 2. Inclusion and exclusion criteria for acceptable papers and operational definitions.

Population or problem

• Well-being in community settings in the USA, Australia, New Zealand or Canada.

Intervention

• Any intervention addressing improvement of well-being using a quality improvement (QI) approach.

Comparison

• Any experimental or non-experimental study with or without comparison groups.

Outcomes

• Community-level well-being, or community capacity to improve.

Exclusion criteria

• Article focuses on drivers of improvement, effectiveness of improvement, etc, but does not use QI approach or describe QI process.

• Article describes approaches to improve community, coalition or programme function (eg, improve attendance of coalition members at meetings) without linkage to community well-being outcome.

• Intervention took place within the walls of a facility with no linkage to community setting.

Operational definitions

• Community: a group of people with diverse characteristics who are linked by social ties, share common perspectives and engage in joint action in geographical locations or settings.

• Community capacity: knowledge, motivation or skills to apply QI approaches to community well-being.

• Community setting: where people eat, live, play, pray or participate in other voluntary activities, where attendance/participation is not mandatory.

  • For example, school site (or any site of mandatory activity) if outside of mandatory attendance hours; outcome is measured at school-level but activities take place in community.

• Community well-being: any health (physical, mental), educational or social outcome measured at an aggregate level.

• Facility: school, correctional (juvenile, jail, prison), hospital, clinic, military.

• Intervention: an activity, evidence-based programme or policy that took place (ie, is not merely proposed).

• QI approach: any QI method, such as Lean, PDSA, Six Sigma or the Model for Improvement; or description of systematic process to improve community well-being.

Stage 3: study selection

Study selection will occur in three phases. The first phase will finalise the search strings and the eligibility criteria. All three reviewers will independently review titles and abstracts for a 2% sample to identify studies as eligible or ineligible for full-text review using the inclusion and exclusion criteria. The reviewers will designate the studies as ‘eligible’, ‘ineligible’ or ‘maybe’ and aim for an inter-rater reliability (IRR) of 80% or higher. If this is not achieved, the definitions of the criteria will be reviewed and adjusted. Despite high IRR, if the search produces irrelevant studies, we will refine the search strings to narrow their focus.

In the second phase, two reviewers (TC and MWT) will each review half the titles and abstracts extracted using the search strings and the inclusion and exclusion criteria in box 2. As before, the reviewer will designate articles as ‘eligible’, ‘ineligible’ or ‘maybe’, with articles designated as ‘maybe’ moving on to the second stage. If articles do not have an abstract and are not obviously excludable based on title, they will be designated as ‘maybe’. Third, the two reviewers will both review the full text of each article designated ‘eligible’ or ‘maybe’ that met the inclusion criteria to determine if it will be included in the scoping review. If conflict arises, a third reviewer (RR) will read the full text and consult with TC and MWT to make a final decision.

Stage 4: data collection and charting the results

Using the JBI Reviewer’s Manual scoping review extraction template²⁸ and the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanations guides,²⁹ we will develop an initial draft of a standardised data charting instrument to extract relevant study information. Elements of the draft data charting form are listed in box 3.

Box 3. Elements of the data charting form.

• Bibliographic information

  • Study ID

  • Article title

  • Publication date

  • Journal details (volume, issue, pages)

  • Study location

  • Extracted by

  • Checked by

• Researcher details

  • Authors and affiliations (as presented on paper)

• Scoping review inclusion and exclusion criteria

  • Study population details (age, sex, race/ethnicity, sample size)

  • Community setting

  • Concept

  • Context

  • Type of study

• Aims and methods

  • Study aims/objectives

  • Methodology

  • Quality improvement approach

  • Intervention details (type, activity, duration, components)

• Study outcomes

  • Classification of community health or well-being outcomes (physical, mental, educational, social well-being or community capacity to improve)

  • Description of study outcomes and results

• Emergent categories

  • Any further information that emerges as important to the study not captured above, and agreed on by the study team

Data to be extracted from the included studies will comprise standard information, such as author, year of publication and study objectives. We will also extract information to examine the application of QI methods in community health interventions, intervention details (type, activity, duration, components), study population and sample size, study setting and a description of outcome measures. If the need for novel data extraction categories emerges during the data collection process, or we find eligible abstracts with missing data, consultation within our research team will guide decisions and be reported with the findings.

Stage 5: data summary, synthesis and report of the results

We will provide both a qualitative and quantitative summary of the scoping review results. PRISMA guidelines³⁰ will guide our qualitative, thematic analysis of the extracted data. We will report on a priori themes related to our research questions and any emergent themes that arise. The goal of the scoping review is to give an overview of what exists and identify gaps, not to meta-analyse or analyse the quality of the studies. The quantitative summary will detail the number and type of studies included.

Assessing study quality is optional in scoping reviews because one of the objectives of conducting such a review is to improve the precision of research questions based on the literature rather than to assess the quality of the published evidence to answer a specific research question. As stated by Munn et al, some of the objectives of scoping reviews are to identify the types of evidence available in a field, to clarify definitions and concepts and to identify knowledge gaps.³¹ This is the context in which our review is performed. There is no clear definition of what community health improvement means or how QI methods have been applied to these settings, and it is possible that our review will uncover a number of heterogeneous approaches that are difficult to compare. Our emphasis in this review will therefore be on describing the kinds of studies that present the use of QI methods in communities with the goal of providing guidance on identifying the kinds of studies that might need to be conducted before a systematic review is appropriate.

Stage 6: consultation with stakeholders

We will not perform consultation for the present scoping review. While community health interventions using QI approaches may exist in the practice setting, reporting and dissemination may not be abundant in the grey literature. Moreover, engaging stakeholder consultation in the process would involve developing a credible protocol and identifying a diverse group of community members and subject matter experts for consultation. We may complete this stage separately at a later date.

Patient and public involvement

No patient was involved.

Ethics and dissemination

This study protocol has been developed to systematically conduct a scoping review of English-language, peer-reviewed articles published since 2000 and synthesise data to identify how communities in the USA, Australia, New Zealand and Canada use QI approaches to address community health and well-being. While this study will neither grade evidence nor draw relationships between use of QI approaches and community-level well-being outcomes, results will provide a better understanding of: (1) how community health improvement has been defined and conceptualised; (2) how and to what extent QI approaches are being used in the community setting to address health and (3) the similarities or differences of how such approaches manifest in the community versus clinical setting. Evidence from recently published studies, such as those on the Spreading Community Accelerators through Learning and Evaluation initiative funded by the RWJF,^32–34 indicates that this novel body of inquiry is of current interest and is being explored. Study results will inform funders, researchers, practitioners and community members on how to better support, design and evaluate community health and well-being improvement interventions. Dissemination of study results will occur through peer-reviewed journals, conferences, presentations and a public health graduate course.