How to Design and Write Your Quality Improvement Study for Publication

Abstract

Objective

To describe a pragmatic process for translating quality improvement (QI) projects into published manuscripts.

Scope

Types of QI work that are generalizable and have broad relevance (to journals and readers), design principles that are important for publishable QI work, how QI manuscript organization might differ from biomedical manuscripts, how to use and not to use Standards for Quality Improvement Reporting Excellence and other guidelines, pitfalls, and how to avoid/repair them.

Introduction

Quality improvement (QI) is a systematic approach to solving problems in health care to effect change and improve how we care for patients and their outcomes.¹ High-impact QI interventions are increasingly being implemented in a variety of settings, both hospital and clinic, spanning academic and community settings. The breadth of this work spans the IOM's 6 aims: safe, effective, patient-centered, efficient, timely and equitable, and increasingly incorporating a broader evaluation of value, including cost.² QI methodology affords the opportunity to identify and measure potential health care gaps, develop and test interventions, and thereby ensure adherence to recommended best practice to improve care for patients. Organizations such as the Institute for Healthcare Improvement (IHI) and the National Quality Forum have helped provide broad dissemination of the importance of QI work in health care, leading many medical providers to obtain QI training and devote time and resources to QI initiatives and projects. Despite more widespread QI awareness, dissemination of QI-specific projects that result in the form of manuscript publication has been limited and variable.

The peer-reviewed QI health care literature has grown over the past 30+ years, but a gap likely exists between the number of QI projects performed and the number published.³ There are multiple reasons for this gap, such as lack of publication venues, lack of training on how to publish QI work, and lack of clarity on how to publish, given the varied methodologic and reporting quality standards.^4-6 In addition, there can be a misperception that QI publications are not as rigorous as traditional research, which can make it challenging for QI project leaders to identify suitable mentorship and resources to successfully take a project to completion and subsequent publication.⁷ Many journals are now dedicated to disseminating knowledge on how to improve the quality of patient care, including Neurology® Clinical Practice, whose mission is to help clinicians provide unbiased high-quality care for all neurologic patients and prioritize articles that address quality and patient outcomes.⁸ Furthermore, many institutions outside of academic medical centers will lead impactful QI work; however, their focus is on improving patient care and they may not subsequently pursue publication. As health care systems become more complex, it is essential to disseminate this improvement knowledge, therefore decreasing the gap between QI implementation and publication.

The objective of this article is to describe strategies in the planning, execution, and reporting of QI projects to streamline the process of translating into manuscript form and improve the likelihood of peer-reviewed publication. Publishing in QI can be challenging because the format is different from traditional research. Therefore, a review of the reporting framework recommended for QI literature is provided with pearls and pitfalls to consider.

Considerations During Early Study Design

At the outset of a QI project, the project team must carefully consider and define the problem they are trying to solve, including the patient population of interest. A stakeholder analysis will help inform assembly of the multidisciplinary team to ensure representatives from all the key departments and disciplines in the process. The team should also practice an important principle in QI that highlights the value of learning from other high-performing teams. Often, this may involve a “go-and-see” or Gemba. The idea of Gemba walks is for improvement project leaders to go to where the interventions will take place.⁹ While local improvement is often the goal of QI, teams in similar health care settings can learn from one another and apply best practices during the early-stage design process.

The team will also need to carefully consider how they will know whether their intervention(s) is a success, considering study design and evidence-based quality measures carefully.¹⁰ Quality measures are specific standardized metrics widely recognized to assess and evaluate quality of care.¹¹ The study team should develop a data collection plan and ensure sufficient statistical expertise to perform a rigorous analysis. Project teams should discuss whether publication is a goal and plan accordingly. When considering specific study trial designs, most QI projects use an interrupted time series design, but a stepped-wedge cluster randomized trial can also be considered.^12,13 Interrupted time series design is used to evaluate the impact of an intervention by measuring the effects at several points in time to allow demonstration of causal relationship between the intervention and the observed change.¹¹ The stepped-wedge cluster randomized trial uses randomly assigned groups to start as the control group and transition to intervention at different points in time.¹² This allows observation of the impact of an intervention over time where all groups will be in the intervention group. A mixed-method approach, incorporating qualitative evaluation of the intervention(s) and/or patient experience, alongside the quantitative results might provide additional depth to the analysis.¹⁴ Pre-post designs can be informative but are more prone to confounding variables that the authors will need to carefully account for in their description of institutional context and discussion. Other study designs that are not ideal for QI publication would be retrospective or ecological study designs. Retrospective study designs rely on historical recall of data, which can be susceptible to biases.¹⁵ Ecological studies focus more on population health rather than individual data and, therefore, can make causality of problems difficult to assess.¹⁶

SQUIRE 2.0

SQUIRE 2.0, or Standards for Quality Improvement Reporting Excellence,¹⁷ is used as a standardized protocol to help provide consistent and accurate publishing of QI projects. This systematic approach spans several sections. It begins with the title and abstract, which define the project's purpose and objectives, followed by the methods, results, and discussion. If the goal of a QI project includes publication, project leaders should review the SQUIRE 2.0 guidelines to ensure it upholds the benchmark for academic rigor. The current SQUIRE 2.0 guidelines are given in the Table.¹⁸ A study in 2021 reported that citation of SQUIRE 2.0 is still inadequate.¹⁸ Early adoption of the SQUIRE 2.0 guidelines is crucial because it ensures clear project articulation and team alignment of goals including standardized reporting of publication. It promotes best practices and enhances the credibility of the article. Inconsistent adherence to these accepted guidelines could result in confusion or missing key components. Furthermore, current literature review with proper citation is essential because this provides context and shows how the project builds on existing knowledge. Finally, it is also important to note that not all components are necessary for every project. Authors should tailor their reporting based on what is needed to describe a complete picture of their project.

Table.

Example QI Publications in Neurology

Study	Design	Population	Setting	Aim/SMART goal	Measure(s)—process, outcome, balancing	Intervention(s)	QI tools used	Key results
Patel AD Epilepsia 202011	Single-center prospective quality improvement study	Pediatric patients with epilepsy	AMC	Decrease low-dose rescue Rx from 3.5% to 1.5% over 2 y and sustain × 1 y	Process	Epic alert, prefilled syringes, pharmacy review of low doses	Key driver diagram, control chart	Centerline shift from average 7.2% nonadherence to 0.22%
Jones FJS Neurology 202012	Single-center prospective quality improvement study	Adult patients with epilepsy	AMC	Increase complete seizure documentation from 49.8% to 70% in 6 mo	Process	SmartPhrase development, education, and feedback	Key driver diagram, control chart	Median weekly complete seizure documentation improved to 78%
Wang C-J JAMA netw open 202313	Multicenter stepped-wedge cluster RCT Implementation science	Adult patients with acute stroke	Secondary and tertiary stroke centers	Reperfusion therapy rate (IVT or EVT)	Process (primary) Secondary outcomes included mortality and mRS (outcome)	STEP intervention—education, toolkit, feedback/audit	PDSA (implementation science)	Reperfusion rate 53.5% compared with 43.9% in control (did not meet statistical significance)
Sather J Joint Comm J Qual and Pt safety 202114	Single-center prospective quality improvement study	Adult patients with SAH or nontraumatic ICH	AMC	Improving the safety and quality of interhospital transfer	Process	Local practice guideline development, transfer acceptance protocol and process, electronic arrival notification and imaging transfer, EHR decision support	Interrupted time series figure	Reduced median ED length of stay from 300 to 150 min
Machline-Carrion MJ JAMA neurol 201915	Multicenter cluster RCT	Adult patients with acute stroke or TIA	Public and private hospital spanning 3 countries	Improve adherence to set of evidence-based stroke measures	Process	Case management, reminders, checklist, education, audit/feedback		Composite adherence 85.3% in intervention vs 77.8% in controls (did not reach statistical significance)
Roy A Neurology® Clinical Practice 202316	Single-center prospective quality improvement study	Adult patients with TIA or nondisabling stroke	AMC	Evaluate impact of a TIA/nondisabling stroke outpatient pathway	Process (primary) Outcome (secondary) 90-d recurrent TIA/stroke Balancing measures—ED and hospital readmissions	Safety checklist to evaluate candidacy for early ED discharge/outpatient expedited evaluation; SmartPhrase incorporating data, Rx, and intended follow-up; coordination to ensure 7-d follow-up stroke and MRI and/or cardiac testing	Process, outcome, and balancing measures	ED LOS decreased from 24.4 h in historical controls to 13.7 h. ED-based MRI imaging was reduced while CT remained stable. 86% followed up within 5 business days
Yarbrough AJ Neurology® Clinical Practice 202217	Single-center prospective study	Neurology patients	AMC	Evaluate impact of a neurology access clinic	Process Cost	Creation of an access clinic M-F daytime for all new general neurology referrals	Cost estimate of intervention	Lead time decreased from 142 to 18 d for new patients; readmission rate was similar

Abbreviations: AMC = academic medical center; EVT = endovascular therapy; IVT = intravascular therapy; mRS = modified Rankin Scale; Rx = prescription; SMART = specific, measurable, achievable, relevant, time-bound.

Permission granted from Ogrinc G, Davies L, Goodman D, et al. SQUIRE 2.0 (Standards for QUality Improvement Reporting Excellence): revised publication guidelines from a detailed consensus process; BMJ Quality & Safety 2016; 25:986–992.

The introduction section focuses on the project's context and identifies gaps in establishing its significance within the broader health care landscape. The methods section for a QI manuscript can be challenging because it may differ considerably from traditional research methodology and communication. This section should describe each of the interventions completed in the QI project. Often, the person who helped plan the intervention or perform the Plan, Do, Study Act (PDSA) cycle is best to provide this information. For projects with multiple PDSA cycles, each intervention should be described in the methods section. The team-oriented focus of QI also facilitates an inclusive approach to authorship of the reporting manuscript.

When describing each intervention, the a priori plan to evaluate the impact of the intervention should be included. Selected measures and measure categories (i.e., structure, process, outcome, or balancing) should be clearly described. For QI project teams in the early design phase who are evaluating how to best measure current state, a literature review of previous similar studies and/or review of previously described measures could be informative. The AAN maintains a portfolio of rigorously designed Quality Measures that are evidence-based, meaningful, valid, reliable, and relevant to the care of patients with neurologic disease.^19,20 Candidate measures might also include quality metrics of high relevance to hospitals and health care systems that are tied to reimbursement, program accreditation, and rankings—including readmissions, clinic access, length of stay, hospital-acquired complications, and stroke core measures.

More than one measure can be used for a given QI project that is being submitted for publication. The article should list the tools used to develop the aim statement, key drivers, and interventions. A clear understanding of how the QI methodology was implemented and followed for a given project is critical for manuscript reviewers and editors.

For QI manuscripts, be careful that QI methodology was used before you submit. If you did not follow one of the methodologies used in improvement science, then it may be best not to submit a manuscript for publication. Examples of methodology can be from the IHI Lean, Six Sigma, or Failure Modes and Effects Analysis.^21-25 Not planning and thinking of QI methodology at the start of a project idea can easily sink manuscripts from being accepted for publication. If using any statistical methodology, this data analysis should be described in the same way as in a traditional research manuscript. For iterative interventions, data should be reflected using a statistical process control chart.^26-29 In addition, it is important that you receive a determination from your local institutional review board (IRB) if the project fits under QI and is, therefore, exempt from IRB review. Often, a research determination form from your IRB is now required when submitting a manuscript to a peer-reviewed journal. It is best to check with local rules regarding how to best approach this subject.

It is also important to use American Society for Quality (ASQ) criteria when determining whether a centerline shift is seen.²⁹ An additional statistically sound approach is to use the aggregate point rule (APR).³⁰ A pitfall is not using any statistical technique when attempting to publish QI. Also, cite the publication for the technique used to determine statistically significant change that occurred based on intervention implementation such as the ASQ or APR manuscripts cited here. It is most ideal to avoid a before vs after analysis. Such a technique is flawed for not controlling all variables as occurs in a traditional research trial. Pre-post analyses can be useful for exploratory improvement studies. However, a major limitation is that even if patient characteristics are similar before and after, there can be confounding system changes that may affect the outcome and it can be difficult to discern the true impact of the intervention itself. An interrupted time series design allows readers to understand trends in the data before the intervention, immediately after, and in the follow-up.³⁰

The results section does not need to be long. It is good practice to start with a description of the population and consider including a demographic table ³¹ Then, describe how the data measure changed because of each intervention being implemented. Track data throughout the life cycle of your QI project so they can be commented on in the results section. It is important to list the observed associations in the data after the intervention is implemented. Often, a description on the impact or observations of the PDSA cycles is given here. A figure showing the data with the interventions annotated on it should be included and referenced in this section.³² This illustrates one of the main differences between QI publications and traditional research. QI projects focus on the continuous feedback and adapting to real-time iterative changes while traditional research projects follow a strict protocol. QI projects also focus more on real-world application in real time while traditional research may focus on theoretical implications. Next, it is also important to discuss points that are outside the 3 standard deviations above or below the mean reported because these points consist of special cause variation.²⁷ Finally, you may observe unintended consequences after implementing interventions or after collecting feedback from a given PDSA cycle. Include this information, if applicable, in the results section.³³ Note that it is important to stick to the facts and not provide commentary on what worked or what did not work. These points are best suited for the discussion section.

In the discussion section, you can highlight the key findings of your QI effort and put them in context of the existing literature. Start with the key findings and link them to the study aim. Discuss how the intervention(s) was associated with the outcome(s) and the potential reasons or explanations for this. For studies with an interrupted time series design,^34,35 address common cause vs special cause variation.^27,36 Be cautious not to overclaim causality. Remember to discuss any balancing measures used. Balancing measures are other data metrics used in conjunction with the current project objectives to ensure that improvement in the primary outcome does not result in unintended negative impacts in other parts of the health care ecosystem.³⁷ Address these specifically and ideally quantitatively. To determine value, consider an evaluation of the actual costs or an estimate, including the cost of the intervention.^38-40

Next, compare your study and findings with the existing literature, being careful to address context.⁴¹ How are the findings like what has been found in other studies? How are they different? What are the unique strengths of the QI project? How was the intervention(s) similar or different? What about context in the other studies, and how might this yield similar or different results? Ideally, you can connect your findings to other published studies in neurology addressing a similar problem/aim, but if they do not exist, looking for studies in other specialties might be needed. If multiple studies had similar conclusions, this strengthens the findings of the study and increases the likelihood that the findings could be generalized at other centers. If an improvement was seen in a process measure only (e.g., door-to-needle time, on-time administration of Parkinson medications, time to postdischarge clinic follow-up, and documentation of seizure safety discussion), provide supporting evidence for the link to an outcome measure (e.g., modified Rankin Scale, falls, and breakthrough seizures) from other studies, and if none exists, be sure to address this in next steps.⁴² In addition, other components that should be discussed are highlighting lessons learned, presenting the iterative changes done, and detailing the sustainability plan for these interventions. These components demonstrate one of the main differences in QI publications vs traditional research. QI projects focus on the adaptive process needed to make quick changes from ongoing feedback, applies practical real-time interventions, and considers future application and sustainability.

Limitations must be carefully addressed in the discussion. What were the limitations of the study design (controls, blinding, randomization, single center vs multicenter)? What factor(s) may have contributed to bias or confounding, and in what ways did you attempt to minimize these limitations? If an improvement was seen after the intervention(s), could factors extrinsic to the intervention(s) have influenced the outcome (e.g., a change in broader hospital processes, staffing, or other resources). Readers will want to understand whether they can replicate this project at their center. What is unique about the institutional context (people, materials, training, processes, policies) that might limit applications to other centers? Ideally, you can demonstrate sustainability of the effect after the interventions are complete to mitigate the Hawthorne effect.⁴³

Finally, like any research study, the discussion is an opportunity to cast vision for the next steps in the field. What are the next steps for the project? Will the interventions be disseminated to other settings? More broadly, what are the implications of the findings for the field and what are the unanswered questions?

Additional QI Professional Development Opportunities

There are many additional opportunities for professional development in QI that exist to further understand the specific details of QI publications. The Institute for Healthcare Improvement Open School has many online courses. More recently, the AAN has formed a workgroup to address the QI training and professional development gap, particularly for trainees and young faculty. Additional QI training tools, including webinars, presentations, and example manuscripts, are available on the AAN QI page (Table).⁴⁴

Summary and Next Steps

As health care providers, we are dedicated to ensuring that our patients receive safe, effective, timely, patient-centered, efficient, and equitable health care, which are key domains of quality care.² How do we as individuals, teams, and communities of clinicians ensure that we are meeting the goal of high-quality care for our patients?

Publication of QI projects using a rigorous SQUIRE 2.0 framework is essential to dissemination of QI results that could potentially be applied or modified to other settings, to eventually implement best practice and mitigate care gaps on a larger level. Publication also provides a mechanism for career advancement for those who have dedicated significant effort to QI initiatives.

Historically, many journals have only published traditional research methodology–based projects, but fortunately, this landscape is changing. An increasing array of journals now publish QI work, including Neurology® Clinical Practice. We believe publication presents a great platform for sharing knowledge because many neurology health care teams and their multidisciplinary colleagues are solving similar problems in their local systems. We hope this guide helps project leaders to be successful in translating their improvement idea into a successful project and thus meet the standards set for publication so that their work can be disseminated.

Appendix. Authors

Name	Location	Contribution
Anup D. Patel, MD	Division of Neurology and Center of Clinical Excellence, Nationwide Children's Hospital, Department of Pediatrics, The Ohio State University College of Medicine, Columbus	Drafting/revision of the manuscript for content, including medical writing for content; major role in the acquisition of data; study concept or design
Laurice Yang	Department of Neurology, Stanford University, Palo Alto, CA	Drafting/revision of the manuscript for content, including medical writing for content; major role in the acquisition of data; study concept or design
Kathryn Kvam	Department of Neurology, Stanford University, Palo Alto, CA	Drafting/revision of the manuscript for content, including medical writing for content; major role in the acquisition of data; study concept or design
Christine Baca, MD, MSHS	Department of Neurology, Virginia Commonwealth University, Richmond	Drafting/revision of the manuscript for content, including medical writing for content; major role in the acquisition of data; study concept or design
Lyell K. Jones	Department of Neurology, Mayo Clinic, Rochester, MN	Drafting/revision of the manuscript for content, including medical writing for content; major role in the acquisition of data; study concept or design

Study Funding

The authors report no targeted funding.

Disclosure

The authors report no relevant disclosures. Full disclosure form information provided by the authors is available with the full text of this article at Neurology.org/cp.