A Next Step: Reviewer Feedback on Quality Improvement Publication Guidelines

Technology and Process Improvement

Dazzled by advances in surgical lasers, the anti-aging drug reservatrol, or Google's warp-speed search of the earth's information on their handheld computers, physicians and nurses may dismiss the significance of their recent diabetic and cardiac quality improvement (QI) work that arguably improves health care more than many technologic advances. Despite the intensity of designing and enacting QI studies to improve the processes, reliability, and safety of medical care, doctors and nurses have historically not captured or reported their data and findings with enough systematic rigor to turn your head from the brilliance of technology or the randomized, controlled, drug trials reported online and in the media even before print publication. Their work, however, is critical because embedding and integrating new knowledge into clinical practice through quality improvement work is ultimately the outcome that will improve the state of this country's health care.

Quality Improvement Guidelines

Because the Institute for Healthcare Improvement (IHI) has stimulated such a surge of QI work, and clinicians want to report their results in peer-reviewed journals, and because reviewers for leading medical journals are unfamiliar with the methodology and evaluation of QI studies, a set of guidelines was developed to guide editors and to ensure that QI article content is consistently complete, statistically sound, and high quality.

Frank Davidoff, MD (IHI), and Peter Batalden, PhD (Dartmouth Medical School), collaborated to create these new publication guidelines. Sixteen items are included within their proposed format—Introduction, Methods, Results, and Discussion (“IMRaD”). Three examples of the sixteen items are: “Problem: Nature and severity of specific local dysfunction or failure; Analytic methods: Statistical and time series techniques used, and specific software; Outcomes: What effects the changes/improvements had on clinical and/or organizational and professional outcomes and processes including benefits, harms, unexpected results, problems, failures.” As an additional aid, the authors cite 67 “Examples of elements and criteria to be considered in reporting guideline items.”¹

These guidelines are part of an effort to improve the quality of health care through setting a standard that benefits: QI researchers performing studies and writing reports; journal editors reviewing manuscripts; and physicians, nurses, and project managers reading and evaluating the published results.

Reviewer Feedback

After attending Drs Davidoff and Batalden's IHI workshop in December 2005, I sent their article containing the publication guidelines to the 500 national reviewers of our Journal to inform, to educate, and to give them a tool. Representative feedback included:

• “This is the main focus of my current research/administrative activities; until now I have been struggling with how to publish the results of our work.”

• “I agree with the guidelines, have used them and find them most helpful.”

• “These guidelines have value in the setting of review of a quality improvement article, as well the structure of questions asked can guide the writing of the article, so it may be prudent to distribute these guidelines to the authors of papers.”

• “I think that the guidelines are enlightening and likely to be useful. Maybe a check-off form could be used, with the reviewer marking each of these guidelines as: met, possibly met, or not met, with specific comments added, as appropriate. I don't think the reviewers should be forced into a narrow format, as it could result in missing the forest for the trees.”

• “Adding a structured approach for reviewing quality assurance genre articles is a good idea.”

• “They are reasonable for QI types of articles, but a more stringent standard such as CONSORT should still apply for formal research studies.”

Outcome

The Permanente Journal has a particular interest in these guidelines because the editors, for a decade, have selected and published QI articles to enhance the development, communication, and dissemination of clinical practices that improve performance. The capstone is our annual publication of the Kaiser Permanente practices recognized as the Vohs Awards for Quality—comprehensive, clinical innovations demonstrating significant results in large populations. Publication guidelines can improve the quality of QI articles and the clinical practices they report.