Adapting Clinical Guidelines for the Digital Age: Summary of a Holistic and Multidisciplinary Approach

Abstract

The Centers for Disease Control and Prevention’s Adapting Clinical Guidelines for the Digital Age initiative aims to redesign and improve guideline development, implementation, and standardization. Historically, aspects of guideline development and implementation have been siloed. This leads to long lag times for guidelines to reach patient care, unnecessary redundancy, and potential for misinterpretation, leading to inconsistencies in how the recommendations are applied. A multidisciplinary, multiorganizational holistic approach brought together experts in guideline development, informatics, communication, implementation, and evaluation to understand and identify problems in guideline development and implementation, define an ideal state with no constraints, and then design a future state that advances the process close to the ideal state. The Adapting Clinical Guidelines for the Digital Age workgroups each worked on one focus area and included experts from the other areas to help analyze the current state and develop holistic solutions for the future state. Each workgroup produced interrelated standards, processes, and tools that can be used across the continuum of guideline development and implementation.

Introduction

In 2018, the Centers for Disease Control and Prevention (CDC) began the Adapting Clinical Guidelines for the Digital Age (ACG) initiative to improve the process for developing and implementing clinical practice guidelines (CPGs).^1,2 The initiative brought together experts in 5 focus areas—guideline creation, informatics, communication, implementation, and evaluation—to understand and identify opportunities for improving the guideline development and implementation process. This overview provides background on the initiative, outcomes, products, and future activities.

The ACG initiative took a systems-thinking approach to process improvement—a way of exploring and developing effective action by looking at the whole system rather than separate parts.³ The primary objective was to identify problems and solutions to reduce lag time between guideline development and implementation^4,5 without creating new or additional problems. In the ACG initiative, the system includes expertise, processes, and organizations involved in the continuum of guideline development and implementation. While other efforts have focused on improving parts of the process, such as guideline development or clinical workflow implementation,^6,7 ACG provides a multidisciplinary, multiorganizational approach that includes the entire system and applies the perspectives of guideline development, informatics, implementation, communication and dissemination, and evaluation with representation from multiple organizations (ie, a holistic approach encompassing the entire system).

Background

Traditionally, the development of written guidelines and digital tools and the implementation of guideline recommendations into patient care have been planned separately and completed sequentially.⁸ Digital tools, such as clinical decision support (CDS) and electronic clinical quality measures (eCQMs), can help with implementing guidelines.⁹ However, there are often barriers to integrating guidelines into digital systems, including a lack of broadly implemented interoperability standards (eg, Fast Healthcare Interoperability Resources [FHIR®]) that would increase consistency across a variety of patient care settings.^10–14 As a result, the current state of guideline development and implementation includes many disconnected systems and unnecessary redundancy, leading to inconsistent guideline interpretation and integration into patient care.¹⁵ A more effective approach would consider systems as a whole across the entire guideline development and implementation continuum.

Participants of the ACG initiative—described in the Methods section—reviewed the entire guideline system, from guideline development to evaluation, to identify problems and solutions to improve guideline adoption, taking into account existing standards and practices for written guidelines.^6,16 This holistic review revealed challenges that often occur downstream in the guideline development and implementation process. For example, problems with developing digital tools or applying recommendations to patient care often stem from issues with how the guideline is written. Guidelines may be vague in their wording if the scientific evidence does not directly support the recommendation. This vagueness makes it challenging to develop digital tools or insert clear recommendations into workflows, creating the potential for misinterpretation (ie, the vague wording is one root cause of downstream implementation challenges). Likewise, the sequential and siloed nature of the traditional development process creates unnecessary redundancy and lag times in translating the evidence into practice.^4,5 There was also no international health IT standard for computable guidelines—a representation of written guideline recommendations in computer-interpretable format—that provided a way to develop multiple types of digital tools (eg, CDS, eCQMs, and case reports) to support implementing and using recommendations in practice.

To consider the entire system of guideline development and implementation and to identify the most difficult problems that limit how easily, quickly, accurately, or consistently guidelines are adopted in patient care, CDC and its partners applied an adapted Kaizen method.¹⁷ Kaizen is a Japanese philosophy for continuous improvement and is part of the Lean methodology aimed at minimizing waste and increasing productivity.^18,19 The adapted Kaizen method is a useful approach for improving guideline development and implementation because it incorporates the perspectives of multiple disciplines across multiple organizations and focuses on complex processes with broad impacts.¹⁷

Methods

Applying the adapted Kaizen method,¹⁷ CDC held an agency-wide, multidisciplinary planning session to set the scope, map processes at a high level, and determine the types of expertise needed for a Kaizen event focused on guideline development and implementation. A multidisciplinary group of partners was identified and included guideline authors, health IT developers (including CDS and eCQM developers), clinicians, patients and patient advocates, medical societies, public health organizations, laboratory professional groups, researchers, evaluators, standards experts, payers, and policymakers (at various levels—eg, national and organizational). In February 2018, CDC convened the group for a 4.5-day Kaizen event designed to map the current state of guideline development and implementation, identify an ideal state, and propose a future state in 5 key areas of focus: guideline creation, informatics, communication, implementation, and evaluation. After the event, 5 workgroups continued working on these focus areas to further develop the details and deliverables of the proposed future state established during the Kaizen event. The workgroups collectively determined that solving the underlying problems identified in the current state would require a redesign of the guideline development and implementation process.

During the 4 years following the Kaizen event, and ongoing for some workgroups, the detailed work occurred during regular workgroup meetings, with some of the work slowing or pausing during the COVID-19 pandemic. Each workgroup determined a method and products that best addressed their respective scope:

• The Guideline Creation Workgroup developed a 12-phase integrated process and model for codevelopment and implementation of written and computable guidelines^20,21 building on international standards for guideline development, including the Institute of Medicine’s CPGs We Can Trust framework and Guidelines International Network-McMaster Development Checklist.^6,16 The integrated process, informed by literature reviews and subject matter expertise from all ACG workgroups, aimed to resolve or mitigate issues during guideline development, such as vague wording in the written guideline, that impact translation into computable guidelines and implementation in patient care. The integrated process incorporates the products of the other ACG workgroups, brings downstream perspectives (ie, informatics, implementation, communication, and evaluation) into guideline development from the beginning, and includes an evaluation phase to monitor guideline impact. Suggested activities and resources support each phase of the integrated process. The activities were validated by experts in their respective fields and through review of literature and industry best practices.

• The Informatics Workgroup established an approach and methodology for developing and implementing computable guidelines based on a health IT standard called FHIR®²² from the international standards development organization, Health Level Seven International (HL7®).²³ The Informatics Workgroup partnered with HL7® to develop, test, and vet a computable guidelines standard: FHIR® Clinical Guidelines Implementation Guide (IG). More commonly known as CPG-on-FHIR®,²⁴ this IG was balloted at HL7® in September 2019.²⁵ The balloting process vetted the standard in the Universal Realm, allowing the standard to be used internationally. The HL7® balloting process included an open review and comment period followed by reconciliation of each comment and final approval by the sponsoring HL7® workgroup.²⁵ The initial version of the standard, called a Standard for Trial Use, was published in February 2021.²⁶ As more guideline efforts use CPG-on-FHIR®, updates continue to be made to the standard as it matures. The next update will be balloted in the September 2023 ballot cycle as part of the continuing maturation of the standard.

• The Communication Workgroup developed a stakeholder analysis tool and communication checklist based on the industry best practice and incorporated its products in the integrated process.²¹ Communication planning needed to begin early in guideline development to assure the target audience is prepared and ready to implement the guideline. As part of communication planning, the stakeholder analysis tool provides a way to determine who may help or hinder the success of a guideline. The communication checklist specifies a standard operating procedure for effective communication and dissemination.

• The Implementation Workgroup developed checklists to help implement computable guidelines.²⁷ The checklists were based on Boxwala et al’s levels of knowledge framework,²⁸ where L1 is the representation of knowledge in a narrative format, L2 is in a semistructured format, L3 is in a structured format, and L4 is in an executable format. Required steps were kept to a minimum to increase flexibility while still indicating core activities needed to create a successful representation of the guideline recommendations at each knowledge level. Since the Guideline Creation workgroup’s integrated process includes details on developing the written guideline, the Implementation workgroup chose not to develop an L1 checklist. These checklists themselves use the FHIR® standard so they are executable in a health IT system.

• The Evaluation Workgroup developed an evaluation framework and tool^29,30 to be used in parallel with the integrated process to maximize the likelihood that desired outcomes are achieved. The framework and tool were based on the relevant evaluation models used in guideline development, informatics, computing, and public health as well as subject matter expertise. This framework provides a way to assess the guideline development process, products, and outcomes in real-time so adjustments can be made as needed and the impact of using the integrated process can be assessed.

The work on the integrated process and evaluation framework was finalized in 2022.The products from each of the ACG workgroups are summarized below and described fully in separate publications.^{20,21,24,27,29,30}

Results

Participants in CDC’s Kaizen event determined that the current state of guideline development and implementation is sequential and siloed, and the written guidelines, CDS, CQMs, and other digital tools are developed separately (left side of Figure). This development approach is composed of sequential steps that build on each other, creating opportunity for misinterpretation in each step analogous to the telephone game. This unnecessary redundancy includes the potential for hundreds of translations of written guidelines into CDS (represented by the first arrow in Figure), based on the estimated number of electronic health record (EHR) vendors in the United States (over 500).³¹ This estimate would be much higher if third party health IT developers were included. Furthermore, based on the number of hospitals (over 6000)³² and ambulatory practices (over 136 000)³³ in the United States, thousands more translations could take place with the integration of CDS into patient care workflows (represented by the second arrow in Figure). Thus, each layer of translation that occurs exponentially increases redundancy and decreases the likelihood of applying the guideline recommendations as intended. Also, the development and implementation of CQMs are generally treated separately from patient care, limiting their use in evaluation of guideline uptake and health outcomes. Evaluation and feedback loops, especially back to the guideline developers, may be inconsistent or nonexistent.

Figure.

ACG Kaizen event outcome: high-level current state and proposed future state for guideline development and implementation. Adapted from CPG-on-FHIR® v1.0.0 figure 1.4.²⁴ Abbreviations: ACG, Adapting Clinical Guidelines for the Digital Age; CPG, Clinical Practice Guidelines; FHIR®, Fast Healthcare Interoperability Resources.

In the proposed future state (right side of Figure), all perspectives are represented from the outset, working together using an iterative approach to produce a package of products that includes the written and computable guidelines, CDS, CQMs, and potentially other derivatives (digital tools) that can help apply the guidelines in patient care. Feedback loops are more consistent and iterative, providing opportunity to monitor the guidelines in practice (process) and to evaluate the guideline quality (product) and its impact (outcomes).

As the scientific evidence indicates that updates or new guidelines are needed, modular updates can be made to the written and computable guidelines and the derivatives that support guideline implementation. While this approach does not remove the potential for mistranslations into patient care, it may reduce interpretive errors regarding the intent of the guideline developers. By using a holistic approach from the beginning, challenges that have historically been identified downstream may be avoided or mitigated. This may decrease the lag time for guideline uptake and increase the likelihood of successful implementation of the recommendations into patient care.

Each of the 5 ACG initiative workgroups produced deliverables to support the proposed future state. Together, these products provide standards, processes, and tools that can help codevelop and evaluate written and computable guidelines, standardize knowledge representation for computable guideline recommendations, and provide checklists to help ensure completeness. Each workgroup’s products are described below with references to the more detailed publications summarized in Table.

Table.

ACG Workgroup Products and Publications.^{20,21,24,27,29,30}

Workgroup	Product(s)	Publication
Guideline Creation	Integrated process model for iterative codevelopment of written and computable guidelines, early integration of implementation and testing, communication and dissemination strategies, and evaluation; and Guideline Development Decision Tool	Matson-Koffman DM, Robinson SJ, Jakhmola P, et al. An Integrated Process for Co-Developing and Implementing Written and Computable Clinical Practice Guidelines. Am J Med Qual. 2023;38(S2):S12-S34. DOI: 10.1097/JMQ.0000000000000137. https://journals.lww.com/10.1097/JMQ.0000000000000137 •Appendix A Integrated Process Tables (https://stacks.cdc.gov/view/cdc/131006)
Informatics	International technical standard for computable guidelines: FHIR® Clinical Guidelines IG (aka CPG-on-FHIR®)	http://hl7.org/fhir/uv/cpg/index.html
Implementation	Detailed checklists to facilitate the development and implementation of knowledge levels L2, L3, and L4, which became part of the CPG-on-FHIR® methodology	http://hl7.org/fhir/uv/cpg/methodology.html#validate-checklists
Communication	Communication and dissemination aspects of the integrated process model and templates for Stakeholder Communication Analysis and Communication Standard Operating Procedures: A Checklist for Effective Communication and Dissemination	Supplementary tables. An Integrated Process for Co-Developing and Implementing Written and Computable Clinical Practice Guidelines. Centers for Disease Control and Prevention. Accessed August 9, 2023. https://journals.lww.com/10.1097/JMQ.0000000000000137 •Appendix B Stakeholder Communication Analysis (https://stacks.cdc.gov/view/cdc/131006) •Appendix C Checklist for Effective Communication and Dissemination (https://stacks.cdc.gov/view/cdc/131006)
Evaluation	Evaluation framework and tool to evaluate the integrated process across each of its 12 phases in terms of process, product, and outcome	Tailor A, Robinson SJ, Matson-Koffman DM, et al. An Evaluation Framework for a Novel Process to Codevelop Written and Computable Guidelines. Am J Med Qual. 2023;38(S2):S35-S45. DOI: 10.1097/JMQ.0000000000000140. https://journals.lww.com/10.1097/JMQ.0000000000000140 Evaluation Tool https://stacks.cdc.gov/view/cdc/131007

Abbreviations: ACG, Adapting Clinical Guidelines for the Digital Age; CDC, Centers for Disease Control and Prevention; CPG, Clinical Practice Guidelines; FHIR®, Fast Healthcare Interoperability Resources.

The Guideline Creation Workgroup modeled a 12-phase integrated process^20,21 that reengineers guideline development to be iterative and cyclical, with concurrent development of the written and computable guidelines accelerating implementation into clinical practice using CDS systems and related tools. Phases 0 to 3 are preparatory and establish the operational framework and governance processes, the guideline topic, key development groups, collaboration platforms, and tooling and develop communication and evaluation plans early so they are ready when the guideline is published. Phases 4 to 7 determine the guideline scope and concurrently draft the written and computable guideline products, with early implementation of guideline components providing opportunity for feedback to adjust the guidelines for maximum effectiveness before they are released. Phases 8 to 9 vet the written guideline products through public and expert opinion, validate the computable guideline products, and then finalize the documentation and derivatives (digital tools) to support implementing the recommendations in patient care. Phases 10 to 12 provide key activities for implementing and monitoring the guideline products at the local level and updating the recommendation(s) when indicated by the evidence. Suggested activities and resources for each phase are detailed in referenced tables. Products from the other 4 workgroups, including the CPG-on-FHIR® standard, are incorporated throughout the 12 phases. Examples illustrate each level of knowledge (ie, L1, L2, L3, and L4) along with a real-world case example for COVID-19, showing how the integrated process was applied and how it shortened the development and implementation timeframe.

The Informatics Workgroup developed a FHIR®-based technical standard for computable guidelines known as CPG-on-FHIR®.²⁴ A foundational tenet of the approach described in CPG-on-FHIR® is the concept of one faithful representation of the written guideline in computable format (ie, the computable guideline) with many ways to implement it. This minimizes the effort required to translate the written guideline into computable form while increasing the ways the computable content can be reconfigured into different types of digital tools (ie, guideline derivatives) to maximize flexibility and minimize burden in implementation.³⁴ CPG-on-FHIR® includes an approach for multidisciplinary collaboration across the lifecycle of guideline-directed care and related activities, a methodology for representing a guideline in different formats (ie, narrative, semistructured, structured, and executable—or L1, L2, L3, and L4), profiles and terminology that include definitions and documentation of how concepts are represented, computable guideline examples, and more. A COVID-19 example is included as a CPG methodology case study, presenting an agile and integrated cross-functional approach.³⁵

The Communication and Dissemination Workgroup incorporated various tools and templates into the integrated process, including the Stakeholder Communication Analysis and Communication Standard Operating Procedures: A Checklist for Effective Communication and Dissemination.²¹ They also identified ways to disseminate written and computable guidelines. Incorporating the communication perspective at the earliest phases (ie, 0 and 1) of the integrated process was key, beginning with the development of a communication plan that outlines the audience, strategy, and tactics for a guideline development and implementation.

The Implementation Workgroup developed detailed checklists to facilitate representing guidelines in different knowledge formats. These checklists became part of CPG-on-FHIR®’s methodology.²⁷ The L2 Checklist ensures that informaticists and knowledge engineers have what they need from L1 content in order to successfully produce L3 content, represented in a semistructured format such as decision trees or flow charts. The L3 Checklist ensures that L3 structured content meets the requirements for implementers to have what they need in order to successfully implement computable content in a specific setting, such as an outpatient primary care clinic or hospital ward. The L4 Checklist supports the process of implementing in a specific setting, helping to ensure successful and faithful implementation of guideline content in executable format within the EHR or through external services that integrate with the EHR.³⁵ Future versions of CPG-on-FHIR® may include revisions to these checklists, such as adding an L1 checklist, as more groups use them in their guideline projects.

The Evaluation Workgroup produced an evaluation framework and tool to be used in parallel with the integrated process during each of its 12 phases.^29,30 The evaluation assesses the process, products, and outcomes for a guideline. For processes, the evaluation is intended to assure steps are carried out as intended. Product evaluation assesses the quality, usability, and user acceptance of the written and computable guideline. Outcomes are evaluated to determine the extent to which the guideline is achieving its intent. For example, to evaluate the process of drafting the scope of the guideline, the tool assesses to what extent the multidisciplinary team engaged in codeveloping the narrative and computable guidelines. To evaluate outcome, an example includes determining to what extent the target audience is using the guideline for patient management.

Discussion

A systems-thinking approach,³ together with the adapted Kaizen method,¹⁷ incorporated the perspectives of many partners and considered many possible solutions to improve the process of CPG development and implementation. For example, the clarity of the wording in the recommendations impacts translation into digital tools and implementation into clinical workflows. Therefore, developing computable guidelines and considering their implementation into clinical workflows while the written guideline is being developed could improve its clarity and adoption in patient care.

The products of the ACG initiative are complementary and interrelated. While the ACG products are meant to be used together, the product of the workgroup most aligned with one’s perspective or expertise would be the best one to begin with. For example, a guideline developer would begin with the integrated process (ie, the Guideline Creation Workgroup’s product) while an informatics expert would begin with the CPG-on-FHIR® IG (ie, the Informatics Workgroup’s product). The Guideline Creation Workgroup’s integrated process model^20,21 provides a comprehensive method and can be tailored based on the needs, resources, and available expertise. However, without a standardized, interoperable, scalable way to faithfully represent guideline recommendations in computable form, codeveloping, disseminating, and implementing a computable guideline—a key feature of the integrated process—would not be possible. The technical health IT standard (CPG-on-FHIR®),²⁴ the primary product of the Informatics and Implementation Workgroups, provides a standardized, interoperable way to faithfully represent guideline recommendations and similar types of knowledge in computable form and checklists for how to create each level of knowledge. The Communication Workgroup’s tools and inputs in the integrated process incorporate communications planning from the beginning.²¹ To measure whether these processes and standards are achieving their aims, the Evaluation Workgroup developed an evaluation framework and tool to assess the integrated process, products, and outcomes of a specific guideline development and implementation effort.^29,30

In a related effort, the World Health Organization (WHO) also set out to develop a new approach to help member nations apply recommendations in practice, an effort called Standards-based, Machine-readable, Adaptive, Requirements-based, and Testable (SMART) Guidelines.^36,37 WHO began by making a specific guideline (antenatal care)³⁸ computable and planned to expand their approach to other guidelines. This is the converse of the CDC-led approach of reviewing and improving the entire system of guideline development and implementation and then applying the improved approach to specific guidelines. CDC and WHO have leveraged each other’s work to help achieve the same goals. The WHO’s SMART Guidelines initiative began using and contributing to CPG-on-FHIR® and plans to incorporate elements of the integrated process and evaluation framework. CDC’s ACG initiative added WHO’s Antenatal Care Guidelines example into CPG-on-FHIR® and learned from their experience in developing computable guidelines.

The various products of the ACG initiative are meant to be used in combination to support a redesigned and integrated process for codevelopment of written and computable guidelines using international health IT standards to promote interoperability. Realizing this approach, however, remains a challenge. The coordination of many partners within guideline development and implementation efforts can be difficult. Some guideline recommendations may not be suitable for computable translation. The adoption of health IT standards necessary to support computable guidelines is fragmented and incomplete.^39,40 In the United States, the Cures Act Final Rule³⁹ and Interim Final Rule,⁴⁰ regulatory policies from the Office of the National Coordinator for Health IT, include several interoperability requirements to improve robust use of FHIR®. In addition to procedural and technical challenges, some organizations are not culturally ready or do not have the resources or technical expertise needed to achieve the proposed future state.

While some projects have begun applying this new approach to guideline development and implementation,¹⁷ more are needed to determine how well the integrated process and computable guideline standards work, what needs to be improved, and the impact of the approach on the ease, speed, accuracy, and consistency of guideline development and implementation. An economic impact analysis would help determine the effect of this new approach on the overall health care system.

The ACG approach allows for modular updates to guideline recommendations and could also work well for living guidelines, defined as an optimization of the standard guideline process, as it allows updating of individual recommendations as soon as new relevant evidence becomes available.⁴¹ Applying a framework for living guidelines⁴² while implementing the computable CPG recommendations would be important to evaluate.⁴³

Conclusion

The ACG initiative’s holistic approach and engagement with a multidisciplinary, multiorganizational group of partners has resulted in a set of innovative products for improved guideline development and implementation that redesign the process, establish health IT standards, provide more comprehensive communication and dissemination planning, and offer an evaluation framework. While the new approach still has some challenges and limitations, it offers a systems-thinking solution for a complex process with many components. The products of the ACG initiative need to be applied to more guideline development and implementation efforts so that they can be further tested and continuously improved. The ultimate goal is a seamless, quick, accurate, and consistent translation and adoption of evidence-based guidelines into patient care.

Acknowledgments

My deepest gratitude to the leads of each ACG workgroup for their intense dedication in guiding the work of their respective group in producing the innovative products described in this article: Dyann Matson-Koffman and Priya Jakhmola (Guideline Creation), Bryn Rhodes and Wendy Blumenthal (Informatics), DuWayne Willet and Michelle Dardis (Implementation), Dana Pitts (Communication), and Ira Lubin, Amrita Tailor, and Susan Robinson (Evaluation). My thanks to Julia Skapik, Blackford Middleton, Laura Fochtman, and Natschja Ratanaprayul for helping hone the initial ideas to summarize the ACG initiative for this article. I appreciate Mindy Hangsleben and Lindsay Wisham who facilitated discussions and were key to the planning, executing, and following up after the Kaizen event. My sincere appreciation to Cindy Johnson, Aubrey Garner, Frank Dilley, Linda Roesch, Pam Lutz, Dana Pitts, Robin Wagner, Matt Eblen, Laura Mann, and Merita Craddock for their help with organizing CDC’s inaugural Kaizen event. I am indebted to Chesley Richards for serving as the executive sponsor and champion of the ACG initiative.

Conflicts of interest

The author has no conflicts of interest to disclose.

Disclaimer

The findings and conclusions in this article are those of the author and do not necessarily represent the official position of the Centers for Disease Control and Prevention.