Living Systematic Reviews and Living Guidelines to Maintain the Currency of Public Health Guidelines

Globally, considerable resources and community trust are invested in the development of evidence-based guidelines for the prevention and treatment of a range of public health issues. Best practice guideline development processes draw on well-conducted systematic reviews of the contemporary evidence base to formulate recommendations. However, evidence underpinning many published public health guidelines may quickly become outdated.

Living systematic review methods, in which new research is continually identified and synthesized, can support the production of “living guidelines” that incorporate evidence into their recommendations as soon as it is available. Such processes have the potential to enhance the speed of translating the latest research evidence into public health practice and improve health outcomes. Living guidelines offer an opportunity to preserve the currency of guidelines and may be especially beneficial for priority public health issues in which the evidence base is uncertain and emerging. This opinion editorial aims to provide guidance and examples on how to apply living guidelines in public health to improve the quality of systematic reviews in public health.

PUBLIC HEALTH GUIDELINES

Guidelines are fundamental for public health to support evidence-based decision making by considering intervention options, scientific evidence to support their effects, and other important considerations (e.g., cost, equity). Ideally, they provide specific recommendations about the appropriate actions for stakeholders to take when addressing a health problem, enabling policymakers and health care providers to select the most effective interventions in the context of any feasibility or practical constraints.

Various international agencies, including the World Health Organization, the Centers for Disease Control and Prevention, and the National Health and Medical Research Council, provide standards for the process of developing public health guidelines.^1–4 Although there are subtle variations in these guideline development processes, the core expectations and processes for guideline development are consistent, including that they are informed by the best available and up-to-date evidence generated from high-quality systematic reviews.^1–5

Traditional approaches to guideline development can be time consuming and sporadic. Guidelines can take years to produce, as they require the establishment of guideline development groups and the conduct of systematic reviews and deliberative processes that consider research evidence in the intended context. A years-long latency period between updates typically follows, often as a result of lack of funding. This means that the research evidence and the recommendations developed from that evidence can become quickly out of date, sometimes before the guideline is published. This slows translation of research evidence and delays health and health care improvements, often by many years.⁶

In the case of some topics, conclusions from research evidence remain stable over time because either little research is produced or the findings are consistent with or do not substantively challenge previous knowledge and understandings, and so no change to a recommendation is required. However, in active research areas, including priority public health research areas (e.g., COVID-19), evidence may be quickly generated with conclusions and recommendations rapidly changing. In such contexts, guideline recommendations are particularly susceptible to rapidly becoming out of date.⁶ For such public health priorities, innovative solutions are required to reduce delays in the evidence to translation pathway.

LIVING SYSTEMATIC REVIEWS

Living systematic reviews (LSRs) offer an opportunity to preserve the currency of the systematic review evidence that underpins guideline recommendations. LSRs are defined as follows:

a systematic review that is continually updated, incorporating relevant new evidence as it becomes available…this means continual surveillance for new research evidence through ongoing or frequent searches and the inclusion of relevant new information into the review in a timely manner so that the findings of the systematic review remain current.^7(p24)

Proposed in 2014, the LSR approach to updating systematic review evidence adopts standard methods, albeit with a higher frequency of record searching, synthesis, and publication, specified a priori. Although the approach can be applied to any type of review, not all systematic review questions are appropriate for an LSR. Three key criteria are required to be met before an LSR is initiated: (1) the review question is a priority for decision making, (2) certainty in the existing evidence is low or very low, and (3) there is likely to be new research evidence.⁷

The feasibility of the LSR approach to updating evidence has been demonstrated in a number of case studies, including a mixed-methods evaluation that highlighted its potential for reducing time and resource requirements.⁸ The feasibility of LSRs is also evidenced by their wide-scale adoption in a relatively short period of time. The first two LSRs were published in the Cochrane Library in 2017, and there are now more than 150 indexed in PubMed. Twenty LSRs have been published by Cochrane and 10 more are under way, three of which are related to public health. Published methods guidance is available to support the best practice conduct of LSRs.⁹

LIVING GUIDELINES

The application of LSR methods to guideline development is a recent innovation that addresses the issue of evidence currency and sporadic updating of guidelines. Living guidelines have been seminally defined as optimization of the guideline development process to allow updating of individual recommendations as soon as relevant new evidence becomes available.¹⁰ Living recommendations are defined as recommendations that are updated as soon as relevant new evidence becomes available.

As with the LSR approach, a living guideline process can be applied to any health topic and applied to an entire guideline or individual recommendations. Similar to LSRs, criteria to consider whether a living guideline process is appropriate include the following: (1) the recommendation is a priority for decision making (e.g., areas of known variation in practice), (2) there is a reasonable chance that the existing recommendations will change with the emergence of new evidence, and (3) there is likely to be new research evidence emerging.¹⁰

Dozens of living guidelines are currently in production, with a number focused on public health priority areas, including COVID-19.^11–15 A framework for developing living practice guidelines in health care,¹⁶ a handbook for developing living guidelines,⁹ and a series of papers have recently been published to support best practice living guideline methods.^17–21

GUIDELINE DEVELOPMENT PROCESS

Although living guideline steps are broadly similar to traditional approaches to guideline production, the more frequent generation of research evidence and subsequent guideline updates necessitates some modification to traditional approaches to enable rapid production. The following are the six key elements of traditional guideline development processes adapted for the production of living guidelines¹⁰ (for more details, see Table A, available as a supplement to the online version of this article at http://www.ajph.org):

• 1.Identifying and synthesizing evidence via LSRs,
• 2.Generating a living evidence profile and evidence for a decision table,
• 3.Convening a living guideline panel and peer reviewers,
• 4.Engaging and supporting agencies that provide guideline approval,
• 5.Identifying appropriate living publication and dissemination pathways, and
• 6.Establishing a sufficient funding and production team.

Another important element is “living prioritization,” which refers to ongoing review and consideration of which guideline recommendations meet the criteria for continual maintenance as a living approach.

CONSIDERATIONS FOR LIVING GUIDELINES IN PUBLIC HEALTH

Living guidelines have been successfully adopted across a broad range of health topics including stroke, diabetes, COVID-19, and maternal health.^14,15,22,23 Living review and guideline processes to date have been generically developed, and practical considerations for their operationalization (including methods for evidence surveillance and frequency) were discussed in a recent Journal of Clinical Epidemiology series.^17–21 However, there are a number of other aspects to consider when applying these methods to public health. A discussion of those we consider most important follows, including examples from existing living review and guideline development teams.

Study Designs and Outcomes

Although LSRs synthesizing research of any study design can underpin living guidelines, most to date have been applied to clinical care and treatment interventions, for which randomized controlled trials and outcomes amenable to synthesis via meta-analysis are standard. Many public health interventions and policies (e.g., COVID-19 mask mandates, infection control measures) are not as amenable to randomized designs. Therefore, evidence from nonrandomized studies, including opportunistic studies in which interventions may already have been implemented as part of standard care, may be required to form guideline recommendations. In such instances, existing public health surveillance data or existing clinical or other databases may be available for synthesis.

In addition, some important outcomes of public health interventions are not typically amenable to pooling via meta-analysis (e.g., acceptability) and are synthesized narratively. Synthesis without meta-analyses of findings from nonrandomized study designs and certain outcomes (e.g., qualitative outcomes) has the potential to increase the complexity and resource demand of LSRs and the development of living guidelines. Although these challenges require careful consideration, they are not insurmountable (see Table B, available as a supplement to the online version of this article at http://www.ajph.org).

Alignment of Research Questions and Living Guidelines

Identification of existing LSRs aligned with the focus of a living guideline can decrease resource requirements and increase feasibility. In such cases, surveillance for updated LSR results can be incorporated into the living guideline workflow. International databases of systematic reviews, such as PROSPERO²⁴ and Cochrane, provide a mechanism to identify prospectively registered LSRs and offer some information on progress.

The feasibility of using existing LSRs for living guidelines relies on the speed of their publication. For fast-moving public health topics (e.g., COVID-19), the possibility of existing or planned LSRs aligned with guidelines is more likely. In these cases, collaborations can be established between LSR author and guideline teams, including collaborations across guideline group jurisdictions to reduce duplication and improve efficiency. Sharing of review findings within these collaborations, including details on new studies or preprint review results, is important in enabling evidence to be incorporated into the guideline development process.

Existing LSRs may be less likely for other public health topics, and conversion of existing systematic reviews or establishment of new LSRs is required to enable a living guideline. Although there are obvious resource implications of initiating a new LSR, it allows an opportunity for direct alignment in research questions between the LSR and guideline, reducing complexity in the guideline development process.

Many challenges of using LSRs to underpin living guidelines are similar to those involved in using standard systematic reviews to inform standard guidelines. LSRs may not be updated rapidly enough, there may be multiple LSRs that cover the same trials, or LSRs might not cover the exact PICO of interest for the guideline. In these cases, existing LSRs can still be helpful in identifying studies or assessing analyses conducted by the guideline developers.

Editorial, Peer Review, and Publication Processes

Coordinated models of peer review, publication, and consideration by guideline approval agencies are necessary if LSRs and living guidelines are to provide timely and current evidence-based guideline recommendations. This can be challenging given that typical peer review to publication processes for systematic reviews can take up to 12 months, and guideline approval panels may meet less frequently than the frequency at which the living guideline is updated.

A growing number of journals and databases have modified their editorial, peer review, and publishing processes to support publication of LSRs (e.g., Cochrane, BMJ). The success of these editorial and publication models relies on the willingness to form close relationships and prompt communication between review author and editorial teams,⁸ with time lines and expectations for submission, review, and publication mutually agreed on and adhered to. The adoption of living guidelines in public health topics will be partly influenced by how many journals recognize the potential impact of this approach and develop accommodating systems and processes.

In the case of living guidelines for which monthly or more frequent searching and synthesis of new evidence are conducted, a flexible approach to review and endorsement by guideline approval agencies outside of planned meetings may be required (see Table B). Online publication platforms are also vital for living guidelines. Platforms such as MAGICApp²⁵ and the Infectious Diseases Society of America’s Practice Guideline App²⁶ can enable publication of multiple guideline versions, collaborative revision, and tracking of changes. Importantly, Web links for any derivative guidelines always take users to the most recent version of the guideline.

Resourcing and Sustainability

Similar to traditional guideline approaches, substantial resourcing (funding and people) is required to produce the initial iteration of a living guideline; however, significantly less resourcing is required to maintain the recommendations once the processes are established. In areas of public health priority that meet living guideline criteria, early experience suggests that although obtaining initial grant funding and engaging guideline development panel members are feasible,²⁷ sourcing ongoing funding (via grants or other mechanisms) and maintaining input over time may be challenging given that much funding operates on a project-based model. Living guidelines therefore may be most appropriate in cases in which stable funding is available (e.g., recurrent health agency funding).

Numerous technological enablers have been developed to reduce the resources required to conduct and maintain LSRs and living guidelines. This includes automation tools for database searching (e.g., machine learning) and identification of eligible research studies (e.g., crowd sourcing, Cochrane Crowd)²⁸ and online collaborative platforms for data extraction, analysis, and appraisal of study quality (e.g., Covidence, EPPIReviewer, Rayyan). Existing data sources (including big data) can also be leveraged for LSRs and living guidelines to reduce resource needs. Adoption of these tools can significantly reduce the resources required for evidence surveillance and generation and improve the sustainability of living guidelines (see Table B).

Living guidelines and LSRs are not intended to be maintained indefinitely. Similar to LSRs, guidance for living guidelines suggests that when any of the three criteria for considering a living guideline are not met, “living mode” should cease and instead standard guideline development methods should be adopted.¹⁰ This can include the living guideline recommendations no longer being a priority for decision makers (e.g., reduced prevalence or mortality of a condition); high certainty of the evidence, in which case new evidence would be unlikely to change recommendations; or no active studies or new evidence of existing or new intervention approaches being imminent. This situation can apply to an entire guideline or recommendations within the guideline, and regular reviews against the criteria should be conducted to determine the ongoing appropriateness of a living approach (see Table B).

Knowledge Translation and Implementation

Knowledge translation is a key issue for living guidelines, and given the novelty of the approach there is still much to learn. The benefits of reliable, continually updated recommendations for public health and health care will be realized only if communication, dissemination and implementation strategies are in place that can respond on the time cycle of guideline updates.²⁹ Newsletters, social media, app-based notification systems, and other flexible, dynamic approaches are likely to be needed. It is vital that knowledge translation mechanisms are in place that clearly communicate the “what is new” in each updated version of the guideline so that health decision makers can remain up to date with recommendations as they are revised.

Concerns have been raised about potential knowledge translation challenges caused by “flip-flopping” recommendations; however, this has not occurred in our experience so far. Usually, knowledge translation efforts focus on “strong” guideline recommendations for which there is a high level of certainty in the underlying evidence. In their nature and with respect to both stroke and COVID-19 guidelines,²² these types of recommendations established from a high level of certainty in the evidence have not substantively changed over time. In contrast, “conditional” or “weak” recommendations, in which the evidence is evolving and certainty in the evidence is low, are more likely to change over time and typically are not used to generate recommendations until the certainty of the evidence is higher.

There has been some limited investigation regarding the process of implementing updated living guideline recommendations³⁰ proportionate to their stage of innovation. The ongoing establishment of living guidelines and the creation of new guidelines will enable further investigation.

CONCLUSIONS

Living guidelines underpinned by LSRs offer an innovative, feasible, and effective solution to improving research on translation pathways by maintaining the currency of public health guidelines and ensuring that recommendations are based on the most up-to-date evidence. Although the feasibility of living guidelines for clinical care–focused public health topics has been demonstrated, adoption of living guidelines for other public health topics is evolving. Given the considerable potential for public health benefit, further application of living guidelines to other public health topics that meet living guideline criteria is warranted.