ABSTRACT
Background and Objectives
The relevance of Cochrane systematic reviews depends on timely completion and updates. This study aimed to empirically assess the lifecycles of Cochrane reviews published from 2003 to 2024, including transitions from protocol to review, update patterns, and withdrawals.
Methods
We extracted data from Cochrane Library publications between 2003 and 2024. Each review topic was identified using a unique six‐digit DOI‐based ID. We recorded protocol publication, review publication, updates, and withdrawals (i.e., removed from the Cochrane Library for editorial or procedural reasons), calculating time intervals between stages and conducting subgroup analyses by review type.
Results
Of 8137 protocols, 71.9% progressed to reviews (median 25.7 months), 2.4% were updated during the protocol stage, and 10.0% were withdrawn. Among 8477 reviews, 64.3% were never updated by the time of our analysis; for those updated at least once, the median interval between updates was 57.2 months. Withdrawal occurred in 2.5% of reviews (median 67.6 months post‐publication). Subgroup analyses showed variation across review types; diagnostic and qualitative reviews tended to have longer protocol‐to‐review times than other types of reviews.
Conclusions
Cochrane reviews show long development and update intervals, with variation by review type. Greater use of automation and targeted support may improve review efficiency and timeliness.
Keywords: Cochrane systematic reviews, lifecycle, meta‐research, review update, withdrawal
1. Introduction
Systematic reviews are essential tools in healthcare, synthesizing evidence to inform clinical practice, guideline development, and policy decisions. Cochrane systematic reviews, hosted in the Cochrane Library, are recognized for their rigorous methodology and structured updating processes and are often considered the “gold standard.” [1, 2, 3] Cochrane systematic reviews typically start as detailed protocols that progress to published reviews, subsequently updated to maintain relevance and accuracy over time. Timely updates are critical, yet maintaining a balance between methodological rigor and speed remains challenging, especially given the increasing availability of automated and AI‐assisted review methodologies [4, 5, 6].
Previous studies have demonstrated variability in the time taken from protocol to full review, update frequency, and withdrawal rates within Cochrane systematic reviews [7, 8]. However, few comprehensive empirical analyses have tracked entire lifecycles, including protocols, published reviews, updates, and withdrawals, over extended periods. This study aims to comprehensively evaluate the lifecycles of a large collection of Cochrane systematic reviews, analyzing key intervals from protocols to reviews, update patterns, and subgroup differences across review types. By providing a detailed characterization, we aim to inform future strategies for the production and maintenance of systematic reviews, particularly the integration of automated technologies.
2. Methods
We systematically extracted data from publicly available records in the Cochrane Library, spanning Issue 1 of 2003 to Issue 12 of 2024. We selected 2003 as the starting point because the Cochrane Library began systematically indexing reviews and protocols with complete metadata from that year onward. This period also represents the majority of Cochrane systematic reviews, ensuring a comprehensive and representative data set. We identified each publication via its unique Digital Object Identifier (DOI). Publications were categorized as protocols, published reviews, updated protocols or reviews, or withdrawal notices. Editorials and methodological articles were excluded.
We used R (version 4.5.1) with the “rvest” package (version 1.0.4) to extract DOI information and publication dates from Cochrane Library webpages. We then cleaned and reshaped the data using the “dplyr” package (version 1.1.4) and performed date calculations with the “lubridate” package (version 1.9.4). Data organization and management were conducted using Microsoft Excel. Each Cochrane systematic review topic has a unique identifier in the format CDXXXXXX (a “CD” prefix followed by six numerical digits), embedded in its DOI, enabling comprehensive lifecycle tracking. For each ID, we documented whether the topic originated as a protocol, whether it transitioned into a published review, how many updates it underwent and when, and whether it was eventually withdrawn and at what point in the timeline.
We calculated intervals (in months) between critical stages, including from the last protocol publication to the first published review, intervals between review updates, and from the last review publication to withdrawal (if applicable). We calculated descriptive statistics, including medians, means, interquartile ranges (IQRs), and percentages. We also conducted subgroup analyses based on review type: diagnostic, intervention, overview, qualitative, rapid, prognosis, and prototype.
3. Results
Figure 1 illustrates the lifecycles of Cochrane systematic reviews, with detailed results presented in Table 1. The Cochrane Library included 8137 records of protocols, of which 2.4% were updated during the protocol stage, while 10.0% were eventually withdrawn (median time = 60.5 months from the most recent protocol to withdrawal). Of note, this count reflects all protocol versions published between 2003 and 2024, including multiple versions for the same topic, such as protocols that were subsequently updated, withdrawn, or progressed to completed reviews. Additionally, 71.9% of protocols progressed to a full review, with a median transition time of 25.7 months (IQR 15.3–42.0), consistent with prior research reporting a median of approximately 2.4 years [7].
Figure 1.
Diagram illustrating the lifecycles of Cochrane systematic reviews. Note: Cochrane systematic reviews do not routinely specify whether further updates will be undertaken or if the evidence is considered conclusive. As a result, information on reviews retired due to conclusiveness is largely unavailable, although most reviews will eventually cease to be updated.
Table 1.
Descriptive statistics of Cochrane systematic reviews, protocols, and update time gaps.
| Type (Count) | Stage | Count (Percentage) | Median Time (Month) | IQRa (Month) | Mean Time (Month) |
|---|---|---|---|---|---|
| Protocols (8137) | With no updates | 7938 (97.6%) | NA | NA | NA |
| With at least 1 updateb | 199 (2.4%) | 42.1 | 18.5–68.2 | 47.4 ± 36.5 | |
| Eventually led to reviewsc | 5852 (71.9%) | 25.7 | 15.3–42.0 | 32.8 ± 25.5 | |
| Eventually withdrawnd | 816 (10.0%) | 60.5 | 37.9–95.5 | 69.4 ± 41.5 | |
| Reviews (8477) | Overalle | NA | 57.2 | 37.5–82.3 | 64.1 ± 36.8 |
| With no update | 5447 (64.3%) | NA | NA | NA | |
| With 1 updateb | 1956 (23.1%) | 63.8 | 38.8–94.6 | 70.4 ± 41.7 | |
| With 2 updatesb | 723 (8.5%) | 54.8 | 39.2–72.0 | 56.7 ± 22.7 | |
| With at least 3 updatesb | 351 (4.1%) | 42.7 | 32.7–54.1 | 44.1 ± 15.4 | |
| Eventually withdrawnd | 211 (2.5%) | 67.6 | 47.8–102.9 | 73.1 ± 40.5 |
Abbreviation: NA, not applicable.
Interquartile range.
For protocols or reviews with exactly one update, the update time is calculated as the difference between the date of the last version and the date of the first version. For protocols or reviews with two or more updates, the (averaged) update time is calculated as the difference between the date of the last version and the date of the first version, divided by the number of updates.
Time from the publication of the final protocol version to the publication of the first full review.
Time from the publication of the final protocol or review version to the withdrawal date, excluding those reviews that had an additional protocol or review published after withdrawal.
Calculated among reviews that underwent at least one update.
Table 2 presents the results of the most frequent subgroups of Cochrane reviews. The median transition times were 31.1 months for diagnostic reviews, 25.6 for intervention reviews, 22.0 for overviews, 32.6 for qualitative reviews, 24.0 for prognosis reviews, and 22.1 for prototype reviews.
Table 2.
Descriptive statistics for subgroups of intervention, diagnostic, overview, prognosis, prototype, qualitative, and rapid reviews.
| Subgroup | Type (Count) | Stage | Count (Percentage) | Median Time (Month) | IQR (Month) | Mean time ± SD (Month) |
|---|---|---|---|---|---|---|
| Intervention | Protocols (7705) | With no updates | 7511 (97.5%) | NA | NA | NA |
| With at least 1 update | 194 (2.5%) | 42.6 | 18.3–69.1 | 47.6 ± 36.8 | ||
| Eventually led to reviews | 5596 (72.6%) | 25.6 | 15.2–42.0 | 32.7 ± 25.7 | ||
| Eventually withdrawn | 782 (10.1%) | 61.0 | 37.8–95.2 | 69.6 ± 41.6 | ||
| Reviews (8141) | With no update | 5154 (63.3%) | NA | NA | NA | |
| With at least 1 update | 2987 (36.7%) | 57.3 | 50.0–123.2 | 64.3 ± 36.8 | ||
| Eventually withdrawn | 210 (2.6%) | 67.7 | 47.8–103.0 | 73.2 ± 40.5 | ||
| Diagnostic | Protocols (248) | With no updates | 245 (98.8%) | NA | NA | NA |
| With at least 1 update | 3 (1.1%) | 43.2 | 30.7–61.2 | 46.9 ± 30.7 | ||
| Eventually led to reviews | 154 (62.1%) | 31.1 | 18.8–48.0 | 36.8 ± 23.9 | ||
| Eventually withdrawn | 25 (10.1%) | 54.1 | 47.1–102.4 | 70.4 ± 38.4 | ||
| Reviews (192) | With no update | 163 (84.9%) | NA | NA | NA | |
| With at least 1 update | 29 (15.1%) | 55.8 | 28.7–77.3 | 54.2 ± 31.1 | ||
| Eventually withdrawn | 0 (0.0%) | — | — | — | ||
| Overview | Protocols (87) | With no updates | 85 (97.7%) | NA | NA | NA |
| With at least 1 update | 2 (2.3%) | 32.5 | — | 32.5 ± 2.1 | ||
| Eventually led to reviews | 63 (72.4%) | 22.0 | 11.3–37.0 | 26.7 ± 21.3 | ||
| Eventually withdrawn | 7 (8.0%) | 51.0 | 33.5–78.5 | 56.4 ± 32.3 | ||
| Reviews (70) | With no update | 63 (90.0%) | NA | NA | NA | |
| With at least 1 update | 7 (10.0%) | 35.6 | 28.7–77.3 | 42.2 ± 41.9 | ||
| Eventually withdrawn | 1 (1.4%) | 62.6 | — | — | ||
| Prognosis | Protocols (47) | With no updates | 47 (100.0%) | NA | NA | NA |
| With at least 1 update | 0 (0.0%) | — | — | — | ||
| Eventually led to reviews | 20 (42.6%) | 24.0 | 14.1–38.4 | 27.8 ± 17.0 | ||
| Eventually withdrawn | 0 (0.0%) | — | ||||
| Reviews (21) | With no update | 20 (95.2%) | NA | NA | NA | |
| With at least 1 update | 1 (4.8%) | 3.9 | — | — | ||
| Eventually withdrawn | 0 (0.0%) | — | — | — | ||
| Prototype | Protocols (8) | With no updates | 8 (100.0%) | NA | NA | NA |
| With at least 1 update | 0 (0.0%) | — | — | — | ||
| Eventually led to reviews | 3 (37.5%) | 22.1 | — | 18.3 ± 8.1 | ||
| Eventually withdrawn | 0 (0.0%) | — | — | — | ||
| Reviews (11) | With no update | 10 (90.9%) | NA | NA | NA | |
| With at least 1 update | 1 (9.1%) | 30.2 | — | — | ||
| Eventually withdrawn | 0 (0.0%) | — | — | — | ||
| Qualitative | Protocols (41) | With no updates | 41 (100.0%) | NA | NA | NA |
| With at least 1 update | 0 (0.0%) | — | — | — | ||
| Eventually led to reviews | 25 (61.0%) | 32.6 | 19.7–38.9 | 33.0 ± 16.8 | ||
| Eventually withdrawn | 1 (2.4%) | 3.3 | — | — | ||
| Reviews (30) | With no update | 29 (96.7%) | NA | NA | NA | |
| With at least 1 update | 1 (3.3%) | 0.3 | — | — | ||
| Eventually withdrawn | 0 (0.0%) | — | — | — | ||
| Rapid | Protocols (1) | With no updates | 1 (100.0%) | NA | NA | NA |
| With at least 1 update | 0 (0.0%) | — | — | — | ||
| Eventually led to reviews | 1 (100.0%) | 4.3 | — | — | ||
| Eventually withdrawn | 0 (0.0%) | — | — | — | ||
| Reviews (13) | With no update | 10 (76.9%) | NA | NA | NA | |
| With at least 1 update | 3 (23.1%) | 17.0 | — | 16.9 ± 10.6 | ||
| Eventually withdrawn | 0 (0.0%) | — | — | — |
Note: Some results are not reported due to the absence or limited availability of data. See the notes in Table 1 for details.
Beyond the protocol stage, the Cochrane Library documented 8477 reviews, of which 64.3% were never updated by the time of our analysis, 23.1% had one update, 8.5% had two updates, and 4.1% had three or more updates. Among reviews that underwent at least one update, the median time between updates was 57.2 months. We report these results stratified by the number of updates (i.e., 1 update, 2 updates, or at least 3 updates) in Table 1, which reflects the natural decrease in sample size with each successive update.
In addition, 211 reviews (2.5%) were ultimately withdrawn, with a median time of 67.6 months from the final review publication to withdrawal. In 54 cases, the withdrawal notice preceded the nominal “final” review date, likely due to editorial arrangements.
4. Discussion
4.1. Implications
This large‐scale empirical analysis demonstrates substantial delays and variability within Cochrane systematic review lifecycles, particularly in protocol‐to‐review transitions and subsequent updates. While longer intervals do not inherently mean that systematic reviews are out‐of‐date, since some topics may not warrant frequent updates, they do highlight the potential risk of lagging behind new evidence, especially in rapidly evolving fields. Our findings align with prior research indicating that many systematic reviews may not keep pace with the rate of new evidence generation [7, 8], raising concerns about their continued practical relevance and applicability in some contexts.
The long median intervals identified raise concerns about the potential obsolescence of reviews, especially in rapidly evolving clinical fields. Although Cochrane policies strongly encourage regular updating, our findings suggest that policy alone is insufficient to ensure timely reviews [9]. Automation and integration of AI technologies could substantially reduce manual workload, streamline updates, and improve the responsiveness of reviews to emerging evidence [10, 11]. Recent advances in AI demonstrate its potential to expedite critical review processes, such as literature screening and data extraction, significantly reducing update intervals [12, 13]. However, while AI shows promise in facilitating the conduct of new systematic reviews, its role in accelerating updates remains less certain due to the complexities of integrating new evidence into existing syntheses and reassessing conclusions [14]. Future research should explore how AI tools might be adapted or developed to address these specific challenges in the updating process.
Our subgroup analyses identified variations in review lifecycles based on review type. Diagnostic and qualitative reviews had notably longer transitions from protocol to review publication. This may not necessarily reflect more complex methodologies, but could instead be due to the greater need for manual effort and human judgment, such as assessing nuanced diagnostic criteria or synthesizing nonquantitative data. Recognizing these labor‐intensive demands, Cochrane and systematic review groups could implement tailored strategies to complement existing resources and infrastructure. These might include expanding capacity, improving workflow efficiency, and developing novel tools such as semi‐automated data extraction or AI‐assisted qualitative synthesis to reduce manual effort and further improve lifecycle efficiency.
In parallel with technological innovations, Cochrane has also introduced focused review formats with simplified reporting requirements to help reduce the workload for authors and editors. These formats may complement automation efforts by allowing for more targeted, question‐specific reviews that can be completed and updated more efficiently, particularly when rapid evidence synthesis is needed.
4.2. Strengths and Limitations
A strength of this study is its extensive scope, covering over two decades and more than 22,000 Cochrane publications (including different versions), enabling robust characterization of lifecycle patterns. However, limitations include reliance on publicly available data, which lack detailed contextual information such as internal editorial decisions, communication with authors, resource constraints, or topic prioritization processes. Without access to these internal factors, it is difficult to determine why some reviews are delayed, never completed, not updated, or withdrawn. Further qualitative research could illuminate these internal dynamics, helping to design more effective strategies for review management.
In addition to qualitative approaches, further empirical research should consider analytical methods to assess the necessity and timing of updates relative to evolving evidence. Some reviews may remain valid without frequent updates, while others quickly become outdated [15, 16]. A lack of updates does not necessarily indicate inefficiency; in many cases, no new studies or no impactful evidence may have emerged to warrant revision. Differentiating between reviews that are outdated and those that remain current due to a stable evidence base is essential for prioritizing updates effectively. Cochrane has recently introduced a decision framework to guide authors and editors in determining when to update reviews, incorporating considerations such as evolving evidence, changes in interventions, and methodological advancements. Continued development of objective, data‐driven criteria could further support prioritization efforts, helping allocate limited resources more effectively across diverse review portfolios.
Future studies could also examine temporal trends, such as whether review development times and update frequencies have changed over successive 5‐year periods. This may reveal shifts in practices and efficiencies over time. Additionally, examining the outcomes and consequences of withdrawn reviews, including whether they are ever reinstated or revised, could provide deeper insights into systematic review lifecycle management and inform future editorial and methodological policies.
5. Conclusions
Our study highlights the need for improved methods to manage the lifecycle of Cochrane systematic reviews. Automated methods and AI could notably enhance efficiency, reduce delays, and ensure timely integration of emerging evidence, thereby maintaining the relevance and clinical utility of systematic reviews. Addressing methodological complexity through tailored strategies, clarifying update criteria, and exploring internal review dynamics could further optimize future systematic review production.
Author Contributions
Shiyin Li: methodology, formal analysis, investigation, writing – original draft. Chong Wu: data curation, writing – review and editing, funding acquisition. Zichen Zhang: data curation, writing – review and editing. Mengli Xiao: writing – review and editing. Mohammad Hassan Murad: writing – review and editing. Lifeng Lin: conceptualization, methodology, data curation, writing – review and editing, supervision, funding acquisition.
Ethics Statement
This study is based on publicly available Cochrane systematic reviews and does not involve human participants or identifiable personal data. Therefore, it was exempt from ethics committee approval.
Conflicts of Interest
The authors declare no conflicts of interest.
Peer Review
The peer review history for this article is available at https://www.webofscience.com/api/gateway/wos/peer-review/10.1002/cesm.70043.
Acknowledgments
ChatGPT‐4o and ChatGPT‐4.5 (OpenAI) were used in June 2025 exclusively to enhance the writing of this manuscript and were not utilized for any other aspects of this study. L.L. was partially supported by the U.S. National Institute of Mental Health (R03 MH128727), the U.S. National Library of Medicine (R21 LM014533, R01 LM012982), the National Institute on Aging (R03 AG093555), and the Arizona Biomedical Research Centre (RFGA2023‐008‐11). The content is solely the responsibility of the authors and does not necessarily reflect the official views of the U.S. National Institutes of Health or the Arizona Department of Health Services. The funders had no role in the study design, data collection, analysis, interpretation, or manuscript writing.
Data Availability Statement
The data used in this manuscript are directly sourced from published Cochrane systematic reviews. They are also available from the corresponding author upon reasonable request.
References
- 1. Jadad A. R., Cook D. J., Jones A., et al., “Methodology and Reports of Systematic Reviews and Meta‐Analyses: A Comparison of Cochrane Reviews With Articles Published in Paper‐Based Journals,” Journal of the American Medical Association 280, no. 3 (1998): 278–280, 10.1001/jama.280.3.278. [DOI] [PubMed] [Google Scholar]
- 2. Green‐Hennessy S., “Cochrane Systematic Reviews for the Mental Health Field: Is the Gold Standard Tarnished?,” Psychiatric Services 64, no. 1 (2013): 65–70, 10.1176/appi.ps.001682012. [DOI] [PubMed] [Google Scholar]
- 3. Seidler A. L., Hunter K. E., Cheyne S., Berlin J. A., Ghersi D., and Askie L. M., “Prospective Meta‐Analyses and Cochrane's Role in Embracing Next‐Generation Methodologies,” Cochrane database of systematic reviews 10 (2020): 000145, 10.1002/14651858.ED000145. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Riaz I. B., Naqvi S. A. A., Hasan B., and Murad M. H., “Future of Evidence Synthesis: Automated, Living, and Interactive Systematic Reviews and Meta‐Analyses,” Mayo Clinic proceedings. Digital health 2, no. 3 (2024): 361–365, [published Online First: 20240608], 10.1016/j.mcpdig.2024.05.023. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Bashir R., Surian D., and Dunn A. G., “Time‐To‐Update of Systematic Reviews Relative to the Availability of New Evidence,” Systematic Reviews 7, no. 1 (2018): 195, 10.1186/s13643-018-0856-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Murad M. H., Wang Z., Chu H., et al., “Proposed Triggers for Retiring a Living Systematic Review,” BMJ Evidence‐Based Medicine 28, no. 5 (2023): 348–352, 10.1136/bmjebm-2022-112100. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Tricco A. C., Brehaut J., Chen M. H., and Moher D., “Following 411 Cochrane Protocols to Completion: A Retrospective Cohort Study,” PLoS One 3, no. 11 (2008): e3684, 10.1371/journal.pone.0003684. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Andersen M. Z., Gülen S., Fonnes S., Andresen K., and Rosenberg J., “Half of Cochrane Reviews Were Published More Than 2 Years After the Protocol,” Journal of Clinical Epidemiology 124 (2020): 85–93, 10.1016/j.jclinepi.2020.05.011. [DOI] [PubMed] [Google Scholar]
- 9. Runjic E., Behmen D., Pieper D., et al., “Following Cochrane Review Protocols to Completion 10 Years Later: A Retrospective Cohort Study and Author Survey,” Journal of Clinical Epidemiology 111 (2019): 41–48, 10.1016/j.jclinepi.2019.03.006. [DOI] [PubMed] [Google Scholar]
- 10. Hasan B., Saadi S., and Rajjoub N. S., et al, “Integrating Large Language Models in Systematic Reviews: A Framework and Case Study Using Robins‐I for Risk of Bias Assessment,” BMJ Evid Based Med 29 (2024): 394–398, 10.1136/bmjebm-2023-112597. [DOI] [PubMed] [Google Scholar]
- 11. Bernard N., Y. Sagawa, Jr. , Bier N., Lihoreau T., Pazart L., and Tannou T., “Using Artificial Intelligence for Systematic Review: The Example of Elicit,” BMC Medical Research Methodology 25, no. 1 (2025): 75, 10.1186/s12874-025-02528-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Konet A., Thomas I., Gartlehner G., et al., “Performance of Two Large Language Models for Data Extraction in Evidence Synthesis,” Research Synthesis Methods 15, no. 5 (2024): 818–824, 10.1002/jrsm.1732. [DOI] [PubMed] [Google Scholar]
- 13. Gartlehner G., Kahwati L., Nussbaumer‐Streit B., et al, “From Promise to Practice: Challenges and Pitfalls in the Evaluation of Large Language Models for Data Extraction in Evidence Synthesis,” BMJ Evid Based Med (2024), In Press, 10.1136/bmjebm-2024-113199. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Bolaños F., Salatino A., Osborne F., and Motta E., “Artificial Intelligence for Literature Reviews: Opportunities and Challenges,” Artificial Intelligence Review 57, no. 10 (2024): 259, 10.1007/s10462-024-10902-3. [DOI] [Google Scholar]
- 15. Brok J., Thorlund K., Gluud C., and Wetterslev J., “Trial Sequential Analysis Reveals Insufficient Information Size and Potentially False Positive Results in Many Meta‐Analyses,” Journal of Clinical Epidemiology 61, no. 8 (2008): 763–769, 10.1016/j.jclinepi.2007.10.007. [DOI] [PubMed] [Google Scholar]
- 16. Jaidee W., Moher D., and Laopaiboon M., “Time to Update and Quantitative Changes in the Results of Cochrane Pregnancy and Childbirth Reviews,” PLoS One 5, no. 7 (2010): e11553, 10.1371/journal.pone.0011553. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
The data used in this manuscript are directly sourced from published Cochrane systematic reviews. They are also available from the corresponding author upon reasonable request.