A new dawn for evidence synthesis: Embracing machine learning technology to generate living evidence maps

Citation counts in MEDLINE have risen from 579,041in 2004 to 981,270 in 2022 [1]. A 2021 publication estimates that there is an average worldwide growth rate of 4 % [2]. While the number of publications has increased across all fields of science, health has had the largest exponential growth as health professionals and researchers seek to generate and apply new knowledge [3]. There was an unprecedented surge in publications during the COVID-19 pandemic. Across both Royal Society of Public Health journals, Public Health and Public Health in Practice, there was a doubling of submissions between 2019 and 2020. Elsevier estimated submissions increased by around 92 % for their journals between February and May 2020 when compared to the previous period in 2019 [4].

With the rising number of scientific publications over the past decades, systematic reviews have been on the rise [5]. By synthesising large volumes of studies and literature, systematic reviews (SRs) are held as the gold standard and appear at the top of the hierarchy of evidence for clinical guidance and health care policy. The Cochrane Collaboration has augmented their dominance. However, there have been numerous critiques; on average it takes over a year to undertake and publish a review [6] and with large volumes of literature being published constantly, SRs can go out of date quickly [7].

Although the COVID-19 pandemic came with great challenges, it created opportunities to innovate. In the UK in February 2020 the National Institute for Health Research funded a collaboration between the EPPI Centre at University College London, the Centre for Reviews and Dissemination at the University of York, and the Public Health, Environments and Society at the London School of Hygiene and Tropical Medicine to produce an evidence map using a semi-automation, machine learning approach to map emerging COVID-19 evidence [8]. These timely living evidence maps enabled efficient identification of emerging evidence to improve prevention, diagnosis, treatment, and management of COVID-19.

Machine learning software has been in development for about 10 years, but recent developments have brought it to the fore. Opportunities include deduplication, automatic clustering of studies, priority screening and semi-automated data extraction. With the rise of technology and artificial intelligence changing all parts of society, researchers should embrace opportunities to innovate literature review methodology, and reimagine the future of evidence synthesis [9].

Evidence maps are a new variant on traditional reviews [10] and defined as a systematic synthesis which visually display relevant evidence to a research question. The scope of the map is generally broader than traditional SRs because machine learning technology is particularly good at dealing with topics with fuzzy boundaries [11]. For example, for complex topics such as what works to address health inequalities the technology can be trained to adeptly identify relevant articles. The Finding Accessible Inequalities Research in Public Health (FAIR) project used the technology to find, organise and describe public health interventions through an inequality lens [12]. These maps increase the visibility of findings and can gather large amounts of evidence, such as systematic reviews and primary studies to provide a more strategic approach to identify any gaps in the evidence. An example of this adoption is seen by UNICEF, who are using these tools across policy-relevant research topics facing children to create accessible evidence for decision makers [13].

There are significant benefits to be had, but also limitations of semi-automated tools. The technology is unlikely to be able to identify all relevant studies with some being missed, and the machine learning algorithm may perpetuate publication bias by learning to identify studies with positive conclusions, ignoring those with opposing results. Researchers must ensure the algorithm does not introduce evidence selection bias into the review and ensure all relevant studies are included. We need to develop experience with these tools so we can maximise their benefits and mitigate their limitations; acknowledging that there will always need to be intellectual input.

Machine learning assisted literature reviews offer numerous benefits and we believe this is the future of evidence synthesis. As researchers continue addressing complex issues such as health inequalities, these difficult problems will require modern solutions. Now is the time to get ahead of the curve as technology develops.