Efficacy of searching in biomedical databases beyond MEDLINE in identifying randomised controlled trials on hyperbaric oxygen treatment

Abstract

Introduction

Literature searches are routinely used by researchers for conducting systematic reviews as well as by healthcare providers, and sometimes patients, to quickly guide their clinical decisions. Using more than one database is generally recommended but may not always be necessary for some fields. This study aimed to determine the added value of searching additional databases beyond MEDLINE when conducting a literature search of hyperbaric oxygen treatment (HBOT) randomised controlled trials (RCTs).

Methods

This study consisted of two phases: a scoping review of all RCTs in the field of HBOT, followed by a a statistical analysis of sensitivity, precision, ‘number needed to read’ (NNR) and ‘number unique’ included by individual biomedical databases. MEDLINE, Embase, Cochrane Central Register of Control Trials (CENTRAL), and Cumulated Index to Nursing and Allied Health Literature (CINAHL) were searched without date or language restrictions up to December 31, 2022. Screening and data extraction were conducted in duplicate by pairs of independent reviewers. RCTs were included if they involved human subjects and HBOT was offered either on its own or in combination with other treatments.

Results

Out of 5,840 different citations identified, 367 were included for analysis. CENTRAL was the most sensitive (87.2%) and had the most unique references (7.1%). MEDLINE had the highest precision (23.8%) and optimal NNR (four). Among included references, 14.2% were unique to a single database.

Conclusions

Systematic reviews of RCTs in HBOT should always utilise multiple databases, which at minimum include MEDLINE, Embase, CENTRAL and CINAHL.

Introduction

Hyperbaric oxygen treatment (HBOT) has been an active research field for decades, leading to the publication of numerous clinical studies investigating effectiveness and safety.[ 1 - 4] HBOT “is the treatment of a disease or medical condition by the inhalation of near-100% (at least 95%) medical grade oxygen at pressures greater than 1 atmosphere absolute (ATA) (101.3 kilopascals [kPa]) in a pressure vessel constructed for that purpose.”[ 1] The resulting hyperoxia leads to a number of effects such as bactericidal properties, release of growth factors, neovascularisation, and immunomodulation.[ 5]

Like all medical fields, literature searches are often employed by researchers and clinicians to inform treatment decisions. It is generally recommended to search numerous databases to ensure rigorousness and avoid missing relevant studies.[ 6 - 10] Based on time and resource constraints, however, this may not always be possible – or even necessary.[ 11] In many cases, it may be preferable to quickly identify a number of relevant studies while reducing the number of non-relevant search results that appear. Searching multiple databases to identify relevant trials among increasing numbers of publications may delay knowledge translation of evidence or prevent swift clinical decision-making. Ideally, the search of a single well-organised and indexed database including all relevant trials would improve efficiency when identifying trials to inform clinical practice and potentially close existing knowledge gaps. Therefore, this study aimed to determine whether searching beyond the Ovid MEDLINE (MEDLINE) database is necessary to identify the extent of the literature when performing a literature search of HBOT randomised controlled trials (RCTs).

Methods

The study is composed of two successive steps: (1) a scoping review of all RCTs in the field of HBOT; and (2) an analysis of the ‘performance’ (i.e., the proportion of included RCTs retrieved) of individual biomedical databases relative to all HBOT RCTs.

STEP 1: SCOPING REVIEW

To identify all available RCTs in the HBOT field, we first conducted a scoping review, and used the PRISMA-ScR reporting guidelines.[ 12] The aim of a scoping review is to “systematically identify and map the breadth of evidence available on a particular topic”.[ 13]

Eligibility criteria

Only RCTs were eligible for inclusion, and could be of any design (e.g., crossover, parallel-group, cluster, factorial). We included all studies conducted with human subjects; either patients, healthy volunteers, or healthcare providers. All contexts were included, such as clinical and simulated settings. Studies using animal populations, tissues, or cell cultures were excluded. Studies were included if they involved at least one treatment described as HBOT, offered either on its own or in combination with other treatments, for both Undersea and Hyperbaric Medical Society (UHMS) approved and non-UHMS approved indications. Diving medicine studies that did not include HBOT in a hyperbaric chamber were not included. Within each study, the comparison group was defined as a group receiving no HBOT or a different HBOT protocol than in the treatment group. Only publications in English were included for feasibility. Conference abstracts, editorials, and commentaries were excluded.

Information sources and search strategy

Based on previous systematic reviews in hyperbaric medicine, the electronic databases MEDLINE (via Ovid), Embase (via Ovid), Cochrane Central Register of Control Trials (CENTRAL), and Cumulative Index to Nursing and Allied Health Literature (CINAHL) were searched without language restrictions, from inception to December 31, 2022.[ 3 , 14 - 16] The Database of Randomized Controlled Trials in Diving and Hyperbaric Medicine was also searched.[ 17]

The MEDLINE search strategy was developed with an information specialist (AD), a practicing hyperbaric medicine physician (SB), and the research team. It was then reviewed by a second trained information specialist using the peer review of electronic search strategies guideline (PRESS).[ 18] The comprehensive MEDLINE strategy was then adapted to the unique subject headings and keywords of Embase, CENTRAL, and CINAHL (Appendix 1 ^{(104.5KB, pdf)} ). To increase the sensitivity of the search strategy, a specific search filter for RCTs was incorporated within each search protocol.

Study selection

Identified references were uploaded to DistillerSR software (Evidence Partners, Ottawa, Canada) and duplicate publications were removed. The research team developed and piloted a screening tool with 20 randomly selected articles. The tool was iteratively refined until inter-rater reliability was deemed to be adequate.

Screening by title and abstract was completed in duplicate by two pairs of independent reviewers (SI, MK, PD, SA). Studies determined to meet the inclusion criteria and those marked as ‘unclear’ proceeded to full-text review. The independent reviewers then determined compliance with inclusion criteria for the full-text articles, again in duplicate, with disagreements resolved through consensus or a third party (CE, SB). The senior author (SB), a practising hyperbaric medicine physician, reviewed the list of included articles to determine if there were any key studies meeting our inclusion criteria that, to his knowledge, were missing from the list.

STEP 2: STATISTICAL ANALYSIS OF DATABASES

Complete search results for each database were downloaded as separate Endnote (Clarivate, Philadelphia, USA) files, and each database was then searched for the title of every included study to determine if the study was indexed or not in each database. This information was recorded and extrapolated in a Microsoft Excel (version 16.65, Microsoft Corporation, Redmond Washington, USA) spreadsheet. Following the same methods as previous studies, we recorded the database of each reference, the number of records identified in each database, and the number remaining after duplicate removal (performed within but not across each database).[ 16 , 18 , 19] We descriptively summarised the number of RCTs that were unique to each database and that were unique to a combination of databases.

Analysis

For the purposes of this analysis, it was assumed that the number of RCTs identified by our search strategy was a reasonable approximation of the ‘true’ number of RCTs in existence, as is generally accepted in the systematic review community.[ 20] In addition, we are confident in our assumption, given that our literature search used wording such as ‘hyperbaric medicine’ and ‘hyperbaric oxygen’ that are both broad and specific to the area of focus. Also, the MeSH term ‘hyperbaric oxygenation’ was created a long time ago, in 1965 (Appendix 1 ^{(104.5KB, pdf)} ) .

From the search strategy of each database, we calculated the following:

• Sensitivity: the number of RCTs retrieved from each database divided by the total number of included articles indexed across databases[ 11]

• Precision: the number of included RCTs identified by a source divided by the number of both included and excluded citations identified by that source[ 9]

• ‘Number needed to read’ (NNR): effectively the inverse of precision which gives a measure of how many RCTs need to be screened to find one that is included[ 9]

• ‘Number unique’ refers to the number of included RCTs that were exclusively identified by each database[ 7]

Results

STEP 1: SCOPING REVIEW

Completion of the literature search identified 5,840 citations. Removal of duplicate articles resulted in 4,859 unique articles across the four databases utilised. After assessing the title and abstract of each reference against our inclusion criteria, 701 references proceeded to full-text screening. Of these, 334 articles were subsequently excluded: six were not in English, 217 were not RCTs, 11 studied animal populations or cell cultures, 39 were not original articles, 47 were not related to HBOT, and 14 were duplicates not initially detected automatically. Therefore, 367 RCTs were included in the analysis. All the details are shown in the PRISMA flow chart (Figure 1).

Figure 1.

PRISMA flow diagram detailing the database searches, the number of abstracts screened, and the full texts retrieved as retrieved from DistillerSR

STEP 2: STATISTICAL ANALYSIS OF DATABASES

Included records for each database

The number of included records from respective databases are presented in Table 1. We found that CENTRAL indexed the highest percentage of records (87.2% total: 7.1% unique to CENTRAL; 80.1% unique to combination of CENTRAL and additional database[s]) while Medline (78.5%) and Embase (77.9%) still indexed the majority of included studies. However, CINAHL indexed only 20.4% of included studies. Of note, these percentages include overlap among databases.

Table 1. Numbers of records uniquely identified by either a single database or a combination of databases. Each trial is counted only once. Overlapping articles are not included in the unique record count per database or database combination. As such, the number of records indicated for a combination of databases does not include records unique to a single database. CENTRAL – Cochrane Central Register of Control Trials; CINAHL – Cumulative Index to Nursing and Allied Health Literature .

Uniqueness status	Database(s)	Included records
	(n)	(%)
Unique to a single database:	MEDLINE	8	2.2
	Embase	11	3.0
	CENTRAL	26	7.1
	CINAHL	7	1.9
Unique to a combination of databases:	MEDLINE + Embase	16	4.4
	MEDLINE + CENTRAL	29	7.9
	MEDLINE + CINAHL	2	0.5
	CENTRAL + Embase	31	8.4
	CENTRAL + CINAH	4	1.1
	MEDLINE + CENTRAL + CINAHL	5	1.4
	MEDLINE + Embase + CINAHL	3	0.8
	MEDLINE + Embase + CENTRAL	171	46.6
	MEDLINE + Embase + CENTRAL + CINAHL	54	14.7
	Total	367	100.0

Meanwhile, the total overlap among multiple databases (i.e., included articles indexed by more than one database) was 85.8%, and 14.7% of included papers were indexed by all four databases. There were a total of 87 included articles indexed outside of MEDLINE, i.e., uniquely present in one of the other three databases or uniquely found across a combination of them, resulting in 23.7% of the articles (Table 2). Each database retrieved unique papers: MEDLINE (2.2%), Embase (3.0%), CENTRAL (7.1%), and CINAHL (1.9%). In total, 14.2% of all included papers were unique to a single database.

Table 2. Number of unique included records retrieved from outside of the MEDLINE database; note: each record is included only once. The number of records indicated for a combination of databases does not include records unique to a single database. CENTRAL – Cochrane Central Register of Control Trials; CINAHL – Cumulative Index to Nursing and Allied Health Literature .

Database(s)	Included records n (%)
Embase	11 (3.0)
CENTRAL	26 (7.1)
CINAHL	7 (1.9)
Embase + CENTRAL	31 (8.4)
Embase + CINAHL	8 (1.8)
CENTRAL + CINAHL	4 (1.1)
Embase + CENTRAL + CINAHL	0 (0.0)
Total articles not found in MEDLINE	87 (23.7)

Precision, and number needed to read (NNR)

MEDLINE hold the highest precision (23.8%) and lowest NNR (4), meaning only four papers were required to be screened to encounter one included paper. CINAHL held the lowest precision, at 5.9% and the highest NNR (17) (Table 3).

Table 3. Results of the various types of searches for HBOT RCTs; yellow cells indicate the best results among the searched databases for each category. CENTRAL – Cochrane Central Register of Control Trials; CINAHL – Cumulative Index to Nursing and Allied Health Literature .

Database	Total number of references retrieved before deduplication and screening (n = 5,480)	Number of included studies retrieved by database (including overlap)	Sensitivity including overlap (%)	Number unique n(%)	Precision (%)	Number needed to read (%)
MEDLINE	1,210	288	78.5	8 (2.2)	23.8	4.2
Embase	1,780	286	77.9	11 (3.0)	16.1	6.2
CENTRAL	1,585	320	87.2	26 (7.1)	20.2	5.0
CINAHL	1,265	75	20.4	7 (1.9)	5.9	16.9

Discussion

No single database indexed all RCTs in HBOT. While CENTRAL was the most sensitive database, the majority of HBOT RCTs were indexed by the CENTRAL, Embase and MEDLINE databases. Our findings showed that almost a quarter (23.7%) of the HBOT RCTs in the literature are not indexed in MEDLINE but can rather be found in other commonly used databases, namely Embase, CENTRAL, and CINHAL. However, MEDLINE remains the most efficient to search, as one included paper was encountered for every four papers identified.

KEY FINDINGS AND INTERPRETATION

These findings offer practical evidence that can be utilised by a variety of stakeholders in the field of HBOT. The results suggest that multi-source comprehensive searches are necessary to identify all included RCTs in hyperbaric medicine. This result is similar to previous studies in other fields.[ 11 , 13 - 15] Specifically, there is no singular database that contains all available RCTs in hyperbaric medicine, indicating that there is much value to searching multiple databases for the purpose of conducting high-quality systematic reviews. Therefore, researchers conducting systematic reviews of RCTs in hyperbaric medicine should not accept the risk of missing any relevant papers. Although our results indicate that CENTRAL indexes a large number of relevant articles, at minimum, researchers should conduct literature searches from all four electronic databases (MEDLINE, Embase, CENTRAL, and CINAHL) to ensure comprehensiveness.

Second, these results may hold alternate implications to clinicians, and possibly patients, who may need to quickly identify a concentrated number of RCTs in hyperbaric medicine. That is, clinicians and patients may prefer to identify the greatest amount of evidence in the shortest amount of time to inform a treatment decision, without the need to be totally exhaustive. With this goal in mind, MEDLINE proved to be the most ‘productive’ database to search. With a ‘number needed to read’ at about four, the MEDLINE database on average requires reading only four articles to come across one relevant article, whereas the number needed to read for CINAHL reached 17. Furthermore, although it did not identify the largest number of RCTs in HBOT, MEDLINE included almost 80% of all RCTs in HBOT.

Searching multiple databases can be difficult, time consuming, and costly. A search conducted in the fewest databases that retrieves a maximum yield of relevant trials and minimum yield of non-relevant trials would be ideal in order to reduce the time and costs associated with searching. Although a large proportion of HBOT RCTs were indexed in MEDLINE (78.5%), we did not assess the quality or the clinical value of the studies retrieved, and it is important to acknowledge that other potentially valuable RCTs may be indexed elsewhere. We deliberately decided to focus purely on identifying the extent of the literature and not to score the quality/value of included RCTs because scoring the value of any RCT must account for numerous parameters. This would require a separate study to be conducted. When interested in a specific area of hyperbaric medicine such as nursing protocols in a hyperbaric environment, one might be better off looking through the CINAHL database (nursing studies) instead of the MEDLINE database. Nevertheless, MEDLINE is available free online, while the other databases searched require institutional subscriptions, which may not be available to all clinicians depending on their institutions and likely are not accessible to most patients. Thus, for a cost-effective overview and readily accessible search capability, MEDLINE may still be preferable.

Given MEDLINE indexes publications from all areas of biomedicine, it may not be entirely surprising that it found the vast majority of RCTs in HBOT. Conversely, CINAHL includes publications related to nursing and health, along with other topics such as behavioural sciences, education and health administration, and logically found only 20% of papers included in this study. While Embase is a European-oriented database, it includes the field of biomedicine with primary areas of focus being toxicology and drug literature. CENTRAL combines multiple sources and focuses on high-quality evidence and is generally considered to be among the richest sources of trials.[ 10] It indeed identified the largest number of papers included in our study (87.2%), but at the expense of more ‘noise’ (i.e., less precision - more non-included papers) than MEDLINE.

STRENGTHS AND LIMITATIONS

The strength of this study is that it offers a methodological insight for conducting systematic reviews of RCTs in hyperbaric medicine. This work will help authors of future systematic reviews of RCTs to optimise their resources and may also help clinicians and possibly patients to optimise efficiency when evidence is needed within a limited timeframe.

The study has several limitations. First, we included a limited number of databases. Nevertheless, these databases have been carefully selected due to their wide use, particularly in healthcare, and large indexation coverage. Second, indexation of journals in databases is susceptible to change over time. We intended an exploratory decade-by-decade analysis for each database to account for this risk. We found that the number of studies was minimal for most databases and decades (often less than 10). Therefore, we decided not to conduct the decade-by-decade analysis as we believed that it would have been misleading in calculation of sensitivity, ‘number needed to read’ and precision for each decade and database. However, given the overall low frequency of journal indexation changes, we believe that the potential impact on our results is only marginal, at most. Third, our results are specific to the literature search algorithm we developed, and we assumed that the average clinician or patient can formulate a search in the same way as that used in this paper. Although the words used in our search were basic and intuitive (e.g., hyperbaric oxygen, randomised), we cannot know for certain what the results might be with searches conducted by other individuals. However, the terminology of hyperbaric oxygen treatment is very specific and was established decades ago (e.g., the MESH term ‘Hyperbaric Oxygenation’ was introduced in 1965). Therefore, there is limited risk of obtaining different results with minor variations in the literature search strategy. Fourth, while including only English-language publications may introduce some degree of bias, this is unlikely to affect the results of this study. Evidence suggests that using language restrictions in systematic reviews in medicine does not introduce systematic bias.[ 19] Further, trials not published in English tend to be difficult to locate and access, and published outside of the databases included here.[ 21]

Conclusions

With all aspects considered, to ensure comprehensiveness and accuracy, systematic reviews of RCTs in hyperbaric medicine should always search multiple databases, which at minimum should include MEDLINE, Embase, CENTRAL, and CINAHL.

Funding Statement

This project was supported by The Ottawa Hospital Anesthesia Alternate Funds Association and the Faculty of Medicine, University of Ottawa with a Tier 2 Clinical Research Chair. The DistillerSR licenses were funded by the Department of Anesthesiology and Pain Medicine, The Ottawa Hospital.