The paucity of high-level evidence for therapy in pediatric cardiology

ABSTRACT

Introduction:

Clinical practice should be based on the highest quality of evidence available. Therefore, we aimed to classify publications in the field of pediatric cardiology in the year 2021 based on the level of scientific evidence

Materials and Methods:

A PubMed search was performed to identify pediatric cardiology articles published in the calendar year 2021. The abstract or manuscript of each study was reviewed. Each study was categorized as high, medium, or low level of evidence based on the study design. Disease investigated, treatment studied, and country of publication were recorded. Randomized control trials (RCTs) in similar fields of neonatology and adult cardiology were identified for comparison. Descriptive statistics were performed on the level of evidence, type of disease, country of publication, and therapeutic intervention.

Results:

In 2021, 731 studies were identified. A decrease in prevalence for the level of evidence as a function of low, medium, and high was found (50.1%, 44.2%, and 5.8%, respectively). A low level of evidence studies was the majority for all types of cardiac disease identified, including acquired heart disease, arrhythmias, congenital heart disease, and heart failure, and for treatment modalities, including circulatory support, defibrillator, percutaneous intervention, medicine, and surgery. In a subgroup analysis, most high-level evidence studies were from the USA (31%), followed by China (26.2%) and India (14.3%). Comparing RCTs, 21 RCTs were identified in pediatric cardiology compared to 178 in neonatology and 413 in adult ischemic heart disease.

Conclusions:

There is a great need for the conduct of studies that offer a high level of evidence in the discipline of pediatric cardiology.

INTRODUCTION

Evidence-based medicine (EBM) is defined as a process that seeks to integrate the best research evidence with clinical expertise and patient values to optimize clinical outcomes for patients.[1,2] It is the cornerstone of clinical practice.[3] The GRADE system for therapeutic interventions rates the certainty of evidence for each outcome into categories of high, moderate, low, and very low based on criteria such as risk of bias.[4] A key factor contributing to such categorization of evidence is study design. Randomized controlled trials (RCTs) and systematic reviews of such trials are considered the highest level of evidence that provides the most valid estimates of the effects of an intervention. The use of medications, surgical interventions, and percutaneous procedures that are supported by lower levels of evidence places patients at risk of being exposed to ineffective or even harmful interventions.

In pediatric cardiology, publications are added to the literature every year; however, the publications have not been analyzed based on their level of evidence. The impression of most faculty at our institution was that high-level evidence-based studies were lacking in pediatric cardiology. Experts have also commented on the lack of RCTs in pediatric cardiology.[5,6,7] Gidding and Harris et al. asserted the need both ethically and clinically for independent rigorous pediatric clinical trials.[5,8] Therefore, we conducted this study aiming to (1) classify the literature on therapeutic interventions in pediatric cardiology according to different levels of evidence, (2) describe the characteristics of RCTs in pediatric cardiology, and (3) compare the number of RCTs in pediatric cardiology with those in two other closely related fields – neonatology and adult cardiology.

MATERIALS AND METHODS

Search strategy

A PubMed query was performed to gather articles in pediatric cardiology from January 1, 2021, to December 31, 2021. The PubMed query consisted of a search for keywords such as “Surgical Procedures, Operative” (Mesh) OR “Therapeutics” (Mesh) AND “Heart Diseases” (Mesh). Studies which included children <18 years of age were used for analysis. We excluded studies related to peripheral vascular disease and those lacking therapeutic interventions. This strategy was developed with the assistance of a librarian to ensure maximal inclusion of relevant studies.

Selection criteria

Studies were included if published in English and available in full text. Studies exclusively related to pediatric cardiology and evaluating therapeutic interventions of any kind involving patients <18 years of age were included. Table 1 shows the exclusion criteria.

Table 1.

Exclusion criteria

	Publications that:
1	Solely described cardiac arrest without describing primary etiology
2	Pertained to animal studies only
3	Pertained only to fetal heart disease
4	Reported on patients 18 years of age or older, even if diagnosed with a CHD
5	Pertained to peripheral vascular diseases outside of Kawasaki disease, truncus arteriosus, transposition of the great vessels, and patent ductus arteriosus
6	Were commentaries, editorials, letters to the editor, or other publications that did not include patients
7	Described only diagnostic tests, imaging, prognosis, or epidemiology
8	Were published outside of the queried dates

CHD: Congenital heart disease

The title and abstract of each study, published in print between January 1, 2021, and December 31, 2021, were manually reviewed by two researchers. In instances where desired information could not be determined by the abstract alone, the entire paper was read. Data extracted included the year of publication, disease subcategory, treatment modality, and type of study design. A third independent researcher provided clarity on the nature of the study design if required.

Design

The diseases addressed by the studies were classified as arrhythmias, congenital heart disease (CHD), heart failure (HF), acquired heart disease (AHD), and multiple. If the data showed multiple diseases, it was listed as “multiple.” The type of intervention described in the study was categorized as medicine, surgery, percutaneous intervention, defibrillator, circulatory, other, multiple, and unknown [Table 2]. Congenital cardiomyopathy, for example, Barth syndrome, was also classified as CHD. Therapies not included in the primary category design, such as the use of stem cells or music, were listed as “other.” The treatment category for the deactivation of ventricular assist devices was classified as “circulatory.”

Table 2.

Categorization of disease, treatment intervention, and types of research studies

Disease	Treatment	Type of study
CHD	Circulatory support	Case–control
AHD	Defibrillator	Case report
Arrhythmia	Medicine	Case series
HF	Multiple	Cohort
Multiple	Others	Cross sectional
	Percutaneous intervention	Literature review
	Surgery	Meta-analysis
		Non-RCT
		Nonsystematic review
		RCT
		Survey
		Systematic review

CHD: Congenital heart disease, RCT: Randomized control trial, HF: Heart failure, AHD: Acquired heart disease

Each study was first classified based on the study design into one of the following types: case report, case series, cohort, case–control, RCT, systematic review, and nonsystematic review. Each study was then placed into one of three categories of levels of evidence modified from the Centre for EBM: high (Levels 1 and 2A), medium (Levels 2B, 2C, and 3), and low (Levels 4 and 5)[3] [Table 3]. After categorizing the studies as described above, the characteristics of the studies identified were evaluated. Finally, to compare the number of RCTs in pediatric cardiology with those in other fields, separate PubMed searches to identify published RCTs in neonatology and adult cardiology were conducted.

Table 3.

Levels of evidence for therapeutic studies adapted from the center for evidence-based medicine[3]

Adapted level of evidence	Level	Type of evidence
High	1	Systematic review of RCTs, individual RCTs, all or none studies, meta-analysis
	2A	Systematic review (with homogeneity) of cohort and case–control studies
Medium	2B	Individual cohort study, non-RCTs
	2C	“Outcomes” research; ecological studies
	3	Individual case–control study
Low	4	Case series, survey
	5	Expert opinion without explicit critical appraisal or based on physiology bench research or “first principles”

RCTs: Randomized control trials

Statistical analysis

Studies were exported from PubMed into the EndNote citation manager. From EndNote, author names, the PubMed identification number, abstract, keywords, notes, and article title were exported into a Microsoft Excel spreadsheet. These variables were entered manually into a second Excel spreadsheet. Analysis was performed on the data collected.

RESULTS

A PubMed search yielded a total of 2112 studies. The general query was further filtered to exclude exclusively e-published work resulting in 1788 total articles. Each of the 1788 studies was carefully scrutinized with the application of inclusion and exclusion criteria. A total of 731 studies were found to meet all specifications.

Types of research studies

Among primary studies, those providing low, medium, and high levels of evidence comprised 50.1%, 44.2%, and 5.75%, respectively [Table 4].

Table 4.

Breakdown of types of research studies published in pediatric cardiology in 2021

Type of study	n (%)
Case–control	29 (4.0)
Case report	206 (28.2)
Case series	156 (21.3)
Cohort	218 (29.8)
Cross sectional	4 (0.5)
Meta-analysis	2 (0.3)
Non-RCT	3 (0.4)
Nonsystematic review	69 (9.4)
RCT	21 (2.9)
Survey	4 (0.5)
Systematic review	19 (2.6)

RCT: Randomized control trial

Countries

The countries of publication included the USA (35.16%), China (11.35%), Japan (6.98%), the UK (4.79%), Turkey (4.10%), Italy (3.56%), India (3.15%), other countries (32.95%), and multi-country (0.96%).

Types of heart diseases

The diseases primarily addressed by the studies were CHD (68.4%), AHD (10.7%), HF (9.8%), arrhythmias (6.4%), and multiple diseases (4.7%) [Table 5].

Table 5.

Breakdown of types of diseases studied in pediatric cardiology in 2021

Disease	n (%)
AHD	78 (10.7)
Arrhythmia	47 (6.4)
CHD	500 (68.4)
HF	72 (9.8)
Multiple	34 (4.7)

CHD: Congenital heart disease, HF: Heart failure, AHD: Acquired heart disease

Types of treatment

Therapeutic interventions addressed by the evidence were surgery (46.4%), percutaneous intervention (16.3%), medications (11.4%), circulatory support (8.9%), other forms of therapy (5.7%), and multiple forms of therapy (10.4%) [Table 6].

Table 6.

Breakdown of types of treatment interventions studied in pediatric cardiology in 2021

Treatment	n (%)
Circulatory support	65 (8.9)
Defibrillator	7 (1.0)
Medicine	83 (11.4)
Multiple	76 (10.4)
Others	42 (5.7)
Percutaneous intervention	119 (16.3)
Surgery	339 (46.4)

Levels of evidence

Low level

Among primary studies on pediatric cardiology, 50.1% were classified as having low levels of evidence (LLE). Within each disease category, LLE consisted of 70.5% of all research on AHD, arrhythmias (66.0%), HF (50.0%), and CHD (45.6%) [Table 7]. With respect to treatment category, LLE publications were maximum in the field related to defibrillators (85.7%), followed by circulatory support (56.9%), percutaneous intervention (63.0%), medications (50.6%), and surgery (47.2%) [Table 8]. With respect to country of publication for LLE studies, most were from the USA (29.8%), followed by China (8.7%) and Turkey (4.9%).

Table 7.

Level of evidence found in each disease category

Disease	Level of evidence
	Low, n (%)	Medium, n (%)	High, n (%)
AHD	55 (70.5)	20 (25.6)	3 (3.8)
Arrhythmias	31 (66.0)	15 (31.9)	1 (2.1)
CHD	228 (45.6)	238 (47.6)	34 (6.8)
HF	36 (50.0)	32 (44.4)	4 (5.6)
Multiple	16 (47.1)	18 (52.9)	0.0

CHD: Congenital heart disease, HF: Heart failure, AHD: Acquired heart disease

Table 8.

Level of evidence found in each treatment intervention

Treatment	Level of evidence
	Low, n (%)	Medium, n (%)	High, n (%)
Circulatory support	37 (56.9)	25 (38.5)	3 (4.6)
Defibrillator	6 (85.7)	1 (14.3)	0.0
Medicine	42 (50.6)	29 (34.9)	12 (14.5)
Multiple	33 (40.7)	38 (46.9)	10 (12.3)
Others	13 (31.0)	20 (47.6)	9 (21.4)
Percutaneous intervention	75 (63.0)	39 (32.8)	5 (4.2)
Surgery	160 (47.2)	171 (50.4)	8 (2.4)

When investigating only LLE, a subgroup analysis was performed to show the most common disease categories within LLE. Our analysis showed that the most common disease categories addressed by LLE were CHD (62.3%), AHD (15.0%), HF (9.8%), and arrhythmias (8.5%), while the most common therapeutic categories addressed by LLE were surgery (43.7%), percutaneous intervention (20.5%), medications (11.5%), circulatory (10.1%), and others (3.5%).

Medium level

Among primary studies on pediatric cardiology, 44.2% were classified as having medium levels of evidence (MLE). Within each disease category, MLE consisted of 47.6% of all research on CHD, HF (44.4%), arrhythmias (31.9%), and AHD (25.6%) [Table 7]. MLE comprised surgery (50.4%), percutaneous intervention (32.8%), medications (34.9%), circulatory support (38.5%), defibrillation (14.3%), multiple treatments (46.9%), and other treatments (47.6%) [Table 8]. Of the medium-level studies, 41.8% were published from the USA, 12.4% from China, and 4.3% from Japan.

When investigating only MLE, a subgroup analysis was performed to show the most common disease categories within MLE. Our analysis showed that the most common disease categories addressed by MLE were CHD (73.7%), HF (9.9%), AHD (6.2%), and arrhythmias (4.6%). The most common therapeutic categories addressed by the medium-level studies were surgery (52.9%), percutaneous intervention (12.1%), medications (9.0%), circulatory support (7.7%), multiple (11.8%), and others (6.2%).

High level

Among primary studies on pediatric cardiology, 5.75% were classified as having high levels of evidence (HLE). Within each disease category, high-level evidence consisted of 6.8% of all research on CHD, HF (5.6%), AHD (3.8%), and arrhythmias (2.1%) [Table 7]. High-level evidence comprised 14.5% of studies on medications, circulatory support (4.6%), percutaneous intervention (4.2%), and surgery (2.4%) [Table 8]. Countries contributing to studies of HLE were the USA (31%), China (26.2%), and India (14.3%).

When investigating only HLE, a subgroup analysis was performed to show the most common disease categories within HLE. Our analysis showed that the most common disease categories addressed by HLE were CHD (81%), HF (9.5%), AHD (7.1%), and arrhythmias (2.4%). The most common therapeutic categories addressed by these RCTs were medications (28.6%), surgery (19%), percutaneous intervention (11.9%), circulatory support (7.1%), and multiple (11.9%), and other forms (21.4%). From January 1, 2021, to December 31, 2021, there were 21 RCTs in pediatric cardiology, 178 in neonatology (birth – 1 month of age), and 413 in adult ischemic heart disease.

In addition, to compare high-level evidence in pediatric cardiology to other pediatric disciplines during the same timeframe (calendar year 2021), we analyzed high-level evidence in pediatric surgery (5.8%), neonatology (9.5%), and pediatric urology (6.5%). We also looked at high-level evidence studies in adult specialties: adult surgery (7%), adult urology (6.3%), and adult cardiology (7.9%).

DISCUSSION

In the hierarchy of study designs, RCTs are considered the most valid and desirable form of evidence to support therapeutic interventions because they most effectively limit both known and unknown confounding and bias.[3,9,10,11] In this study, we reviewed all publications about therapeutic interventions in pediatric cardiology over 1 year and classified them into high, MLE, and LLE. Our study showed that only 5.75% of the pediatric cardiology literature consisted of RCTs or high-level evidence. The low number (21) of RCTs in pediatric cardiology over 1 year contrasts starkly with the high number of RCTs during the same period in the field of neonatology (178) and in adult ischemic heart disease (413). These findings suggest that most published research in the field of pediatric cardiology has only low- or medium-level evidence. Researchers and research funding agencies should focus on designing studies that will have a high level of evidence. This supports results found by Drury et al., who highlighted that recent literature in pediatric cardiac surgery contains few late-phase clinical trials and that the published trials are small, single-center studies of low value and uncertain quality.[7]

Our results parallel the percentage of studies constituting high-level evidence in other specialties. Despite a larger number of RCTs in neonatology (178), they constituted only 9.5% of all research in that specialty as compared to 5.6% in pediatric cardiology. In addition, the percentages of high-level evidence in pediatric surgery and pediatric urology seemed to mirror those seen in adult subspecialties. Interestingly, despite the growing emphasis on EBM, the prevalence of RCTs being performed continues to be low, regardless of the population studied. A comprehensive study by Gnanalingham et al. of 30 years of scientific literature in 25 clinical journals found the proportion of RCTs in multiple medical fields to be as follows: anesthesia (18%), psychiatry (9.6%), medicine (8.1), pediatrics (6.4%), and surgery (5.3%).[12]

In our evaluation of the geographical origins of studies, regardless of the level of evidence, the countries of publication were the USA (35.16%), China (11.35%), Japan (6.98%), the UK (4.79%), Turkey (4.10%), Italy (3.56%), and India (3.15%). Our study found that high-level evidence in pediatric cardiology originates mostly from the US, China, and India. The bibliometric analysis demonstrates that the predominance of high-level evidence, regardless of field of study, is generated by the US, the UK, and Canada.[13] Catalá-López et al. have stated that this might be due to a heightened focus on scientific collaboration and biomedical investigation, increased fiscal resources, and increased scientific productivity as a result of more available intellectual capital.[13] Lower-income countries face additional challenges in research related to poor data organization, accessibility, and funding.[13]

The challenges of conducting RCTs are substantial, including difficulties in recruitment, the need to adequately follow-up patients for significant durations, safety monitoring, and costs. Other authors have discussed the specific challenges of performing RCTs in surgical fields.[14,15] These include relative infrequency of a disease state, lack of community equipoise about the standards of care, methodologic issues with effective blinding and randomization, patient biases against invasive treatment arms, and lack of funding.[14,16,17] Pediatric cardiology shares these as well as other challenges. Gidding ascertained that these included the relative rarity of individual diseases, the heterogeneity of presentation, rapid changes in technology making older diagnostic and therapeutic techniques obsolete, the importance of individual physician skill to outcome, the preponderance of interventions that cannot be masked like medications can be masked with a placebo, and variations in surgical skills and techniques that make it difficult to standardize interventions.[8] CHD, for example, as described by Abdulla, is a rare entity that is often accompanied by a wide spectrum of medical problems, thus reducing the generalizability of the patients.[18]

Nevertheless, pediatric cardiology researchers should not consider these problems as insurmountable because other disciplines with similar challenges have managed to overcome them through organized efforts and multicenter collaboration. For example, Moss et al. successfully completed a multicenter randomized trial comparing two types of surgical interventions in preterm infants with perforated necrotizing enterocolitis.[19] Recommendations for randomized trial design that apply to surgical interventions have been published.[20] Moss et al. emphasized the tradition in pediatric surgery of accepting case series data when making decisions and the lack of equipoise among surgeons as a self-imposed barrier to the conduct of rigorous studies.[16] Similar self-imposed barriers likely exist in pediatric cardiology as well. The lack of rigorous studies in pediatric cardiology resulting in continued interventions on patients based on LLE should be considered unacceptable. This concern was addressed by the National Heart, Lung, and Blood Institute through the establishment of the Pediatric Heart Disease Clinical Research Network in 2001 and its continued evolution.[21,22] The collaborative efforts of this group have yielded several RCTs and observational studies thus far and have incorporated investigator-initiated funding for some clinical trials.[21] We hope that further collaborative studies yield higher standards of evidence and ultimately improve patient care outcomes in pediatric cardiology.

Limitations

Our search process was robust, comprising multiple journals published in many countries, made possible through the use of electronic databases. This systematic assessment yielded a large number of studies published over 1 year, and each study was subsequently rigorously evaluated and classified by study design. We attempted to minimize interobserver reliability by utilizing a third reviewer in cases where levels of evidence were not immediately clear. Therefore, our results can be extrapolated to pediatric cardiology as a whole. Limitations included analysis of publications within a 1-year time frame, thus precluding analysis of larger trends. Studies were also confined to those having therapeutic implications. Our analysis was restricted to study design and did not focus on the quality of individual studies. Our search strategy based on keyword assignment in PubMed might have failed to show several relevant studies from this time frame.

CONCLUSIONS

The development of EBM in pediatric cardiology is still in its early stages. There is a paucity of high-level evidence to support therapeutic interventions in pediatric cardiology, particularly for circulatory support and cardiac surgery. We hope to draw attention to the great need for clinical research in pediatric cardiology and pediatrics as a whole that will produce high-level evidence for common conditions and therapies. The creation of collaborative research networks can foster such research.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.