Publication bias in pediatric emergence delirium: a cross-sectional analysis of ClinicalTrials.gov and ClinicalTrialsRegister.eu

Abstract

Objectives

Emergence delirium (ED) is a frequent and potentially serious complication of general anaesthesia in children. Although there are various treatment strategies, no general management recommendations can be made. Selective reporting of study results may impair clinical decision-making. We, therefore, analysed whether the results of completed registered clinical studies in patients with paediatric ED are publicly available or remain unpublished.

Design

Cross-sectional analysis.

Setting

ClinicalTrials.gov and ClinicalTrialsRegister.eu.

Participants and outcome measures

We determined the proportion of published and unpublished studies registered at ClinicalTrials.gov and ClinicalTrialsRegister.eu that were marked as completed by 1st September 2018. The major trial and literature databases were used to search for publications. In addition, the study investigators were contacted directly. For published trials, time to publication was calculated as the difference in months between study completion date and publication date.

Results

Of the 44 registered studies on paediatric ED, only 24 (54%) were published by September 2019. Published trials contained data from n=2556 patients, whereas n=1644 patients were enrolled in unpublished trials. Median time to publication was 19 months. Studies completed in recent years were published faster, but still only 9 of 24 trials were published within 12 months of completion.

Conclusion

There is a distinct publication gap in clinical research in paediatric ED that may have an impact on meta-analyses and clinical practice.

Strengths and limitations of this study.

• This study quantitates the amount of research waste in paediatric emergence delirium assessed as (a) the number and (b) sample sizes of published and unpublished completed clinical studies.

• The precise reasons for non-publication of the studies included in this analysis remain unknown.

• Strengths of findings as well as directions of individual unpublished studies remain unknown.

• Study registers other than ClinicalTrials.gov and ClinicalTrialsRegister.eu were not analysed.

Introduction

Emergence delirium (ED) can be a very stressful event for both patients and caregivers during general anaesthesia in children. Although it may also develop in adults, ED is much more common in paediatric patients, with prevalences between 25% and 80% depending on the definition of ED.¹ Symptoms usually begin shortly after emergence from anaesthesia and can be very frightening including self-inflicted injury or accidental removal of catheters and other medical devices. Although episodes of ED are usually short-lived, it has been suspected that ED may be associated with long-term behavioural disturbances such as eating disorders, sleeping disorders and separation anxiety.²

The exact pathophysiology of ED is not yet understood. However, several risk factors are known: young age, use of volatile anaesthetics (especially sevoflurane), type of surgery (increased risk for otorhinolaryngeal and ophthalmological procedures), parental as well as patient anxiety and pre-existing behavioural problems.³ Whereas anxiety and behavioural problems can be addressed by non-pharmacological interventions, most of these risk factors cannot be modified and prompt the preoperative and/or perioperative administration of various medications including benzodiazepines, alpha-2 agonists, propofol, opioids and ketamine.^{4 5}

However, although it is evident that all of these drugs may have beneficial effects in specific settings to reduce the rates of ED, no universal recommendations can be derived from the existing literature for this very common and potentially serious complication. This is a typical situation in the treatment of paediatric patients, where many treatment decisions are still based on incomplete clinical data, and off-label use of various drugs is common. One important factor for the lack of clinical consensus data might be a publication bias. It is twice as likely that a positive outcome of an intervention is reported than a negative one.⁶ Such selective reporting of positive results is likely to influence clinical decision-making. We, therefore, investigated potential publication bias and time to publication in registered clinical trials on ED in children. This is a cross-sectional study.

Methods

Purpose of this study

The purpose of this analysis is to characterise publication status, patient numbers, topics of investigation, study localisation and time to publication of completed clinical trials in paediatric emergence ED, with the ultimate goal to obtain an insight into transparency and potential research waste in this important area of medicine.

Research reporting guideline

The STROBE criteria (STrengthening the Reporting of OBservational studies in Epidemiology) were applied for design, analysis and reporting of this study.⁷

Identification of clinical trials

Two databases were assessed to identify registered clinical trials on paediatric ED reported as completed by 1st September 2018: (1) the ClinicalTrials.gov database provided by the U.S. National Library of Medicine and (2) the European Union Clinical Trials Register at ClinicalTrialsRegister.eu. Search criteria were: keywords ‘emergence delirium’ and ‘emergence agitation’ with the query selection parameters ‘completed studies’ and ‘child (0–17 years)’. Database was closed on 1st September 2019. Data were downloaded for further analysis.

Search for publications of completed trials

To identify publications related to the registered and completed trials, ClinicalTrials.gov, PubMed and Google Scholar were searched for NCT number, EudraCT number, study title, principal investigator, study sponsor and keywords generated from the study title. If no respective publication was found, the principal investigators were contacted by email and/or ResearchGate and were asked to provide information about whether the study was published in a source not covered by PubMed or Google Scholar. The authors were contacted once more if they did not reply within 4 weeks.

Data analysis

The following variables were analysed: age of participants, condition, number of participants, intervention, availability of study results, completion and publication dates (time to publication) and country of sponsor. The variables ‘age of participants’ and ‘condition’ refer to the inclusion criteria of a respective clinical study. Both variables were reviewed categorially in order to ensure that only paediatric studies with patients with ED were considered in the present analysis. The variable ‘number of participants’ refers to the sample size of a given clinical study. Numbers and population sizes were calculated for both published and unpublished studies. The variable ‘intervention’ provided information about the main research tropic of a respective clinical study. Time to publication was calculated as the difference in months between study completion date and publication date in order to ascertain when results were made publicly available after completion of the study. The variable ‘country of sponsor’ provided information about the geographic localisation of the study. A detailed overview of the data is provided in the referenced online supplemental table 1. Trials were categorised into eight groups according to their main research topic. Missing data were not imputed. All statistical analyses were performed in SPSS V.20 (IBM Corporation) using standard methods for descriptive statistics. No sensitivity analyses were conducted.

Patient and public involvement

No patient involved.

Results

Publication status of studies

We identified a total of 47 studies that were reported as completed in the two trial databases. Of these, three unpublished studies were completed in less than 1 year before closing of the database. Because the U.S. Food and Drug Administration (FDA) allows a time frame of 1 year between completion and publication of the study as specified in the FDA Drug Administration Amendments Act (FDAAA),⁸ these three studies were excluded from the analysis. Of the remaining 44 studies, 24 were published and 20 were unpublished. Nine principal investigators of the unpublished studies could not be contacted by email or through the ResearchGate social network. Of the remaining 11, 2 replied and confirmed that the study results had not been published yet (figure 1). Publication rates varied considerably with the country of the sponsor (table 1) and the main topic of the investigation (table 2).

Figure 1.

Flowsheet: details of the study selection process.

Table 1.

Published (n=24) and unpublished (n=20) completed studies on paediatric emergence delirium by country

Countries	Published studies (n)	Unpublished studies (n)
Belgium	2	0
Brasil	0	1
Canada	1	0
China	0	2
Egypt	1	0
Greece	1	0
India	1	1
Italy	1	0
Kenya	0	1
South Korea	8	4
Thailand	2	1
Turkey	3	2
USA	4	8

Table 2.

Publication status of studies registered as completed on ClinicalTrials.gov and ClinicalTrialsRegister.eu involving children with emergence delirium

Topic of investigation	Overall number of studies	Number and percentage of published studies	Number of patients enrolled in unpublished studies
Dexmedetomidin	13	5 (38)	598
Diagnostic criteria	6	2 (33)	326
Non-pharmacological interventions	5	4 (80)	100
Opioids	5	2 (40)	322
Other drugs	5	4 (80)	66
Propofol	4	3 (75)	100
Volatile anaesthetics	3	1 (33)	132
Midazolam	3	3 (100)	0

The numbers of published and unpublished studies for each year of study completion (2007–2018) is shown in figure 2. An increasing number of publications over the years can be observed as well as an increasing proportion of unpublished studies, which even exceeded the number of published studies in the last 3 years.

Figure 2.

Distribution of published (n=24) and unpublished (n=20) trials by year of completion.

Patient numbers

All studies involved both genders. Published trials contained data from n=2556 patients, whereas n=1644 patients were enrolled in unpublished trials. Median size of published trials was 90 (IQR 68–136), range 40–418, whereas median size of unpublished trials was 80 (IQR 55–100), range 22–156 participants. Of note, the number of patients enrolled in unpublished studies significantly exceeded those in published studies during the last years (figure 3).

Figure 3.

Distribution of patient count stratified by publication status and year.

Time to publication

Median time to publication was 19 (IQR 12–27), range 3–104 months. More recent studies were published faster, but still only 9 of 24 trials were published within 12 months after completion as warranted by the FDAAA (figure 4).

Figure 4.

Time to publication (time between completion of the trial and publication of results) in months by year of completion. Timeline mandated by the U.S. Food and Drug Administration Amendments Act (FDAAA) of 2007.

Discussion

Almost every second completed registered clinical trial on paediatric ED remains unpublished, making results from 1644 enrolled study patients unavailable for clinical decision-making. Given the high prevalence of ED and its potentially serious manifestations, this significant publication bias is both surprising and unsatisfying.

This lack of study results may directly influence clinical practice. An illustrative example is the use of dexmedetomidine. Two published studies could show a reduction of incidence and degree of ED following premedication with intranasal dexmedetomidine.^{9 10} However, dexmedetomidine, like most potent sedatives, causes an unpleasant burning sensation when applied intranasally.¹¹ Oral application might therefore be a better choice for anxious children. One recent study showed that 1 µg/kg oral dexmedetomidine for premedication provided satisfactory sedation levels, but was not effective in preventing ED.¹² On the other hand, we identified an unpublished registered trial (NCT03357718)¹³ that used 2 µg/kg, so it is not known whether oral dexmedetomidine at higher doses might be as effective as intranasal application. For 3 of the 20 unpublished studies, preliminary results are available at ClinicalTrials.gov, and all 3 studies compared dexmedetomidine with placebo. However, their preliminary results are as conflicting as the published ones: no positive effects of intramuscular (NCT01535287)¹⁴ and intravenous (NCT01901588)¹⁵ dexmedetomidine, respectively, but reduction of ED when eightfold higher intravenous doses had been used (NCT00857727).¹⁶ Another unpublished study (NCT03171740)¹⁷ compared premedication with intranasal dexmedetomidine to oral midazolam. Intraoperative or postoperative dexmedetomidine application was investigated in five registered trials (NCT01901588,¹⁵ NCT03779282,¹⁸ NCT00857727,¹⁶ NCT01895023,¹⁹ NCT01535287)¹⁴ the results of which are not available (yet) to the public. Especially with regard to different doses and potential cardiocirculatory side effects of intravenous dexmedetomidine, the data of these 482 patients would be very interesting.

Similar considerations can be made for several study topics summarised in table 2. Minimising pain with intraoperative Fentanyl given at a mean dose of 2.5 µg/kg at the end of surgery reduced the incidence of ED in a study by Cohen et al²⁰. However, in the context of postoperative delirium in the PICU, we could recently show that fentanyl increases the risk for delirium in a dose-dependent way and that this could probably be attributed to substance-specific anticholinergic effects.²¹ Therefore, it would be very interesting to see the results of the 322 patients from the three unpublished registered trials (NCT02753725,²² NCT03010540,²³ NCT03062488)²⁴ on intraoperative fentanyl given at different doses.

Unfortunately, the low publication rate for studies on ED that we found in our analysis is in line with other published observations. Anderson et al²⁵ recently reported that only 38.3% of all completed or prematurely terminated trials registered at ClinicalTrials.gov were published and we came to similar conclusions when testing for publication bias in fields as diverse as paediatric liver transplantation²⁶ or autism.²⁷ Publication of the results gathered in clinical trials involving human subjects is considered an ethical imperative.²⁸ In 2007, it became a legal obligation in the USA to register all clinical trials in advance and publish their results within 12 months of completion.⁸ Interestingly, despite this federal law, the USA was the country of investigation found to have the highest rate of unpublished studies. Timely publication of the results is another issue that we investigated in our study. Only 9 of the 24 published studies were published within 12 months of completion, and we did not observe a trend to shorter publication intervals during recent years.

Limitations

Our study has several limitations. First, we only analysed clinical trials that were registered either at ClinicalTrials.gov or ClinicalTrialsRegister.eu. Therefore, some studies registered in smaller national registers may have been missed. Second, our analysis relies on the accuracy of data input in the respective register. Third, we can only speculate about the reasons why half of the investigators chose not to publish their results, as we did not receive respective information after contacting them directly. Last, it is likely that some of the recently completed studies will be published eventually, but still considerably later than the 12 months warranted by the FDAAA.

Conclusion

There is a distinct publication gap in clinical research in paediatric ED. Although this does not call into question the results of published studies, it should raise awareness that many aspects of the current treatment options are not exactly known. Larger numbers of published trials are immensely helpful to either support or challenge existing data which would further improve clinical practice. In addition, timely publication of study results helps to improve patient care and avoids unnecessary exposure to research, in particular, if a similar research question is being investigated repeatedly due to a lack of transparency.