Randomized controlled trials of antibiotics for neonatal infections: a systematic review

Abstract

Aims

Antibiotics are a key resource for the management of infectious diseases in neonatology and their evaluation is particularly challenging. We reviewed medical literature to assess the characteristics and quality of randomized controlled trials on antibiotics in neonatal infections.

Methods

We performed a systematic search of PubMed, Embase and the Cochrane Library from January 1995 to March 2010. Bibliographies of relevant articles were also hand-searched. We included all randomized controlled trials that involved neonates and evaluated the use of an antibiotic agent in the context of a neonatal infectious disease. Methodological quality was evaluated using the Jadad scale and the Cochrane Risk of Bias Tool. Two reviewers independently assessed studies for inclusion and evaluated methodological quality.

Results

A total of 35 randomized controlled trials were evaluated. The majority were conducted in a single hospital institution, without funding. Median sample size was 63 (34–103) participants. The most frequently evaluated antibiotic was gentamicin. Respectively, 18 (51%) and 17 (49%) trials evaluated the therapeutic or prophylactic use of antibiotics in various neonatal infections. Overall, the methodological quality was poor and did not improve over the years. Risk of bias was high in 66% of the trials.

Conclusions

Design and reporting of randomized controlled trials of antibacterial agents in neonates should be improved. Nevertheless, the necessity of implementing such trials when antibacterial efficacy has already been established in other age groups may be questioned and different methods of evaluation should be further developed.

Introduction

Randomized controlled trials (RCTs) are accepted as the design of choice to demonstrate the efficacy of a pharmaceutical product and they are mandatory to support a marketing authorization. Disease-affected neonates are given a great number of drugs, but a large proportion of these products is still prescribed without or outside the terms of their product license (off label) 1, 2. Neonatal clinical research presents multiple challenges, which are often difficult to address 3 and despite significant efforts to promote high quality drug research in paediatrics, neonates are still less likely to be involved in a RCT 4–6.

Management of infectious diseases is an everyday challenge for neonatologists. Bacterial infections account for a major part of the neonatal morbidity and mortality worldwide and antibiotic therapy is a key resource for the management of sick newborns 7, 8. This therapeutic class exhibits its pharmacological effects exclusively on pathogenic bacteria and consequently any direct effects on human cells and tissues are, by definition, adverse 9. This has major implications for the assessment of the efficacy of antibiotics, mainly dependent on the susceptibility of the causal pathogen, and it also underlines the importance of their safety evaluation that may vary between age groups.

There is a substantial lack of information on antibiotic use in neonates. We are currently evaluating ciprofloxacin as part of the TINN (Treat Infection iN Neonates) project supported by the 7^th Framework Programme of the European Commission 10 and we decided to consider previously conducted trials in order to inform future research. Although several reviews have been published by the Cochrane Collaboration on the treatment of specific infections, we were more interested in providing a global view of RCTs of antibiotics and in discussing specific difficulties surrounding neonatal research that impact on the quality of the trials. Therefore, we conducted a systematic review of the extant medical literature to assess the characteristics and quality of RCTs evaluating the use of antibiotics in neonatal infections.

Methods

Search strategy

Relevant RCTs were identified through electronic literature searches using the following databases: MEDLINE, EMBASE and the COCHRANE Library (CENTRAL and Cochrane Database of Systematic Reviews), all from January 1995 to March 2010. We used the following abbreviated search strategy: ‘Neonates’ AND ‘Antibiotics’ AND ‘Randomized controlled trial’ (full search strategy given in Appendix S1). Bibliographies of relevant articles were also hand-searched for the same purpose.

Study selection

Studies were eligible if (i) they were RCTs defined as any prospective study where participants were randomly allocated to study groups, (ii) they recruited neonates, an age group defined from birth up to and including 27 days of life in term newborns, or from birth to a post-menstrual age of 40 weeks and 27 days in preterm newborns 11 and (iii) their main objective was to evaluate the effects of an antibiotic agent in the context of a neonatal infection. All RCTs where the unit of randomization was not the neonate were not retained. Also, abstracts, letters, duplicates, preliminary publications and reviews were excluded.

Validity assessment

All retrieved titles and abstracts were initially screened by one reviewer (FK) and full texts of potentially relevant articles were obtained. A second reviewer (EJA) who was blinded to the initial assessment, independently screened a sample of abstracts that comprised all abstracts for which the first reviewer obtained full text, plus a random selection of abstracts rejected at the initial screening. Both reviewers further examined retrieved full texts with regard to eligibility criteria.

Data extraction

To extract data we developed a standardized data collection form that was pre-tested in 10 randomly selected included articles and then modified accordingly. Data extracted from each trial referred to the study settings, journal and year of publication, funding source, design, clinical context, number of participants, antibiotic under test, control intervention and concomitant medications. Use of antibiotics was defined as ‘therapeutic’ or ‘prophylactic’ towards a specific neonatal infection. Prophylactic use concerned both primary and secondary prophylaxis.

Quality assessment

The methodological quality of each trial was evaluated using the Jadad scale and the Cochrane Risk of Bias (ROB) Tool. The Jadad scale is a validated three-item quality assessment tool of randomization, blinding and description of patient withdrawals and dropouts 12. The maximum score assigned to a trial is 5 although a score of at least 3 is considered to denote good methodological quality. The ROB tool was implemented based on the guidelines of the Cochrane Collaboration 13. It addresses six methodological domains: sequence generation, allocation concealment, blinding, incomplete outcome data, selective outcome reporting and other sources of bias. For each study, ROB is described as low (all six domains are judged to be at low ROB) or high (one or more domains are judged to be at high ROB) or unclear (one or more domains are judged to be at unclear ROB and none at high risk). Reporting of the primary outcomes' definitions, power calculation and study conclusions, were also assessed 14. Quality evaluation was performed independently by two reviewers (FK, EJA) who arrived at consensus for all items.

Data analysis

Descriptive data included absolute numbers (percentages) for categorical variables and medians [first quartile (Q1)–third quartile (Q3)] for continuous variables. The Kruskall-Wallis rank sum test was used to compare the Jadad score in different groups. Analysis was performed with SAS version 9.2 software (SAS Inc, Cary, North Carolina, USA).

Results

Flow of included studies

Electronic search yielded a total of 2198 reports. Selection of articles is presented in Figure 1. Overall, 47 RCTs were identified, 31 involved only neonates, four involved neonates and young infants and 12 involved neonates, older children and even adults. These 12 RCTs focused on clinical conditions that were more prevalent in children and adults and only two mentioned the exact number of neonates included (proportion of neonates 0.5% and 9%). Consequently, these RCTs were excluded, resulting in a total of 35 articles retained for analysis (references are given in Appendix S2).

Figure 1.

Study screening process

Agreement between reviewers

The second reviewer was provided with a sample of 150 abstracts, 80 initially selected for full-text reviewing and 70 initially excluded. The second reviewer agreed with all initially excluded abstracts and among those initially selected, she agreed with 72 and disagreed with eight. After discussion, these eight abstracts were considered irrelevant. Following full text review, there was complete agreement between the two reviewers about the 35 articles retained.

Trial characteristics

Table 1 reports the main characteristics and Figure 2 displays the number of trials by year of publication. Most of the studies were conducted nationwide, were exclusively hospital-based and authors were mainly from Europe and North America. Funding was specified in only seven articles (20%). Median sample size was 63 (34–103) and age at inclusion ranged from birth to 90 days. All trials were two parallel arm RCTs. Antibiotics under test and the clinical context of their evaluation are presented in Table 2. Gentamicin was the most frequently studied antibiotic. In 26 RCTs (74%), tested antibiotics were administered intravenously whereas in the remaining 26% administration was intravenous or oral 3%, oral 3%, by topical application 6% or without information on the route of administration 14%. Duration of treatment ranged between 1 and 42 days (missing data n = 12). In most trials, the control group had received the same antibiotic as the test group but with a different administration schema. Description of concomitant medications was provided in approximately half of the trials. Most articles were published in specialty journals.

Table 1.

Characteristics and methodology for 35 randomized controlled trials evaluating the effects of antibiotics in neonatal infectious diseases

	ALL RCTs (n = 35)
	Number (%)
Continent of corresponding author
Africa	3 (9%)
Asia	10 (28.5%)
Europe	10 (28.5%)
North America	12 (34%)
Journal type
Specialty	29 (83%)
General Medical*	6 (17%)
Number of centres
Single centre	30 (85%)
Multicentre national	1 (3%)
Multicentre international	2 (6%)
Not stated	2 (6%)
Funding
Industry	2 (6%)
Institution	5 (14%)
Unknown	28 (80%)
Sample size [median (Q1–Q3)]	63 (34–103)
Minimal age at inclusion†, days (range)	0–7
Maximal age at inclusion‡, days (range)	0–90
Treatment in the control group
Same antibiotic as the test group	18 (52%)
Usual care (no additional antibiotic drug)	10 (28%)
Active reference antibiotic treatment	5 (14%)
Placebo	2 (6%)
Concomitant therapy
Yes, detailed	18 (52%)
Not detailed or not stated	17 (48%)
Jadad score [median (Q1–Q3)]	2 (1–2)
= 0	1 (3%)
= 1	12 (34%)
= 2	16 (46%)
= 3	4 (11%)
= 4	2 (6%)
Risk of bias
Low	3 (9%)
Unclear	9 (25%)
High	23 (66%)
Primary outcome
Defined§	23 (66%)
Not defined	12 (34%)
Power calculation
Stated	14 (40%)
Not stated	21 (60%)
Study primary statistical analysis
Statistically significant	15 (43%)
Not statistically significant	17 (48%)
Not performed/No details on analysis	3 (9%)
Authors' conclusions
Positive	22 (63%)
Negative	13 (37%)

^*Journals that publish articles from any specialty.

^†seven missing data.

^‡seven missing data.

^§Defined explicitly in the article or used in power calculation.

Figure 2.

Trends in publication of RCTs

Table 2.

Tested antibiotics and context of evaluation in the 35 relevant randomized controlled trials in neonates

Antibiotics	Infection/disease	Number of RCTs (%)
		Therapeutic use	Prophylactic use
Gentamicin		12 (34%)
	Suspected or proven bacterial sepsis or focal infection	8*
	Meconium aspiration syndrome		1
	Type of infection not stated	3*
Ampicillin + aminoglycosides		5 (14%)
	Neonatal pneumonia (+ gentamicin)	2
	Meconium aspiration syndrome (+ gentamicin/amikacin)		2
	Bacterial infection in high risk infants (+ netilmicin)		1
Vancomycin		4 (11%)
	Nosocomial coagulase-negative staphylococci infections		2
	Necrotizing enterocolitis		1
	Catheter-related bloodstream infections (+ heparin lock)		1
Erythromycin		4 (11%)
	Chronic lung disease with/without Ureaplasma urealyticum colonization		3
	Infectious conjunctivitis		1
Fucidic acid		2 (6%)
	Catheter-related blood stream infections (+ heparin lock)		1
	Infectious conjunctivitis	1
Amoxicillin		1 (3%)
	Catheter-related blood stream infections		1
Benzathine penicillin		1 (3%)
	Congenital syphilis		1
Linezolid		1 (3%)
	Sepsis due to resistant Gram + ve bacteria	1
Teicoplanin		1 (3%)
	Coagulase-negative staphylococci infections		1
Amikacin		1 (3%)
	Suspected or proven bacterial infection	1*
Azithromycin		1 (3%)
	Chronic lung disease		1
Ceftazidim		1 (3%)*
	Suspected or proven sepsis	1*
Various antibiotics		1 (3%)
	Suspected or proven sepsis	1

^*Antibiotic efficacy was evaluated upon pharmacokinetic parameters.

Seventeen (49%) RCTs estimated the effects of several antibiotics for the prevention of infections or infectious complications (Table 2). In more than half of these trials (n = 10), the control group received usual care without additional antibiotic treatment. The tested antibiotic or combination of antibiotics was compared with a placebo or the same antibiotic administered for different durations in two trials each and a reference antibiotic agent in three trials.

Eighteen (51%) RCTs assessed the efficacy of antibiotics for the treatment of proven or suspected neonatal infections (Table 2). Efficacy was evaluated upon clinical parameters (n = 5), pharmacokinetic parameters (mainly serum trough and peak concentration thresholds, n = 9) or both pharmacokinetic and clinical parameters (n = 4). Among trials that focused only on clinical efficacy, two compared an antibiotic (e.g. linezolid, fusidic acid) against an active reference antibiotic treatment and three aimed to compare two different treatment durations of the same antibiotic combination (n = 2) or of various antibiotics prescribed according to sensitivity reports, not predefined by the study protocol (n = 1). Trials that focused on the pharmacokinetic parameters to determine efficacy compared different administrations of gentamicin (once vs. twice daily or multiple daily doses or every 48 h: n = 10, standard dose vs. loading dose: n = 1), ceftazidime and amikacin (once or twice daily dose: n = 1 for each molecule) (Table 2). Concomitant therapy by beta-lactam antibiotics was described in nine out of these 13 trials (69%).

Quality assessment

Overall, median Jadad score was 2 and only two trials presented a score of 4 (Table 1). The quality of trials slightly improved over the years (median Jadad score 1995–1999 = 2, 2000–2004 = 2, 2005–2009 = 2.5, P value 0.32). No difference in the Jadad score was observed among RCTs evaluating prophylactic antibiotics and those evaluating therapeutic antibiotics (median Jadad score 2 for each of the two categories, P value 0.47).

The ROB was high for 23 (66%) trials, low for three (9%) and unclear for nine (25%) (Table 1). High ROB was mostly related to absence of blinding or selective reporting of outcomes as most primary outcomes were either not pre-specified or incompletely reported (Table 3). Also, additional sources of bias included early termination of the trials (n = 6), selective reporting of subgroups (n = 2), imbalance in baseline characteristics (n = 2) and differential diagnostic activity between study groups (n = 1).

Table 3.

Risk of bias assessment for the 35 relevant randomized controlled trials in neonates

Domains	Risk of bias assessments – n (%)
	High	Unclear	Low
Sequence generation	1 (3%)	21 (60%)	13 (37%)
Allocation concealment	1 (3%)	19 (54%)	15 (43%)
Blinding	7 (20%)	9 (26%)	19 (54%)
Incomplete data	3 (9%)	6 (17%)	26 (74%)
Selective reporting	13 (37%)	9 (26%)	13 (37%)
Other sources of bias	11 (31%)	17 (49%)	7 (20%)

Uncertainty about ROB was mainly related to incomplete reporting of sequence generation or allocation concealment in particular. However, most studies did not present missing data for outcomes or at least reasons for missing data were not related to outcome (Table 3). Also, blinding was rated as low ROB because although most articles were unblinded or did not address this domain explicitly (n = 24), assessment of primary outcomes was not influenced by the absence of blinding. Trials with a low ROB presented a higher median Jadad score than those with an unclear or high ROB but differences were not statistically significant (median Jadad score: low ROB = 3; unclear or high ROB = 2 for each of the two categories; P value: 0.10).

One in three trials did not explicitly specify the nature of the primary outcome(s) or use any power calculation (Table 1). Although primary outcomes were mostly of a clinical nature such as the cure or relapse of an infection, their definitions were highly variable between studies. Also, multiple primary outcomes were reported in 24 (69%) of RCTs.

In most RCTs, statistical calculation to ensure adequate power was not described (n = 21, 60%) and statistical hypotheses on which to base analysis were not formulated (n = 23, 66%). Only one trial was described as a non-inferiority study. In addition, authors' conclusions were not always in keeping with statistical analysis as the majority of them claimed positive research results (n = 22, 63%) although differences between groups were statistically significant in only 15 (43%) trials (Table 1).

Discussion

This study assessed the characteristics and methodological quality of all published RCTs evaluating antibiotics in neonatal infections within the last 15 years. While antibiotics are extensively prescribed in neonatology 15, 16, our literature review found few and mainly low quality RCTs. Also, there was no increase in the number and quality of RCTs over the years, although these indicators have globally increased in paediatric research 17. Hence, the extant literature is not suitable to inform clinical practice about antibiotics in neonatology.

The RCT endeavouring to show superiority is a common approach to evaluate clinical efficacy in research, by comparing two treatment arms, the first receiving the compound under evaluation and the second a placebo or a reference compound. The results are simple to interpret but obviously, non-significant results do not mean equivalence between the two treatment arms. Conversely, a non-inferiority trial aims to demonstrate that the experimental treatment is not inferior to the reference treatment. Such RCTs usually require a higher number of patients and a validated comparator and, therefore, are rarely implemented in paediatrics 18. Usually, RCTs of antibiotics for licensing purposes in adults are non-inferiority trials unless the infection is of such a nature that a placebo is relevant, which is clearly not the case in infected neonates 19. In addition, pharmacokinetic studies or pharmacokinetic–pharmacodynamic studies are usually descriptive, although randomization can be used to compare different doses or dosage schedules.

Overall poor methodological quality can question the relevance of trials' results but also the ethical basis of conducting such trials. Drug research evidence in paediatrics has already been reported as less robust than that in adults but this clearly tends to be more obvious in neonates and infants 20, 21. The median Jadad score of RCTs in our review was lower than the mean Jadad score of 3.22 of RCTs published in 2007 that involved children 6. This is of significant concern as neonates are particularly vulnerable compared with older paediatric patients and adults.

Many trials in our review failed to implement or provide details on key quality features of trial design. Assessment of selection criteria and primary outcomes is essential for trial validity and comparison between trials but the majority of trials did not specify them or used customized assessments. For obvious ethical reasons, use of a placebo arm is often not possible in neonatal infections. However, it is disappointing that despite widespread dissemination of quality and reporting requirements for RCTs 22, 23 more efforts were not pursued to blind care providers, explicitly report outcomes and methods of random sequence generation and allocation concealment. Such findings concerning methodological errors or omissions in reporting are in accordance with previous Cochrane reviews on the evaluation of antibiotics for specific neonatal infections. All but one review stated that current evidence from randomized controlled studies was either insufficient or inadequate to draw any reliable conclusions.

Also, statistical analysis and interpretation of data were often inconsistent and a large proportion of trials were probably underpowered to demonstrate treatment superiority. Frequently, the authors concluded the equivalence between two therapies when no statistical difference was found.

The paucity of RCTs and their poor quality highlight the difficulties in conducting studies assessing the effects of antibiotics in neonates, despite regulatory initiatives that encourage drug studies in neonatology 24. Several factors may explain this situation:

1. Although suspected infections are quite frequent in neonatal care, microbiologically evaluated infection is rare: 1–2 cases per 1000 live births for both early and late onset infections.

2. Antibiotic treatment has to be started as early as possible and merely upon clinical suspicion 25. This makes the design and analysis of trials challenging because participants are often randomized before the microbiologically valuable population is defined.

3. Antibiotics with proven efficacy and safety in older paediatric patients or even adults have entered the neonatal care arena because clinicians perceive them to have a more useful spectrum of activity than the currently used ‘older’ antibiotics. Although widely used, few if any antibiotics have been systematically evaluated so that it is difficult to choose a validated comparator.

4. It is also extremely difficult to standardize antibiotic treatment within multicentre trials, as the choice of the empirical treatment, depending on local epidemiology and clinical context, varies between units.

5. As antibiotics are hard to allocate on a random basis, the absence of ‘clinical equipoise’ is a major shortcoming for the ethical validity of a trial. In addition, enrolment in clinical trials may be challenging in this age group particularly in emergency situations 26, 27.

Finally, funding source was rarely reported. This, combined with the frequent single centre character, may reflect both poor infrastructure and limited financial support for trials in neonates. It may also reflect the lack of interest of pharmaceutical companies to sponsor trials that may be complex and expensive to conduct, with few financial incentives. This is unfortunate given the emphasis on this age-group from regulators 24 that encourage drug studies in neonatology through national or international collaborations 28.

We based our quality evaluation exclusively on peer review publications, as dissemination of scientific knowledge relies mainly on reporting of trials. Our review stresses the fact that investigators should further improve the quality of design and reporting of RCTs of antibiotics in neonates. Nevertheless, there can be situations in neonatology where implementation of such trials may not be feasible, such as infections due to rarely encountered pathogens. In these situations, we feel that the input of other study designs, that are potentially more adapted to the evaluation of this therapeutic class and the context of neonatal infections, should not be discarded. In fact, regulators acknowledge that RCTs of antibacterial agents in neonates may not be required when their efficacy has already been established in adults or older children 19, 29 as the target of antibiotics is bacterial and the nature of a pathogen is unlikely to vary with patients' age. However, determining the appropriate dosage regimen and safety of use in neonates remains crucial. Such data can be obtained by pharmacokinetic/pharmacodynamic (PK/PD) studies using PK/PD markers highly correlated with pathogen eradication (bacterial efficacy), therapeutic response (clinical efficacy) and even resistance data, as in adult patients 30, 31. Studies may also be well-designed observational controlled trials that in addition may assess long term toxicity 32 which is a crucial issue in neonatal drug development.

Conclusion

RCTs of antibacterial agents to treat or prevent infections in neonates are poorly designed, reported and interpreted. Even if these problems could be overcome, random allocation of antibiotics may not be feasible or even unnecessary when the main research question is to assess the effectiveness and safety of antibiotics in usual clinical practice. Different methods of evaluation should also be considered and further developed to circumvent such difficulties and ensure the efficient and safe use of antibiotics in neonatology.

Financial disclosure

This work is part of the FP7 TINN (Treat Infection iN Neonates) project supported by the European Commission under the Health Cooperation Work Programme of the 7th Framework Programme (Grant agreement n°223614) and coordinated by Professor Evelyne Jacqz-Aigrain. Funding source had no role in the study design, data collection and analysis, decision to publish or preparation of the manuscript.

Author contributions

Dr Florentia Kaguelidou and Pr. Evelyne Jacqz-Aigrain participated in design, data collection, analysis and interpretation of the data. Dr Mark Turner, Professor Imti Choonara, Professor John van Anker, Dr Paolo Manzoni, Professor Corinne Alberti, Professor Jean-Paul Langhendries and Professor Evelyne Jacqz-Aigrain participated in data interpretation. This paper was principally drafted by Dr Florentia Kaguelidou and Professor Evelyne Jacqz-Aigrain, and was critically reviewed and subsequently approved by each co-author in its final form.

Competing Interests

All authors have completed the Unified Competing Interest Form (available on request from the corresponding author) and declare no support from any organization for the submitted work, no financial relationships with any organizations that might have an interest in the submitted work in the previous 3 years and no other relationships or activities that could appear to have influenced the submitted work.

Supporting Information

Additional Supporting Information may be found in the online version of this article at the publisher's web-site:

Appendix S1

Search strategies for MEDLINE, EMBASE and the Cochrane Library

Appendix S2

References of retained RCTs on neonates and young infants