Clinical-trial design and selected drug safety issues for community acquired pneumonia

The early antibiotic trials, using historic controls, showed impressive mortality benefits for patients with bacteremic pneumococcal infection (1,2). The idea that antibiotic therapy may be useful to patients, because it kills infecting organisms, has been an attractive incentive and argument for physicians to expand their use for other infections, sometimes even in the absence of evidence that benefits exceed the risks. The last 50 or 60 years has witnessed an epidemic of antibiotic use, extending to indications such as otitis media, acute bacterial sinusitis and acute exacerbations of chronic bronchitis. In the absence of strong evidence of a health benefit from placebo-controlled clinical trials, there is no anchor for interpreting active-comparison trials. Like all drugs, antibiotics are associated with risks, and the recent placebo-controlled trials in acute bacterial sinusitis (3) serve an important reminder that insofar as self-limited infections resolve on their own, antibiotic treatment will be associated with risks but little or no benefit.

The evaluation of drug safety and efficacy are inextricably linked. High-quality evaluations of both are essential to characterize the risk-benefit profile of medications. Inadequate evaluations of either safety or efficacy compromise the knowledge base for physicians and patients. This paper will focus on both aspects.

Evaluations of efficacy

The current FDA guidelines on trial design for community acquired pneumonia have several weaknesses (4). The failure to insist on intention-to-treat analysis is an important limitation. Dropping out subjects who did not receive drug therapy for 48–72 hours or who died due to other causes undermines the randomization and converts a randomized trial into an observational cohort study. The failure to insist on double-blinding in the conduct of these trials is another weakness. Additionally, non-inferiority trials, in the absence of an anchor, may be difficult to interpret.

The use of “evaluable” patients in an analysis rather than the preferred intention-to-treat analysis (5) may lead to bias. In a meta-analysis of antibiotic trials comparing fluoroquinolones with other antibiotics in community-acquired pneumonia (6), Salkind and colleagues reported the results for both evaluable patients (odds ratio [OR] = 1.37; 95% CI = 1.11 to 1.68) and intention-to-treat analysis (OR = 1.22; 95% CI = 1.02 to 1.47). This difference in ORs may represent 15% to 30% of the typical non-inferiority margin used in comparative trials of antibiotics.

The failure to use double blinding is often associated with bias. Not only does the randomization need to be concealed (7), but investigators and patients should be blinded to treatment to avoid bias in the ascertainment or assessment of the outcomes of trials. In a meta-analysis that compared fluoroquinolones or macrolides with beta-lactams in community acquired pneumonia (8), Shefer and colleagues reported the results among trials with adequate concealment of randomization (relative risk [RR] = 0.96; 95% CI = 0.61 to 1.52) and among trials with unclear or inadequate concealment of randomization (RR = 0.68; 95% CI = 0.53 to 0.86). The difference, a measure of the bias associated with “open” trials, represents about 25% to 50% of the typical non-inferiority margins used in comparative trials.

While the absence of previous placebo-controlled trials makes the interpretation of comparative trials difficult, the widespread use of antibiotics for community acquired pneumonia as standard treatment largely precludes the conduct of placebo-controlled trials at this time, even for outpatient events. Pneumococcal pneumonia can be rapidly fatal in untreated patients. Physicians, patients, and human subjects committees would object to the ethics of using placebo treatment although withholding antibiotic treatment for limited periods of time is likely to be an acceptable and useful evaluation strategy in the setting of mild disease. Despite the current standards about treatment, data suggest that in some circumstances, a placebo-controlled trial or trial with an arm temporarily withholding antibiotic therapy might be ethical. In a meta-analysis of trials of non-severe community-acquired pneumonia (9), Mills and colleagues report that among subjects with Chlamydia pneumoniae or Mycoplasma pneumonia, treatment failures occurred in 8.8% of the 215 receiving antibiotics active against atypical agents and 10.4% of the 211 receiving beta-lactams (RR = 0.97; 95% CI = 0.87 to 1.07). In this setting, beta-lactams are functional placebos. In other words, in certain settings, such as when these atypical agents are the cause of the pneumonia, even placebo-controlled trials might be ethical.

Evaluations of safety

The FDA drug evaluation process includes pre-clinical studies to assess toxicity and a series of clinical studies in humans to define efficacy and identify potential safety problems. In the evaluation of efficacy, the sponsor has a particular outcome in mind, and the trials are designed and powered to test the drug effect on a pre-specified end point. The safety evaluation, on the other hand, is ad hoc. Safety data are collected and reported. There are usually many safety findings of minor common side effects, and there is no effort to adjust for multiple testing. To notice and define an emerging safety issue from among the welter of data coming from the animal studies and the Phase I-III trials requires a kind of “diagnostic” act of recognition. The FDA guidance on pre-market risk assessment recognizes the “exploratory” nature of safety analyses (10). When signals are apparent as they were for telithromycin (liver toxicity in rats, dogs and moneys) or for sparfloxacin (QT prolongation in dogs), it is nonetheless essential that these signals are fully and fairly evaluated in human studies.

The numbers of subjects evaluated in the pre-approval setting are adequate to identify common adverse events. After approval, the drug is typically used in large numbers of patients, some of whom had been excluded from the trials. In the post-approval setting, there are two major components to the ongoing evaluation of drug safety, the FDA adverse-event reporting system and additional studies conducted by sponsors, some of which are post-market commitments.

During 1969–2002, the FDA received 2.3 million adverse-drug reports (ADR) on 6000 marketed drugs (11). Even though ADRs, an incomplete case series, represent the weakest form of epidemiologic evidence, they are often responsible for drug withdrawals. During 1978–2003, 25 drugs, including temafloxacin for hemolytic syndrome and grepafloxacin for prolonged QT and arrhythmias, were removed from the market on the basis of either case reports or ADRs (11). Trovafloxacin use was restricted due to heptatoxicity, and more recently, sparfloxacin and gatifloxacin were removed from the market. Though the fluoroquinolones represent an important advance in antibiotic therapy, their toxicities can be serious, and they include QTc prolongation, torsades de pointes, tendonitis, glucose dysregulation, phototoxicity, nephritis, hepatitis, hemolytic uremic syndrome, eosinophilic pneumonia and seizures (12–14).

Like terfenadine and cisapride, the fluoroquinolones are known to prolong the QT interval on the electrocardiogram. In patch-clamp studies, Kang and colleagues evaluated several fluoroquinolones for their affinity to the cardiac human ether-a-go-go related gene (HERG) potassium channel, which is one mechanism of QT prolongation (15). The ratio, HERG IC₅₀/peak plasma concentration, was lowest for sparfloxacin (10), highest for levofloxacin (76), and intermediate for grepafloxacin (16) and moxifloxacin (20). Sparfloxacin and grepafloxacin have been removed from the market for QT prolongation. Whether moxifloxacin may increase the risk of QT prolongation, torsades and sudden death remains an open question.

In the CAPRIE trial, Anzueto and colleagues compared moxifloxacin with levofloxacin in 394 hospitalized adults aged 65 and older with community acquired pneumonia (16). They excluded subjects who were severely ill. Only 71% of those randomized were judged “evaluable.” The cure rates were 93% for moxifloxacin and 88% for levofloxacin (95% CI for the difference = −2% to 12%). Safety was evaluated in a companion article (17). Compared with levofloxacin, moxifloxacin was associated with a significant increase in the QTc interval (p = 0.03), an increase in the composite outcome of ventricular arrhythmic events (RR = 1.6; 95% CI = 0.8 to 3.5), and an increased risk of death during therapy (n = 6 vs 3; RR = 2.0; 95% CI = 0.5 to 8.0). On the basis of these data, Morganroth and colleagues conclude that moxifloxacin has “comparable cardiac rhythm safety” (17).

The CAPRIE trial was, however, seriously underpowered to evaluate either ventricular arrhythmic events or total mortality. The failure to find a significant difference in a small trial provides little assurance of safety. Indeed, the point estimate was a two fold increase in total mortality. Let us assume for the moment that moxifloxacin is associated with 3 extra deaths among 200 treated patients. In the setting of a life-threatening infection, if the drug provides important advantages compared with other available therapies, the overall risk-benefit profile may be attractive. But in the setting of more mild infections such as acute bacterial sinusitis and perhaps some mild forms of community-acquired pneumonia, which are rarely fatal, the extra 3 deaths per 200 treated patients would represent a serious safety matter, one that is rare enough that it is unlikely to be recognized by practicing clinicians.

Many post-market commitments are never completed by sponsors (18). Others are poorly designed. Faich and colleagues reported another study to evaluate the safety of moxifloxacin (19). All 18,409 subjects received moxifloxacin for 5 to 10 days for a variety of respiratory indications, including mild or moderate community-acquired pneumonia. While there was an external safety committee, ECG data were available on less than half of the cardiac events. Importantly, there was no control group. In the absence of a control group, this study provides little or no useful information about safety. Indeed, more marketing than science, it is what FDA officials have called a “seeding” study (20).

Sponsors often lack a symmetric interest in safety and efficacy. A primary difficulty with non-inferiority designs is that poorly conducted studies, which increase noise, are biased to toward the conclusion of non-inferiority even in the presence of important differences. The conduct of telithromycin safety study 3014, which included 24000 patients, is an example (21–23). This randomized trial, which was plagued by suspect and fraudulent data, did not detect an increase in hepatic adverse events. The adverse-event reporting system, a far weaker form of evidence, suggested that telithromycin was associated with acute liver failure at a rate 3.5 to 11 times higher than other antibiotics used for similar indications (24). The FDA needs to insist on the conduct of high-quality studies to provide adequate evidence about drug safety (25).

For antibiotics, drug safety issues are more complex than risk-benefit decisions for individual patients. The epidemic use of antibiotics has contributed to the development of drug resistance (26). In the Netherlands, for instance, where penicillin use is 4 defined daily doses (DDD) per 1000 inhabitants daily, 2% of pneumococcal isolates are penicillin resistant; but in France, where use is 10 DDD per 1000, 45% are resistant. The cross-national correlation between drug use and resistance is 0.84 (95% CI = 0.62 to 0.94). Excessive use of antibiotics in mild conditions provides little benefit for patients who receive the drug and simply contributes to drug resistance. The possibility of adverse health effects on others in the community makes the evaluation of risk-benefit especially complex. While the presence of antibiotics represents a powerful force for developing resistance, rapid reductions in the inappropriate use of antibiotics, even if they were to happen immediately, would be slow to affect a reversal of resistance, which is likely to occur largely by drift.

Concluding observations

The preferred design for clinical trials is a superiority trial that uses double blinding methods and intention-to-treat analyses. According to human-subjects conventions, if active comparison treatments are used, control patients should receive the optimal known therapy in appropriate doses and for appropriate durations. In severe community-acquired pneumonia, reductions in mortality are an attractive outcome. In double blind trials, patient-reported outcomes may be an important addition, and if investigator-declared resolution of community-acquired pneumonia remains an outcome of interest, it will be important to specify exact criteria for resolution and to maintain blinding of patients and physicians. Data safety and monitoring committees should be used for all trials of sufficient size and duration. Where safety signals are apparent during development, they should be aggressively evaluated in high-quality studies. Risks of antibiotics such as QTc interval prolongation and sudden death may be acceptable in the presence of convincing benefits in the setting of severe infections; but the risk-benefit profile derived from severe infections cannot necessarily be generalized to mild or self-limited infections, where the same drug-associated risks are likely to exceed the benefits. Complete and proper evaluation of both safety and efficacy in specific settings is essential to define the risk-benefit profiles of all drugs, including antibiotics.