Are selective COX 2 inhibitors superior to traditional non steroidal anti-inflammatory drugs?

Selective cyclo-oxygenase 2 (COX 2) inhibitors, including celecoxib (Celebrex) and rofecoxib (Vioxx), are hypothesised to have a lower risk of gastrointestinal complications than traditional non-steroidal anti-inflammatory drugs.¹ In September 2000 the celecoxib long term arthritis safety study, better known as CLASS, was published in JAMA.² This trial, widely cited and distributed, concluded that a COX 2 inhibitor was associated with a lower incidence of complications than traditional non-steroidal anti-inflammatory drugs. What was much less widely publicised were criticisms that contradicted this conclusion.

CLASS was reported as a three arm trial comparing celecoxib 800 mg/day with ibuprofen 2400 mg/day and diclofenac 150 mg/day in osteoarthritis or rheumatoid arthritis. Clinically relevant upper gastrointestinal ulcer complications (bleeding, perforation, or obstruction) and symptomatic ulcers during the first six months of treatment were described as the two main outcome measures, comparing incidence rates for celecoxib and a traditional non-steroidal anti-inflammatory drug (fig 1). It was concluded that, compared with the traditional non-steroidal anti-inflammatory drug, celecoxib “was associated with a lower incidence of symptomatic ulcers and ulcer complications combined.”³ The trial was funded by celecoxib's manufacturer Pharmacia.

Figure 1.

Main results according to published report (top) and pre-specified protocol (bottom). Alternate definition of ulcer related complications, pre-specified by the FDA, included more stringent criteria to address serious gastrointestinal bleeding. P values are from log rank tests.

An article in the Washington Post in August 2001³ and two letters published in JAMA in November 2001⁴,⁵ drew attention to the fact that complete information available to the United States Food and Drug Administration contradicted these conclusions. The paper reporting CLASS² actually referred to the combined analysis of the results of the first six months of two separate and longer trials. The protocols of these trials differed markedly from the published paper in design, outcomes, duration of follow up, and analysis.

Two comparisons were originally planned: celecoxib versus ibuprofen, and celecoxib versus diclofenac. The Food and Drug Administration was concerned that selective COX 2 inhibitors could interfere with the benefits of COX 2 in ulcer healing.⁶ This could lead to a long term increase of ulcer related complications that occur without warning symptoms.⁴ Therefore the pre-specified primary outcome was ulcer related complications, not symptomatic ulcers, in both trials, while the maximum duration of follow up was 15 and 12 months respectively.^7–8

A two step procedure was planned to control for a type 1 error: after comparing celecoxib with the non-steroidal anti-inflammatory drugs combined, a pairwise comparison of celecoxib with each of the two non-steroidal anti-inflammatory drugs, ibuprofen and diclofenac, had to be done. The protocol explicitly specified that celecoxib would be claimed to be different from the traditional non-steroidal anti-inflammatory drug only if both overall and pairwise comparisons were statistically significant for ulcer related complications.⁷

Analysis according to a pre-specified protocol showed similar numbers of ulcer related complications in the comparison groups (fig 1).⁷,⁸ Almost all the ulcer complications that had occurred during the second half of the trials were in users of celecoxib (fig 2). When an alternate definition of ulcer related complications (pre-planned by the Food and Drug Administration) was used, a non-significant trend was found in favour of diclofenac (fig 1).⁷,⁸ These results clearly contradict the published conclusions.² They were available when the manuscript was submitted, but were neither referred to in the article² nor reported to JAMA.⁹

Figure 2.

Kaplan-Meier estimates for ulcer complications according to traditional definition. Results are truncated after 12 months, no ulcer complications occurred after this period. Adapted from Lu 2001.⁷

Two issues cause concern. Firstly, the authors' explanations⁹ for these serious irregularities were inadequate. They failed to justify the post hoc changes in design, outcomes, and analysis and provided an unconvincing explanation for considering the six month follow up only. They argued that a large and differential dropout rate had occurred during the later stage of the trial, which depleted patients with gastrointestinal adverse events preferentially in the groups taking non-steroidal anti-inflammatory drugs and that these patients were at higher risk of developing ulcer related complications.⁹ However, the absolute number of dropouts and withdrawals, both overall and due to gastrointestinal adverse events, increased gradually, without any sudden increase after six months, and withdrawal rates stayed roughly constant in different treatment groups during the entire follow up period. In addition, there was no robust evidence that gastrointestinal adverse events were actually a risk factor for ulcer related complications.^7–8

Secondly, the flawed findings published in the original article² appear to be widely distributed and believed. About 30 000 reprints of CLASS were bought from the publisher (W Bartolotta, personal communication), and a recent search of the Science Citation Index yielded 169 articles citing it, more than 10 times as many citations as for any other article published in the same issue. This wide distribution and citation has coincided with the sales of celecoxib increasing from $2623m in 2000 to $3114m in 2001.¹⁰

Publishing and distributing overoptimistic short term data using post hoc changes to the protocol, while omitting disappointing long term data of two trials, which involved large numbers of volunteers, is misleading. While some of the problems related to CLASS were partially covered in the news sections of BMJ¹¹ and other journals, it was not emphasised how flawed the trial actually was,² and how inadequate the authors' justifications.⁹ Consequently, CLASS may still be relied on by many physicians without reference to these flaws. In our experience most still believe the findings published originally.² For example, most of 58 physicians attending an osteoarthritis workshop in Berne, Switzerland, in December 2001 had not realised that CLASS was seriously biased.

In contrast with the CLASS trial,² the VIGOR trial,¹² which was similar in design and outcomes, found an unequivocal benefit of another selective COX 2 inhibitor, rofecoxib, over traditional non-steroidal anti-inflammatory agents. Four potential reasons for this discrepancy warrant further exploration. Firstly, aspirin was used concurrently by about 20% of patients in CLASS (but not in VIGOR). Secondly, naproxen, rather than diclofenac (which has greater COX 2 selectivity¹), was used as the comparator in VIGOR. Thirdly, CLASS employed higher doses of celecoxib than usual, and finally rofecoxib has considerably higher COX 2 selectivity than celecoxib.¹

Two things need to happen now. Firstly, an “industry independent,” individual patient data meta-analysis of all large scale, long term trials of selective COX 2 inhibitors must be performed to include both published and unpublished data. Secondly, the wide dissemination of the misleading results of the CLASS trial has to be counterbalanced by the equally wide dissemination of the findings of the reanalysis according to the original protocol. If this is not done, the pharmaceutical industry will feel no need to put the record straight in this or any future instances.