Endoscopic ulcers are neither meaningful nor validated as a surrogate for clinically significant upper gastrointestinal harm

“A correlate does not a surrogate make”¹

Gastrointestinal complications are common among chronic NSAID/aspirin users. Despite the importance of this problem, randomized controlled trials with ulcer complications as the primary endpoint are rare. Theoretically, effective prevention strategies could be developed rapidly if a surrogate endpoint was available that reliably correlated with the true clinical outcome. In this issue of CGH, a group of opinion leaders make the argument that endoscopic ulcers might serve as a surrogate endpoint for clinically significant ulcer complications (abbreviated as ‘UGI harm”) ². Their manuscript was based on a “white paper” intended for submission to the US Food and Drug Administration and initially produced by a commercial vendor under a contract from Pfizer followed by input from the authors. The authors note that because evidence examining a direct relationship between endoscopic ulcers and UGI harm was unavailable, they sought “indirect evidence from a range of interventions and risk factors relating to the direction and magnitude on endoscopic ulcers and clinical outcomes”. Among the studies they examined, they found consistent and reproducible effects on UGI harm that paralleled endoscopic ulcers and recommended endoscopic ulcers as a possibly surrogate endpoint for UGI harm.

Surrogate endpoints

Because expensive large randomized clinical trials are generally needed to identify differences in infrequent events, there has long been an interest in identifying alternative outcomes, or surrogate endpoints that would allow smaller and more efficient studies. There are many examples of successful surrogate markers. For example, blood pressure is an important risk factor for cardiovascular-related mortality (ie, among hypertensive patients a 5% reduction in cardiovascular-related mortality and a 10% reduction in stroke is obtained for every 1 mm Hg reduction in blood pressure) ^{3, 4}. Almost 20 years ago criteria to validate surrogate endpoints in phase 3 trials were proposed ⁵. Essentially these criteria required that the surrogate must be a correlate of the true clinical outcome and fully capture the net effect of treatment on the clinical outcome. Many proposed surrogate markers have failed to meet these criteria ^{1, 6, 7}. Fleming identified four reasons for failure including 1) The surrogate is not in the causal pathway of the disease process, 2) Of several causal pathways of disease, the intervention affects only the pathway mediated through the surrogate, 3) The surrogate is not in the pathway of the intervention’s effect or is insensitive to its effect, and 4) The intervention has mechanisms of action independent of the disease process ⁶. Twaddell added an additional explanation: “even if an intervention has an effect on a surrogate marker and that marker is clearly in the causal pathway of the clinical end point, the effect may not persist long enough for the drug to alter the long-term clinical outcome. The drug may seem to be efficacious because of its short-term effect on the surrogate marker, but have no effect on the clinical outcome” ⁷. An example of failure of a generally reliably surrogate is the effect of calcium channel blockers in hypertensive patients. Reduction in blood pressure have proven to be a reliable surrogate endpoint for the evaluation of diuretics, however the favorable antihypertensive effects of calcium channel blockers were offset by other mechanisms making the surrogate fail ⁶.

Candidate surrogate markers are typically identified in phase 2 trials designed to identify whether a new intervention is biologically active and for guiding decisions about whether the intervention is promising enough to justify large definitive trials with clinically meaningful outcomes. Candidate surrogate endpoints must be rigorously validated in phase 3 trials. In the absence of a very well validated surrogate endpoint, the primary endpoint should be the true clinical outcome ⁶.

The authors proposing endoscopic ulcer be considered as a surrogate rely on correlations to support their hypothesis ². Possibly because of space limitations, each item of evidence was covered very briefly with little or no discussion of the weaknesses or exceptions. Importantly, the potential surrogate marker itself commonly used (a gastric or duodenal lesion ≥3 mm with significant depth) is in actuality often difficult to separate from an erosion. “Significant depth” has never been defined ^{8, 9}. One can ask, why 3 mm and not 5 mm, 10 mm, or some other number? How is the size to be measured (ie, in any one direction (ie, 3 mm × 0.5 mm?), minimal for any direction, etc)? 0ne person’s erosion may be another’s ulcer ^{8, 9}. For example, a blinded analysis of a teaching tape used to train investigators for detection of endoscopic ulcers for a pharmaceutical company sponsored study was performed by a group of experienced NSAID researchers. The experienced endoscopists disagreed that many of the lesions described as endoscopic ulcers were in fact ulcers ¹⁰.

Other studies suggest that the endoscopic determination of the presence or absence of an endoscopic ulcer, at least as done in large multicenter trials, has a definite and not insignificant false-positive rate. For example, although actual ulcers are extremely rare among H. pylori negative asymptomatic patients not taking any gastrotoxic drugs, 4.2% of volunteers who initially had negative endoscopies and were receiving placebo were found to have endoscopic ulcers in a 12 week trial (ie, an annual incidence of 16.8%) ¹¹. It is indeed rare for large multicenter trials to “calibrate” the observers to ensure consistency in diagnosis. In addition, video or still photographic images are rarely taken and review to confirm the reported findings. The authors noted that increasing age was associated with an increased prevalence of endoscopic ulcers and ulcer complications. Most correlations are incidental and not causative. For example, it has long been known that UGI bleeding increases with age independently of whether the patient is receiving NSAIDs ^{12, 13}. As noted above, the validity of a surrogate can only be confirmed by clinical trials with the endpoint being the true clinical outcome. The outcome of most interest is the presence of UGI harm defined as bleeding, perforation, or obstruction. The authors noted that one potential weakness of their analyses was that it remains unclear whether or how often endoscopic erosion/ulcers themselves transform into chronic or clinical ulcers and if so, which features allow one to predict such an occurrence ². Although this outcome is different from UGI harm as defined above, there are data that endoscopic ulcers and chronic ulcers are not directly related. For example, one of the first animal experiments producing ulcers with drugs was the production of gastric ulcer following the chronic administration of cinchophen. In that model the course of gastric injury was first a typical acute severe mucosal damage developed, followed by recovery of the mucosa despite continuing the drug. Then, a small fraction of dogs then went on to develop chronic gastric ulcer {Bollman, 1938 20560/id}. It has long been known that frequency of gastric mucosal damage seen in acute studies in man is much greater than noted in endoscopic surveys of chronic aspirin users. In acute studies, the damage is almost universally present and is usually diffuse and moderately severe. In contrast, in chronic studies the damage is usually mild except for a small percent with true ulcers and is reported in only 20–50% of subjects) ^{15, 16}.

Ulcers associated with UGI harm tend to be chronic and to recur in the same location suggesting that the process is different from that which produces endoscopic ulcers, especially when using locally gastrotoxic drugs ¹⁷. If UGI harm is typically the result of transformation of a particular lesion, how can that lesion be identified and separated from the often diffuse injury seen in a typical study using an NSAID with both topical and systemic actions?

If one accepted the surrogate maker, how would one use it?

The authors suggest that there is a quantitative aspect of endoscopic ulcers (ie, COX-2 inhibitors cause fewer than traditional NSAIDs and appear safer, antisecretory drug therapy reduces bad outcomes and endoscopic ulcers) ². First, is the hypothesis restricted to the few NSAIDs examined or universal? There are many papers ranking NSAIDs in terms of risk for UGI complications and the risk has been found to vary greatly ^18–21. Prior to the introduction of COX-2 inhibitors, the prodrug sulindac was noted to rarely cause endoscopic ulcers. According to the endoscopic ulcer surrogate hypothesis, it should be a safe NSAID and early studies were very encouraging that this would be the case. For example, short term blood loss studies using chromium-tagged red blood cells showed that sulindac was no more damaging than placebo and contrasted markedly to results with other available NSAIDs ²². Endoscopic studies using normal volunteers also showed that sulindac rarely caused erosions or significant gastric mucosal injury ^{23, 24}. Subsequent studies provided a biologic basis for these observations; as a prodrug and the active form was not present in the stomach. In addition, both products, sulindac sulfoxide and sulindac sulfide, were almost insoluble in acidic gastric contents thus preventing topical injury ²⁴. Unfortunately, the apparent lack of gastrotoxicity in short term studies did not reliably predict that sulindac was a particularly safe NSAIDS for when UGI bleeding within 30 days of exposure to one of seven NSAIDs was examined in 88,044 patients, sulindac users had the highest rate of UGI tract bleeding ²⁵. Another NSAID thought to have a low propensity to cause endoscopic ulcers, nabumetone, ²⁶ was given to high risk patients who had previously suffered NSAID-associated UGI bleeding. No evidence of protection was seen and bleeding was common ²⁷. If the endoscopic ulcer hypothesis does not hold for all orally administered NSAIDs, one would like to know whether it would be meaningful for alternate routes of drug administration such as rectal, parenteral, or enteric coated or for drugs other than NSAIDs (ie, iron, pivampicillin, microencapsulated or wax matrix potassium) which are also associated with endoscopic ulcers ²⁸?

Can the effect on endoscopic ulcer prevention be used to identify the best therapy?

In the original omeprazole vs. low dose misoprostol and the omeprazole vs. low dose ranitidine studies, low dose misoprostol was more effective for ulcer prevention among those without H. pylori infection than was omeprazole; omeprazole and ranitidine were not statistically different ²⁹. Low dose misoprostol was also superior to omeprazole for healing endoscopic gastric ulcers among H. pylori negative NSAID users ²⁹. Standard dose misoprostol proved superior to lansoprazole in preventing recurrence of endoscopic ulcers in a head to head comparison ³⁰. Using the proposed surrogate endpoint one might conclude that low dose misoprostol would be the preferred gastroprotective agent. In epidemiology studies, misoprostol as the combination drug Arthrotec® has consistently been associated with reduced events and deaths ³¹ and is the only drug shown in large randomized clinical trials to reduce UGI harm among chronic NSAID users ³². Epidemiologic studies suggest that PPIs are somewhat effective although the percent reduction remains unclear; there is no evidence suggesting a dose response effect ³¹. H₂-receptor antagonists appear to be marginally useful ³¹. Thus, despite the large number of studies using endoscopic ulcers as an endpoint, the lack of true outcome data prevents clinicians from making a reliable judgment regarding which gastroprotective strategy is superior or to assess their relative effectiveness.

The major gastroprotective strategies have been evaluated in single center studies using the clinical endpoint UGI harm (ie, bleeding). These studies have been done in high risk patients defined as those who had bled from NSAID ulcers. In that population, misoprostol showed essentially no benefit ²⁷ and omeprazole proved to be clinically ineffective both for preventing recurrent ulcers ³³ or UGI bleeding ^{27, 34}. Celecoxib alone was also not clinically effective ^{33, 34}. Clearly, for high risk patients, the proposed surrogate marker failed to correlate to actual outcome confirming again the admonition against using surrogates that have not been rigorously validated.

Where do we go from here?

A useful a surrogate endpoint must be able to be reliably identified, and highly and consistently reproducible, with minimal interobserver variability. Endoscopic ulcers as currently used meet none of these prerequisites. If standardization were to be accomplished, then rigorous validation would still be needed to confirm that its use was applicable to different interventions. Such validation will require actual clinical trials and the results may not be applicable for drugs with different mechanisms of action (eg, NSAIDs with systemic and topical activity vs. systemic only). The authors point out it may be difficult or impossible to currently do a placebo controlled trial because of ethical considerations ². Although, it may or may not be unethical not to use placebo, it may not be necessary in that misoprostol has been shown to reduce UGI harm making non-inferiority studies a viable option. Such trials are unlikely to be funded by PPI manufacturers which, as we noted previously, are particularly vulnerable to class effects, such that any new indications would usually be generalizable to all PPIs ³¹. The Obama administration has instituted a government program to compare different therapies and thus may provide a mechanism for studies to compare preventive strategies and possibly identify valid surrogate markers. Considering all the problems and uncertainties currently surrounding endoscopic ulcers, we believe it is unlikely they will ever become more than a marketing tool of uncertain relevance. We doubt endoscopic ulcers will disappear as an endpoint for as Haynes and Haynes recently commented “Unfortunately, when medical hypotheses are disproved, they act more like zombies than corpses, revived by the sorcerers of Mammon, aided and abetted by the inertia of medical practice” ³⁵.