Presumed benefit: lessons from the American experience with marrow transplantation for breast cancer

Few stories in medicine are as sobering as the American experience with autologous bone marrow transplantation (ABMT) for treating breast cancer. It is a story of young women dying from aggressive disease, well meaning physicians trying to be equally aggressive in treating it, and lawyers arguing that insurers should pay the bill. It is also a story of professional interests, weak research, financial gain, politics, and fraud. Because of its potential relevance to complex cancer therapies currently in development (such as gene therapy) we recount here the story and its lessons.

Summary points

• For over 10 years bone marrow transplantation for breast cancer was seen as an example of the general dilemma about who should pay for costly new life saving therapies

• This characterisation obscured the more basic question: Did it work?

• Intermediate outcomes and inadequate controls made preliminary evidence misleading

• Statements by physicians in the literature and the general press reinforced the presumption of benefit, as did the decision of government bodies to mandate insurance coverage

• The findings of major randomised trials did not support the use of the therapy

• This experience provides lessons relevant to complex cancer therapies currently in development

Early reports

Bone marrow transplantation was first performed to treat primary bone marrow disorders, but in the late 1970s it started to be used also for “rescuing” patients (using their own marrow) after supralethal chemotherapy or radiation for solid tumours.¹ By the mid-1980s there were strong proponents for using it this way in advanced breast cancer—on the basis that higher chemotherapy doses would be expected to kill more tumour cells. The enthusiasm for this hypothesis was evident in comments made to the New York Times in 1989 by the head of the breast cancer section at the National Cancer Institute: “The evidence is absolutely convincing that the dose intensity is correlated with survival.”² But other oncologists were more sceptical. In the same article one pointed out that the notion of dose-response was purely theoretical and also applied to toxicity: “It's a hypothesis . . . and higher dosages are more toxic.”²

And there were few data. The first report in the general medical literature on the treatment's efficacy for breast cancer appeared in the Annals of Internal Medicine in 1988.³ The article reported on 172 women from 27 studies. The summary response rate (defined as tumour shrinkage ⩾50%) was 58%. There were no controls. A few months later the Annals published a review that included 159 women and noted an 80% response rate.⁴ In both articles the authors were cautious, concluding: “Critical evaluation will require controlled trials,” and “response rates that are probably superior to the best available with conventional therapies . . . although not yet associated with improved survival.”

Comments made to the press, however, were less cautious. In the news section of the Journal of the National Cancer Institute, one author said: “I think this shows that ABMT can be a very effective form of treatment.”⁵ A similar sentiment appeared in the Los Angeles Times: “A combination of bone marrow transplants and very high doses of anti-cancer drugs may be able to double the survival rate of patients with advanced breast cancer, a Boston researcher reported last week.”⁶

Patients in the media

Other factors converged to make bone marrow transplantation for breast cancer a big story. Women's issues were prominent. Breast cancer was both common and feared. Transplantation was a source of hope—a technologically advanced procedure. And soon there was the added dimension of money.

In December 1989 the Washington Post described a Johns Hopkins study of 20 women treated with bone marrow transplantation and reported “partial success” (while acknowledging that the prospects for long term survival were not yet known).⁷ The article reassured readers that the high cost of the treatment ($75 000-100 000) “is usually covered by health insurance or Medicaid.”

Four months later a 35 year old mother of three proved this statement wrong. Page 1 of the Post's Metro section declared: “Maryland Mother's Chance of Life Hinges on Trial; Patient Sues Insurers For Cancer Treatment Cost.”⁸ Pamela Pirozzi had been advised that her “best chance of surviving more than a year” was a transplant, but her insurance company had refused coverage, stating the procedure was still “experimental.” Armed with a “list of insurance companies in other states that, when challenged, have agreed to pay for the procedure,” the Pirozzis sued.

A federal judge ruled in her favour: “To require that the plaintiff or other plan members wait until somebody chooses to present statistical proof . . . that would satisfy all the experts means that plan members would be doomed to receive medical procedures that are not state of the art.”⁹ The same month another federal judge ordered a Massachusetts insurer to pay for a Boston woman to receive a transplant in North Carolina.¹⁰

State of the art or state of the theory?

American insurance companies argued that bone marrow transplantation was experimental and thus not covered by their policies: “We view ABMT for breast cancer as investigational and experimental, because the treatment has not proved to be safe and effective.”¹⁰ Such a broad definition did not fare well in courts or the court of public opinion. It also highlighted the dilemma for policy makers: how do you define “experimental therapy”?

In November 1990 the Blue Cross Blue Shield Association of insurers provided a pragmatic definition: any therapy being studied in a randomised trial. After years of inconclusive data a National Cancer Institute panel had just decided to conduct a major randomised trial to test the effectiveness of bone marrow transplantation. The national study was to recruit 1200 women with metastatic breast cancer or high risk regional disease. The association promised at least $10m to help fund the trial.¹¹

It was the first time a private insurer had agreed to fund a trial of treatment.¹² The decision was a reaction to the controversy and the lawsuits. The senior vice president of the association pointed out the benefits of the decision: “By funding some of the clinical care costs associated with the National Cancer Institute trials, we're able to provide access to this treatment while contributing to the search for a definitive answer to the question of whether or not it works.”¹²

Health services research and ABMT

There was sufficient controversy in the medical community to attract non-oncologists. In April 1992 the Journal of Clinical Oncology published a structured review of the data by an epidemiologist: 72 studies and abstracts on the use of bone marrow transplantation for breast cancer—but no randomised trial.¹³ After applying three simple inclusion criteria (peer review, survival or response as an outcome, and more than 10 patients), the reviewer had only 10 case series left to summarise (238 patients). Six large series of conventional therapy and 45 randomised trials (comparing various agents) served as a crude comparison group. The author concluded that survival rates after transplantation and conventional chemotherapy were essentially the same.

The same month JAMA published an article on bone marrow transplantation for breast cancer—ironically addressing the cost effectiveness of the procedure.¹⁴ Given that there were no good data on effectiveness, this was predicted using case series data on response rates and a decision analytic model. In retrospect the model's predictions were half right. The estimated 27% three year survival after transplantation was similar to that observed in the subsequent randomised trial, but the estimate of 14% survival for those treated with conventional chemotherapy was less than half that actually observed.¹⁵ The reported cost effectiveness ratio was $115 800 per life year. Although the authors were clear that these data were to be used while awaiting trial results, the presumption of benefit was also clear: “Using reasonable assumptions, ABMT provided substantial benefit but at a cost that may be untenable.”¹⁴

The juxtaposition of these two articles was striking. On the one hand was the lack of evidence of effectiveness. On the other was the conclusion that it was effective but too expensive. The authors of the two articles tried to reconcile the “apparently different conclusions” in a letter to JAMA,¹⁶ but the sound bite was already out: “High cost marrow treatment helps fight breast cancer.”¹⁷

Insurance coverage and lawsuits

Despite the controversy about whether the therapy worked, more women were taking their insurers to court. By 1990 one group of insurers had been sued by over a dozen patients and consumer advocacy groups to cover the treatment.¹¹ The same year saw three decisions from the federal courts over bone marrow transplantation for breast cancer. By 1993, there were 19.

The most visible lawsuit that year took place in Riverside, California. Nelene Fox was a 39 year old mother of three diagnosed with breast cancer in 1991 and soon found to have metastatic disease. Her insurer, Health Net, refused her request for ABMT stating it did not cover “investigational” procedures. With the aid of 1700 donors, Ms Fox's family raised $210 000. She underwent the treatment in 1992 and died a few months later. Her husband and attorney brother sued Health Net for their refusal to pay, and the jury ruled in favour of the family. The award, $89.1m, was the largest “ever levied against an insurance company for refusing to provide health coverage benefits.”¹⁸

While some insurers were denying payment (and going to court), others were giving in. In February 1994 the New England Journal of Medicine published its first article on the issue—on the variation in insurance coverage.¹⁹ The authors (including one of the nation's most prominent transplanters) found no relation between the clinical characteristics of patients and the insurer's decision about payment and concluded that coverage decisions were “arbitrary and capricious.”¹⁹ The choice of adjectives echoed in newspapers across the country.

The investigators argued that insurers must support clinical research, but their definition of clinical research seemed unduly broad. Most of the 533 patients they described were involved in trials that had nothing to do with whether transplantation was more effective than conventional therapy. About half the patients were in phase I or II studies; about half in randomised trials involving subsidiary comparisons (such as a trial of an anti-emetic after transplantation). Only 15 patients were randomised not to receive bone marrow transplantation and, even in these cases, the transplant was merely delayed. Nevertheless, the authors were clear about the value of research (and the presumptive value of treatment): “Policy restrictions that limit access to clinical trials are likely to delay the evaluation of therapeutic programs and to result in the relegation of patients to outdated and inferior treatments.”¹⁹

Politics and policy

The presumption of benefit was widespread. Because investigators were struggling to enrol patients in the randomised trials, even the mundane issue of patient accrual made headlines.²⁰ Four years into enrollment, the trial of patients with metastatic breast cancer had only half of the 549 patients needed. Many physicians and patient advocacy groups believed the question was not in doubt: transplantation was better.²¹ Others pointed out the strong financial incentives for medical centres to perform transplants.²²

In September 1994, while the National Cancer Institute was arguing that there was scientific uncertainty about the effectiveness of transplantation, a different branch of the government weighed in.²⁵ The Office of Personnel Management ordered all health plans serving federal employees to grant coverage of autologous bone marrow transplantation for breast cancer within 24 hours or risk being dropped from the programme. The decision affected 350 health plans serving 9 million people.

The decision was officially based on several factors. Nearly a third of the programme's health plans already covered the procedure or were about to. Virginia, the home of many federal employees, had recently mandated state-wide coverage (one of seven states ultimately to do so). Finally, the increasing number of lawsuits highlighted the downsides of not covering the procedure.

There was also political pressure. In October 1993 54 members of congress wrote to demand that the Office of Personnel Management cover the procedure.²³ They cited statistics from a report at Duke University claiming transplantation and high dose chemotherapy were “eight times more effective than conventional dose therapy.” (The statement was loosely based on an article on bone marrow transplantation in high risk primary breast cancer reporting a 72% 5-year survival rate compared to 35% in historical controls.²⁴) In June 1994 the directors of five major academic cancer treatment programmes wrote to the office, presenting even more favourable data. Finally, Representative Eleanor Holmes Norton led a hearing in August 1994, with testimonies from federal employees who had been denied coverage. Despite hearing the National Cancer Institute position that “formal scientific evaluation (ought) to proceed the routine use of such a toxic and expensive therapy”²⁵ she called for the Office of Personnel Management to re-evaluate its policy.

Fraud and findings

In 1995 the Journal of Clinical Oncology reported on the first randomised trial of bone marrow transplantation in metastatic breast cancer.²⁶ South African oncologists led by W R Bezwoda reported a complete response rate (no evidence of tumour) of 51% in women randomised to transplantation, compared with 4% in those receiving conventional therapy. The benefit in median survival was even more impressive: 90 weeks versus 45. The article was cited about 300 times before concerns about its validity. Four years later at the 1999 American Society of Clinical Oncology meeting, Bezwoda reported equally impressive results for transplantation in high risk primary breast cancer. At the same meeting, however, four other randomised trials were presented (Lotz JP; Peters W; Scandanavian Breast Cancer Group).¹⁵ None supported the use of the therapy (table). Bezwoda's results suddenly seemed too good to be true. Leading oncologists decided an on site review was essential.²⁷

The review team identified multiple problems: the protocol was different from that presented at the 1999 meeting, the enrollment criteria were different, eligibility was not documented, there were no signed consent forms. Moreover, the team was repeatedly denied access to the control patients. The vice president of the society made the logical inference to the Guardian (London) “You could conclude that they might not exist.”²⁸

Autologous bone marrow transplantation was being viewed in a different light. The finding of no benefit was well publicised in the general press. In February 2000 Aetna Insurance announced it would no longer cover the procedure outside federally sponsored trials.²⁹ A New England Journal of Medicine editorial summarised what was for many a new way of thinking: “To a reasonable degree of probability [AMBT for metastatic breast cancer] has been proved to be ineffective and should be abandoned in favour of well justified alternative experimental approaches.”³⁰

Lessons

For over 10 years desperately ill women had sought bone marrow transplantation as their best chance for survival. Many physicians encouraged this judgment. Fearing bad publicity and lawsuits insurers reluctantly agreed to pay the considerable charges. A strong presumption of benefit and equally strong financial interests impeded progress towards finding an answer.

The obvious lesson from these events was articulated in the New York Times by two of the treatment's most visible critics. “As a society we have to accept that rigorous evaluation of a new treatment is essential . . . Skipping this step may seem like a compassionate act, but it can have devastating consequences.”³¹

It is important to remember that preliminary evidence can be misleading: intermediate outcomes (such as response rates) may not correlate with survival, and historical controls may not be comparable. And proponents can be persuasive. The lesson is familiar: it is the case for randomisation. Future research may still show the utility of a chemotherapeutic regime for breast cancer which requires autologous bone marrow transplantation, but for now this story serves as a good example of why scepticism is important in medicine. There are also less obvious lessons.

Firstly, it is premature to raise the question of cost effectiveness when effectiveness is unknown. Even though the report on cost effectiveness was explicit about the existence of ongoing trials, the statement that transplantation provides “substantial benefit but at a cost that may be untenable” served only to reinforce the presumption of benefit. It also added weight to the prevailing view of payers as institutions that would deny access to life saving services simply to save money. When presented data about cost effectiveness, many clinicians assume that effectiveness is reasonably established.

Secondly, establishing what is “experimental” is an important role for government. Accepting that new therapies are experimental is difficult in our culture. Given the increasingly commercial nature of medicine, we can expect aggressive promotion of new therapies. Without authoritative statements saying otherwise, benefit will be presumed and enrollment in randomised trials will suffer. Having the National Institutes of Health work with payers to define what is experimental could benefit both. Insurers could fund experimental therapies in randomised trials: that would provide them with much needed limits and the public with financial support for major trials.

Thirdly, public officials should not mandate coverage in the absence of clear data. The decisions by the Office of Personnel Management and seven state governments sent the wrong message. It was counterproductive to have one arm of government apparently assert the treatment was no longer experimental while another was trying to make the uphill case for an experiment.

Finally, the news media watchdog role should be extended to health care. The media was slow to see that there was more to the story than the question of how to pay for a costly, new, life saving therapy. Proponents were successful in characterising the case against transplantation as simply about money. Yet proponents of “advances” will always be more vociferous than detractors: they usually have stronger interests (both professional and financial) in arguing for a particular technology than detractors have in arguing against it. Given a public primed to believe in medical breakthroughs, the press should focus on evidence of effectiveness before raising arguments about money. All would be well served by a press that displayed the same scepticism about pronouncements from medicine as it does with pronouncements from government.

Figure.

Timeline of major events

Table.

Summary of four randomised trials of bone marrow transplantation for breast cancer originally reported at the 1999 American Society of Clinical Oncology meeting (South African trial excluded)

Study	No of patients and disease stage	Overall survival (%)
		Follow up (months)	Bone marrow transplantation	Control
Metastatic breast cancer
Philadelphia group15	199, stage IV	36	32	38
French group (Lotz)	61, stage IV	60	30	19
High risk primary breast cancer
Cancer and leukaemia group B (Peters)	874, ⩾10 nodes	36	78	80
Scandinavian group	525, ⩾10 nodes	20*	85	85

^*Estimate from abstract