Priority setting for new technologies in medicine:
A Case Study

Peter A. Singer, MD, MPH, FRCPC
Sun Life Chair and Director, University of Toronto Joint Centre for Bioethics
Douglas K. Martin, PhD
Research Associate, University of Toronto Joint Centre for Bioethics
Mita Giacomini, PhD
Assistant Professor, Clinical Epidemiology & Biostatistics,
McMaster University
Laura Purdy, PhD
Bioethicist, University of Toronto Joint Centre for Bioethics

All correspondence and reprint requests should be sent to:
Dr. Peter A. Singer
University of Toronto Joint Centre for Bioethics
88 College Street
Toronto
Ontario
M5G 1L4
CANADA

tel: 416-978-4756
fax: 416-978-1911

e-mail: peter.singer@utoronto.ca.

Preliminary results were presented at the Second International Conference on Priorities in Health Care, British Medical Association, London UK, 8-10 October 1998.

Word Count: 1,956

ABSTRACT

Objective:To describe priority setting for new technologies in medicine.

Design: Qualitative study using case studies and grounded theory.

Setting: Two committees advising on priorities for new technologies in cancer and cardiac care in Ontario, Canada.

Participants: The two committees and their 26 members.

Outcome measures: Accounts of priority setting decision making gathered by reviewing documents, interviewing members, and observing meetings.

Results: We identified six inter-related domains of priority setting for new technologies in medicine: the institutions in which the decision are made, the people who make the decisions, the factors they consider, the reasons for the decisions, the process of decision making, and the appeals mechanism for challenging the decisions.

Conclusions: We developed a model of priority setting for new technologies in medicine.
 

What this paper adds

Although the individual elements of priority setting in health care have been described, they have not been collected into an integrated model.

Using case studies of priority setting for new technologies in cancer and cardiac care in Ontario, we describe a priority setting model in six domains: the institutions in which the decision are made, the people who make the decisions, the factors they consider, the reasons for the decisions, the process of decision making, and the appeals mechanism for challenging the decisions.

INTRODUCTION

Because demand for health care exceeds the supply of resources allocated to finance it, priority setting is a problem for every health care system in the world. But how should we set priorities within health systems?

Two key issues lie at the heart of priority setting – legitimacy ("Under what conditions should authority over priority setting be placed in the hands of a particular organization, group or person?") and fairness ("When does a patient or clinician have sufficient reason to accept as fair particular priority setting decisions?").

At present, decision makers and the public have difficulty determining whether particular priority setting decisions are legitimate and fair. In order to connect what should be done to make legitimate and fair priority setting decisions with actual practice, a first step is to understand how groups make these decisions.,

Since innovation is the primary driver of escalating health care costs, these issues are particularly acute for new technologies. The purpose of this study is to develop a model describing priority setting in one specific context – new technologies in cancer and cardiac care.

METHODS

Design

This study used qualitative methods of case studies and grounded theory. ,

Setting

We studied 2 cases – the Cancer Care Ontario Policy Advisory Committee and the Cardiac Care Network of Ontario Expert Panel on Intracoronary Stents and Abciximab (a glycoprotein IIB/IIIA inhibitor). The Cancer Care Ontario Policy Advisory Committee "manage[s] the selection and introduction of all new drugs within the funds provided." The Cardiac Care Network of Ontario Expert Panel on Intracoronary Stents and Abciximab was mandated to "review current literature and practice ... and recommend, where possible, a cost-effective, multi-year plan for stent volumes and use of Abciximab that supports the principles of quality of care, access and affordability.", p.1

Sampling and sample size

Of 26 committee members, 21 were interviewed (11 of 15 from Cancer Care Ontario and 10 of 11 from the Cardiac Care Network of Ontario). The membership of the cancer committee included 3 lay members (1 of whom was a patient), a government representative, a pharmacist, a nurse, 2 administrators, and 7 oncologists. The membership of the cardiac committee included 1 lay member (a patient), a government representative, an administrator, a health economist, and 7 cardiologists or cardiac surgeons. All meetings of both committees were observed for at least 12 months from the formation of each committee in 1997 to December 1998. At this point, the analysis was "saturated"; that is, no new major domains emerged.

Data Collection

Documents reviewed included information about the 2 organizations, the written mandate of both committees, committee minutes, correspondence to committee members, and committee reports. Semi structured interviews were conducted by a single interviewer (DKM), either in person or over the telephone, and audio-taped and transcribed. The interviewer asked respondents to describe their role on the committee and evaluate their effectiveness; describe the committee process; indicate whether the process was fair; and indicate whether the decisions were fair.

Data Analysis

We analyzed the data in three steps.9, pp. 61-142 First, using open coding, we identified passages of text that related to a theme or idea, and then grouped similar concepts into conceptual categories (e.g. benefit). Second, using axial coding, we further developed the conceptual categories and compared them with each other (i.e., the 6 domains of the model). Third, using selective coding, we developed a model by relating the domains to a central theme and to each other (i.e. the metaphor of a gem). The analysis was conducted simultaneously with data collection.

The trustworthiness of our findings was enhanced in three ways. First, two investigators in addition to the primary analyst coded the raw data from one meeting to ensure the authenticity of the coding scheme; the final coding scheme was developed by consensus and used for the analysis. Second, two investigators familiar with all the primary data developed the interpretation. Third, a draft of this paper was endorsed by members of both committees in a "member check."

Research Ethics

This study was approved by the Committee on Use of Human Subjects of the University of Toronto. Each committee member interviewed provided consent.

RESULTS

We identified six inter-related domains of priority setting for new technologies in medicine: the institutions in which the decision are made, the people who make the decisions, the factors they consider, the reasons for the decisions, the process of decision making, and the appeals mechanism for challenging the decisions.

Institutions

Priority setting in both committees was established within a legitimate organizational context. Both organizations were both created by the Ministry of Health to advise the Ministry. Although the mandate of both committees included priority setting, they struggled with this mandate. The cardiac committee debated about the distinction between making recommendations for clinical practice and for funding priorities. The cancer committee decided to advocate for increased funding if it found itself in the situation of denying an effective treatment to patients because of funding limits.

People

A key element of fairness described by committee members was that multiple stakeholder perspectives were represented. A difference emerged between the 2 committees with respect to the participation of lay committee members. The three cancer committee lay members were more satisfied with their participation than the lone cardiac lay member. One of the cancer committee lay members said,

"I think on access issues I’ve been effective. … I’d say that my frustrations have been fewer than I thought they would be at the start. . . . it could be that just having community reps with this perspective sitting on that committee makes them have that awareness ... So, I won’t say I’ve been personally totally successful, but I think the process has been more successful than not in fulfilling what I think is my role."The lay member on the cardiac committee questioned his effectiveness:"I'm a businessperson, and to walk into a medical panel where they're talking a great deal of medical topics that I knew very little about, it's very hard for me to have the confidence to question what they were doing. You try to some extent but, if there was a matter of conflict it would be very easy for me to defer to their expertise … I think if there were two of us that might have helped … So one doesn’t feel quite so overwhelmed by the rest of the panel."This comparison suggests that a critical mass of public participation is required.

Factors

The individual factors that shaped the decisions of both committees were: benefit, evidence, harm, cost, cost effectiveness, and pattern of death. Benefit had the greatest role in the deliberations.

Evidence represented the degree of certainty with which the benefit was known. Sometimes the committee had to balance benefit against evidence. For instance, the cardiac committee compared high quality evidence of a small benefit of stent use for patients with "favourable coronary artery lesions" in comparison to lower quality evidence of a potentially large benefit for patients with "unfavourable" lesions.

When there was evidence of both significant benefits and harm, participants felt that individual patients, with advice from their physicians, were best suited to decide.

The total cost of providing a treatment to a group of patients led to a discussion of access and equity. For example, the cancer committee decided that patients should be treated equally without regard to whether they happened to belong to a relatively large or small group of patients.

Although available for only a minority of the drugs under consideration, formal cost effectiveness analyses were used to support decisions that were primarily made on grounds of benefit and evidence. A cardiac committee member said:

"Without having hard numbers on the cost effectiveness, we did use the concept, at least, of cost effectiveness … in deciding, for example, not to recommend funding for the use of stenting in areas where we thought cost effectiveness would be unattractive."The patterns of death of patients with cancer compared to cardiac disease influenced the deliberations of the committees. For example, it was recognized that patients with metastatic colorectal cancer deteriorated and died according to a progressive trajectory which could be forestalled but not reversed. By contrast, the use of stents in patients with cardiogenic shock might not only prevent death, but potentially return patients to their earlier state of health. The possibility of "saving" patients, even if remote, tended to influence the allocation of resources.

Reasons

The reasons underlying both committee’s decisions did not rest on individual factors like those described above. Rather, both committees made decisions based on clusters of factors. Moreover, actual decision making was more complex than simply one drug and its attendant cluster of factors. Some decisions involved clusters of drugs, each with their own cluster of factors, for a single disease. Other decisions involved clusters of factors, clusters of drugs and clusters of diseases. To illustrate, we present two examples from the cancer committee deliberations (see box).

Examples of decision clusters

Example 1 Clusters of factors for one drug in one disease

Ralitrexed for the treatment of colorectal cancer: In randomized comparisons with the standard therapy (5FU), Ralitrexed showed equivalent benefit (prolongation of survival and response rate). The toxicity of Ralitrexed was thought to be different than 5FU, but not worse. Although Ralitrexed is more convenient to give, it is approximately 200 times more expensive. Therefore, the panel reasoned that because Ralitrexed is no better than the standard therapy in terms of benefit or harm, and much more expensive, it should not be funded. At a subsequent meeting, the panel decided to recommend funding for Ralitrexed for patients with excessive 5FU toxicity, or for patients who lived beyond a specific distance from a treatment centre.

Example 2  Clusters of factors for two drugs in two diseases

Pamidronate and Clodronate for the treatment of myeloma: Based on evidence of equal quality, Pamidronate has shown better symptom relief and prevention of complications (decreased bone pain and decreased number of fractures). In addition, in one study Pamidronate showed a survival advantage. Pamidronate is much more costly than Clodronate. The population of myeloma patients is small, therefore the overall cost was expected be modest; however, a lack of alternatives for those patients made the need great. Some, but not all, hospitals were currently providing Pamidronate, so province-wide funding was required to ensure equal access to all myeloma patients. The panel decided that Pamidronate, but not Clodronate, should be funded because – in the context of a small population of patients with great need – there existed benefits of enhanced survival rates and a better quality of life.

Following the decision in the context of myeloma, the committee considered Pamidronate and Clodronate for the treatment of breast cancer. The cluster of factors attending Pamidronate and Clodronate in the context of breast cancer treatment were similar to those in the context of myeloma. There were two significant differences: the evidence for oral clodronate in breast cancer was much stronger than for myeloma; and the population of patients with breast cancer was much larger than with myeloma and, therefore, the overall cost to the program was larger. The panel decided to fund IV Clodronate for those patients who could not tolerate oral Clodronate (for which alternate funding mechanisms existed) and that they would fund Pamidronate only for patients who tried and could not tolerate Clodronate.

Example 1 shows how clusters of factors were used to develop a reason. Example 2 shows how clusters of factors, clusters of drugs and clusters of diseases converge when reasoning through more complex decision making. This reasoning process enabled the committee members to periodically review previous decisions to evaluate the consistency of their reasoning.

Process

A key element of the process was transparency. Participants identified other aspects of committee process that contributed to fairness: acknowledging conflicts of interest, providing the opportunity for everyone to express views, ensuring that all committee members understand the deliberations, maintaining honesty, building consensus, ensuring availability of external expert consultation, ensuring appropriate agenda setting, maintaining effective chairing, and ensuring timeliness in making funding decisions to get effective new technologies to patients.

Appeals

Participants emphasized that an appeals mechanism was a key element of fairness. In response to challenges of some decisions, the cancer committee decided it should revisit decisions if new evidence or new arguments became available.

DISCUSSION

Although it may not be generalizable, we developed a model of priority setting in one specific context – new technologies in medicine. Our model can be likened to a gemstone. Six domains -- institutions, people, process, factors, reasons, and appeals -- form the facets. Each facet may be more or less perfect (legitimate or fair), and contributes to the perfection (legitimacy or fairness) of the whole.

However, just because a group makes priority setting decisions in a particular way, that does not make it "right." Our goal was to describe how these groups made priority setting decisions, not to prescribe how they should make them. We did not seek to justify what makes a particular domain more or less fair.

The closest analogues to our study are the work of Foy et al and Hope et al. Foy et al. found that priority setting decisions regarding new cancer drugs were based on "evidence threshholds" – cut off points determined from information on effectiveness. Hope et al. described the use of evidence of effectiveness, equity, and patient choice in a health authority’s priority setting decisions. Our study describes these factors in priority setting decisions, but also places the factors in a model that includes institutions, people, process, reasons, and appeals. Of course, our model developed in the context of new technologies for cancer and cardiac care in one province of Canada may not be generalizable to other contexts such as priority setting by regional health authorities or hospitals.

Although the elements of our model have been discussed by others,, , , , , the novelty lies in integrating these elements based on evidence from case studies, and the perspectives of decision makers.

The next step will be to harmonize this description of how groups do make priority setting decisions, with ethical accounts of how they should make such decisions. ,

Contributors: Peter Singer, the principal investigator, initiated the research, discussed core ideas, and participated in the collection, analysis, and interpretation of the data, and was primary author. Douglas Martin participated in design of the study, collected the data, was the primary data analyst, and participated in writing. Mita Giacomini consulted on the design, participated in data collection, analysis and interpretation, and writing of the paper. Laura Purdy contributed to analysis and interpretation of the data and writing of the paper.

Acknowledgements: We are grateful to Cancer Care Ontario and the Cardiac Care Network of Ontario, and to the members of their committees, for agreeing to participate in this research. We also thank Professor Bernard Dickens for providing a legal perspective on the issues in this article. Soren Holm, Steven Lewis, Martin McKneally, and Gilbert Sharpe provided helpful comments on an earlier version of the article.

Source of funding: Dr. Singer is supported by a Canadian Institutes of Health Research Investigator award. Dr. Giacomini is supported by a National Health Research Scholar award from Health Canada. This research project was funded by grants from the Medical Research Council of Canada (#MA-14675) and the Physicians' Services Incorporated Foundation of Ontario (#98-08).

Competing interests: None declared

REFERENCES

1.Daniels N, Sabin JE. Limits to Health Care: Fair Procedures, Democratic Deliberation and the Legitimacy Problem for Insurers. Philosophy and Public Affairs 1997; 26(4): 303-50
2.Singer PA. Resource allocation: Beyond evidence-based medicine and cost-effectiveness analysis. ACP Journal Club 1997; 127(3): A16-8.
3.Martin DK, Singer PA. Priority Setting and Health Technology Assessment: Beyond evidence based medicine and cost effectiveness analysis. In: Ham C, Coulter A, eds. Priorities in Health Care. Buckingham, UK: Open University Press, in press.
4.Newhouse JP. Medical care costs: How much welfare loss? Journal of Economic Perspectives 1992; 6(3): 3-21.
5.Yin RK. Case Study Research: Design and Methods. Thousand Oaks, CA: Sage Publications, Inc., 1994, p. 13.
6.Strauss AL, Corbin JM. Basics of qualitative research: Grounded theory procedures and techniques. Newbury Park, CA: Sage Publications, Inc., 1990.
7.Garland A (Acting Regional Director, Ontario Ministry of Health), letter to Hollenberg C (President & CEO, CANCER CARE ONTARIO), 30 July 1997.
8.Cardiac Care Network of Ontario Expert Review Panel on Intracoronary Stents and Abciximab. Final Report and Recommendations. 1998.
9.Foy R, So J, Rous E, Scarffe JH. Perspectives of commissioners and cancer specialists in prioritising new cancer drugs: impact of the evidence threshold. British Medical Journal 1999; 318: 456-9.
10.Hope T, Hicks N, Reynolds DJM, Crisp R, Griffiths S. Rationing and the health authority. British Medical Journal 1998; 317: 1067-9.
11.Green J. Commentary: Generalizability and validity in qualitative research. British Medical Journal 1999; 319: 420-1.
12.Klein R, Day P, Redmayne S. Managing Scarcity: Priority Setting and Rationing in the National Health Service. Buckingham, UK: Open University Press, 1996.
13.Klein R. Puzzling out priorities: Why we must acknowledge that rationing is a political process. British Medical Journal 1998; 317: 959.
14.Ham C. Priority setting in health care: learning from international experience. Health Policy 1997; 42: 49-66.
15.Ham C. Tragic choices in health care: lessons from the Child B case. British Medical Journal 1999; 319: 1258-61.
16.Holm S. Goodbye to the simple solutions: the second phase of priority setting in health care. British Medical Journal 1998; 317: 1000-2.
17.Daniels N, Sabin J. The ethics of accountability in managed care reform. Health Affairs 1998; 17(5): 50-64.
18.Daniels N, Sabin JE. Last chance therapies and managed care: Pluralism, fair procedures, and legitimacy. Hastings Center Report 1998; 28(2): 27-41.
19.Daniels N, Sabin JE. Limits to Health Care: Fair Procedures, Democratic Deliberation and the Legitimacy Problem for Insurers. Philosophy and Public Affairs 1997: 26(4): 303-50.