Priority setting for new technologies in medicine: qualitative case study

Abstract

Objective

To describe priority setting for new technologies in medicine.

Design

Qualitative study using case studies and grounded theory.

Setting

Two committees advising on priorities for new technologies in cancer and cardiac care in Ontario, Canada.

Participants

The two committees and their 26 members.

Main outcome measures

Accounts of priority setting decision making gathered by reviewing documents, interviewing members, and observing meetings.

Results

Six interrelated domains were identified for priority setting for new technologies in medicine: the institutions in which the decision are made, the people who make the decisions, the factors they consider, the reasons for the decisions, the process of decision making, and the appeals mechanism for challenging the decisions.

Conclusion

These domains constitute a model of priority setting for new technologies in medicine. The next step will be to harmonise this description of how priority setting decisions are made with ethical accounts of how they should be made.

Introduction

Because demand for health care exceeds the supply of resources allocated to finance it, setting priorities is a problem for every healthcare system in the world. But how should we set priorities within health systems?

Two key issues lie at the heart of setting priorities—legitimacy (under what conditions should authority over priority setting be placed in the hands of a particular organisation, group, or person?) and fairness (when does a patient or clinician have sufficient reason to accept as fair particular priority setting decisions?).¹

At present, decision makers and the public have difficulty determining whether particular priority setting decisions are legitimate and fair. A first step in deciding what should be done to make such decisions legitimate and fair, is to understand how groups make these decisions.²,³

Since innovation is the primary driver of escalating healthcare costs,⁴ priority issues are particularly acute for new technologies. We conducted a study to develop a model describing priority setting in a specific context—new technologies in cancer and cardiac care.

Participants and methods

We used qualitative methods of case studies and grounded theory⁵,⁶ to study two cases: the Cancer Care Ontario policy advisory committee and the Cardiac Care Network of Ontario expert panel on intracoronary stents and abciximab (a glycoprotein IIb/IIIa inhibitor). The Cancer Care Ontario policy advisory committee “manage[s] the selection and introduction of all new drugs within the funds provided” (letter from A Garland, acting regional director, Ontario Ministry of Health, to president of Cancer Care Ontario, 30 July 1997). The Cardiac Care Network of Ontario expert panel on intracoronary stents and abciximab was mandated to “review current literature and practice . . . and recommend, where possible, a cost-effective, multi-year plan for stent volumes and use of Abciximab that supports the principles of quality of care, access and affordability.”⁷

Sampling and sample size

We interviewed 21 of the 26 committee members (11 of 15 from Cancer Care Ontario and 10 of 11 from the Cardiac Care Network of Ontario). The cancer committee included three lay members (one of whom was a patient), a government representative, a pharmacist, a nurse, two administrators, and seven oncologists. The cardiac committee included one lay member (a patient), a government representative, an administrator, a health economist, and seven cardiologists or cardiac surgeons. All meetings of both committees were observed for at least 12 months from the formation of each committee in 1997 to December 1998. At this point, the analysis was saturated—that is, no new major domains emerged.

Data collection

We reviewed information about the two organisations, the written mandate of both committees, committee minutes, correspondence to committee members, and committee reports. Semistructured interviews were conducted by a single interviewer (DKM), either in person or over the telephone, and tape recorded and transcribed. The interviewer asked respondents to describe their role on the committee and evaluate their effectiveness; describe the committee process; indicate whether the process was fair; and indicate whether the decisions were fair. We also observed, tape recorded, and transcribed the meetings of both committees.

Analysis of data

We analysed the data in three steps. Firstly, using open coding, we identified passages of text that related to a theme or idea and then grouped similar concepts into conceptual categories (such as benefit). Secondly, using axial coding, we further developed the conceptual categories and compared them with each other (the six domains of the model). Thirdly, using selective coding, we developed a model by relating the domains to a central theme and to each other (the metaphor of a gem). The analysis was conducted simultaneously with data collection.

The trustworthiness of our findings was enhanced in three ways. Firstly, two investigators in addition to the primary analyst coded the raw data from one meeting to ensure the authenticity of the coding scheme; the final coding scheme was developed by consensus and used for the analysis. Secondly, two investigators familiar with all the primary data developed the interpretation. Thirdly, a draft of this paper was endorsed by members of both committees in a “member check.”

Research ethics

This study was approved by the committee on use of human subjects of the University of Toronto. Each committee member interviewed provided consent.

Results

We identified six interrelated domains of priority setting for new technologies in medicine: the institutions in which the decisions are made, the people who make the decisions, the factors they consider, the reasons for the decisions, the process of decision making, and the appeals mechanism for challenging the decisions.

Institutions

Priority setting in both committees was established within a legitimate organisational context. Both organisations were created by the Ministry of Health to advise the ministry. Although the mandate of both committees included priority setting, they struggled with this. The cardiac committee debated about the distinction between making recommendations for clinical practice and for funding priorities. The cancer committee decided to advocate increased funding if it found itself denying an effective treatment to patients because of funding limits.

People

A key element of fairness described by committee members was that multiple stakeholder perspectives were represented. A difference emerged between the two committees with respect to the participation of lay committee members. The three lay members of the cancer committee were more satisfied with their participation than was the lone lay member of the cardiac committee. One of the cancer committee's lay members said:

I think on access issues I've been effective . . . I'd say that my frustrations have been fewer than I thought they would be at the start . . . it could be that just having community reps with this perspective sitting on that committee makes them have that awareness . . . So, I won't say I've been personally totally successful, but I think the process has been more successful than not in fulfilling what I think is my role.

The lay member on the cardiac committee questioned his effectiveness:

I'm a businessperson, and to walk into a medical panel where they're talking a great deal of medical topics that I knew very little about, it's very hard for me to have the confidence to question what they were doing. You try to some extent but, if there was a matter of conflict it would be very easy for me to defer to their expertise . . . I think if there were two of us that might have helped . . . So one doesn't feel quite so overwhelmed by the rest of the panel.

This comparison suggests that a critical mass of public participation is required.

Factors

The individual factors that shaped the decisions of both committees were benefit, evidence, harm, cost, cost effectiveness, and pattern of death. Benefit had the greatest role in the deliberations.

Evidence represented the degree of certainty with which the benefit was known. Sometimes the committee had to balance benefit against evidence. For instance, the cardiac committee compared high quality evidence of a small benefit from stents in patients with “favourable coronary artery lesions” with lower quality evidence of a potentially large benefit in patients with “unfavourable” lesions. When there was evidence of both benefits and harm, participants felt that individual patients, with advice from their physicians, were best suited to decide.

The total cost of providing a treatment to a group of patients led to a discussion about access and equity. For example, the cancer committee decided that patients should be treated equally regardless of whether they belonged to a relatively large or small group of patients.

Although formal data on cost effectiveness were available for only a minority of the drugs under consideration, these data were used to support decisions that were primarily made on grounds of benefit and evidence. A cardiac committee member said:

Without having hard numbers on the cost effectiveness, we did use the concept, at least, of cost effectiveness . . . in deciding, for example, not to recommend funding for the use of stenting in areas where we thought cost effectiveness would be unattractive.

The patterns of death of patients with cancer compared with cardiac disease influenced the deliberations of the committees. For example, the committee recognised that deterioration and death in patients with metastatic colorectal cancer could be slowed but not reversed. By contrast, the use of stents in patients with cardiogenic shock might not only prevent death but potentially return patients to their earlier state of health. The possibility of “saving” patients, even if remote, tended to influence the allocation of resources.

Reasons

The reasons underlying both committees' decisions did not rest on individual factors such as those described above. Rather, both committees made decisions based on clusters of factors. Moreover, actual decision making was more complex than simply one drug and its attendant cluster of factors. Some decisions involved clusters of drugs, each with their own cluster of factors, for a single disease. Other decisions involved clusters of factors, clusters of drugs, and clusters of diseases. The box gives two examples from the cancer committee's deliberations. The first example shows how clusters of factors were used to develop a reason. The second example shows how clusters of factors, clusters of drugs, and clusters of diseases converge when reasoning through more complex decision making. This reasoning process enabled the committee members to periodically review previous decisions to evaluate the consistency of their reasoning.

Examples of decision clusters

Clusters of factors for one drug in one disease: raltitrexed for colorectal cancer

In randomised comparisons with the standard treatment (fluorouracil), raltitrexed showed equivalent benefit (on survival and response rate). The toxicity of raltitrexed was thought to be different from that of fluorouracil but not worse. Although raltitrexed is more convenient to give, it is about 200 times more expensive. Therefore, the panel reasoned that because raltitrexed is no better than the standard treatment in terms of benefit or harm, and much more expensive, it should not be funded. At a subsequent meeting, the panel decided to recommend funding for raltitrexed for patients with excessive fluorouracil toxicity or for patients who lived beyond a specific distance from a treatment centre.

Clusters of factors for two drugs in two diseases: pamidronate and clodronate for myeloma and breast cancer

On the basis of evidence of equal quality, pamidronate has been shown to give better symptom relief and prevention of complications (decreased bone pain and decreased number of fractures) than clodronate. In addition, one study showed a survival advantage with pamidronate. Pamidronate is much more costly than clodronate. The population of patients with myeloma is small, therefore the overall cost was expected be modest; however, a lack of alternatives for those patients made the need great. Some, but not all, hospitals were providing pamidronate, so province-wide funding was required to ensure equal access for all myeloma patients. The panel decided that pamidronate but not clodronate should be funded because (in the context of a small population of patients with great need) it provided enhanced survival rates and a better quality of life.

After making the decision on treatment of myeloma, the committee considered the same two drugs for breast cancer. The cluster of factors attending pamidronate and clodronate in treatment of breast cancer was similar to those for myeloma. However, the evidence for oral clodronate in breast cancer was much stronger than for myeloma, and many more patients had breast cancer than myeloma, which would make the overall cost to the programme larger. The panel decided to fund intravenous clodronate for patients who could not tolerate oral clodronate (for which alternate funding mechanisms existed) and that they would fund pamidronate only for patients who had tried and could not tolerate clodronate.

Process

A key element of the process was transparency of decisions within the committee, although the reasoning was not widely publicised outside the committee. Participants identified other aspects of committee process that contributed to fairness: acknowledging conflicts of interest, providing the opportunity for everyone to express views, ensuring that all committee members understand the deliberations, maintaining honesty, building consensus, ensuring availability of external expert consultation, ensuring appropriate agenda setting, maintaining effective chairing, and ensuring timeliness in making funding decisions to get effective new technologies to patients.

Appeals

Participants emphasised that an appeals mechanism was a key element of fairness. In response to challenges of some decisions, the cancer committee decided it should revisit decisions if new evidence or new arguments became available.

Discussion

Although our findings may not be generalisable, we developed a model of priority setting in new technologies in medicine. Our model can be likened to a gemstone. Six domains—institutions, people, process, factors, reasons, and appeals—form the facets. Each facet may be more or less perfect (legitimate or fair) and contributes to the perfection (legitimacy or fairness) of the whole.

However, just because a group makes priority setting decisions in a particular way does not make it “right.” Our goal was to describe how these groups made priority setting decisions, not to prescribe how they should make them. We did not seek to justify what makes a particular domain more or less fair.

The closest analogues to our study are the work of Foy et al⁸ and Hope et al.⁹ Foy et al found that priority setting decisions regarding new cancer drugs were based on “evidence thresholds”—cut-off points determined from information on effectiveness. Hope et al described the use of evidence of effectiveness, equity, and patient choice in a health authority's priority setting decisions. Our study describes these factors in priority setting decisions, but also places the factors in a model that includes institutions, people, process, reasons, and appeals. Our model developed in the context of new technologies for cancer and cardiac care in one province of Canada may not be applicable to other contexts such as priority setting by regional health authorities or hospitals.¹⁰

Although the elements of our model have been discussed by others,^11–17 the novelty lies in integrating these elements on the basis of evidence from case studies and the perspectives of decision makers. The next step will be to harmonise this description of how groups make priority setting decisions with ethical accounts of how they should make such decisions.¹

What is already known on this topic

Limited resources for health care mean that setting priorities is essential in most health systems

Although the individual elements of priority setting in health care have been described, they have not been collected into an integrated model

What this study adds

A model of priority setting for new technologies in cancer and cardiac care in Ontario was produced

The model had six domains: the institutions in which the decision are made, the people who make the decisions, the factors they consider, the reasons for the decisions, the process of decision making, and the appeals mechanism for challenging the decisions

Editorial by Daniels