Using Nominal Group Technique to Identify Key Attributes of Oncology Treatments for a Discrete Choice Experiment

Abstract

Background. Responding to rising oncology therapy costs, multiple value frameworks are emerging. However, input from economists in their design and conceptualization has been limited, and no existing framework has been developed using preference weightings as legitimate indicators of value. This article outlines use of the nominal group technique to identify valued treatment attributes (such as treatment inconvenience) and contextual considerations (such as current life expectancy) to inform the design of a discrete choice experiment to develop a preference weighted value framework for future decision makers. Methods. Three focus groups were conducted in 2017 with cancer patients, oncology physicians, and nurses. Using the nominal group technique, participants identified and prioritized cancer therapy treatment and delivery attributes as well as contextual issues considered when choosing treatment options. Results. Focus groups with patients (n = 8), physicians (n = 6), and nurses (n = 10) identified 30 treatment attributes and contextual considerations. Therapy health gains was the first priority across all groups. Treatment burden/inconvenience to patients and their families and quality of evidence were prioritized treatment attributes alongside preferences for resource use and cost (to patients and society) attributes. The groups also demonstrated that contextual considerations when choosing treatment varied across the stakeholders. Patients prioritized existence of alternative treatments and oncologist/center reputation while nurses focused on administration harms, communication, and treatment innovation. The physicians did not prioritize any contextual issues in their top rankings. Conclusions. The study demonstrates that beyond health gains, there are treatment attributes and contextual considerations that are highly prioritized across stakeholder groups. These represent important candidates for inclusion in a discrete choice experiment seeking to provide weighted preferences for a value framework for oncology treatment that goes beyond health outcomes.

Rising pharmaceutical spending on oncology therapies in the United States has demonstrated a need for priority setting and value assessment (VA) of treatment options.^1,2 These new VA models move away from solely focusing on clinical benefits to also include wider “value” considerations.^3,4 Value-based pricing (VBP) seeks to achieve a fairer and needs-based provision of care, incorporating other requirements deliberated by payers and stakeholders. In recent years, there has been an emerging body of literature and frameworks looking to include VA under VBP.^5,6 The Professional Society for Health Economics and Outcomes Research recently recommended the need to develop understanding of what drives payer preferences and incorporate “novel,” non-health treatment characteristics into future frameworks and economic evaluations.⁶ By doing so they argue that we could achieve the most effective use of resources and establish a price control measure whereby drug prices reflect the value they bring to a patient. Thus, it may be possible to avoid “suboptimal resource allocation” whereby higher prices for medicines are established regardless of their benefits.^4,7

The United States (US) comprises a multi-payer health care system where payers (e.g., private insurance plans or public payers like Medicaid and Medicare) cover members’ service costs. Private insurance plans offer a choice of health care “products” (plans) based on extent of coverage and co-pay arrangements, catering to more preferences and needs.⁸ Consumers’ willingness-to-pay (WTP) and valuation of a health plan often align with their ability to pay.⁹ Unlike other multi-payer systems such as Japan or Germany, in the US no mandate for creating a set price achieved through negotiation or unilateral agreement among payers exists. Price negotiation is a key feature of single-payer systems such as the United Kingdom, which are financed through taxation and where the government has control of spending.

Forms of health technology assessment (HTA) and existing mechanisms to determine cost-effectiveness such as the quality-adjusted life year (QALY) are not employed universally in US formal decision making.¹⁰ In the United Kingdom, the National Health Service (NHS) set a £20,000 to £30,000 QALY price limit for health care services (although alternative assessments of the appropriate threshold exist¹¹). Thus, understanding how different payers’ value health care could be used to inform future VAs is important.

US Value Frameworks

There exists lack of agreement among the suggested VA frameworks as to which criteria define a treatment’s value, who should determine “value,” and whether frameworks should incorporate non-health treatment considerations (such as the inconvenience to patients and families), or consider the influence of contextual issues such as a patient’s current life expectancy prior to treatment. Only the recently amended Institute for Clinical and Economic Review’s (ICER) value framework has a formal voting policy roundtable process for including wider “benefits and disadvantages” of a treatment. Yet discussions on how these results can be incorporated into the final VBP calculations and who should determine the value are still ongoing and do not address the issue of contextual issues influencing value.¹² An “absence of any notion of sacrifice when it comes to defining weights for different attributes within any given framework” is not unique to dealing with the non-health attributes but affects all treatment attributes being included in value frameworks.¹³ Moreover, the context in which “value” is being determined, preferences of different payer groups, and potentially non–cost-effective considerations of society add further complexity and have the potential to influence attribute weighting.¹⁴ Traditional economic welfare theory stipulates that those often best placed to determine the value of a good or service are the consumers who wish to maximize their wellbeing (utility), and greater utility gain indicates higher value.^15,16 Capturing preferences when there is a lack of current real-world data through presenting trade-offs between utility gain from different characteristics (attributes) is dependent on the contextual framing of the task.

Contingent Valuation Methodology: Discrete Choice Experiments

When revealed preference (RP) data, showing real-life actions and consumption of existing market goods are unavailable, contingent valuation (CV) offers an alternative approach. CV is a stated preference (SP) approach presenting hypothetical market scenarios to individuals who are asked to imagine the loss or gain of a particular good/service and state their WTP for the gain or willingness to accept (WTA) the loss. These values are interpreted as measures of value for the good/service allowing value estimation of commodities that currently do not exist within the market.¹⁶ One increasing popular SP methodology used to “simulate” real-world market options are discrete choice experiments (DCE).¹⁷ DCEs are a preference elicitation approach that presents two or more alternative options to participants and asks them choose to between them.¹⁸ Due to the method’s theoretical foundation in behavior theory, DCEs are readily able to capture human behavior.¹⁹ Relatively new to health economics, DCEs have a firm methodological background in transport and environmental economics for capturing and measuring broad well-being utilities with preferences, thus providing both a mechanism for looking at potential benefits beyond QALYs to include broader aspects of well-being and a person’s health consistent with emerging VA as well as a means for testing how different contextual information may influence valuations.^20,21

DCEs are founded in random utility theory and examine respondent trade-offs between a good/service’s attributes in order to capture preferences.^22-24 This differs from other SP methods such as visual analogue scales, ranking/rating, standard gamble, and time-trade-off commonly used for cost-utility analysis to provide utility values of a good/service suitable for the construction of QALYs. Instead, DCEs move away from this traditional approach of a service/good as objects of utility and instead assumes utility as being derived from the attributes of the good/service (e.g., a treatment’s cost, length, and convenience).²⁵ Within economics, an underpinning assumption is that individuals make utility maximizing choices revealing “an underlying (latent) utility function.”^26(p145),27 Researchers can then capture the impact of attributes on the benefit derived from a good/service, estimating utilities and if a payment vehicle attribute (i.e., cost) is included, WTP or WTA.²⁸

DCEs typically have five stages:

1. Identify appropriate attributes

2. Define and assign attribute levels

3. Generate an experimental design

4. Administer the questionnaire (collect data)

5. Analyze choices

Attribute identification is a crucial stage as this ensures that the functional form of the utility function for the good/service can be determined.²⁹ Despite an increase in DCEs within health economics, the literature on generating attributes is frequently poorly reported.^30-32 In their examination of attribute development and construction, Coast and Horrocks,³¹ Coast and colleagues,³⁰ and Vass and colleagues³³ all refer to the potential for qualitative work to inform the process. One key advantage of qualitative research is that it allows researchers to develop a deeper understanding as to the contextual factors and internal decision-making processes that are informing choices, resulting in more reliable results. The most commonly cited qualitative methods for attribute identification are focus groups and interviews.³³

Maintaining the balance between describing a good/service adequately using a manageable number of attributes for the respondent is essential.^31,32 Distilling information into a controllable number of attributes is key to experimental design development as a DCE that is too cognitively burdensome could lead to respondents taking shortcuts or ignoring information.³⁴

Nominal group technique (NGT) has been highlighted as a means for attribute identification due to its structured approach and ranking methodology within a face-to-face focus group setting.³⁵ NGT is a consensus focus group methodology that differs from standard focus group methodology using open discussion throughout and instead is a structured four-stage process that provides a quick and clear methodology for capturing participant responses and getting them to prioritize their thoughts and perspectives both as a group and individually.^36,37 The key benefit of NGT is that it requires participants to prioritize attributes into a manageable list, thus addressing concerns of cognitively burdensome DCEs. Furthermore, the methodology has the distinct advantage over other qualitative consensus methods such as the “Delphi methodology” as it is quick in ensuring groups reach a consensus through face-to-face discussions and has a higher response rate success. This is particularly pertinent for studies with a short project completion timeframe.³⁸ In contrast, the Delphi methodology can take months to complete with multiple rounds of surveys being distributed to participants.³⁷

Study Objective

This study sought to inform the development of the first stage of an oncology VA framework incorporating valued oncology treatment attributes from multiple stakeholder groups and contextual considerations that could influence preferences. These will be used to design a DCE that quantifies preferences and trade-offs and goes beyond the existing cost-effectiveness approach focused on health measured using QALYs.

Methods

Nominal Group Technique

The focus groups consisted of a facilitator (CS) and a session moderator (AB). Sessions took up to 60 minutes. The four stages of this study’s NGT are outlined below.³⁷

1. Idea: Following facilitator introduction participants had 10 minutes to silently reflect and list all cancer treatments attributes they considered important.

2. Round Robin: One by one, participants stated attributes while the facilitator wrote them on a board. Then the moderator supplemented this list with additional attributes considered in existing value frameworks that had not been stated by the group. These were generated by a scoping review of VBP and value frameworks. Groups stated if any should be added to their collective list.

3. Clarification: Groups discussed each attribute, possible overlapping and confirmed a definition for each.

4. Ranking: This took two forms. First, silently the participants independently wrote a ranked attribute list in order of importance. Second, participants publicly voted to reach a consensus of the top six most important attributes. The focus groups were audio recorded and transcribed. Transcriptions were anonymized to protect respondent.

Recruitment

Participants were recruited from the Memorial Sloan Kettering Cancer Centre (MSKCC). Participants were considered eligible if they were English speaking, 18 to 70 years old, and either previous or existing MSKCC cancer patients or currently employed by MSKCC as oncology nurses or physicians. The participants were divided into groups according to their experiences (patient, nurse, or physician); thus, each of these three perspectives could be considered independently. Patients were recruited through an MSKCC patient support network. The chair of that network reached out to the membership and invited them to participate in this study. Nurses and physicians were recruited through a convenience sample of available clinicians working at MSKCC. In particular, physicians were trainees on rotation and therefore covered a number of specialties but were younger and less senior than average. Participants were compensated for their time through a $50 Amazon Gift Card. Ethical approval was granted by the University of Glasgow and MSKCC (December 2016).

Data Analysis

Participants’ individual attribute ranking scores were combined to produce a list of each focus group’s cumulative scores to take forward in the DCE.³⁹ The higher the score the lower the attribute was prioritized. Additionally, an attribute list from the public voting exercise across the three groups was collated. This allows for the differences between the group consensus and individual rankings to be examined. If there was overlap in the attributes names, the attributes were combined.

Qualitative analysis of the transcripts was undertaken using NVivo software package.⁴⁰ Due to the structured format of the focus groups, a deductive analysis approach was taken.⁴¹ That is, the total attribute list across the groups was used as a predetermined framework for the thematic analysis.⁴¹ Key terms used by participants to describe attributes were coded according to the attribute they were describing. This process identified themes and contextual considerations associated with the attributes, and also helped recognize interactions or themes across multiple attributes. Attributes were then grouped as either contextual considerations or attributes of the treatment option. Those attributes identified by the focus groups referring to health-related features of treatment outcomes were not the focus of the study and would be presented as part of QALYs within the DCE. These analyses were used to determine attributes (and their levels) for inclusion in the DCE and help design the contextual information presented to DCE participants.

Results

Study Sample

Twenty-four subjects participated in the focus groups held at MSKCC in February 2017: 8 previous or existing cancer patients, 10 oncology nurses, and 6 physicians working within oncology. The sociodemographic characteristics of the participants are shown in Table 1.

Table 1.

Participant Sociodemographic Characteristics

	Patients (n = 8), % (n)	Nurses (n = 10), % (n)	Physicians (n = 6)v	Total, % (n)
Sex
Male	37% (3)	10% (1)	33% (2)	25% (6)
Female	63% (5)	90% (9)	67% (4)	75% (18)
Age (years)
18–30	12.5% (1)	—	17% (1)	8% (2)
31–40	25% (2)	30% (3)	83% (5)	42% (10)
41–50	50% (4)	10% (1)	—	21% (5)
51–60	12.5% (1)	40% (4)	—	21% (5)
61–70	—	20% (2)	—	8% (2)

The sample was predominantly (75%) female. Both the patients and nurses groups were had a wide range of age groups while no physicians over 40 years old were recruited. Only two participants had prior focus group experience. Two patients had experience of caring for a cancer patient. No physicians had personal experiences of cancer while three nurses did. All participants were in full-time employment.

Attributes Identified

Thirty attributes were identified (Table 2). Five scoping review attributes were added. Nurses chose “any existing treatments” and “cost to population.” Physicians chose “fairness,”“disease severity,” and “can be used for more than 1 treatment.” Patients added none. As previously stated, treatment attributes that were health-related and would be considered part of a QALY were excluded. Participants often used different terms and phrases to describe similar concepts which have been grouped together (Table 2). Table 3 presents the top ranked non-health and resource use attributes in addition to contextual considerations raised.

Table 2.

Attributes and Contextual Considerations Identified Across Focus Groups

	Identified Criteria	Patients	Physicians	Nurses
Treatment attributes	Quality of evidence: Level of evidence/how well established the treatment is/how does the treatment work?	✓a	✓	✓
	Treatment inconvenience/burden: Burden on family and caregivers/lifestyle modifications/length of time/timeline of treatment	✓a	✓a	✓
Resource use and cost attributes	Costs/insurance coverage/financial toxicity/barriers	✓	✓a	✓
	Does it impact others consumption of resources	✓
	Societal costs and consequences/costs to population/fairness		✓a	✓
Contextual considerations	Alternative options/any existing treatments/psychological benefits hope/patient expectations	✓a	✓	✓a
	Disease severity		✓	✓
	Compassionate staff/communication	✓		✓a
	Do I have a voice in the treatment plan?	✓
	Can you offer/recommend any psychosocial support?	✓
	Reputation of oncologist/cancer center	✓a
	Aspects of prevention			✓
	Will it be helping in any research of future treatment	✓
	Can be used for more than 1 treatment		✓	✓a

^aDenotes group voted highest ranked attributes.

Table 3.

Top Ranked Contextual Considerations, Non-Health and Resource Use Attributes

		Ranking
Focus Group	Activity	1	2	3
		Patients	Individuala	How well established the treatment is (Scored: 43)	Reputation of the oncologist, cancer center, group (Scored: 46)	Lifestyle modifications (Scored: 53)
Groupa	How well established the treatment is (6 votes)		Reputation of the oncologist, cancer center, group (3 votes)	Alternative options (3 votes)
Physicians	Individuala	Burden and inconvenience to patients (Scored: 28)	Financial toxicity (Scored: 29)	Burden and inconvenience on family and caregivers (Scored: 40)
	Groupa	Financial toxicity (5 votes)	Burden and inconvenience to patients (4 votes)	Societal costs and consequences (2 votes)
Nurses	Individuala	Communication (Scored: 60)	Patient expectations (Scored: 63)	Barriers (Scored: 67)
	Groupa	Communication (6 votes)	Innovation (6 votes)	Patient expectations (4 votes)

^a“Individual” refers to the participants’ silent (private) ranking, while “group” denotes the public voting process.

Definitions by Groups: Treatment Attributes

Quality of Evidence

The patients prioritized availability of evidence, and stated that their top ranked attribute across both ranking exercises was “how well established the treatment is.” This was defined as the amount of data available to them and linked the attribute with a treatment’s efficacy as the more evidence available, the clearer their understanding about the true efficacy of a treatment.

The example of what I would think is at the time I was treated there was a treatment that was considered better, but Sloan Kettering didn’t want to use it because it wasn’t as well-established. Now it’s the standard of care for a long time. (Patient F)

Similarly, physicians reported that aspects of evidence quality were important to their decision making, describing how the “degree of uncertainty about the outcomes” of a treatment would affect their final decision. Nurses also valued quality of evidence, defining it as a result of staff working experience of a treatment, describing that having existing first-hand experience of a treatment’s effect on patients would provide them with a type of “data” and that the more “data” they had would influence their choice of a treatment option.

I was looking at that from the provider’s perspective. What is our experience with it, so we’re more comfortable talking to the patients about it. (Nurse H)

However, only the patients prioritized evidence quality in their group voting.

Treatment Inconvenience/Burden

The impact of the treatment on the lives of the patients and their families was a commonly prioritized theme across the groups, despite using different attribute names. For the physicians, what the patients termed “lifestyle modifications,” was referred to as two “burden and inconvenience” attributes; one concerning patients and the other the family and caretakers. “Burden and inconvenience on patients” resulting from treatments was highlighted as being separate from provider burden and from “burden on family members and caretakers” though very much connected with one another with one participant describing the relationship as similar to that of a car “driver” and “passenger.” This “burden and inconvenience on patients” was further described as encompassing other attributes listed by the group in NGT stages 1 and 2: “ease of administration,”“additional testing,” and “scan interval.”

The “burdens” of a treatment were prioritized by the patients and termed “lifestyle modifications.” When trying to define the attribute and why it was key to their choosing a treatment, patients categorized other attributes such as “location for treatment” and “disruption to family” as being components of “lifestyle modifications.” The participants associated need for patients to make changes to their daily lives with their family as part of “lifestyle modifications.”

And now the frequency of visits can be a barrier, if they have to come every week and they’re trying to work, you know. And maybe they don’t finish treatment. (Nurse H)

Resource Use/Cost Attributes

Highly ranked by the physicians as “financial toxicity” (defined as the financial cost and burden of the illness on the patient and their families) and “societal costs and consequences,” costs and resource use was discussed across groups.⁴²“Financial toxicity” was deemed inclusive of “out-of-pocket costs” and physicians discussed how these costs could be termed indirect and direct costs.

Would it be cleaner to do direct and indirect costs, of like, when you think of financial toxicity the direct costs are the sum that is on the bill due to the drug, right? And that you can talk about with the patient, and that’s standard regardless of the patient’s circumstances. And then there are the indirect costs—missing work, someone has to comp you to pay for parking and go through tolls, but they are actually two unique things, and one is independent of the patient and the other one is very patient- and circumstance-dependent. (Physician D)

Patients further discussed that resources (defined as “time from staff, time from family, cost of drugs, and cost of surgery”) could be categorized as two separate attributes: “cost/insurance coverage” (costs to the patient) and “from the health care system,” which was then redefined as part of the “does it impact on others by consuming an inordinate amount of resources” consumption of resources’ attribute.

My thought was just that there could be a lot of treatment plans out there, a lot of things that could be done, some which consume a lot of resources and I think you might be offered something that consumes a lot of resources from the hospital, from the insurance, and just the sense that every time you consume a resource, there’s a possibility that means there’s less resources to go for other people. (Patient D)

The issue of societal costs and “fairness” were also considered within this attribute as well as “disease rarity.” Physicians agreed that rarity was included yet would not be discussed with patients.

At least in my mind it’s separated from the things that you normally talk to about—or talk about a patient [ . . . ] It has—there is something about valuing a cancer treatment based on its rarity that is more of a societal issue than something you talk about with the patient. (Physician I)

Nurses defined “cost to the patients and family” as an aspect of their highly prioritized “barriers” attribute and separate from understanding the “big picture” and “where the dollars are going in health care,” which was termed “cost to population.” This latter attribute was discussed as a possible cost-effective preventative measure as early detection could potentially have positive implications.

It’s almost like not only early detection. It’s, you know—diet, lifestyle, sleep, stress management, to not get the cancer in the first place. That would be super effective per cost. (Nurse J)

Definitions by Groups: Contextual Considerations

As shown in Table 2, the top ranked contextual considerations across the groups were “alternative options,”“communication,”“patient expectations,”“innovation,” and “reputation of the oncologist, cancer center.” All focus groups stated the importance of knowing if alternative treatments were available. The patients listed it as a top ranked treatment attribute. For patients, knowing their “end of the line” was essential and for nurses understanding if treatments were “standard of care” or “investigational” (their definition for “any alternative options”) would help their work. This ability to communicate effectively with patients and manage patients hope and expectations was discussed by both physicians and nurses as a key treatment consideration. This sense of responsibility on staff to ensure that the patient had realistic expectations of the treatment was discussed as being linked to “communication,” another highly ranked attribute, defined as staff being able to support patients and families.

I think it’s more widely encompassed by communication, and that covers length of treatment, compliance, availability, you know, so it covers a range of other things that are all covered in ongoing communication with the patient and their family. (Nurse D)

The “innovation” of a treatment was a top three nurse ranked attribute who stated it would be a feature of a treatment they would consider as “innovation” which could provide more “targeted treatment” compared to existing treatments, thus fewer side-effects and better quality of life. Last, “reputation of the oncologist/cancer center” was highlighted by patients as key to treatment choice as this can be tied to an institute. Furthermore, patients highlighted that center choice could affect whether any “offer/provision of psychosocial support” was provided and whether they had “compassionate staff.”

Discussion

The findings of this study are twofold, highlighting both methodological strengths of NGT for DCEs and those aspects of treatments that are not health outcomes yet are highly valued by patients, nurses and physicians.

Methodologically, this study has highlighted how NGT, as a rapid and structured form of qualitative consensus methodology, has been used to show that non-health attributes and contextual considerations are important considerations for treatment choice beyond the health outcome and cost attributes usually included in a cost-per-QALY VA. Thorough analysis of the transcriptions providing the research team with definitions, specific terms, and key themes, NGT can aid the design of more realistic hypothetical choice tasks in a DCE through its quick and concise ranking approach. This is invaluable information that can help ensure that the scenarios that will be presented to DCE respondents are evidence based in real life and can help result in a survey with face validity. These identified attributes will now be taken forward into the design process of a US population-wide DCE designed to explore whether (and to what extent) these non-health attributes will be traded against health. The groups highlighted clear similarities and differences between the perspectives of the stakeholders. These differences in rankings will be tested further in the initial design processes of the DCE with different versions used in consultation about the design and phased piloting of scenarios and the contextual information with those who have experienced cancer and who currently work within the oncology field.

The results highlight that for US stakeholders cost to the individual is a key concern and possible treatment “barrier.” This is unsurprising given the US multi-payer health care system and existing evidence suggesting that ability to pay aligns with consumers’ valuations of a treatment.⁹ This demonstrates that in order to elicit preferences for non-health outcomes of treatment, the treatment costs presented in the DCE will either need to be disclosed or some statement about costing will be required to ensure participant engagement.

This study demonstrates that wider contextual issues are of importance to stakeholders. Attributes such as communication and reputation of the oncologist, cancer center/group, and patient expectations are not “treatment attributes” but rather “context” considerations that will require valuing in the future design of a DCE (the next stage of this study). The strength of influence these considerations have on preferences for treatments has yet to be determined and will be a key research question taken forward.

Limitations

Guidance on NGT sample size has not been clarified in the literature and this is not the first study to use a small number of focus groups.³⁹ The research team were aware that the restriction of project timeframe to conduct focus groups may have limited the breadth of attributes mentioned, which may have been brought up in subsequent focus groups. Therefore, the team has consulted a qualitative research methodology specialist (now a permanent member of the research team) to help with the further development of the DCE. Following her advice to ensure the validity of the resulting DCE survey, a think-aloud study with cancer patients has been included in the piloting stages of the DCE and will be reported in a subsequent publication. The research team would encourage that if it is not possible to conduct focus groups until reaching saturation in the results then extensive pretesting of the DCE is essential.

Conclusion

The importance of “treatment burden/inconvenience” and “quality of evidence” was evident across the groups and will be used as the treatment-specific attributes within the next stage DCE. These two attributes are already included in some of the US value frameworks such as the National Comprehensive Cancer Network’s evidence blocks, which includes “quality of evidence” as one of the five elements of value, and ICER, who recognizes that wider issues (such as burden on family) should be considered in their VA framework.^12,43 Yet the article also highlights that for those individuals working with oncology patients, the role of communication and management of patient expectations is also a key issue requiring further investigation. A key aim of the wider research project is to explore whether individuals would trade health for an improvement in these non-health outcomes and thus, informed by this study, the DCE will be able to help determine the validity of the incorporation of non-health outcomes into VBP decision making.