Abstract
BACKGROUND:
The landscape for hemophilia A prophylaxis is rapidly expanding from factor VIII replacement therapy to include novel treatments such as nonfactor replacement therapies that may enhance coagulation (e.g., emicizumab) or inhibit anticoagulant pathways (e.g., fitusiran and concizumab). For payers, this expansion presents challenges in balancing well-established treatments with new options that cost more and have lesser known real-world safety and efficacy.
OBJECTIVE:
To evaluate likely coverage practices for hemophilia A prophylaxis therapies among U.S. payers given evolving real-world data on safety and efficacy.
METHODS:
A 3-round modified Delphi process was conducted with representatives of U.S. commercial health plans who had considerable expertise in managing populations of patients with hemophilia. Round 1 consisted of an online questionnaire; round 2 involved an online discussion about the aggregated results from round 1; and round 3 allowed participants to revise their responses from round 1 based on insights gained during round 2. Questions elicited ratings, rankings, and estimates on access restrictions based on given safety and efficacy information for hemophilia A prophylaxis therapies. Consensus was reached if ≥ 74% of panelists (14 of 19) were within 1 SD of the median group estimate during round 3.
RESULTS:
19 Payers participated in the research. Among them, 94% dealt with commercial insurance, 94% with Medicare, and 81% with Medicaid; 79% had spent ≥ 5 years in their current role. Panelists reported limited access restrictions on hemophilia A prophylaxis therapies; the most common restrictions were prior authorization (n = 16, 84%) and quantity level limits (n = 13, 67%). Tiering and step therapy were reported by 7 respondents (39%). Respondents agreed that there was an 80% median likelihood that ≥ 9 additional patients with any safety event (e.g., thrombotic event, death) per year would trigger access restrictions, with the median likelihood of restrictions increasing to 95% for another ≥ 10 patients with safety events per year. Respondents also agreed that > 5 thrombotic events requiring treatment per patient per year would have a 98% median likelihood of leading to access restrictions and that ≥ 5 years of real-world safety and efficacy data would be highly likely (95% median likelihood) to affect coverage decisions. Noncoverage was highly unlikely (ranked fifth or sixth of 6 by 14 respondents), as was no restriction-coverage parity (ranked sixth of 6 by 10 respondents). All else being equal, cost continues to affect access policies, with respondents agreeing that a 13%-30% difference in net cost may lead to preferred formulary treatment for a drug with superior efficacy and noninferior safety, inferior efficacy and noninferior safety, or noninferior efficacy and inferior safety.
CONCLUSIONS:
Payers prefer treatments with well-understood efficacy, safety, and cost over newer treatments with uncertain long-term effects. Relatively unrestricted access to legacy and new hemophilia A prophylaxis will likely continue unless additional real-world safety concerns or major cost differences emerge.
What is already known about this subject
Novel treatment options for hemophilia A are rapidly emerging, including gene therapy and nonfactor replacement options.
Little is known about how payers will change their coverage approaches in light of these novel therapies and how they interpret emerging evidence in the context of formulary decisionmaking practices.
What this study adds
To our knowledge, this study is the first to assess systematically how payers interpret and use real-world evidence on cost, safety, and efficacy to guide coverage decisions in a rapidly evolving treatment landscape for rare diseases such as hemophilia A.
In lieu of empirical evidence, the use of the Delphi method to elicit payer assessments of comparative evidence on cost, efficacy, and safety of various treatment options serves as a novel way to derive consensus around coverage practices and strategies.
Hemophilia is a genetic bleeding disorder that prevents the blood from clotting normally and can result in uncontrolled bleeding in various areas of the body. Hemophilia is considered a rare disease; estimates indicate it affects approximately 20,000 people in the United States.1 Hemophilia A (a quantitative lack of clotting factor VIII [FVIII]) is more prevalent than hemophilia B (a quantitative lack of clotting factor IX), and approximately 50% of patients with hemophilia A are afflicted with a severe form of the disease (level of FVIII < 1%) that is linked to increased risk of spontaneous bleeding episodes.2 There is currently no cure for hemophilia A, except for a liver transplant. Most treatments available are focused on preventing serious bleeding events in patients, which can lead to permanent joint damage, long-term complications, and death.3
The current standard treatment for patients with hemophilia A who have a significant bleeding phenotype is FVIII prophylaxis to prevent bleeding events, which requires intravenous administrations 2 or 3 times per week, depending on the half-life of the product. FVIII can also be used on demand for treating breakthrough bleeding events.4 Approximately 30% of patients with hemophilia A who regularly take prophylaxis develop inhibitors to the exogenous FVIII replacement therapy; these inhibitors prevent the treatment from working, resulting in more severe and frequent bleeding.5 FVIII replacement therapies, which were first used as prophylaxis in 1992, are therapies with either a standard half-life (mean 8 to 12 hours) or an extended half-life (mean 14 to 19 hours).6,7
Although FVIII remains the standard of care, with a long history of safe and effective use among the hemophilia A population, the treatment landscape is rapidly expanding to include novel treatments such as nonfactor treatments and gene therapy.8,9 Emicizumab (Hemlibra) was the first new, nonfactor product approved (in 2017) for the prevention of bleeding events in patients with hemophilia A without inhibitors. Unlike FVIII therapies that are administered intravenously, emicizumab is administered through subcutaneous injection either once weekly, once biweekly, or once every 4 weeks.10
Short-term clinical evidence suggests that emicizumab may offer potentially better efficacy and more convenience than currently available FVIII treatments.11 Yet, despite numerous ongoing trials, emicizumab’s long-term safety and efficacy are unknown; in comparison, those of FVIII have been well established.12-15 Given that currently no head-to-head trials are comparing FVIII replacement therapy and emicizumab, definitive conclusions cannot be made regarding the benefits of one treatment over the other.
For U.S. payers, this new treatment landscape presents certain challenges. First, because hemophilia A is a rare disease, payers may be hesitant to establish access restrictions or set preferential treatment policies for one product over another. Further, given the lack of evidence around longterm safety outcomes and the short real-world track record of new treatments compared to those of known, established treatment options such as FVIII, payers must weigh incomplete, but emerging, real-world data during their decision making. In addition, off-label combination use and potential for future genetic interventions may further disrupt the treatment landscape, for example, by limiting the risk of a patient developing inhibitors,16,17 presenting even greater challenges for payers. In order to establish effective current and future market strategies in the face of this uncertainty, an understanding of payer perspectives is critical with regard to how potential changes in available evidence affect their decision making.
Methods
STUDY DESIGN
In order to evaluate areas of consensus regarding how U.S. commercial health plans are likely to manage prophylaxis therapies for patients with hemophilia A but without inhibitors, a modified 3-round Delphi process was conducted between June and October 2020. The Delphi method is a structured and scientifically rigorous approach for eliciting expert opinions and reducing variance in responses through an iterative process.18 This approach provides insight on how payers rate and rank information on safety and efficacy, and how their responses are affected by the ratings and rankings given by their colleagues. Given that treatments for hemophilia A have until recently been largely limited to a single class, the Delphi approach was best suited for gathering insights in the face of uncertainty regarding the evolving treatment landscape.19,20
This study used a modified Delphi process informed by a targeted literature review and was conducted entirely online, building on the RAND/UCLA Appropriateness Method.21-23 The modified Delphi process comprised 3 rounds: (1) completion of a web-based questionnaire; (2) participation in an online discussion forum; and (3) submission of final responses to the questionnaire from round 1. Round 1 of the Delphi process served to establish baseline estimates of the payers’ assessments of the impact of information about safety, efficacy, and costs on access policies. In order to avoid oversaturation and participant fatigue, in round 2 the traditional in-person discussion was replaced with an online discussion forum that allowed participants to comment on the aggregated findings from round 1. The online discussion forum in round 2 presented aggregate group responses from round 1 along with key areas of agreement and disagreement among the initial responses, and it provided follow-up questions as needed on a password-protected web page. Participants were able to either comment directly on the materials provided or respond to another participant’s comments. The study team was able to tie responses to individual participants, but participants were blinded to each other’s identities to avoid a biased discussion.
Last, in round 3, the original questionnaire was administered to all panel participants, along with information about their round 1 responses, in order to discern whether the group discussion and aggregated results from round 2 altered their initial round 1 assessments. This study was determined to be exempt from institutional review board oversight by the Advarra Institutional Review Board.
PANELIST SELECTION
Study participants were recruited for the Delphi panel through expert recommendations and the PRECISIONvalue Market Insights Advisory Network,24 a proprietary database comprising more than 2,000 managed care professionals, developed and owned by Precision Medicine Group, LLC, the parent company of the authors. Payers were eligible to participate if they (a) were current employees of or consultants to U.S. commercial health plans, (b) were directly involved in making drug formulary or coverage policy decisions related to hemophilia, (c) held a title or role equivalent to medical director or pharmacy director, and (d) were willing and able to provide verbal informed consent.
All panelists completed an initial survey to describe their demographics, training, job title, years of experience, experience with hemophilia A, and health plan size. Participants were compensated $300 for each round they completed, totaling $900 for completion of all study rounds.
QUESTIONNAIRE
The study team identified, through a targeted literature review, key areas of uncertainty around current and future coverage practices for hemophilia treatment that are facing U.S. commercial health plans. Then, they developed the Delphi questionnaire under the guidance of 2 scientific advisors: a clinical expert in hemophilia and related disorders, and an expert on budgetary and treatment-related questions in the management of patient populations with hemophilia.
The final questionnaire contained 17 questions designed to elicit ratings, rankings, and uncertainty estimates around the effects of new data related to safety, efficacy, and cost of treatments on coverage policies for patients with hemophilia A without inhibitors. Among the 17 questions were 9 likelihood-related questions, 6 parameter estimation–related questions, and 2 questions related to ranking. For likelihood questions, panelists were asked to estimate, on a scale of 0% (highly unlikely) to 100% (highly likely), the likelihood of a particular response occurring in the presence of new data, as suggested by the RAND/UCLA Appropriateness Method User’s Manual.21
Wherever necessary, definitions of key concepts (e.g., safety events) were provided in order to ensure that the questionnaire would be interpreted consistently. For parameter estimation questions, panelists were asked to provide threshold estimates (e.g., for costs or the frequency of safety events) that would trigger changes in their coverage decision making (on a given scale). Last, for ranking questions, panelists were asked to rank the relative likelihood of specific policies given information on safety, efficacy, and cost. For all ranking questions, panelists were provided with a fixed number of options to choose from.
The study team administered the introductory survey and questionnaire using Welphi, an online platform specializing in Delphi processes.25
STATISTICAL ANALYSIS
Descriptive analyses were conducted in order to summarize panel members’ relevant demographic characteristics (e.g., age, sex), information pertaining to their years of experience in the payer setting, and information on current coverage policies for hemophilia A treatments in their health plans (Table 1).
TABLE 1.
Respondent Demographics (N = 19)a
| Sex | |
|---|---|
| Male | 15 (79) |
| Female | 4 (21) |
| Education level | |
| Master’s degree | 3 (16) |
| Doctorate | 5 (26) |
| Medical degree (MD, OD) | 10 (53) |
| No response | 1 (5) |
| Role | |
| Pharmacy director | 8 (42) |
| Medical director | 8 (42) |
| Other | 3 (16) |
| Time in role (years) | |
| 3-5 | 2 (11) |
| 6-10 | 5 (26) |
| ≥ 11 | 10 (53) |
| No response | 2 (11) |
| Coverage channelb | |
| Commercial | 15 (94) |
| Medicare | 15 (94) |
| Medicaid | 13 (81) |
| No. of covered lives in health planc | |
| 1,000,000 | 1 (7) |
| 1,000,000-4,999,999 | 4 (27) |
| 5,000,000-7,499,999 | 4 (27) |
| 7,500,000,-10,000,000 | 2 (13) |
| > 10,000,000 | 4 (27) |
aData are n (%).
bResponse options were not mutually exclusive; 16 respondents answered the question.
c15 respondents answered the question.
MD = doctor of medicine; OD = doctor of optometry.
To determine whether consensus was reached during the Delphi process, the study team considered various consensus definitions and thresholds identified in a recent systematic literature review.26 Congruent with the review, for questions with likelihood thresholds, responses of 0%-25% likelihood were considered “highly unlikely,” whereas responses of 75%-100% likelihood were considered “highly likely.” Furthermore, consensus was considered to be present if at least 74% of panelists (i.e., 14 of 19 panelists) were within 1 SD of the median group estimate during round 3. Because many of the questions in this study aimed to elicit decision-making threshold values, rather than likelihoods of future trends, the study team deemed clustering of participant responses around a measure of centrality (e.g., SD) as the most appropriate definition of expert consensus.
Results
PARTICIPANTS
A total of 74 payer experts were invited to participate in the study. Of these, 23 (31%) agreed to participate, 20 (27%) completed round 1, and 19 (26%) completed all 3 rounds. Respondents completing all 3 rounds were predominantly male (n = 15, 79%), had a medical degree (n = 10, 53%), served as either a pharmacy director or medical director at a health plan (n = 16, 84%), had more than 10 years’ experience in their role (n = 10, 53%) and represented commercial (n = 15, 94%) and/or Medicare health plans (n = 15, 94%). A detailed breakdown of panelists’ demographic characteristics and background expertise is presented in Table 1.
CURRENT MANAGEMENT PRACTICES
Almost all of the 19 respondents who completed all 3 rounds reported wide coverage of the treatments most commonly prescribed for patients with hemophilia A without inhibitors. These treatments included FVIII therapies such as antihemophilic factor (recombinant) (Advate) and antihemophilic factor (recombinant), PEGylated (Adynovate), and the nonfactor therapy emicizumab. Respondents characterized their health plan’s management of hemophilia A prophylaxis therapies as either open (n = 8, 42%) or moderate (n = 11, 58%), meaning some modest level of management, such as formulary tiers and prior authorizations, exists for the therapeutic area. The most commonly cited management tactic was prior authorization (n = 16, 84%; Table 2).
TABLE 2.
Management Practices Respondents’ Health Plans Use to Manage Hemophilia A Prophylaxis Treatments
| Management Practice | Responses, n (%)a |
|---|---|
| Level of engagement the health plan had in managing the hemophilia A prophylaxis treatment categoryb | |
| Moderate (e.g., products on different formulary tiers, prior authorization required) | 11 (58) |
| Open (e.g., all products available with minimal requirements, no prior authorization) | 8 (42) |
| Hemophilia treatment dispensing network and HTC pharmaciesb | |
| Limited to a select group of specialty pharmacies/HTCs (≥3) | 10 (53) |
| Only allows dispensing from 1 or 2 specialty pharmacies/HTCs | 4 (21) |
| Open to a broad set of specialty pharmacies/HTCs; allows buy and bill | 5 (26) |
| Currently covered products on health plan’s formulary (regardless of possible restrictions such as step therapy or prior approval) for the prevention of bleeding events in patients with hemophilia A (Please check all that apply) | |
| Advate (antihemophilic factor VIII [recombinant], plasma/albumin-free method)c | 18 (100) |
| Adynovate (antihemophilic factor VIII [recombinant], PEGylated)b | 19 (100) |
| Afstyla (antihemophilic factor [recombinant])c | 16 (89) |
| Alphanate (antihemophilic factor [human])c | 17 (94) |
| Eloctate (antihemophilic factor [recombinant])b | 18 (95) |
| Esperoct (turoctocog alfa pegol)c | 14 (78) |
| Helixate (antihemophilic factor [recombinant])d | 15 (83) |
| Hemlibra (emicizumab-kxwh)b | 19 (100) |
| Hemofil (antihemophilic factor [human])c | 18 (100) |
| Jivi (antihemophilic factor [recombinant])b | 16 (84) |
| Kogenate (antihemophilic factor [recombinant])b | 17 (89) |
| Kovaltry (antihemophilic factor [recombinant])c | 17 (94) |
| Novo8 (antihemophilic factor [recombinant])b | 18 (95) |
| Nuwiq (antihemophilic factor [recombinant])b | 17 (89) |
| Recombinate (antihemophilic factor [recombinant])b | 19 (100) |
| Xyntha (antihemophilic factor [recombinant])b | 18 (95) |
| Management tactics currently employed in pharmacy or medical benefits by health plan in managing hemophilia A prophylaxis therapeutic area (Please check all that apply)c | |
| Placing products on different tiers | 7 (39) |
| Step therapy | 7 (39) |
| Prior authorizations | 16 (84) |
| Quantity limits | 12 (67) |
| Exclusions | 4 (22) |
| Medical policy | 14 (78) |
| Assay management | 10 (56) |
| Require use of HTC | 7 (39) |
| Require billing through pharmacy benefit | 9 (50) |
| Require billing through medical benefit | 7 (39) |
| Othere | 1 (9) |
aPercentages were calculated on the basis of the number of participants who responded to a particular item, as set with each item in column 1.
bN = 19.
cN = 18.
dN = 15.
eN = 11.
HTC = hemophilia treatment center.
EFFECT OF SAFETY AND EFFICACY DATA ON COVERAGE PRACTICES
Respondents were presented with a number of scenarios related to possible new efficacy and safety data and asked how they would make coverage decisions on the basis of those data. Respondents were asked to report the likelihood that they would restrict access to a treatment given information on worsening efficacy (e.g., large number of breakthrough bleeding events) and safety (e.g., thrombotic events, deaths, or any safety event) per patient per year or per number of additional patients per year.27 Further, likelihood questions were asked in order to gather additional details around how safety and efficacy data may affect coverage decisions.
Across each safety/efficacy scenario, the likelihood of access restrictions being imposed increased; respondents reached consensus that 9 or 10 additional patients with any safety event per year equated to an 80% median likelihood (MLH) (SD 31%) of resulting in new access restrictions for a hemophilia A treatment, whereas more than 10 additional patients with any safety event per year equated to a 95% MLH (SD 32%) of resulting in additional access restrictions for a hemophilia A prophylaxis therapy. When asked about specific types of safety events, respondents agreed that more than 5 additional deaths per 1,000 cases per year would have a 75% MLH (SD 32%) of triggering access restrictions, whereas more than 5 breakthrough bleeds requiring treatment per patient per year would have an 80% MLH (SD 31%) of causing a change in coverage policies. Similarly, respondents agreed that 5 thrombotic events requiring treatment per patient per year would be highly likely (MLH 85%, SD 32%) to trigger an access restriction for a particular medication, and the MLH increased to 98% (SD 32%) for more than 5 thrombotic events per patient per year.
EFFECT OF REAL-WORLD EVIDENCE ON COVERAGE PRACTICES
Information from clinical trials tends to drive access decisions early in the life of a new therapy.28 Over time, however, real-world data can build a clearer picture of how a drug performs across heterogeneous patients in various treatment settings.29,30 Respondents were asked to evaluate a variety of real-world data scenarios and reached consensus that 4 or more years of real-world data would be highly likely (MLH 80%, SD 30%) to be used in coverage decisions that may either restrict or expand access for one product over another.
When participants were offered real-world data scenarios, 2 themes emerged from their responses (Table 3). First, respondents agreed that new, emerging “real-world” safety data showing an increase in the number of deaths related to a treatment would be highly likely (MLH 81%, SD 30%) to trigger a formulary restriction for that treatment. Similarly, respondents agreed that new, emerging “real-world” efficacy data showing a decrease in the use of rescue medications to control breakthrough bleeding related to a drug would be highly likely (MLH 75%, SD 32%) to trigger a formulary restriction for the therapy without the accompanying real-world data.
TABLE 3.
Detailed Delphi Questionnaire and Participant Responses (N = 19)
| Topic/Question | Likelihood/Parameter Statements Rated | Round 1 | Round 3 | Within 1 SD of Median in Round 3, n (%) | Consensus Reacheda | ||
|---|---|---|---|---|---|---|---|
| Median | SD | Median | SD | ||||
| No. of additional deaths per 1,000 cases per year | |||||||
| In determining access policies for medications to treat hemophilia A without inhibitors, how likely is it that you will restrict access (nonpreferred tier, step therapy, etc.) to drug A for prophylaxis compared to drug B based on the following: | 1 | 20 | 28 | 20 | 28 | 12 (63) | No |
| 2 | 25 | 31 | 25 | 28 | 13 (68) | No | |
| 3 | 23 | 33 | 40 | 30 | 11 (58) | No | |
| 4 | 32 | 35 | 40 | 32 | 12 (63) | No | |
| 5 | 33 | 36 | 60 | 31 | 11 (58) | No | |
| > 5 | 45 | 40 | 75 | 32 | 14 (73) | Yes | |
| In determining access policies for medications to treat hemophilia A without inhibitors, how many additional deaths per 1,000 cases per year among users of drug A for prophylaxis compared to drug B would definitely cause you to restrict access to drug A? (Please enter a number of reported deaths in your health plan causing you to restrict access.) | – | 10 | 25 | 10 | 24 | 15 (79) | Yes |
| What is the likelihood that you will restrict access to drug A for prophylaxis compared to drug B under the following scenarios? | No. of breakthrough bleeding events requiring treatment per patient per year | ||||||
| 1 | 8 | 18 | 10 | 24 | 15 (79) | Yes | |
| 2 | 18 | 21 | 25 | 23 | 14 (73) | Yes | |
| 3 | 30 | 26 | 45 | 25 | 13 (68) | No | |
| 4 | 49 | 35 | 56 | 30 | 11 (58) | No | |
| 5 | 60 | 37 | 65 | 31 | 10 (52) | No | |
| >5 | 82 | 37 | 80 | 31 | 15 (79) | Yes | |
| What is the likelihood that you will restrict access to drug A for prophylaxis compared to drug B under the following scenarios? | No. of thrombotic events requiring treatment per patient per year | ||||||
| 1 | 17 | 31 | 25 | 29 | 15 (79) | Yes | |
| 2 | 30 | 32 | 40 | 30 | 10 (53) | No | |
| 3 | 50 | 36 | 50 | 32 | 12 (63) | No | |
| 4 | 68 | 35 | 80. | 30 | 12 (63) | No | |
| 5 | 82 | 37 | 85 | 32 | 14 (74) | Yes | |
| > 5 | 99 | 37 | 98 | 32 | 14 (74) | Yes | |
| What is the likelihood that you will restrict access to drug B for prophylaxis compared to drug A based on additional patients with any safety event (inhibitor development, thrombotic events/blood clots, death, etc.)? | No. of additional patients with any safety event per year | ||||||
| 1 or 2 | 15 | 26 | 15 | 29 | 13 (68) | No | |
| 3 or 4 | 25 | 32 | 30 | 30 | 15 (79) | Yes | |
| 5 or 6 | 32 | 37 | 41 | 33 | 11 (58) | No | |
| 7 or 8 | 48 | 36 | 51 | 34 | 10 (53) | No | |
| 9 or 10 | 65 | 34 | 80 | 31 | 15 (79) | Yes | |
| > 10 | 80 | 36 | 95 | 32 | 15 (79) | Yes | |
| Please indicate the likelihood that a particular number of years of “real-world” data on safety and efficacy are needed for you to use these data in coverage decisions that may lead you to either restrict or expand access to one product over another. | No. of years of real-world data | ||||||
| 1 | 13 | 29 | 10 | 27 | 13 (68) | No | |
| 2 | 28 | 31 | 32 | 29 | 10 (53) | No | |
| 3 | 50 | 30 | 50 | 28 | 8 (42) | No | |
| 4 | 65 | 34 | 80 | 30 | 15 (79) | Yes | |
| ≥ 5 | 80 | 34 | 95 | 31 | 11 (58) | No | |
| What is the most likely restriction to access you would use in hemophilia A based on new safety and efficacy information? Please rank the options, where 1 is the most likely restriction and 6 is the least likely restriction you would impose. | Nonpreferred tier status | 3 | 2 | 4 | 1 | 14 (73) | Yes |
| Step therapy | 3 | 1 | 3 | 1 | 13 (68) | No | |
| Prior authorization | 3 | 1 | 2 | 1 | 17 (90) | Yes | |
| Noncoverage | 5 | 2 | 5 | 2 | 16 (84) | Yes | |
| No restriction-coverage parity | 5 | 2 | 6 | 2 | 13 (68) | No | |
| Medical policy | 4 | 2 | 3 | 1 | 13 (68) | No | |
| How much of a relative percentage point difference of patients with no bleeding events would trigger you to restrict access to one product and treat the other product as preferred for hemophilia A without inhibitors? | – | 25 | 12 | 25 | 12 | 14 (74) | Yes |
| Assuming no change in efficacy for drug B, what is the likelihood that new, emerging “real-world” safety data will trigger a formulary restriction for Drug A if the real-world safety data show an increase in the following events with drug A? | No. of blood clots | 50 | 28 | 50 | 25 | 12 (63) | No |
| Inhibitor development | 49 | 33 | 50 | 29 | 12 (63) | No | |
| No. of deaths | 78 | 31 | 81 | 30 | 14 (74) | Yes | |
| Assuming no change in safety for drug A or drug B, what is the likelihood that new, emerging “real-world” efficacy data will trigger a formulary restriction for drug A if real-world efficacy data shows a decrease in the following events with drug B? | No. of bleeding events | 60 | 32 | 60 | 35 | 9 (47) | No |
| No. of joint bleeds | 63 | 31 | 60 | 32 | 10 (53) | No | |
| Use of rescue medications to control breakthrough bleeding | 68 | 32 | 75 | 32 | 14 (74) | Yes | |
| What is the minimum net cost difference that might result in preferred formulary treatment of drug A over drug B? | Assume Drug A has noninferior efficacy and noninferior safety to Drug B | 15 | 8 | 15 | 8 | 12 (63) | No |
| Assume drug A has noninferior efficacy and superior safety to Drug B | 13 | 13 | 15 | 10 | 12 (63) | No | |
| Assume drug A has superior efficacy and noninferior safety to drug B | 10 | 18 | 13 | 8 | 14 (74) | Yes | |
| Assume drug A has inferior efficacy and noninferior safety to drug B | 28 | 32 | 30 | 30 | 14 (74) | Yes | |
| Assume drug A has noninferior efficacy and inferior safety to drug B | 26 | 36 | 25 | 35 | 14 (74) | Yes | |
| Assume drug A has inferior efficacy and inferior safety to drug B | 50 | 36 | 60 | 33 | 11 (58) | No | |
| In the future, you will need to consider all of the trade-offs between hemophilia A prophylaxis options relative to available efficacy and safety data, cost, convenience, patient preference, among other factors. Given these descriptions, please rank in order how you will prioritize managing each class, from 1 (highest priority to manage) to 3 (lowest priority to manage). | Class A has decades of proven efficacy, established safety, and well-understood costs. | 1 | 1 | 1 | 1 | 10 (52) | No |
| Class B has noninferior efficacy and safety to class A, no longterm efficacy or safety information, and is an ~15% premium to class A, on average. | 2 | 1 | 2 | 1 | 9 (47) | No | |
| Class C has noninferior efficacy and safety to classes A and B, no longterm efficacy or safety information, and costs ~5 times higher than class A, but a potential average of ~95% lower cost after 5 years. | 3 | 1 | 2 | 1 | 7 (37) | No | |
| Please think about the current standard of care, unmet needs, average costs, and so on in the hemophilia A prophylaxis category. Given all those factors, what is your upper limit of how many years of experience with gene therapy would be required to justify the expected up-front cost of gene therapy? | – | 5 | 3 | 5 | 3 | 16 (84) | Yes |
| Assuming a comparable efficacy and safety profile, how much more are you willing to pay per patient per month to offer your patients greater convenience with a subcutaneous therapy (vs an intravenous therapy)? | – | 547 | 2126 | 579 | 1418 | 13 (68) | No |
| Users of emicizumab (Hemlibra) who have severe hemophilia A without inhibitors may, at some point in their treatment, require supplemental, concomitant use of either another therapy (likely a FVIII replacement product) or untested, increased dosing with Hemlibra to maintain prophylaxis. Safety data on combination therapy or increased emicizumab dosing will emerge with ongoing experience but has not been collected to date. How is your health plan likely to respond to the following 2 scenarios? | Scenario 1: concomitant use of emicizumab and an FVIII replacement therapy for sustaining prophylaxis | ||||||
| No change in policies | 14 | 31 | 15 | 27 | 16 (84) | Yes | |
| Prior approval for the addition of FVIII | 85 | 33 | 80 | 34 | 14 (74) | Yes | |
| Prior approval for emicizumab if used concomitantly with FVIII | 88 | 23 | 90 | 27 | 15 (79) | Yes | |
| Noncoverage of emicizumab for concomitant prophylaxis with FVIII | 28 | 27 | 25 | 30 | 15 (79) | Yes | |
| Other | 8 | 18 | 7 | 16 | 15 (79) | Yes | |
| Scenario 2: increased dosing of emicizumab beyond indicated for sustaining prophylaxis | |||||||
| No change in policies | 10 | 31 | 13 | 26 | 15 (79) | Yes | |
| Prior approval for the addition of FVIII | 80 | 33 | 75 | 34 | 14 (74) | Yes | |
| Prior approval for emicizumab if used concomitantly with FVIII | 20 | 31 | 25 | 34 | 11 (58) | No | |
| Noncoverage of emicizumab for concomitant prophylaxis with FVIII | 23 | 35 | 30 | 35 | 12 (63) | No | |
| Other | 10 | 24 | 6 | 15 | 16 (84) | Yes | |
aConsensus was considered to be present if at least 74% of panelists (i.e., 14 of 19 panelists) were within 1 SD of the median group estimate during round 3.
FVIII = clotting factor VIII.
EFFECT OF COST ON COVERAGE DECISIONS
Coverage decisions are often a balancing act between efficacy, safety, and cost. Respondents were offered a wide range of scenarios comparing 2 hypothetical treatments for patients with hemophilia A without inhibitors: the relative efficacy and safety varied between scenarios as inferior, noninferior, superior; cost varied as less than, equal to, or more than. These terms correspond with the terminology payers are most likely to see when reviewing clinical evidence to compare available treatments. Results for all scenarios are presented in Table 4.
TABLE 4.
Payer Responses to Scenarios Varying the Relative Cost, Efficacy, and Safety of 2 Hypothetical Treatments for Hemophilia Aa
| Scenario | Cost Options | Round 1 Results (n = 20) | Round 3 Results (n = 19)b | ||||
|---|---|---|---|---|---|---|---|
| Parity | Drug A Preferred | Drug B Preferred | Parity | Drug A Preferred | Drug B Preferred | ||
| 1. Assume drug A has noninferior efficacy and noninferior safety to drug B. | Drug A cost = Drug B cost | 19 (95) | 0 (0) | 1 (5) | 19 (100) | 0 (0) | 0 (0) |
| Drug A cost < Drug B cost | 3 (15) | 17 (85) | 0 (0) | 3 (16) | 16 (84) | 0 (0) | |
| Drug A cost > Drug B cost | 2 (10) | 1 (5) | 17 (85) | 3 (16) | 0 (0) | 16 (84) | |
| 2. Assume drug A has noninferior efficacy and superior safety to drug B. | Drug A cost = Drug B cost | 12 (60) | 6 (30) | 2 (10) | 11 (58) | 7 (37) | 1 (5) |
| Drug A cost < Drug B cost | 1 (5) | 17 (85) | 2 (10) | 2 (11) | 16 (84) | 1 (5) | |
| Drug A cost > Drug B cost | 7 (35) | 5 (25) | 8 (40) | 9 (47) | 4 (21) | 6 (32) | |
| 3. Assume drug A has superior efficacy and noninferior safety to drug B. | Drug A cost = Drug B cost | 9 (45) | 11 (55) | 0 (0) | 9 (47) | 10 (53) | 0 (0) |
| Drug A cost < Drug B cost | 5 (25) | 1 (5) | 14 (70) | 5 (26) | 1 (5) | 13 (68) | |
| Drug A cost > Drug B cost | 10 (50) | 3 (15) | 7 (35) | 11 (58) | 1 (5) | 7 (37) | |
| 4. Assume drug A has inferior efficacy and noninferior safety to drug B. | Drug A cost = Drug B cost | 5 (25) | 0 (0) | 15 (75) | 5 (26) | 0 (0) | 14 (74) |
| Drug A cost < Drug B cost | 3 (15) | 17 (85) | 0 (0) | 3 (16) | 16 (84) | 0 (0) | |
| Drug A cost > Drug B cost | 7 (35) | 2 (10) | 11 (55) | 8 (42) | 2 (11) | 9 (47) | |
| 5. Assume drug A has noninferior efficacy and inferior safety to drug B. | Drug A cost = Drug B cost | 9 (45) | 5 (25) | 6 (30) | 10 (53) | 3 (16) | 6 (32) |
| Drug A cost < Drug B cost | 6 (30) | 1 (5) | 13 (65) | 6 (32) | 0 (0) | 13 (68) | |
| Drug A cost > Drug B cost | 11 (55) | 5 (25) | 4 (20) | 10 (53) | 5 (26) | 4 (21) | |
| 6. Assume drug A has inferior efficacy and inferior safety to drug B. | Drug A cost = Drug B cost | 1 (5) | 0 (0) | 19 (95) | 2 (11) | 0 (0) | 17 (89) |
| Drug A cost < Drug B cost | 5 (25) | 1 (5) | 14 (70) | 5 (26) | 0 (0) | 14 (74) | |
| Drug A cost > Drug B cost | 2 (10) | 0 (0) | 18 (90) | 2 (11) | 0 (0) | 17 (89) | |
aData are n (%); data in some rows may not add up to 100 due to rounding.
bDuring round 3, consensus was achieved (green text) when at least 74% of panelists (i.e., 14 of 19 panelists) were within 1 SD of the median group estimate.
Among the 18 scenarios, 9 stood out because respondents reached consensus. Unsurprisingly, 100% of respondents agreed that parity was the likely outcome in the scenario where the 2 products being compared had noninferior efficacy and safety, and equal cost. Respondents also indicated a high likelihood of preferring a product with superior or noninferior efficacy and safety if the pricing was equal to or better than that of the comparator. Approximately a quarter of respondents (n = 5, 26%), however, indicated they would maintain parity in access if the product with superior efficacy and noninferior safety was less costly than the alternative. Respondents also showed an inclination to offer preferential coverage for less expensive products if they provided some advantage with either efficacy or safety, or when no clear advantage existed for one over the other (e.g., noninferior efficacy and noninferior safety).
Interestingly, respondents reached consensus that a 30% difference in net costs would lead to preferred formulary treatment if the drug had inferior efficacy but noninferior safety, whereas a 25% net cost difference would lead to preferred formulary treatment for a product with noninferior efficacy and inferior safety.
EFFECT OF CHANGES IN TREATMENT STRATEGIES AND EMERGING THERAPEUTICS
Users of emicizumab who have severe hemophilia A without inhibitors may, at some point during their treatment, require supplemental, concomitant use of a FVIII replacement product, or untested higher dosing with emicizumab, to maintain prophylaxis. Safety data on the combination therapy or higher emicizumab dosing have not been collected to date. Respondents were asked to react to a series of scenarios that occurred among their covered patients. When offered the options of no change in policy, prior approval for the addition of FVIII, prior approval for emicizumab if used concomitantly with FVIII, and no coverage for combination therapy, the respondents reached consensus that the most likely outcome would be prior approval for the addition of FVIII when emicizumab dosing was titrated (MLH 75%, SD 34%). In a scenario where FVIII therapy was required in addition to emicizumab, prior approval for emicizumab to be used in a combination was highly likely (MLH 90%, SD 27%).
Finally, in exploring the potential coverage of gene therapy, respondents were asked to indicate the upper limit of how many years of experience with gene therapy would be required to justify the expected initial costs of gene therapy. Respondents agreed that 5 years of real-world experience (SD 3 years) would be necessary.
Discussion
Expert assessments did not notably shift between rounds 1 and 3 of the Delphi elicitation. Round 2 provided the experts an opportunity to explain the responses they gave during round 1, to ask questions, and to react to overall responses in an anonymized online discussion board. During round 2, panelists provided 188 comments across 21 discussion topics. During the discussion, panelists pointed to the small numbers of patients and limited long-term evidence for efficacy and safety for many treatments as primary challenges in managing the hemophilia treatment category; they suggested relying heavily on providers, the U.S. Food and Drug Administration, and the market to determine access to treatments and to interpret available evidence on safety and efficacy. After the round 2 discussion, convergence was achieved for several key concepts, as detailed below.
Payers are under increasing pressure to manage costs in light of a wide range of emerging therapies that treat rare conditions. Although these new treatments offer important alternatives for patients, they often do so at substantially higher costs.31,32 Additionally, with new therapies comes much uncertainty, as most are tested on only a small population in clinical trials, and their effectiveness and potential adverse events in the real-world setting among a heterogeneous patient population are not known.33
For patients with hemophilia A, FVIII has for decades been an effective prophylaxis against bleeding events, despite some limitations (mainly around dosing method and risk of thrombosis). New treatments such as emicizumab and concizumab are disrupting how payers have managed hemophilia A prophylaxis to date, generally providing unrestricted access to FVIII therapies.16,17 Our study of payer practices in the hemophilia A therapeutic space found that payers are willing to add new treatments such as emicizumab to their formulary with few restrictions (e.g., prior approval) despite the lack of real-world data. Our study suggests, however, that as real-world data emerge, payers are likely to react to new information on safety, efficacy, and cost with strategies for limiting drug access that align with traditional formulary management practices. Existing Delphi studies in hemophilia focused on obtaining physicians’ consensus on treatment decision-making practices, and on patient outcomes and costs.34-39 This study is, to our knowledge, the first of its kind, aimed at obtaining payers’ consensus on coverage practices for hemophilia A prophylaxis treatments for which real-world data on safety and efficacy are evolving.
Other research has shown that highly innovative treatments, especially for rare diseases, often lack long-term evidence and therefore introduce considerable uncertainty into payers’ decision-making processes.40 Real-world data can bridge this evidence gap if they are collected systematically and if the collection process is documented appropriately.41,42 To date, however, no universally accepted definitions exist for key real-world evidence thresholds that are used to evaluate trade-offs between treatment options for rare diseases.
STRENGTHS AND LIMITATIONS
The use of the Delphi process as a structured technique to elicit expert feedback enabled the study team to capture perspectives and opinions from a broad spectrum of experts. This methodology facilitated an in-depth exploration of attitudes and opinions that is not possible with surveys. Given that only 23 of the 74 invited experts agreed to participate in the interviews, self-selection bias may be present. Panelists were compensated for their participation in this project, and they may also be more enthusiastic about the study subject than their peers who elected not to participate. To reduce researcher bias, 2 external scientific advisors—a clinical expert in hemophilia and related disorders, and an expert on budgetary and treatment-related questions in the management of hemophilic patient populations—reviewed the questionnaire for both content and design.
Other potential sources of bias that may have affected the experts include social desirability bias (individuals might seek to adjust their responses to achieve a desirable group consensus),43 the bandwagon effect and dominance (individuals are likely to respond similarly to the group or align with one particularly vocal individual),44 anchoring and adjustment bias (a specific point estimate or reference provided in the question influences the experts), the availability heuristic (experts might overstate specific likelihoods if they recently experienced a particular phenomenon), range–frequency compromise (when provided with a range of options, experts might apportion responses evenly), and overconfidence (experts might provide confidence intervals that are too narrow).45 The study team made efforts to minimize these biases by requiring experts to complete rounds 1 and 3 independently and maintaining anonymity throughout the entire Delphi process. Because of the small sample size, the results may not be generalizable, nor is the aim of any expert elicitation research to be generalizable. Nevertheless, we believe that the study results provide unique views of and valuable insights from experts regarding key issues surrounding coverage decision-making practices and trade-offs between long-term information on cost, efficacy, and safety in the hemophilia A treatment space.
Conclusions
U.S. payers strongly prefer treatments with known efficacy and safety, and well-understood costs, over newer treatments with uncertain long-term effects for patients with hemophilia A without inhibitors. Relatively unrestricted access will likely continue for legacy and new hemophilia A prophylaxis unless additional real-world safety concerns or major cost differences emerge. Payers in our sample reached a consensus that real-world evidence on the safety and efficacy of treatment holds the key to coverage decisions in the management of hemophilia A. Payers were likely to impose some type of access restriction or require prior approval for the combination therapy of emicizumab and FVIII. These perspectives on access policies may help mitigate payer uncertainty when making coverage decisions regarding hemophilia A prophylaxis treatments.
REFERENCES
- 1.Centers for Disease Control and Prevention. What is hemophilia? Last reviewed July 17, 2020. Accessed March 15, 2021. https://www.cdc.gov/ncbddd/hemophilia/facts.html
- 2.Centers for Disease Control and Prevention. A new study of hemophilia occurrence finds many more cases in the United States. Reviewed July 17, 2020. Accessed March 15, 2021. https://www.cdc.gov/ncbddd/hemophilia/features/keyfinding-hemophilia-occurrence-US.html
- 3.Konkle BA, Skinner M, Iorio A. Hemophilia trials in the twenty-first century: defining patient important outcomes. Res Pract Thromb Haemost. 2019;3(2):184-92. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Srivastava A, Elena Santagostino E, Dougall A, et al. WFH Guidelines for the Management of Hemophilia. Haemophilia. 2020;26(Suppl 6):1-158. [DOI] [PubMed] [Google Scholar]
- 5.Witmer C, Young G. Factor VIII inhibitors in hemophilia A: rationale and latest evidence. Ther Adv Hematol. 2013;4(1):59-72. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Srivastava A, Brewer AK, Mauser-Bunschoten EP, et al. Guidelines for the management of hemophilia. Haemophilia. 2013;19(1):e1-47. [DOI] [PubMed] [Google Scholar]
- 7.Malec LM, Cheng D, Witmer CM, et al. The impact of extended half-life factor concentrates on prophylaxis for severe hemophilia in the United States. Am J Hematol. 2020;95(8):960-65. [DOI] [PubMed] [Google Scholar]
- 8.Arruda VR, Doshi BS, Samelson-Jones BJ. Emerging therapies for hemophilia: controversies and unanswered questions. F1000Res. 2018;7:F1000 Faculty Rev-489. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Doshi BS, Arruda VR. Gene therapy for hemophilia: what does the future hold? Ther Adv Hematol. 2018;9(9):273-93. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Hemlibra. Prescribing information. Genentech. Revised 2021. Accessed March 29, 2021. https://www.gene.com/download/pdf/hemlibra_prescribing.pdf
- 11.Reyes A, Révil C, Niggli M, et al. Efficacy of emicizumab prophylaxis versus factor VIII prophylaxis for treatment of hemophilia A without inhibitors: network meta-analysis and sub-group analyses of the intra-patient comparison of the HAVEN 3 trial. Curr Medical Res Opin. 2019;35(12):2079-87. [DOI] [PubMed] [Google Scholar]
- 12.Taki M, Fukutake K, Matsushita T, et al. Inhibitor development, safety, and efficacy of Advate® in previously untreated patients with hemophilia A in a postmarketing surveillance in Japan. Int J Hematol. 2019;109(1):70-78. [DOI] [PubMed] [Google Scholar]
- 13.Nogami K, Takedani H, Shima M, et al. Perioperative safety and hemostatic efficacy of Advate® in patients with hemophilia A in a postmarketing surveillance in Japan. Int J Hematol. 2018;108(1):22-29. [DOI] [PubMed] [Google Scholar]
- 14.Humphries TJ, Rauchensteiner S, Tückmantel C, Pieper A, Maas Enriquez M, Mathew P. Sucrose-formulated recombinant factor VIII dosing flexibility in prophylaxis regimens: experience from postmarketing surveillance studies. Adv Hematol. 2015;2015:431268. doi:10.1155/2015/431268 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Aledort L, Milligan S, Watt M, Booth J. A retrospective observational study of rurioctocog alfa pegol in clinical practice in the United States. J Manag Care Spec Pharm. 2020;26(4):492-503. doi:10.18553/jmcp.2020.26.4.492 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Rind D, Walton S, Agboola F. Valoctocogene roxaparvovec and emicizumab for hemophilia A: effectiveness and value. Evidence report. Institute for Clinical and Economic Review. October 26, 2020. Accessed March 15, 2020. https://icer.org/wp-content/uploads/2020/10/ICER_Hemophilia-A_Evidence-Report_102620.pdf
- 17.Franchini M, Mannucci PM. Non-factor replacement therapy for haemophilia: a current update. Blood Transfus. 2018;16(5):457-61. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Helmer O. Analysis of the future: the Delphi method. RAND. March 1967. Accessed March 15, 2021. https://www.rand.org/pubs/papers/P3558.html
- 19.Ayanian JZ, Landrum MB, Normand SL, Guadagnoli E, McNeil BJ. Rating the appropriateness of coronary angiography–do practicing physicians agree with an expert panel and with each other? New Engl J Med. 1998;338(26):1896-904. [DOI] [PubMed] [Google Scholar]
- 20.Campbell SM, Cantrill JA, Roberts D. Prescribing indicators for UK general practice: Delphi consultation study. BMJ. 2000;321(7258):425-28. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Fitch K, Bernstein SJ, Aguilar MD, Burnand B, LaCalle JR. The RAND/UCLA Appropriateness Method User’s Manual. Santa Monica, CA: Rand; 2001. [Google Scholar]
- 22.Khodyakov D, Grant S, Barber CE, Marshall DA, Esdaile JM, Lacaille D. Acceptability of an online modified Delphi panel approach for developing health services performance measures: results from 3 panels on arthritis research. J Eval Clin Pract. 2017;23(2):354-60. [DOI] [PubMed] [Google Scholar]
- 23.Khodyakov D, Grant S, Denger B, et al. Practical considerations in using online modified-Delphi approaches to engage patients and other stakeholders in clinical practice guideline development. Patient. 2020;13(1):11-21. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.PRECISIONvalue. Exclusive, actionable insights from former payers. Accessed March 15, 2021. https://www.precisionvalue.com/services/#anchor4
- 25.Welphi. About. 2020. Accessed March 15, 2021. https://www.welphi.com/en/About.html
- 26.Diamond IR, Grant RC, Feldman BM, et al. Defining consensus: a systematic review recommends methodologic criteria for reporting of Delphi studies. J Clin Epidemiol. 2014;67(4):401-09. [DOI] [PubMed] [Google Scholar]
- 27.Mahlangu J, Oldenburg J, Callaghan MU, et al. Bleeding events and safety outcomes in persons with haemophilia A with inhibitors: a prospective, multi-centre, non-interventional study. Haemophilia. 2018;24(6):921-29. [DOI] [PubMed] [Google Scholar]
- 28.Hurwitz JT, Brown M, Graff JS, Peters L, Malone DC. Is real-world evidence used in P&T monographs and therapeutic class reviews? J Manag Care Spec Pharm. 2017;23(6):613-20. doi:10.18553/jmcp.2020.26.12.1604 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Schilsky RL. Finding the evidence in real-world evidence: moving from data to information to knowledge. J Am Coll Surg. 2017;224(1):1-7. [DOI] [PubMed] [Google Scholar]
- 30.Najafzadeh M, Schneeweiss S. From trial to target populations–calibrating real-world data. N Engl J Med. 2017;376(13):1203-05. [DOI] [PubMed] [Google Scholar]
- 31.Ollendorf DA, Chapman RH, Pearson SD. Evaluating and valuing drugs for rare conditions: no easy answers. Value Health. 2018;21(5):547-52. [DOI] [PubMed] [Google Scholar]
- 32.Schlander M, Dintsios C-M, Gandjour A. Budgetary impact and cost drivers of drugs for rare and ultrarare diseases. Value Health. 2018;21(5):525-31. [DOI] [PubMed] [Google Scholar]
- 33.Kennedy-Martin T, Curtis S, Faries D, Robinson S, Johnston J. A literature review on the representativeness of randomized controlled trial samples and implications for the external validity of trial results. Trials. 2015;16:495. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Athale A, Giguere A, Barbara A, Krassova S, Iorio A. Developing a twosided intervention to facilitate shared decision-making in haemophilia: decision boxes for clinicians and patient decision aids for patients. Haemophilia. 2014;20(6):800-06. [DOI] [PubMed] [Google Scholar]
- 35.Barnes C, Brown SA, Curtin J, Dunkley S. When is enough … enough? Developing consensus of definition of failure of immune tolerance induction in patients with haemophilia and inhibitors. Haemophilia. 2014;20(4):e275-79. [DOI] [PubMed] [Google Scholar]
- 36.Benson G, Auerswald G, Elezović I, et al. Immune tolerance induction in patients with severe hemophilia with inhibitors: expert panel views and recommendations for clinical practice. Eur J Haematol. 2012;88(5):371-379. [DOI] [PubMed] [Google Scholar]
- 37.Bonnet P, Gringeri A, Gomperts E, et al. Modeling costs and outcomes associated with a treatment algorithm for problem bleeding episodes in patients with severe hemophilia a and high-titer inhibitors. Am Health Drug Benefits. 2011;4(4):219-31. [PMC free article] [PubMed] [Google Scholar]
- 38.Fischer K, Lewandowski D, Janssen MP. Estimating unknown parameters in haemophilia using expert judgement elicitation. Haemophilia. 2013;19(5):e282-88. [DOI] [PubMed] [Google Scholar]
- 39.Iorio A, Puccetti P, Makris M. Clotting factor concentrate switching and inhibitor development in hemophilia A. Blood. 2012;120(4):720-27. [DOI] [PubMed] [Google Scholar]
- 40.Conti RM, Gruber J, Ollendorf DA, Neumann PJ. Valuing rare pediatric drugs: an economics perspective. Working paper 27978. National Bureau of Economic Research. October 2020. Accessed March 15, 2021. https://www.nber.org/papers/w27978 [Google Scholar]
- 41.Hampson G, Towse A, Dreitlein WB, Henshall C, Pearson SD. Real-world evidence for coverage decisions: opportunities and challenges. J Comp Eff Res. 2018;7(12):1133-43. [DOI] [PubMed] [Google Scholar]
- 42.Facey KM, Rannanheimo P, Batchelor L, Borchardt M, de Cock J. Real-world evidence to support payer/HTA decisions about highly innovative technologies in the EU–actions for stakeholders. Int J Technol Assess Health Care. 2020;36(4):459-68. [DOI] [PubMed] [Google Scholar]
- 43.Ecken P, Gnatzy T, von der Gracht HA. Desirability bias in foresight: consequences for decision quality based on Delphi results. Technol Forecast Soc Change. 2011;78(9):1654-70. [Google Scholar]
- 44.Winkler J, Moser R. Biases in future-oriented Delphi studies: a cognitive perspective. Technol Forecast Soc Change. 2016;105:63-76. [Google Scholar]
- 45.O’Hagan A. Expert knowledge elicitation: subjective but scientific. Am Stat. 2019;73(Suppl 1):69-81. [Google Scholar]
