Determining value in the treatment of activated PI3Kδ syndrome in Spain: a multicriteria decision analysis from the perspective of key stakeholders

ABSTRACT

Introduction:

Activated phosphoinositide 3-kinase (PI3K)δ syndrome (APDS) is an ultra-rare inborn error of immunity (IEI) combining immunodeficiency and immune dysregulation. This study determined what represents value in APDS in Spain from a multidisciplinary perspective applying multicriteria decision analysis (MCDA) methodology.

Methods:

A multidisciplinary committee of nine experts scored the evidence matrix. A specific framework for orphan drug evaluation in Spain and the weights assigned by a panel of 98 evaluators and decision-makers was used. Re-evaluation of scores was performed.

Results:

APDS is considered a very severe disease with important unmet needs, including misdiagnosis and diagnostic delay. Current management is limited to treatment of symptoms with off-label use of therapies supported by limited evidence. Therapeutic benefit is partial, resulting in limited disease control. Haematopoietic stem cell transplantation (HSCT), the only potential curative alternative, is restricted to a reduced patient population and without evidence of long-term efficacy or safety. All options present a limited safety profile. Data on patients’ quality of life are lacking. APDS is associated with high pharmacological, medical and indirect costs.

Conclusions:

APDS is considered a severe disease, with limited understanding by key stakeholders of how treatment success is assessed in clinical practice, the serious impact that has on patients and the associated high economic burden. This study brings to light how MCDA methodology could represent a useful tool to complement current clinical and decision-making methods used by APDS experts and evaluators.

Introduction

Activated phosphoinositide 3-kinase (PI3K)δ syndrome (APDS) is a very severe and complex disease, an ultra-rare inborn error of immunity (IEI), estimated to affect fewer than 1-2 persons per 1,000,000 worldwide, caused by autosomal-dominant mutations in genes coding for subunits of PI3Kδ (1). These lead to hyperactive signalling of the PI3Kδ/Akt/mTOR pathway (2), resulting in abnormal development and maturation of immune cells, immunodeficiency and immune dysregulation (2,3).

Most patients are diagnosed in childhood-adolescence (median diagnosis age at 12 years [6.5-21.5] (2)) presenting with a wide variety of clinical manifestations (3,4). Over 90% of patients experience some form of disease manifestation by the time they are 6 years old (2). The clinical course is generally unpredictable and heterogeneous from patient to patient. Due to the progressive nature of APDS, patients are likely to experience an increased risk of disease complications over time (e.g. severe upper respiratory infections, development of benign lymphoproliferation and, subsequently, development of autoimmunity and gastrointestinal manifestations (2,3)). These may result in life-threatening complications and, together with haematologic malignancies, particularly B-cell lymphoma, contribute to the majority of deaths (3).

There are currently no approved treatments for APDS in Spain. Disease management is limited to symptomatic treatments including antimicrobials, immunoglobulin replacement therapy (IRT) and off-label immunosuppressants and immunomodulators (corticosteroids, rituximab and sirolimus) which do not change the natural history of the disease, do not resolve all disease symptoms and are associated with severe adverse events (AEs) (2,3). Haematopoietic stem cell transplantation (HSCT) represents the only potential curative therapy. Given its associated morbidity, patients with APDS (and their caregivers) are expected to experience impaired quality of life (QoL) as commonly seen with other IEI (5). There is currently no specific guideline, protocol, consensus document or treatment algorithm on APDS available in Spain (nor globally) to support decision-making.

It is widely recognised that healthcare systems face distinct challenges for the evaluation and decision-making of medicines for rare and ultra-rare conditions (6). Key reasons include lack of disease awareness and knowledge and its unmet needs, impeding to ascertain the value contribution of any new treatment.

Since information on APDS is scarce, it is of interest to generate evidence on the current situation and unmet needs as well as to determine what represents value when evaluating treatment alternatives so as to drive informed evaluation and decision-making by healthcare systems.

Reflective multicriteria decision analysis (MCDA) offers a methodological framework that allows determination of what represents value in a given medical condition, considering all criteria relevant for healthcare decision-making in a transparent and systematic manner and from the perspective of relevant stakeholders. MCDA methodology is already being used by regional healthcare services, health technology assessment agencies and hospital pharmacy services in Spain, especially in complex areas such as rare and ultra-rare diseases and evaluation of orphan drugs (ODs) (7).

The aim of this study was to apply MCDA methodology to determine and discuss what represents value in the treatment of APDS through a multidisciplinary perspective in Spain.

Methods

Study design

The study was designed following good methodological practices (8,9). The MCDA framework specifically developed and validated by Spanish stakeholders involved in the evaluation of ODs and decision-making at national, regional and hospital level was selected (10).

Panel design and training

A multidisciplinary panel of nine experts from reference centres in IEI (five physicians and four hospital pharmacists [HPs] from six Spanish regions) was invited to participate online in November 2022. Physicians were chosen based on their practical experience in managing APDS patients. HPs are experts in evaluation and decision-making. The study was coordinated by the consultancy company. The training (explaining the methodology and its interpretation) of the experts participating in the MCDA was conducted by the consultancy company.

Adapted MCDA value framework

An adaptation of the EVIDEM MCDA framework was used. The adaptation was developed for the evaluation of ODs in Spain by a group of Spanish evaluators (10). The particularities presented by ODs (such as the limitation of information available or the differences in clinical development compared to non-ODs) were considered in the adaptation of the framework.

Evidence matrix

A systematic literature review was conducted between August and September 2022 to retrieve relevant information for each of the OD MCDA framework criteria. Published evidence was searched using biomedical databases (PubMed/Medline, Cochrane, Medes). The search included published articles in English or Spanish without time span restriction complemented using grey literature sources (e.g. Google Scholar, clinical practice guidelines (CPGs), patient association websites and official European and Spanish healthcare evaluation bodies’ webpages). A total of 75 publications were finally included for the synthesis of evidence. Retrieved information was used to populate the MCDA OD framework criteria shown in Table 1.

TABLE 1 -.

Adapted multicriteria decision analysis value framework specifically developed for the evaluation of orphan drugs in Spain (10)

Quantitative criteria
Domain – Impact of the disease: • Disease severity • Unmet needs
Domain – Results observed in: • Efficacy/effectiveness • Safety/tolerability • Quality of life (patient-reported outcomes)
Domain – Type of benefit: • Therapeutic impact
Domain – Economic consequences: • Cost of intervention • Other medical costs • Non-medical (indirect) costs
Domain – Understanding the intervention: • Quality of evidence
Contextual/Qualitative criteria
Domain – Regulatory context: • Mandate and scope of healthcare system and population priorities and access • Common goal and specific interests
Domain – Feasibility: • System capacity and appropriate use of intervention

Matrix scoring

Experts scored the evidence matrix individually prior to group discussion of results (November 2022), during which all experts shared their scoring results and debated based on their individual rationale. Scoring was performed considering the information presented in the matrix and experts’ own experience and perception. Quantitative criteria were scored using an ordinal scale that ranged from 0 to 5 in increasing order of value contribution perceived by the experts. Contextual criteria were scored in a categorical scale with three levels: negative, neutral and positive.

Discussion meeting and retest

An expert group meeting was performed with all participants to discuss the results obtained and the reasoning behind the scores of each criterion. After the session, participants were requested to repeat the scoring exercise to assess the consistency of results.

Data analysis

Value scores were collected individually from each participant, transferred to a common database and analysed using Microsoft Excel software. For each quantitative criterion, the mean, standard deviation (SD) and the range of minimum and maximum scores were calculated. Contextual criteria scoring was assessed as the percentage of experts considering each scoring option. Initial and final results (re-scoring after discussion) were analysed using the Wilcoxon test to assess for statistically significant differences.

Results

Quantitative criteria scoring

Quantitative criteria scores are shown in Fig. 1.

FIGURE 1 -.

Quantitative criteria value scoring results. Min = minimum; Max = maximum; n = number of experts; SD = standard deviation. The black dots correspond to the mean of the scores.

“Severity of the disease” was one of the most highly scored criteria (4.1±0.3). APDS was considered a very severe and complex disease due to the associated morbidity and high risk of mortality. “Unmet needs” (4.1±0.6) was highly scored due to the lack of specific treatments and the substantial misdiagnosis and diagnosis delay. “Other medical costs” also received a high score (3.7±0.5) considering the high use of healthcare resources derived from the hospitalisations due to complications and the hospital administration of treatments like IRT. “Cost of treatment” received a significant score (3.4±0.5), due to the chronic use of a combination of therapies and specially HSCT, as well as “Non-medical costs” (3.4±0.5) due to the burden assumed by patients and their families/carers as patients need to visit the hospital regularly to receive treatment or due to complications. A score of 3.0±0 was assigned to “Patient-Reported Outcomes”, reflecting a perceived efficacy/safety balance of current alternatives. The “Therapeutic impact” of available treatments was considered moderate (2.9±0.3) as reported clinical outcomes are suboptimal. “Efficacy/effectiveness” was scored with 2.8±0.4 since current alternatives are partially effective and symptom-based, resulting in limited disease control. “Safety and Tolerability” was scored with 2.4±0.5 as the therapeutic options available have a moderate safety with limitations (i.e. immunosuppressants increasing infection risk or sirolimus presenting severe AEs over the long term). The “Quality of evidence” supporting currently available treatments was considered low (2.0±0.5) as the available data are based only on clinical experience and not on formal regulatory and/or published evidence.

Qualitative (contextual) criteria scoring

Figure 2 shows scoring results for contextual criteria.

FIGURE 2 -.

Qualitative (contextual) criteria value scoring results. HC = healthcare.

“Mandate and scope of the healthcare system and population priorities and access”: most participants (67%) believe APDS is aligned with the Spanish National Health System (NHS) priorities as it is a rare disease; 33% assigned a neutral score since IEI (including APDS) is not reflected in healthcare plans. “Common Goals and specific Interests”: most participants scored positively (67%) believing that a new treatment for APDS should not encounter access barriers; 33% scored neutral given the lack of specific guidelines. “System capacity and appropriate use of intervention”: most participants (89%) scored positively considering that the Spanish NHS is well prepared for the introduction of a specific drug for the treatment of APDS. However, one expert (11%) scored neutral, highlighting some institutional limitations might exist for patients’ derivation to reference hospitals.

Re-scoring

The Wilcoxon test performed showed no statistically significant differences between initial scoring and re-scoring results except for the “Non-medical cost” criterion (−0.66; p<0.05). After re-scoring, mean scores increased slightly (+0.1) for the “Disease severity”, “Unmet needs” and the “Patient-Reported Outcomes” criteria. A larger increase in mean values was reported for the “Other Medical costs” (+0.3) and the “Indirect costs” (+0.6) criteria. Experts assigned lower scores to the “Efficacy/effectiveness” (−0.1), “Safety/tolerability” (−0.3), the “Cost of treatment” (−0.1) and the “Quality of evidence” (−0.2) criteria.

Discussion

To our knowledge, this represents the first study to determine current management and what represents value in APDS in Spain using reflective MCDA methodology and by a multidisciplinary panel of expert stakeholders involved in patients’ management, evaluation of treatments for rare diseases and decision-making. Criteria scoring and reflective group discussion allowed a holistic identification of key value drivers and the unmet needs for this disease.

APDS is considered a very severe disease. Immune deficiency and immune dysregulation can progress to irreversible organ damage and life-threatening complications, and especially B-cell lymphoma, usually associated with chronic Epstein-Barr virus infections, representing the main cause of mortality in APDS (2-4,11). APDS presents patient management challenges, including inter-patient heterogeneity and the presence of at least two severe symptoms in most patients: lymphoproliferation (>70% of patients) with progression to lymphoma and gastrointestinal manifestations. The high risk of lymphoma is concerning, presenting by the late teens or early adulthood (78% cumulative risk at 40 years of age, with a median age at diagnosis of malignancy of 19 years), representing the main cause of mortality (2). The lack of treatments specifically developed and approved for APDS, substantial misdiagnosis and diagnosis delay remain key challenges, with many patients suffering from recurrent severe infections since childhood leading to a marked worsening of prognosis. Because of the complexity of disease management, experts identified the need for a consensus protocol in order to establish best practices and support decision-making. Current available alternatives are only partially effective and symptom-based, resulting in limited disease control. Antimicrobial prophylaxis and IRT were considered partially effective in preventing and controlling the risk of infections but do not target the core underlying cause of immune dysregulation or immune deficiency. Immunosuppressant and immunomodulatory agents only have partial effect against lymphoproliferation with limited benefit when treating gastrointestinal manifestations (e.g. enteropathy) without reducing the risk of infections. HSCT use is limited to a small subset of patients (9%-13%) (2,3,12-14), particularly in those with severe infections and/or dysregulation (gastrointestinal disease) and lymphoma and sometimes requiring multiple transplants.

The lack of published, long-term data represents a major limitation to assess the effectiveness of current treatments, including HSCT. Current options present limitations in their safety and tolerability profiles: IRT is generally well tolerated. In contrast, sirolimus is associated with severe AEs, including pancreatic atrophy, metabolic complications, thrombosis, aphthous ulcers and stomatitis, which are difficult to manage and may even increase the risk of malignancy (2,3,15). HSCT is associated with AEs in 91% of patients with APDS, including infectious complications, graft-versus-host disease, organ toxicity and renal failure and transplant-related mortality reported in 14% of patients after 2 years (13,14). There is no clear guidance on whether and when to perform HSCT in APDS patients. Spanish consensus guidelines (2020) recommended that, in patients with common variable immunodeficiency phenotype with immune dysregulation (including autosomal-dominant PI3Kδ mutations), HSCT should be considered after an individual has failed first-line therapies with abatacept, PI3K or JAK inhibitors, or in cases of incomplete response (16). No long-term data are available to determine whether HSCT addresses the manifestations of hyperactive PI3K outside of the immune system. Therefore, it is unknown whether it fully targets the underlying disease. In this sense, experts claimed for the need of a targeted therapy that achieves this objective, providing predictable and long-term symptom control.

Severe and recurrent APDS symptoms can substantially impact on the QoL of these patients. The lack of published data and of specific APDS QoL questionnaires was considered as a major limitation and a specific unmet need. Besides, the need for several prophylactic and chronic treatments for multiple symptoms makes it difficult to assess the directly measurable QoL benefit of long-term treatment.

Treating APDS involves considerable pharmaceutical burden and pharmacological costs with the need to co-administer several therapies, even higher for HSCT, which in APDS patients can be up to three times more expensive than the average HSCT cost given the potential associated complications (e.g. use of defibrotide for liver problems derived from the transplant) and the need for chronic IRT treatment post-transplant. APDS involves considerable use of medical resources, which increases significantly with age due to complications such as pulmonary sequelae or the development of lymphoma requiring frequent, extended follow-up visits, hospitalisations and complementary care. APDS is perceived to be associated with relevant indirect costs due to the burden and productivity losses assumed by patients. The quality of evidence regarding available treatments is low, based on clinical experience, cohort studies and case series. At the time of the study there were no clinical trials published to support the repositioning of currently used off-label alternatives. Recently, a randomised, placebo-controlled phase 3 trial study observed the selective PI3Kδ inhibitor leniolisib which targets the root cause of the disease, reducing lymphadenopathy and significantly increasing naïve B-cell percentage, while being overall well tolerated (17). In addition to leniolisib, two other PI3Kδ inhibitors have been investigated in APDS: nemiralisib (18) and seletalisib (19).

Re-scoring scores after reflexive group discussion increased slightly for the “Disease severity” criterion due to the difficulty of symptom management and the associated high risk of lymphoma. Higher increases in mean scoring values for “Other Medical costs” and “Indirect costs” criteria reflected the sharing of clinicians’ personal experience on associated burden for patients and families. On the other hand, re-scoring values were lower for “Efficacy/effectiveness” mainly due to the lack of long-term data demonstrating durable and predictable symptom control with current options and for “Safety/tolerability” due to strong concern about the severe AEs associated with the chronic use of sirolimus and the important safety risks associated with HSCT discussed during the group session. Mean re-scoring of “Cost of treatment” lowered slightly since it was understood that, although the HSCT cost is very high, it only applies to a small proportion of patients. The “Quality of evidence” scored lower in re-scoring, as the experts considered there is lack of formal published evidence.

The concept of what represents value in a given condition can vary among healthcare professionals, resulting in a range of definitions. The reflective component of the MCDA methodology used in this study allowed to understand and discuss the rationale behind experts’ scores for each value criterion, and to understand the perspectives of different stakeholder profiles contributing to increased awareness and knowledge and collegiate decision-making.

Changes in re-scores after discussion showed how some experts changed their perceptions after gaining knowledge and understanding the rationale provided by colleagues who shared their own practical experience with managing APDS patients. This is especially relevant in ultra-rare diseases, for which no/limited published evidence is usually available. Spanish experts recommended promoting identification and registration of patients with APDS at international level to contribute to much needed evidence generation. Increased APDS awareness across the wide medical community is necessary, as potential cases may not be recognised and timely referred to expert immunologists. Experts claimed for the need of a consensus protocol to ensure efficient coordination across levels of care and to establish best practices in the care pathway for these patients in Spain.

This study has several strengths. Using MCDA methodology, each criterion was evaluated systematically, transparently and objectively. The experts classified the exercise as useful and that the exchange of opinions enriched individual analysis and assessments, reflected during group discussion and changes during re-scoring. The experts were selected based on their practical expertise, while trying to achieve a balanced geographical representation.

However, this study has some first, the limited number of experts participating in the study. The reason for choosing a relatively small panel size in MCDA exercises facilitates group discussions and sharing of perspectives, allowing an in-depth analysis of the different value criteria. Nonetheless, the number of experts involved in this study is in accordance with those from previous, similar MCDA studies (10,16,20) and resembles the number of experts participating in regional and hospital evaluation committees in Spain. Additional future work with a larger group of experts could be warranted to validate and complement the study findings. Secondly, this study does not include the patients’ perspective. At present, patient representatives are not routinely involved in evaluation and/or decision-making processes in Spain. Therefore, in order to reflect the current situation, they were not involved. And finally, at the time of this study, lack of data and published evidence may have affected the scoring. Thus, results might change when new data become available.

Conclusions

Reflective MCDA methodology has allowed the determination of what represents value and the identification of key unmet needs in APDS from the point of view of a multidisciplinary group of experts, considering a wide range of criteria to drive clinical assessment, evaluation and decision-making. It is expected that this study will raise awareness, promote further work to fill evidence gaps and contribute to informed decision-making in APDS.

Abbreviations (alphabetical order)

AEs = adverse events; APDS = activated PI3Kδ syndrome; HPs = hospital pharmacists; HSCT = haematopoietic stem cell transplantation; IEI = inborn error of immunity; IRT = immunoglobulin replacement therapy; MCDA = multicriteria decision analysis; NHS = National Health System; ODs = orphan drugs; PI3Kδ = phosphoinositide 3-kinase (PI3K) δ; PROs = patient-reported outcomes; QoL = quality of life.

Funding Statement

Conflict of Interest: RA has received honoraria from Akcea Therapeutics, Biogen, Boehringer Ingelheim, Clovis Oncology, CSL Behring, Janssen, Sobi, Pharming Group N.V., Sanofi and Zogenix International, also receiving honoraria from Pharming for participation in this study; CA, LA, ELG, JLP and CR have received honoraria from Pharming for participation in this study; ON has received consultation honoraria from Pharming and invited as a speaker by Grifols; JGR has participated in remunerated medical advisory board for Pharming; JBT and RF are employees of Pharming. SS and AG are employees of Omakase Consulting which received funding from Pharming to develop and conduct this study and the elaboration of the manuscript. None of the authors have received honoraria for the review of the manuscript.

Financial support: This study was sponsored by Pharming Group N.V.

Authors contribution: AG has been responsible for the conceptualisation and design of the study; MRA, CA, LA, ELG, ON, JLP, JGR, CRG performed the scores; JBT, RF and SS analysed the data; AG developed the draft of this manuscript. All authors have reviewed and approved this manuscript.

Availability of Data and Material: The datasets used and analysed during the current study are available from the corresponding author on reasonable request.

Consent for publication: All co-authors have consented to publication of this manuscript.