Implementing Outcomes-Based Managed Entry Agreements for Rare Disease Treatments: Nusinersen and Tisagenlecleucel

Abstract

Background and Objective

Enthusiasm for the use of outcomes-based managed entry agreements (OBMEAs) to manage uncertainties apparent at the time of appraisal/pricing and reimbursement of new medicines has waned over the past decade, as challenges in establishment, implementation and re-appraisal have been identified. With the recent advent of innovative treatments for rare diseases that have uncertainties in the clinical evidence base, but which could meet a high unmet need, there has been renewed interest in the potential of OBMEAs. The objective of this research was to review the implementation of OBMEAs for two case studies across countries in the European Union, Australia and Canada, to identify good practices that could inform development of tools to support implementation of OBMEAs.

Methods

To investigate how OBMEAs are being implemented with rare disease treatments, we collected information from health technology assessment/payer experts in countries that had implemented OBMEAs for either nusinersen in spinal muscular atrophy or tisagenlecleucel in two cancer indications. Operational characteristics of the OBMEAs that were publicly available were documented. Then, the experts discussed issues in implementing these OBMEAs and specific approaches taken to overcome challenges.

Results

The OBMEAs identified were based on individual outcomes to ensure appropriate use, manage continuation of treatment and in two cases linked to payment schedules, or they were population based, coverage with evidence development. For nusinersen, population-based OBMEAs are documented in Belgium, England and the Netherlands and individual-based schemes in Bulgaria, Ireland, Italy and Lithuania. For tisagenlecleucel, there were population-based schemes in Australia, Belgium, England and France and individual-based schemes in Italy and Spain. Comparison of the OBMEA constructs showed some clear published frameworks and clarity of the uncertainties to be addressed that were similar across countries. Agreements were generally made between the marketing authorisation holder and the payer with involvement of expert physicians. Only England and the Netherlands involved patients. Italy used its long-established, national, web-based, treatment-specific data collection system linked to reimbursement and Spain has just developed such a national treatment registry system. Other countries relied on a variety of data collection systems (including clinical registries) and administrative data. Durations of agreements varied for these treatments as did processes for interim reporting. The processes to ensure data quality, completeness and sufficiency for re-analysis after coverage with evidence development were not always clear, neither were analysis plans.

Conclusions

These case studies have shown that important information about the constructs of OBMEAs for rare disease treatments are publicly available, and for some jurisdictions, interim reports of progress. Outcomes-based managed entry agreements can play an important role not only in reimbursement, but also in treatment optimisation. However, they are complex to implement and should be the exception and not the rule. More recent OBMEAs have developed document covenants among stakeholders or electronic systems to provide assurances about data sufficiency. For coverage with evidence development, there is an opportunity for greater collaboration among jurisdictions to share processes, develop common data collection agreements, and share interim and final reports. The establishment of an international public portal to host such reports would be particularly valuable for rare disease treatments.

Supplementary Information

The online version contains supplementary material available at 10.1007/s40273-021-01050-5.

Key Points for Decision Makers

Outcomes-based managed entry agreements can in some circumstances help resolve uncertainties relating to the value and optimal use of a rare disease treatment, but they should be supported by a covenant agreed by all stakeholders providing assurance that all necessary actions will be taken to ensure that the data collected will be sufficient for decision making.

When used, outcomes-based managed entry agreement constructs for data collection and reports of clinical results should be published and shared to enable agreement of core data across jurisdictions, data aggregation and to support health system learnings.

An international public repository for outcomes-based managed entry agreement reports akin to the HTA database hosted by the International Network of Agencies for Health Technology Assessment would be valuable.

Background

Because of the difficulties associated with clinical research in rare diseases, treatments often come to market with a small evidence base involving uncontrolled or small randomised controlled trials of short duration, studying outcomes that may not clearly demonstrate patient benefit [1]. This leads to a range of uncertainties relating to the population appropriate for reimbursement, and to the determination of clinical and cost effectiveness. Alongside this, rare disease treatments are often associated with high prices, which challenge standard willingness-to-pay thresholds and raise concerns about budget impact, even after negotiation of financial/commercial agreements. Some countries have special appraisal and reimbursement mechanisms that allow flexibility in evaluation of rare disease treatments [2], such as leniency in critical assessment/review of evidence, flexibility in base-case assumptions for economic modelling and broader consideration of value. Despite these adapted approaches, some treatments for which substantial clinical benefit or large gains in quality-adjusted life-years are estimated are still associated with major uncertainties, which means they cannot be recommended for use or full reimbursement in a health system. Alongside this, the high unmet need, childhood onset and severity of many rare diseases means there is often strong clinical, patient/parent and political lobbying to provide access to such rare disease treatments. In these situations, one solution to access could be an outcomes-based managed entry agreement (OBMEA)1 that seeks to manage the clinical and financial uncertainties [3, 4].

Outcomes-based managed entry agreements can be used with any form of health technology and occur at two levels for a range of purposes including resolution of uncertainties in clinical evidence, management of budget impact, and improved understanding of real-life and long-term effectiveness to optimise care:

Individual: ensuring only eligible patients receive treatment, sometimes with assessment of outcomes to determine treatment continuation (appropriate use) or linked to payment schemes (paying only if response achieved or refund if response not achieved)

Population: using coverage with evidence development (CED)/post-licensing evidence generation as a conditional coverage mechanism post-health technology assessment (HTA)/reimbursement that requires evidence collection to inform re-appraisal or pricing and reimbursement re-negotiations

However, even with prevalent conditions, implementation of OBMEAs has been found to be challenging [5, 6] owing to administrative burdens on health providers/payers and for population-based schemes, the insufficiency of data to inform re-appraisal and funding of data collection [7]. Pricing and reimbursement processes are the responsibility of the Ministry of Health nationally and/or a payer, which may have a “jurisdiction” that is national, relates to a region in the country or to a population covered by the health plan. Furthermore, these OBMEAs must be agreed alongside confidential commercial arrangements, thus the constructs of the OBMEAs are often kept confidential or published in local languages that have made sharing difficult. As a consequence, different agreements arise for the same treatment in different jurisdictions. This represents a challenge for the marketing authorisation holder (company/health technology sponsor), but for rare diseases this is also an important inefficiency as sharing of outcomes data across jurisdictions could create more robust evidence for re-appraisal [8, 9].

Work Package (WP) 10 of the European Commission-funded H2020 IMPACT HTA project has undertaken research to develop an appraisal framework suitable for rare disease treatments that includes consideration of when and how to implement OBMEAs. For this, IMPACT HTA WP10 drew on the European Commission-funded H2020 COMED project. The COMED systematic review of CED schemes [10] identified a range of challenges, many of which seemed transferable to rare disease treatments. We categorised these into issues relating to:

• Establishment (including purpose, stakeholder responsibilities, agreement on eligibility and continuation criteria, outcomes to be studied)

• Implementation (including research protocol, data collection infrastructure, monitoring to improve data quality and completeness, amendment processes, feedback mechanisms for physicians and patients, resourcing, ethical issues)

• Re-appraisal (including ability to implement a revised reimbursement decision).

Reckers-Droog et al. [10] suggested that future research should seek to deepen understanding of the challenges of OBMEAs for different health technologies and consider impacts for all stakeholders including clinicians and patients. Hence, IMPACT HTA WP10 organised workshops with all stakeholders involved in National Institute for Health and Care Excellence (NICE) Highly Specialised Technologies (HST) OBMEAs (called managed access agreements). These OBMEAs gather data from a variety of health system sources for treatments of ultra-rare conditions that are provided in a national specialised service in NHS England. Then EX undertook a desktop review of the national web-based registry system of the Italian Medicines Agency, AIFA [11]. This in-depth evaluation of all AIFA registries over a 15-year period identified 88 registries for rare disease indications and explored their form, evolution and impact.

This work from England and Italy showed some differences in approaches to OBMEAs, but with a commonality of issues in relation to clinician (physicians and pharmacists) burden of data entry and assessment. As a result, it was decided to review the implementation of OBMEAs for two specific rare disease treatments across a much broader range of countries. The purpose was to explore how different health systems had established and implemented both individual and population-based OBMEAs and identify good practices that could inform development of tools to support OBMEAs for rare disease treatments.

Methods

Given emerging findings in other parts of IMPACT HTA WP10 research, two different treatments designated as orphan medicinal products (OMP) by the European Medicines Agency were chosen as case studies: nusinersen and tisagenlecleucel. Nusinersen (Spinraza^®) has European Union regulatory authorisation for 5q spinal muscular atrophy (SMA) and is administered as an intrathecal injection with four loading doses in the first 2 months and then a maintenance dose every 4 months thereafter, continued dependent on response. Tisagenlecleucel (Kymriah^®) is a one-off CAR-T therapy with European Union regulatory authorisation for patients aged up to 25 years with refractory/relapsed B-cell acute lymphoblastic leukaemia or adults with relapsed/refractory diffuse large B-cell lymphoma. These provide contrasting cases of: non-cancer vs cancer; ongoing vs one-off treatment; babies/adults vs adolescents/adults; antisense oligonucleotide vs advanced therapeutic medicinal products; different HTA/reimbursement routes within organisations.

With a good understanding of the processes to undertake OBMEAs at NICE and AIFA, a template was developed to capture relevant information about OBMEAs in other jurisdictions. In February 2020, the template was sent to HTA/payer/academic experts in each country in the European Union, European Economic Area, Canada, Australia and New Zealand that we had worked with in the documentation of appraisal processes in another workstream of IMPACT HTA WP10. They were asked to complete the templates using publicly available information. We sent several reminder e-mails over the coming months, but this coincided with the emergence of the COVID-19 pandemic and thus we were sensitive to health system pressures. Therefore, we contacted a market access expert from each marketing authorisation holder to identify the countries where their products had been reimbursed and those that were likely to have an OBMEA and targeted repeated follow-ups to these countries.

The completed templates were reviewed by KF to clarify entries and reduce duplication. Each revised table was validated by the relevant HTA expert. The HTA/payer/academic experts (authors) that contributed detailed information were invited to two virtual workshops in autumn 2020 to discuss their entries in the revised tables, differences in processes, new initiatives that had been implemented to overcome previous challenges with OBMEAs and the potential for future collaboration in relation to OBMEAs for rare disease treatments.

Results

Data Returns

Nusinersen

In the data collection phase in summer 2020, it was recognised that the following countries had not authorised/reimbursed nusinersen or were unlikely to have an OBMEA: Croatia, Denmark, Estonia, Greece, Iceland, Liechtenstein, Luxembourg, Malta, New Zealand, Romania and Slovakia and their non-response was not pursued beyond one follow-up. Canada, Finland, Scotland, Sweden and Switzerland did not respond. In Scotland, nusinersen is reimbursed for SMA type 1 and from July 2019 it could be prescribed for SMA types 2/3 via the new ultra-orphan pathway. This agreement lasts for a period of up to 3 years while further evidence on effectiveness is generated, but no other details are available2. Canada, Sweden and Finland have decentralised systems, where the provinces/hospitals/councils negotiate pricing and reimbursement and any OBMEA, but these agreements are confidential.

For Australia, France, Germany, Hungary, Norway and Slovenia, we were informed that nusinersen was reimbursed, often with strict inclusion criteria, but without an OBMEA. According to the marketing authorisation holder, the Czech Republic does have a scheme that assesses effectiveness in children after 1 year of treatment and outcomes-based continuation criteria are applied, but no further details are available.

Templates returned from Austria, Portugal and Spain provided general responses about OBMEA construction and had few details for the specific treatment because the agreements are confidential. Austria made regional reimbursement decisions on nusinersen including for indications and any associated OBMEA, but it is known that one region collected motor milestones in the German SMArtCare registry3.

For the remaining countries, templates were returned showing CED (population-based) schemes in Belgium, England and the Netherlands (Table 1). Responses from Bulgaria, Ireland, Italy and Lithuania showed individual-based OBMEAs used to determine eligibility, continuation and, in some cases, total budget caps that are intended to be used in pricing and reimbursement renegotiations (Table 2).

Table 1.

OBMEAs for nusinersen: coverage with evidence development

Country Organisation	Belgium National Institute for Health and Disability Insurance (INAMI)	England National Institute for Health and Care Excellence (NICE)	The Netherlands Zorg Instituut (ZIN)
P&R indication sought	5q SMA	5q SMA Types I, II, III and pre-symptomatic patients genetically destined to develop SMA	5q SMA
P&R date	1/9/18	July 2019	1. ≤ 9.5 years old 1/8/18 (fully reimbursed) 2. > 9.5 years old 1/1/20 (conditional)
P&R special process	Conditional—Chapter IV of positive list (Joint negotiation in BENLUXA with Netherlands)	Conditional on OBMEA and commercial agreement organised by NHS England and Improvement (NHSE&I)	OBMEA possible for OMPs (Joint negotiation in BENLUXA with Netherlands) 1. Ultra-rare, financial agreement until 1/21, with extension, “informal OBMEA” initiated after formal CED for other sub-population 2. Conditional reimbursement with legal OBMEA
OBMEA population	5q SMA not needing permanent ventilation 1. ≤ 9.5 years old – 1st symptoms before 6 mos, disease < 26 weeks – 1st symptoms 6–20 mos, disease < 94 mos – Pre-symptomatic babies with genetic diagnosis and 2 or 3 SMN2 copies 2. > 9.5 years old	5q SMA Types I–III and pre-symptomatic excluding – Permanent ventilation/ tracheostomy – Intrathecal injection not feasible – Had spinal fusion surgery – Severe contractures that prohibit measurement of motor milestones – Who gained ability to ambulate independently, but are no longer able to (except children who lost independent ambulation in prior 12 mos)	1. ≤ 9.5 years old—informal OBMEA – 1st symptoms before 6 months, disease < 26 weeks – 1st symptoms 6–20 mos, disease < 94 mos – Pre-symptomatic babies with genetic diagnosis and 2 or 3 SMN2 copies 2. > 9.5 years old—Legal OBMEA Extensive exclusion criteria
OBMEA Duration	28 months (up to max of 3 years)	5 years, with minimum of 3 years’ data collection	1. Informal OBMEA planned to start—no end date 2. Legal OBMEA—7 years including HTA re-assessment (290 pats recruited in first 2 years)
Purpose of OBMEA	Resolve uncertainties – Number pats treated in real life number pats treated by SMA type – Age at symptom appearance – Disease duration at onset of Rx – Real-life efficacy/safety: mortality, ventilation, performance, discontinuation reasons – Long-term efficacy/safety – Evidence in populations with no determined added value to date	To resolve clinical uncertainties in NICE appraisal with – Clinical data collection – PRO data collection – Resource utilisation data collection	1. In reimbursed population (≤ 9.5 years old), informal OBMEA to evaluate real-life effectiveness, determine appropriate use and cost effectiveness 2. For conditional coverage (> 9.5 years old), CED, legal OBMEA to collect enough evidence to determine if it is clinically and cost effective compared to no use (historic registry data)
Use in re-appraisal	Re-appraisal for P&R re-negotiations to convert to standard reimbursement or change in target population etc or to extend OBMEA	– 1 year review for a specific population (type III, non-ambulant) – Reassessment in year 4 using at least 3 years of OBMEA data collection and any additional evidence from other sources. – Revised commercial agreement – Decision by year 5 to reimburse or not, no further OBMEA	1. Continuing process to refine start and stop criteria, collecting long-term follow up data and if it does not work as well as expected, reassessment can be initiated with potential to change advice to not reimburse or renegotiate price 2. Legal requirement for P&R re-negotiations
Stakeholders involved	– INAMI Commission for Reimbursement proposed OBMEA to MAH, who accepted and developed OBMEA with MoH and INAMI MEA taskforce – Clinicians	– NICE can propose OBMEA according to clear criteria, developed in collaboration with MAH, NHSE&I, registry holder, clinicians and patient groups – Patient groups liaise with patients to explain OBMEA, feedback issues, input to data monitoring discussions – Other members of the Managed Access Oversight Committee (MAOC) clinicians from 4 expert centres, academics responsible for development of PROM – National Clinical Panel assesses the patients included against the eligibility and continuation criteria on an annual basis and discuss other clinical issues that arise such as difficult cases requiring clinical judgement	– ZIN appraisal committee proposed both the informal and the legal OBMEAs to Minister of Health – Research group send application with research protocol to ZIN for review and approval this forms basis of legal agreement – Clinical expert centre, patient group, health insurers, bureau of university hospitals, pharmacists (and for legal OBMEA MoH)
Treatment stopping criteria	Permanent ventilation	– Permanent ventilation (≥ 16 h/day for 21 consecutive days in the absence of acute reversible infection) or requirement of insertion of permanent tracheostomy. – Total worsening in outcome score corroborated by two consecutive measurements. A scaled equivalent of these losses would apply if a domain was unmeasurable/not suitable: > 2 points on horizontal kick or 1 point on other HINE scores excluding voluntary grasp > 4 points on the CHOP INTEND scale > 3 points on the Revised Hammersmith scale – Inability to regain ambulation within 12 months of nusinersen initiation – Inability to give intrathecal administration because of spinal fusion surgery	1. 9.5 years old reimbursed population, informal OBMEA—not known yet, will be evaluated in the informal OBMEA 2. 9.5 years old CED, legal OBMEA– not stipulated, but important to establish if reimbursement is agreed after OBMEA
Data sources	– INAMI and Insurance Funds have established a bespoke registry with healthdata.be that links into the internationally coordinated SMA registry Data from the registry will be combined with INAMI reimbursement data – MAH can also collect data through other routes	– Completion of ongoing trials – SMA REACH registry (organised by leading children’s hospital) – NHS Blueteq (System for high-cost drugs) – PROMs being developed	1. Dutch registry 2. Dutch registry developed for use by clinicians in the expert centre and data will be compared with historical Dutch cohort
Data collected	To resolve stated uncertainties, details confidential	– Nusinersen (use, first dose, dose, combination therapy, parental perceived benefit) – Survival, need for ventilation (+ respiratory events) – Motor function (various measures) – Scoliosis – Fractures – Nasogastric tube, gastrostomy placement – For each group a “primary” endpoint is defined in the DCA and many other endpoints as well	1. ≤ 9.5 years old, reimbursed population, informal OBMEA (data collection still under development) 2. > 9.5 years old, CED, legal OBMEA A lot of data to assess clinical and cost effectiveness with specific research questions compared to historical control. Primary endpoints: If HFMSE score ≥ 5, decrease in HFMSE over 4 years follow-up. MCID—75% reduction in the average decrease compared to the historical cohort. (HFMSE decreases by an average of 0.8 points per year in historical cohort, so there is a clinically relevant difference if the HFMSE score decreases by an average of ≤ 0.2 points per year). IF HFMSE < 5, RULM is the primary outcome measure. Secondary efficacy measures: RULM, EQ-5D, SF-36, PedsQoL, SMA-FRS, fatigue (measured with the ES9HPT, ESBBT and ESWT), Safety: side effects, serious side effects Economics: budget impact and cost effectiveness.
Data Analysis Plan	?	Developed by MAH	1. Expert centre 2. Research group from expert centre
Frequency of data collection	Annually, with an initial delay: so 2020, 2021, 2022	Twice per year (at least 4 months apart) to coincide with 6-monthly follow-ups or 4-monthly administrations	1. ≤ 9.5 years old, informal OBMEA—not known yet 2. > 9.5 years old, legal OBMEA—every 3–6 months
Interim data reviews	None	SMA REACH (registry) 6-monthly – Summary reports to MAOC to monitor data completeness and case ascertainment. – Pseudo-anonymised individual reports to Clinical Panel to monitor eligibility and stopping rules MAH provides annual summary data At end of year 1 provide evidence submission to NICE re non-ambulant type III	1. Aggregated public data reported in ZIN annual report “monitoring orphan drugs in practice” 2. Group meets 6-monthly to review status of research—no. of patients treated, no. on waiting list, no. stopped treatment etc and reported in annual report about use of OMPs
Data quality process	No	– Mandatory fields to be completed – Registry holder undertakes site monitoring of data entry and remote monitoring of completeness. – MAOC checks case ascertainment figures from another NHS data source (Blueteq)	Organised by the research centre
Funder for data collection	INAMI for the new SMA registry MAH	MAH	1. No standard system for informal OBMEA 2. MAH
Funder for data processing	MAH	MAH and SMA REACH	1. No standard system for informal OBMEA 2. MAH

CED Coverage with Evidence Development, CHOP INTEND Children`s Hospital of Philadelphia Infant Test of Neuromuscular Disorders, ES9HPT Endurance Shuttle Nine Hole Peg Test, ESBBT Endurance Shuttle Box and Block Test, ESWT Endurance Shuttle Walk Test, FRS Functional Rating Scale, HFMSE Hammersmith Functional Motor Scale Expanded, HINE Hammersmith Infant Neurological Examination, INAMI National Institute for Health and Disability Insurance, MAH Marketing Authorisation Holder, MAOC Managed Access Oversight Committee, mos months, MoH Ministry of Health, MCID minimum clinically important difference, NHIF National Health Insurance Fund, NHSE&I NHS England and Improvement, NICE National Institute for Health and Care Excellence, OBMEA Outcomes-Based Managed Entry Agreement, OMP orphan medicinal product, P&R pricing and reimbursement, PROM patient-reported outcome measure, Rx treatment, RULM revised upper limb module, SMA spinal muscular atrophy, ZIN Zorg Instituut

Table 2.

OBMEAs for nusinersen: individual agreements

Country Organisation (respondent if not from organisation)	Bulgaria National Health Insurance Fund (TB, Founding President of National Council on Prices and Reimbursement of Medicinal Products)	Ireland Health Service Executive (HSE) (AOL, NCPE)	Italy Italian Medicines Agency (AIFA) (EX, formerly AIFA)
P&R indication sought	5q SMA	Children aged < 18 years with genetically confirmed SMA types I, II or III	5q SMA types I–III No age limitation
P&R date	29/9/19, inclusion in NHIF list 1/1/20	December 2017 following HTA not deemed to be cost effective and further pricing negotiations	28/9/17
P&R special process	Prescription for individual patients according to protocol approved by special commission in NHIF	Ultra-rare Conditional reimbursement according to protocol published by MoH	Fund for innovative drugs achieved “full” innovation score valid for 3 years (28/09/2017–17/09/2020), with P&R contract re-negotiations annually http://www.aifa.gov.it/sites/default/files/6-Spinraza_v1.0.pdf
OBMEA population	Children with 5q SMA aged < 18 years excluding those with - Anamnesis of peripheral or CNS disease - Severe scoliosis - Coagulation disorders - Hypersensitivity with manifestations of severe anaphylactic reactions to active substance or excipients - Those in a clinical trial for SMA Patients cannot receive treatment with other drugs paid by NHIF for the same indication, which are not included in the protocol	National Drugs Management Protocol states: Children aged < 18 years with genetically confirmed SMA types I, II or III excluding those with contraindications and - Patients with other life-limiting conditions - SMA type 0 - Clinical and genetic diagnosis of SMA not fulfilled - Current participation in a clinical trial with an investigational gene therapy for SMA - Comorbidities that might preclude lumbar puncture Patients prioritised on the basis of clinical need and the potential for treatment benefit	Genetically confirmed 5q SMA types I–III with no more than 4 copies of SMN2 gene. SMN1 for naive patients
OBMEA duration	3 years	28 months? Planned to start in August 2019, but some delays in execution	1 year and reviewed annually, often beyond innovation status, for many years 1st PR negotiation 28/09/17 https://www.aifa.gov.it/sites/default/files/Determina_1611-2017_Spinraza.pdf PR re-negotiation 29/9/18 https://www.gazzettaufficiale.it/eli/id/2018/09/27/18A06132/sg
Purpose of OBMEA	Obligatory requirement for some form of MEA as condition for positive drug list This OBMEA to ensure appropriate use and treatment discontinuation	Resolve uncertainties about long-term benefit by ensuring appropriate use including treatment discontinuation	Manage clinical uncertainty by ensuring appropriate use including treatment discontinuation
Use in re-appraisal	(OB)MEAs valid for 3 years until reimbursement re-negotiation	As defined in protocol, originally planned for review in November 2021	Part of the annual review of P&R contract
Stakeholders involved	NHIF and MAH	- Healthcare payer: primary care eligibility and reimbursement service - HSE medicines management programme (HSE MMP) to review individual applications for treatment against protocol (developed with clinicians)	- AIFA establishes registry for use by hospitals, can access anonymised data to produce reports - MAH funds registry - Regions: authorise prescribing centres - Health managers authorise clinicians and pharmacists - Clinicians/pharmacists: dispensing and access depends on entries into the registry for initiating and continuing treatment
Treatment stopping criteria	Assessment of performance status according to HFMSE, CHOP INTEND and HINE for SMA type 1 and RULM for SMA type 2 and re-evaluation of exclusion criteriaa	Differentiated for SMA types, based on motor/mobility scales, respiratory and patient/clinician viewsb	None
Data sources	Hospitals	Hospital clinicians (paper based), submitted to HSE-MMP for review	Drug-specific national web-based registry
Data collected	Confidential	As outlined in protocolb: - Baseline patient characteristics - Treatment information - Outcomes at 12 months to determine treatment continuation, different for each SMA type	Baseline patient characteristics Inclusion criteria: Patient with a genetically confirmed diagnosis of SMA 5q (mutations in the SMN1 gene), alleged SMA phenotype (types 1, 2, 3a or 3b) For patients diagnosed with SMA type 2, the forced vital capacity measurement to be collected Tests: CHOP INTEND, Motor milestones HINE, or HFMSE Information about whether patient has already been treated and appears in early access registry (as for all the AIFA registries) Date of nusinersen administration and dose First assessments after the loading dose and each four months of treatment - Clinician overall assessment - % change in score (CHOP INTEND, motor milestones HINE or HFMSE) - Disease progression, patient death, drug toxicity Treatment continuation decision End of treatment, date, reason, end-points (Electronic Supplementary Material)
Data analysis plan	NHIF	HSE-MMP	AIFA https://www.aifa.gov.it/en/-/attivazione-del-registro-spinraza-12-12-2017-
Frequency of data collection	Confidential	Annually	Continuum in clinical practice (medicine cannot be dispensed for any visit until data entered)
Interim data reviews	Depends on OBMEA	Not stipulated	Annually, and specifically after end of innovation status
Data quality process	Confidential	Not stipulated	Standard summary graphs and charts can be produced and sent to stakeholders for review
Funder for data collection	No special funding	Not funded	MAH
Funder for data processing	Confidential	Not funded	AIFA

Country Organisation (respondent if not from organisation)	Latvia NHS of Latvia (payer) [unvalidated]	Lithuania National Health Insurance Fund (AV, prior head of HTA)	Poland MoH/National Health Fund (AZ, AOTMiT) (unclear if OBMEA proposals implemented)
P&R indication sought	5q SMA as per SPC indication	5q SMA as per SPC indication	5q SMA as per SPC indication
P&R date	May 2019	First case reimbursed July 2017	January 2019
P&R special process	Special state programme for OMPs	Special process for reimbursement of OMP and medicines for treatment of very rare conditions is regulated by special order of Minister of Health. The order specifies decision making body (Very Rare Conditions Committee) and the process flow	Positive drugs list, drug programme B.102
OBMEA population	Genetically confirmed diagnosis of SMA: - Pre-symptomatic genetically confirmed SMA, 2/3 SMN2 gene copies - SMA I, 2 copies of SMN2 gene, up to 6 months old - SMA I, 3 copies of SMN2 gene, up to 8 months old - SMA II/III, SMN2 gene copy number ≥ 2, up to 12 years old Excluding those: - Who need breathing support (including CPAP or supplemental oxygen) to provide spo2 > 95% - With significant contractures that would prevent full use of motor function rating scale - With severe scoliosis (radiographic Cobb angle > 40°) Patient or guardian understands start/stop treatment criteria and agrees with treatment and multidisciplinary state of health evaluation regimec	Excluded non-ambulant SMA type 3 Then, based on individual case submission	Pre-symptomatic and symptomatic patients with diagnosed SMA 5q confirmed by genetic testing. Patients are approved by the coordinating team in the SMA treatment programme Most important exclusion criteria: - Scoliosis, which makes intrathecal injection impossible - Deterioration (different level in various scales) in the scale appropriately selected for age and type of sma (chop intend, hine or hfmse) - Drainage of the cerebrospinal fluid - Pregnancy
OBMEA duration	Confidential	3 years	2 years in line with P&R contract
Purpose of OBMEA	Appropriate use and discontinuation of treatment	Appropriate use: hospital justifies to NHIF for each treatment cycle of each patient that there is no disease worsening since the previous treatment cycle	Proposal from AOTMiT to evaluate uncertainty in clinical efficacy, especially long term (no data for SMA 0, IV and lack of randomised research for asymptomatic patients) with appropriateness and continuation criteria
Use in re-appraisal	In reimbursement re-negotiations, particularly evaluating number of patients treated	P&R renegotiation at 3 years or earlier if utilisation cap exceeded	Unknown
Stakeholders involved	- NHS develops criteria for the agreement and performs analysis - Childrens’ University Hospital	- NHIF may develop OBMEA with advice from clinical experts about how to assess effectiveness - Hospital: submit information for each patient before each maintenance cycle with physician’s conciliumd about patient response based on HFMSE	- AOTMiT Transparency Council can recommend OBMEA to MoH - MoH Economic Commission, which includes experts from the NHFe, negotiates with MAH and the P&R contract may include and OBMEA but this is confidential
Treatment stopping criteria	By decision of multidisciplinary team at any time if any of the following criteria are met: 1. Reduction in age-appropriate motor function rating compared with baseline; at 1 year, then every 6 months 2. Invasive or non-invasive ventilation > 16 h per day, for 21 consecutive days, without acute infectious disease 3. Contraindications to lumbar puncture or technically impossible to perform 4. Patient monitoring and international SMA care standards are not followed appropriately 5. Lack of equality between patients and patients’ relatives (non-adherence or insufficient patient nutrition provided) 6. Significant adverse drug reactions 7. Patient or carer requests to discontinue Criteria for starting and stopping treatment can be updated at any time according to latest scientific evidence	Physicians’ concilium in treating hospital: if individual’s disease is worsening, treatment should be discontinued (opinion, data not shared)	AOTMiT proposition was to discontinue if no improvement from baseline by at least 1 point on CHOP INTEND, HINE or HFMSE on 2 assessments taken 4 months apart
Data sources	Childrens’ University Hospital	NHIF from hospital	Clinicians
Data collected	Medicine utilisation data	Information about starting and stopping of treatment for each patient as supplied by hospital (data and opinion from physicians’ concilium)	Patient data to inform eligibility and treatment continuation decisions
Data analysis plan	None	None	None
Frequency of data collection	After 1 year, then every 6 months	Before each maintenance cycle (4-monthly)	Proposed before each maintenance cycle (4-monthly)
Interim data reviews	Unknown	NHIF produces annual report on budget expenditure for ultra-rare conditions by therapeutic area	Unknown
Data quality process	Unknown	NHIF has routine checks on data quality	Unknown
Funder for data collection	NHS	NHIF	Unknown
Funder for data processing	NHS	NHIF	Unknown

AIFA Italian Medicines Agency, AOTMiT Agency for Health Technology Assessment and Tariff System, CHOP INTEND Children’s Hospital of Philadelphia Infant Test of Neuromuscular Disorders, CNS central nervous system, CPAP HFMSE Hammersmith Functional Motor Scale Expanded, HINE Hammersmith Infant Neurological Examination, HSE MMP Health Services Executive Medicines Management Programme, MEA , MAH marketing authorisation holder, MoH Ministry of Health, NHIF National Health Insurance Fund, NHS National Health Service, OBMEA outcomes-based managed entry agreement, P&R pricing and reimbursement, RULM revised upper limb module, SMA spinal muscular atrophy

^aNHIF requirements: Section B on page 4: https://www.nhif.bg/get_file?uuid=9AECFAA6771D62DBE05400144FFB42AE (Bulgarian)

^bhttps://www.hse.ie/eng/about/who/acute-hospitals-division/drugs-management-programme/protocols/hse-nusinersen-spinraza-protocol.pdf

^chttp://www.vmnvd.gov.lv/uploads/files/5d0a2aa94a54d.pdf and http://www.vmnvd.gov.lv/lv/veselibas-aprupes-pakalpojumi/retas-slimibas

^dConvocation of 3 or more physicians to give advice

^eNarodowy Fundusz Zdrowia

An initial response was received from Latvia that documented an individual-based scheme, but because of health system pressures, it has not been possible to validate the data summary. In Poland, it is unclear if the scheme proposed by the HTA body was implemented. These are shown in Table 2 in italic to enable comparisons to the individual-based schemes that were implemented, and for which there is validated information.

Tisagenlecleucel

In the data collection phase, it was determined from respondents or the marketing authorisation holder that at that point in time, Bulgaria, Estonia, Hungary, Ireland, Latvia, Lithuania, Slovakia and Slovenia had not authorised/reimbursed tisagenlecleucel and that Croatia, Cyprus, Denmark, Greece, Iceland, Liechtenstein, Luxembourg, Malta, New Zealand and Romania did not have OBMEAs. Contacts in the Czech Republic, Finland and Switzerland did not respond. For Portugal, a general response about the process was received indicating that any agreement would be made with INFARMED and the marketing authorisation holder, but that there was a preference for simple discounts.

Norway, Scotland and Sweden had reimbursed tisagenlecleucel through standard appraisal and reimbursement processes without need for an OBMEA. The Netherlands only uses population-based OBMEA for non-cancer OMPs, but can establish an individual-based agreement to ensure appropriate use. It did not use this for tisagenlecleucel, but gave reimbursement for acute lymphoblastic leukaemia only. In Poland, there were originally applications for emergency access for a limited group of patients, then applications for reimbursement were made.

Jørgensen et al. [12] stated that in Germany during the initial 12-month free-pricing period, the marketing authorisation holder made arrangements with the health insurers, covering 60% of the eligible population, for an outcomes-based rebate for patients who died within a certain period. After the early benefit assessment, the reimbursement mechanism has reverted to a simple confidential discount.

In Austria, for tisagenlecleucel, there was a national agreement on the reimbursed indication agreed by the national specialist group of haematologists4 and outcomes data are documented in the European Society for Blood and Marrow Transplantation. Reimbursement decisions (including any OBMEA) were agreed in regions and are confidential.

In Canada, CADTH recommended that tisagenlecleucel be provided via interprovincial agreements with clear eligibility criteria. It was recommended that standardised outcomes data be collected in a pan-Canadian registry to generate real-world evidence for consideration in future assessments to assess longer term effectiveness, safety and cost effectiveness5. No further details are published.

Templates were returned showing CED schemes in Australia, Belgium and England and an individual-based scheme in Italy. Publications and public reimbursement information described CED in France and an individual-based scheme in Spain. These are summarised in Table 3.

Table 3.

OBMEAs for tisagenlecleucel

Country Organisation (respondent if not from organisation)	Australia Medical Services Advisory Committee (Pharmaceutical Benefits Advisory Committee)	Belgium INAMI	England NICE
P&R indication sought	ALL, DLBCL	ALL, DLBCL as per SPC indication	ALL, DLBCL as per SPC indication
P&R date	April 2019	1/6/19	ALL, 16/11/18; DLBCL, 1/2/19
P&R special process	None	Conditional, Chapter IV of positive list	Cancer Drugs Fund
OBMEA population	ALL indication as per regulation (DLBCL reimbursed November 2019)	ALL, DLBCL as per SPC indication	ALL, DLBCL, SPC indication with extensive eligibility criteria (Electronic Supplementary Material)
OBMEA duration	2 years	2 years (could be extended up to 3 years)	ALL: 4 years 7 months; DLBCL: 4 years
Purpose of OBMEA	Manage clinical, economic and financial uncertainties in the March 2019 submission for ALL, particularly - Patient not successfully infused - Patient receives second dose - Patient numbers overestimated - Durability of response - Proportion receiving SCT - Need for ongoing IVIG (Co 3 years, vs 6 years from MoH) CED	Resolve uncertainties relating to efficacy/safety (particularly longer term), therapeutic added value - Success rate of production and turn-around time - Optimal selection of patients (high chance of response, low risk of severe toxicity) - Number of patients treated in real life - OS, PFS 6, 12 and 20 months (DLBCL only) - Number who had SCT without disease progression (i.e. Bridge-to-transplant) - Medical resource utilisation - Number receiving tocilizumab and duration - Number not infused and reason - Is capacity of accredited centres sufficient? CED	Resolve clinical uncertainties ALL: - Immaturity of data that do not fully support the curative nature of tisagenlecleucel - Rate of subsequent sct - Need for IVIG and duration of treatment DLBCL: - Immaturity of data (OS) to support the curative nature of tisagenlecleucel - Need for IVIG and duration of treatment CED
Use in re-appraisal	Re-appraisal of clinical and cost effectiveness and budget impact	Re-appraisal for price re-negotiations or alteration in eligibility and other criteria	Re-appraisal of clinical and cost effectiveness after data collection period (no extension of OBMEA permitted)
Stakeholders involved	- MAH, MSAC, MoH - Highly specialised tertiary centres - Australian BMT Registry	- INAMI Commission for Reimbursement proposed OBMEA to MAH, who accepted and developed OBMEA with MoH and INAMI MEA taskforce - Clinicians - Clinical centres accredited by MAH	- Signatories: MAH, NICE, NHSE&I, PHEa - Non-signatory: BSBMTCTb - Phased approach in accredited centres with national clinical panel to judge eligibility against eligibility criteria
Data sources	Completion of ongoing trial Health service data Australian BMT Registry	Completion of ongoing trials Clinician completed forms INAMI reimbursement data	MAH: ongoing phase II open-label trials and phase I study of lymphomas BSBMTCT registry NHS SACT dataset
Data collected	Minimum dataset: - Indication for use of CAR-T - PFS - Complications, use of high-cost medications, late-onset aes, saes - Use and duration of IVIG - Reinfusion with any CAR-T Sub-group analyses of patients infused with non-optimal cell numbers	To resolve stated uncertainties, details confidential	From clinical trials: OS; and for DLBCL also PFS and IVIG use From BMT register (if data sharing is possible) ALL, nNumber and time to stem cell transplant From NHS datasets (including SACT and other data) DLBCL: OS, IVIG use (if possible)
Data analysis plan	?	Confidential	- Data analysis plan specified in data collection agreement - Methodologies for analyses from NHS SACT database are publishedc
Frequency of data collection	Registry enters info for each patient as treated, including those unsuccessfully or not infused	Yearly (allowing for delay in data collection, thus end of 2021, end of 2022)	- Clinical trials, as per protocol - SACT data uploaded 2-monthly by NHS trusts - Registry data uploaded after patient has transplant and gives consent
Interim data reviews	No	No	- MAH asked for updates on clinical trials - SACT and BSBMTCT aggregated data reviews (with MAH, NICE, NHSE&I, PHE) at 9 and 12 months and annually of OS
Data quality process	?	None	- Clinical trial standard processes - NHS systems map local data to SACT information standard based on NHS data dictionaryd. PHE employ data liaison officers to review data and chase trusts for missing/erroneous data for CDF data reports. All CDF reports are subject to a review by NICE, NHSE&I and PHE prior to sharing externally or publication - BSBMTCT perform regular checks and validation tests on registry
Funder for data collection	MAH	Insurance funds (MAH for their bespoke analyses)	- MAH funds ongoing clinical trial - NHSE&I, SACT, Blueteq, registry
Funder for data processing	MAH	MAH	- MAH funds analysis of clinical trials - NHSE&I fund analysis of SACT, Blueteq, registry data

Country Organisation (respondent if not from organisation)	France Haute Autorité de Santé (partnere, HAS reportsf,g)	Italy AIFA (EX, formerly AIFA)	Spain MoH (JE, Jorgensen 2020)
P&R indication sought	ALL, DLBCL as per SPC indication	ALL, DLBCL as per SPC indication	ALL: DLBCL as per SPC indication
P&R date	July 2019	13/8/19	January 2019
P&R special process	No	Fund for innovative oncological drugs (Law 232, 2016), fully innovative (Innovation status valid 13/08/2019–12/08/2022—with innovativeness confirmation in July 2020)	No
OBMEA population	ALL, DLBCL as per SPC indication	ALL, DLBCL with clear eligibility criteria (Electronic Supplementary Material)	ALL: DLBCL with extensive eligibility criteria stated in national clinical assessment (Electronic Supplementary Material)
OBMEA duration	Unknown	Linked to P&R contract (18 months)	Confidential
Purpose of OBMEA	Resolve uncertainties about efficacy, safety and complexity of treatment process CED	Manage clinical uncertainty based on outcome metrics Individual: “payment at result” (staged payments) 3 payments at baseline, 6 months, 12 months	- Ensure equitable, safe and efficient use and monitor patients - Provide long-term evaluation of outcomes in clinical practice to resolve uncertainties that remain after clinical trials Individual: outcomes-based staged payments
Use in re-appraisal	Annual HTAs to reassess improvement in the clinical benefit (SMR/ASMR rating)	Ongoing, part of the regular P&R contract reviews	P&R renegotiations
Stakeholders involved	HAS Hospitals Registry holder MAH	- AIFA hosts national medicines registry, for PaR, MAH accesses anonymised individual data to verify the payment - Pharmacists responsible for payments in case of PaR. - Regions authorise centres and health managers authorise clinicians and pharmacists (dispensing dependent on registry entry by clinician)	MoH National Pricing Committee outline data to be collected by Regions with agreement from MAH Designated hospitals responsible for administering CAR-T
Data sources	- National CAR-T registry data platform hosted by Lymphoma Academic Research Organisation (LYSARC) - Ongoing clinical trials and PAES - Patients who received treatment through early access ATU scheme	AIFA national web-based registry for medicines, data collection determined from AIFA Technical Scientific Committee report and produces reports based on anonymised data	VALTERMED national web-based registry system launched in November 2019 to pilot the CAR-Ts, but not fully functioning by early 2021, hence MAH organising data collection
Data collected	(On all patients eligible for treatment, not only those actually treated) Patient characteristics Time from failure of last treatment to apheresis Tisagenlecleucel production time Therapeutic strategies before and after injection (e.g. tocilizumab, IVIG) Key outcomes at 28 days, 100 days, 6 months and then 6 monthly - Survival - Remission status - Disease progression - AEs Reasons for treatment failure and subsequent management including re-injection	ALL: Diagnosis: information on relapse, first diagnosis date, genetic risk factors Treatment: leukapheresis date, expected infusion date, weight, infusion (or reason for not infusing), reason for delay of administration Immediate follow-up: chemotherapy regimen, need for bridging therapy Follow-up: disease response status, other ALL treatments, AEs End of follow-up: various outcomes about cause including death DLBCL: Diagnosis: stage, International Prognostic Index, first diagnosis date Treatment: leukapheresis date, expected infusion date, date of infusion (or reason for not infusing), reasons for delay of administration Immediate follow-up: chemotherapy regime, need for bridging therapy Follow-up: disease response, other anti-lymphoma treatments, AEs End of follow-up: various outcomes about cause including death (Electronic Supplementary Material)	Patient characteristics Disease characterisation Leukapheresis, CAR-T production Administration of tisagenlecleucel Post-infusion and long-term safety monitoring Disease relapse/progression Response evaluation (ALL), complete response with early negative residual disease maintained for at least 18 months due exclusively to treatment with tisagenlecleucel
Data analysis plan	Part of HTA process	Confidential	Unknown
Frequency of data collection	Data reported by hospitals quarterly	Continuum in clinical practice	Continuum in clinical practice
Interim data reviews	Quarterly reports from the LYSARC registry	Depends on duration of contracts for P&R (18 months) and innovation status (12 months)	On VALTERMED, regions intended to have access to their own aggregated data for review. National reports on utilisation
Data quality process	Within registry platform	Probably, but process unclear	Intended registry platform, built-in Currently MAH responsible
Funder for data collection	MAH	MAH	Regional authorities/hospitals
Funder for data processing	Registry holder/MAH	AIFA	MoH

AE adverse event, AIFA Italian Medicines Agency, ALL acute lymphoblastic leukaemia, AOTMiT Agency for Health Technology Assessment and Tariff System, ATU authorisation for temporary use, BMT bone marrow transplant, BSBMTCT British Society of Blood and Marrow Transplantation and Cellular Therapy, CED coverage with evidence development, DLBCL diffuse large B-cell lymphoma, HAS Haute Autorité de Santé, INAMI National Insurance for Health and Disability Insurance, IVIG intravenous immunoglobulin, LYSARC Lymphoma Academic Research Organisation, MAH marketing authorisation holder, MoH Ministry of Health, MSAC Medical Services Advisory Committee, NHIF National Health Insurance Fund, NHSE&I NHS England&Improvement, NICE National Institute for Health and Care Excellence, OBMEA outcomes-based managed entry agreement, OS overall survival, PAES post-authorisation efficacy study, P&R pricing and reimbursement, PFS progression-free survival, PHE Public Health England, PRO patient-reported outcomes, SACT systemic anti-cancer therapy, SAE serious adverse event, SCT stem cell transplant, SPC Summary of Product Characteristics, VALTERMED VALor TERapeutico en la Practica Clinica Real de los MEDicamentos de Alto Impacto Sanitario y Economico

^aDLBCL only

^bALL only

^chttp://www.chemodataset.nhs.uk/nhse_partnership

^dSystemic Anti-Cancer Therapy Data Set (datadictionary.nhs.uk)dSystemic Anti-Cancer Therapy Data Set (http://www.datadictionary.nhs.uk)

^eHospinomics IMPACT HTA Partner, member of HAS CEESP

^fhttps://www.has-sante.fr/upload/docs/application/pdf/2018-12/kymriah_ldgcb_pic_ins_avis3_ct17238.pdf

^gKYMRIAH_LAL_12122018_AVIS_CT17202 (has-sante.fr)

Country Comparisons Reimplementing OBMEAs

Every jurisdiction has processes in place to develop commercial agreements/financial Managed Entry Agreements (MEA) within their pricing and reimbursement/commissioning process, many of which have a clear legal underpinning. These commercial agreements are highly confidential and generally only involve the marketing authorisation holder and the payer/Ministry of Health and for our case studies, range from simple discounts to budget caps (Belgium, Bulgaria, Latvia [unvalidated], Lithuania, Spain).

Despite the confidentiality of these commercial agreements, public information was available for many jurisdictions to populate our OBMEA templates for the two case studies. As shown in Tables 1, 2, and 3, information was available about the rationale and design of the OBMEA in different countries covering topics that elucidate the purpose of the OBMEA, population given access, duration of agreement, stakeholder involvement, data to be collected and reporting mechanisms. These elements are presented in the following sections with augmentation of intelligence from the HTA expert workshops.

Establishing the OBMEA

Some countries have standing committees that oversee the establishment of all OBMEAs (such as Belgium) inviting in clinical/appraisal experts relevant to the treatment. In England, the case studies come from two separate programmes within NICE, the Cancer Drugs Fund and the HST programme. They had different processes for the establishment of OBMEAs, thus the form of data collection agreements is slightly different for the case studies (but rationalisation of these OBMEA processes is underway for the future).

Outcomes-based managed entry agreements are sometimes limited to specific groups of medicines such as (high-cost) OMPs or those with conditional or exceptional marketing authorisations, nusinersen was considered via this route in The Netherlands. In some countries, OBMEAs are mandated for all medicines in a certain category, for example, in Italy those with a full innovation status must have a national registry that at least determines appropriate use (guiding eligibility for treatment and continuation criteria), as was the case for nusinersen, whereas tisagenlecleucel was the first product to undergo the “Payment at Results” OBMEAs in AIFA. In other countries, the use of an OBMEA for nusinersen and tisagenlecleucel was considered on a case-by-case basis to support appropriate use in individual patients and/or data collection to resolve clinical (and economic) uncertainties.

The OBMEAs for nusinersen were initiated between 2017 and 2020 and as shown by Whittal et al. [13] more evidence from studies in different sub-populations, clinical study follow-ups and real-world effectiveness might have been available for the later reimbursement decisions. This could have potentially resolved some uncertainties and impacted their need for OBMEA. For tisagenlecleucel, decisions were made at very similar times between January and August 2019.

Form of OBMEAs

In most cases, the purposes of the OBMEAs for nusinersen and tisagenlecleucel were clear with specific clinical and/economic uncertainties named relating to population characteristics, treatment administration, real-life and long-term efficacy, expensive concomitant or future treatments, and healthcare resource utilisation data. However, the link to appraisal reports, which would have been expected to be the documented source of the uncertainties, was not always clear.

For nusinersen, the individual-based OBMEAs in Table 2 show that they were generally used to collect outcomes to ensure “appropriate use” and determine treatment continuation. The table shows that even for these simpler forms of OBMEAs, a range of outcomes needs to be collected.

Only Italy and Spain implemented the more complex individual-based OBMEA that used individual outcomes to inform a staged payment model for tisagenlecleucel (Table 3). The population-based schemes in our case studies were illuminating because they included many elements of the individual-based schemes (with strict appropriateness criteria) but they went further to require data aggregation for re-analysis at a specific point in the future (Tables 1 and 3).

OBMEA Population and Duration

The reimbursed populations and inclusion criteria for the OBMEAs differ somewhat across countries. For nusinersen, Belgium simply refers to the regulatory authorised indication. Others state a variety of exclusion criteria that relate to ability to administer treatment (nusinersen), treatment stage and potential for response (tisagenlecleucel). These features are likely to feature in the clinical judgement of patient suitability in Belgium, even if not explicitly stated.

The differing age (and thus stage) restrictions for nusinersen probably arise from different value judgements made by appraisal committees about the extension of evidence from small trials in symptomatic young children to pre-symptomatic children and older patients. Two countries are of particular note.

As the indication for reimbursement sought to include all sub-types of SMA, this meant that in England, nusinersen was evaluated through the NICE Technology Appraisal programme. However, if a restriction had been made to type I, it would have probably been evaluated through the HST route (as was the case for onasemnogene abeparvovec). In the HST programme, a different decision-making criteria framework is used, including higher willingness-to-pay thresholds. In which case, an OBMEA may not have been needed, but access would have been restricted to specific patients.

As a pilot in the BENELUXA initiative, Belgium and the Netherlands published a joint assessment report on the clinical and cost effectiveness of nusinersen. This informed a joint pricing negotiation, but each country implemented its own OBMEA as shown in Table 1. In the Netherlands, as the assessment report had identified that clinical research only included patients aged up to 9.5 years and that evidence was not transferable to older patients (SMA types II and III), different reimbursement decisions were made for sub-populations. For those aged 9.5 years and older, a legally based OBMEA is ongoing, whereby the health system will pay for nusinersen and its administration and the marketing authorisation holder must pay the costs of research to collect data to resolve stated uncertainties. For children aged under 9.5 years, reimbursement was agreed but, using a new arrangement for selected OMPs that seeks to evaluate real-life effectiveness and optimise treatment, an informal OBMEA (that does not involve the marketing authorisation holder) will start in 2021. In contrast, Belgium requested data collection on the entire population, but has no mandate to ensure these data are collected.

In all countries, treatment and data collection for both cases are assigned to a few expert centres that generally require some type of authorisation or accreditation. Only the Netherlands appears to have provided a sample size estimate for the legal OBMEA. In 2019, estimates were made of initially 170 and then 290 patients, with a recruitment period of 2 years. In February 2021, 300 patients had been screened and two-thirds of those were to be treated with nusinersen. For most other countries, the number of patients was explicitly named as a key uncertainty and other financial schemes were put in place to ensure budget caps.

Several countries have implemented OBMEAs with a duration of 2–3 years for these treatments, whereas in England, data collection is a minimum of 3 years to enable re-appraisal within 4.5–5 years or earlier. In the Netherlands, a legal OBMEA with nusinersen is planned for 7 years, which goes beyond the usual 4-year period permitted, given the small number of patients anticipated.

OBMEA Data Collection and Analysis

Most countries are collecting data via a range of approaches that depend on each treatment, its uncertainties and the existing health data ecosystem infrastructure (clinical and/or reimbursement or insurance data), with an assumption that ongoing clinical studies will be completed. For tisagenlecleucel, links have been made to existing chemotherapy datasets and national registries for bone marrow transplant, whereas for nusinersen links to clinical registries have been important or have needed to be established.

Italy and Spain have a national web-based platform for which bespoke data collection requirements are created for each medicine/therapeutic indication. This not only allows enforcement of eligibility and continuation criteria, but should encourage timely entry of data to access treatment. However, in Italy, the national platform is not linked to the hospital reimbursement system and thus duplicate data entry is required by the prescribers. In Spain, the platform is being established and was not fully functional for tisagenlecleucel.

Costs of OBMEA are unclear, apart from in the well-established system in Italy, where for an OMP registry the marketing authorisation holder pays approximately 30,000 Euros every three years, with a 10,000 Euros maintenance fee from the fourth year. This includes the construct of the data collection forms after the pricing and reimbursement negotiation, updates on data collection and aggregated standard reports.

Only England, the Netherlands and Spain stated an intention to capture patient-reported outcomes for these cases. In England, research was commissioned at the outset of the OBMEA for nusinersen to identify existing patient-reported outcomes that could capture quality-of-life issues for patients with SMA. None adequately captured the issues that were considered important by patient representatives. As a result, the collection of patient-reported outcomes is yet to be operationalised. However, this data collection is a requirement of the English OBMEA and is expected to form part of the evidence submissions at the time of re-appraisal.

In England, the nusinersen OBMEA was undertaken within NICE’s HST programme, which requires each patient/guardian and their treating physician to sign an informed assent form6. These not only provide consent for use of data, but also confirm understanding of the OBMEA process including the conditions under which treatment could be terminated and the implications of withdrawal (stopping treatment access).

Detailed data analysis plans were not in place in most countries for the nusinersen and tisagenlecleucel OBMEA at the outset. Interim data are understandably not intended to be published, but some intend to provide updates on progress in recruitment in annual reports about OMP use and most with CED intend to issue public reports after re-appraisal.

Discussion

Outcomes-based managed entry agreements arise from appraisal discussions or pricing and reimbursement negotiations to manage outstanding uncertainties (and high prices). The Organisation for Economic Co-operation and Development report [7] stated that little information is shared or published about OBMEAs in terms of treatments involved, OBMEA design and results (analyses and decision). This is important because OBMEAs require careful governance to address data ownership, audit, transparency and appeal [14].

Our research has found information about the purpose, duration and data to be collected in the OBMEAs for nusinersen and tisagenlecleucel in several jurisdictions. However, this information was only available following close interaction with experts (co-authors of this paper) who knew where to find this information on their national websites and who could help us translate key information. This has enabled a detailed comparison of the constructs of these OBMEA case studies, which provides valuable insights that can improve future use and implementation of OBMEAs with rare disease treatments.

Since this research was started, there has been increasing interest in approaches to manage uncertainties with the one-off cell and gene therapies, many of which have been developed for rare diseases. These therapies rely on benefits realisation in the future and thus innovative pricing models based on individual outcomes have been used in some cases [15–17]. Pricing models have not been the subject of this work as issues related to pricing are subject to confidentiality agreements. The goal of this work was to identify what OBMEA information was available for two specific cases and encourage further collaboration to support all jurisdictions and stakeholders in sharing what can easily be made public. Work around innovative pricing schemes will clearly continue and it will be important to consider it alongside this work on OBMEAs.

The nusinersen and tisagenlecleucel case studies demonstrate that OBMEAs are agreed, at a minimum, in negotiation with the marketing authorisation holder and the payer, and for CED in several countries, physicians and research groups are involved in the design of data collection. Only England and the Netherlands mentioned the involvement of patient groups. In the Netherlands, a covenant was signed by all stakeholders for the legal OBMEA (CED) of nusinersen. This gave written assurance to the Minister of Health that the OBMEA would be conducted so that it was capable of drawing conclusions about clinical effectiveness at the end of the agreement. This purposive approach to data collection showing the commitment of all stakeholders appears to be an important change in the Dutch process to ensure the effectiveness of the OBMEA and could be an important model for other systems.

Given the investment required from all stakeholders in the design and conduct of population-based schemes (CED), it is imperative that steps are taken to enable them to deliver meaningful results. They should have the rigour of a pragmatic trial with clear delineation of uncertainties to be resolved (research questions and associated outcomes), sample size determination (number of patients and duration of study), data management processes for data from all sources and statistical analysis plan. However, these aspects were not clear for all the CED examples in our case studies.

Maximising Data Quantity and Quality

The outcomes studied in the OBMEA case studies to inform eligibility, continuation and effectiveness decisions were based on measurements used in clinical trials, but not all were standard in clinical practice. This adds burdens for clinicians, patients and their families. However, in many countries, the OBMEA will result in more clinical data than was available in the original submission from the marketing authorisation holder. Hence, it is important to ensure the data, from both individual-based and population-based schemes are used not only to optimise care for the patient but also to contribute to health system efficiency by determining optimal treatment regimes.

Only The Netherlands stipulated a sample size for their OBMEA. This demonstrated how sample size considerations for an OBMEA go beyond that for a clinical study as the conditional reimbursement implies that all eligible patients should have access to treatment. In chronic rare diseases, the prevalent population will be eligible for study at the outset and those who should recieve treatment first in this cohort may need to be prioritised. When all have been offered treatment, only newly eligible cases will be available. When recruitment is complete for the OBMEA, consideration needs to be given as to what happens with other eligible patients. For rare diseases, it would seem valuable to study these patients and this was recognised in the Netherlands, where an additional study will be established after recruitment to the legal OBMEA (not funded by the marketing authorisation holder). In England, this is considered on a case-by-case basis and for nusinersen those not eligible for the OBMEA will form a comparative cohort with risk adjustment to allow comparison, if possible.

Although the data to be collected in the OBMEA were similar in different countries, the duration of agreements differed and the rationale for the duration was unclear. It may have been linked to decision-making milestones, such as pricing and reimbursement negotiations. Only the Netherlands clearly stated the expected recruitment period and duration of follow-up. Such information is important for CED and should be clearly documented, taking account of the prevalent and incident population in the health jurisdiction and the timing of the most important outcome assessments.

In summer 2020, EUnetHTA Joint Action 3 published its first Post-Launch Evidence Generation pilot data collection report for nusinersen [22]. Eight research questions addressed uncertainties in SMA sub-populations covering the number of patients treated, patient characteristics, treatment duration, dosing in different patient groups, quality-of-life and patient-reported outcomes, long-term safety and efficacy, and validation of new outcome measures for disease progression effects. A minimum dataset was then defined to collect relevant real-world data from nusinersen registries in the participating countries (Italy, Croatia, Finland, Norway, Portugal, The Netherlands). Outcomes were to be measured at baseline, 6 and 18 months and a brief statistical analysis plan was presented. As EUnetHTA Joint Action 3 comes to an end in 2021, it is unclear how well this study will progress and as can be seen from this case study research, countries have initiated their own data collection schemes for nusinersen.

Alongside bespoke data collection, marketing authorisation holders were expected to continue their planned clinical programmes for the case studies, completing ongoing studies, and providing longer term follow-up and information about various sub-populations. However, none of the OBMEAs referred to regulatory post-licensing data requirements and how that may bolster evidence in the future and be used alongside data arising from the OBMEA.

To maximise information coming from disparate sources, linking of records is needed. This can be more easily achieved within health systems for secondary use of administrative data, where consent is presumed in law. However, challenges can arise when linkage to other sources, such as clinical registries is required. The AIFA registry is a stand-alone system that does not link to other data, which is seen as a limitation by many countries that have good electronic health records. For this reason, the new VALTERMED medicines’ registry system in Spain is of particular interest, not only because of its national status, but because it intends to include patient-reported outcomes and to link to other health service data.

Several countries are using national registries to collect data, SMA disease registries for nusinersen and stem cell transplantation registries for tisagenlecleucel. The aim is to link to individual patient data and thus national registries are preferred. However, as the RWE4Decisions initiative states [9], more needs to be done to bring the HTA/payer and clinical communities together to ensure that clinical registries are aligned with HTA/payer needs. This needs to include consideration of how the European registries for rare diseases being developed by the European Reference Networks can be used in jurisdictions.

Our case studies also show that there is opportunity for HTA/payers to collaborate across jurisdictions to not only align their appraisals to come to a common agreement on decision-relevant uncertainties, but to also agree a core dataset for data collection post-HTA. This could substantially improve timeliness and cost and efficiency of data collection [3] and optimise re-appraisal. However, it is difficult to achieve in practice. The BENELUXA joint assessment of nusinersen led to agreement on the need for an international SMA registry, but despite this, each country of that partnership developed its own OBMEA as outlined in Tables 1 and 2. The challenges were also recognised by EUnetHTA who stated that for rare diseases “collaboration among Member States in the definition of the minimum dataset and in gathering evidence coming from different registries and databases is crucial for HTA” [18].

In the national web-based OBMEA systems, data checking elements are embedded and automatic summary reports are produced that update on the level of data collection. Where a variety of data sources is used and several treatment centres are involved, careful monitoring processes are needed if sufficient data are to be collected, but not all countries have organised this for the case studies. This is essential not only to ensure data quality but to monitor patient recruitment and data completeness of all required assessments. NICE uses a multi-stakeholder Managed Access Oversight Committee for nusinersen. This committee meets every 6 months to discuss data collection issues that arise in clinical practice and monitors completeness and quality of data. One challenging issue that had to be faced in 2020 was the high level of missing data as a result of cancelled clinic visits because of the COVID-19 pandemic. As a result, alternative treatment continuation protocols have been developed and data analysis plans modified.

Capturing Patients’ Perspectives

In the field of rare diseases, knowledge about the disease evolves, particularly when the first disease-modifying medicine comes to market in an area of high unmet need. Therefore, evidence generation post-reimbursement should not only seek to resolve uncertainties found in the HTA, but also to capture new knowledge about the disease (particularly important if comparisons are made to natural history) and optimisation of treatment in real life. For this, involvement of patients and their informal caregivers would seem essential. However, there was little evidence of patient involvement in the OBMEA case studies. A few of the OBMEAs indicated the plan to collect patient-reported outcomes, which are considered particularly important in rare diseases [19, 20]. However, there is often no suitable patient-reported outcome for a rare disease (as was the case for nusinersen in England) and thus other mechanisms are needed to capture patients’ perspectives. As part of this project, KF/EK/SU have worked with the patient group leaders involved in NICE OBMEAs (including for spinal muscular atrophy) to develop a patient group submission template for re-appraisal. This will enable patient groups to gather evidence from patients about the conduct of the OBMEA and the benefits and disbenefits of treatment that are not captured in the quantitative data collection [21].

Examples of Successful OBMEA

Coverage with evidence development schemes have been criticised for their inability to collect sufficient good quality data in a timely manner to inform re-appraisal/pricing and reimbursement re-negotiations. As our case studies were about the establishment and implementation of OBMEAs, not re-appraisal, examples of OBMEAs that had made a difference were discussed in the expert workshops. Australian experts shared examples of OBMEAs for four rare cancers and one for lumacaftor/ivacaftor in cystic fibrosis. Of these, two are ongoing, one was suspended by the sponsor, one led to a change in reimbursement population and one led to no change.

In the English Cancer Drugs Fund, brentuximab (NICE TA524) and obinutuzumab (NICE TA629) for the treatment of rare cancers have completed their OBMEAs and resulted in full reimbursement. Elosulfase alfa for mucopolysaccharidosis type IVA was the first OBMEA enacted in the NICE HST programme. The treatment was due to be re-appraised in December 2020, but this has been delayed. The scope of the re-appraisal was agreed with stakeholders in December 2019. However, in February 2020, the re-appraisal was suspended “because the company has not provided an evidence submission that is adequate for the committee to make a decision and will not accept the terms of the charging procedure for HSTs”7. One challenge with this re-appraisal is that it will now be undertaken under a different decision-making process. Elosulfase alfa (and the other early appraisals) did not consider cost effectiveness and willingness-to-pay thresholds, but these are now clearly stated in their methods8.

Thus, the jury is still out as to whether OBMEA can be successful for rare disease treatments, but perhaps through greater collaboration among HTA/payers and with stakeholders, there can be more transparency about uncertainties that exist, constructs for data collection, and sharing of results that will optimise treatment and improve health service efficiency.

In Italy, where the majority of OBMEAs are individual-based schemes, changes have related to confidential price re-negotiations or optimisation of use, such as alteration of continuation criteria. Only a few treatments have been withdrawn from reimbursement and this was as a result of safety issues raised by regulators9 [11]. Given the increasing complexity of treatments coming to market, authors agree there is value in the use of OBMEAs for optimisation of use, to perhaps alter the eligible population or criteria for treatment continuation, and to ensure good organisation of care such as specialist centre capacity and to monitor the quality of care in each centre.

New National OBMEA Initiatives

In 2019, Germany passed the GSAV law that that gives the Federal Joint Committee increased authority to impose data collection requirements and price reductions if data do not support added value. It is anticipated this will be used for OMPs, but no further guidance has been issued. In 2019, Scotland also implemented a new “ultra-orphan pathway” enabling designated medicines to have an initial HTA review by the Scottish Medicines Consortium to identify key uncertainties, then requiring the marketing authorisation holder to submit a 3-year data collection plan to the Scottish Government to resolve the uncertainties. Given the small size of Scotland, it is likely that the subsequent re-appraisal will include data beyond Scotland, but this process is not transparent and few details are available.

Conclusions

These case studies show that OBMEAs are being implemented to manage uncertainties relating to the real-life and long-term clinical effectiveness and inform economic evaluations of rare disease treatments in several jurisdictions. This research focuses on understanding how the challenges in the establishment and implementation of OBMEAs for two different forms of rare disease treatment (nusinersen and tisagenlecleucel) have been addressed to deliver data of sufficient quantity and quality to inform re-appraisal.

During the IMPACT HTA WP10 research programme, a strong message has emerged from all stakeholders that OBMEAs should not become routine practice, as their implementation is burdensome for all parties. This aligns with past research that has queried the sustainability and effectiveness of OBMEAs [4] and suggested that OBMEAs should be the exception and not the norm. This was strongly reiterated by all involved in this case study research.

If OBMEAs are to be a credible conditional reimbursement route for rare disease treatments, the costs and feasibility of collecting sufficient data to inform decisions must be scrutinised at the outset, then steps must be undertaken to ensure data quality and completeness. One important mechanism to achieve this could be a covenant agreed by all stakeholders providing assurance that all necessary actions will be taken to ensure that the data collected will be sufficient for decision making as implemented in The Netherlands for nusinersen.

Bringing together the learnings about establishment of OBMEAs for these two case studies of rare disease treatments with previous literature about when to conduct an OBMEA, the IMPACT HTA WP10 research team has developed a checklist to determine the feasibility of undertaking an OBMEA for a rare disease treatment [23]. This checklist could be used by an individual jurisdiction, or across jurisdictions, as for some rare diseases (particularly the ultra-rare), collaborative approaches seem essential. Collaboration could foster discussions at the time of appraisal to determine decision-relevant uncertainties and a minimum dataset to be collected by all and shared in aggregated form for future re-appraisal. IMPACT HTA WP10 has issued tools to support implementation of OBMEA [24, 25] that could help standardise national approaches and support collaboration, but the tools need to be piloted, and adapted with experience.

Another important element to ensuring the efficiency and effectiveness of OBMEAs for rare disease treatments is to encourage transparency in reporting the plans for, interim updates on status and results of OBMEAs. Such information should be publicly available and thus perhaps, the HTA database, now hosted by the International Network of Agencies for HTA, could be enhanced to host such information. As the Organisation for Economic Co-operation and Development indicates [7], such sharing could reduce duplication of effort across countries and enable learning, which is particularly important for rare disease treatments.

Supplementary Information

Below is the link to the electronic supplementary material.

Declarations

Funding

Jaime Espin, Karen M. Facey, Elena Nicod, Entela Xoxi, Sheela Upadhyaya and Anna Zaremba were funded for this research from the European Commission’s Horizon 2020 research and innovation programme for the IMPACT-HTA project (grant number 779312). The results presented here reflect the authors’ views and not the views of the European Commission. The European Commission is not liable for any use of the information communicated. The other authors did not receive funding for this research.

Conflicts of Interest

Karen M. Facey, Jaime Espin and Entela Xoxi have received fees from various pharmaceutical companies for speaking and advisory roles, none of which relates to the topics in this paper. Elena Nicod is employed part-time by Dolon Ltd, no conflicts arise with this work. Emma Kent, Angèl Link, Elena Nicod^, Aisling O’Leary, Inneke van de Vijver, Anna Zaremba, Tatyana Benisheva, Andrius Vagoras and Sheela Upadhyaya have no conflicts of interest that are directly relevant to the content of this article.

Ethics Approval

No ethical approval was required for this study.

Consent to Participate

Completed templates included consent.

Consent for Publication

All authors have provided consent for publication.

Availability of Data and Material

All data are provided in Tables 1, 2 and 3, no programming has been undertaken.

Code Availability

Not applicable.

Authors’ Contributions

EX, EN, SU and KF developed the case study template. EN issued invitations to complete the template. KF/SU contacted marketing authorisation holders to determine which countries might have OBMEAs. All authors completed a template and answered queries from KF. KF amalgamated responses and all authors (except TB and AV) contributed to several discussion meetings about results, clarification of processes and key messages. KF led the writing of the paper and all authors critically reviewed drafts and approved the manuscript.