Multiple Sclerosis International Federation guideline methodology for off-label treatments for multiple sclerosis

Abstract

Background

A total of 2.8 million people are living with multiple sclerosis and due to disparities in access to medicines, the ability to treat this condition varies widely. Off-label disease-modifying therapies are sometimes more available or affordable in different health systems. Appropriate methodology is integral in creating high-quality and trustworthy guidelines. In this article, we outline Multiple Sclerosis International Federation’s (MSIF) approach to creating guidelines for off-label treatments for multiple sclerosis.

Methods

We use the Guidelines International Network (GIN)-McMaster Guideline Development Checklist and the Grading of Recommendations, Assessment, Development and Evaluations (GRADE) Evidence-to-Decision (EtD) framework. We developed detailed health descriptors for health outcomes and the panel drafted PICO (Population, Intervention, Comparator, Outcome) questions and prioritised outcomes. We collaborate with independent organisations, which systematically review and collate the information. We are actively engaging stakeholders and consulting with relevant organisations, boards, working groups and individuals.

Results

The draft guideline recommendations will be published for open comment and stakeholders will be encouraged to endorse and disseminate the guidelines. Our methodology ensures integrity and transparency in the criteria, evidence and judgement used to make recommendations.

Conclusions

This approach will facilitate transparent creation of high-quality and trustworthy guidelines, and allow the global guidelines to be adopted or adapted into national settings.

Background

Global estimates suggest that 2.8 million people are diagnosed with multiple sclerosis (MS), ¹ 25% of them residing in low- and middle-income countries. There are disparities in the availability of disease-modifying therapies (DMTs) for people with MS. Experts in 14% of countries surveyed report having no licensed DMTs available in the country. In the African region, 60% of countries have no licenced DMTs, and this number is even higher, 70%, when analysing low-income countries around the world. ² Off-label DMTs are sometimes more available or affordable. The Atlas of MS highlights that at least 89 countries use at least one off-label DMT to treat MS. The need for guidance for off-label treatments is highlighted in the recently published article ‘Ethical use of off-label disease-modifying treatments for multiple sclerosis’. ² To address this need, MSIF has decided to develop guidelines for commonly used off-label DMTs for MS.

We outline the detailed MSIF methodology for reviewing evidence and making recommendations in this paper. We base our methods on Grading of Recommendations, Assessment, Development and Evaluations (GRADE) and the Guidelines International Network (GIN)-McMaster Guideline Development Checklist.^{( 3 )} Significant attention and controversy surround off-label treatments for MS. We have therefore implemented a process that is strict, clearly structured, transparent and ensures that conflicts of interests (COIs) are carefully managed.

Purpose of the guidelines

The purpose of the global MS guideline is to support clinicians and people with MS to make decisions when considering specific off-label DMTs and to improve health equity in access to MS care. Off-label DMTs do not have regulatory approval for MS and the evidence for their use in MS may be limited, with uncertainty in the efficacy and safety of the DMTs.

Guidelines aim to systematically address evidence and considerations to support decisions for patients, clinicians and decision-makers with a systematic review of the existing evidence and a structured approach to making recommendations. The national and local setting influences guidelines and it is important that any global recommendations are adapted to meet the needs of people with MS and the local health system. A transparent process allows criteria, evidence and judgements to be carefully assessed and developed further.

This guideline should inform clinical practice, health policy and reimbursement decisions in low-resource settings, where a range of on-label treatments are not available or affordable. The aim is to improve health equity globally, regionally and with marginalised populations, such as refugees, people without health insurance and people with low socio-economic status. The primary end users of this guideline are people with MS, clinicians treating MS and health policymakers involved in treatment reimbursement decisions. The guideline may identify future research priorities, where there is a clear lack of evidence.

It is important to note that guidelines should not be used out of context for reimbursement, clinical practice, legal or policy decisions. Good clinical practice always requires the clinician and person with MS to consider both desirable and undesirable health outcomes of any intervention to make a decision. If the guideline recommendations are used to deny access to viable alternative treatments, this is a misinterpretation of the recommendation and potentially harmful.

Methods

Guideline development process

The guideline development process is based on the GIN-McMaster Guideline Developers Checklist and the GRADE approach.^3,4 We are using GRADE’s Guideline Development and Implementation tool GRADEpro. ⁵ Please refer to Figure 1 for the overview of the process.^{( 3 )} This methods paper uses only publicly available information and did not consult patients/clients; therefore, research ethics board approval was not applicable.

Figure 1.

Outline and structure of the guideline process.

Figure 1 shows the outline and structure of the guideline process. ³ The guideline development group is comprised of representatives from MSIF and the McMaster University team. The Technical Support Team is formed by the Cochrane MS and Rare Diseases of the CNS Review Group and the WHO Collaborating Centre in Evidence-Based Research Synthesis and Guideline Development. The guideline oversight committee is MSIF’s International Working Group on Access (IWGA).

Panel composition and training

We select the guideline panel using the following criteria:

1. Multidisciplinary proficiency and experience; research, clinical practice in a variety of settings (clinicians, allied health professionals), people with MS, pharmaceutical policy and regulation, access to treatment, pharmacology, guideline methodology and development.

2. International diversity; WHO world regions and countries from the different World Bank income categories.

3. Gender balance.

4. Representation from key neurological organisations that have been involved in guideline development for MS.

The panel members are required to complete the online INGUIDE panel member training program. ⁶ The training modules give an overview of the guideline development process, including rating patient-important outcomes, rating the certainty of evidence and creating recommendations. The guideline conduct and group process have also been guided by the Guideline Participant Tool to ensure fair and effective participation.^4,7

Managing COIs

MSIF remains committed to the impartiality of the outcomes of the guideline process. No funding from the pharmaceutical industry is used in this project.

We follow the GIN principles for managing COIs. ⁸ The panel members complete a standard COI form, following the World Health Organization (WHO) template, requiring the declaration of conflicts relating to the specific DMTs considered. An independent organisation, the European Academy of Neurology (EAN) (EAN Ethics and Quality Task Force) assessed the COI forms and made recommendations regarding panel members’ COI management. In line with GIN principles, any individuals with identified conflicts are able to actively take part in the discussion, but are recused from making judgements and take part in voting. The panel members are required to update their COI if they change during the guideline process. To protect the guideline panel from any outside influence, they are required to sign a non-disclosure agreement to not divulge details of the discussions before the publication of the guideline. Panel membership is kept out of the public domain during the development process to additionally protect panelists from pressure and outside influence.

Generating PICO questions and ranking outcomes

The guideline panel prioritised questions based on common clinical decisions in low-resource settings, where a range of on-label treatments are presently not available or affordable. The selected populations (P) and sub-populations aim to cover all forms of MS with emphasis on sub-populations that are expected to be of particular relevance. The interventions (I) and comparators (C) were prioritised based on consideration of the breath of relevant options in these settings.

The guideline panel carefully selected outcomes (O) by the following process^3,9,10:

1. All commonly used and relevant MS health outcomes are listed.

2. Health descriptors are generated for all health outcomes ¹¹ to ensure all panel members agreed on a specific definition of an outcome. A small focus group of international people with MS were also able to comment on the health descriptors. Please see Appendix 1 (see Supplemental material) for the 11 descriptors: disability or dependency, relapse of MS, adverse events, long-term adverse events, serious adverse events, cognitive decline in MS, quality-of-life impairment, new or enlarging T2-weighted MRI lesions, new gadolinium-enhancing positive T1-weighted MRI lesions and mortality.

3. The guideline panel members independently rate each health outcome numerically on a 1–9 scale based on the importance for people with MS (7–9, critical; 4–6, important; and 1–3, of limited importance). The aggregated results are discussed in a teleconference and the final list of critical, important and excluded outcomes are agreed. We only assess the critical and important outcomes for the systematic reviews and the guideline recommendations.

Evidence synthesis

To ensure high-quality standards and independence, we are collaborating with Cochrane Multiple Sclerosis and Rare Diseases of the CNS Review Group IRCCS Istituto delle Scienze Neurologiche di Bologna, AUSL di Bologna (Italy), the WHO Collaborating Centre in Evidence-Based Research Synthesis and Guideline Development Direzione Generale Cura della Persona Salute e Welfare, Regione Emilia-Romagna, Bologna (Italy) and the Department of Epidemiology Lazio Regional Health Service-ASL Rome, Unit of Research synthesis and dissemination, Guidelines and HTA, Rome GRADE Center for the systematic and rapid reviews.

We are conducting systematic reviews for the effects on health benefits and harms following the Cochrane methods process ¹² to ensure all the relevant evidence is considered and GRADE methodology is followed to assess carefully the quality of evidence. The clinical data is compiled into Summary of Findings tables for clear presentation of the results.^10,12

We use the GRADE Evidence-to-Decision (EtD) framework and synthesise evidence from rapid reviews to inform the framework criteria (please see Box 1^13–18). The EtD criteria include the benefits and harms, values, resource requirements and cost-effectiveness, equity, acceptability and feasibility.

Box 1 EtD framework

Box 1.

Evidence-to-Decision (EtD) framework

The Grading of Recommendations, Assessment, Development and Evaluations (GRADE) EtD framework supports by developing high-quality recommendations.^(13–15) The framework aims to create a transparent and clear structure to outline the rationale for recommendations.

The criteria used for a clinical recommendation–population perspective are as follows:

1. Is the problem a priority?

2. How substantial are the desirable anticipated effects?

3. How substantial are the undesirable anticipated effects?

4. What is the overall certainty of the evidence of effects?

5. Is there important uncertainty about or variability in how much people value the main outcomes (values)?

6. Does the balance between desirable and undesirable effects favour the intervention or the comparison?

7. How large are the resource requirements (costs)?

8. What is the certainty of the evidence of resource requirements (costs)?

9. Does the cost-effectiveness of the intervention favour the intervention or the comparison?

10. What would be the impact on health equity?

11. Is the intervention acceptable to key stakeholders?

12. Is the intervention feasible to implement?

Results

Interpretation of recommendations

In our approach to recommendations, we follow the GRADE approach describing the strength and certainty of recommendations for or against an intervention. The strength of a recommendation is expressed as strong (‘the guideline panel recommends…’) or conditional (‘the guideline panel suggests…’). The interpretation is as follows^{( 19 )}:

Strong recommendation

• For patients: most individuals in this situation would want the recommended course of action, and only a small proportion would not.

• For clinicians: most individuals should follow the recommended course of action. Formal decision aids are not likely to be needed to help individual patients make decisions consistent with their values and preferences.

• For policymakers: the recommendation can be adopted as policy in most situations. Adherence to this recommendation according to the guideline could be used as a quality criterion or performance indicator.

• For researchers: the recommendation is supported by credible research or other convincing judgments that make additional research unlikely to alter the recommendation. On occasion, a strong recommendation is based on low or very low certainty in the evidence. In such instances, further research may provide important information that alters the recommendations.

Conditional recommendation

• For patients: the majority of individuals in this situation would want the suggested course of action, but many would not. Decision aids may be useful in helping patients to make decisions consistent with their individual risks, values and preferences.

• For clinicians: different choices will be appropriate for individual patients, and clinicians must help each patient arrive at a management decision consistent with the patient’s values and preferences. Decision aids may be useful in helping individuals to make decisions consistent with their individual risks, values and preferences.

• For policymakers: policymaking will require substantial debate and involvement of various stakeholders. Performance measures about the suggested course of action should focus on whether an appropriate decision-making process is duly documented.

• For researchers, this recommendation is likely to be strengthened (for future updates or adaptation) by additional research. An evaluation of the conditions and criteria (and the related judgments, research evidence and additional considerations) that determined the conditional (rather than strong) recommendation will help to identify possible research gaps.

The GRADE EtD framework supports by developing high-quality recommendations.^(13–15) The framework aims to create a transparent and clear structure to outline the rationale for recommendations.

MSIF is collaborating with McMaster GRADE Centre, WHO Collaborating Centre for Infectious Diseases, Research Methods and Recommendations to facilitate unbiased methods and chair the panel discussions.

Stakeholder engagement

We are taking an active approach to stakeholder engagement and consulting with relevant organisations, boards, working groups and individuals. The draft guideline recommendations will also be published for open comment for public input and correction of possible inaccuracies. Stakeholders will be encouraged to endorse and disseminate the guidelines.

Publishing and updating the guideline

We will publish the resulting guideline in a peer-reviewed scientific journal and ensure it will be available open-access without a fee. When relevant new information is published, it will be important to ensure the guidelines are updated in a timely fashion.

Limitations

We have prioritised transparency and reliability of the process using well-established methods for evidence reviews and guideline development. This is imperative for a topic that may cause controversy and which will need to be considered in different health system contexts. This rigour requires a considerable time commitment from the guideline development group, technical support team and the panel members. We decided to take a collaborative approach with wide stakeholder feedback, requiring time and commitment from a number of organisations. The strict COI assessment caused some limitations when selecting panel members, and restricting panel member’s contribution during the process.

Conclusion

We have presented a new approach for rigorous and transparent evidence-based guideline development by MSIF. This work utilises the widely adopted GRADE methodology for guideline development, and employs Cochrane systematic reviews for evidence assessment. We have described how we will effectively engage stakeholders and manage COIs to ensure the integrity of the guideline. We hope this guideline will support equity in access to MS treatment by providing evidence-based recommendations on commonly used off-label treatments.