Use of pragmatic randomized trials in multiple sclerosis: A systematic overview

Abstract

Background:

Pragmatic trials are increasingly recognized for providing real-world evidence on treatment choices.

Objective:

The objective of this study is to investigate the use and characteristics of pragmatic trials in multiple sclerosis (MS).

Methods:

Systematic literature search and analysis of pragmatic trials on any intervention published up to 2022. The assessment of pragmatism with PRECIS-2 (PRagmatic Explanatory Continuum Indicator Summary-2) is performed.

Results:

We identified 48 pragmatic trials published 1967–2022 that included a median of 82 participants (interquartile range (IQR) = 42–160) to assess typically supportive care interventions (n = 41; 85%). Only seven trials assessed drugs (15%). Only three trials (6%) included >500 participants. Trials were mostly from the United Kingdom (n = 18; 38%), Italy (n = 6; 13%), the United States and Denmark (each n = 5; 10%). Primary outcomes were diverse, for example, quality-of-life, physical functioning, or disease activity. Only 1 trial (2%) used routinely collected data for outcome ascertainment. No trial was very pragmatic in all design aspects, but 14 trials (29%) were widely pragmatic (i.e. PRECIS-2 score ⩾ 4/5 in all domains).

Conclusion:

Only few and mostly small pragmatic trials exist in MS which rarely assess drugs. Despite the widely available routine data infrastructures, very few trials utilize them. There is an urgent need to leverage the potential of this pioneering study design to provide useful randomized real-world evidence.

Background

With the emergence of new treatment options in routine care, persons with multiple sclerosis (pwMS) and their caregivers increasingly have a choice between various therapeutic strategies. This requires solid evidence to inform decisions in daily routine. Pragmatic trials are randomized clinical trials designed to directly inform treatment decisions in practice. ¹ They are a key element of real-world evidence ² by leveraging the increasingly available sources of routinely collected data and overcoming the traditional shortcomings of non-randomized real-world evidence.^3,4 Conversely to traditional trials, they compare real-world care alternatives rather than testing experimental interventions under development, they deliver interventions such as drugs without measures that would never occur in practice to increase the adherence to them, and they use comparators that do exist in routine care, not such as placebo controls. They primarily consider health-related, patient-relevant outcomes, as these are critical to decision-making, rather than focusing on more mechanistic outcomes such as biomarkers which are more relevant for research and development of treatments. Pragmatic trials aim to answer the question what the best treatment choice is, conversely to traditional trials that investigate the interventions themselves.

Most recently, the COVID-19 pandemic has brought back memories of the great medical progress made in the 1980s with large pragmatic trials in cardiology.^1,5,6 Such “large simple trials”^7,8 were the blueprint for the now legendary RECOVERY trial, ⁹ which rapidly delivered the first drug with proven survival benefit for COVID-19 during the pandemic. This trial, designed in a few days and conducted with unprecedented speed, recruiting more than 10,000 patients in 3 months, has reinvigorated the idea of pragmatic trials.

These lessons and opportunities raise the question of where we stand in other areas—including in MS. The ability to leverage real-world data infrastructure and overcoming laborious, costly, and often slow data collection procedures has been identified as key to generate “randomized real-world evidence.” ¹⁰ MS is carefully studied in numerous international observational studies based on excellent routine data infrastructures^11,12 making this field another prime candidate for pragmatic trials. We aimed to systematically investigate to what extent such conditions may have led to pragmatic trials being conducted in MS, what characteristics they have, and which clinical questions they aim to answer.

Methods

Design

We generated a systematic overview of the pragmatic randomized trials research landscape in MS and followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement. ¹³

Eligibility criteria and assessment of pragmatism

We included any pragmatic randomized trial assessing any intervention and comparator on any health-related outcome in pwMS that reports on trial findings and was published as a journal article in English language with no restriction of the publication year. We considered trials that randomized participants individually (i.e. individuals were randomly assigned to an intervention) or per cluster (i.e. a group of individuals, e.g. communities or hospital ward, was randomly assigned to an intervention).

To determine if trials are “pragmatic,” we assessed the pragmatism of potentially eligible trials with the PRragmatic–Explanatory Continuum Indicator Summary-2 (PRECIS-2). PRECIS-2 has nine domains that can impact the pragmatism (eligibility criteria, recruitment, setting, organization, flexibility in delivery of the intervention, flexibility in adherence of the intervention, follow-up, primary outcome, and primary analysis; Table 1). ¹⁴ Each domain was independently assessed by two reviewers (J.H. and P.J., G.J.N., K.D., T.V.T.N., or L.G.H.) based on a 5-point scale (according to PRECIS-2 categories, i.e. 1 = very explanatory, 2 = rather explanatory, 3 = equally pragmatic and explanatory, 4 = rather pragmatic, 5 = very pragmatic, or no information). Trials were considered “pragmatic” if none of the PRECIS-2 domains was assessed below 3. We skipped the PRECIS-2 assessment and directly excluded trials with a clearly explanatory design feature (i.e. blinding, placebo-controlled), trials with a crossover design (i.e. the way how interventions are delivered may not be true care options), and self-labeled feasibility trials when the intervention was under development and the trials primarily intended to assess process-related outcomes (e.g. intervention acceptance).

Table 1.

PRECIS-2 domains with examples.

Domain	Pragmatic approach (derived from) 15 with notional examples for MS research
Eligibility criteria	Include anyone of the target population of interest (e.g. the persons with MS who in usual care would have to choose between the alternatives that the trial randomly allocates)
Recruitment	Recruitment from usual care (e.g. without extra resources or incentives)
Setting	Conduct the trial in usual care (e.g. MS centers or participants’ home)
Organization	Identical organization as usual care (e.g. involving clinicians or nurses with same qualification and experience as in routine)
Flexibility in delivery of the intervention	Full flexibility in how the intervention is delivered (e.g. follow standard clinical routine)
Flexibility in adherence of the intervention	Full flexibility in how participants engage with the intervention (e.g. no special/artificial measures to enforce adherence to drug use, such as pill-count)
Follow-up	Identical intensity of measurement and follow-up as in usual care (e.g. embedded in routine clinic appointments)
Primary outcome	Primary outcome(s) with decision-relevance to participants (e.g. relapses, quality-of-life)
Primary analysis	Include all participants in the analysis (e.g. no per-protocol analysis)

PRECIS-2: PRragmatic–Explanatory Continuum Indicator Summary-2.

Trials that were self-labeled as “pragmatic” and met the other eligibility criteria were assessed by PRECIS-2 but included irrespective of their individual domain PRECIS-2 scores (we considered these to be most inclusive in our approach, acknowledging that self-labeling of pragmatism is a poor predictor of pragmatism). ¹⁶

Information sources and search strategy

We searched for relevant trials via MEDLINE/Ovid, using the, to our knowledge, the only available validated search strategy for pragmatic trials ¹⁵ in combination with a search strategy for MS ¹⁷ (date of last search 1 December 2022; Appendix 1). Given the limited sensitivity of the pragmatic trial search strategy (46%), we additionally searched Scopus for references that cited the CONSORT-extension reporting guideline for pragmatic trials ¹⁸ or the seminal article describing the concept of pragmatic trials (published in 1967; ¹⁹ republished in 2009) ²⁰ and mentioned “Multiple Sclerosis” in their title, abstract, journal name, keyword, or any other database field (date of last search on 28 March 2023).

To quantify the sensitivity of the pragmatic trial search component in our search strategy, we manually screened a random sample of 100 articles identified in MEDLINE/Ovid (last search 19 December 2022) using our search strategy but removing all pragmatic trial-related search terms (i.e. the same keywords and index terms related to randomized trials and MS as above, but not restricted to keywords that indicate pragmatic design features; Appendix 1).

Study selection

One reviewer (J.H.) screened titles, abstracts, and full texts for eligibility. A second reviewer (P.J. or L.G.H.) confirmed the assessment, if necessary.

Data extraction

For data extraction, we used the PragMeta database infrastructure that is described in detail elsewhere. ²¹ Briefly, one reviewer (J.H.) extracted information on trial topic (i.e. treatment, supportive care, etc.), if the trial was self-labeled as “pragmatic,” type of MS, number of participants randomized, type of intervention and comparison, primary or co-primary outcome(s) (if not specified, all outcomes mentioned), outcome assessor, length of follow-up, unit of randomization, number of trial sites, number of trial arms, use of routinely collected data, trial registration and protocol availability, country of conduct, funding, patient and public involvement, and bibliographic information (i.e. authors, publication year, and journal).

Data analysis

We used descriptive statistics (range, interquartile range (IQR), and total/sum for continuous variables; number and percentages for categorical variables). We used R (version 4.2.2) for data analysis.

Results

Our search retrieved 420 records with 53 eligible publications reporting on 48 trials^22–69 (Appendices 1 and 2).

The 48 trials were published between 1967 and 2022 (median 2015; using the first publication if there were multiple ones; Figure 1), of which 12 (25%) were self-labeled as “pragmatic.” The trials were published in 25 different journals, with 19 trials (40%) being published in Multiple Sclerosis Journal and Clinical Rehabilitation. Twenty-nine trials (60%) referred to a trial registration and 19 (40%) to a protocol. Most of the trials (n = 33; 69%) were publicly funded without any support by the industry (Table 2; Appendices 3 and 5).

Figure 1.

Time trend of trials showing sample size, intervention type, and length of follow-up.

m: months; n: number of randomized participants.

Table 2.

Trial characteristics (n = 48).

	n (%)
Clinical characteristics
Number of randomized patients, median (range; IQR; overall)	82 (15–1053; 42–160; 7534)
Type of MS a
Relapsing-remitting MS	36 (75)
Secondary progressive MS	31 (65)
Primary progressive MS	28 (58)
Not specified	7 (15)
Intervention type
Pharmacological/treatment	7 (15)
Non-pharmacological/supportive care a	41 (86)
Lifestyle	18 (38)
Behavior	13 (27)
Care management	6 (13)
Physiotherapy	6 (13)
Comparison type
Usual care	25 (52)
Wait-list	8 (17)
Lifestyle	4 (8)
Drug	7 (15)
No intervention	2 (4)
Behavior	1 (2)
Physiotherapy	1 (2)
Design characteristics
Multinational	4 (8)
Multicenter	25 (52)
Involvement of patient representatives
Yes	4 (8)
Unclear	11 (23)
Not reported	33 (69)
Use of routinely collected data
Yes a	10 (21)
Recruitment purpose	10 (21)
Type of data: registry	5 (10)
Type of data: health records	1 (2)
Type of data: not reported	3 (6)
Type of data: other	1 (2)
Outcome collection	1 (2)
Type of data: health records	1 (2)
Unclear	2 (4)
Not reported	36 (75)
Reporting characteristics
Self-labeled as “pragmatic”	12 (25)
Referred to a trial registry entry	29 (60)
Referred to a trial protocol	19 (40)
Funding
Public	33 (69)
Public, supported by industry	4 (8)
Industry	4 (8)
No funding received	1 (2)
Not reported	6 (13)

IQR: interquartile range; MS: multiple sclerosis.

^aMore than one category possible.

Clinical questions of the included trials

Trials included participants with relapsing-remitting MS (n = 36; 75%), and/or secondary progressive MS (n = 31; 65%), and/or primary progressive MS (n = 28; 58%) or did not specify the type of MS (n = 7; 15%) (Tables 2 and 4; Appendices 3 and 5).

Table 4.

Individual trial description (n = 48).

Authors and reference	Country of conduct	Type of MS	N	Intervention, comparison, and main outcome b	Follow-up (m)
Non-pharmacological interventions (n = 41)
Riemenschneider et al. 62	Denmark	RRMS	84	Aerobic exercise intervention vs usual care on relapse and the global brain atrophy rate	12
Straudi et al. 63	Italy	RRMS, SPMS, PPMS	36	Physical exercises vs wait-list on walking endurance	3
Langeskov-Christensen et al. 66	Denmark	RRMS, SPMS, PPMS	86	Aerobic exercise intervention vs wait-list on cognitive performance, depression, and cardiorespiratory fitness b	12
Torkhani et al. 56	France	RRMS, SPMS, PPMS	35	Mindfulness-based intervention and physical exercises vs lifestyle intervention on fatigue, mobility, and quality of life	2
Williams et al. 2021 61	Australia	RRMS, SPMS, PPMS	50	In-person physical exercises vs home-based physical exercises on gait speed	2
Callesen et al. 25	Denmark	RRMS, SPMS, PPMS	71	Physical exercises vs wait-list on walking speed	2.5
Lincoln et al. 44 and Janiaud et al. 71 a	The United Kingdom	RRMS, SPMS, PPMS	449	Cognitive rehabilitation vs usual care on quality of life	12
Mayo et al. 45 a	Canada	RRMS, SPMS, PPMS	137	Personally adapted physical exercises vs guideline-based physical exercises on peak oxygen consumption	12
Arntzen et al.22,72	Norway	RRMS, SPMS, PPMS	80	Physical exercises vs usual care on balance	7.5
Feys et al. 30 a	Belgium	Not specified	42	Remote physical exercises vs wait-list on physical fitness, walking capacity and perceived ability, functional mobility, and quality of life	3
Freeman et al. 26 a	The United Kingdom	SPMS, PPMS	140	Physical exercises vs usual care on motor function	9
Gunn et al. 36 a	The United Kingdom	SPMS	56	Self-management program vs usual care on walking ability and quality of life b	7
Jongen et al. 40	The Netherlands	RRMS	158	Cognitive theory–based intervention vs usual care on self-efficacy	6
Renfrew et al. 52	The United Kingdom	RRMS, SPMS, PPMS	85	Ankle-foot orthoses vs functional electrical stimulation on walking speed	12
Boesen et al. 23 a	Denmark	RRMS, SPMS, PPMS	427	Multidisciplinary rehabilitation vs wait-list on quality of life	6
McClurg et al. 46 a	The United Kingdom	RRMS, SPMS, PPMS	191	Massage vs usual care on bowel dysfunction	6
Solari et al. 53	Italy	SPMS, PPMS	78	Palliative care management vs usual care on quality of life and burden	6
Stuifbergen et al. 54 a	The United States	RRMS, SPMS, PPMS	183	Computer-assisted cognitive rehabilitation vs usual care on neurocognitive competence	6
Vermöhlen et al. 58	Germany	Not specified	70	Hippotherapy vs usual care on balance	3
Khayeri et al. 41	Iran	Not specified	140	Happiness principle–based cognitive and behavioral intervention vs usual care on depression, stress, anxiety, and fatigue b	3
Kiropoulos et al.	Australia	RRMS	30	Cognitive behavioral therapy vs usual care on depressive symptoms	5
Bogosian et al. 24	The United Kingdom	SPMS, PPMS	40	Remote mindfulness intervention vs wait-list on distress	3
Papeix et al. 49	France	RRMS, SPMS, PPMS	50	Integrated multidisciplinary care approach vs usual care on quality of life	12
Carter et al. 27	The United Kingdom	RRMS, SPMS, PPMS	120	Physical exercises vs usual care on exercise behavior	9
Finch et al. 31 a	Canada	RRMS, SPMS, PPMS	15	Massage vs wait-list on self-efficacy b	2
Medina-Perez et al. 68	Spain	RRMS	42	Physical exercises vs usual care on maximal contraction, muscle power, and muscle endurance b	6
Paul et al. 50 a	The United Kingdom	RRMS, SPMS, PPMS	30	Web-based physiotherapy vs usual care on walking speed	3
Garrett et al. 33 b	Ireland	RRMS, SPMS, PPMS	316	Physical exercises vs no intervention on physical impact	2.5
Humphreys et al. 39	The United Kingdom	RRMS, SPMS, PPMS	151	Cognitive behavioral–based intervention vs wait-list on global health	8
Thomas et al. 55 b	The United Kingdom	RRMS, SPMS, PPMS	164	Fatigue management program vs usual care on fatigue, self-efficacy, and quality of life	4
Learmonth et al. 43	The United Kingdom	Not specified	32	Physical exercises vs usual care on walking speed	3
Cooper et al. 26	The United Kingdom	RRMS, SPMS	24	Computerized cognitive behavioral therapy vs usual care on depressive symptoms	5.25
Dodd et al. 29	Australia	RRMS	76	Physical exercises vs usual care on walking endurance	2.5
Miller et al. 47	The United Kingdom	SPMS, PPMS	30	Physiotherapy vs usual care on physical and psychological impact	4
Grossman et al. 35	Switzerland	RRMS, SPMS	150	Mindfulness intervention vs usual care on quality of life, depression, and fatigue	6
Velikonja et al. 57	Slovenia	RRMS, SPMS, PPMS	20	Sports climbing vs yoga on spasticity, cognitive function, mood/depressive symptoms, attention, and fatigue b	2.5
Higginson et al. 38	The United Kingdom	RRMS, SPMS, PPMS	52	Palliative care vs usual care on anxiety and palliative care concerns/needs, pain, caregiver burden, and costs b	6
Hermens et al. 37	Italy; Spain; Belgium	Not specified	35	Technology-supported physical exercises intervention vs usual care on motor function	2
Ward et al. 59	The United Kingdom	Not specified	114	Personalized education and information vs standardized information on falls or skin sores	12
O’Hara et al. 48	The United Kingdom	RRMS, SPMS, PPMS	183	Self-management program vs no intervention on mobility, activities of daily living, quality of life, and need for assistance b	6
Pozzilli et al. 51	Italy	RRMS, SPMS, PPMS	201	Home care vs usual care on disability, fatigue, mood/depressive symptoms, quality of life, resource use, and costs b	12
Pharmacological interventions (n = 7)
Cutter et al. 28	Russia; Poland; Italy; France; Croatia; the United States; Mexico; Spain; Austria; Turkey; Belgium; Argentina; Germany; Finland	RRMS	861	Glatiramer acetate lower vs higher dose on medication satisfaction	6
Giovannoni et al. 34	Australia; Belgium; Brazil; Canada; Croatia; Czech Republic; Denmark; France; Germany; Israel; Italy; Mexico; Netherlands; Poland; Russia; Serbia; Spain; Sweden; Ukraine; the United Kingdom; the United States	RRMS	628	Alemtuzumab vs interferon beta-1a (Rebif) on disability b	24
Weinstock-Guttman et al. 60	The United States	RRMS	50	Immediate discontinuation natalizumab and initiated another DMT vs two more natalizumab infusions on lesion activity and relapse	12
Calkwood et al. 64	The United States; Canada	RRMS	1053	Switch to fingolimod vs standard DMT (Avonex, Copaxone, Rebif, Betaseron, or Extavia) on treatment satisfaction	6
Mazdeh et al. 67	Iran	RRMS, SPMS, PPMS	90	Interferon beta-1a (Avonex or Rebif) vs interferon beta-1b (Betaferon) on disability and relapse b	24
Hoogstraten et al. 65	The Netherlands	RRMS	29	Corticotrophin vs vitamin B on disability, functioning, mobility, and activities of daily living b	12
Millar et al. 69	The United Kingdom	Not specified	350	Corticotrophin and vitamin B vs vitamin B alone on disability and relapse b	19

DMT: disease-modifying therapy; MS: multiple sclerosis; N: number of randomized participants; RRMS: relapsing-remitting multiple sclerosis; SPMS: secondary progressive multiple sclerosis; PRMS: primary progressive multiple sclerosis.

^aTrials self-labeled as “pragmatic.”

^bMain outcome(s) extracted (i.e. primary or co-primary); if not specified as such, we extracted all outcomes mentioned (indicated by^b).

Most trials assessed non-pharmacological interventions (n = 41; 85%) with primarily lifestyle (n = 18; 38%) and/or behavioral (n = 13; 27%) components (Figure 1 and Table 3). Only seven trials assessed drugs (15%; alemtuzumab, ³⁴ corticotrophin,^65,69 glatiramer acetate, ²⁸ natalizumab, ⁶⁰ interferon beta-1a, ⁶⁷ and fingolimod); ⁶⁴ these were published between 1967 and 2019 (median 2014) and had a median of 350 participants (IQR = 70–745; overall 3601). The most frequent comparator was usual care (n = 25; 52%).

Table 3.

PRECIS-2 assessment.

PRECIS-2 domain	Total (48 trials)	Self-labeled as “pragmatic” only (12 trials)	Drugs only (7 trials)	Use of RCD only (10 trials)
	Median (IQR, range); n	Median (IQR, range)	Median (IQR, range)	Median (IQR, range)
Eligibility criteria a	4 (4–5, 2–5); 48	4.5 (4–5, 2–5)	4 (3–4, 3–5)	4.5 (4–5, 2–5)
Recruitment a	4 (4–5, 3–5); 35	4 (4–5, 3–5)	5 (5–5, 5–5)	4.5 (4–5, 3–5)
Setting	4.5 (4–5, 3–5); 48	5 (4–5, 4–5)	5 (4–5, 4.5–5)	5 (4–5, 4–5)
Organization	4.5 (4–5, 2–5); 44	4 (3–4, 2–5)	5 (5–5, 3–5)	4.5 (3.25–5, 3–5)
Flexibility—intervention delivery	5 (4–5, 3–5); 37	5 (3–5, 3–5)	5 (5–5, 5–5)	5 (4–5, 3–5)
Flexibility—intervention adherence	3.5 (3–4, 2–5); 26	3 (2.25–4, 2–5)	4 (4–4, 4–4)	3.5 (3–4, 3–4)
Follow-up	4 (3–4, 1–5); 47	3 (3–4, 1–5)	4 (3.5–4, 3–4)	4 (3–4, 3–5)
Primary outcome	5 (5–5, 1–5); 6	5 (5–5, 1–5)	5 (4.75–5, 4–5)	4 (3–4, 3–5)
Primary analysis	5 (4–5, 2–5); 37	4 (3–4, 2–5)	5 (5–5, 5–5)	5 (4.25–5, 3–5)

IQR: interquartile range; n: number of trials with sufficient information to assess individual PRECIS-2 domains; PRECIS-2: PRagmatic Explanatory Continuum Indicator Summary 2; RCD: routinely collected data.

Individual trial assessment per domain is provided in Appendix 4; 1 = very explanatory; 2 = rather explanatory; 3 = equally pragmatic/explanatory; 4 = rather pragmatic; 5 = very pragmatic. ⁷⁰

^aFor the one cluster-randomized trial, we could not identify sufficient information to assess the domains eligibility criteria and recruitment on cluster level.

Primary outcomes were diverse, assessing or example, quality of life, physical functioning, and disease activity. The median length of follow-up was 6 months (range = 2–24; IQR = 3–12) (Tables 2 and 4 and Figure 1; Appendix 5).

Design characteristics

The 48 trials included a median of 82 participants (range = 15–1053; IQR = 42–160; overall = 7534; Figure 1). Only three trials (6%) included >500 participants. All trials randomized participants individually; there was only one cluster-randomized trial. Trials had a median of two arms (range = 2–4; IQR = 2–3) and a median of two trial centers (range = 1–191; IQR = 1–5). They were conducted mostly in the United Kingdom (n = 18; 38%), Italy (n = 6; 13%), the United States, and Denmark (each n = 5; 10%); only four trials (8%) were conducted internationally. Outcomes were mostly assessed in a non-blinded fashion (n = 28; 58%). Only four trials (8%) clearly stated information on patient and public involvement. The 10 trials (21%) that clearly reported the use of routinely collected data (mostly registry-based data; n = 5; 10%) leveraged these data for recruitment, and only 1 trial (2%) used routinely collected data for outcome ascertainment (Tables 2 and 4; Appendices and 5).

Pragmatic trial features

None of the identified trials was very pragmatic (i.e. scoring 5) in all domains of the PRECIS-2 tool, but 14 trials (29%) scored rather pragmatic or very pragmatic (i.e. 4 or higher) in all domains (Appendices 4 and 5).

Most trials were very pragmatic (median PRECIS-2 score 4.5–5) for the domains setting (IQR = 4–5), organization (IQR = 4–5), flexibility in intervention delivery (IQR = 4–5), primary outcome (IQR = 5–5), and primary analysis (IQR = 4–5). For example, they were conducted in a similar setting to which the results were intended to be applied (e.g. a home-delivered intervention), ²⁶ the expertise and resources needed for the organization of the trial did not much deviate from usual care (e.g. trained staff who is not available in usual care), ³³ the delivery of the intervention was as in usual care (e.g. the way of how to deliver the intervention was left to the discretion of the therapist), ²² the primary outcome was very relevant for decision-making of patients (e.g. quality of life), ³⁹ and the primary analysis was according to the intention to treat (i.e. not only considering those that strictly adhered to a protocol). ⁶²

With a median PRECIS-2 score of 3.5–4 in the domains eligibility criteria (IQR = 4–5), recruitment (IQR = 4–5), flexibility in intervention adherence (IQR = 3–4), and follow-up (IQR = 3–4), the pragmatic design features were less pronounced. For example, the eligibility criteria excluded persons likely to receive the intervention in routine care (e.g. pregnant women ²² or elderly ⁴⁵ ). Other factors limiting pragmatism included an artificially intensive resource use for participant recruitment (e.g. advertisement ⁴² ), that unusual measures were used to monitor and promote intervention adherence (e.g. monitored through phone calls ³⁰ ), or that there were artificially intensive (long and frequent) follow-up visits deviating from usual care (e.g. daily diary completion for 6 weeks ⁴⁶ ) (Table 3 and Figure 2; Appendices 4 and 5).

Figure 2.

PRECIS-2 assessment wheels of individual trials (n = 48).

PRECIS-2: PRagmatic Explanatory Continuum Indicator Summary-2.

Trials self-labeled as “pragmatic” (PRECIS-2 wheels highlighted in blue); missing dot indicates insufficient information to assess a PRECIS-2 domain.

The PRECIS-2 assessment results for trials self-labeled as “pragmatic” (n = 12; 25%), trials assessing drugs (n = 7; 15%), and trials using routinely collected data (n = 10; 21%) were not clearly different to the overall sample (Table 3).

Discussion

We identified 48 pragmatic trials in the field of MS, most of them with small sample sizes, more recently published, and very few assessing drugs. The spectrum of their research questions and primary outcomes is very wide and design features that may indicate pragmatism, such as the use of routinely collected data, were rare.

The first pragmatic trial in MS was published in 1967 comparing a treatment with corticotrophin with usual care, ⁶⁹ the same year as the seminal article by Schwartz and Lellouch introduced the concept of pragmatism for clinical trials.^19,20 Until about 10 years ago, there were almost no pragmatic trials in MS, and when they finally became a bit more common, they remained relatively small, almost always with fewer than 100 patients. There are no such large pragmatic trials in MS as the famous GISSI trial (Gruppo Italiano per lo Studio della Sopravvivenza nell’Infarto Miocardico) from Italy, which enrolled more than 11,000 patients from February 1984 to June 1985 and demonstrated the mortality benefit of early thrombolytic treatment with streptokinase in acute myocardial infarction. ⁷³ Large simple trials have the advantage that they can uncover the typically small to modest average treatment effects that we can unfortunately only expect with most modern treatments, ⁷⁴ and they can also reveal subgroup effects that allow us to better personalize therapies and identify those patients who may expect larger benefits. This requires a heterogeneous study population and not a population homogenized according to strict eligibility criteria that lacks diversity. Too restricted samples have been also highlighted in rehabilitation trials in MS as the main hurdle to evidence-based decision-making. ⁷⁵

While the high incidence of myocardial infarction (GISSI) or COVID-19 (RECOVERY) certainly facilitated the conduct of these large trials to some degree, MS has a higher prevalence as a chronic condition and pwMS are often treated in structured care settings with scheduled regular follow-up visits, structured assessment of the most decision-relevant outcomes (such as the Expanded Disability Status Scale (EDSS)), and systematic data collection embedded in routine care. The MSBase registry alone includes systematic follow-up of more than 92,000 patients with routinely collected data. ¹² Using such data infrastructures for randomized trials could provide directly applicable real-world evidence with pragmatic trials, saving costs and resources, among other benefits.

A classic example of this approach comes (again) from cardiology, the Thrombus Aspiration in ST-Elevation Myocardial Infarction (TASTE) trial in Sweden, which was based on a quality control registry and showed that patients with myocardial infarction did not benefit from thrombus aspiration as coronary intervention. In this trial with more than 7000 randomized patients, the data quality was excellent (perfect completeness, zero losses to follow-up) and the costs were minimal (US$50 per patient). ⁷⁶

In MS, the trials that we identified were rather small compared to the examples from cardiology outlined above. Among the largest ones was the CONFIDENCE study ²⁸ with almost 1000 participants from 88 study centers in 14 countries that answered the question which of two doses of glatiramer acetate provided more medication satisfaction (in this study, pwMS preferred glatiramer acetate 40 mg three times weekly to 20 mg once daily). The CONFIDENCE study still had features that were not perfectly in line with a pragmatic intend (e.g. it had an intense follow-up assessment which deviates from usual care). An example of a trial assessed as pragmatic among all PRECIS-2 domains was from Boesen et al., who determined in more than 400 pwMS whether a 4-week inpatient multidisciplinary rehabilitation resulted in a better quality of life compared to 6 months on a wait-list to receive the rehabilitation afterwards ²³ (the scheduled rehabilitation was favorable for several quality of life-related measures). Pragmatism was facilitated by direct recruitment of participants from usual care, the provision of commonly available rehabilitation measures, and clearly decision-relevant outcomes. These two studies present characteristics which are typical for the designs of the other pragmatic trials we found. Unfortunately, these were often not only small, but more than half of them also lacked a blinded outcome assessment, which indicates another area for improvement.

Overall, the PRECIS-2 domains that were often (very) pragmatic across the 48 trials in MS were related to the setting, recruitment, and primary outcome. This may be a direct consequence of having the structured care settings with regular follow-up and the structured assessment of decision-relevant outcomes that are typical for MS care. In this regard, the specific conditions of MS care appear to indeed facilitate pragmatic trials. On the contrary, despite the widely available and often excellent routinely collected data infrastructures (e.g. registries), very few trials seem to utilize this source. Only 10 trials clearly reported the use of routinely collected data, mostly for recruitment, and only 1 trial (2%) used these data for outcome ascertainment. There is an urgent need to leverage the potential of this pioneering study design to provide useful randomized real-world evidence.

Trial teams wishing to use pragmatic designs may encounter external barriers. Clinical trials are increasingly encompassed by an extensive regulatory framework ⁷⁰ aiming to uphold ethical, legal, and quality standards. Existing guidance and procedures have been developed with a focus on traditional drug-oriented trials such as those for drug development and approval, but they do not focus on trials with a pragmatic intend that compare treatment choices; however, these same systems can be a barrier against reform, if regulatory personnel is unfamiliar with pragmatic trial design. ⁷⁷ The domains for which we found lower overall pragmatism scores (i.e. eligibility criteria, recruitment procedures, adherence, and follow-up) may all have structural barriers against pragmatic design features, for example, ethical concerns about including vulnerable populations or using electronic consent forms, and requirements for close attention to adherence and extensive follow-up measures.

Our study has several limitations. First, we aimed to provide an overview of the research agenda with pragmatic trials in MS and cannot ensure completeness—we will have missed some pragmatic trials. However, we included every trial published in a PubMed-indexed journal that was labeled as “pragmatic” by the authors. We also searched for studies using a validated search filter specifically for pragmatic trials and used established search components for MS. The screening of studies was done by one reviewer alone and we used only one literature database, but we supplemented our search by screening all articles that cited two key articles on pragmatic trials, regardless of whether they were indexed in PubMed or not. Overall, it is unlikely that the trials we may have missed would have changed the overall interpretation of our findings.

Second, the PRECIS-2 assessment was performed in different teams involving four different second reviewers. However, with one consistent first reviewer, who was experienced in the assessment of pragmatism and following strictly the PRECIS-2 guidance provided by the PRECIS-2 team, ⁷⁸ we ensured to have very consistent PRECIS-2 ratings.

Third, retrospective assessment of pragmatism is challenging because it is sometimes difficult to judge whether the trial situation is similar to the target situation in which the healthcare decision is to be made. The main criteria we used to exclude most of the trials were, however, objective and independent of the target situation (e.g. use of placebo controls), which limits the impact of this aspect.

Forth, the evaluation of pragmatism depends substantially on the reporting quality. While we were not able to assess all PRECIS-2 domains in all trials due to insufficient information, the pragmatism of some trials will be seen differently by those who were involved in the trials or had closer insights in the original setting and circumstances. While this may have changed some domain assessments, we do not believe that this would substantially change the overall impression of a trial and thus the main interpretation of our findings.

Overall, we found pragmatic features that have been commonly applied in MS trials, illustrating a promising development. Lessons learned from COVID-19 for clinical trials include that “Pragmatism, integration in clinical care, efficient administration, promotion of collaborative structures, and enhanced integration of existing data and facilities might be several of the legacies of COVID-19 on future randomized trials.” ⁷¹ In Switzerland, the publicly funded, investigator-initiated MultiSCRIPT project was launched most recently by our group. ⁷² It is a continuous series of pragmatic trials evaluating healthcare innovations for MS, fully embedded in the Swiss Multiple Sclerosis Cohort (SMSC) and based entirely on routinely collected data from more than 1000 pwMS. ¹¹ MultiSCRIPT aims to resemble these lessons, and we hope it inspires others. The rise of the pragmatic approach is promising; there is still a long way to go in MS research, but the first steps have been taken.

Conclusion

There are only very few and mostly small pragmatic trials in MS and rarely assessing drug treatments. Despite the widely available and often excellent routinely collected data infrastructures, very few trials seem to utilize this source. There is an urgent need to leverage the potential of this pioneering study design to provide useful randomized real-world evidence.