Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2022 Aug 18.
Published in final edited form as: Mult Scler Relat Disord. 2020 Aug 5;45:102434. doi: 10.1016/j.msard.2020.102434

Idebenone does not inhibit disability progression in primary progressive MS

Peter Kosa a, Tianxia Wu b, Jonathan Phillips a, Mika Leinonen c, Ruturaj Masvekar a, Mika Komori a,#, Alison Wichman a, Mary Sandford a, Bibiana Bielekova a,*
PMCID: PMC9386688  NIHMSID: NIHMS1620173  PMID: 32784117

Abstract

Background:

Multiple sclerosis (MS) is a chronic, immune-mediated neurodegenerative disorder of the central nervous system (CNS). While current MS therapies target the inflammatory processes, no treatment explicitly targets mitochondrial dysfunction and resulting axonal loss. Therefore, the aim of this study was to determine whether idebenone inhibits mitochondrial dysfunction and accumulation of disability in primary progressive MS (PPMS) and to enhance understanding of pathogenic mechanisms of PPMS progression using cerebrospinal fluid (CSF) biomarkers.

Methods:

The double-blind, placebo-controlled Phase I/II clinical trial of Idebenone in patients with Primary Progressive MS (IPPoMS; NCT00950248) was an adaptively designed, baseline-versus-treatment, placebo-controlled, CSF-biomarker-supported trial. Based on interim analysis of the 1-year pre-treatment data, change in the area under the curve of Combinatorial Weight-Adjusted Disability Score (CombiWISE) became the primary outcome, with >80% power to detect ≥40% efficacy with 28 patients/arm treated for 2 years in baseline versus treatment paradigm. Changes in traditional disability scales and in brain ventricular volume were secondary outcomes. Exploratory outcomes included CSF biomarkers of mitochondrial dysfunction (Growth/differentiation factor 15 [GDF15] and lactate), axonal damage (neurofilament light chain [NFL]), innate immunity (sCD14), blood brain barrier leakage (albumin quotient) and retinal nerve fiber layer thinning.

Results:

Idebenone was well tolerated but did not inhibit disability progression or CNS tissue destruction. Concentrations of GDF15, secreted predominantly by astrocytes and choroid plexus epithelium in vitro, increased after exposure to mitochondrial toxin rotenone, validating the ability of this biomarker to measure intrathecal mitochondrial damage. CSF GDF15 levels correlated strongly with age and MS patients had CSF levels of GDF15 significantly above age-adjusted healthy volunteers, with highest levels measured in PPMS. Idebenone did not change CSF GDF15 levels.

Conclusion:

Mitochondrial dysfunction exceeding normal aging reflected by age-adjusted CSF GDF15 is present in the majority of PPMS patients, but it is not inhibited by idebenone.

Keywords: Idebenone, Primary progressive multiple sclerosis, GDF15, Mitochondrial dysfunction, Clinical trail

1. Introduction

Weak efficacy of immune-targeting disease-modifying treatments (DMTs) in progressive multiple sclerosis (MS) suggests that the immune system may not be the sole or even the main driver of central nervous system (CNS) destruction in late stages of MS. Many alternative, neuro-degenerative mechanisms have been described on pathology of progressive MS, with perhaps the strongest evidence implicating mitochondrial dysfunction (Lassmann et al., 2012). With the inability to measure these putative pathogenic processes in living MS patients, it is unknown which one of them associates with MS severity and which may represent an epiphenomenon. Additionally, while progressive MS has been divided into primary-progressive (PPMS) and secondary-progressive MS ([SPMS], preceded by relapsing-remitting [RRMS] stage), it was unclear whether SPMS and PPMS represent pathophysiologically distinct diseases.

Consequently, we sought to develop and validate biomarkers of potentially pathogenic intrathecal processes such as inflammation, mitochondrial dysfunction or neuronal loss, and to identify molecular differences between clinical MS subtypes. Achieving this goal required access to large numbers of longitudinal cerebrospinal fluid (CSF) samples collected from deeply-phenotyped progressive MS subjects. Thus, we devised in parallel two adaptive, randomized, CSF biomarker-supported, placebo-controlled clinical trials in progressive MS to establish SPMS and PPMS cohorts for comparative analysis, while providing patients with a possibility of direct clinical benefit of interventional trials, since there were no FDA-approved treatments for progressive MS at that time.

The first trial, “The double-blind combination of Rituximab by IntraVenous and IntraThecAl injection versus placebo in patients with Low-Inflammatory SEcondary progressive MS” (RIVITALISE; NCT01212094) tested the hypothesis that intrathecal B cells support compartmentalization of inflammation to CNS through tertiary lymphoid follicles and thus contribute to disability progression. The RIVITALISE trial was terminated prematurely after the pre-determined interim analysis showed insufficient depletion of intrathecal B cells (Komori et al., 2016 ). The second trial, “The double-blind, placebo-controlled Phase I/II clinical trial of Idebenone in patients with Primary Progressive MS” (IPPoMS; NCT00950248) is reported here.

Idebenone (2,3-dimethoxy-5-methyl-6-(10-hydroxydecyl)-1,4-benzoquinone) is a synthetic quinone similar to the naturally occurring coenzyme Q10 (CoQ10) differing by the presence of a shorter, less lipophilic tail responsible for its distinct physicochemical properties. CoQ10 is a lipophilic constituent of the mitochondrial electron-transport chain (ETC). Dysfunctional ECT generates reactive oxygen species (ROS), such as highly reactive semiquinones, at complexes I and III that in turn oxidize ECT components, initiating a vicious circle of mitochondrial damage. Cellular response to resulting oxidative injury increases expression of “detoxifying enzymes”, such as cytosolic NAD (P)H:quinine oxidoreductases (NQO1 and NQO2). NQOs substitute the production of reactive semiquinones with relatively stable hydroquinones (Haefeli et al., 2011). While lipophilic CoQ10 cannot access this cytosolic detox system, more hydrophilic idebenone can shuttle electrons between the dysfunctional ECT in the mitochondrial membrane and cytosolic NQO1. The NQO1-reduced idebenone can then bring electrons downstream of dysfunctional complex I, partially restoring adenosine triphosphate (ATP) production (Haefeli et al., 2011) and decreasing mitochondrial damage (Suno and Nagaoka, 1989). This mechanism explains efficacy of idebenone in Leber’s hereditary optic neuropathy, a genetic dysfunction of ECT complex I (Klopstock et al., 2011).

MS-associated demyelination leads to increased ATP demand in demyelinated axons (Mahad et al., 2009, Zambonin et al., 2011). Presumably, when the increased energy demand exceeds efficiency of ATP generation (which decreases in physiological aging), mitochondrial dysfunction, especially of the complex I, ensues (Campbell et al., 2012, Campbell et al., 2011, Dutta et al., 2006). As expected, this upregulates expression of NQOs in MS lesions (van Horssen et al., 2006), providing an opportunity for therapeutic effect of idebenone in MS. Consequently, the IPPoMS trial tested the hypothesis that mitochondrial dysfunction contributes to neuronal damage in PPMS and idebenone, through its mitochondria-protective function, will partially ameliorate disability progression in PPMS patients. This hypothesis has effectively three parts: 1. That mitochondrial dysfunction, in excess of natural aging is associated with PPMS; 2. That idebenone at least partially ameliorates this mitochondrial dysfunction and 3. That the idebenone-driven improvement of mitochondrial function will slow down disability progression in PPMS. An integral part of the trial was identification/validation of CSF biomarker(s) of mitochondrial dysfunction applicable to MS and investigation whether idebenone treatment exerts discernable effect on such intrathecal biomarker(s).

The traditional, although not very sensitive biomarker of mitochondrial dysfunction is elevated lactate in biological fluids. More recently, growth/differentiation factor 15 (GDF15) has been proposed as a more sensitive biomarker of mitochondrial respiratory chain dysfunction in the peripheral blood in a spectrum of metabolic mitochondrial disorders (Koene et al., 2015, Montero et al., 2016, Poulsen et al., 2020) as well as neurodegenerative diseases of the CNS, including MS (Miyaue et al., 2020, Nohara et al., 2019). However, the role of GDF15 in CSF as a biomarker of intrathecal mitochondrial dysfunction has not been determined and therefore it has become an exploratory outcome of this study along with sCD14, previously identified as a marker of intrathecal activation of myeloid lineage, both on cellular level (Han et al., 2014) and as a soluble biomarker (Komori et al., 2015). We also measured change in CSF NFL as a marker of axonal damage and albumin quotient as marker of subtle blood brain barrier dysfunction.

2. Methods

2.1. Patients

Patients were prospectively enrolled at the National Institutes of Health (NIH), Bethesda, USA. The eligibility criteria included diagnosis of PPMS according to 2005 McDonald’s criteria (Polman et al., 2005), age range 18–65 (inclusive), Expanded Disability Status Scale (EDSS) range of 1–7, ability to provide informed consent, commitment to use birth control if able to become pregnant, not receiving any immunomodulatory/immunosuppressive therapies for at least 3 months before enrollment, and no exposure to idebenone, CoQ10, or other dietary supplements for at least 1 month before enrollment in the study. Exclusion criteria included clinically significant medical disorders that could cause CNS tissue damage or limit its repair, a history of hypersensitivity reaction to idebenone or CoQ10, pregnancy, abnormal baseline blood tests, and immunosuppressive therapies. The study was approved by the Scientific review committee of the National Institute of Neurological Disorders and Stroke (NINDS), by Combined Neuroscience Institutional Review Board of the NIH and monitored by an independent Data and Safety Monitoring Board (DSMB). All patients provided a written informed consent.

2.2. Trial design

IPPoMS was a randomized Phase I/II safety/efficacy baseline-versus-treatment trial with an adaptive trial design (Fig 1A): one-year pre-treatment baseline, followed by two-year double-blind, treatment phase randomized into a daily dose of 2250mg of idebenone (3 × 750mg per day) and placebo. The block stratified (block of two) 1:1 randomization with age as a single condition (50 < age ≥ 50) was performed by the NIH pharmacy that released the randomization code to the investigators after the trial completion and locking of the data. Patients were followed every 6 months by neurological, neuroimaging, and research biomarker/immunological evaluation (Fig 1A). An Investigational New Drug (IND; #104,895) for idebenone was obtained from the Food and Drug Administration (FDA) with the principal investigator acting as IND sponsor. Santhera Pharmaceuticals provided idebenone for the study free of charge, under collaborative agreement with NIH/NINDS.

Fig. 1. IPPoMS trial.

Fig. 1.

(A) An overview of the IPPoMS trial design consisting of one-year pre-treatment baseline and 2 years of double-blind treatment phase, schedule of clinical and MRI evaluations, collection of biological samples, and laboratory safety evaluation procedures. (B) CONSORT diagram of IPPoMS trial. The Pre-treatment safety population includes all enrolled patients, Safety population included randomized patients that received at least one dose of idebenone/placebo, Intention-to-treat population was used to evaluate the effect of idebenone on the primary outcome and included patients that completed at least one six-month follow-up evaluation after randomization. Completer population includes patients that finished two years of the double-blind phase of the trial.

2.3. Trial endpoints

The primary outcome was the treatment-induced change in disability progression, assessed with the Area Under the Curve (AUC) of Combinatorial, Weight-adjusted disability score (CombiWISE) (Kosa et al., 2016). For more details on primary outcome, see Supplementary Data. Secondary outcomes included progression of enlargement of brain ventricular volume, progression in EDSS-plus (Cadavid et al., 2017), progression of lower extremity disability assessed by 25FW (Schwid et al., 1997), progression of upper extremity/ fine motor movements disability assessed by 9-hole peg test (9HPT) (Goodkin et al., 1988), progression of neurological disability assessed by Scripps Neurological Rating Scale (SNRS) (Sipe et al., 1984), and progression of neurological disability assessed by EDSS (Kurtzke, 1983). Exploratory outcomes included progression of retinal nerve fiber layer (RNFL) thinning detected by optical coherence tomography (OCT), progression in cognitive dysfunction assessed by Symbol Digit Modalities Test (SDMT) (Smith, 1982), progression of neurological disability assessed by Multiple Sclerosis Functional Composite (MSFC) (Fischer et al., 1999), and therapy-induced changes in CSF albumin quotient, sCD14, lactate, GDF15, and neurofilament light chain (NFL). The primary safety endpoints included premature discontinuation of study treatment, serious adverse events (SAE), and all adverse events (AE).

For additional Methods see Supplementary Data.

3. Results

3.1. Patients

Between November 1, 2009 and July 23, 2015, 85 patients were assessed for eligibility and enrolled into the IPPoMS trial (Fig 1B), with 77 patients randomized into a daily dose of 2250mg (3 × 750mg per day) idebenone (N=39) or placebo (N=38). Out of those, 38 patients in the idebenone group and 35 patients in the placebo group, underwent at least one follow-up visit in the double-blind phase (Table 1). All demographic and baseline characteristics were balanced between trial groups. 33 patients in idebenone group and 33 patients in placebo group completed the two-year double-blind phase. The one-year baseline follow-up on 85 enrolled patients amounted to 88.6 patient years, averaging 1.04 (standard deviation [SD] = 0.17) years. The average duration of the double-blind phase was 1.80 (SD=0.46) years for the 39 patients randomized to idebenone and 1.81 (SD=0.49) years for the 38 patients randomized to placebo, constituting 70.01 and 68.73 patient years, respectively (Table 2).

Table 1.

Demographic data (Intention-To-Treat Population).

Idebenone Placebo

Patients (N) 38 35
Gender (% females) 42.1 51.4
Age at baseline (years; mean, ± SD, range) 56.7 ( ± 6.9, 42.1–65.7) 55.5 ( ± 7.1, 35.9–70.2)
Age at disease onset (years; mean, ± SD, range) 43.0 ( ± 11.3, 18.5–63.7) 43.2 ( ± 9.0, 24.2–57.7)
Time since disease onset at baseline (years; mean, ± SD, range) 13.7 ( ± 9.1, 0.2–38.7) 12.3 ( ± 7.8, 1.6–30.1)
Body Mass Index (mean, ± SD, range) 25.7 ( ± 9.12, 16.4–33.9) 25.8 ( ± 4.3, 19.9 – 39.0)
History of Mononucleosis (% positive) 26.9 26.1
Northern European Ancestry (% positive) 77.3 81.0
Family History of MS (% positive) 31.6 21.2
History of Smoking (% positive) 75.7 90.0

Table 2.

Extent of exposure or follow-up.

Duration of follow-up (years) (Pre-treatment phase safety population)
Phase Group Mean SD Min Median Max N Patient years
Pre-treatment Total 1.04 0.17 0.3 1.02 1.71 85 88.6
Duration of follow-up or extent of exposure (years) (Safety population)
Phase Group Mean SD Min Median Max N Patient years
Pre-treatment Idebenone 1.07 0.13 0.96 1.02 1.71 39 41.59
Placebo 1.05 0.13 0.90 1.01 1.69 38 40.04
Double-blind Idebenone 1.80 0.46 0.00 1.95 2.11 39 70.01
Placebo 1.81 0.49 0.01 1.97 2.09 38 68.73

3.2. Safety and tolerability

There were 115 AEs recorded during the pre-treatment phase in 40 patients (47.1% of the Pre-treatment Safety Population [PSP]), out of those 26 represented SAEs in 12 patients (14.1% of the PSP) (Supplementary Table 2). The most frequent AEs during the pre-treatment phase were falls (8.2% of the PSP) and urinary tract infections (UTIs, 7.1% of the PSP). In the Safety Population (SP) there were 108 AEs in the idebenone group (N=39) that occurred during the double-blind phase in 26 patients, representing 66.7% of the population. Out of those, 35 were classified as SAEs and occurred in 11 patients (28.2% of the population). In the placebo arm of the SP (N=38), there were 99 AEs that occurred in 21 patients (55.3% of population). 39 of those occurring in 10 patients (26.3% of the population) were classifies as SEAs. There were two deaths in the placebo arm; one occurred in a patient following stroke, bowel obstruction, pneumonia, and sepsis. The second death was related to cholelithiasis. The most frequent AEs in the SP were falls (12.8% and 13.2% of the idebenone and placebo SPs, respectively), and UTIs (7.7% and 10.5% of the idebenone and placebo SP, respectively). There was no statistically significant difference in occurrence and severity of AEs between the treatment groups indicating that a daily dose of 2250mg idebenone was well tolerated.

3.3. Primary endpoint results

The primary endpoint of the trial was the effect of idebenone compared to placebo on the disability progression using AUC of CombiWISE (Fig 2A) in the baseline-versus-treatment paradigm. CombiWISE was assessed every 6 months: months −12, −6, and 0 during the pre-treatment baseline and months 0, 6, 12, 18, and 24 during the double-blind phase (Fig 2B). The mean change of CombiWISE AUC between baseline and double-blind phase in the idebenone group was −0.13 (SD=2.17) and it did not reach statistical significance (p=0.77); the mean change of CombiWISE AUC within the placebo group was −1.04 (SD=2.87), reaching p-value of 0.03 (Fig 2C); providing evidence for measurable placebo effect. Between-group difference of CombiWISE AUCs tested using the Analysis of Covariance (ANCOVA) method (with baseline AUC, baseline CombiWISE, and baseline age as covariates) estimated a treatment difference of 0.15 (95% Confidence Interval [CI]: −0.75–1.05) between idebenone and placebo and didn’t reach statistical significance (p=0.74) (Fig 2D). The lack of statistically significant inhibition of disease progression as measured by CombiWISE was also confirmed using the piecewise linear mixed-effect model (Supplementary Table 3).

Fig. 2. Primary endpoint of IPPoMS trial.

Fig. 2.

(A) The primary endpoint of the IPPoMS trial was based on measurements of the Area Under the Curve (AUC) of CombiWISE scores collected every six months between the pre-treatment baseline (blue lines and blue-shaded area, month −12 to 0) and treatment phase (red line and red-shaded area, month 0 – 24). (B) Group data of CombiWISE values for placebo (top) and idebenone (bottom) show accumulation of disability measured by CombiWISE in both groups as depicted by medians (red bars) and interquartile ranges (blue whiskers) at 6 month-visits. (C) The difference in CombiWISE AUC between baseline and treatment phase was not statistically significant in either placebo (top) or treatment (bottom) group and (D) there was no statistical significance and AUC during the treatment phase between placebo and idebenone group (red bars and blue whiskers represent group median and interquartile range, respectively). (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

3.4. Secondary endpoints results

All components of CombiWISE (EDSS, SNRS, 25FW, and 9HPT) were also tested for efficacy of idebenone in inhibiting disease progression using a piecewise linear mixed-effects model that estimated the slopes for the baseline and treatment phases. There was no statistically significant difference in slopes of disease progression between baseline and double-blind phase in either placebo or idebenone group, nor was there a significant difference in slopes of disease progression during the double-blind phase between idebenone and placebo group (Fig 3, Supplementary Table 3).

Fig. 3. Secondary endpoint outcomes in IPPoMS trial.

Fig. 3.

Group data for CombiWISE components EDSS, SNRS, 25FW, and 9HPT in placebo (left) and idebenone (right) arm measured every 6 months during the 3-year trial. Red bars represent medians and blue whiskers show interquartile range. EDSS = Expanded Disability Status Scale, SNRS = Scripps Neurological Disability Scale, 25FW = timed 25 foot walk, 9HPT = 9 hole peg test. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

An analysis of ventricular volume AUCs showed no statistically significant difference in enlargement of ventricular volume between idebenone and placebo (average difference between baseline and treatment phase AUC of −244 [SD=1094.9] and 35.4 [SD=1091.1], respectively, Fig 4). The lack of statistically significant inhibition of ventricular volume enlargement was also confirmed by piecewise linear mixed-effects model (Supplementary Table 3).

Fig. 4. Ventricular volume as a secondary endpoint of IPPoMS trial.

Fig. 4.

(A) Group data for ventricular volume in placebo (left) and idebenone (right) arm assessed every 6 months during the 3-year IPPoMS trial. (B) The comparison of calculated AUC during baseline and treatment phase of the IPPoMS trial in placebo (left) and idebenone (right) arm. (C) Comparison of AUCs of the treatment phase for ventricular volume between placebo and idebenone arm shows no statistically significant difference. Red bars represent medians and blue whiskers show interquartile range. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

A composite measure EDSS-plus showed no statistically significant difference in median time to disease progression between idebenone and placebo group (23.1 months [CI: 12.8-NA] and 23.7 months [CI: 19.8-NA], respectively, [the upper limit of the 95% CI was not estimated due to small number of patients at risk], Supplementary Figure 1).

3.5. Exploratory outcomes results

OCT, MSFC, and SDMT analyzed using a piecewise linear mixed-effects model showed no significant difference in the RNFL outcome or in cognitive abilities measured by SDMT between baseline and double-blind phase in idebenone and placebo group (Fig 5, Supplementary Table 3). There was a statistically significant disability progression measured by MSFC in the idebenone group (p=0.0023), suggesting worsening of disability. This is likely a result of significantly different baseline slopes (p=0.0448) between idebenone and placebo group: while there was a negative baseline slope in the placebo group (suggesting worsening) there was a positive baseline slope in the idebenone group (suggesting improving). However, there was no statistically significant difference in treatment slopes between idebenone and placebo.

Fig. 5. Imaging and clinical exploratory outcomes of IPPoMS trial.

Fig. 5.

Group data showing disease progression measured by composite scale MSFC, cognitive test SDMT and OCT outcome RNFL during the 3-year trial in placebo (left) and idebenone (right) arm. Red bars represent medians and blue whiskers show interquartile range. MSFC = Multiple Sclerosis Functional Composite, SDMT = Symbol Digit Modality Test, OCT = Optical Coherence Tomography, RNFL = Retinal Nerve Fiber Layer. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

3.6. Development of GDF15 as a biomarker of intrathecal mitochondrial dysfunction

While GDF15 was proposed as sensitive biomarker of mitochondrial dysfunction in periphery, this protein plays physiological role in CNS tissue, where it regulates food intake and energy expenditure in response to metabolic and toxin-induced stress, by activating neurons localized in the area postrema and nucleus tractus solitarius of the brainstem. Therefore, we sought to mechanistically confirm that CNS cells release GDF15 to CSF in response to mitochondrial dysfunction, modelled by in-vitro cultures of primary human CNS cells and cell lines exposed to mitochondrial toxin, rotenone. In the absence of rotenone, we observed preferential secretion of GDF15 by primary human astrocytes and choroid plexus epithelial cells (HCPEpiC), while neurons, CNS-derived endothelial, and microglia cell lines did not secrete GDF15. Addition of rotenone resulted in increased GDF15 secretion in astrocytes and HCPEpiC (2.6-fold and 1.7-fold, respectively); no GDF15 secretion was observed in other tested cell lines (Supplementary Figure 2). Thus, we conclude that increased CSF levels of GDF15 may reflect overall mitochondrial damage in CNS tissue, but is unlikely to reflect mitochondrial dysfunction specifically in neurons or demyelinated axons.

To determine if GDF15 CSF levels are elevated in PPMS subjects in excess of physiological aging, we assessed CSF GDF15 levels from patient CSF samples across all MS subtypes: RRMS (N=129), SPMS (N=70), and PPMS (N=116), and healthy volunteers (HV, N=39, Supplementary Table 4). We observed a strong, statistically significant correlation with age in all diagnostic categories (Fig 6A). Therefore, we used the linear regression model from the HV cohort to adjust MS patient data for physiological aging processes. These age-adjusted residuals of CSF GDF15 levels were significantly higher in progressive MS subgroups compared to HV, and in PPMS compared to both SPMS and RRMS subgroups (Fig 6B).

Fig. 6. Examining associations among age, GDF15 concentration, and diagnoses.

Fig. 6.

(A) Simple linear regression (red lines) predicting GDF15 from age, separately for HVs and subjects with RRMS, SPMS, and PPMS. The equation of linear regression line is shown in all graphs, followed by correlation coefficient and p-value, coefficient of variation, and number of samples analyzed. (B) Age-adjusted GDF15 values across diagnoses, calculated from residuals from the HV model. Red horizontal lines represent mean of the group and red whiskers show 95% confidence interval (CI) of the mean. Blue points represent values that are above the upper 95% CI of healthy volunteers (HV limit); blue % numbers show proportion of subjects above HV limit in each diagnostic group. **** p<0.0001, ** p<0.05, *p<0.05. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

Thus, we conclude that CSF GDF15 levels reflect, at least partially, intrathecal mitochondrial dysfunction, increase with physiological aging, but after adjusting for normal aging process, they remain elevated in progressive MS, especially in PPMS subtype. This validates utility of CSF GDF15 as a pharmacodynamic marker of intrathecal mitochondrial dysfunction in IPPoMS trial.

3.7. CSF biomarker results

To mechanistically explore intrathecal effects of idebenone, we measured CSF biomarkers of mitochondrial damage (i.e., GDF15 and lactate), of activation of myeloid lineage (sCD14), of subtle blood brain barrier dysfunction (albumin quotient) and of axonal damage (NFL) at baseline and at month 12 of treatment phase as exploratory outcomes. The statistical analysis revealed strong effect of age on GDF15, NFL, and sCD14, therefore age was used a covariate in the ANCOVA analysis. There was a statistically significant decrease in lactate levels between baseline and double-blind phase in both idebenone and placebo group. An increase of borderline statistical significance was identified between baseline and treatment values of albumin quotient (p=0.0319) and GDF15 (p=0.0251) in the idebenone group, however, the increase in GDF15 disappeared after age adjustment (Supplementary Figure 3). Moreover, an analysis of only patients with baseline CSF GDF15 levels exceeding physiological aging (postulating that these patients had intrathecal mitochondrial dysfunction) resulted in no statistically significant treatment-induced changes. Statistically significant increase of sCD14 levels between baseline and treatment phase has been observed in both idebenone and placebo group (p=0.0389 and p=0.0239, respectively). None of the CSF biomarkers showed statistically significant differences between placebo and idebenone group in either baseline or treatment phase (Fig 7, Supplementary Table 5).

Fig. 7. CSF biomarkers as exploratory outcomes of IPPoMS trial.

Fig. 7.

Baseline versus treatment comparison of albumin quotient, soluble CD14, GDF15, Neurofilament light chain (NFL), and L-lactate in placebo (left) and idebenone (right) arm. Red bars represent medians, blue dotted lines represent healthy control cut-offs. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

4. Discussion

The identification of contrast enhancing lesions (CELs) as surrogate marker of MS relapses allowed rapid screening of immunomodulatory treatments for efficacy in MS, leading to significant expansion of therapeutic options for RRMS. However, CELs, relapses, and the efficacy of these drugs on progression of clinical disability decrease with MS duration and patients’ age (Filippi et al., 2001, Tremlett et al., 2008, Wolinsky et al., 2000). On average, immunomodulatory MS drugs have zero efficacy on disability progression measured by EDSS after mean age of 53 years (Weideman et al., 2017).

Pathologists offered multiple alternative processes that may underlie disability progression in the absence of formation of new MS lesions: mitochondrial dysfunction linked to demyelination of axons, associated oxidative stress, endoplasmic reticulum stress, excitotoxicity and compartmentalized inflammation (Lassmann, 2018). Only interventional clinical trials may determine which of these processes are truly pathogenic, versus represent an epiphenomenon. But because we lack sensitive outcomes, these clinical trials in progressive MS must study large cohorts, making them extremely expensive and rare. Additionally, without pharmacodynamic readout, our understanding of the effects of these drugs on disease mechanisms remains limited. Finally, when using insensitive outcomes, or outcomes susceptible to bias, the success in Phase II trials does not guarantee a validation in Phase III studies as recently demonstrated by biotin for the treatment of progressive MS (Tourbah et al., 2016 ) (https://www.businesswire.com/news/home/20200310005779/en/MedDay-Reports-Top-Line-Data-Phase-III-Trial). Therefore, the goal of our research program on progressive MS, initiated a decade ago, was to deeply characterize longitudinal cohorts of PPMS and SPMS patients to compare, develop, and validate sensitive outcomes for Phase II trials and to collect longitudinal CSF samples to gain mechanistic insight into biological processes that correlate with disease progression. We selected two candidate pathogenic processes to study within investigator-initiated progressive MS trials: 1. Compartmentalized inflammation (targeted in RIVITALISE trial (Komori et al., 2016 )) and 2. Intrathecal mitochondrial dysfunction (targeted in IPPoMS trial).

While idebenone was safe and well-tolerated, the IPPoMS trial did not achieve efficacy on any of its primary or secondary endpoints, providing high confidence that idebenone does not inhibit disability progression or CNS tissue destruction in PPMS. One concern may be raised that IPPoMS used primary outcome (i.e., CombiWISE) that has not been tested in any clinical trial before. The adaptive part of the protocol demonstrated unequivocally (Kosa et al., 2016) that the published power calculation for brain atrophy measured by Structural Image Evaluation, using Normalization, of Atrophy (SIENA) in SPMS (Altmann et al., 2009) is overly optimistic. This conclusion is fully supported by recent report of MS-SMART clinical trial (De Angelis et al., 2020) that reported much smaller progression of brain atrophy over two years, measured by SIENA in the placebo arm (i.e., −1.29% +/− 1.1 SD; n = 99) in comparison to the original publication by Altman et al (i.e., −2.47% +/− 1.57 SD; n= 28) (Altmann et al., 2009). In fact, the progression of brain atrophy observed over 2-year follow up in MS- SMART trial is fully compatible with our measurements from IPPOMS one year pre-treatment baseline (Kosa et al., 2016). Using brain atrophy, or any outcome other than CombiWISE would require cohort size that greatly exceeded possibilities of the investigator-initiated, single center trial. Nevertheless, following facts make us confident about validity of CombiWISE outcome: 1. The analyses of 58 measured outcomes in the baseline phase of the IPPoMS protocol were pre-determined in the protocol and CombiWISE was a clear winner (Kosa et al., 2016); 2. CombiWISE was developed using strictly data-driven approach and its ability to measure disease progression was validated in two independent cohorts of progressive MS patients. In fact, both placebo and idebenone treatment arms demonstrated continued, measurable disability progression on CombiWISE (Fig 2B) during the two years of follow-up, supporting high sensitivity of this clinical outcome. 3. Comparison of CombiWISE with EDSS in 303 MS patients demonstrated excellent correlation (i.e., Rho = 0.9805, p<0.0001) (Kosa et al., 2016). Finally, the fact that we observed no therapeutic effect of idebenone, not even a trend, on any traditional clinical, MRI or CSF NFL outcomes, strongly supports our conclusion that the results of the primary outcome are valid and that idebenone exerts no therapeutic benefit in PPMS.

Why is idebenone ineffective? Is it because our PPMS cohort lacked evidence of intrathecal mitochondrial dysfunction or because idebenone exerted no discernable effect on such intrathecal process? The CSF biomarkers provide some mechanistic insight: The lack of efficacy on NFL (even a trend of efficacy) is consistent with the observed lack of efficacy on disability progression and brain atrophy. In both idebenone and placebo groups, we measured comparable, statistically significant increases in CSF sCD14 and decreases in CSF lactate over two years of follow-up. Thus, we conclude that these effects are not related to treatment. Increase in sCD14, a biomarker expressed only on the surface of monocytes, macrophages and microglia, that was previously associated with progressive MS likely reflects increased activation of these cells during MS progression (Komori et al., 2015). The decrease in CSF lactate levels is counterintuitive. Because we are not aware of other published data that would support decrease in CSF lactate levels during MS evolution, we should keep in mind that this result may be false positive, perhaps due to undisclosed changes in measurement methodology by NIH clinical laboratory. Alternatively, as CSF lactate can be produced by activated immune cells and by astrocytes and oligodendrocytes as an alternative food source for axons in the form of “lactate shuttle” (Philips and Rothstein, 2017), decrease in CSF lactate levels may be due to decreased production of lactate by these alternative lactate sources during MS evolution. This hypothesis should be tested in future studies.

We put enormous effort to attempt to validate CSF GDF15 as a surrogate marker of MS-associated mitochondrial dysfunction. We observed results that support this concept: 1. Increase in GDF15 secretion by CNS cells after treatment with mitochondrial toxin rotenone in-vitro; 2. Increase in CSF GDF15 levels with natural aging; 3. Increase in HV age-adjusted CSF GDF15 levels in all stages of MS, with further, statistically significant increase in both progressive MS subgroups (66–68% of subjects) in comparison to RRMS (40% of subjects, Fig 6B). However, remaining results caution the interpretation that GDF15 is unequivocal biomarker of MS-associated mitochondrial dysfunction. Based on MS pathology studies, the mitochondrial dysfunction should affect mostly neurons (i.e., demyelinated axons), but we did not see significant increase in GDF15 secretion from neurons exposed to rotenone; the cells that significantly increased production of GDF15 in response to rotenone were astrocytes. This observation supports previously-described glial secretion and neuronal consumption of GDF15 in mice, where GDF15 exerts trophic support for dopaminergic and motor neurons (Strelau et al., 2003, Strelau et al., 2009). Thus, although age-related increase in CSF GDF15 levels, observed both in HV and MS cohorts may represent mitochondrial dysfunction, it may also reflect age-related neuronal loss and gliosis. If CSF GDF15 is true biomarker of mitochondrial dysfunction of CNS cells, which our data don’t unequivocally support, then idebenone not only did not inhibit mitochondrial dysfunction in PPMS, but may have even slightly exacerbate it based on increase in GDF15 (and albumin quotient) that reached borderline significance in the idebenone group. Albumin quotient was measured as a candidate biomarker of subtle blood brain barrier damage, previously associated with progressive MS. Therefore its increase would also be considered detrimental (LeVine, 2016). Nevertheless, we are extremely cautious about making conclusions from such subtle changes of borderline significance, as these likely represent Type 1 errors.

Can we speculate why was idebenone ineffective in PPMS? The clinical trials on Leber’s hereditary optic neuropathy (LHON) (Klopstock et al., 2011, Maresca et al., 2013, Yu-Wai-Man et al., 2016), a genetic disorder of Complex I ECT affecting mostly young adults show only weak efficacy and narrow therapeutic window for the ability of idebenone to restore axonal function after acute demyelinating event; it is likely that most PPMS patients were outside of this therapeutic window. Second, in LHON studies, idebenone was more effective in preventing further demyelination, which may not be a frequent event in the older PPMS population targeted in IPPOMS trial. But perhaps most importantly, both our in-vitro data and recently published mechanistic data in rodent-derived CNS cells (Jaber et al., 2020) point to the fact that idebenone can limit mitochondrial dysfunction in astrocytes, but not in neurons. The genetic defect of LHON affects all CNS cells and astrocytic mitochondrial dysfunction may exacerbate neuronal dysfunction in this disease. In contrast, in MS the mitochondrial dysfunction is secondary to demyelination, and therefore it would not be expected to affect astrocytes. Unfortunately, this knowledge was not available when IPPoMS trial was initiated.

Although both RIVITALISE and IPPoMS clinical trials were negative, our experience shows that incorporating CSF biomarkers to early phases of progressive MS drug development advances our understanding of MS disease mechanisms and generates clinically useful tools (Barbour et al., 2017). This fulfils the promise of obtaining generalizable knowledge that represents the ethical basis of early clinical trials with uncertain outcomes.

Supplementary Material

MM6
MM5
MM4
MM3
MM2
MM1

Acknowledgments

The study was supported in part by the intramural research program of the National Institute of Allergy and Infectious Diseases (NIAID) (grant no. NS003055-11) of the National Institutes of Health (NIH) and by a clinical trial agreement (CTA#2009-0004) between NIH and Santhera Pharmaceuticals that provided idebenone without restriction for the trial. We would like to thank Günther Metz for the management of the collaborative alliance between NIH and Santhera Pharmaceuticals. We also thank IPPoMS DSMB members, Drs. Mitchell T. Wallin (DSMB chair), Walter Royal, Carlos Mora, and Ludwig Kappos for their careful trial monitoring and Dr. Judy Starling from NIH pharmacy for supervising randomization. Finally, we thank patients and their caregiver without whom this study would not be possible.

5. Funding source

The study was supported in part by the intramural research program of the National Institute of Allergy and Infectious Diseases (NIAID) (grant no. NS003055-11) of the National Institutes of Health (NIH) and by a clinical trial agreement (CTA#2009-0004) between NIH and Santhera Pharmaceuticals that provided idebenone without restriction for the trial.

Footnotes

CRediT authorship contribution statement

Peter Kosa: Formal analysis, Data curation, Writing - original draft, Writing - review & editing, Visualization. Tianxia Wu: Formal analysis, Software, Methodology. Jonathan Phillips: Investigation. Mika Leinonen: Formal analysis, Software, Methodology. Ruturaj Masvekar: Investigation. Mika Komori: Investigation. Alison Wichman: Investigation. Mary Sandford: Investigation. Bibiana Bielekova: Conceptualization, Methodology, Investigation, Writing - original draft, Writing - review & editing, Supervision, Project administration, Funding acquisition.

Declaration of Competing Interests

Mika Leinonen is an employee of Santhera Pharmaceuticals. All remaining authors declare no conflict of interest the subject matter or materials discussed in this manuscript.

Supplementary materials

Supplementary material associated with this article can be found, in the online version, at doi:10.1016/j.msard.2020.102434.

References

  1. Altmann DR, Jasperse B, Barkhof F, et al. , 2009. Sample sizes for brain atrophy outcomes in trials for secondary progressive multiple sclerosis. Neurology 72 (7), 595–601 Feb 17. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Barbour C, Kosa P, Komori M, et al. , 2017. Molecular-based diagnosis of multiple sclerosis and its progressive stage. Ann. Neurol 82 (5), 795–812 Nov. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Cadavid D, Cohen JA, Freedman MS, et al. , 2017. The EDSS-Plus, an improved endpoint for disability progression in secondary progressive multiple sclerosis. Mult. Scler 23 (1), 94–105 Jan. [DOI] [PubMed] [Google Scholar]
  4. Campbell GR, Ohno N, Turnbull DM, Mahad DJ, 2012. Mitochondrial changes within axons in multiple sclerosis: an update. Curr. Opin. Neurol 25 (3), 221–230 Jun. [DOI] [PubMed] [Google Scholar]
  5. Campbell GR, Ziabreva I, Reeve AK, et al. , 2011. Mitochondrial DNA deletions and neurodegeneration in multiple sclerosis. Ann. Neurol 69 (3), 481–492 Mar. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. De Angelis F, Connick P, Parker RA, et al. Amiloride, fluoxetine or riluzole to reduce brain volume loss in secondary progressive multiple sclerosis: the MS-SMART four-arm RCT. Southampton (UK) 2020. [PubMed] [Google Scholar]
  7. Dutta R, McDonough J, Yin X, et al. , 2006. Mitochondrial dysfunction as a cause of axonal degeneration in multiple sclerosis patients. Ann. Neurol 59 (3), 478–489 Mar. [DOI] [PubMed] [Google Scholar]
  8. Filippi M, Wolinsky JS, Sormani MP, Comi G, 2001. European/Canadian Glatiramer acetate study G. Enhancement frequency decreases with increasing age in relapsing-remitting multiple sclerosis. Neurology 56 (3), 422–423 Feb 13. [DOI] [PubMed] [Google Scholar]
  9. Fischer JS, Rudick RA, Cutter GR, Reingold SC, 1999. The multiple sclerosis functional composite measure (MSFC): an integrated approach to MS clinical outcome assessment. National MS society clinical outcomes assessment task force. Mult Scler. 5 (4), 244–250 Aug. [DOI] [PubMed] [Google Scholar]
  10. Goodkin DE, Hertsgaard D, Seminary J, 1988. Upper extremity function in multiple sclerosis: improving assessment sensitivity with box-and-block and nine-hole peg tests. Arch. Phys. Med. Rehabil 69 (10), 850–854 Oct. [PubMed] [Google Scholar]
  11. Haefeli RH, Erb M, Gemperli AC, et al. , 2011. NQO1-dependent redox cycling of idebenone: effects on cellular redox potential and energy levels. PLoS One 6 (3), e17963. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Han S, Lin YC, Wu T, et al. , 2014. Comprehensive immunophenotyping of cerebrospinal fluid cells in patients with neuroimmunological diseases. J. Immunol 192 (6), 2551–2563 Mar 15. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Jaber SM, Ge SX, Milstein JL, VanRyzin JW, Waddell J, Polster BM, 2020. Idebenone has distinct effects on mitochondrial respiration in cortical astrocytes compared to cortical neurons due to differential NQO1 activity. J. Neurosci 40 (23), 4609–4619 Jun 3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Klopstock T, Yu-Wai-Man P, Dimitriadis K, et al. , 2011. A randomized placebo-controlled trial of idebenone in Leber’s hereditary optic neuropathy. Brain 134 (Pt 9), 2677–2686 Sep. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Klopstock T, Yu-Wai-Man P, Dimitriadis K, et al. , 2011. A randomized placebo-controlled trial of idebenone in Leber’s hereditary optic neuropathy. Brain Jul 25. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Koene S, de Laat P, van Tienoven DH, et al. , 2015. Serum GDF15 levels correlate to mitochondrial disease severity and myocardial strain, but Not to disease progression in adult m.3243A>G carriers. JIMD Rep. 24, 69–81. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Komori M, Blake A, Greenwood M, et al. , 2015. Cerebrospinal fluid markers reveal intrathecal inflammation in progressive multiple sclerosis. Annals of neurology 78 (1), 3–20 Jul. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Komori M, Lin YC, Cortese I, et al. , 2016. Insufficient disease inhibition by intrathecal rituximab in progressive multiple sclerosis. Ann. Clin. Transl. Neurol 3 (3), 166–179 Mar. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Kosa P, Ghazali D, Tanigawa M, et al. , 2016. Development of a sensitive outcome for economical drug screening for progressive multiple sclerosis treatment. Front. Neurol 7, 131. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Kurtzke JF., 1983. Rating neurologic impairment in multiple sclerosis: an expanded disability status scale (EDSS). Neurology 33 (11), 1444–1452 Nov. [DOI] [PubMed] [Google Scholar]
  21. Lassmann H, van Horssen J, Mahad D, 2012. Progressive multiple sclerosis: pathology and pathogenesis. Nat. Rev. Neurol 8 (11), 647–656 Nov 5. [DOI] [PubMed] [Google Scholar]
  22. Lassmann H, 2018. Multiple sclerosis pathology. Cold Spring Harb. Perspect. Med 8 (3) Mar 1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. LeVine SM, 2016. Albumin and multiple sclerosis. BMC Neurol. 16, 47 Apr 12. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Mahad DJ, Ziabreva I, Campbell G, et al. , 2009. Mitochondrial changes within axons in multiple sclerosis. Brain 132 (5), 1161–1174. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Maresca A, la Morgia C, Caporali L, Valentino ML, Carelli V, 2013. The optic nerve: a “mito-window” on mitochondrial neurodegeneration. Mol. Cell. Neurosci 55, 62–76 Jul. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Miyaue N, Yabe H, Nagai M, 2020. Serum growth differentiation factor 15, but not lactate, is elevated in patients with Parkinson’s disease. J. Neurol. Sci 409, 116616 Feb 15. [DOI] [PubMed] [Google Scholar]
  27. Montero R, Yubero D, Villarroya J, et al. , 2016. GDF-15 is elevated in children with mitochondrial diseases and is induced by mitochondrial dysfunction. PLoS One 11 (2), e0148709. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Nohara S, Ishii A, Yamamoto F, et al. , 2019. GDF-15, a mitochondrial disease biomarker, is associated with the severity of multiple sclerosis. J. Neurol. Sci 405, 116429 Oct 15. [DOI] [PubMed] [Google Scholar]
  29. Philips T, Rothstein JD, 2017. Oligodendroglia: metabolic supporters of neurons. J. Clin. Invest 127 (9), 3271–3280 Sep 1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Polman CH, Reingold SC, Edan G, et al. , 2005. Diagnostic criteria for multiple sclerosis: 2005 revisions to the “McDonald Criteria” Ann. Neurol 58 (6), 840–846 Dec. [DOI] [PubMed] [Google Scholar]
  31. Poulsen NS, Madsen KL, Hornsyld TM, et al. , 2020. Growth and differentiation factor 15 as a biomarker for mitochondrial myopathy. Mitochondrion 50, 35–41 Jan. [DOI] [PubMed] [Google Scholar]
  32. Schwid SR, Goodman AD, Mattson DH, et al. , 1997. The measurement of ambulatory impairment in multiple sclerosis. Neurology 49 (5), 1419–1424 Nov. [DOI] [PubMed] [Google Scholar]
  33. Sipe JC, Knobler RL, Braheny SL, Rice GP, Panitch HS, Oldstone MB, 1984. A neurologic rating scale (NRS) for use in multiple sclerosis. Neurology 34 (10), 1368–1372 Oct. [DOI] [PubMed] [Google Scholar]
  34. Smith A, 1982. Symbol digit modalities test: manual. Loas Angeles. CA: Western Psychol. Serv. [Google Scholar]
  35. Strelau J, Schober A, Sullivan A, Schilling L, Unsicker K, 2003. Growth/differentiation factor-15 (GDF-15), a novel member of the TGF-beta superfamily, promotes survival of lesioned mesencephalic dopaminergic neurons in vitro and in vivo and is induced in neurons following cortical lesioning. J. Neural Transm Suppl. 65, 197–203. [DOI] [PubMed] [Google Scholar]
  36. Strelau J, Strzelczyk A, Rusu P, et al. , 2009. Progressive postnatal motoneuron loss in mice lacking GDF-15. J. Neurosci 29 (43), 13640–13648 Oct 28. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Suno M, Nagaoka A, 1989. Inhibition of brain mitochondrial swelling by idebenone. Arch. Gerontol. Geriatr 8 (3), 299–305 May. [DOI] [PubMed] [Google Scholar]
  38. Tourbah A, Lebrun-Frenay C, Edan G, et al. , 2016. MD1003 (high-dose biotin) for the treatment of progressive multiple sclerosis: a randomised, double-blind, placebo-controlled study. Mult. Scler 22 (13), 1719–1731 Nov. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Tremlett H, Zhao Y, Joseph J, Devonshire V, Neurologists UC, 2008. Relapses in multiple sclerosis are age- and time-dependent. J. Neurol. Neurosurg. Psychiatry 79 (12), 1368–1374 Dec. [DOI] [PubMed] [Google Scholar]
  40. van Horssen J, Schreibelt G, Bo L, et al. , 2006. NAD(P)H:quinone oxidoreductase 1 expression in multiple sclerosis lesions. Free Radic. Biol. Med 41 (2), 311–317 Jul 15. [DOI] [PubMed] [Google Scholar]
  41. Weideman AM, Tapia-Maltos MA, Johnson K, Greenwood M, Bielekova B, 2017. Meta-analysis of the age-dependent efficacy of multiple sclerosis treatments. Front. Neurol 8, 577. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Wolinsky JS, Narayana PA, Noseworthy JH, et al. , 2000. Linomide in relapsing and secondary progressive MS: part II: MRI results. MRI Analysis Center of the University of Texas-Houston, health science Center, and the North American Linomide investigators. Neurology 54 (9), 1734–1741 May 9. [DOI] [PubMed] [Google Scholar]
  43. Yu-Wai-Man P, Votruba M, Burte F, La Morgia C, Barboni P, Carelli V, 2016. A neurodegenerative perspective on mitochondrial optic neuropathies. Acta Neuropathol. 132 (6), 789–806 Dec. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Zambonin JL, Zhao C, Ohno N, et al. , 2011. Increased mitochondrial content in remyelinated axons: implications for multiple sclerosis. Brain 134 (Pt 7), 1901–1913 Jul. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

MM6
MM5
MM4
MM3
MM2
MM1

RESOURCES