Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2020 Nov 1.
Published in final edited form as: Ann Neurol. 2019 Oct 2;86(5):671–682. doi: 10.1002/ana.25592

Telomere Length Is Associated with Disability Progression in Multiple Sclerosis

Kristen M Krysko 1, Roland G Henry 1, Bruce A C Cree 1, Jue Lin 2; University of California, San Francisco MS-EPIC Team1, Stacy Caillier 1, Adam Santaniello 1, Chao Zhao 1, Refujia Gomez 1, Carolyn Bevan 3, Dana L Smith 2, William Stern 1, Gina Kirkish 1, Stephen L Hauser 1, Jorge R Oksenberg 1, Jennifer S Graves 1,4
PMCID: PMC7135931  NIHMSID: NIHMS1566036  PMID: 31486104

Abstract

Objective

To assess whether biological aging as measured by leukocyte telomere length (LTL) is associated with clinical disability and brain volume loss in multiple sclerosis (MS).

Methods

Adults with MS/clinically isolated syndrome in the University of California, San Francisco EPIC cohort study were included. LTL was measured on DNA samples by quantitative polymerase chain reaction and expressed as telomere to somatic DNA (T/S) ratio. Expanded Disability Status Scale (EDSS) and 3-dimensional T1-weighted brain magnetic resonance imaging were performed at baseline and follow-up. Associations of baseline LTL with cross-sectional and longitudinal outcomes were assessed using simple and mixed effects linear regression models. A subset (n = 46) had LTL measured over time, and we assessed the association of LTL change with EDSS change with mixed effects models.

Results

Included were 356 women and 160 men (mean age = 43 years, median disease duration = 6 years, median EDSS = 1.5 [range = 0–7], mean T/S ratio = 0.97 [standard deviation = 0.18]). In baseline analyses adjusted for age, disease duration, and sex, for every 0.2 lower LTL, EDSS was 0.27 higher (95% confidence interval [CI] = 0.13–0.42, p < 0.001) and brain volume was 7.4mm3 lower (95% CI = 0.10–14.7, p = 0.047). In longitudinal adjusted analyses, those with lower baseline LTL had higher EDSS and lower brain volumes over time. In adjusted analysis of the subset, LTL change was associated with EDSS change over 10 years; for every 0.2 LTL decrease, EDSS was 0.34 higher (95% CI = 0.08–0.61, p = 0.012).

Interpretation

Shorter telomere length was associated with disability independent of chronological age, suggesting that biological aging may contribute to neurological injury in MS. Targeting aging-related mechanisms is a potential therapeutic strategy against MS progression.


The pathological processes underlying progression in multiple sclerosis (MS) are not yet understood. However, chronological age has been consistently associated with rate of non–relapse-related disability accumulation in MS.13 In pediatric MS, there are very few cases with a primary progressive (PP) phenotype (0.9% vs 8.5% in adults) and children with relapsing onset MS take longer to reach the secondary progressive (SP) phase of disease (32 vs 18 years).4 In adults, chronological age has been associated with time to disability milestones independent of disease duration, with older patients experiencing shorter time intervals to ambulatory dysfunction.2,3,5,6 The average age at diagnosis of progressive MS in adults is 10 years older than that of relapsing–remitting (RR) MS,7 and age at onset of progression is highly similar between PP and SP disease (about 45 years).5,8 Despite these observations suggesting that accumulation of irreversible disability appears to be age-dependent, the role of biological aging in MS progression remains to be determined.

Investigation of telomere length is a novel strategy to assess the contribution of biological aging to MS disability progression. Shortened telomeres are upstream of important senescence-related changes in the immune system that could impact MS phenotypes. Improving the understanding of this relationship is important because targeting aging-related mechanisms may be a potential therapeutic strategy.9

Telomeres contain proteins and nucleotide repeats (TTAGGG) at the end of chromosomes that shorten with each cell division, making telomere shortening a marker of cellular aging.10,11 Telomere shortening is accelerated by oxidative stress and DNA damage, whereas telomerase promotes telomere length replenishment.10 Lifestyle factors such as smoking, psychological stress, diet, and exercise are associated with telomere length.10,12 Shortened telomeres are observed in chronic aging-related illnesses including cardiovascular disease13 and dementia,14 and in autoimmune diseases such as systemic lupus erythematosus and rheumatoid arthritis.15 Although telomere biology is not yet understood in MS, one prior study found that individuals with PP MS had shorter telomeres than controls. This was not seen for RR or SP MS.16

We aimed to assess whether biological age as measured by telomere length is associated with the progression of clinical disability and brain volume loss in individuals with MS, independent of chronological age and disease duration, in both cross-sectional and longitudinal analyses. We also aimed to determine the proportion of the effect of chronological age on disability explained by telomere length.

Patients and Methods

Study Design and Participants

This is an observational cohort study of individuals aged 18 to 65 years meeting diagnostic criteria17 for MS or clinically isolated syndrome (CIS) at the University of California, San Francisco (UCSF). The EPIC study design and cohort characteristics were previously reported.18,19 Individuals seen at the UCSF MS Center between July 2004 and September 2005 were offered enrollment. There was preferential recruitment of ambulatory individuals and recent onset disease, although all MS subtypes were included. Individuals were followed with clinical assessment and magnetic resonance imaging (MRI) yearly for 5 years, with re-evaluation up to 10 years from baseline. Retention at years 8 to 10 in the EPIC study was 91%.18

Protocol Approvals and Patient Consent

The UCSF Committee on Human Research approved the study, and participants provided informed consent.

Clinical Measures

Age, disease duration, smoking status (never, former, current), pack-years of smoking, and body mass index (BMI) were recorded. Disease-modifying therapy (DMT) exposure prior to baseline and between baseline and year 10 was classified as untreated, platform therapy (interferon-betas, glatiramer acetate, teriflunomide, mycophenolate mofetil, azathioprine, methotrexate, intravenous immunoglobulin, steroids), high-potency therapy (fingolimod, dimethyl fumarate, natalizumab, rituximab, ocrelizumab, alemtuzumab, mitoxantrone, cyclophosphamide), or platform plus high-potency therapy, as categorized in prior studies of this cohort.18 Trained staff performed the Expanded Disability Status Scale (EDSS) score and the Multiple Sclerosis Functional Composite (MSFC). The primary outcome of EDSS measures clinical disability with a score from 0 to 10, with normal neurologic examination scoring 0 and higher scores indicating more severe disability.20 The MSFC includes tests of neurologic disability including cognition (Paced Auditory Serial Addition Task), upper extremity function (9-Peg Hole Test), and ambulation (25-foot walk), combined and standardized with a z score.21 Relapses during follow-up were captured by patient self-report at annual study visits.

Leukocyte Telomere Length Measure

Real-time quantitative polymerase chain reaction assays were used to measure leukocyte telomere length (LTL), which was expressed as the ratio of the abundance of telomere versus a single copy gene (T/S) with the approach described by Cawthon in 2002,22 adapted by the Blackburn laboratory as described by Lin et al 2010.23 The single copy gene used was human beta-globin. The T and S concentrations for each sample were measured in triplicate wells, and the averages of T and S concentrations were used to calculate the T/S ratio. This was done twice to obtain 2 T/S values. When the duplicate T/S values varied by >7%, the sample was run a 3rd time and the 2 closest values were averaged to obtain the final T/S ratio. All samples from the same participants were run in the same batch. The average interassay coefficient of variation for this study was 3.0% (±2.2%). These assays were performed on stored baseline DNA samples for the entire cohort and at multiple timepoints for a subset of 46 individuals. These T/S ratios represent the average telomere length across leukocytes. Although T/S ratios were not compared to base pairs measured by Southern blot analysis in samples from this study, these can be converted to base pairs using a conversion formula derived in the same assay laboratory.24 We expect LTL to decrease by about 24 base pairs per year based on prior literature.25

MRI Measures

High-resolution 3-dimensional T1-weighted, T2-weighted, and proton density sequences were obtained on a 3T MRI scanner. SIENAX was used to determine T1-weighted volumes including normalized brain volume, white matter volume, gray matter volume, and cortical gray matter volume, which were secondary outcomes.

Statistical Analyses

For descriptive analyses, mean and standard deviation (SD) or median and range were reported as appropriate. LTL was compared between sexes with Student t test and by smoking status with analysis of variance. The relationships of chronological age, disease duration, BMI, and smoking pack-years (among smokers) with LTL were assessed with Pearson correlation (r). The primary predictor of interest, LTL (T/S ratio), was scaled by −0.2 for all analyses for ease of model interpretation with expression of coefficients per 0.2U decrease in LTL. Scaling of LTL by 0.2 was chosen because this represents the SD of the T/S ratio in the cohort, and thus reported coefficients for LTL represent the change in outcomes associated with a single SD decrease in LTL.

Cross-Sectional Analyses

In primary analyses, unadjusted and adjusted linear regression models were used to assess the association between LTL and EDSS at baseline. Secondary analyses included similar models for secondary outcomes, including MSFC z score, total brain volume, white matter volume, gray matter volume, and cortical gray matter volume.

Covariates for adjusted models included confounding factors based on prior knowledge, including chronological age and sex as shown in the proposed causal diagram (Fig 1A). Although disease duration may not be a confounder, it was also adjusted for because it is a strong determinant of the outcome, and could potentially be a confounder if longer disease duration leads to shorter telomeres due to oxidative stress associated with inflammation. Smoking status and HLA-DRB1*15:01 were evaluated as potential confounders, and it was decided a priori that these variables would be included if they led to a 10% or greater change in the coefficient for LTL. In an additional analysis, the cohort was restricted to smokers and included adjustment for smoking pack-years, as this has been previously associated with LTL,26 with evaluation for confounding by this cumulative measure of smoking exposure. BMI was also evaluated as a potential confounder by evaluating for an association between BMI and LTL, and evaluating for a change in LTL coefficient when BMI was added to the models, because BMI has previously been associated with telomere length.26 It was not anticipated that DMT would be a confounder of the relationship between LTL and disability/MRI outcomes, because there is no prior evidence that treatment would be associated with the predictor (LTL). To test this assumption, it was evaluated whether LTL values differed by level of DMT exposure by testing for an association between DMT exposure category (untreated, platform, high potency, platform plus high potency) and LTL with regression models. DMT would only be adjusted for in models of interest if associated with LTL in this cohort.

FIGURE 1:

FIGURE 1:

Proposed causal diagram of the association between telomere length and disability (A) and between age and disability mediated by telomere length (B). (A) Cumulative number of cell divisions, telomerase activity, genetic factors, and environmental factors affect telomere length, which is a marker of biological age, and we assess its association with disability. Chronological age and sex are considered confounders, whereas disease duration is hypothesized to be a mediator. (B) The same relationships are shown, but the association between chronological age and disability is the primary focus, with mediation by telomere length. Black arrows represent the relationships of interest. MS = multiple sclerosis.

Disease duration was natural log transformed to satisfy the assumption of linearity, and 0 values were coded as 0.1 before transformation to avoid missing data. The potential interaction between sex and LTL was evaluated based on a prior observation that suggested decreased maintenance of LTL in men compared to women.27 Examination of pairwise correlation coefficients and variance inflation factors was used to assess for colinearity between age, disease duration, and LTL.

Mediation Analyses

Mediation analyses of cross-sectional data were used to assess the proportion of the effect of chronological age on disability (EDSS) mediated by biological age as measured by telomere length. The proposed model is in the causal diagram (see Fig 1B). First, the association between age and telomere length with adjustment for sex was assessed. Then, the overall effect of chronological age on EDSS adjusted for sex was calculated with a regression model. Next, the direct effect of chronological age on EDSS not mediated by LTL was determined by adjusting for LTL (again adjusted for sex as well). Finally, using the above model estimates, the indirect effect of age on EDSS mediated by LTL was evaluated to obtain the percentage explained. Bootstrap 95% confidence intervals (CIs) were calculated for the point estimate of the mediation analysis. Disease duration was not adjusted for in this analysis to avoid bias, given that it is considered to be a mediator in the hypothesized causal pathway (see Fig 1).

Longitudinal Analyses of the Entire Cohort

To assess the association of baseline telomere length with change in outcomes (EDSS, MSFC z score, brain volume metrics) over time in the whole cohort, mixed effects linear regression models with random slopes and intercepts with an interaction term between baseline LTL and visit were employed, with visit modeled as categorical, as the assumption of linearity was violated if it was modeled as continuous. The interaction terms and overall interaction p values were of interest, as they allow assessment of whether baseline telomere length predicts different slopes or rates of change in outcomes over time. Model results were graphed by displaying the predicted outcome trajectory for mean LTL and LTL 2 SDs above and below the mean to allow evaluation of the interaction effect. These models were also adjusted for baseline chronological age, sex, and disease duration.

Analyses of Matched Pairs

A nested case–control study was performed to evaluate the association of baseline LTL with development of SP MS, and the association of change in LTL with disability and MRI metrics. From the parent cohort, 23 participants who developed secondary progression and had DNA available at multiple timepoints were selected. These participants were matched 1:1 on baseline age, sex, disease duration, and EDSS score to participants who remained classified as having relapsing MS with DNA available at the same timepoints. This subset of 46 individuals had LTL measured at multiple timepoints up to 10 years from baseline, in addition to longitudinal clinical and MRI metrics.

Adjusted conditional logistic regression analysis was performed in the 23 matched pairs to assess the association of baseline LTL with outcome status of SP versus RR MS, accounting for the paired nature of the data and adjusting for baseline age, disease duration, and sex. Unadjusted and adjusted mixed effects linear regression models with random slopes and intercepts were used to assess the association of telomere length as a time-varying predictor with change in outcomes (EDSS, MSFC z score, brain volume metrics), accounting for the paired nature of the data. These repeated measures models allow use of all data points over time, rather than only the difference in a variable between baseline and the final timepoint. The time-varying predictor was LTL at each visit, and the repeated measures outcome was the disability or MRI metric for the same visit. This preplanned analysis resulted in a single coefficient for time-varying LTL, which can be interpreted as the change in outcome associated with change in LTL.28 Baseline age, sex, and disease duration were adjusted for as above.

To determine whether the primary model above, with a single time-varying LTL predictor, accurately depicted the effect of change in LTL, sensitivity analyses were pursued. Additional models were performed that separated the effect of baseline LTL (between-person differences) from change in LTL (within-person differences) by including 2 predictors for LTL in each model (baseline LTL and time-varying LTL change from baseline at each visit), with methods previously described.28 Within-person changes were isolated from between-person differences because the latter are more subject to residual confounding. To determine whether the simpler, original model was not driven by between-person differences, coefficients for change in LTL from the models including both baseline LTL and change in LTL as separate predictors were compared to coefficients for time-varying LTL as a single predictor in the original models. These were compared for substantive differences, as well as statistically with z tests. Based on the analysis plan, if there were no substantive differences in the coefficients, this means the change effect is appropriately depicted by modeling with a single time-varying LTL predictor, and the original model would be reported to reduce complexity of model interpretation.

Model Assessment and Sensitivity Analyses

Stata 15 (StataCorp, College Station, TX) was used for all analyses, and all tests were 2-sided with alpha of 0.05. Model diagnostics were performed to assess assumptions of regression, and no violations were present in the final models. EDSS was modeled as a numerical outcome, and given the ordinal nature of the scale, statistical assumptions were carefully evaluated including assumptions of linearity for continuous predictors, normal distribution for residuals, and constant variance of fitted values to ensure statistical models were appropriate. As an additional assessment, sensitivity analyses with bootstrap CIs were performed for all models with EDSS as the outcome.

Sensitivity analyses excluding patients who remained with CIS over the study were also performed to decrease baseline heterogeneity. The association between baseline LTL and annualized relapse rate (ARR) was also evaluated among those with RR MS at baseline with negative binomial regression, with adjustment for age, sex, and disease duration with an offset by follow-up duration. ARR was not adjusted for in models evaluating the association between LTL and disability/MRI outcomes, as relapses and recovery from relapses may be mediators along the causal pathway of interest.

Results

Baseline Participant Characteristics

Of 517 in the original cohort, 516 had DNA available at the baseline visit and were included in this study. There were 12 individuals with only a baseline visit, whereas the remainder contributed to longitudinal data for this study, with 434 contributing at least 5 visits.

Mean age was 42.6 years, and 68.6% were female. All MS subtypes were included. There was a wide range of disease duration (median = 6 years, range = 0–45 years) and disability (EDSS median = 1.5, range = 0–7). LTL was normally distributed in the cohort, and mean LTL (T/S ratio) was 0.97 (SD = 0.18), which corresponds to 5,615 base pairs when converted using a formula derived in the same assay laboratory (Table 1).24

TABLE 1.

Baseline Characteristics of Participants (N = 516)

Characteristic Cohort
Age, mean yr (SD) 42.6 (9.8)
Female sex, n (%) 354 (68.6)
Disease duration, median yr (range) 6 (0–45)
Smoking status, n (%)
 Current 64 (12.4)
 Former 157 (30.5)
 Never 295 (57.2)
MS subtype, n (%)
 RR MS 367 (71.1)
 CIS 80 (15.5)
 SP MS 47 (9.1)
 PP MS 17 (3.3)
 PR MS 4 (0.8)
 Unclear 1 (0.2)
DMTa before baseline, n (%)
 Untreated 151 (29.3)
 Platform 331 (64.2)
 High potency 6 (1.2)
 Platform plus high potency 28 (5.4)
Leukocyte telomere length, mean T/S ratio (SD) 0.97 (0.18)
EDSS, median (range) 1.5 (0.0–7.0)
Total brain volume, mean mm3 (SD) 1,506.8 (90.8)
a

Platform therapy: interferon-betas, glatiramer acetate, teriflunomide, mycophenolate mofetil, azathioprine, methotrexate, intravenous immunoglobulin, steroids. High-potency therapy: fingolimod, dimethyl fumarate, natalizumab, rituximab, ocrelizumab, alemtuzumab, mitoxantrone, cyclophosphamide.

CIS = clinically isolated syndrome; DMT = disease-modifying therapy; EDSS = Expanded Disability Status Scale; MS = multiple sclerosis; PP = primary progressive; PR = progressive relapsing; RR = relapsing–remitting; SD = standard deviation; SP = secondary progressive; T/S = telomere to somatic DNA.

Cross-Sectional Association between Telomere Length and Clinical and MRI Outcomes

Higher age (r = −0.29, p < 0.001) and disease duration (r = −0.16, p < 0.001) were associated with shorter telomere length, as expected. The T/S ratio was 0.01 lower for every year of age. Using a conversion formula between T/S ratio and base pair length derived in the same assay laboratory,24 there was an estimated average lower LTL of 24 base pairs per year. However, there was no association between LTL and sex (mean T/S ratio: female, 0.97; male, 0.96; p = 0.36), BMI (r = −0.02, p = 0.62), or smoking status (mean T/S ratio: smokers, 1.0; former smokers, 0.95; never smokers, 0.97; p = 0.12). Among current/former smokers (n = 221), pack-year exposure was not associated with LTL (r = −0.09, p = 0.21). Despite not being associated with LTL in our sample, sex was retained in adjusted models, given it is considered a confounder based on prior knowledge.27,29 Level of DMT exposure prior to baseline was not substantively or statistically associated with baseline LTL (mean T/S ratio: untreated, 0.99; platform, 0.96; high potency, 0.96; platform plus high potency, 0.92) without (p = 0.28) or with (p = 0.24) adjustment for age and sex. In the subset of 46 with longitudinal LTL measurements, LTL was also not associated with DMT exposure. Due to there being no statistical or substantive association between DMT and LTL in this cohort, DMT was not adjusted for in models of interest.

In primary unadjusted analysis, for every 0.2U lower LTL, EDSS was 0.41U higher (95% CI = 0.27–0.56, p < 0.001; Table 2). Remarkably, in analysis adjusted for chronological age, sex, and disease duration, for every 0.2U lower LTL, EDSS was 0.27U higher (95% CI = 0.13–0.42, p < 0.001). Those with shorter telomere length had lower MSFC z score in the univariable analysis (β = −0.09, 95% CI = −0.16 to −0.03, p = 0.006), but this association was not statistically significant after adjustment (β = −0.05, 95% CI = −0.12 to 0.02, p = 0.13).

TABLE 2.

Unadjusted and Adjusted Linear Regression Analyses of Cross-Sectional Association of Leukocyte Telomere Length with Baseline Clinical and Magnetic Resonance Imaging Outcomes, Expressed per 0.2U Decrease in Telomere Length (N = 516)

Unadjusted Adjustedb
Outcomea βc 95% CI p βc 95% CI p
Disability score (EDSS) 0.41 0.27 to 0.56 <0.001 0.27 0.13 to 0.42 <0.001
MSFC z score −0.09 −0.16 to −0.03 0.006 −0.05 −0.12 to 0.02 0.13
Total brain volume, mm3 −22.3 −30.6 to −14.0 <0.001 −7.4 −14.7 to −0.10 0.047
Total WM volume, mm3 −7.4 −11.5 to −3.3 <0.001 −4.0 −8.0 to −0.04 0.048
Total GM volume, mm3 −14.7 −20.1 to −9.2 <0.001 −3.4 −7.8 to 1.0 0.13
Cortical GM volume, mm3 −12.9 −17.5 to −8.3 <0.001 −3.1 −6.8 to 0.61 0.10
a

n = 516 for EDSS, n = 511 for MSFC, and n = 515 for brain volume metrics. Each row represents a separate model.

b

Adjusted for chronological age, sex, and disease duration.

c

Per 0.2U decrease in mean telomere to somatic DNA ratio (leukocyte telomere length).

CI = confidence interval; EDSS = Expanded Disability Status Scale; GM = gray matter; MSFC = multiple sclerosis functional composite; WM = white matter; β = linear regression coefficient.

In unadjusted analyses of MRI metrics, 0.2U lower LTL was associated with 22.3mm3 lower total brain volume (95% CI = 14.–30.6, p < 0.001), 7.4mm3 lower white matter volume (95% CI = 3.3–11.5, p < 0.001), 14.7mm3 lower total gray matter volume (95% CI = 9.2–20.1, p < 0.001), and 12.9mm3 lower cortical gray matter volume (95% CI = 8.3–17.5, p < 0.001; see Table 2). After adjustment for chronological age, sex, and disease duration, although all point estimates suggested lower brain volumes associated with shorter LTL, only associations with total brain and white matter volume reached statistical significance (see Table 2). For every 0.2U lower LTL, there was 7.4mm3 lower total brain volume (95% CI = 0.10–14.7, p = 0.047) and 4.0mm3 lower white matter volume (95% CI = 0.04–8.0, p = 0.048).

Mediation Analyses

An association between age and LTL was demonstrated. As expected, for every 10-year increase in age, LTL was 0.05U lower (95% CI = 0.04–0.07, p < 0.001). Age was also associated with EDSS. For every 10-year increase in age, EDSS was 0.51U higher (95% CI = 0.37–0.65, p < 0.001). The direct effect of age on EDSS not mediated by LTL was then estimated; for every 10-year increase in age, EDSS was 0.43U higher (95% CI = 0.29–0.58, p < 0.001). The indirect effect of age on EDSS mediated by LTL, or percentage explained, was calculated to be 15.1% (95% CI = 6.9–26.7%).

Longitudinal Analyses of the Entire

Cohort Regardless of baseline LTL, individuals tended to have worsening of EDSS and declining brain volume over 10 years. In unadjusted and adjusted analyses of the entire cohort, lower baseline LTL was associated with higher EDSS over time on average. However, baseline LTL was not strongly associated with the rate of change in EDSS over time, with similar EDSS trajectory regardless of baseline LTL (interaction p = 0.06; lack of interaction effect shown in Fig 2A, B; interaction coefficients are given in the Supplementary Table). Findings were similar for secondary outcomes (Table 3). Those with lower baseline LTL tended to have lower brain volumes on average over the 10 years, although the rates of decline did not vary by baseline LTL (interaction p = 0.33; lack of interaction effect is shown in Fig 2C, D with similar brain volume trajectory regardless of baseline LTL; interaction coefficients are given in the Supplementary Table). Although the interaction between baseline LTL and time since cohort entry was statistically significant in the model for cortical gray matter volume (p = 0.02), the trajectory or rate of change of cortical brain volume appeared similar regardless of baseline LTL as shown in Figure 2E and F, suggesting no evidence of a relevant interaction (interaction coefficients are given in the Supplementary Table).

FIGURE 2:

FIGURE 2:

Predicted trajectory of Expanded Disability Status Scale (EDSS; A, B), brain volume (C, D), and cortical gray matter volume (E, F) over 10 years by baseline telomere length with longitudinal analyses in the entire cohort with unadjusted and adjusted mixed models. Predictions are based on longitudinal mixed effects models of the entire cohort for outcomes of EDSS (A, unadjusted; B, adjusted), total brain volume (C, unadjusted; D, adjusted), and cortical gray matter volume (E, unadjusted; F, adjusted). Leukocyte telomere length (LTL) was treated as continuous in the models, but key LTL values including the approximate mean value (1), and 2 standard deviations below (0.6) and above (1.4) the mean are shown graphically to display model results. Shaded areas represent 95% confidence limits. EDSS increased and brain volume decreased over time regardless of baseline LTL, with similar trajectories despite baseline LTL, and no relevant interaction between LTL and time since enrollment. T/S ratio = telomere to somatic DNA ratio. Adjusted models were adjusted for baseline age, sex, and disease duration. Year 7 was excluded from the models due to very few participants completing this visit.

TABLE 3.

Unadjusted and Adjusted Linear Regression Analyses of Association of Baseline LTL with Longitudinal Clinical and Magnetic Resonance Imaging Outcomes, Expressed per 0.2U Decrease in Telomere Length (N = 516)

Unadjusted Adjustedb
Outcomea βc 95% CI p Interactiond Year by LTL p βc 95% CI p Interactiond Year by LTL p
Disability score (EDSS) 0.41 0.27 to 0.56 <0.001 0.06 0.25 0.11 to 0.39 0.001 0.06
MSFC z score −0.09 −0.16 to −0.02 0.008 0.36 −0.05 −0.12 to 0.02 0.15 0.33
Total brain volume, mm3 −22.5 −31.1 to −14.0 <0.001 0.50 −10.5 −18.0 to −3.0 0.006 0.52
Total WM volume, mm3 −7.5 −11.8 to −3.2 0.001 0.64 −4.3 −8.4 to −0.14 0.043 0.64
Total GM volume, mm3 −14.8 −20.3 to −9.2 <0.001 0.18 −6.4 −11.0 to −1.9 0.006 0.21
Cortical GM volume, mm3 −12.9 −17.7 to −8.2 <0.001 0.01 −5.9 −9.8 to −2.1 0.003 0.02
a

n = 516 for EDSS and brain volume metrics, n = 514 for MSFC. Each row represents a separate model.

b

Adjusted for baseline chronological age, sex, and disease duration.

c

Per 0.2U decrease in mean telomere to somatic DNA ratio (leukocyte telomere length).

d

Interaction effect sizes are shown graphically for EDSS, total brain volume, and cortical gray matter volume in Figure 2, as they cannot be represented by a single coefficient. Graphs demonstrate no meaningful difference in trajectory of outcomes by baseline telomere length, suggesting no relevant interaction between LTL and year since enrollment. Full interaction effect estimates are shown in the Supplementary Table.

CI = confidence interval; EDSS = Expanded Disability Status Scale; GM = gray matter; LTL = leukocyte telomere length; MSFC = multiple sclerosis functional composite; WM = white matter; β = linear regression coefficient at visit 1.

Analyses of Matched Pairs

In adjusted analysis in the subset of 46 patients with LTL measured over time and modeled as a single time-varying LTL predictor, change in LTL was associated with change in EDSS; for every 0.2U lower LTL, EDSS was 0.34U higher (95% CI = 0.08–0.61, p = 0.012) over 10 years of follow-up. There was no statistically significant association between change in LTL and change in MSFC z score or brain volume metrics in analyses of this subset (Table 4). On average, LTL decreased by an estimated 24 base pairs per year in this subset.

TABLE 4.

Unadjusted and Adjusted Linear Regression Analyses of Association of Change in Leukocyte Telomere Length with Longitudinal Clinical and Magnetic Resonance Imaging Outcomes in the Subset with Telomere Length Measured Longitudinally, Expressed per 0.2U Decrease in Telomere Length (n = 46)

Unadjusted Adjustedb
Outcomea βc 95% CI p βc 95% CI p
Disability score (EDSS) 0.31 0.05 to 0.57 0.021 0.34 0.08 to 0.61 0.012
MSFC z score 0.07 −0.03 to 0.17 0.15 0.08 −0.02 to 0.18 0.10
Total brain volume, mm3 −9.1 −20.6 to 2.4 0.12 −6.2 −17.6 to 5.2 0.29
Total WM volume, mm3 −3.4 −10.7 to 3.8 0.35 −2.4 −9.7 to 4.9 0.52
Total GM volume, mm3 −4.4 −11.3 to 2.6 0.22 −2.7 −9.3 to 4.0 0.44
Cortical GM volume, mm3 −0.55 −6.2 to 5.1 0.85 0.13 −5.4 to 5.7 0.96
a

n = 46. Each row represents a separate model.

b

Adjusted for baseline chronological age, sex, and disease duration.

c

Per 0.2U decrease in mean telomere to somatic DNA ratio (leukocyte telomere length).

CI = confidence interval; EDSS = Expanded Disability Status Scale; GM = gray matter; MSFC = multiple sclerosis functional composite; WM = white matter; β = linear regression coefficient.

We performed sensitivity analyses to ensure models with LTL as a single time-varying predictor appropriately depicted the effect of change in LTL. Within-person changes were isolated from between-person differences by using 2 predictors for LTL (baseline LTL and time-varying change in LTL). Coefficients for time-varying change in LTL in these sensitivity models (including baseline LTL) were compared to coefficients for time-varying LTL as a single predictor in the original models. There were no substantive or statistically significant differences observed between the coefficients for time-varying LTL as a single predictor compared to coefficients for time-varying change in LTL from baseline (in the models also including baseline LTL) for any outcome model. Thus, as planned in advance, the original models were reported for ease of interpretation and are interpreted to demonstrate the association of change in LTL with change in outcomes.

In conditional logistic regression of the subset of 46 patients with change in LTL measured over time adjusted for baseline age, disease duration, and sex, for every 0.2U decrease in baseline LTL, individuals had 1.4 times the odds of converting to SP MS by treating physician assessment (95% CI = 0.61–3.4, p = 0.40), although this observation was not statistically significant.

Model Assumptions and Sensitivity Analyses

Model diagnostics showed no violations of assumptions of regression in the final models. There was careful evaluation of models with EDSS as an outcome to ensure treatment as a numeric variable was appropriate, with no evidence of nonlinearity and with normality of residuals and constant variance of fitted values. Sensitivity analyses with bootstrap 95% CIs for all models with EDSS as an outcome demonstrated results that did not differ substantively from the original models; the original models were reported. These diagnostics and sensitivity analyses supported that statistical models used were appropriate. Smoking status and HLA-DRB1*15:01 were evaluated as potential confounders in the models, although these did not meet the requirement for a 10% or more change in coefficient for LTL decided a priori, so they were not retained in final models. Among smokers, adding adjustment for pack-years of smoking exposure did not substantively change the LTL coefficients, with no evidence of confounding by pack-years, and thus this was not retained in final models. Additionally, when BMI was included in multivariable analyses, there was no substantive change in LTL coefficients, so BMI was not retained. There was no statistically significant interaction between sex and LTL, so this was also not retained in final models. There was no evidence for problematic colinearity between age, disease duration, and LTL, as there were no pairwise correlations greater than 0.8 between these variables and variance inflation factors were less than 1.5.

Sensitivity analyses excluding those who remained with CIS (n = 27) over follow-up showed no substantive differences in results in cross-sectional or longitudinal analyses. In unadjusted negative binomial regression among those with RR MS (n = 359), there was no statistically significant association between baseline LTL and ARR over follow-up (incidence rate ratio = 1.10 [95% CI = 0.95–1.26, p = 0.19] for every 0.2U decrease in LTL). However, after adjustment for age, disease duration, and sex, there was a statistically significant association between baseline LTL and ARR. For every 0.2U decrease in LTL, relapse rate was 1.27 times higher (95% CI = 1.10–1.46, p = 0.001).

Discussion

Supporting the hypothesis that biological aging contributes to MS progression, shorter telomere length was associated with greater disability and brain atrophy. Participants with shorter telomere length had higher EDSS and lower brain volumes at baseline, with differences maintained over 10 years on average. In the subset of study participants with telomere length measured at multiple timepoints, decline in telomere length was associated with worsening EDSS. Mediation analyses28 were used to evaluate the percentage of the chronological age effect on disability mediated by telomere length, and 15% of the effect was accounted for by LTL. Together, these results link biological aging with MS phenotype and motivate further study of telomere length and the DNA damage response in MS progression.

This study suggests that individual variability in biological aging may contribute to the heterogeneity in MS course. Individuals with the same birthdate have been shown to have large differences in estimates of true biological age.30 These differences can predict risk of chronic illness and mortality.30 Our results further support studying biological aging processes independent of simple chronological age.

Potential mechanisms for the contribution of biological aging to disability progression include increased oxidative stress with somatic aging,10 decreased remyelination capacity,3133 and altered immune function associated with immune senescence.34,35 Another possible explanation for the observed association is that short telomeres may be a marker for comorbidities such as cardiovascular disease and diabetes,10 which are also associated with disability progression in MS.36,37 Lifestyle factors such as smoking, psychological stress, poor diet, and lack of exercise are associated with shortened telomeres,12 and the effect of lifestyle on biological aging could be a potential mechanism for their association with MS progression.3840

The association of LTL with all outcome measures was attenuated after adjustment for chronological age. This confounding by chronological age was expected given that telomere length shortens with age10,25 and age is associated with disability progression.13,5,6 Age-adjusted models could be considered overly conservative, because the effect of LTL demonstrated in these models does not include the contribution of biological aging that occurs under the umbrella of effects associated with chronological age. However, the strength of our approach was to include this conservative model adjusting for birthdate and disease duration. That a chronological age–independent effect of telomere length on disability and brain volume was observed suggests that the DNA damage response and resulting downstream factors play specific roles in MS disability progression beyond other effects of chronological aging.

Baseline telomere length did not predict the trajectories for either worsening disability or brain volume loss in this cohort. However, because the dataset includes a large range of ages and disease durations, there may be more power to assess associations of LTL with MS outcomes in cross-sectional analyses than power to assess baseline LTL as a predictor of MS outcomes over 10 years of follow-up. It is also possible that the effect of biological age could be fixed by the baseline timepoint. Alternatively, the change in telomere length over time might alter disability trajectory rather than be dependent on the baseline telomere length. This finding is supported by the association between change in LTL and change in EDSS in the nested case–control portion of the study.

Limitations of this study include that we were unable to measure change in LTL over time for all individuals in this legacy cohort because DNA was not sampled at timepoints after baseline for most participants. We addressed this limitation by including the case–control subset analysis of subjects with multiple DNA timepoints available who experienced progression to SP MS during the study and matched them by age, sex, and disease duration with individuals who remained with RR MS. In this analysis, decline in telomere length over time was associated with worsening disability over 10 years. Although sampling in the case–control portion was related to the outcome, selection was unrelated to the exposure (LTL value), and thus we do not expect selection bias to have caused the observed association. The small sample size of this nested case–control portion of the study was underpowered to detect effects on secondary MRI metrics. Additional limitations include the possibility of reverse causation. It is possible that greater MS severity activates mechanisms that cause DNA damage or impairs the ability of telomerase to maintain telomere length, thereby leading to telomere shortening. Furthermore, we cannot exclude the possibility of unmeasured confounding of the associations found in this study. It is unclear why smoking status, cumulative smoking exposure, and BMI were not associated with LTL in this cohort, as these have previously been associated in healthy individuals.26 For face validity, we still performed secondary analyses to determine whether including these potential confounders in our multivariable analyses changed our point estimates, and they did not. Additionally, although DMT was not associated with LTL in our cohort and thus analyses were not adjusted for DMT, the potential influence of more potent medications on LTL may require additional study, particularly given that few were on high-potency DMT at baseline in this cohort. Additionally, EDSS was modeled as a numeric variable, even though it is an ordinal scale; however, this choice was supported by the model diagnostics, which showed no violations of assumptions. Although these models focus on change in EDSS score, which may have different interpretations across the range of the scale, the main goal was to assess for an association between LTL and disability, which was demonstrated. Lastly, there was no matched control group included in the study to determine whether MS patients have accelerated telomere erosion; however, the rate of decline in T/S ratio per year in this study of MS participants is consistent with that reported in healthy individuals.25

It was also found that shorter telomere length was associated with higher relapse rate in adjusted analyses, suggesting that when holding chronological age fixed, those with worse biological age may have a greater number of relapses, and relapse rate may be a potential mediator of the effect of biological age on disability. However, this analysis is limited by the lack of adjudication of relapses in the cohort, with the possibility that fluctuations in symptoms or progression could be categorized as relapses. Thus, this relationship requires additional study.

Strengths of this study include our novel investigation of telomere length, the ultimate biological clock, on disability progression in MS. We study this in a large cohort of well-characterized participants with MS, and use both cross-sectional and longitudinal analyses with robust statistical models to evaluate the association of telomere length with MS disability and brain volume.

Both immunological and central nervous system (CNS) aging may contribute to MS progression. In this study, LTL was used as a general marker of biological aging as previously done in several conditions including cardiovascular disease and dementia,10 even though LTL can vary across cell types.41 This general marker of biological aging was associated with progression, although aging processes specific to the CNS may also be implicated.42

The observation that telomere length, a somatic marker of biological aging, contributes to MS disability is consistent with our prior work on reproductive aging in women with MS. Levels of anti-Mullerian hormone, which correlate with ovarian aging and function, were associated with disability and brain volumes in cross section and over time.43 Taken together, these studies suggest that targeting aging-related mechanisms may be a potential therapeutic strategy in MS to delay disability progression.9 Additionally, this work highlights the importance of treating comorbidities associated with biological aging, such as cardiovascular disease and diabetes, that contribute to disability progression. Finally, improving biological aging through lifestyle factors such as exercise, stress reduction, diet, and smoking avoidance might reduce disability worsening.

In conclusion, this study demonstrates that shorter telomere length is associated with greater disability and brain atrophy independent of chronological age and MS disease duration. Future directions include evaluating differences in LTL across leukocytes including lymphocyte subsets in MS, exploring potential mechanisms for this association, and identifying potential therapeutic targets aimed at biological aging.

Supplementary Material

supplementary appendix
supplementary table

Acknowledgment

This study was funded by grants from the National Multiple Sclerosis Society (RG-1607-25103; principal investigator [PI], J.S.G.) and NIH (RO1NS26799; PI, S.L.H.). K.M.K. is funded by a Sylvia Lawry Physician Fellowship through the National Multiple Sclerosis Society (FP-1605-08753; PI, K.M.K.).

We thank the patients and research coordinators who participated in this study; the Elizabeth Blackburn laboratory for advice and support of the study; and Drs C. McCulloch and A. Lazar, who provided statistical advice.

Footnotes

Potential Conflicts of Interest

Nothing to report.

Members of the UCSF MS-EPIC Team are available as an online supplementary file.

References

  • 1.Confavreux C, Vukusic S. Age at disability milestones in multiple sclerosis. Brain 2006;129(pt 3):595–605. [DOI] [PubMed] [Google Scholar]
  • 2.Trojano M, Liguori M, Bosco Zimatore G, et al. Age-related disability in multiple sclerosis. Ann Neurol 2002;51:475–480. [DOI] [PubMed] [Google Scholar]
  • 3.Martinelli V, Rodegher M, Moiola L, Comi G. Late onset multiple sclerosis: clinical characteristics, prognostic factors and differential diagnosis. Neurol Sci 2004;25(suppl 4):S350–S355. [DOI] [PubMed] [Google Scholar]
  • 4.Harding KE, Liang K, Cossburn MD, et al. Long-term outcome of paediatric-onset multiple sclerosis: a population-based study. J Neurol Neurosurg Psychiatry 2013;84:141–147. [DOI] [PubMed] [Google Scholar]
  • 5.Scalfari A, Neuhaus A, Daumer M, et al. Age and disability accumulation in multiple sclerosis. Neurology 2011;77:1246–1252. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Freilich J, Manouchehrinia A, Trusheim M, et al. Characterization of annual disease progression of multiple sclerosis patients: a population-based study. Mult Scler 2018:24:786–794. [DOI] [PubMed] [Google Scholar]
  • 7.Tremlett H, Paty D, Devonshire V. The natural history of primary progressive MS in British Columbia, Canada. Neurology 2005;65: 1919–1923. [DOI] [PubMed] [Google Scholar]
  • 8.Tutuncu M, Tang J, Zeid NA, et al. Onset of progressive phase is an age-dependent clinical milestone in multiple sclerosis. Mult Scler 2013;19:188–198. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Chen C-D, Zeldich E, Li Y, et al. Activation of the anti-aging and cognition-enhancing gene klotho by CRISPR-dCas9 transcriptional effector complex. J Mol Neurosci 2018;64:175–184. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Blackburn EH, Epel ES, Lin J. Human telomere biology: a contributory and interactive factor in aging, disease risks, and protection. Science 2015;350:1193–1198. [DOI] [PubMed] [Google Scholar]
  • 11.Bernadotte A, Mikhelson VM, Spivak IM. Markers of cellular senescence. Telomere shortening as a marker of cellular senescence. Aging 2016;8:3–11. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Lin J, Epel E, Blackburn E. Telomeres and lifestyle factors: roles in cellular aging. Mutat Res 2012;730:85–89. [DOI] [PubMed] [Google Scholar]
  • 13.Haycock PC, Heydon EE, Kaptoge S, et al. Leucocyte telomere length and risk of cardiovascular disease: systematic review and meta-analysis. BMJ 2014;349:g4227. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Forero DA, González-Giraldo Y, López-Quintero C, et al. Meta-analysis of telomere length in Alzheimer’s disease. J Gerontol A Biol Sci Med Sci 2016;71:1069–1073. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Georgin-Lavialle S, Aouba A, Mouthon L, et al. The telomere/-telomerase system in autoimmune and systemic immune-mediated diseases. Autoimmun Rev 2010;9:646–651. [DOI] [PubMed] [Google Scholar]
  • 16.Guan J-Z, Guan W-P, Maeda T, et al. Patients with multiple sclerosis show increased oxidative stress markers and somatic telomere length shortening. Mol Cell Biochem 2015;400:183–187. [DOI] [PubMed] [Google Scholar]
  • 17.Ian MW, Alistair C, Gilles E, et al. Recommended diagnostic criteria for multiple sclerosis: guidelines from the international panel on the diagnosis of multiple sclerosis. Ann Neurol 2001;50:121–127. [DOI] [PubMed] [Google Scholar]
  • 18.UCSF MS-EPIC Team, Cree BAC, Gourraud P-A, et al. Long-term evolution of multiple sclerosis disability in the treatment era. Ann Neurol 2016;80:499–510. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.UCSF MS-EPIC Team, Cree BAC, Hollenbach JA, et al. Silent progression in disease activity-free relapsing multiple sclerosis. Ann Neurol 2019;85:653–666. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Kurtzke JF. Rating neurologic impairment in multiple sclerosis: an Expanded Disability Status Scale (EDSS). Neurology 1983;33:1444–1452. [DOI] [PubMed] [Google Scholar]
  • 21.Fischer JS, Rudick RA, Cutter GR, Reingold SC. The Multiple Sclerosis Functional Composite Measure (MSFC): an integrated approach to MS clinical outcome assessment. National MS Society Clinical Outcomes Assessment Task Force. Mult Scler 1999;5:244–250. [DOI] [PubMed] [Google Scholar]
  • 22.Cawthon RM. Telomere measurement by quantitative PCR. Nucleic Acids Res 2002;30:e47. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Lin J, Epel E, Cheon J, et al. Analyses and comparisons of telomerase activity and telomere length in human T and B cells: insights for epidemiology of telomere maintenance. J Immunol Methods 2010; 352:71–80. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Farzaneh-Far R, Lin J, Epel E, et al. Telomere length trajectory and its determinants in persons with coronary artery disease: longitudinal findings from the heart and soul study. PLoS One 2010;5:e8612. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Müezzinler A, Zaineddin AK, Brenner H. A systematic review of leukocyte telomere length and age in adults. Ageing Res Rev 2013;12: 509–519. [DOI] [PubMed] [Google Scholar]
  • 26.Valdes AM, Andrew T, Gardner JP, et al. Obesity, cigarette smoking, and telomere length in women. Lancet 2005;366:662–664. [DOI] [PubMed] [Google Scholar]
  • 27.Barrett ELB, Richardson DS. Sex differences in telomeres and lifespan. Aging Cell 2011;10:913–921. [DOI] [PubMed] [Google Scholar]
  • 28.Vittinghoff E, Glidden DV, Shiboski SC, McCulloch CE. Regression methods in biostatistics. 2nd ed. New York, NY: Springer, 2012. [Google Scholar]
  • 29.Golden LC, Voskuhl R. The importance of studying sex differences in disease: the example of multiple sclerosis. J Neurosci Res 2017;95: 633–643. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Belsky DW, Caspi A, Houts R, et al. Quantification of biological aging in young adults. Proc Natl Acad Sci U S A 2015;112:E4104–E4110. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Chari DM, Crang AJ, Blakemore WF. Decline in rate of colonization of oligodendrocyte progenitor cell (OPC)-depleted tissue by adult OPCs with age. J Neuropathol Exp Neurol 2003;62:908–916. [DOI] [PubMed] [Google Scholar]
  • 32.Sim FJ, Zhao C, Penderis J, Franklin RJM. The age-related decrease in CNS remyelination efficiency is attributable to an impairment of both oligodendrocyte progenitor recruitment and differentiation. J Neurosci 2002;22:2451–2459. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Goldschmidt T, Antel J, König FB, et al. Remyelination capacity of the MS brain decreases with disease chronicity. Neurology 2009;72: 1914–1921. [DOI] [PubMed] [Google Scholar]
  • 34.Rawji KS, Mishra MK, Michaels NJ, et al. Immunosenescence of microglia and macrophages: impact on the ageing central nervous system. Brain 2016;139:653–661. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Shaw AC, Goldstein DR, Montgomery RR. Age-dependent dysregulation of innate immunity. Nat Rev Immunol 2013;13:875–887. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Marrie RA. Comorbidity in multiple sclerosis: implications for patient care. Nat Rev Neurol 2017;13:375–382. [DOI] [PubMed] [Google Scholar]
  • 37.Marrie RA, Rudick R, Horwitz R, et al. Vascular comorbidity is associated with more rapid disability progression in multiple sclerosis. Neurology 2010;74:1041–1047. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Manouchehrinia A, Tench CR, Maxted J, et al. Tobacco smoking and disability progression in multiple sclerosis: United Kingdom cohort study. Brain 2013;136(pt 7):2298–2304. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Motl RW, Pilutti LA. Is physical exercise a multiple sclerosis disease modifying treatment? Expert Rev Neurother 2016;16:951–960. [DOI] [PubMed] [Google Scholar]
  • 40.Hadgkiss EJ, Jelinek GA, Weiland TJ, et al. The association of diet with quality of life, disability, and relapse rate in an international sample of people with multiple sclerosis. Nutr Neurosci 2015;18: 125–136. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Blackburn EH. Telomere states and cell fates. Nature 2000;408: 53–56. [DOI] [PubMed] [Google Scholar]
  • 42.Musella A, Gentile A, Rizzo FR, et al. Interplay between age and neuroinflammation in multiple sclerosis: effects on motor and cognitive functions. Front Aging Neurosci 2018;10:238. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Graves JS, Henry RG, Cree BAC, et al. Ovarian aging is associated with gray matter volume and disability in women with MS. Neurology 2018;90:e254–e260. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

supplementary appendix
supplementary table

RESOURCES