Psychometric Properties of the SymptoMScreen Questionnaire in a Mild Disability Population of Patients with Relapsing–Remitting Multiple Sclerosis: Quantifying the Patient’s Perspective

Abstract

Crucial elements for achieving optimal long-term outcomes in multiple sclerosis (MS) are patient confidence and effective physician–patient communication. Patient-reported instruments may provide the means to fill the gap in currently available clinician-rated measures. The SymptoMScreen (SMSS) is a brief self-assessment tool for measuring symptom severity in 12 neurologic domains commonly affected by MS. We conducted a non-interventional study to assess the dimensional structure and item characteristics of the SMSS. A total of 218 patients with relapsing–remitting MS and mild disability (median Expanded Disability Status Scale score 2.0) were studied. Symptom severity was low (SMSS score 13.5, interquartile range 4.2–27), fatigue being the domain with the highest impact. A non-parametric item response theory, i.e., Mokken analysis, found that the SMSS is a robust one-dimensional scale (overall scalability index H  0.60) with high reliability (Cronbach’s alpha  0.94). The confirmatory factor analysis model confirmed the unidimensional structure (comparative fit index 1.0, root-mean-square error of approximation  0.001). Samejima’s model fitted well an unconstrained model with different item difficulties. The SMSS shows appropriate psychometric characteristics and may constitute a valuable and easy-to-implement addition to measure the symptom severity in clinical practice.

Key Summary Points

Patients’ perspectives may differ from healthcare professionals’ priorities

Understanding the impact of multiple sclerosis (MS) symptoms from the patients’ perspective may facilitate shared decision-making

The SymptoMScreen is a self-rated questionnaire to assess symptom severity in different key neurologic domains commonly affected by MS. In addition to its good psychometric properties, it is a rapid and reliable tool to administer in clinical practice

Introduction

Multiple sclerosis (MS) is a chronic inflammatory demyelinating disease of the central nervous system that affects different functional systems and is associated with progressive disability [1]. The last 10 years has witnessed a dramatic increase in the diversity of available therapies for the management of MS. In this context, patients, who are commonly less informed than heathcare professionals on such aspects as treatment efficacy and the profiles of side effects, have a limited number of tools available on which to base their decision-making regarding therapy [2–4].

The assessment of patient perspectives can provide invaluable clinical information that may otherwise be lost when clinical evaluation is the primary source of information [3, 4]. Patient-reported outcomes are designed to capture patient perceptions of their health condition, functional level, and quality of life. However, they have been inconsistently employed in the assessment of patients with MS [5, 6].

The SymptoMScreen questionnaire (SMSS) is a self-assessment tool for measuring symptom severity in 12 critical neurologic domains commonly affected by MS: mobility, hand function/dexterity, spasticity and stiffness, pain, sensory symptoms, bladder control, fatigue, vision, dizziness, cognition, depression, and anxiety [7]. In the few studies on the SMSS conducted to date, the SMSS total score was found to correlate with clinical disability, patient-perceived physical and psychological impact, and work productivity (as assessed by the Expanded Disability Status Scale [EDSS], 29-item Multiple Sclerosis Impact Scale, and Working Productivity and Impairment Questionnaire, respectively) [7–9]. The SMSS may encourage patients to communicate symptom severity in multiple overt and less tangible key standardized domains. In addition, this instrument can help clinicians focus the discussion of symptom management during the visit, track severity, and serve as basis for making specific referrals [7]. The SMSS takes less than 5 min to complete and may constitute a useful tool to quantify symptoms in clinical practice [7]. However, the whole spectrum of its psychometric properties has not yet been studied and reported.

The aim of this study was to assess the dimensional structure and item characteristics of the SMSS in the management of MS.

Methods

A non-interventional, cross-sectional study was conducted in 17 MS units throughout Spain (PERCEPTIONS-MS Study). Key eligibility criteria included an age of at least 18 years, a diagnosis of relapsing–remitting MS according to the 2010 revised McDonald criteria [10], and an EDSS score of 0–5.0 [11]. Written informed consent was obtained from all subjects. The study was approved by the institutional review board of the Hospital Universitari Clínic (Barcelona, Spain) and conformed with the Helsinki Declaration of 1964, as revised in 2013, concerning human and animal rights, as well as with Springer’s policy concerning informed consent.

The SMSS consists of 12 items which are assessed on a seven-point Likert scale that ranges from 0 (not at all affected) to 6 (total limitation) [7]. The total score ranges from 0 to 72, with higher scores indicating more severe symptom endorsement.

Statistical Analysis

The distribution of the SMSS scores was evaluated across different levels of MS severity, including disease duration, total number of relapses, EDSS score, and number of disease-modifying therapies. The early-stage MS definition was used to establish the disease duration cutoff (≤ 3 years from first documented clinical attack) [8]. We evaluated mean differences and linear trends using one-way analysis of variance and linear regression.

A non-parametric item response theory (IRT) procedure, the Mokken scale analysis, and a confirmatory factor analysis (CFA) were performed to assess the underlying dimensions of the SMSS. Each of the 12 items was required to have a scalability coefficient (H_i) of ≥ 0.30 and an overall scale scalability index (H) of ≥ 0.30 [12, 13]. A parametric Samejima’s Grade Response Model (GRM) was also conducted to further assess the information and discrimination of SMSS items [14]. We considered a comparative fit index (CFI) value > 0.95 as an acceptable model fit and root-mean-square error of approximation (RMSEA) values of < 0.08 and < 0.05 as adequate and good model fit, respectively. We performed all analyses with R version 3.6.0 (https://cran.r-project.org/bin/windows/base/old/3.6.0/) using the libraries mokken, lavaan, and ltm.

Results

A total of 218 patients were enrolled in the study and included in the analysis (mean age 39.1 ± 9.5 years, 70.6% female). Symptom severity was low, and fatigue was the domain with the highest score (Table 1).

Table 1.

Clinical characteristics of patients enrolled in the study

Clinical characteristics	Values (N = 218 patients)
Time since diagnosis (years); mean (SD)	6.6 (4.5)
EDSS score; median (IQR)	2.0 (1.0, 4.0)
SMSS score; median (IQR)	13.5 (4.2, 27.0)

EDSS Expanded Disability Status Scale, IQR interquartile range, SD standard deviation, SMSS SymptoMScreen (questionnaire)

The SMSS scores showed a significant linear trend relationship with EDSS severity (p value < 0.001), with means ranging from 7.5 (lower EDSS category) to 28.9 (EDSS value 4), and total number of relapses (≤ 2 vs. ≥ 3; p = 0.02). No significant relationships were found between SMSS scores and disease duration (≤ 3 vs. > 4 years; p = 0.07) and between SMSS scores and number of MS treatments (1, 2, ≥ 3; p = 0.08), but the average SMSS scores (± standard deviation) did show a clear linear trend for disease duration (12.8 ± 11.2 and 16.6 ± 14, respectively) and for total number of treatments (13 ± 13.8, 15.6 ± 12.8, and 18.4 ± 13.3, respectively).

Internal Reliability

The SMSS showed high reliability (Cronbach’s alpha 0.94, bootstrap 95% confidence interval 0.93–0.95).

Non-Parametric (Mokken) Item Response Theory

The SMSS is a robust unidimensional scale with an overall scalability index of H = 0.60, and all items showing scalability coefficients of H_i > 0.50 (Table 2).

Table 2.

Psychometric characteristics of the patients as assessed according to the 12 items on the SMSS

SMSS items	Hi	Discriminationa	Informationa	CFA standardized coefficients
1	0.572	2.039	5.89	0.722
2	0.618	2.459	8.17	0.783
3	0.623	2.685	8.34	0.793
4	0.644	3.041	11.25	0.833
5	0.629	2.504	8.65	0.784
6	0.593	2.444	6.28	0.757
7	0.634	2.458	8.84	0.803
8	0.579	2.155	6.87	0.724
9	0.585	2.039	6.40	0.738
10	0.658	2.828	11.53	0.821
11	0.528	1.716	5.32	0.647
12	0.557	1.902	5.82	0.696

CFA Confirmatory factor analysis, H_i Scalability coefficient

^aDiscrimination and information coefficients were from Samejima’s Grade Response Model [14]

Item Characteristics: Grade Response Model

The item characteristic curves from Samejima’s GRM showed adequate probability patterns for response categories with an item-ranking order for discrimination that also agreed with the observed order for item information (Table 2).

Factor Structure

The CFA model confirmed the unidimensional structure suggested by Mokken analysis (RMSEA 0.001, CFI  1.0) (Table 2). All item coefficients are intrinsically linked regardless of the different techniques (non-parametric IRT, CFA, parametric IRT) as would be expected when a unidimensional domain underlies all of them (Fig. 1).

Fig. 1.

Confirmatory factor analysis (CFA) model. Lower diagonal displays the correlation graphs, upper diagonal presents Pearson’s correlation estimates, main diagonal displays the distribution of parameter estimates. Discrimination and information coefficients are from Samejima’s Grade Response Model (GRM), H_i Scalability coefficient

Discussion

Fatigue, pain, heat intolerance, mobility problems, visual impairment, anxiety, depression, and cognitive impairment are commonly found to be associated with work difficulties and poor productivity in MS patients [15]. MS is a complex disease that cannot be reduced to walking capacity, and patients must be assessed in many different aspects in each clinical examination [3, 5, 6]. In addition, the perspectives and preferences of a patient may and often do differ from the priorities of healthcare professionals. Heesen et al. found that vision, cognition, and mobility were the most important bodily functions from a patient perspective in a sample of 171 subjects with relapsing–remitting MS [16]. In the present study, visual impairment was not considered to be relevant by physicians—in contrast to patients’ opinion. The results of another quality of life assessment in MS involving 380 MS patients and 135 neurologists found a disagreement related to factors impacting health-related quality of life, with physicians focusing on physical aspects while patients considered vitality, pain, and mental health also to be relevant [17]. The management of such disruptive MS-related problems should be part of patient-centered care [3].

Clinician-rated EDSS is still the most common instrument used to assess the effectiveness of clinical interventions and to monitor disease progression despite the several shortcomings of this assessment tool [18], including inadequate assessment of several critical domains, such as cognitive performance, quality of life, and energy level [18, 19]. In this context, patient-reported outcome instruments may provide the means to fill the current gap in patient-centered impairment measures.

Our study found that the SMSS is a robust unidimensional scale that shows good psychometric properties regarding internal consistency and item discrimination and information. The SMSS shows appropriate convergent validity with the EDSS and total number of relapses as well as a linear trend with other surrogate indicators of severity.

Green et al. [7] developed the SMSS to help patients to identify and communicate symptom severity in multiple functional domains relevant for MS. Using this scale, patients rate symptoms that lead to symptom-related limitations, as opposed to measures of disability like the EDSS [7, 8]. In a sample of 1865 patients with MS, the pain, ambulation, and fatigue domains of the SMSS significantly predicted patients’ perception of health [20].

This study has a major limitation. The study population included a sample of clinically stable relapsing–remitting MS patients with low physical disability. The results may thus not be generalizable to less stable subjects and other clinical subtypes (secondary or primary progressive forms). Despite this limitation, the sample of 218 subjects was managed in 17 different MS units on a national level, representing relapsing–remitting patients across the country.

Conclusion

Understanding the impact of symptoms from the patients’ perspective may facilitate shared decision-making in MS care. The SymptoMScreen is a quick, comprehensive, and reliable instrument to assess the severity of symptoms in clinical practice.