Design of a Non-Interventional Study to Validate a Set of Patient- and Caregiver-Oriented Measurements to Assess Health Outcomes in Spinal Muscular Atrophy (SMA-TOOL Study)

Abstract

Introduction

There is a need to optimize the current clinical outcome measures in spinal muscular atrophy (SMA) incorporating patients’ and caregivers’ perspectives. The aim of this study is to evaluate the psychometric properties (validity, reliability and sensitivity to change) of a set of existing questionnaires and newly created items grouped in a “toolbox” to assess the impact of SMA on the physical, psychological and activities of daily living domains of the patient’s life.

Methods

This non-interventional, prospective study will be conducted at 12 neuromuscular clinics specialized in the management of patients with SMA in Spain. An expert panel of pediatric and adult neurologists, rehabilitation physicians, and a patient representative participated in the study design and selected key disease dimensions to explore and their respective measurements: mobility-independence, fatigue and endurance, pain, fatigability, breathing and voice, sleep and rest, and vulnerability. Patients aged 2 years or older with a confirmed diagnosis of 5q-autosomal recessive SMA (genetic confirmation of homozygous deletion or heterozygosity predictive of loss of function of the SMN1 gene) will be recruited.

Planned Outcomes

The development of robust outcome measures in collaboration with the patient community is essential to determine what is meaningful to patients and their caregivers. This study will provide us with a comprehensive set of tools to better capture the course of the disease and the response to treatments.

Key Summary Points

The therapeutic landscape in spinal muscular atrophy has been changing in the past decade, with key improvements in motor and respiratory dimensions.

Outcome measures traditionally used in clinical trials are inadequate to assess the full range of disease severity in real life.

Patient-reported instruments remain largely understudied in spinal muscular atrophy.

This study will provide a comprehensive set of tools incorporating patients’ and parent caregivers’ preferences and meaningful outcomes.

Digital Features

This article is published with digital features, including a summary slide, to facilitate understanding of the article. To view digital features for this article, go to https://doi.org/10.6084/m9.figshare.13379642.

Introduction

Spinal muscular atrophy (SMA) is an autosomal recessive neuromuscular disease caused by a homozygous deletion or mutation of the survival motor neuron 1 (SMN1) gene on chromosome 5q13, leading to progressive muscle weakness and atrophy [1, 2]. Although SMA is clinically very heterogeneous, it usually has a huge impact on the health-related quality of life of patients, their families and society [3–5].

The landscape of SMA has changed over the past few years [6]. Different therapeutic approaches such as antisense oligonucleotides, small molecules or gene therapy show increased life expectancy and motor and respiratory improvements [7]. However, it has become increasingly apparent that outcome measures used in clinical trials are inadequate to assess the full range of disease severity in real life, both from the patients’ and caregivers’ perspective and from the clinicians’ perspective [8–16].

Respiratory and bulbar functions as well as mobility were identified as the priority areas in which treatments should have an effect in a survey involving 822 patients with type II and III SMA from Europe [9]. In a qualitative study conducted in the United States, 123 patients, caregivers and clinicians considered that outcome measures should assess other important features of SMA, including the ability to perform daily activities, respiratory function, swallowing, fatigue and endurance [10]. Mobility, self-toileting and feeding, spending time alone, independence and being engaged in social activities/building relationships were the aspects of quality of life and activities of daily living that matter most as possible treatment benefits in a survey of 298 adults with type I–IV SMA and caregivers in the USA [11]. A recent qualitative study carried out in Spain by the Spinal Muscular Atrophy Foundation (FundAME) assessed five focus groups with different types of SMA patients and parents to identify which were the main areas of daily life that were impacted by the disease [12]. Mobility, fatigue and endurance, pain, scoliosis, contractures and hip dislocation, feeding, breathing and voice, sleep and rest, vulnerability, infections and hospitalizations, and time spent in caring activities were the most impactful areas. Other initiatives to assess patients’ perception of well-being, the impact of a particular activity of daily living on their quality of life and patients’ expectations regarding the effects of therapies on their lifestyle, such as the European Patient Expectation Survey SMA (EUPESMA-2019), are still ongoing [16]. Therefore, well-designed patient- and caregiver-oriented outcome measures are needed to ensure that we are measuring the most relevant and clinically meaningful outcomes.

The aim of this study is to assess the psychometric properties of a set of existing questionnaires and new items to gather information on the impact of SMA on physical, psychological, social and functional domains from the patients’ and caregivers’ perspective.

Methods

Design

The SMA-TOOL is a prospective, non-interventional study that will be conducted at 12 hospital-based neuromuscular clinics specialized in the management of patients with SMA in Spain. The psychometric properties (reliability, construct validity, concept validity, and sensitivity to change) of a set of patient- or proxy-reported outcome questionnaires will be assessed. There will be two study visits: one at enrollment and the other at the 4-month follow-up.

This study is conducted in accordance with the Good Clinical Practice Guidelines of the International Conference on Harmonisation and with the ethical principles of the Declaration of Helsinki. The study protocol was approved by the institutional review board of the Hospital Universitari de Bellvitge (Barcelona, Spain; reference code: PR264/20).

Selection of Disease Dimensions and Measurements

A multidisciplinary research team of pediatric and adult neurologists and rehabilitation physicians with expertise in neuromuscular disorders, and a patient representative (FundAME) participated in the study design and selection of key disease dimensions to explore. A review of the literature was conducted to identify current measurements. Finally, the research team selected the following dimensions and their respective assessment instruments:

1. Mobility-independence to perform daily activities: SMA Independence Scale (SMAIS) [16, 17]

2. Fatigue and endurance: Neuro-QoL Fatigue test [18]

3. Pain: Neuro-QoL Short Form—Pediatric Pain and PROMIS Pain Interference tests [18, 19]

4. Fatigability items

5. Breathing and voice

6. Sleep and rest

7. Vulnerability

The last four dimensions will be assessed using specific items developed by the FundAME qualitative study [12].

Participants

The study will recruit patients with confirmed diagnosis of 5q-autosomal recessive SMA (genetic confirmation of homozygous deletion or heterozygosity predictive of loss of function of the SMN1 gene) aged from 8 years or their parents, if aged between 2 and 8 years. Written informed consent will be obtained from all subjects. Exclusion criteria will be any medical or psychological condition that according to the investigator’s judgment might compromise the ability of the patient or their parents to provide the informed consent. The patients will be recruited consecutively from October 2020 to March 2021.

Measurements

The SMA Independence Scale (SMAIS) is a self-report questionnaire to assess changes in the amount of assistance that patients with type 2 and non-ambulatory type 3 SMA require to perform daily activities in the past 7 days [17]. The SMAIS includes 29 items assessing activities such as washing and hygiene, dressing, eating and drinking, picking up and moving objects, mobility and strength, chores, writing and using a computer. Each item is scored using a five-point scale, with higher scores indicating greater independence. There are two versions: a patient-reported instrument for individuals aged 12 or older and an observer-reported version for caregivers of individuals between the ages of 2 and 11 years. The same items are included in both versions.

The NeuroQoL Fatigue is a self-report, computerized adaptive test (CAT) to assess sensations ranging from tiredness to an overwhelming, debilitating and sustained sense of exhaustion that decreases one’s capacity for physical, functional, social and mental activities in the past 7 days [18]. Patients aged 8–17 years will use an 11-item pool, and adults a 19-item version. Each item employs five response frequency options (“never” to “always”). Item scores are summed to create a total row score which will be transformed to a T-score. High scores indicate worse (undesirable) self-reported health. NeuroQoL Fatigue will be answered by the parents in patients under 8 years of age.

The NeuroQoL Pain is a self-report, 10-item measure to assess pain in the past 7 days in patients aged 8–17 years [18]. It is conceptually divided into components of quality (e.g. the nature, characteristics, intensity, frequency and duration of pain), behaviors (e.g. verbal and nonverbal actions that communicate pain to others) and interference (e.g. impact of pain on physical, mental and social activities). Each item employs five response frequency options (“never” to “usually”). Item scores are summed to create a total row score which will be transformed to a T-score. High scores indicate worse (undesirable) self-reported health. NeuroQoL Pain will be answered by the parents of patients under 8 years of age.

The Patient-Reported Outcomes Measurement Information System (PROMIS) Pain Interference Adult Form is a CAT self-report instrument based on a 40-item pool that measures the consequences of pain in the past 7 days on relevant aspects of a person’s life and may include the extent to which pain hinders engagement with social, cognitive, emotional, physical and recreational activities [19]. Each item has five response options (“not at all” to “very much”). Item scores are summed to create a total row score which will be transformed to a T-score. A higher PROMIS T-score represents more of the concept being measured.

Fatigability, breathing and voice, sleep and rest, and vulnerability in the past month will be assessed using a set of items developed in a qualitative study previously conducted by FundAME [12]. Items will be completed by the parents for patients under 8 years of age.

Fatigability

The fatigability assessment consists of ten self-report items plus three extra items for non-walking patients and four extra items for walking patients (Table 1). Each item has five response options (“never to always” for the first five items and “impossible” to “easy” in the following items).

Table 1.

Fatigability

During the last month…
	Never	Almost never	Sometimes	Almost always	Always
1. Have you needed to rest often during the day or for long periods because you felt tired?
2. Have you needed to choose your activities during the day to be able to do what you wanted?
3. Have you had problems to maintain your posture during the day because you felt tired?
4. Have there been activities you were able to do in the morning but could not do in the afternoon or evening/night (you ran out of energy as the day went on)?
5. If you made a bigger exertion than usual, tiredness lasted until the next day?

During the last month, how much difficulty have you had in successfully completing the following activities that included repetitive or continuous movements?
	Impossible	Very difficult	Difficult	Somewhat easy	Easy	Not applicable
6. Taking notes on paper
7. Sending text messages
8. Combing your hair
9. Brushing your teeth
10. Eating by yourself

During the last month, have you been able to do the following movements several times with the same strength and speed?
	Impossible	Very difficult	Difficult	Average (not easy or difficult)	Easy
(For non-walking patients)
11. Sit up straight in the chair when your back is supported
12. Maintain the position of your head
13. Repeat specific movements when you are playing

	Impossible	Very difficult	Difficult	Average (not easy or difficult)	Easy
(For walking patients)
14. Walk up a whole flight of stairs
15. Keep the pace when you walk
16. Rise from a chair or the bed
17. Get in or out of a car

Breathing and Voice

The breathing and voice assessment consists of eight self-report items, two related to the time at which the response is given and six in the past month as the reference period (Table 2). The first item is answered “yes/no,” the last item is answered “yes/no/sometimes or I don’t know,” and the rest of items have five response options (“I don’t have difficulties” to “it is impossible”).

Table 2.

Breathing and voice

1. Can you speak?
Yes	No

During the last month…
	I have no difficulty	It’s a bit difficult	It's difficult	It's very difficult	It's impossible	Not applicable
2. Can you cough effectively (expelling mucus)?
3. How difficult is it for you to make yourself understood when speaking with someone you know?
4. How difficult is it for you to make yourself understood when speaking with someone you don't know?
5. How difficult is it for you to make yourself understood when speaking by phone?
6. How difficult is it for you to participate in group conversations?
7. How difficult is it for you to make yourself understood with background noise?

Have you had changes in the quality of your voice apparently caused by secretions in your throat (wet voice)?
Yes	No	Sometimes

Sleep and Rest

The sleep and rest assessment consists of three self-reported items (Table 3). Each item has five response options (“never” to “always”).

Table 3.

Sleep and rest

During the last month…
	Never	Almost never	Sometimes	Almost always	Always
1. Have you woken at night to ask for help to be able to move in the bed?
1a. On the nights you have woken, how many times do you usually wake? (Depends on 1)	1	2–3	More than 3

	Never	Almost never	Sometimes	Almost always	Always
2. Have you woken at night because the ventilation machine was bothering you?
3. Have you gotten up tired after sleeping at night?

Vulnerability

The vulnerability assessment consists of nine self-reported items. Each item has five response options (“never” to “almost every day” for the first item and “never to always” for the other eight items; (Table 4).

Table 4.

Vulnerability

How many times have you choked during the last month (when eating, speaking, because of mucus, etc.)?
Never	Very few days	Some days	Many days	Almost every day

During the last month…
	Never	Almost never	Sometimes	Almost always	Always
2. When you have choked, did you feel you couldn’t breathe? (Depends on 1)
3. Has the risk of choking made you always eat with your usual assistant?
4. Has the risk of choking limited the kind of food you eat when accompanied by someone who is not your usual assistant?
5. Has the risk of choking limited the kind of food you can eat by yourself?

During the last month, has the risk of having respiratory infections limited your school life, your social life or your visits to health centers and hospitals?
Never	Almost never	Sometimes	Often	Almost always

During the last month, how often have you been unable to recover an upright posture?
Never	Almost never	Sometimes	Often	Almost always	Not applicable

During the last month, have you felt at risk because you couldn't recover your posture? (Depends on previous item)
Never	Almost never	Sometimes	Often	Almost always

During the last month, has the risk of not being able to recover your posture led you to not being able to be alone?
Never	Almost never	Sometimes	Often	Almost always

In addition to the instruments of the “toolbox,” the Clinical Global Impression—Severity (CGI-S), the Clinical Global Impression—Improvement (CGI-I) and the Patient Global Impression—Improvement (PGI-I) scales will also be administered [20]. The CGI-S is a seven-point single-question scale that requires the clinician to rate the severity of the patient’s illness at the time of assessment (from “normal, not at all ill” to “among the most extremely ill patients”). The CGI-I is a single question rated on a seven-point scale that requires the clinician to compare the patient's overall clinical status to baseline (from “very much improved” to “very much worse”). The PGI-I is a single question asking the patient to rate their condition compared with how it was at baseline (from “very much better” to “very much worse”).

Study Flow

Investigators will collect sociodemographic and clinical data and complete the CGI-S scale using an electronic case report form designed for this study. Patients, or their parents when they are under 8 years of age, will complete questionnaires and items grouped in the “toolbox” using a tablet or touch panel. At 4 months of follow-up, all the above information and measures will be collected again adding the CGI-I and PGI-I scales. Further details of the study flow are shown in Fig. 1.

Fig. 1.

Study flow

Sample Size

No consensus exists to define the sample size of studies assessing psychometric properties of patient-reported outcomes [21]. One recommendation is to enroll 2–20 subjects per item, so a 40-item questionnaire such as the PROMIS (the longest one to be administered in the study) would require at least 80 subjects [22]. The plan is to recruit a total of 150 patients.

Statistical Analysis

Data analyses will be performed using SAS^® statistical software. All patients participating in the study who meet the eligibility criteria will be included in the study population. Continuous data will be presented as number of observations (N), mean, standard deviation (SD), minimum, Q1, median, Q3 and maximum. Data for categorical and ordinal variables will be presented as counts, proportions or percentages. In both cases, the number of missing data (N missing) will be specified.

For all analysis, the number of patients with available data will be indicated. Missing data on different variables will not be replaced, except for scoring algorithms used for patient-reported outcomes. Normal distribution of data will be assessed. In cases where the normality assumptions are met, parametric tests will be used (to compare means and proportions), and appropriate nonparametric tests will be used if the data are not normally distributed. All statistical tests will be two-sided with an α level of 0.05.

Demographic and Clinical Variables

Demographic (age, gender, living status and educational level) and clinical characteristics will be recorded for all evaluable patients. Clinical data includes the following variables: time since SMA diagnosis, type of SMA, number of SMN2 copies, functional classification according to the World Health Organization motor milestones, ambulation stage, muscle strength (elbow flexors, knee extensors and flexors), use of ventilator support and/or gastrointestinal tube [yes/no], history of scoliosis surgery [yes/no], and the use of active treatment for SMA [yes/no].

Primary Objective Analyses

The reliability, construct validity, and concept validity of SMAIS and the new items included in the “toolbox” (fatigability, breathing and voice, sleep and rest, and vulnerability) will be measured. For the new items, it will first be necessary to group them into dimensions and questionnaires according to the previous hypothesized structure and then to perform an exploratory factorial analysis to assess construct validity. The scores of each dimension or questionnaire will be calculated, and analysis of reliability, concept validity and sensitivity to change will be conducted for these questionnaires. Construct validity of the SMAIS will be assessed by conducting confirmatory factorial analysis to confirm the dimension structure of the original version of the questionnaire. Reliability will be assessed in terms of internal consistency by means of Cronbach’s alpha coefficient, which must be more than 0.7.

Concept validity will be assessed by comparing the scores of the questionnaires between different patient groups according to previous hypotheses: scores obtained in each subscale will be compared according to clinical data such as type of SMA, number of SMN2 copies, functional classification, ambulation stage, muscle strength, use of ventilator support and/or gastrointestinal tube, history of scoliosis surgery, use of active treatment for SMA, CGI-S and CGI-I scores using a one-way ANOVA followed by Bonferroni post hoc test for multiple comparisons, to determine which groups differ from the rest. Pearson or Spearman correlations between scores in new items and SMAIS score and other instruments (NeuroQoL Fatigue, NeuroQoL Pain Children short form, and PROMIS Adult Pain Interference) will be presented.

Sensitivity to change or longitudinal validity will be assessed by comparing the scores of the questionnaires at first and second visits in the group of patients who change their clinical status according to investigator’s and patient’s opinion using the CGI and the PGI scales, respectively. Effect sizes will be used to estimate the magnitude of changes. Effect size will be defined as the difference between baseline and final visit mean scores divided by the standard deviation at baseline. An effect size of 0.2 is equivalent to a small effect size, 0.5 to a medium effect size, and values of 0.8 or over are considered to be large effect sizes.

Potential Risks of Participating in the Study

This is a non-interventional study to evaluate the psychometric properties of a set of assessment scales. The patients will be treated by participating investigators following current clinical practice and according to their judgment. Subjects can leave the study at any time for any reason without any consequences.

Research burden is defined as the amount of effort required by the respondent, the amount of stress, and the length and frequency of scales and questionnaires [23]. The study design team took into account this problem, aiming a minimally disruptive clinical research. Such approach may improve trial recruitment and retention.

Discussion

Major changes in the therapeutic management of patients with SMA are currently occurring with a crucial impact on the natural history of the disease [6].

SMA is a heterogeneous disorder affecting from non-ambulant young children to ambulant adults. Different reliable clinician-rated motor function scales have been used in research and clinical practice, including the Children’s Hospital of Philadelphia Infant Test of Neuromuscular Disorders, Hammersmith Functional Motor Scale-Expanded, Motor Function Measure, and Revised Upper Limb Module. Relevant milestones, including sitting, standing or walking are more objectively observed and measured with these instruments compared to smaller but meaningful changes in ability to perform daily activities [7, 10]. Moreover, there is an urgent need to measure the impact of changes in theses scales on activities of daily living and quality of life of SMA patients.

In this context, new challenges and opportunities have emerged to emphasize the role of patients’ and caregivers’ voice. Patient-reported outcomes are designed to capture patient perceptions of their health condition, functional level and quality of life [24]. The systematic assessment of patient perspectives has the capacity to provide invaluable clinical information that could otherwise be lost when relying on clinical evaluation alone [25]. Messina et al. identified 36 instruments currently used in clinical trials to evaluate the impact of SMA on quality of life, activities of daily living, and caregiver burden [26]. Only ten included a combination of items exploring all these dimensions, and only six were specifically developed for SMA. Mercuri and Muntoni recently coordinated a workshop with advocacy groups, physicians and physical therapists with a specific experience in patient-reported outcome measures in neuromuscular disorders [16]. They reported that scales assessing activities of daily living and caregiver burden are more likely to capture changes when used longitudinally than health-related quality of life measures. Despite these efforts, patient-reported instruments remain largely understudied, especially in adult patients [27].

Consequently, there is a need to supplement the current clinical outcome measures in SMA incorporating patients’ and parent caregivers’ preferences and meaningful outcomes, especially on activities of daily living domains. Regulatory agencies, scientific societies and advocacy groups have been working to improve understanding, measurement and incorporation of the perspectives and priorities of patients with SMA and their caregivers in therapy development, regulatory review processes and clinical practice [15, 16, 26].

This study will validate the psychometric properties (validity, reliability and sensitivity of change) of a set of tools selected by a multidisciplinary team including patients representatives. The approach was to assess a combination of existing instruments along with new items identified and developed in a previous qualitative study with patients and caregivers led by FundAME in Spain.

Our study protocol has some limitations. First, the study population may not be representative of the entire spectrum of SMA because the lack of inclusion of children in the first 2 years of life. Second, parents will complete questionnaires when their children are less than 8 years old. Previous research shows that children’s and parents’ perceptions of the degree of illness may often differ [11, 28]. Third, in order to avoid participant fatigue, some of the key domains previously identified in SMA could not be included in the final set of instruments to assess.

Despite these limitations, the results obtained will provide a comprehensive set of tools with complementary information beyond motor aspects from a patients’ perspective that would be useful to assess the effectiveness of treatments in research and clinical practice. A sample of 150 patients including children and adults managed in 12 different neuromuscular clinics at the national level will allow results to be generalized to community practice. This study protocol is also a good example of a patient advocacy group working successfully with a multidisciplinary team of healthcare professionals caring for patients with SMA, and may serve as a model for similar initiatives in other countries.

Study Status

This study is ongoing. The expected end date of patient recruitment is March 1, 2021.