Reliability of home-based, motor function measure in hereditary neuromuscular diseases

Abstract

Objective

To evaluate the reliability of the motor function measure (MFM) scale in the assessment of disease severity and progression when administered at home and clinic and assess its correlation with the Paediatric Outcomes Data Collection Instrument (PODCI).

Methods

In this prospective study, two assessors rated children with hereditary neuromuscular diseases (HNMDs) using the MFM at the clinic and then 2 weeks later at the patients’ home. Intraclass correlation coefficient (ICC) was calculated for the reliability of the MFM and its domains. The reliability of each item was assessed and the correlation between MFM and three domains of PODCI was evaluated.

Results

A total of 48 children (5–17 years of age) were assessed in both locations and the MFM scale demonstrated excellent inter-rater reliability (ICC, 0.98). Weighted kappa ranged from excellent to poor. Correlation of the home-based MFM with the PODCI domain ‘basic mobility and transfers’ was excellent, with the ‘upper extremity’ domain was moderate, but there was no correlation with the ‘happiness’ domain.

Conclusion

The MFM is a reliable tool for assessing patients with HNMD when used in a home-based setting.

Introduction

Standardized functional examination scales for children with hereditary neuromuscular diseases (HNMDs) are designed to assess multiple functional variables in different scenarios systematically.¹ The assessments follow a detailed description of the functional phenotype in each patient with a progressive neuromuscular disease and are useful for prognosis and assessing outcomes in clinical trials.¹ However, the clinical presentations of HNMDs are varied. For example, in certain types of hereditary polyneuropathies, the disease progresses slowly,² whereas in other HNMDs, loss of the ability to walk independently occurs at an early stage.³ Indeed, in type II spinal muscular atrophy (SMA) children never walk by themselves, but in type III SMA, the loss of this ability is variable.⁴

According to the International Classification of Functioning, Disability and Health, it is advisable that the clinical examination and assessment of progression of the disease in patients with HNMD are based on an evaluation of body function and structure (i.e. muscular strength and range of movement) as well as an assessment of daily activities and participation.⁵ Previously, muscular strength was the most commonly used method for defining outcomes to treatments.^6–8 However, patients with the same muscular strength may show different performance abilities during their daily activities. For this reason, assessment of daily activities using functional scales is now one of the preferred methods for evaluating the progression in HNMD.^1,5,9,10 In a clinical setting, functional scales may demonstrate a child’s ability to fulfil many daily tasks.¹ While the clinical assessment of capability to perform a task requires a controlled environment that minimizes variables during the execution of the tasks, the assessment of performance depends on the environment.¹¹ Therefore, the assessment of capability is preferred over the assessment of performance when early meaningful changes in disability are being investigated.¹¹

Some of the most frequently used scales for assessing children with HNMD are the Hammersmith functional motor scale, the North Star Ambulatory Assessment scale and the Motor Function Measure (MFM).^1,9,10 The North Star and the Hammersmith scales were specifically designed for assessing mobility in children with Duchenne muscular dystrophy (DMD) and SMA types II and III.^9,10 However, the MFM has been validated in patients with different neuromuscular diseases and includes items for proximal and distal abilities as well as axial functioning. The assessment scale has been used for follow-up in DMD, SMA, and hereditary polyneuropathies;^12,13 and in DMD has been used for assessing primary outcomes after experimental therapy.¹⁴ The MFM scale has also been proposed as a guide for anticipated treatment when the loss of ambulation is predicted.¹⁵

Although functional scales should be administered in a clinical setting, for many children with HNMD attending a hospital for an assessment may not be easy. For example, musculoskeletal and respiratory comorbidities that require several assistive devices make the trip to a hospital a burdensome task. In clinical research, the loss of subjects due to problems with transportation to the hospital can be a major difficulty,¹⁶ so home-based clinical and functional assessments of patients offer a solution to the problem.¹⁷ To date, the reliability of functional scales has been tested in hospital-based settings and studies that have evaluated the reliability of functional scales in a home-based setting are scarce.^1,16 To the best of our knowledge, the reliability of the MFM has only been assessed in a hospital-based setting.¹ Therefore, the primary objective of this study was to evaluate the reliability of the MFM when administered in a home-based setting by comparison with a hospital-based setting. A secondary objective was to investigate the correlation between home-based MFM scale scores and Paediatric Outcome Data Collection Instrument (PODCI) scores. The PODCI is a subjective measurement tool designed to provide a standardized method of assessing outcomes in paediatric musculoskeletal conditions and has been validated in Spanish.^18,19 The PODCI has been used in the fields of orthopaedics and rheumatology, musculoskeletal disorders related to cerebral palsy and other paediatric pathologies such as arthrogryposis.^18,20

Patients and methods

Study population

Children with HNMD (i.e. DMD, Becker’s muscular dystrophy, limb-girdle muscular dystrophy, facioscapulohumeral muscular dystrophy, SMA or hereditary polyneuropathies) and attending the Instituto de Ortopedia Infantil Roosevelt, Bogota, Colombia were eligible for this prospective study, which took place between April 2011 and December 2012. The inclusion criteria were: (i) confirmed diagnosis of HNMD; (ii) < 18 years of age; (iii) confirmatory genetic diagnostic test or, supporting data from a typical clinical phenotype, muscular biopsy and/or electromyographic examination. The exclusion criteria were: (i) severe cognitive impairment (i.e. patients who were not able to follow instructions) or severe disruption of communicative ability; (ii) recent surgery or concurrent disease that prevented the evaluations.

The Committee on Research Ethics of the Universidad Nacional de Colombia, School of Medicine, Bogota, and the Board Committee of the Instituto de Ortopedia Infantil Roosevelt, Bogota approved the study. Both committees followed the ethical standards of the Helsinki Declaration. Written informed consent was obtained from all parents.

Administration of the MFM

The MFM has 32 items separated into three domains: D1, standing position and transfers (13 items); D2, axial and proximal limb motor function (12 items); and D3, distal limb motor function (7 items).¹ Items were scored according to the user manual of the Spanish version that is available through the MFM homepage (http://www.mfm-nmd.org/le-manuel-utilisateur.aspx). Each item may be scored from 0 to 4 (0, does not initiate movement or the starting position cannot be maintained; 1, partially completes the exercise; 2, completes the exercise with compensations, slowness or obvious clumsiness; 3, completes the exercise with a standard pattern).¹ Scores are expressed as a percentage of a maximum possible score. By contrast, the PODCI is a quality of life scale that explores seven domains of a child’s daily functioning, perception of illness, and happiness.¹⁸ For this present study, three domains of the PODCI were selected (i.e. ‘transfers and basic mobility’, ‘upper extremity functioning’ and ‘happiness’).

Two observers, both with experience of the MFM scale, performed the evaluations 2 weeks apart, one at the clinic, the other located at the patient’s home. Firstly, an independent, physical therapist administered the initial hospital-based MFM. Subsequently, a third-year physical medicine and rehabilitation resident (E.R.C.) administered the home-based MFM. Prior to the study, the two observers met with the head of the Department of Rehabilitation (F.O.C.) and agreed on a standardized approach for administering the scale either at the hospital or at home. During the home-based administration of the MFM it was not possible to use a height-adjustable table or chair, a mat or a stretcher. Instead, firm surfaces that allowed children to keep elbows at 90 degrees with support to forearms were used. Chairs available in the house appropriate to the children’s height that allowed them to keep feet flat on the ground with knees at 90 degrees were used. Items that required a supine starting position were evaluated in the children’s or other beds. For the home-based assessment, parents answered questions relating to ‘basic mobility and transfers’, ‘upper extremities functioning’ and ‘happiness’ domains of the PODCI.

Statistical analyses

For reliability tests, a sample size of approximately 50 patients has been suggested to be adequate.²¹ Continuous data were reported as medians and interquartile ranges and categorical data were reported as counts and percentages. For the inter-rater reliability of the MFM total score and its domains, the intraclass correlation coefficient (ICC; two-factors, mixed effects) was determined.²² The better the agreement among raters, the closer the ICC was to 1.0.

For the reliability of each item, the total observed percentage of agreement (concordance index) and weighted kappa coefficient were calculated. The weighted kappa coefficient is useful in reliability studies when data are ordinal and it considers absolute and relative agreement.²³ Weighted kappa was interpreted as follows: poor (<0.40), moderate (0.40–0.60), good (0.61–0.80) and excellent (>0.80) agreement.²⁴ Spearman’s rho was calculated in the assessment of correlation between the home-based MFM and the PODCI scores. The correlation between the MFM total score with each of the PODCI domains was assessed as was the relationship between the score of each of the MFM domains with each of the PODCI domains.

All data analyses were performed using SPSS® software version 22 for Mac® (IBM Corp, Armonk, NY, USA). A P-value < 0.05 was considered to indicate statistical significance.

Results

Forty-eight children, between 5 and 17 years of age were assessed, of whom 37 (77.1%) were boys. Overall, 26 (54.2%) children could not walk, 14 (29.2%) used a conventional wheelchair exclusively, four (8.3%) used a motorized wheelchair exclusively, eight (16.7%) used both types of wheelchairs and two (4.2%) used noninvasive ventilation. The clinical characteristics, MFM and PODCI scores of all patients are shown in Table 1.

Table 1.

Clinical characteristics of the children participating in the study and their motor function measure (MFM) and Paediatric Outcomes Data Collection Instrument (PODCI) scores.

Diagnosis	n (%)	Age, years	MFM total score hospital-based	MFM total score home-based	PODCI ‘upper extremity’ domain	PODCI ‘mobility’ domain	PODCI ‘happiness’ domain
SMA 2	4 (8.3)	5.0 (5, 17)	38.5 (22.9, 47.9)	38.5 (22.9, 48.9)	66.5 (60, 93)	19.0 (7, 37)	82.0 (80, 100)
SMA 3	4 (8.3)	9.5 (6, 15)	84.8 (80.2, 88.5)	82.2 (80.2, 86.4)	81.0 (76, 90)	74.5 (67, 80)	90.0 (70, 100)
CMT	2 (4.2)	12.0 (9, 15)	85.9 (82.2, 89.5)	85.9 (85.4, 86.4)	71.5 (67, 76)	87.0 (87, 87)	30.0 (25, 35)
LGMD	6 (12.5)	13.0 (6, 17)	56.0 (30, 91.6)	60.9 (31.2, 93.7)	69.0 (43, 90)	64.0 (27, 81)	70.0 (70, 85)
FSHMD	2 (4.2)	17.0 (17, 17)	66.1 (46.8, 85.4)	67.1 (46.8, 87.5)	78.5 (76, 81)	67.0 (40, 94)	45.0 (45, 45)
DMD	26 (54.2)	12.0 (5, 17)	44.3 (22.9, 88.5)	44.2 (22.9, 88.5)	57.0 (14, 100)	20.0 (0, 88)	62.5 (20, 100)
CM	4 (8.3)	10.0 (5, 17)	79.6 (51, 87.5)	78.6 (50.0, 94.7)	85.5 (57, 100)	68.0 (20, 100)	87.5 (65, 100)
Total	48 (100.0)	11.5 (5, 17)	51.0 (22.9, 91.6)	51.0 (22.9, 94.7)	71.0 (14, 100)	37.5 (0, 100)	70.0 (20, 100)

Data are expressed as n of patients (%) or median (interquartile range).

SMA 2, type 2 spinal muscular atrophy; SMA 3, type 3 spinal muscular atrophy; CMT, Charcot-Marie-Tooth disease; LGMD, limb-girdle muscular dystrophy; FSHMD, facioscapulohumeral muscular dystrophy; DMD, Duchenne muscular dystrophy; CM, congenital myopathy.

The reliability test results by domains are shown in Table 2. For inter-rater reliability (i.e. hospital-based and home-based), the ICC of the MFM total score was 0.98 (95% confidence interval 0.97, 0.99; data not shown). The inter-rater reliability for D1 (standing position and transfers) was 0.98 and was greater than the other two domains. For D2 (axial and proximal limb motor function) the ICC was 0.97 and for D3 (distal limb motor function) the ICC was 0.90.

Table 2.

Intraclass correlation coefficient (ICC), concordance index (CI) and weighted kappa coefficient (Kappa) for each of the motor function measure (MFM) domains.

MFM domain	MFM score hospital-based median (interquartile range)	MFM score home-based median (interquartile range)	ICC	CI >90% n	CI 81–90% n	CI 71–80% n	CI 61–70% n	Kappa (excellent) n	Kappa (good) n	Kappa (moderate) n	Kappa (poor) n
D1	7.6 (2.5, 57.6)	9.2 (2.5, 62.8)	0.98	2	10	1	0	10	3	0	0
D2	81.4 (58.7, 96.5)	82.4 (58.3, 94.4)	0.97	1	7	2	2	4	5	1	2
D3	80.9 (71.4, 89.2)	80.9 (76.1, 91.6)	0.90	1	2	2	2	0	6	1	0

MFM domain: D1, standing position and transfers (13 items); D2, axial and proximal limb motor function (12 items); D3, distal limb motor function (7 items).¹

MFM scores are expressed as a percentage of a maximum possible score (100%).

For ICC values, the better the agreement among raters, the closer the ICC is to 1.0.²²

Weighted kappa was interpreted as follows: poor (<0.40), moderate (0.40–0.60), good (0.61–0.80) and excellent (>0.80) agreement.²⁴

The weighted kappa coefficient was excellent for 14 items, good for another 14 items, moderate for two items and poor for another two items (Table 2). The concordance index was >90% for four items, 81–90% for 19 items, 71–80% for five items and 61–70% for a further four items (Table 2).

The home-based assessed MFM total score showed excellent correlation with the ‘transfer and basic mobility’ domain score of the PODCI (r = 0.92, P < 0.001) (Figure 1), moderate correlation with the ‘upper extremity’ domain score (r = 0.68, P < 0.001) (Figure 2) and no correlation with the ‘happiness’ domain score (r = 0.1, not statistically significant) (Figure 3).

Figure 1.

Scatter diagram showing the correlation between scores from the home-based motor function measure (MFM) scale and scores from the ‘basic mobility and transfer’ domain of the Paediatric Outcomes Data Collection Instrument (r = 0.92, P < 0.001). Scores are expressed as a percentage of a maximum possible score.

Figure 2.

Scatter diagram showing the correlation between scores from the home-based motor function measure (MFM) scale and scores from the ‘upper extremity’ domain of the Paediatric Outcomes Data Collection Instrument (r = 0.68, P < 0.001). Scores are expressed as a percentage of a maximum possible score.

Figure 3.

Scatter diagram showing the correlation between scores from the home-based motor function measure (MFM) scale and scores from the ‘happiness’ domain of the Paediatric Outcomes Data Collection Instrument (r = 0.1, not statistically significant). Scores are expressed as a percentage of a maximum possible score.

The association between each of the three MFM domains (D1, D2 and D3) and each of the PODCI domains was assessed (data not shown). For the PODCI ‘upper extremity’ domain, the correlation was good with D1 (r = 0.7, P < 0.001), and moderate with D2 (r = 0.6, P < 0.001) and D3 (r = 0.5, P < 0.001) (data not shown). For the PODCI ‘basic mobility and transfers’ domain, the correlation was excellent with D1 (r = 0.9, P < 0.001) and D2 (r = 0.8, P < 0.001) and good with D3 (r = 0.7, P < 0.001). For the PODCI ‘happiness’ domain, there was no correlation with all three MFM domains (r = 0.1, r = 0.1, and r = 0.2, for D1, D2, and D3, respectively).

Discussion

While the reliability of any functional assessment scale may be hampered by inter-rater or environmental differences, the results of this present study suggest that the situational reliability (i.e. the conditions under which measurements are made) of the MFM scale and its domains is excellent. Therefore, the MFM may not only be administered by different assessors but also in an uncontrolled setting in a patient’s home.

This present study found that the inter-rater reliability indices were high and similar to those of other research that had been performed in standardized environments.¹ The present study used weighted kappa statistics to assess the reliability of each of the MFM items and ‘good’ or ‘excellent’ results were obtained in 28/32 items. In addition, 23/32 of the MFM items had a concordance index above 80%. Again, these findings are similar to other research.¹ For example, in the original publication of the MFM scale, moderate to excellent kappa statistics were reported for the 32 items (0.51–0.94), which had been assessed in the same controlled hospital environment.¹ In a Portuguese validation study of the MFM that used video recordings of an examination that also took place in a controlled environment, the authors reported excellent weighted kappa statistics for all 32 items (0.93–1.0).²⁵ In another clinic-based study that attempted to validate the use of several outcome measures in two congenital muscular dystrophies, the ICC was 0.92 for the MFM total score and 0.94, 0.90 and 0.78 for D1, D2 and D3, respectively.²⁶

A shorter version of the MFM that includes 20 items from the original version, which is suitable for children under 7 years of age, has been validated.²⁷ However, this present study used the long version of the MFM scale (32 items) for the entire sample, which included five children less than 5 years of age. Although the long version of the MFM may not be well suited for young children due to its completion time and achievement of some items, no patient difficulties were observed during its implementation in the present study.

In the present study, some items showed better agreement than others. For instance, the inter-rater reliability for D1 (standing position and transfers) was better than for the other two domains. The D3 (distal limb motor function) domain obtained the lowest ICC value. The low reliability obtained for some of the items in this domain may be due to different observer perceptions. Using item 20 as an example, to obtain a high score the child must “tear the sheet of paper folded in 4”.¹ As was discussed with the observers after the test, these instructions may lead to misinterpretation as one observer interpreted that the sheet of paper had to be folded four times and the other observer interpreted that it has to be folded twice so a four squared sheet of paper was obtained. Therefore, the score of the item depends on the interpretation of the observer rather than on the setting where the test took place.

Ideally, for accurate objective measurements, standardization must be part the evaluation process and recommendations suggest that the MFM scale should be administered using standard equipment that is often available in a physiotherapy clinic.¹ The present study, besides the potential for patient and rater variability, introduced environmental variability. Indeed, although the patients lived in the urban area of Bogota, the conditions inside the houses varied for home to home. On some occasions, the space for assessment was small and the availability of a mat and height-adjustable chair, which are required for some items of the MFM, was limited. However, even though the environment was variable, the observers tried to keep to most of the conditions that are recommended by the MFM user manual.¹ We are of the opinion that variability in the environment has little impact on the perception of the functional ability of patients with HNMD measured with the MFM. For example, for item 12 that requires adequate space and furniture, the weighted kappa (i.e. reliability) was excellent. This result reinforces our hypothesis that the scoring of the MFM items was independent of the environment where the test was administered.

For the patient with HNMD, functional assessment at home may assist follow-up, particularly in clinical trials. Many patients with HNMD have mobility issues and so attending clinics may be problematic and studies have shown that patients drop-out or are excluded from clinical trials because of transportation issues.²⁸ Additionally, some families have observed that the child’s willingness to co-operate with a functional evaluation was better in a familiar setting compared with a hospital environment.¹⁶ In fact, some studies have proposed that, in patients with neuromuscular diseases, a home-based functional assessment may improve adherence to experimental treatments.^16,17 Therefore, reliable, functional examination tools in the clinical, as well as home-based environments would be invaluable.

This present study found moderate to high correlations between the home-based MFM domains and the PODCI domains. In the context of disability, these correlations suggest a relationship between capability (i.e. clinical tests for assessing activities in a home-based controlled environment) and performance (i.e. daily activities assessed using a questionnaire).¹¹ We believe that the high correlation between the PODCI and the home-based MFM evaluation shows the importance of families’ perception of the children’s performance in everyday environments.

The PODCI is a composite scale that explores items related to quality of life.¹⁸ One of the aims of this present study was to assess the correlation of MFM scale scores with three domains of the PODCI, including the ‘happiness’ domain. The evaluation of happiness of a child is imperative, especially in advanced stages of the disease when children require motorized systems for mobility and ventilation support.²⁹ Moreover, survival of children with DMD has improved in recent years due to interventions with noninvasive ventilation, leading to concerns about ethics, happiness and quality of life.^29,30 The present study found a poor correlation between MFM scores and the PODCI domain of ‘happiness’. We believe that the weak correlation may be related to the possibility that the PODCI ‘happiness’ domain explores issues for children that do not depend on the severity of their motor disability. In addition, perhaps the five questions in that domain are merely an approximation to the construct of happiness or maybe they were difficult to understand for a child with DMD. Moreover, the construct of happiness may not be easily assessed in patients with neuromuscular diseases.^30,31

The present study had some limitations. For instance, the sample size was small and most of the children had DMD. Most of the remaining patients had proximal weakness (i.e. dystrophies and SMA). Therefore, because the sample did not include other pathologies, generalizations from these present results cannot be made. Ideally, reliability assessments and correlations between scales should be established for each separate neuromuscular disease since the distribution of muscular weakness, contractures, and compensations are different. Another limitation of the study was that bias may have been introduced because parents and not the clinical assessor answered some of the questions in the PODCI. Further studies are warranted to confirm these current results.

In conclusion, this present study showed that the MFM is a reliable tool for the assessment of patients with HNMD in a home-based environment as well as a hospital-based setting and this may well ease the burden on patients and their families. The present study found that children’s perception of happiness did not correlate with our functional measurements and this reinforces the need for a better understanding of the construct of happiness in patients with HNMD.

Declaration of Conflicting Interests

The authors declare that there are no conflicts of interest.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Publication of this paper was funded by Vitalchem Laboratories de Colombia S.A.