Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2022 Dec 1.
Published in final edited form as: Pediatr Neurol. 2021 Sep 24;125:34–39. doi: 10.1016/j.pediatrneurol.2021.09.012

Reliability of the telemedicine application of the Gross Motor Function Measure-88 in patients with leukodystrophy

Francesco Gavazzi 1,2, Laura Adang 1,3, Amy Waldman 1, Amanda K Jan 1, Geraldine Liu 1, Scott A Lorch 4, Sara B DeMauro 4, Justine Shults 5,6, Samuel R Pierce 7, Elizabeth Ballance 7, Tracy Kornafel 7, Ann Harrington 7, Allan M Glanzman 7,*, Adeline Vanderver 1,3,*
PMCID: PMC8629609  NIHMSID: NIHMS1742910  PMID: 34624609

Abstract

Objective:

Leukodystrophies are a rare class of disorders characterized by severe neuromotor disability. There is a strong need for research regarding the functional status of people with leukodystrophy which is limited by the need for in person assessments of mobility. The purpose of this study is to assess the reliability of the Gross Motor Function Measure-88 (GMFM-88) using telemedicine compared to standard in-person assessments in patients with leukodystrophy.

Methods:

A total of 21 subjects with a diagnosis of leukodystrophy (age range: 1.79 – 52.82 years) were evaluated by in-person and by telemedicine evaluations with the GMFM-88 by physical therapists. Interrater reliability was assessed through evaluation of the same subject by two independent raters within a 3-week period (n=10 encounters), and intrarater reliability was assessed through blinded rescoring of videorecorded assessments after a 1-week time interval (n=6 encounters).

Results:

Remote assessments were performed by caregivers in all 21 subjects using resources found in the home with remote guidance. There was agreement between all paired in-person and remote measurements (Lin’s concordance correlation ≥ 0.995). Bland Altman analysis indicated that the paired differences were within ± 5%. Intra-rater and inter-rater reliability demonstrated an intra-class correlation coefficient of > 0.90.

Conclusions:

These results support that remote application of GMFM-88 is a feasible and reliable approach to assess individuals with leukodystrophy. Telemedicine application of outcome measures may be of particular value in rare diseases and those with severe neurologic disability that impacts the ability to travel.

Keywords: child development/physiology, child, motor skills/physiology, outcome assessment (health care), physical therapy/methods, psychometrics/methods, leukodystrophy, reliability

INTRODUCTION

Leukodystrophies are a broad class of genetic disorders that affect the myelin of the brain which often result in severe neurologic disability in early childhood. Children with leukodystrophies are noted to have a low health-related quality of life,1–4 and prolonged hospitalizations that cost over $59 million per year.2 There is a need to lessen the burden of disease through improved symptomatic care and targeted therapeutics.5–15 One barrier to the development of novel therapies for leukodystrophy is the lack of comprehensive natural history studies describing the functional abilities of patients with the disorder. Standardized assessments of motor function will likely be a key outcome in future natural history studies and future clinical trials.16 Motor outcomes that are valid and reliable are critical for monitoring the change in neurologic function over time.

Because of the rarity of leukodystrophy, enrollment in natural history studies via traditional in-person assessments may require travel to a specialty medical center, time off work for caregivers, and transportation of medically fragile individuals which may limit enrollment.17,18 The ability to capture motor function in a more natural environment without the fatigue or risks of travel may prove to be a powerful approach to more accurate and frequent outcome assessments. One potentially useful tool to assess motor function which may avoid these challenges is the use of remote assessments via telemedicine. Remote assessments would allow for a broader and more diverse population to participate in research, including people who live in long-term care facilities and isolated rural areas19, but few outcome assessments are validated for use with telemedicine approaches.

The field of child neurology has adapted rapidly to the ongoing COVID-19 pandemic by increasing the use of telemedicine encounters to address the needs of patients.20 However, motor assessments via telemedicine has been mostly limited to adults with neurological conditions which largely focus on measures of upper extremity function21 or walking22. Other approaches to assess motor function using telemedicine require specialized equipment23 which limits its broad use in clinical practice. Camden and Silva (2020) report that therapists value telehealth evaluations while noting that use of norm-referenced and criterion-referenced assessments in telehealth have been not implemented due to a lack of information on how to complete these tests remotely.

One commonly used outcome measure that may be adaptable to the telehealth environment is the Gross Motor Function Measure-88 (GMFM-88), which was originally developed to assess children with cerebral palsy.24 The GMFM-88 is divided into 5 subdomains: subdomain A (lying and rolling skills), subdomain B (sitting skills), subdomain C (crawling and kneeling skills), subdomain D (standing skills), and subdomain E (walking, running, and jumping skills). The GMFM-88 requires equipment which is easily available in clinical practice and no special technology so that it’s use in telehealth may be promising. In addition, the use of the GMFM-88 has been recently reported in patients with leukodystrophy.8,25–26 In our previous work, we reported a significant correlation between the GMFM-88 and a disease specific scale for children with Aicardi Goutières Syndrome (AGS).25 However, the reliability of the GMFM-88 has not been reported in patients with leukodystrophy and the assessment of the GMFM-88 via telemedicine has not been described to date. In this study, we sought to assess the reliability of the GMFM-88 using telemedicine compared to standard in-person assessments in patients with leukodystrophy.

METHODS

All subjects were recruited from the IRB-approved protocol, Myelin Database and Biorepository Project (MDBP) at the Children’s Hospital of Philadelphia. Participants were included if they had completed the GMFM-88 during an in-person visit occurred after January 1st, 2020. The family of two patients refused to participate to this project. All subjects carried a molecularly-confirmed leukodystrophy diagnosis. A total of 21 subjects (age range 1.3 – 52.5 years) were evaluated during sequential in-person and telemedicine encounters. The subject’s diagnosis, age at symptom onset, and disease severity were determined by medical records review, as well as all GMFM-88 administrations available before the year 2020.

All in-person GMFM-88 have been administered by physical therapists who had extensive experience using the GMFM-88 and followed the instructions of the GMFM-88 manual.27 The remote assessments were performed by two trained physical therapists that administered and scored the children live, having the patients’ parents assisting their children. One adult subject was evaluated without the assistance of any caregiver. All assessments were performed without the use of orthotics.

The families participating in this study were provided with information prior to the research encounter regarding the necessary environment and technology needed for home evaluations (Supplemental Table 1). Sessions were conducted over a secure video conferencing platform (Cisco WebEx). Existing home equipment, including typical home furniture, was selected to be as close as possible to the standard equipment and for subsequent evaluations and logged for consistency, in accordance with the GMFM-88 manual.27 All 88 items were administered during the virtual encounter, and time needed for the assessment was similar to in-person assessment. Session were recorded with a fixed camera angle.

Composite scores and were calculated and compared between assessments. We assessed inter-rater reliability by administering the GMFM remotely to 10 patients of the original cohort at an average interval of 2 weeks by 2 different therapists. Six of the 10 subjects’ telemedicine visits were recorded for reliability assessments. Intra-rater reliability was assessed using these video-recorded assessments by measuring the degree of agreement between two separate scorings of the same video administration after a 1-week time interval (range 6–8 days).

Statistics

Stata 1628 was used for analysis with two-sided tests and a p-value < 0.05 as the criterion for statistical significance. Descriptive and correlative statistics were used to compare the assessments which were done in person to the remote assessments. Descriptive statistics included tabulation of categorical variables and computation of means, standard deviations (SD), medians, ranges (minimum to maximum values), interquartile-ranges (IQR, median to 75th percentile) of continuous variables.

To evaluate agreement, we used the concordance correlation coefficient (ρc)29,30 that evaluates the degree to which pairs of observations fall on the 45° line through the origin (the line of perfect concordance). The concordance correlation ρc is the product of the Pearson correlation and a bias correction factor. McBride (2005)30 suggested that ρc<0.90 indicates poor agreement, while 0.90 – 0.95; 0.95 – 0.99; and > 0.99 suggest moderate, substantial and almost perfect levels of agreement, respectively. We used the bootcor command in Stata 16.0 to obtain ρc, with 95% confidence intervals (CI) obtained via bootstrap sampling. Bootstrap resampling does not require an assumption of bivariate normality for the two measures being compared and can also be applied if the samples are not independent. We used the concord command in Stata to obtain a scatter plot of values with the line of perfect concordance.

We also used Bland-Altman analysis to evaluate agreement. We used the agree command in Stata to plot the difference versus average of remote and in-person GMFM-88, with a horizontal line at the average difference and the upper and lower limits of agreement. We also obtained 95% CI for the average difference and upper and lower limits of agreement using the agree command. As suggested in Zagreb31 acceptable values for the limits of agreement (within which 95% of differences between measurements would be expected to fall) should be established a priori. Bland-Altman analysis assumes normality of the differences. We evaluated the normality assumption of individual variables and paired differences using diagnostic plots (histograms and plots of quantiles of the variable under consideration versus quantiles of the normal distribution [Q-Q plot]) and Shapiro-Wilk and tests for skewness and kurtosis with the agree command.

There were 21 subjects with both in-person and remote assessments of GMFM-88. Of these 21 subjects, 4 (4/21, 19%) had 1 prior in-person assessments; 5 (5/21, 23.8%) had 2 prior in-person assessments; 1 (1/21, 4.8%) had 3 prior in-person assessments; and 11 (11/21, 52.4%) had 4 prior in-person assessments. Remote and in-person assessments were compared by individual level profile plots for each patient. Individual level regression lines of score versus age were fitted to the previous scores on each patient and were used to obtain a predicted value at the age of remote assessment. The agreement between the remote and predicted values from the individual level regressions was then evaluated using the same approaches that were used to evaluate agreement between the remote and prior value. This analysis was considered to be exploratory and descriptive only.

To evaluate inter-rater reliability we estimated the intra-class correlation coefficient (ICC) for a two-way mixed effects model. We used the bootcor command to use bootstrap resampling to obtain the 95% CI for the ICC because the distribution of the scores deviated from normality. To evaluate intra-rater reliability we also estimated the ICC (with 95% CI) using the bootcor command in Stata 16.0.

RESULTS

Demographics

A total of 21 subjects with leukodystrophy were enrolled in this study. Nine of these subjects were male (42.9%). The mean age at the last in-person evaluation prior to the remote evaluation was of 9.6 years (range 1.3–52.5, SD 11.0). The mean age at the telemedicine assessment was 10.1 years (range 1.8–52.8, SD 11.0). The mean difference between age at telemedicine assessment and age at baseline assessment was 0.5 years (range 0.4–0.7, SD 0.1). Of the 21 subjects, 17 individuals had a confirmed diagnosis of AGS (81%), 2 had a diagnosis of POLR3-related leukodystrophy (9.5%), 1 had a diagnosis of Alexander Disease (AxD) (4.7%), and 1 had a diagnosis of TUBB4A-related leukodystrophy (4.7%).

GMFM-88 evaluations

We first evaluated the agreement between GMFM-88 remote administration total score and the closest in-person GMFM-88 total score. Diagnostic plots (Q-Q plot and histogram) and tests for normality suggested that the distribution of GMFM-88 remote and in-person scores deviated from normality; however, the normality assumption was reasonable for the paired differences (GMFM-88 minus remote scores). The mean difference was 0.2 (95% CI = −0.9 to 1.4; lower limit of agreement = −4.8 [95% CI = −6.8 to −2.8]; upper limit of agreement = 5.3 [95% CI = 3.3 to 7.3]; Lin’s concordance correlation coefficient = 0.997 [95% CI = 0.993 to 0.998]. (Figure 1 A and figure 1B).

Figure 1:

Figure 1:

Open in a new tab

A. Scatter plot of GMFM-88 in-person and remote administration (total score) with the line of perfect concordance. Lin’s concordance correlation ρc (with 95% CI) is also provided (ρc = 0.997 [95% CI = 0.993 to 0.998]). B. The Bland-Altman plot displays the variance between remote and in-person assessments. The lines represent ± 5% variation (upper and lower limits of agreement for the paired differences).

We next compared the GMFM-88 remote administration raw score with the predicted values from the individual level regressions using all previous in-person assessments (n=17 individuals had at least two prior assessments, n=12 individuals had 3 or more assessments). Lin’s concordance coefficient for GMFM-88 Remote versus predicted scores was 0.994 (95% CI = 0.975 to 0.998) (Supplemental Figure 1A). The evaluation of the Bland Altman plot visualized agreement (Supplemental Figure 1B). The average difference (remote - predicted GMFM-88) was 0.3 (95% CI = −1.5 to 2.0). The lower limit of agreement was −6.5 (95% CI = −9.5 to −3.4) and the upper limit of agreement was 7.0 (95% CI = 3.9 to 10.0). Individual level profile plots of the prior scores that excluded and included the remote assessment on each patient are shown in Figure 2A and 2B.

Figure 2:

Figure 2:

Open in a new tab

A. Profile Plots Excluding Remote Assessment of GMFM-88 B. Profile Plots Including Remote Assessment of GMFM-88. Time at remote assessment is 0.

Interrater reliability was assessed through remote evaluation of the same subject by two independent raters within a 3-week period (n=20 encounters for n=10 subjects). The estimated ICC (95% CI) was 0.996 (95% CI = 0.964 to 0.999). Agreement between raters was determined using Lin’s concordance coefficient and Bland Altman plots (Supplemental Figure 2A-B). Concordance between raters was 0.995 (95% CI = 0.961 to 0.999). The average difference between rater scores was −1.7 (95% CI = −3.9 to 0.5). The lower limit of agreement was −7.8 (95% CI = −11.6 to −4.0) and the upper limit of agreement was 4.3 (95% CI = 0.5 to 8.1).

Intra-rater reliability was assessed by having one clinician score the same encounter twice via a blinded scoring of a video recording of the encounter (n=6 subjects; ICC = 0.999; 95% CI = 0.996 to 1.000). Agreement was determined using Lin’s concordance coefficient and Bland Altman plots (Supplemental Figure 3A-B). With respect to agreement, Lin’s concordance coefficient for the scores on both raters was 0.999 (95% CI = 0.985 to 1.000). The average difference between second and first scores was 0.2 (95% CI = −1.1 to 1.5). The lower limit of agreement was −2.3 (95% CI = −4.5 to 0.0) and the upper limit of agreement was 2.7 (95% CI = 0.4 to 5.0).

DISCUSSION

Our results demonstrated agreement between the in-person and telemedicine assessments of the GMFM-88 while finding high levels of interrater and intrarater reliability as indicated by ICCs above 0.99. The population studied represents a broad range of function, with scores ranging across the spectrum of GMFM-88 scores so we would predict that this study may be relevant to most individuals with leukodystrophy. Future research is needed to confirm the reliability of both the in person and remote assessment of the GMFM-88 in a larger sample of subjects with leukodystrophy.

In this investigation, we sought to validate the use of the GMFM-88 via telemedicine more broadly in individuals affected by leukodystrophies, including AGS, POLR3-related leukodystrophy, AxD, and TUBB4A-related leukodystrophies. While many individuals affected by leukodystrophies demonstrate a floor effect on outcome measures due to the severity of neurologic dysfunction, the chosen population demonstrates a broad range of gross motor function.32,33 The leukodystrophies selected for this study tend to have a more static course compared to other leukodystrophies25,33–35 which needs to be confirmed by future natural history studies of motor function. This allows for the comparison of outcomes that were obtained asynchronously but it is unknown if our results may have been affected by the time period in between the in-person and remoted assessments.

One key challenge to understanding rare neurologic disorders is recruitment and bias towards including more mobile individuals.32 Enrollment in natural history studies may be biased against individuals who are have greater disease severity or who live at a greater distance from large academic medical centers. Our results suggest that the GMFM-88 may be a valid tool to assess function remotely which may allow for enrollment of a more diverse population and more frequent assessments in future natural history studies of longitudinal motor function. Future research investigating the remote assessments of other outcome measure of gross motor function as well as assessments of fine motor skills and activities of daily living would allow for more a complete assessment of the functional ability of individuals with leukodystrophy.

While these results are promising, this study has several limitations. Our cohort was limited in number due to rarity of occurrence of leukodystrophies and the ability of families to travel to a central site for in person assessments. However, the inability to recruit larger numbers of participants and to obtain regular study assessments in this population underscores the need to develop alternatives to in-person assessments. Additionally, inclusion in this study required baseline participation at a single centralized site with a high level of expertise with leukodystrophy and the collection of standardized outcome assessments. The ability to generalize this approach with other remote site facilities will need to be addressed in future studies. It is should be considered that technological limitations, such as unavailability of electronic devices, inconsistent internet connections, or other technical difficulties, which have been reported by Rametta et al (2020) during telehealth encounters with children, may make the remote application of the outcomes measures such as the GMFM impossible. The absence of the equipment required to complete the GMFM-88 might limit its application in some home environments. Finally, the use of caregivers to assist in the completion of the GMFM may be difficult for some caregivers, especially for older and more physically dependent children so that alternative assessments which are less physically demanding will need to be developed.

In summary, we validated the use of remote GMFM-88 application using comparison to baseline in-person data. As future therapeutic trials for the leukodystrophies become available, the remote GMFM-88 may serve as a valuable tool to supplement in-person traditional outcome evaluations.

Supplementary Material

1

Supplemental Figure 1: A. Scatter plot of GMFM-88 predicted and remote administration (total score) with the line of perfect concordance. Lin’s concordance correlation ρc (with 95% CI) is also provided (ρc = 0.994 [95% CI = 0.975 to 0.998]). B. The Bland-Altman plot to evaluate agreement between GMFM-88 predicted and remote administration (total score). The Bland-Altman plot displays a horizontal dashed line at 0 and a horizontal solid line at the average difference between the paired differences (between remote and in-person GMFM-88). Horizontal dashed lines are also provided at the upper and lower limits of agreement for the paired differences.

2

Supplemental Figure 2: A. Scatter plot of total scores from provider one versus provider two. Lin’s concordance correlation ρc (with 95% CI) is also provided (ρc = 0.995 [95% CI = 0.961 to 0.999]). B. Bland-Altman plot to evaluate agreement between the total scores on two raters. The Bland-Altman plot displays a horizontal dashed line at 0 and a horizontal solid line at the average difference between the paired differences (between rater one and rater two). Horizontal dashed lines are also provided at the upper and lower limits of agreement for the paired differences.

3

Supplemental Figure 3: A. Scatter plot of second and first measurements of GMFM-88 on each of 6 individuals with the line of perfect concordance. Lin’s concordance correlation ρc (with 95% CI) is also provided (ρc = [95% CI =]). B. The Bland-Altman plot to evaluate agreement between the first and second GMFM-88 total scores on each of six individuals who were each measured by the same rater. The Bland-Altman plot displays a horizontal dashed line at 0 and a horizontal solid line at the average difference between the paired differences (between the second and first measurement). Horizontal dashed lines are also provided at the upper and lower limits of agreement for the paired differences.

4

Acknowledgments

Conflict of interest disclosures

Dr. Gavazzi, Dr. Lorch, Dr. DeMauro, and Dr. Glanzman are supported by NIH, U01NS106845

Dr. Glanzman receives compensation for consulting for Roche and Biogen and licensing royalties related to the CHOP INTEND.

Dr. Adang is supported by NIH, U54NS115052, and U01NS106845

Dr. Waldman reports grants from NIH, U54NS115052, Pennsylvania Department of Health, and United Leukodystrophy Foundation. Dr. Waldman has also received research support for investigator-initiated research from Ionis Pharmaceuticals. She receives honoraria from UpToDate.

Dr. Harrington is supported by NIH, U54 NS115052

Dr. Shults is supported by NIH, U54NS115052. She receives research support from Eli Lily and Company, Homology, Takeda.

Dr. Pierce, A. Jan, T. Kornafel, G. Liu have no conflict of interest to disclose

Dr. Vanderver is supported by research funds from NIH, U54 NS115052, and U01NS106845, Eli Lilly, Biogen, Takeda, Homology, Passage Bio, Illumina, the PMD foundation, NINDS, NCATS and NICHD

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

REFERENCES

  • 1.Mirchi A, Pelletier F, Tran LT, et al. Health-Related Quality of Life for Patients With Genetically Determined Leukoencephalopathy. Pediatr Neurol 2018;84:21–6. [DOI] [PubMed] [Google Scholar]
  • 2.Brimley CJ, Lopez J, van Haren K, et al. National variation in costs and mortality for leukodystrophy patients in US children’s hospitals. Pediatric neurology 2013;49:156–62.e1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Nelson C, Mundorff MB, Korgenski EK, Brimley CJ, Srivastava R, Bonkowsky JL. Determinants of health care use in a population-based leukodystrophy cohort. J Pediatr 2013;162:624–8.e1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Bonkowsky JL, Nelson C, Kingston JL, Filloux FM, Mundorff MB, Srivastava R. The burden of inherited leukodystrophies in children. Neurology 2010;75:718–25. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Ahrens-Nicklas R, Schlotawa L, Ballabio A, et al. Complex care of individuals with multiple sulfatase deficiency: Clinical cases and consensus statement. Mol Genet Metab 2018;123:337–46. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Adang LA, Sherbini O, Ball L, et al. Revised consensus statement on the preventive and symptomatic care of patients with leukodystrophies. Molecular genetics and metabolism 2017;122:18–32. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Escolar ML, West T, Dallavecchia A, Poe MD, LaPoint K. Clinical management of Krabbe disease. Journal of neuroscience research 2016;94:1118–25. [DOI] [PubMed] [Google Scholar]
  • 8.Sessa M, Lorioli L, Fumagalli F, et al. Lentiviral haemopoietic stem-cell gene therapy in early-onset metachromatic leukodystrophy: an ad-hoc analysis of a non-randomised, open-label, phase 1/2 trial. Lancet 2016;388:476–87. [DOI] [PubMed] [Google Scholar]
  • 9.Groeschel S, Kuhl JS, Bley AE, et al. Long-term Outcome of Allogeneic Hematopoietic Stem Cell Transplantation in Patients With Juvenile Metachromatic Leukodystrophy Compared With Nontransplanted Control Patients. JAMA neurology 2016;73:1133–40. [DOI] [PubMed] [Google Scholar]
  • 10.Rosenberg JB, Kaminsky SM, Aubourg P, Crystal RG, Sondhi D. Gene therapy for metachromatic leukodystrophy. Journal of neuroscience research 2016;94:1169–79. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Van Haren K, Bonkowsky JL, Bernard G, et al. Consensus statement on preventive and symptomatic care of leukodystrophy patients. Mol Genet Metab 2015;114:516–26. [DOI] [PubMed] [Google Scholar]
  • 12.Boucher AA, Miller W, Shanley R, et al. Long-term outcomes after allogeneic hematopoietic stem cell transplantation for metachromatic leukodystrophy: the largest single-institution cohort report. Orphanet journal of rare diseases 2015;10:94. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Costello DJ, Eichler AF, Eichler FS. Leukodystrophies: classification, diagnosis, and treatment. The neurologist 2009;15:319–28. [DOI] [PubMed] [Google Scholar]
  • 14.Crow YJ, Vanderver A, Orcesi S, Kuijpers TW, Rice GI. Therapies in Aicardi-Goutieres syndrome. Clinical and experimental immunology 2014;175:1–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Meesilpavikkai K, Dik WA, Schrijver B, et al. Efficacy of baricitinib in the treatment of chilblains associated with the type I interferonopathy Aicardi-Goutieres syndrome. Arthritis Rheumatol 2019. [DOI] [PMC free article] [PubMed]
  • 16.Bunnell BE, Sprague G, Qanungo S, et al. An Exploration of Useful Telemedicine-Based Resources for Clinical Research. Telemed J E Health. 2020;26(1):51–65. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Camden C, Silva M. Pediatric Teleheath: Opportunities Created by the COVID-19 and Suggestions to Sustain Its Use to Support Families of Children with Disabilities. Phys Occup Ther Pediatr. 2021;41(1):1–17. [DOI] [PubMed] [Google Scholar]
  • 18.Utidjian L, Abramson E. Pediatric Telehealth: Opportunities and Challenges. Pediatr Clin North Am. 2016. Apr;63(2):367–78. [DOI] [PubMed] [Google Scholar]
  • 19.Wechsler LR, Tsao JW, Levine SR, et al. American Academy of Neurology Telemedicine Work Group. Teleneurology applications: Report of the Telemedicine Work Group of the American Academy of Neurology. Neurology. 2013. February 12;80(7):670–6. doi: 10.1212/WNL.0b013e3182823361. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Rametta SC, Fridinger SE, Gonzalez AK, et al. Analyzing 2,589 child neurology telehealth encounters necessitated by the COVID-19 pandemic. Neurology. 2020. September 1;95(9):e1257–e1266. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Veras M, Kairy D, Rogante M, Giacomozzi C, Saraiva S. Scoping review of outcome measures used in telerehabilitation and virtual reality for post-stroke rehabilitation. J Telemed Telecare. 2017. July;23(6):567–587. [DOI] [PubMed] [Google Scholar]
  • 22.Venkataraman K, Amis K, Landerman LR, Caves K, Koh GC, Hoenig H. Teleassessment of Gait and Gait Aids: Validity and Interrater Reliability. Phys Ther. 2020. April 17;100(4):708–717. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Jagos H, David V, Haller M, et al. A Framework for (Tele-) Monitoring of the Rehabilitation Progress in Stroke Patients: eHealth 2015 Special Issue. Appl Clin Inform. 2015. December 23;6(4):757–68. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Nordmark E, Hägglund G, Jarnlo GB. Reliability of the gross motor function measure in cerebral palsy. Scand J Rehabil Med. 1997. Mar;29(1):25–8 [PubMed] [Google Scholar]
  • 25.Adang LA, Gavazzi F, Jawad AF, et al. Development of a neurologic severity scale for Aicardi Goutières Syndrome. Mol Genet Metab 2020;130:153–60. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Beschle J, Döring M, Kehrer C, et al. Early clinical course after hematopoietic stem cell transplantation in children with juvenile metachromatic leukodystrophy. Mol Cell Pediatr 2020;7:12. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Russell DJ, Rosenbaum PL, Wright M, Avery LM. Gross Motor Function Measure (GMFM-66 and GMFM-88) User’s Manual. 2nd ed: Wiley; 2013 [Google Scholar]
  • 28.StataCorp. 2019. Stata Statistical Software: Release 16. College Station, TX: StataCorp LLC. [Google Scholar]
  • 29.Lin LI (1989) A Concordance Correlation Coefficient to Evaluate Reproducibility. Biometrics, 45, 255–268. [PubMed] [Google Scholar]
  • 30.McBride GB (2005) A proposal for strength-of-agreement criteria for Lin’s concordance… NIWA Client Report: HAM2005–062.
  • 31.Biochem Med (Zagreb). 2015. Jun; 25(2): 141–151. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Adang L, Gavazzi F, De Simone M, et al. Developmental Outcomes of Aicardi Goutières Syndrome. J Child Neurol 2020;35:7–16. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Gavazzi F, Charsar BA, Williams C, et al. Acquisition of Developmental Milestones in Hypomyelination With Atrophy of the Basal Ganglia and Cerebellum and Other TUBB4A-Related Leukoencephalopathy. J Child Neurol 2021:883073821000977. [DOI] [PMC free article] [PubMed]
  • 34.Prust M, Wang J, Morizono H, et al. GFAP mutations, age at onset, and clinical subtypes in Alexander disease. Neurology 2011;77:1287–94. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Gauquelin L, Cayami FK, Sztriha L, et al. Clinical spectrum of POLR3-related leukodystrophy caused by biallelic POLR1C pathogenic variants. Neurol Genet 2019;5:e369. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

1

Supplemental Figure 1: A. Scatter plot of GMFM-88 predicted and remote administration (total score) with the line of perfect concordance. Lin’s concordance correlation ρc (with 95% CI) is also provided (ρc = 0.994 [95% CI = 0.975 to 0.998]). B. The Bland-Altman plot to evaluate agreement between GMFM-88 predicted and remote administration (total score). The Bland-Altman plot displays a horizontal dashed line at 0 and a horizontal solid line at the average difference between the paired differences (between remote and in-person GMFM-88). Horizontal dashed lines are also provided at the upper and lower limits of agreement for the paired differences.

NIHMS1742910-supplement-1.pdf (33.4KB, pdf)
2

Supplemental Figure 2: A. Scatter plot of total scores from provider one versus provider two. Lin’s concordance correlation ρc (with 95% CI) is also provided (ρc = 0.995 [95% CI = 0.961 to 0.999]). B. Bland-Altman plot to evaluate agreement between the total scores on two raters. The Bland-Altman plot displays a horizontal dashed line at 0 and a horizontal solid line at the average difference between the paired differences (between rater one and rater two). Horizontal dashed lines are also provided at the upper and lower limits of agreement for the paired differences.

NIHMS1742910-supplement-2.pdf (30.2KB, pdf)
3

Supplemental Figure 3: A. Scatter plot of second and first measurements of GMFM-88 on each of 6 individuals with the line of perfect concordance. Lin’s concordance correlation ρc (with 95% CI) is also provided (ρc = [95% CI =]). B. The Bland-Altman plot to evaluate agreement between the first and second GMFM-88 total scores on each of six individuals who were each measured by the same rater. The Bland-Altman plot displays a horizontal dashed line at 0 and a horizontal solid line at the average difference between the paired differences (between the second and first measurement). Horizontal dashed lines are also provided at the upper and lower limits of agreement for the paired differences.

NIHMS1742910-supplement-3.pdf (29.3KB, pdf)
4
NIHMS1742910-supplement-4.docx (12.4KB, docx)

RESOURCES