Determining the Validity of Conducting Rating Scales in Friedreich Ataxia through Video

ABSTRACT

Background

The Friedreich Ataxia Rating Scale (FARS) and the Scale for the Assessment and Rating of Ataxia (SARA) are commonly used neurological rating scales in Friedreich ataxia (FRDA). The modified Friedreich Ataxia Rating Scale (mFARS) has been accepted as an appropriate outcome measure for clinical trials in FRDA.

Objectives

The COVID‐19 pandemic has resulted in limited face‐to‐face interactions with individuals involved in natural history studies and clinical trials. The aim of this study was to determine the validity of conducting the mFARS and SARA through video.

Methods

Individuals who had the mFARS administered face‐to‐face in the previous 6 months were invited to participate. Participants were sent instructions and asked to have a carer present to assist. The mFARS and SARA were then administered by video. Differences between face‐to‐face and video scores and the reliability between scores obtained face‐to‐face and by video were examined.

Results

The mFARS and SARA were conducted by video with 19 individuals. Excellent test–retest reliability was seen in the mFARS lower limb coordination (ICC = 0.96, 95% CI 0.90–0.98) and upright stability sections (ICC = 0.97, 95% CI 0.93–0.99), total mFARS (ICC = 0.97, 95% CI 0.92–0.99) and SARA scores (ICC = 0.98, 95% CI 0.95–0.99).

Conclusions

Excellent test–retest reliability was demonstrated in the majority of the mFARS sections, and in the total mFARS and SARA scores, suggesting that video is a valid method of conducting these scales. This method enables inclusion of participants who are unable to travel to study sites. A larger cohort will be required to further validate the use of video mFARS and SARA for future studies.

Friedreich ataxia (FRDA) is the most common of the inherited ataxias, affecting 1 in 29,000 individuals. ¹ FRDA is caused by a homozygous GAA triplet repeat expansion in intron 1 of the FXN gene in 96% of affected individuals. The remaining individuals are compound heterozygous for a GAA expansion in one allele and a point mutation, deletion or insertion in the other allele. ² , ³ Characteristics of FRDA include gait and limb ataxia, scoliosis, dysarthria and cardiomyopathy, with the average age of onset being between the ages of 10 and 15. ⁴

While there is currently no approved disease modifying treatment for FRDA, there are many clinical trials in progress studying potential therapeutic candidates. It is therefore crucial that the outcome measures used in these trials are accurate and sensitive. Clinician administered neurological rating scales can be used to measure disease progression in FRDA and are commonly used as primary endpoints in clinical trials. Of these rating scales, the Friedreich Ataxia Rating Scale (FARS) is the most widely used. ⁵ The FARS is an outcome measure in the Friedreich Ataxia Clinical Outcome Measures Study (FA‐COMS), which is a large natural history study comprising over 1000 participants from sites in the United States, Canada, Australia and New Zealand. ⁶ , ⁷ The FARS has good inter‐rater reliability ⁵ and consists of three components: functional staging of ataxia, a measure of activities of daily living, and a neurological examination. ⁸

The modified Friedreich Ataxia Rating Scale (mFARS) is derived from the neurologic component of the FARS and consists of four sections: bulbar, upper limb coordination, lower limb coordination and upright stability. ⁷ The overall validity and structure of the mFARS ⁷ as well as its excellent test–retest reliability ⁹ have been confirmed. The mFARS has recently been accepted by the US Food and Drug Administration (FDA) as an appropriate primary outcome measure for clinical trials in FRDA. ¹⁰

The Scale for the Assessment and Rating of Ataxia (SARA) is another widely used rating scale for measuring FRDA. ¹¹ The SARA consists of eight items assessing gait, stance, sitting, speech and limb kinetic function. ¹¹ The SARA is used as a primary outcome measure in the European Friedreich's Ataxia Consortium for Translational Studies (EFACTS), ¹² , ¹³ and in various clinical trials, including the study of the Efficacy and Safety of Nicotinamide in Patients With Friedreich Ataxia (NICOFA). ¹⁴

The global coronavirus (COVID‐19) pandemic has resulted in limited face‐to‐face interactions with individuals involved in natural history studies and clinical trials, with these visits often being conducted either through telephone or by video. Despite the presence of the COVID‐19 pandemic, it is important that clinicians consider alternative methods to ensure the ongoing collection of natural history data and retention of participants in clinical trials. There are no published studies that have examined the validity or feasibility of conducting clinician administered rating scales in the FRDA population remotely, and therefore the aim of this study was to determine the validity of conducting the mFARS and the SARA through video.

Methods

Individuals with genetically confirmed FRDA who had completed a face‐to‐face mFARS assessment, either as part of a pre‐existing natural history study or a clinical trial, within the 6 months prior to video assessment were invited to participate.

Prior to the assessment, participants were sent digital video files with instructions and asked to have at least one person present to assist with the assessment. Participants were sent consent forms to complete via email, and informed consent was obtained before the video assessments were scheduled. Participants were required to have access to a computer, tablet, laptop or smartphone with camera and microphone capabilities. Participants were asked to be barefoot and wear shorts or pants that could be easily pulled up to above the knees. They were asked to have a solid chair, preferably a kitchen chair, instead of a couch or a cushioned seat, to sit on and asked to have another chair close by to put their feet on for the lower limb section. The assessments were completed over a video conferencing platform in real time and assessed by the same rater (MBD), who was blinded to the previous scores. Assessments were only performed if the video quality was judged to be adequate by the rater.

During the video assessment, participants completed items from the mFARS and SARA examinations. For the bulbar section, participants completed the cough and speech items with verbal instructions from the rater over video.

For the upper limb section, the assisting person was instructed to imitate the rater's hand movements to complete the nose‐finger, and dysmetria items. The finger to finger, rapid alternating movements of hands and finger tap items were completed by the participant with verbal instructions from the rater over video.

The lower limb coordination section was completed with participants seated on a solid chair with their feet placed on a separate chair. Participants followed verbal instructions by the rater to complete the heel along shin slide and heel to shin tap items.

For the upright stability section, participants completed the sitting item with verbal instructions from the rater. Ambulant participants who were able to complete the standing items (feet together, eyes open and closed; feet apart, eyes open and closed; tandem stance; stance on dominant foot) could obtain assistance to get into position for the tasks, which were then timed and assessed by the rater. The assisting person was asked to stand next to the participant during these items to ensure the participant's safety as well as to make sure the assessments were completed correctly. The tandem walk and gait items were performed with verbal instructions from the rater if the participant was able to complete these tasks, with the assisting person close by for safety.

The SARA items which were similar to the mFARS items were scored concurrently (gait, stance, sitting, speech, nose‐finger test, fast alternating hand movements, heel‐shin slide) with the mFARS. For the finger chase task, the assisting person was instructed to imitate the rater's hand movements, with the participant following the movements. This item was then assessed by the rater.

The mFARS is scored out of a total of 93, and the SARA out of 40. Higher scores in both scales indicate more severe disease. All face‐to‐face assessments were conducted prior to the video assessments.

Descriptive analyses were used to describe the characteristics of the participants as well as the mFARS and SARA scores obtained at face‐to‐face and video assessments. Paired t‐tests were used to examine the differences between the face‐to‐face and video assessment scores for the bulbar and upper limb coordination sections, and Wilcoxon signed‐rank tests were used for the remaining sections, total mFARS and SARA scores as these were found to be not normally distributed. In previous face‐to‐face assessments, the SARA was not always completed at the same time as the mFARS, hence the participants' most recent face‐to‐face SARA score was used for comparison to the video SARA scores, and these assessments were approximately 12 months apart. Intraclass correlation coefficient (ICC) was used to examine test retest reliability between mFARS and SARA scores obtained face‐to‐face and by video. Bland–Altman plots were also generated, and 95% limits of agreement presented.

Statistical analyses were performed using Stata Statistical Software: Release 16 (StataCorp. 2019. College Station, TX: StataCorp LLC). Ethics Committee approval for this study was obtained through the Monash Health Human Research Ethics Committee (HREC 02114A).

Results

The mFARS and SARA were conducted by video with 19 individuals with FRDA. Participant characteristics at the time of the video assessments are shown in Table 1.

TABLE 1.

Participant characteristics at video assessments (n = 19)

Female gender (%)	11 (57.9)
Age (yr), mean (SD), range	36.9 (15.2), 14–64
Onset age (yr), mean (SD), range	19.2 (10.5), 7–40
Disease duration (yr), mean (SD), range	17.7 (10.4), 3–48
Time from face to face to video assessment for mFARS (days), mean (SD), range	129.4 (28.0), 87–174
Time from face to face to video assessment for SARA (days), mean (SD), range	262 (129.1), 98–489

Abbreviations: SD, standard deviation; mFARS, Modified Friedreich Ataxia Rating Scale; SARA, Scale for the Assessment and Rating of Ataxia.

The mean face‐to‐face and video assessment scores for each section of the mFARS as well as the total mFARS and SARA scores are presented in Table 2. Significant differences were demonstrated between face‐to‐face and video assessments for the upper limb coordination section (−2.95, P < 0.01) and the total mFARS score (Z = −2.40, P = 0.01). There were no significant differences between face‐to‐face and video scores in the other mFARS sections or the SARA.

TABLE 2.

Differences between face to face and video assessments (n = 19)

	Face to face assessment	Video assessment	Difference between face to face and video assessments
Bulbar, mean (SD), range	0.89 (0.46), 0–2	0.84 (0.37), 0–1	0.05 (0.52), P = 0.67
Upper limb coordination, mean (SD), range	11.37 (5.72), 0–21	14.32 (5.94), 0–22	−2.95 (3.15), P < 0.01
Lower limb coordination, median, range	9, 2–16	10, 2–16	Z = −1.68, P = 0.10
Upright stability, median, range	27, 17–34	25, 15.7–34	Z = 1.82, P = 0.07
Total mFARS score, median, range	54, 21–68.5	53, 26.7–73	Z = ‐2.40, P = 0.01
Total SARA score, median, range	(n = 17) 15.5, 7–31	(n = 19) 18.5, 7.5–31.5	(n = 17) Z = −1.31, P = 0.20

Abbreviations: SD, standard deviation; mFARS, Modified Friedreich Ataxia Rating Scale; SARA, Scale for the Assessment and Rating of Ataxia.

Intraclass correlation coefficient evaluated test retest reliability (Table 3): values ≥0.90 are generally considered excellent reliability, and this is shown for total mFARS and SARA scores, as well as the lower limb and upright stability subscales of the mFARS. The bulbar and upper limb subscales demonstrated poor and good reliability respectively. Bland–Altman plots depicting the difference in scores between face‐to‐face and video assessments are shown in Fig. 1. In these plots, the differences between the face‐to‐face and video scores are plotted on the y‐axis, against the mean of each pair on the x‐axis. This difference is calculated as: (video assessments) – (face‐to‐face assessments). The 95% limits of agreement are represented between the dotted horizontal lines and summarized in Table 4.

TABLE 3.

Intraclass correlation coefficients between face‐to‐face and video assessments (n = 19)

Bulbar	0.38 95% CI: −0.57, 0.76
Upper limb coordination	0.86 95% CI: 0.63, 0.94
Lower limb coordination	0.96 95% CI: 0.90, 0.98
Upright stability	0.97 95% CI: 0.93, 0.99
Total mFARS score	0.97 95% CI: 0.92, 0.99
Total SARA score (n = 17)	0.98 95% CI: 0.95, 0.99

Abbreviations: mFARS, Modified Friedreich Ataxia Rating Scale; SARA, Scale for the Assessment and Rating of Ataxia.

FIG. 1.

Plots are shown for bulbar, upper limb coordination, lower limb coordination and upper stability sections, and total mFARS and SARA scores. The dotted horizontal lines represent the 95% limits of agreement.

TABLE 4.

95% limits of agreement between face‐to‐face and video assessments

Bulbar	−1.068, 0.8331
Upper limb coordination	−3.263, 8.322
Lower limb coordination	−3.011, 4.422
Upright stability	−4.813, 4.095
Total mFARS score	−5.856, 11.26
Total SARA score (n = 17)	−3.38, 5.647

Abbreviations: mFARS, Modified Friedreich Ataxia Rating Scale; SARA, Scale for the Assessment and Rating of Ataxia.

Discussion

This study sought to determine the validity of conducting the mFARS and the SARA, two commonly used neurological rating scales, through video. Good to excellent test–retest reliability were demonstrated in the majority of the mFARS sections, and most importantly, in the total mFARS and SARA scores, suggesting that video is a valid method of conducting the mFARS and the SARA. Poor reliability was shown in the bulbar section, and this could be attributed to the small sample size in this study.

The differences between the face‐to‐face and video scores of the mFARS and the SARA were examined to determine if the change captured over the time period was reflective of natural history. Significant differences in the scores were not expected based on previous natural history data ¹⁵ as well as a recent study of test–retest reliability of the mFARS showing no meaningful change between scores obtained between the first and second timepoint (ICC 0.95). ⁹ However, in this study, significant differences were demonstrated in the upper limb coordination section, which was subsequently reflected in the total mFARS score. This finding could be explained by the challenge of assessing fine finger movements via video in a subscale that is prone to variability. While overall video quality was judged to be sufficient by the rater, the quality did vary depending on the connection on the day of the assessment, which may have affected the rating of some of the items. The duration between visits could have also been a factor – the mean time was 129.4 days (range 87–124 days) between visits in this study, compared to the test–retest reliability study, ⁹ where the median time between visits was 42 days (range 26–97 days). In addition, more than one rater was involved in administering face‐to‐face mFARS assessments, which could have contributed to this result. The small sample size in this study could also have played a role. Another limitation of this study is the extended time period between the face‐to‐face and video SARA assessments, as functional deterioration in people with FRDA is expected during this time period. This was due to the timing of the face‐to‐face SARA assessments, as they were not always completed at the same time as the face‐to‐face mFARS. The participants' most recent face‐to‐face SARA score was used for comparison to the video SARA scores which meant that these assessments were up to 12 months apart.

Both the SARA and the mFARS demonstrate similar sensitivity to measuring disease progression in large natural history studies, and show comparable results for power calculations in the context of clinical trials. ¹⁶ There are a number of advantages of using the SARA; the scale is quicker to administer, is more user friendly, and has excellent interrater variability. ¹⁷ However, the mFARS arguably provides a more comprehensive assessment of an individual's functional status, with its sub‐sections enabling specific domains of FRDA to be studied separately. ⁷

Participant retention is challenging in natural history studies. In a large study of 812 subjects evaluating the predictors of disease progression in FRDA, a retention rate of 65% at Year 1 was reported. By Year 5, the retention rate was 52%. ⁶ Disease progression and travel difficulty due to long distances as well as time or financial restrictions, are often cited as factors preventing participants from returning to study sites. Conducting these rating scales remotely enables the inclusion of participants who would otherwise be lost to follow up. Selecting a natural history cohort that is representative of the general disease population is another limitation in rare disease research, ⁶ and this study has demonstrated that participants at various stages of disease progression (mFARS scores range from 21 to 68.5, with ages from 14 to 64) are able to be included.

The COVID‐19 pandemic has meant that face‐to‐face assessment of participants in clinical trials has not been possible in many countries. The data presented in this study means that the mFARS and SARA can be administered and used to assess subjects enrolled in clinical trials during this pandemic.

Recently, a video‐based assessment of the SARA has been developed, ¹⁸ indicating the need for measures that are able to be performed remotely. The SARA^home enables items of the SARA to be performed independently by participants at home. Performance of these items are recorded on a tablet or smartphone camera which are then evaluated by an examiner at a later date. A validation study found that video rating of the SARA^home highly correlated with the conventional complete SARA score (r = 0.925, P < 0.0001). Feasibility of the SARA^home has also been demonstrated in a pilot study of home recordings in 12 ataxia patients.

A limitation of this study was the relatively small sample size. A larger cohort will be required to further validate the use of video mFARS and SARA for future studies.

Author Roles

(1) Research project: A. Conception, B. Organization, C. Execution; (2) Statistical Analysis: A. Design, B. Execution, C. Review and Critique; (3) Manuscript: A. Writing of the first draft, B. Review and Critique.

G.T.: 1B, 2A, 2B, 3A

E.M.Y.: 1C, 2C, 3B

I.R.W.: 1C, 3B

L.A.C.: 1A, 1B, 2A, 2C, 3B

M.B.D.: 1A, 1B, 1C, 2C, 3B

Disclosures

Ethical Compliance Statement

Ethics Committee approval for this study was obtained through the Monash Health Human Research Ethics Committee (HREC 02114A). Participants were sent consent forms to complete via email, and informed consent was obtained before the video assessments were scheduled and completed. We confirm that we have read the Journal's position on issues involved in ethical publication and affirm that this work is consistent with those guidelines.

Funding Sources and Conflict of Interest

The Friedreich's Ataxia Research Alliance (USA) and the Murdoch Children's Research Institute provide ongoing financial support. LAC is supported by a Medical Research Future Fund Next Generation Career Development Fellowship under Grant APP1143098. The authors declare that there are no conflicts of interest relevant to this work.

Financial Disclosures of All Authors for Previous 12 Months

The authors declare that there are no additional disclosures to report.