Content Validity of the Spinocerebellar Ataxia Composite Score as a Measure of Disease Progression in Patients with Spinocerebellar Ataxia

Abstract

Spinocerebellar ataxia (SCA) composite score (SCACOMS) is a statistically-derived composite measure comprising weighted items that are sensitive to change during early-stage disease. SCACOMS items and weights include the functional Scale for the Assessment and Rating of Ataxia Gait (12%), Stance (17%), Sitting (8%), and Speech (10%) items, and the Clinician Global Impression of Change (CGI) (53%). The content validity of SCACOMS is yet to be established. Semi-structured qualitative interviews were conducted with individuals with SCA (N = 24) and healthcare professionals (HCPs) who treat SCA (N = 2) to evaluate the relevance and weighting of SCACOMS items for assessment of SCA progression. Interviews were audio recorded, transcribed, coded, and analyzed using ATLAS.Ti v23 software, following established methods. SCACOMS items measured all sign and symptom concepts reported by > 50.0% of individuals with SCA, including difficulties with speech (79.2%), balance (75.0%), and gait (66.7%). Of SCACOMS items, individuals with SCA ranked Gait as most important for assessing disease progression (45.8%) and CGI as least important (58.3%). When weighting SCACOMS items, individuals with SCA assigned the highest weight to Gait (mean [standard deviation], 32.8% [13.24]) and the lowest weight to Sitting (12.9% [7.98]). HCPs varied the item weights dependent on SCA severity. Agreement with the statistically-derived weighting of SCACOMS varied, with 41.6% of individuals with SCA indicating that CGI weighting was too high. Overall, all participants indicated that SCACOMS could detect meaningful changes and/or disease stabilization. This study supports the content validity of SCACOMS in SCA; however, SCACOMS item weights may warrant adjustment.

Supplementary Information

The online version contains supplementary material available at 10.1007/s12311-025-01896-x.

Introduction

Spinocerebellar ataxias (SCAs) are a heterogeneous group of rare, autosomal, dominantly inherited neurodegenerative disorders characterized by progressive degeneration of the cerebellum [1]. Although the approximately 50 SCA subtypes exhibit clinical heterogeneity, patients commonly present with gait ataxia and imbalance, incoordination of the limbs, visual disorders, and dysarthria [2, 3], which results in long-term disability [4]. These symptoms typically emerge in mid-adulthood (between 30 and 50 years of age) but may also appear during childhood or later in life (over 50 years of age), depending on the underlying genetic mutation [5]. Polyglutamine-encoding CAG repeat expansions cause the majority of the most common SCA subtypes (SCA1, SCA2, SCA3, and SCA6) that emerge in mid-adulthood [1]. The progression of SCA symptoms has been shown to impact patients’ activities of daily living (ADLs) [6], significantly affect health-related quality of life [7, 8], and increase the risk of early mortality [9, 10].

Clinical outcome assessments (COAs) have been developed to evaluate the signs, symptoms, and impacts of SCA across the disease trajectory, and to understand what may constitute clinically meaningful changes in response to a disease-modifying therapy [11]. Until recently, COAs had not typically developed content using criteria to promote optimal sensitivity for the detection of SCA disease progression [12]. This limitation may have inhibited the detection of small, yet clinically meaningful, changes in patient function during early-stage disease and/or within a study of limited timeframe (e.g., 1 year) [12].

To address the need for an adequately sensitive COA within a study of limited timeframe, SCA composite score (SCACOMS), a composite measure of disease progression for detecting changes in patients with early-stage SCA, was developed [13]. SCACOMS was derived using 2 natural history datasets to examine patient disease progression terms of individual item scores from a set of existing validated measures [14, 15, 16]. Using partial least squares regression modeling, COA items were selected for inclusion on the SCACOMS instrument based on observed responsiveness, with weights assigned to each item based on the item’s relative sensitivity for measuring disease progression [12]. During SCACOMS development, the Clinician Global Impression of Change (CGI) displayed the highest sensitivity to change with SCA disease progression (53.0%), followed by each item from the functional Scale for the Assessment and Rating of Ataxia (f-SARA): Stance (17.0%), Gait (12.0%), Speech (10.0%), and Sitting (8.0%) [13]. While key requirements of psychometric scale validation including sensitivity to detecting clinically meaningful decline have been demonstrated for SCACOMS, its content validity is yet to be established.

Here, we report the findings from qualitative interviews with individuals with CAG repeat expansion SCA subtypes and healthcare professionals (HCPs) with clinical expertise in SCA. Interviews were conducted to evaluate the content validity of SCACOMS as a measure of disease progression for investigation of treatment effects in clinical trials. Assessment of the relevance of individual items comprising SCACOMS and their respective relative weighting is also reported.

Methods

Study Design

This was a qualitative interview study conducted with United States-based individuals with SCA and HCPs with clinical expertise in treating SCA. Interviews were conducted in accordance with the United States Food and Drug Administration Patient-Focused Drug Development Guidance [17], and were designed to evaluate SCA sign and symptom concepts and SCACOMS item relevance in the context of predicting SCA disease progression.

Participants

Individuals with SCA who met the eligibility criteria were recruited via Engage Health, Inc. (eligibility criteria are shown in Supplementary Table 1). Eligible individuals with SCA were aged 18–75 years, had a confirmed diagnosis of SCA1, SCA2, SCA3, or SCA6 (CAG repeat expansion subtypes), and had at least 6 years of education or equivalent. Eligible HCPs were those who interacted with ≥ 1 individual with SCA at least twice a year and had clinical experience using existing COAs in SCA.

Interview Process

One-to-one interviews with individuals with SCA and with HCPs were conducted in English via a teleconference platform between April and May 2024 using semi-structured discussion guides (Supplementary Table 2). Interviews with individuals with SCA and HCPs consisted of 3 parts: (1) collection of background information; (2) open concept elicitation to understand the signs, symptoms, and impacts of SCA; and (3) SCACOMS instrument review and discussion of item relevance and contribution to the assessment of disease progression.

During the first part of the interview, demographic and health information from individuals with SCA was collected, and pertinent demographic information, including SCA treatment experience, was ascertained from HCPs. Following this, all participants took part in an open-ended concept elicitation phase, which was designed to collect the perspectives of individuals with SCA and HCPs on how SCA signs and symptoms impact daily life, and understand how the disease progresses over time. In the final part of the interview, participants were asked cognitive debriefing questions to evaluate the individual items included on the SCACOMS instrument for relevance in the context of reflecting SCA disease progression. The relative weighting of each SCACOMS item was ranked by perceived importance for tracking meaningful changes in SCA symptoms; items were assigned a percentage value with the total score across all items to equal 100.0%. Interviews with individuals with SCA incorporated visual aids to facilitate discussions about complex concepts and enhance accessibility of the interview questions [17].

Data Analysis

A descriptive content analysis was employed to analyze the frequency of concepts described during the interviews with individuals with SCA and HCPs. Interviews were audio recorded then transcribed; transcripts were subsequently anonymized and coded using ATLAS.Ti v23 software. Following established qualitative research methods [18], 2 codebooks (1 for individuals with SCA and 1 for HCPs) were developed to analyze the interview transcripts. The transcripts were reviewed by multiple coders to minimize bias. Cognitive debriefing analyses were conducted on the SCACOMS instrument to assess the perceived relevance of included items, the response scale, item relative weighting, and whether clinically meaningful changes may be adequately detected [19]. Interview quotes are included as illustrative examples and to provide depth [20].

Ethical Considerations

The study was approved by a centralized independent Institutional Review Board (IRB; Salus Institutional Review Board, Austin, TX, USA) on April 1, 2024. All participants provided informed consent to take part in the interviews and could withdraw at any time. The study was conducted in accordance with the 1964 Declaration of Helsinki.

Results

Participant Demographics and Clinical Characteristics

Interviews were conducted with 24 individuals (54.2% female; mean [range] age, 54 [34–75] years) with genetically confirmed SCA. The most prevalent subtype among individuals with SCA was SCA3 (n = 10; 41.7%) (Table 1). Self-reported disease severity was equally distributed between mild, moderate, and severe SCA (all severity groups n = 8; 33.3%).

Table 1.

Demographics and clinical characteristics for individuals with SCA

Demographics	Individuals with SCA (N = 24), n (%)
Sex, n (%)
Male	11 (45.8)
Female	13 (54.2)
Age in years, mean (range)	54.0 (34–75)
Ethnicity, n (%)
White	18 (75.0)
Black African American	3 (12.5)
Asian	2 (8.3)
Asian and White	1 (4.2)
Education level, n (%)
Some college (no degree)	4 (16.7)
Associate’s degree	4 (16.7)
Bachelor’s degree	8 (33.3)
Master’s degree	7 (29.2)
Doctoral degree	1 (4.2)
Work status, n (%)
Working full-time	7 (29.2)
Working part-time	4 (16.7)
Homemaker	2 (8.3)
Retired	5 (20.8)
On disability allowance	6 (25.0)
Clinical characteristics
Age in years at diagnosis, mean (range)	44.3 (22–72)
SCA subtype, n (%)
SCA1	6 (25.0)
SCA2	6 (25.0)
SCA3	10 (41.7)
SCA6	2 (8.3)
Self-reported SCA severity, n (%)
Mild	8 (33.3)
Moderate	8 (33.3)
Severe	8 (33.3)

SCA, spinocerebellar ataxia

Two HCPs participated in the interviews. Both HCPs were female neurologists and reported extensive experience in treating a wide range of SCA subtypes. Both HCPs had prior clinical experience using the f-SARA and CGI to assess SCA disease progression and had previously been exposed to SCACOMS.

SCA Symptoms, Progression, and Impact on Daily Life

When asked to describe their signs and symptoms, individuals with SCA reported a total of 29 concepts related to their experience of SCA. Speech difficulties (n = 19; 79.2%), stance/balance issues (n = 18; 75%), and gait/walking difficulties (n = 16; 66.7%) were the most frequently reported concepts by individuals with SCA (Table 2). All concepts reported by over 50% of individuals with SCA were measured by SCACOMS items.

Table 2.

SCA sign and symptom concepts reported by individuals with SCA and those considered most bothersome

Concept	Frequency of report (N = 24), n (%)a	Reported as most bothersome (N = 24), n (%)
Speech difficulties	19 (79.2)	7 (29.2)
Stance/balance issues	18 (75.0)	12 (50.0)
Gait/walking difficulties	16 (66.7)	9 (37.5)
Swallowing issues	9 (37.5)	0
Weakness	9 (37.5)	2 (8.3)
Coordination	8 (33.3)	1 (4.2)
Vision issues	5 (20.8)	0
Tremors	5 (20.8)	0
Muscle cramping	4 (16.7)	0
Tiredness	4 (16.7)	2 (8.3)
Peripheral neuropathy	3 (12.5)	0
Dizziness	3 (12.5)	0
Dexterity/fine motor skills	3 (12.5)	0

^aConcepts reported by > 2 individuals with SCA

SCA: spinocerebellar ataxia

Additional concepts reported by > 2 individuals with SCA related to weakness, coordination, and difficulty using muscles for actions such as swallowing and hand movement. Of the sign and symptoms concepts reported, stance/balance issues (n = 12; 50.0%), gait/walking difficulties (n = 9; 37.5%), and speech difficulties (n = 7; 29.2%) were considered to be the most bothersome to individuals with SCA.

When asked how SCA impacts daily life, individuals with SCA reported a total of 43 impacts on daily life across 10 broad concepts. The most frequently reported concepts included implementing adaptive behaviors (n = 19; 79.2%), such as the use of walking aids to prevent falling, and physical function difficulties (n = 18; 75.0%), which included impacts to activities involving gross and fine motor skills (Table 3). Additionally, half of the individuals with SCA (n = 12, 50.0%) reported that the signs and symptoms of SCA impacted their sleep, ADLs, and cognitive function.

Table 3.

Concepts impacting the daily lives of individuals with SCA

Concept	Frequency of report (N = 24), n (%)a
Implementing adaptive behaviors	19 (79.2)
Physical function	18 (75.0)
Sleep	12 (50.0)
ADL	12 (50.0)
Cognitive function	12 (50.0)
Relationships or social function	9 (37.5)
Recreational/leisure activities	8 (33.3)
Emotional function	7 (29.2)
Household chores	7 (29.2)
Work	5 (20.8)

^aIndividuals with SCA had the opportunity to report multiple concepts, thus the frequency counts are not mutually exclusive

ADL: activities of daily living; SCA: spinocerebellar ataxia

One individual with severe SCA commented on the impact of symptoms on daily life: “Well, just making my own lunch, I can’t do that anymore. I have to have assistance with that, with making all my meals, actually.”

A second individual with severe SCA commented on the impact of symptoms on daily life: “Now that my kids are older, it’s very hard for me to even interact with my kids or play with my kids, or toss a baseball or kick a soccer ball.”

When asked to describe the signs and symptoms of SCA, both HCPs (n = 2; 100.0%) reported difficulties with gait and walking. Other symptoms mentioned by the HCPs included coordination, dizziness, speech, stance, tremors, vision, and cognition/mental health/emotions (all n = 1; 50.0%). When considering the impact of SCA on daily life, both HCPs (n = 2; 100.0%) reported progressive difficulties with physical function, fine motor tasks, ADLs, and working.

One HCP commented on the signs and symptoms of SCA: “The thing people talk about tends to be walking and then a little bit speech.”

Importance Ranking of SCACOMS Items

To evaluate overall impressions of SCACOMS as a clinical tool for assessing SCA disease progression, individuals with SCA were presented with each SCACOMS item individually to evaluate their perceived relevance, comprehension of each item, and its corresponding response scale.

All items comprising SCACOMS were well understood by individuals with SCA and were considered relevant for measuring disease progression. When ranking SCACOMS items from most to least important, Gait and Speech were frequently ranked as most important. Almost half of the individuals with SCA (n = 11; 45.8%) ranked Gait as the most important item for assessing disease progression; 37.5% (n = 9) ranked the Gait item as the second most important. Speech was rated as most important and second most important item by 33.3% (n = 8) and 37.5% (n = 9) of individuals with SCA, respectively. CGI was ranked as the least important item (n = 14; 58.3%) (Fig. 1). The Sitting item was considered the least (n = 9; 37.5%) or second least (n = 9; 37.5%) important item by most individuals with SCA.

Fig. 1.

Ranking of importance of SCACOMS items by individuals with SCA. One individual with SCA ranked all 5 SCACOMS items as equally most important (rank 1). GI: Clinician Global Impression of Change; SCA: spinocerebellar ataxia; SCACOMS: spinocerebellar ataxia composite score

When considering the importance of SCACOMS items, HCPs ranked Gait as the most (n = 1; 50.0%) or second most (n = 1; 50.0%) important item for those with early-stage SCA. The Sitting item was ranked as the least important by both HCPs (n = 2; 100.0%). There were differences between HCPs in the perceived importance of the CGI item. During early-stage SCA, 1 HCP (n = 1; 50.0%) ranked the item as the most important and the other HCP (n = 1; 50.0%) ranked it as the second least important. For severe SCA, 1 HCP (n = 1; 50.0%) ranked Stance as the most important SCACOMS item, whereas the other HCP maintained that CGI was the most important item.

Weighted Contribution of SCACOMS Items

Following the SCACOMS item importance ranking, individuals with SCA were presented with a pie chart visual aid that introduced the discussion with all SCACOMS items as contributing equally (Supplementary Fig. 1). Individuals were asked to indicate how much each slice of the pie should change (i.e., relative percentage contribution) to reflect the importance of each item for assessing disease progression. When considering the relative contribution of SCACOMS items, individuals with SCA attributed the highest weight to the Gait item (mean [SD]: 32.8% [13.2]) and the lowest weight to the Sitting item (12.9% [8.0]) (Fig. 2). When examining responses stratified by SCA disease severity, the relative weights of SCACOMS items were similar, with all groups assigning the highest weight to the Gait item (mean range: 29.3–37.1%) and the lowest weight to the Sitting item (mean range: 11.8–13.5%) (Supplementary Fig. 2). Those with moderate or severe SCA were more likely to assign a higher weight to the CGI item (mean range: 14.0–18.8%) than those with mild SCA (mean: 8.7%) at the expense of the Gait item.

Fig. 2.

Assigned mean relative weighting of SCACOMS items by individuals with SCA (a) and statistically-derived weighting of SCACOMS items provided for reference (b). CGI: Clinician Global Impression of Change; SCA: spinocerebellar ataxia; SCACOMS: spinocerebellar ataxia composite score

Following the assignment of relative weights to the SCACOMS items, individuals with SCA were then asked to review and indicate their level of agreement or disagreement with the current, statistically-derived, SCACOMS items weights using a rating scale ranging from 1 (strongly disagree) to 5 (strongly agree). While the degree of agreement with the statistically-derived weights of the SCACOMS items varied among the individuals with SCA, 41.6% (n = 10) strongly disagreed with the current SCACOMS weighting, citing that the weight of the CGI item was too high (Table 4). Most of these individuals with SCA explained that their disagreement was because they considered their lived experience of the disease and assessment of individual symptoms directly to be more important than the clinician perspective. Additionally, individuals with SCA reported that the CGI was based on a subjective opinion of an HCP who may not know them well enough to provide an accurate assessment of their overall health. Despite this, some individuals with SCA (n = 5; 20.8%) generally agreed with the overall SCACOMS weighting and acknowledged the importance of the CGI item for clinical assessment.

Table 4.

Agreement with the statistically-derived weighting of SCACOMS items reported by individuals with SCA

	Agreement rating for statistically-derived weights of SCACOMS items, n (%)					Average rating, mean (SD)
	1 (strongly disagree)	2	3	4	5 (strongly agree)
All individuals with SCA (N = 24)	10 (41.7)	3 (12.5)	6 (25.0)	4 (16.7)	1 (4.2)	2.3 (1.3)
By SCA severity
Mild (n = 8)	4 (16.7)	0	2 (8.3)	1 (4.2)	1 (4.2)	2.4 (1.6)
Moderate (n = 8)	4 (16.7)	1 (4.2)	2 (8.3)	1 (4.2)	0	2.0 (1.2)
Severe (n = 8)	2 (8.3)	2 (8.3)	2 (8.3)	2 (8.3)	0	2.5 (1.2)

SCA: spinocerebellar ataxia; SCACOMS: spinocerebellar ataxia composite score; SD: standard deviation

Upon analysis, there were no differences between SCA severity groups in overall agreement with the statistically-derived weights for SCACOMS items (Table 4).

Similar to individuals with SCA, HCPs were also asked to consider the relative contribution of each SCACOMS item for evaluating SCA progression. HCPs assigned the highest weight to the CGI item (mean, 30.0%) and the lowest weight to the Sitting item (mean, 5.0%) (Fig. 3a). When considering patients at different stages of the disease, HCPs assigned varying weights to SCACOMS items dependent on SCA severity. For those with mild SCA, HCPs assigned more weight to the Gait item (mean, 29.0%) than for later-stage disease (mean, 12.5%) (Fig. 3b and c). Conversely, for later-stage SCA, the Sitting item was assigned more weight (mean, 15.5%) compared with mild disease (mean, 8.0%).

Fig. 3.

Assigned mean relative weighting of SCACOMS items by HCPs (a) and assigned relative mean weighting of items stratified by SCA disease severity (b and c). CGI: Clinician Global Impression of Change; HCP: healthcare professional; SCA: spinocerebellar ataxia; SCACOMS: spinocerebellar ataxia composite score

Following this, HCPs were asked to provide their agreement rating with the current, statistically-derived, weighting of items on the SCACOMS instrument, using the same rating scale as individuals with SCA. One HCP (n = 1; 50.0%) disagreed with the statistically-derived CGI weighting, suggesting that the overall contribution should be reduced because they reported that the CGI item may not capture the long-term disease impacts or directly measure clinical symptoms. The second HCP (n = 1; 50.0%) agreed with the overall weighting of the SCACOMS items but suggested that the overall contribution of the Sitting item could be increased at the expense of the CGI item.

Defining Clinically Meaningful Change and Stability in SCA Using SCACOMS

To understand what may constitute clinically meaningful change in the context of SCACOMS, individuals with SCA were asked whether stability in their SCACOMS item scores would be meaningful. Discussions were at the individual item level and did not consider item weights.

Most individuals with SCA (n = 22/24; 91.7%) reported that overall stability in their SCACOMS score would be meaningful. When considering the timeframe for meaningful stability, the majority of individuals with SCA (n = 17/22; 77.3%) agreed that stability in their SCACOMS score for 1 year would be meaningful. The remaining individuals with SCA reported that SCACOMS stability for 3 years (n = 3/22; 13.6%) or for the rest of their life (n = 2/22; 9.1%) would be required to be meaningful.

Following this, individuals with SCA were asked whether stability would be considered meaningful if it occurred in just 1 SCACOMS item. Most individuals with SCA reported that stability in 1 (n = 6/24; 25.0%) or 2 (n = 10/24; 41.7%) SCACOMS items would be meaningful; 1 individual (n = 1/24; 4.2%) suggested that stability across all 5 items would be required to be meaningful. Overall, the SCACOMS item that was most frequently reported by individuals with SCA as the most important for stability was Speech (n = 17/24; 70.8%), followed closely by Gait (n = 15/24; 62.5%). However, this varied when stratifying by disease severity; mild SCA was Speech (n = 6/8; 75.0%), moderate SCA was Gait (n = 7/8; 87.5%), and severe SCA was Speech (n = 6/8; 75.0%) (Table 5). While CGI was considered the least important item for meaningful stability overall (n = 6/24; 25.0%), a proportion of individuals with mild and moderate SCA (both n = 3/8; 37.5%) reported that stability on the CGI item would be meaningful.

Table 5.

Frequency of report of the most important SCACOMS items for meaningful stability by individuals with SCA

	Frequency of report of SCACOMS item, n (%)
	Gait	Stance	Sitting	Speech	CGI
All individuals with SCA (N = 24)	15 (62.5)	7 (29.2)	7 (29.2)	17 (70.8)	6 (25.0)
SCA severity
Mild (n = 8)	5 (62.5)	3 (37.5)	3 (37.5)	6 (75.0)	3 (37.5)
Moderate (n = 8)	7 (87.5)	1 (12.5)	2 (25.0)	5 (62.5)	3 (37.5)
Severe (n = 8)	3 (37.5)	3 (37.5)	2 (25.0)	6 (75.0)	0

CGI: Clinician Global Impression of Change; SCA: spinocerebellar ataxia; SCACOMS: spinocerebellar ataxia composite score

One individual with mild SCA commented on the SCACOMS items that were considered important for meaningful stability: “Speech and Sitting, because those are key to my job.”

One individual with moderate SCA commented on the SCACOMS items that were considered important for meaningful stability: “I think Gait and the Clinician Impression are very important. So mostly Gait.”

One individual with severe SCA commented on the SCACOMS items that were considered important for meaningful stability: “[Speech]. I would be able to continue to talk to my boys and my family.”

Individuals with SCA were then asked to describe what would constitute minimal meaningful worsening for each SCACOMS item. For nearly all SCACOMS items (Gait, Stance, Speech, and CGI), most patients considered a 1-level change as minimal meaningful worsening; a median 1.5-level change was reported as meaningful for the Sitting item (Supplementary Table 3).

Similarly, HCPs were also asked to describe what they considered to be meaningful stability for individuals with SCA in the context of SCACOMS. Both HCPs (n = 2; 100.0%) reported that the definition of meaningful stability in particular SCACOMS items was dependent on the severity of SCA. For individuals with mild SCA, HCPs indicated that stability in the Gait (n = 1; 50.0%) and CGI (n = 1; 50.0%) items would be considered meaningful because these appear to be universally impacted across all individuals with SCA as the disease manifests. There was agreement between HCPs regarding what constituted meaningful stability for those with moderate SCA, with both HCPs (n = 2; 100.0%) reporting that stability in the Speech item would be meaningful. For those with severe SCA, 1 HCP (n = 1; 50.0%) noted that stability in both the Sitting and Speech items would be meaningful, whereas the other HCP (n = 1; 50.0%) suggested that stability in the CGI and Stance items would be meaningful. Regarding the duration of stability on SCACOMS items, both HCPs (n = 2; 100.0%) indicated that the stability of items that they considered most important in mild disease (i.e., Gait and CGI) for 1 year would be meaningful for individuals with SCA.

Following this, HCPs were asked about what may constitute meaningful worsening in the Gait or CGI items. For Gait, 1 HCP (n = 1; 50.0%) reported that meaningful worsening for individuals with mild SCA would be a 1-level change, specifically from a score of 1 to a score of 2, and for moderate SCA, a 1-level change from a score of 2 to a score of 3 would be regarded as meaningful. One HCP (n = 1; 50.0%) discussed meaningful worsening in the context of the CGI item and indicated that a 1-level change would be meaningful, irrespective of disease severity. Regarding meaningful worsening with a treatment, both HCPs (n = 2; 100.0%) confirmed that a delay to symptom progression in any of the SCACOMS items would be meaningful for those individuals with mild disease.

Discussion

The development of composite scoring algorithms that offer a highly responsive assessment of patient change constitute a valuable class of tool to assess potential treatment effects in clinical trials. While adequate responsiveness to clinical change is a key metric inherent to their development, responsiveness does not always consider patient feedback. Qualitative research evaluating the perspectives of individuals with SCA offers important insights when examining the overall relevance and patient centricity of these measures.

This qualitative interview study evaluating the content validity of SCACOMS indicates that the statistically-derived composite measure captures relevant and clinically meaningful outcomes for the assessment of SCA progression. Importantly, all sign and symptom concepts reported by > 50.0% of individuals with SCA are assessed by SCACOMS items. All items comprising SCACOMS and their associated response scales were well understood by individuals with SCA and HCPs, and meaningful changes for each item were identified. However, agreement with the statistically-derived weights of the items was limited, with almost half of individuals with SCA and 1 HCP reporting that the weight of the CGI item was too high.

There were no novel concepts identified during the concept elicitation phase of the interview compared with previously published SCA concept libraries [21], suggesting that the current study offered a comprehensive overview of the signs, symptoms, and impacts of SCA. The perceived importance of SCACOMS items by individuals with SCA was comparable to a recently published f-SARA content validity study [22]. Notably, the most frequently reported SCA impacts (gait, balance, and speech) are assessed by SCACOMS. During discussions, some individuals with SCA indicated that the addition of SCACOMS items assessing vision, fine motor skills, neuropathy, and health-related quality of life may improve the clinical meaningfulness of overall assessment of disease progression. While evaluation of those symptoms is not specifically captured by SCACOMS, we suggest that these could be evaluated as part of the CGI item assessment, particularly if the individual feels that clinical assessment of particular symptoms may offer key insights into their SCA progression.

In this study, Gait was considered to be the most important item for assessment of disease progression, followed by Speech and Stance. The importance of gait assessment is highlighted by the effort to study it with maximal sensitivity. For instance, Ilg et al. investigated the use of digital measures to capture disease progression in SCA3 over 1-year [23]. Sitting and CGI were similarly ranked as the least important SCACOMS items for evaluation of symptom progression by individuals with SCA. Several individuals considered that the CGI item was of lower value compared with the f-SARA items because it did not assess SCA symptoms directly, nor clearly represent how the patient functioned. The CGI was also perceived as a subjective assessment, which also limited its acceptability. This perspective was reflected in the consistently lower weight contributions of the CGI item relative to the individual symptom items. Interestingly, the weighting of the CGI item was associated with SCA severity, with those with mild SCA assigning the lowest weight and those with moderate or severe SCA assigning a higher weight to this item. These differences may be because individuals with moderate or severe SCA have had more interactions with HCPs during the course of their disease and thus place greater importance on the HCPs’ assessment of their disease progression. Indeed, both HCPs emphasized the importance of listening to the patient’s perspective, and their description of how SCA impacts their daily life, to understand how symptoms have changed between visits and over longer periods of time. Consistent with the weighting data from individuals with SCA, HCPs also suggested that the weighting of SCACOMS items should differ based on SCA severity, with notable differences in the Gait and Sitting item weightings between early-stage and later-stage SCA.

Most individuals with SCA and HCPs reported that SCACOMS could detect clinically meaningful changes and stabilization of disease. The majority of individuals with SCA and HCPs reported that overall stability in SCACOMS score for 1 year would be meaningful in terms of disease progression; however, meaningful change varied depending on the SCACOMS item. Most individuals with SCA frequently reported that stability on the Gait and Speech items would be required to be meaningful, with few considering stability on the CGI item meaningful. Conversely, 1 HCP regarded stability on the CGI as most meaningful, irrespective of SCA severity, as this provided a global overview of disease progression, including the overall signs, symptoms, and impacts of SCA on daily life. In addition, most individuals with SCA and HCPs reported that SCACOMS could detect clinically meaningful worsening of disease. A 1-level change in most SCACOMS items was reported as an important indicator of meaningful worsening by most individuals with SCA, irrespective of SCA severity.

Overall, the clinical meaningfulness of the items measured by SCACOMS for assessment of disease progression in individuals with SCA was demonstrated. However, the weights of the items may not optimally reflect the impacts that patients perceive to be most clinically important. Examination of the perceived relevance of the differential statistically-derived weighting of previously cross-validated SCACOMS items [13] from individuals with SCA was an important step to understand how SCACOMS measurement properties may be optimized to ensure patient centricity. Reconciling the statistically-derived item weights and the perspectives of individuals with SCA is a complex undertaking, given that the ultimate goal is to develop a more responsive measure to SCA disease progression. Consideration must be taken to balance what may be competing priorities; for example, the effectiveness of the statistically-derived measure alongside the desire to measure concepts that are of central importance to patients. While the challenge of this integration should not be underestimated (it has not, to our knowledge, been performed for other COAs because of uniform item weights), there are tangible gains to be made for patients and HCPs alike with the implementation of a COA with differential item weights that offers a reliable and responsive evaluation of symptoms individuals perceive as key indicators of disease progression. Development of a COA with these properties will be particularly important for assessment of disease-modifying therapies.

The findings from this study support SCACOMS as a clinically meaningful and highly sensitive composite outcome measure that may detect compelling treatment effects in trials of disease-modifying therapies for SCA. Despite this, we recognize that the weighting of individual items may require further examination to incorporate the patient voice. The study has several limitations that require acknowledgment. Individuals with SCA were recruited via a third-party agency in the United States, which may have introduced sample bias. Furthermore, whilst the overall sample was balanced in terms of SCA severity, individuals with specific CAG repeat expansion SCA subtypes were included (SCA1, SCA2, SCA3, and SCA6), which may limit the generalizability of the results to the other approximately 50 subtypes of SCA. Similarly, the sample only included individuals with at least 6 years of education, which may preclude generalizability to those with lower levels of education. The inclusion criteria did not control for cognitive function or mental-emotional status, which may affect individuals’ answers, particularly signs and symptoms and impact on daily life. Future studies could either control for this or compare those with varying cognitive function and mental-emotional statuses to gain a clearer perspective on how this may affect results. Finally, while the data from HCPs offered a clinical perspective, the sample size was limited to 2 and should be interpreted with caution. Despite this, the study demonstrated novel application of qualitative interviews with individuals with SCA to ascertain the perceived agreement with the statistically-derived differential weighting of SCACOMS items.

Conclusions

Overall, this study supports the content validity of SCACOMS in SCA. SCACOMS was reported to be a valid and relevant measure in the context of assessing clinically meaningful SCA disease progression, particularly in early-stage disease. The overall SCACOMS instrument was well understood by individuals with SCA and HCPs; however, the relative contributions of individual SCACOMS items may warrant adjustment to better reflect their importance to individuals with SCA.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Author Contributions

MP, MWB, VC, GL, and JS contributed to study conception and design. Material preparation and data collection was performed by MH, KR, LA-W, NS, RD, KJ, MH, and CB. All authors contributed to data analysis and interpretation. All authors commented on the manuscript during development and approved the final version.

Funding

This study was supported by Biohaven Pharmaceuticals, Inc.

Data Availability

The datasets generated and analyzed during the current study are available from the corresponding author on reasonable request.

Declarations

Patient Consent and Ethical Approval

All eligible individuals with SCA and the HCPs provided informed consent to participate in the interviews and could withdraw at any time. The study was conducted in accordance with the 1964 Declaration of Helsinki.

Competing Interests

Maggie Heinrich, Katja Rudell, Naomi Suminski, Rinchen Doma, Kavita Jarodia, Mahak Jain, and Chris Buckley are employees of Parexel International and have been compensated as consultants by Biohaven Pharmaceuticals Inc. Linda Abetz-Webb received consultancy fees from Parexel International for this study. Jeremy Schmahmann has received compensation as a consultant for Biohaven Pharmaceuticals Inc., is site Principal Investigator for the Biohaven clinical trials NCT03701399 and NCT02960893, and has received an unrestricted grant from Biohaven for an imaging biomarker study in multiple system atrophy. Susan Perlman has been a member of an advisory board or consultant for Biogen, Biohaven Pharmaceuticals, Inc, Erydel, PTC, Quince Therapeutics, Reata Pharmaceuticals, and Steminent. Liana Rosenthal is an Associate Professor in the Department of Neurology at the Johns Hopkins University School of Medicine. She engaged in this research as a private consultant or advisor and not in her capacity as a Johns Hopkins faculty member, and was compensated. Liana Rosenthal also receives: research funding support from the NINDS, the Daniel B. and Florence E. Green Family Foundation, and the Macks Family Foundation; programmatic support from the Gordon and Marilyn Macklin Foundation; and salary support for her role as site Principal Investigator for research studies with Biohaven Pharmaceuticals, Pfizer, and EIP Pharma and for serving on the Clinical Events Committee for a research study with Functional Neuromodulation. Liana Rosenthal also serves on steering committees for the Parkinson Study Group’s research study with both UCB and Bial Pharmaceuticals, and on advisory boards for Reata pharmaceuticals, Biogen pharmaceuticals, Biohaven Pharmaceuticals, Merck Healthcare KGaA, and was a one-time consultant for UCB pharmaceuticals.Michele Potashman, Melissa Wolfe-Beiner, Vlad Coric, and Gilbert L’Italien are employed by and hold stock/stock options in Biohaven Pharmaceuticals Inc.