VALIDATION OF THE TRIPLE TIMED UP-AND-GO TEST IN LAMBERT-EATON MYASTHENIA

Abstract

Introduction:

There are no validated, practical and quantitative measures of disease severity in Lambert-Eaton Myasthenia (LEM).

Methods:

Data from the DAPPER trial were analyzed to assess 3TUG reproducibility and relationships between 3TUG times and other measures of LEM severity.

Results:

The coverage probability technique showed ≥ 0.90 probability for an acceptable 3TUG difference of ≤ 0.2, indicating that it is reproducible in LEM patients. Correlation between 3TUG times and Lower Extremity Function Scores was significant in subjects who continued and in those who were withdrawn from 3,4-diaminopyridine free base. Worsening patient-reported Weakness Self-Assessment Scale scores and Investigator Assessments corresponded with prolongation of 3TUG times.

Discussion:

The 3TUG is reproducible, demonstrates construct validity for assessment of lower extremity function in LEM patients, and correlates with changes in patient and physician assessments. These findings, along with prior reliability studies, indicate 3TUG is a valid measure of disease severity in LEM.

INTRODUCTION:

Lambert-Eaton Myasthenia (LEM) is a rare and debilitating disorder of neuromuscular transmission caused by autoantibodies to the P/Q-type voltage-gated calcium channels (VGCC) of the presynaptic neuromuscular junction.¹ Patients typically present with weakness in the shoulder, hip, and thigh muscles² and autonomic dysfunction.³ Dysphagia and respiratory failure⁴ can be prominent in some patients. The decline in lower extremity strength leads to impaired mobility⁵ and poor quality of life.⁶

Several single-center studies^7–11 have demonstrated improvement in LEM-associated weakness after administration of the potassium channel antagonist^12–14 3,4-diaminopyridine free base (DAP). Previous treatment trials of DAP in LEM have used change in strength of selected muscles,¹¹ the Quantitative Myasthenia Gravis Score (QMG)^{12, 13} or myometry^{8, 11} as primary outcome measures, with CMAP amplitude^{8, 11–13} as a secondary outcome measure in some trials. Autonomic function testing, patients’ subjective symptoms and physicians’ disease classification have also been used as measures of disease severity in LEM trials. None of these measures have been validated in LEM, and others, including the QMG, do not measure important causes of LEM-related disability, such as proximal lower extremity weakness and diminished mobility. Electrodiagnostic studies are time consuming and the availability of some techniques is limited to specialized centers, making them impractical as routine clinical measures of disease severity.

An optimal outcome measure for LEM should fulfill the basic requirements proposed by Hobart et al.,¹⁵ that it be clinically practical and scientifically sound on the basis of validity, reliability and responsiveness.

The Timed Up-and-Go (TUG) test¹⁶ is a simple objective measure of mobility that has been validated primarily in geriatric populations with parkinsonism^{17, 18} and impaired cognition.^{19, 20} It requires the patient to rise up from the seat of a straight backed armchair, walk 3 meters at a normal pace, turn around, walk back to the chair turn and sit back down. A variation of this test, the Triple Timed Up-and-Go (3TUG) test, requires 3 repetitions (laps), and assesses the lower extremity weakness and fatigue or facilitation characteristic of LEM. The 3TUG has been demonstrated to have excellent test-retest reproducibility and interrater reliability in patients with non-LEM neuromuscular disease²¹ but has not been fully validated in LEM.

This study was designed to further validate the 3TUG in LEM patients by confirming test-retest reproducibility and interrater reliability, establishing construct validity through correlation with other measures of LEM-specific disability, and assessing its responsiveness to patient and provider-reported measures of disease severity.

METHODS:

This is a secondary analysis of data from the DAPPER clinical trial, a double-blind, placebo-controlled withdrawal study of DAP in patients with LEM (NCT01511978).²² The trial consisted of 4 stages (eFigure 1): Acclimation (0.5 days), Baseline (2 days), Withdrawal (up to 3.5 days) and DAP Reinstitution (0.5 to 2 days). Data from participants who completed at least the acclimation and baseline observation stages were included in this analysis. Participants were randomized to receive DAP or placebo as per home schedules. Time points for non-3TUG measures (eFigure 1) were determined based on when they were performed prior to randomization or during withdrawal, and were matched with the closest 3TUG time. Lower Extremity Function Scale (LEFS) scores were obtained during Acclimation (Day 0) and again at the end of withdrawal or time of early advancement; all other baseline measurements were obtained during Baseline (Day 2).

Oversight:

The Duke University Health System Institutional Review Board exempted this study (Pro00083458) from review.

Outcome Measures:

The 3TUG was performed before and after the first DAP doses of the afternoon and evening with 4 test times daily during the acclimation stage and 6 times daily (including before and after first morning dose) during baseline and withdrawal stages.

CMAPs were measured in the muscle determined to be most responsive to DAP during acclimation and were reviewed for quality by a blinded observer (eTable 1). CMAP amplitude was measured before and after the first doses of the morning and afternoon during baseline and withdrawal.

The LEFS (eFigure 2) is a 20-item patient-reported outcome measure commonly used to assess mobility in patients with orthopedic conditions.²³ The best possible score is 80, and a change of ±9 points indicates a clinically meaningful change in functional ability. While this tool has not been used in patients with LEM, the predominant lower extremity and hip girdle weakness in LEM is consistent with symptoms experienced by patients with musculoskeletal problems.

The Weakness Self Assessment Scale (W-SAS) is a secondary efficacy measure created by the DAPPER trial sponsor (eFigure 3). It features 7 categories with numerical values that allow a participant to rank weakness along a continuum from “Much Much Weaker (−3)” to “Much Much Stronger (+3).” W-SAS was performed 2 hours following the first DAP doses of the afternoon and evening with 2 test times daily during baseline and 3 times daily (including first morning dose) during the withdrawal stage.

The Investigator Assessment of Treatment Effect is a 5-item categorical scale created by the DAPPER trial sponsor (eFigure 4). It was performed by a blinded investigator to assess overall disease severity at the conclusion of withdrawal or at the time of early advancement. Participants were assessed along a continuum from “Much worse than during baseline (0)” to “Much improved from baseline (4).”

Statistical Analysis:

Reproducibility and reliability analyses were performed using data from participants who completed the acclimation and baseline stages. The other analyses included only randomized participants. Since the goal of this validation study was to evaluate 3TUG performance over time in participants who are clinically stable (continuous DAP) and clinically changing (controlled DAP withdrawal), between-group comparisons of outcome measures for those who continued DAP and those who were withdrawn from DAP were not performed; these have been reported previously.²² All analyses were performed using a percentage change from baseline rather than the absolute values at the pre-specified time points. Data were analyzed using SAS software (version 9.4, SAS Institute, Cary, NC).

Test-retest reproducibility and interrater reliability of the 3TUG

Time-matched 3TUG times recorded by the same on-site observer for the same participant on 2 consecutive days (Figure 1) were analyzed to assess test-retest reproducibility. Participants continued their home doses of DAP during these stages, and only data from participants with values for both time points were included. Agreement between 2 observations of the same 3TUG was determined by comparing 3TUG times recorded by an onsite observer and a remote observer who viewed video-taped 3TUGs. Only participants with observations by 2 different observers were included (n=46). The coverage probability (CP) method was used to assess agreement between paired observations.²⁴ A CP value is the probability that the ratio between paired observations falls within a pre-established range: it is calculated by dividing the number of observed ratios within the acceptable range by the total number of comparisons. For this study it was established a priori that agreement would be demonstrated by a CP ≥ 0.90. Point estimates and a 95% CI for the CP were calculated for an acceptable difference of ≤ 20%. A sensitivity analysis for an acceptable difference of ≤10% was also performed. Bland-Altman plots were constructed.

Figure 1.

Percent change in CMAP amplitude versus percent change in 3TUG time during withdrawal (Stage II). CMAP amplitude and 3TUG time are standardized to baseline (Day 2). Lines of best fit show an association between percent change in CMAP amplitude and percent change in 3TUG time as DAP is progressively withdrawn on Days 3 (A) and 4 (B). The association is not apparent on Day 5 (C).

3TUG, Triple Timed Up and Go test; DAP, 3,4-diaminopyridine free base; CMAP, compound muscle action potential

3TUG times vs CMAP amplitudes

The association between change in 3TUG time recorded by an onsite observer and change in time-matched CMAP by treatment group was assessed using data from subjects with both non-missing post-dose 3TUG times and CMAP amplitudes for the first afternoon dose at baseline and during each day of the withdrawal phase. Percent change in 3TUG time and CMAP amplitude were calculated using the following formula:

$$\left[\left(\text{Post-dose withdrawal }-\text{ Post-dose baseline}\right)\text{/}\left(\text{Post-dose-baseline}\right)\right]\times \text{ 100}$$

Descriptive statistics for the mean absolute and percentage change in 3TUG time and CMAP amplitude were calculated. A multivariable linear regression was fit to the data by randomization group using the pre-specified model:

$${\Delta \text{CMAP}}_{\text{i}}{=\Delta \text{3TUG}}_{\text{i}}\text{+ Pyridostigmine bromide }\left(\text{PB}\right)\text{ + age + assistive device}.$$

Only statistically significant (p < 0.05) adjustment variables were reported in the final model due to risk of overfitting with the small sample size. A simplified univariate linear regression was reported: ΔCMAP_i = Δ3TUG_i

3TUG times vs LEFS scores

The association between change in 3TUG time recorded by an onsite observer and change in LEFS scores by treatment group was assessed using data from participants with time-matched post-dose 3TUG times and LEFS scores from baseline and at the end of withdrawal. Percent change in 3TUG time was calculated using the following formula:

$$\left[\left(\text{Withdrawal value }-\text{ Baseline value}\right)\text{/}\left(\text{Baseline value}\right)\right]\times \text{ 100}$$

Raw change in the LEFS score was calculated using the following formula:

$$\left(\text{Withdrawal value }-\text{ Baseline value}\right)$$

Descriptive statistics were calculated for the change in 3TUG time and LEFS score at each time point. Spearman correlations between the time-matched 3TUG time and LEFS scores and percent change in 3TUG time and raw change in LEFS scores were determined.

3TUG times vs W-SAS

The association between change in 3TUG time and change in W-SAS score was assessed by comparing the last available time-matched 3TUG time and W-SAS score in the withdrawal stage (Day 4 or 5) and the last available time-matched W-SAS and 3TUG time during baseline stage (Day 2). Descriptive statistics were calculated for the percent change in 3TUG time and change in numerical W-SAS score for each randomization group.

3TUG times vs Investigator Assessment of Treatment Effect

The association between change in 3TUG time recorded by an onsite observer and overall Investigator Assessment was determined by comparing the change in 3TUG time to the investigator assessment at the end of withdrawal. Only participants with documented values for both 3TUG and the Investigator Assessment at the end of withdrawal were included in the analysis.

RESULTS:

Test-retest reproducibility of the 3TUG

Forty-six pairs of observations recorded by onsite observers were analyzed (Table 1). The CP for agreement in time-matched observations on consecutive days is 0.93 (95% CI: 0.82–0.99) for an acceptable range of ≤20%, and 0.67 (95% CI: 0.54–0.81) for an acceptable range of ≤10% (eFigure 5).

Table 1.

Reproducibility and Reliability of 3TUG

Parameter	Test-retest reproducibility*		Interrater reliability†
	Test 1	Test 2	Observer 1	Observer 2
Mean 3TUG, s	10.2	10.0	10.0	10.0
SD, s	2.9	2.8	2.8	2.9
Mean difference, s	−0.2		0.0
Min, s	−2.5		−0.5
Max, s	4.3		0.8
SD, s	1.2		0.2

^*Post-dose following the first afternoon dose on Day 0 and Day 1.

^†Post-dose following the first afternoon dose on Day 1. 46 paired observations are represented in each analysis.

3TUG, triple timed up and go test; s, seconds; SD, standard deviation

Interrater reliability of the 3TUG

Forty-six pairs of observations from baseline were analyzed (Table 1). The CP for agreement between unblinded and blinded observers for the same 3TUG test was 1.00 (95% CI: 0.92– 1.00) for an acceptable range of ≤20%, and 1.00 (95% CI: 0.92 – 1.00) for an acceptable range of ≤10% (eFigure 6).

3TUG time vs CMAP amplitude

Between baseline and the last post-dose CMAP of the withdrawal period (Study Day 5 or early advancement), 3TUG times increased by a mean of 1.5% (95% CI −0.4 – 0.6) in those who continued DAP (n = 8) and by a mean of 32.9% (95% CI 16.4 – 49.3) in those who were withdrawn from DAP (n = 8) (eTable 2). In this same time period, the CMAP decreased by a mean of −6.9% (95% CI −21.2 – 7.5) in those who continued DAP and by a mean of −40.9 (−3.4mV, CI −60.5 – −21.3) in those who were withdrawn from DAP (eTable 3). Scatterplots with a line of best fit suggest a trend of decreasing CMAPs with increasing 3TUG times (Figure 1). Linear regression revealed a significant (p < 0.01) association between 3TUG time and CMAP amplitude during the second day of the withdrawal stage (Study Day 4) in those who were withdrawn from DAP; in these subjects an increase of 1% in the 3TUG time was associated with a −1.05% (95% CI: −1.52, −0.57) change in CMAP amplitude (Table 2). This association was not significant at baseline or at the end of withdrawal (Study Day 5) in either group.

Table 2.

Association between change from baseline post-dose CMAP amplitude and change from baseline post-dose 3TUG

Group	Continuous DAP		Withdrew from DAP
Model*	Effect estimate (95% CI)	P-value	Effect estimate (95% CI)	P-value
Day 3†	−0.70 (−3.33, 1.92)	0.55	−0.01 (−2.20, 2.18)	0.99
Day 4‡	−0.43 (−1.65, 0.79)	0.43	−1.05 (−1.52, −0.57)	<0.01
Day 5§	0.32 (−1.69, 2.32)	0.71	−0.52 (−1.59, 0.56)	0.29

^*ΔCMAP_i = Δ3TUG_i

^†continued DAP N=10, withdrawn from DAP N=14;

^‡continued DAP N=9, withdrawn from DAP N=12;

^§continued DAP N=8, withdrawn from DAP N=8. Only 3TUG-CMAP pairs with CMAPs considered acceptable by the blinded reviewer were included in the analysis for each day

CMAP, compound muscle action potential; 3TUG, triple timed-up-and-go test; DAP, 3,4-diaminopyridine free base

3TUG times vs LEFS scores

Between baseline and the LEFS at the end of the withdrawal stage, the 3TUG time decreased by a mean of 1.9% (0.1 secs) in the group that continued DAP (n = 13) and increased by 126.3% (14.4 secs) in the group that was withdrawn from DAP (n = 15). LEFS scores decreased by a mean of 2 points in the continued treatment group and decreased by 24 points in the group that was withdrawn from DAP. Spearman correlation showed a strong negative correlation between the 3TUG time and the total LEFS score prior to re-institution of DAP in the continued DAP group (r = −0.64, p = 0.02) and in those who were withdrawn from DAP (r = −0.64, p = 0.01).

3TUG times vs W-SAS

Between baseline and the end of the withdrawal stage, the 3TUG time increased by a mean of 2.1% (0.2 secs) in the group that continued DAP (n = 13) and increased by 81.5% (9.3 secs) in the group that was withdrawn from DAP (n = 16). Among those who were withdrawn from DAP, an increase in 3TUG time was associated with a greater decline in W-SAS score (Figure 2A). Worsening of ≥3 W-SAS points corresponded with a mean prolongation of ≥73.6% of 3TUG time. Across both treatment groups, 3/11 (27.3%) of participants who reported feeling “Much, Much Weaker” were unable to perform the 3TUG.

Figure 2.

A. Number of responders by change in W-SAS score. The numerical value represents the mean percent change in 3TUG time for each W-SAS score.

B. Number of responders by change in Investigator Assessment of overall treatment effect. The numerical value represents the mean percent change in 3TUG time for each investigator assessment category.

W-SAS, Weakness Self-Assessment Scale; 3TUG, Triple Timed Up and Go test; DAP, 3,4-diaminopyridine free base; PI, Principal Investigator

3TUG times vs Investigator Assessment of Treatment Effect

Blinded investigators assessed participants at baseline and at the end of the withdrawal period. Participants who were graded as “Much Worse” than at baseline had a mean increase of ≥94.3% in their 3TUG time and all were withdrawn from DAP (n = 18); no participants who continued DAP (n = 13) were rated as “Much Worse” (Figure 2B). Across both treatment groups, 3TUG times deteriorated by ≥30% in 81.3% (13/16) of participants who were rated as “Somewhat Worse” (4/16) or “Much Worse” (12/16). Among those rated as “Much Worse,” 91.7% (11/12) experienced a ≥30% deterioration in 3TUG. A total of 3 participants who were rated as “Much Worse” were not able to perform the 3TUG. No participants were reported as “Somewhat Better” or “Much Better.”

DISCUSSION:

Results of this study demonstrate that the 3TUG has both excellent reproducibility and inter-rater reliability in a population of LEM patients on stable therapy and has content and face validity for the assessment of lower extremity dysfunction. The DAPPER trial demonstrated that the W-SAS accurately captures patient perceptions of weakness²² and analysis of the 3TUG reveals that prolongations of 3TUG times correspond with declines in W-SAS scores. Additionally, deteriorations in 3TUG times are congruent with worsening as assessed by blinded investigators. Together these results indicate that the 3TUG is responsive to patient- and clinician-reported changes in disease severity. This offers an advantage over electrophysiologic measures because it can be performed without special equipment and technical training.

While there is no “gold standard” of LEM severity for comparison, the 3TUG can be compared with other measures of lower extremity function and mobility, such as the validated LEFS, with which it correlated well at the end of the withdrawal period, both in participants who continued DAP and in those who were withdrawn from DAP. Prior studies²³ have established a clinically meaningful change in the LEFS score to be ±9 points, consistent with observations in the DAPPER LEM participants who were withdrawn from DAP (mean −22, range −33 to −8), providing content validity of the 3TUG as an assessment of lower extremity function.

This study had several limitations related to the comparator instruments and small sample sizes. The effect estimate of CMAP amplitude and 3TUG times was significant only in the middle of the withdrawal period (Study Day 4) for those who were withdrawn from DAP. This was unexpected because prior reports have demonstrated increases in CMAP amplitude following DAP administration.^{10, 11, 14, 25} Several factors may have played a role in these inconsistent results, among them the small number of participants, particularly at the end of withdrawal due to rescue, and differences in the CMAP technique used at different study sites.^26–28 Indeed, CMAPs from only 10/12 (83%) of participants who continued DAP met pre-determined criteria for acceptability (eTable 3) on the first day of withdrawal (Study Day 3) and the number of acceptable CMAP studies was lower at each subsequent assessment. This pattern was also observed in the group that was withdrawn from DAP: only 14/18 (78%) of participants had acceptable CMAP studies on Study Day 3, and the number of acceptable CMAP studies declined at each subsequent assessment. These findings highlight the need for careful attention to the technical aspects of performing electrodiagnostic measures, and the potential limitations of using CMAP amplitudes in multicenter clinical trials unless rigorous training is employed. These concerns are shared with regard to other electrodiagnostic measures, as recently highlighted in an editorial on the use of Motor Unit Number Index (MUNIX) in clinical trials.²⁹ An alternative methodology using longitudinal analysis strategies, such as mixed effects modeling, in an adequately-powered study might provide a more accurate assessment of the relationship between pre- and post-dose 3TUG times and CMAP amplitudes, and could also assess clinically meaningful co-variates such as age, use of pyridostigmine and assistive devices.

Although the CP for agreement in the test-retest reproducibility of the 3TUG was 0.93 for an acceptable range of ≤ 20%, the CP for an acceptable range of ≤10% was 0.67. This suggests the possibility of skew and dispersion in the data. The original DAPPER analysis did not report data from Day 0 because of concerns that fatigue due to travel could have an unpredictable effect on the data.

This study expands upon the current knowledge of outcome measures in LEM and demonstrates that the 3TUG is reproducible in LEM patients, consistently scored by observers, and has both content and construct validity for LEM-associated disability. The 3TUG is a practical, validated outcome measure for clinical assessment of LEM patients that is suitable for use in the clinic, as well as in clinical trials.

Abbreviations:

LEM

Lambert Eaton Myasthenia

3TUG

Triple Timed Up-and-Go test

DAP

3,4-diaminopyridine free base

QMG

Quantitative Myasthenia Gravis Score

DAPPER

Effectiveness of 3,4-Diaminopyridine in Lambert-Eaton Myasthenic Syndrome trial

CMAP

Compound Muscle Action Potential

W-SAS

Weakness Self-Assessment Scale

LEM-ADL

Lambert-Eaton Myasthenia Activities of Daily Living

LEFS

Lower Extremity Function Scale

CP

Coverage Probability

MCID

Minimal Clinically Important Difference