External validation of a simple clinical tool used to predict falls in people with Parkinson disease

Abstract

Background

Assessment of fall risk in an individual with Parkinson disease (PD) is a critical yet often time consuming component of patient care. Recently a simple clinical prediction tool based only on fall history in the previous year, freezing of gait in the past month, and gait velocity <1.1 m/s was developed and accurately predicted future falls in a sample of individuals with PD.

METHODS

We sought to externally validate the utility of the tool by administering it to a different cohort of 171 individuals with PD. Falls were monitored prospectively for 6 months following predictor assessment.

RESULTS

The tool accurately discriminated future fallers from non-fallers (area under the curve [AUC] = 0.83; 95% CI 0.76 –0.89), comparable to the developmental study.

CONCLUSION

The results validated the utility of the tool for allowing clinicians to quickly and accurately identify an individual’s risk of an impending fall.

1. Introduction

Falls among individuals with Parkinson disease (PD) are prevalent, frequently recurrent, and disabling [1]. Fall incidence generally increases with disease progression, especially from early to middle stages. Commonly identified fall risk factors include a past history of falls, freezing of gait (FOG), impaired balance, and orthostatic hypotension [2]. Nonetheless there is a wide range of frequency of falling, and disease severity does not appear to be an accurate predictor of a future fall [2]. Thus, although much is known about falls and fall risk in PD, a clinician’s ability to accurately predict the absolute risk of an impending fall for an individual patient remains a significant challenge.

Currently a variety of standardized balance assessment tools like the Functional Gait Assessment [3] and Mini-Balance Evaluation Systems Test [4] are used to predict the risk of future falls in individuals with PD. While these measures have demonstrated relatively high accuracy for predicting falls (as measured by the area under the receiver operating characteristic curve (AUC) ≥0.80), they can be time-consuming and require specialized equipment. Recently, Paul and colleagues developed a simple clinical prediction tool based only on history of at least one fall in the past year, FOG in the past month, and gait speed <1.1 m/s [5]. The tool, which easily could be adopted in routine patient care, discriminated near term (i.e. 6 month) future fallers with high accuracy (AUC, 0.80; 95% CI 0.73–0.86).

Clinical prediction tools need to be externally validated to ensure their generalizability, accuracy, and clinical utility [6]. The clinical prediction tool [5] was internally validated using a sample of individuals with PD. The purpose of the present study was to externally validate the tool in a different cohort of persons with PD [7]. We hypothesized that the tool would demonstrate high accuracy in discriminating future fallers in the longitudinal study, comparable to that in the original developmental study [5].

2. Participants and methods

Participants selected for the external validation study were enrolled in a 2-year multicenter longitudinal cohort study designed to monitor the progression of disability and quality of life [7]. Institutional review board approval was obtained at each participating institution, and all participants provided written and informed consent. Community-dwelling individuals over age 40 were included if they had been diagnosed by a neurologist with idiopathic PD, determined to be between Hoehn & Yahr Stages I-IV (mild to moderatel disease severity), and scored ≥ 24 on the Mini-mental State Examination. Individuals were excluded if they had been diagnosed with atypical parkinsonism or had previous surgical intervention specifically for PD (e.g. deep brain stimulation). Assessments were conducted at 6-month intervals for a total of 24 months. All assessments were performed by a physical therapist at the University of Utah, Boston University, Washington University in St. Louis, or University of Alabama at Birmingham. Participants were assessed in the “on” state, defined as 1–2 to hours following anti-PD medication administration. Demographic information, PD profile and severity of motor signs were collected at baseline and quantified using the motor section of the Movement Disorders Society Unified Parkinson’s Disease Rating Scale (MDS-UPDRS-III).

To validate the results of the fall prediction tool in the original developmental sample [5], we used data collected at baseline, 6 months, and 12 months. For the first predictor variable, i.e. the occurrence of at least one fall during the previous year, we combined retrospective 6-month fall history data from the baseline and 6-month assessments [6]. Fall history was determined using a forced-response paradigm, in which choices included none, once, 2–10 times, weekly, or daily. Falls were defined as unintentionally coming to rest on the ground or other lower surface without being exposed to overwhelming external force or a major internal event. For the second and third predictor variables, we used FOG and gait speed data that were collected at the 6-month assessment only. FOG was defined as sudden and transient motor blocks when initiating or continuing walking [9]. In concert with Paul et al. [5], FOG was determined using the answer to Question 3 of the FOG Questionnaire (FOG-Q) [8], which asks “Do you feel that your feet get glued to the floor while walking, making a turn, or when trying to initiate walking?” Participants reporting an answer greater than or equal to 1, which indicates experiencing freezing at least once monthly, were classified as having a positive FOG history. Self-selected gait speed <1.1 m/s was determined using the mean of two gait speed trials collected during a 10-m walk. The 10-m walk was conducted in a wide hallway and the time (seconds) to complete the walk was recorded using a stopwatch. Participants were allowed to use assistive devices if necessary. For the outcome variable, the occurrence of at least one fall during the 6-month period immediately following the assessment of predictor variables, we used 6-month fall history that had been collected at the 12-month assessment.

2.1. Data analysis

The external validation analysis included only those participants with complete fall history data. Among those participants, there were no missing FOG or gait speed data. The faller and non-faller groups were compared on demographic, PD profile, and physical performance measures using t-tests for independent samples (α = 0.05). A logistic regression model was used to evaluate how well the three predictor variables (i.e. at least one fall in the past year (yes/no)), freezing of gait within the past month (yes/no), and self-selected gait speed <1.1 m/sec (yes/no) predicted the occurrence of at least one fall (yes/no) during the 6 months following predictor assessment. Model assumptions were examined without evidence of significant violations. Calibration, or distribution of predicted risk, was examined using the Hosmer–Lemeshow test (Table 2). A chi-square (χ²) contingency table and AUC were used to determine the discriminative ability of the clinical prediction tool and to calculate sensitivity and specificity. All analyses were performed with SPSS version 19 (IBM Corporation, NY).

Table 2.

Multivariate model coefficients and predicted probability of falling according to risk category for validation sample and development sample.

Predictor variable	Validation sample		Development sample
	Regression coefficient	OR (95% CI)	Regression coefficient	OR (95% CI)
Fell in previous 12 mo, yes/no	2.83 (1.84, 3.81)	16.90 (6.30, 45.00)	1.76 (1.10, 2.42)	5.80 (3.00, 11.22)
FOG in past mo, yes/no	0.05 (30.76, 0.88)	1.06 (0.47, 2.40)	0.87 (0.17, 1.60)	2.39 (1.19, 4.80)
Self-selected gait speed <1.1 m/s, yes/no	0.87 (0.07, 1.67)	2.38 (1.07, 5.31)	0.62 (30.04, 1.27)	1.86 (0.96, 3.58)
Risk categorya	Number of participants who fell/total number of participants scoring in this category	Predicted (actual) probability of falling, %	Number of participants who fell/total number of participants scoring in this category	Predicted (actual) probability of falling, %
Low (0)	4/47	1 (9)	8/43	17 (19)
Med (>0 to <8)	15/53	30 (28)	36/73	51 (49)
High (>8)	47/71	66 (66)	76/89	85 (85)

^aWeights are assigned to each of the three predictor variables based on the regression coefficients of the 3-predictor regression (ie, falling in past month = 6, freezing of gait in the past month = 3, gait speed <1.1 m per second = 2). The sum of the predictor weights give a maximum score of 11 and a minimum score of 0. Probability of falling in the next 6 months is then classified (based on the sum of the predictor weights) as low (0), moderate (>0 to < 8) or high (≥8) [7].

3. Results

The external validation cohort included 171 participants with PD (Table 1). Sixty-six (38.6%) participants reported at least one fall during the 6 months following predictor assessment. Additional baseline variables stratified by fallers vs. non-fallers in the 6 months following predictor assessment are presented in Table 1.

Table 1.

Baseline characteristics of fallers and non-fallers.

Variable	Non-fallers (n = 105)	Fallers (n = 66)	p-value
Demographics and PD profile
Women, n (%)	44 (42)	30 (46)	0.38
Age, years	65.51 (9.13)	68.55(9.53)	0.04
Body mass index (kg/m2)	27.34 (5.70)	27.62 (7.88)	0.79
Physical activity scale for the elderly; Range 0–361	165.46 (88.18)	114.73 (81.36)	0.00
Any fall in previous 12 months, n (%)	36 (34)	60 (91)	0.00
Parkinson’s disease duration, years	4.75 (3.81)	6.56 (4.21)	0.00
MDS-Unified Parkinson’s disease rating: motor score; Range, 0–132	29.96 (10.79)	35.00 (15.36)	0.01
FOG in past month, n (%)	38 (36)	42 (64)	0.00
FOG severity; Range, 0–16*	2.01 (3.11)	4.56 (4.17)	0.00
Physical performance measures
Functional gait assessment (0–30)	22.59 (4.96)	17.62 (6.65)	0.00
Timed up and go (seconds)	9.83 (3.95)	13.30 (7.76)	0.00
10m walk: self selected speed (m/s)	1.26 (0.23)	1.10 (0.24)	0.07
10m walk <1.1 m/s	26 (25)	34 (52)	0.00
10m walk: max speed (m/s)	1.71 (0.37)	1.45 (0.35)	0.03
6min walk (meters)	494.75 (112.31)	392.74 (95.44)	0.00

Values are mean (SD) unless otherwise indicated.

Significant (p < 0.05) univariate comparisons in bold.

FOG in past month = yes/no based on question 3 of the FOGQ.

FOG severity was calculated as the sum of items 3–6 on the FOGQ.

Similar to the developmental sample [7], a test of predictor variables against a constant-only model distinguished 6-month future fallers from non-fallers with high accuracy (χ² [3, n = 171] = 64.34, p < 0.001; AUC = 0.83 [95% CI 0.76–0.89]), sensitivity = 90.9% (95% CI 81.30–96.60), specificity 65.7% (95% CI 55.80–74.70). The result generated a positive likelihood ratio of 2.65 (95% CI 2.10–3.49) and a negative likelihood ratio of 0.14 (95% CI 0.05–0.30). A comparison of predictor variable data from each study appears in Table 2.

Risk categories for the external validation sample were calculated based on the proposed scoring criteria from Paul et al. [5] (Table 2). The validation sample had a similar risk gradient as the development sample (Table 2 and Supplemental Digital Content) and calibration on average was correct. The Hosmer–Lemeshow test indicated no lack of fit between predicted and observed fallers (χ² = 2.5 (df = 5, p = 0.77)). The largest discrepancy between the external validation sample and the developmental sample was noted in the relatively lower estimated probability of falling for the low risk category (n = 4).

4. Discussion

This study externally validated the accuracy of the fall prediction tool by meeting and exceeded the accuracy reported in the developmental study [5,6,9]. The results supported the assertion that the clinical prediction tool allows for accurate identification of individuals with PD who are at high risk of an impending fall. Such information should serve as a starting point for in-depth examination of additional risk factors (e.g. physiologic; cognitive) and may facilitate the rapid implementation of fall prevention strategies [2]. The simple tool takes less than five minutes to administer. Fall and FOG history can be gathered on patient intake forms prior to seeing the provider. Preferred gait speed can be easily collected by timing (in seconds) how long it takes a patient to walk a short distance (e.g., 10m) and reported (in m/s) similarly to other vital signs (e.g. heart rate, blood pressure). (See Supplemental Digital Content detailing the prediction tool).

Our findings confirmed that the occurrence of at least one fall in the past year is a strong predictor of future falls and emphasized the need for clinicians to screen for risk by asking all patients this simple question [10]. Our findings also highlighted the relevance of gait speed below 1.10 m/sec as a strong predictor of future falls. Our results, however, did not confirm the independent contribution of FOG to future falls. The latter finding may have been due to the different proportion of fallers in our sample compared to the development sample [5].

The largest discrepancy between our findings and those of the developmental study pertained to the relatively lower predictive accuracy of the tool for individuals with a low risk of an impending fall (Table 2). One reason for the discrepancy could have been related to differences in sample characteristics. Our sample contained more individuals in earlier stages of the disease with lower disease severity. This difference warrants further investigation of the fall risk prediction tool in individuals identified at low fall risk for whom evolution to a higher fall risk category is uncertain.

The findings presented herein should be interpreted within the context of study limitations. First, our method of recording fall history had the potential to be less accurate than fall history collected using a diary. Although there are concerns of potential inaccuracy of relying on patient self-report of falls as opposed to a falls diary, patient recall is the current standard of care for gathering this data in clinical practice [11]. The necessity of this information is reflected in current position statements and guidelines for quality care in PD [10–12]. Future research should investigate the impact of the tool in clinical practice as well as examine the potential benefit of adding a history of recurrent falls to the accuracy of the fall prediction tool [9].

Appendix A. Supplementary data

Supplementary data related to this article can be found at http://dx.doi.org/10.1016/j.parkreldis.2015.05.008.