Temporal External Validation of the TWIST Prediction Tool for Time to Independent Walking after Stroke

Abstract

Background.

The TWIST tool uses clinical measures at 1 week post-stroke to predict whether patients will achieve independent walking by 4, 6, 9, 16 or 26 weeks post-stroke. TWIST was developed with 93 participants from 2 hospitals and was at least 83% accurate at all timepoints. Validation of TWIST is necessary before implementation into clinical practice.

Objective.

The study aimed to perform temporal external validation of the TWIST prediction tool in a single hospital site.

Methods.

Adults with stroke were recruited if they had new lower limb weakness and were unable to walk independently. A TWIST score out of 4 was calculated for each participant using age, knee extension strength and Berg Balance Test score at 1 week post-stroke. Discrimination and calibration of TWIST were calculated for each timepoint and the model was refined as required.

Results.

There were 89 participants in this validation cohort (43% women, median age 69 years, 56% moderate-severe stroke). Predictions were valid for most TWIST scores at most timepoints. The TWIST tool was too optimistic for participants with TWIST scores of 0 and 2 at 16 and 26 weeks, and a TWIST score of 3 at all timepoints. The TWIST tool and clinical interpretations were refined. Discrimination of the TWIST tool is good (C-statistic > 0.8) for all timepoints.

Conclusions.

TWIST is a temporally externally validated prediction tool that combines routine tests at 1-week post-stroke to predict the probability of an individual patient achieving independent walking by 4, 6, 9, 16 or 26 weeks post-stroke.

Introduction

Stroke affects almost 12 million people worldwide each year, ¹ with almost two-thirds experiencing difficulty walking early after stroke. ² The likelihood of regaining independent walking is an important consideration for clinicians, patients and families when planning rehabilitation and discharge location.^3,4 Information about how soon a patient is expected to achieve independent walking is particularly relevant for anticipating length of inpatient stay and requirements for family support after discharge. However, patients and families report a lack of prognostic information after stroke, ⁵ and clinicians express reluctance to provide prognostic information due to concerns around accuracy of the information. ⁶

Several well-known factors have strong relationships with walking outcomes post-stroke,^7-14 and clinicians can accurately predict whether a patient will achieve independent walking or not after stroke.^15-17 However prediction accuracy is poor when clinicians are asked to identify how long it will take to achieve independent walking,^15-17 regardless of clinician confidence or years of post-stroke experience. ¹⁵ This low prediction accuracy suggests that clinicians are not able to apply their knowledge systematically to make accurate predictions for individual patients. Prediction tools are designed to bring prediction information into clinical practice through a structured, step-by-step format to support clinician decision making. ¹⁸ The EPOS tool is the only validated walking prediction tool to date. ¹⁹ EPOS was developed to predict independent walking at a single timepoint of 6 months post stroke and was validated for 3 months post-stroke. Knowing whether independent walking will be achieved at timepoints earlier or later than 3 months post-stroke would also assist discharge planning.

The Time to Walking Independently after STroke (TWIST) prediction tool was developed with a sample of 93 people from 2 hospital sites using simple bedside measures at 1 week post-stroke. ¹¹ TWIST combines age (< 80 years, ≥ 80 years), balance (Berg Balance Test score < 6, 6-15, ≥ 16) and knee extension strength (Medical Research Council (MRC) grade < 3, ≥ 3) to generate a TWIST score out of 4 which is then used to predict whether a patient will achieve independent walking by 4, 6, 9, 16 or 26 weeks post-stroke (Figure 1). ¹¹ TWIST predictions are at least 83% accurate at each timepoint, ¹¹ outperforming physiotherapist predictions for patients who achieved independent walking within the first 3 months and similar to physiotherapist predictions at 4 and 6 months. ¹⁵

Figure 1.

TWIST prediction tool. Age, knee extension strength at 1 week post-stroke and Berg Balance score at 1 week post-stroke are combined for a TWIST score out of 4. First published in Neurorehabilitation and Neural Repair. ¹² TWIST, Time to walking independently after Stroke.

The aim of this study was to perform a temporal external validation and refinement of the TWIST tool in a new cohort. We anticipated that the performance of the TWIST tool would be lower in this validation cohort than in the original development cohort, prompting refinement of the clinical interpretation of TWIST predictions.

Methods

A single site prospective observational study was conducted. This was a temporal external validation study which means the validation took place with a new cohort at a different time point, but at the same center as the development cohort. Patients were consented into the study between 1^st January 2021 and 18th September 2024 from an integrated stroke unit at a large urban hospital in Auckland, New Zealand. All patients admitted to the stroke unit over this period were considered for inclusion. Recruitment was halted for 6 months from August 2021 due to a COVID-19 lockdown limiting research access to the stroke unit. Follow-up assessments were not affected by COVID-19. Study reporting follows the STROBE and TRIPOD+AI guidelines.

Participants

Adults aged 18 years and older within 1 week of new ischemic or hemorrhagic stroke were recruited for the validation cohort. The inclusion criteria were the same as the development cohort: new lower limb weakness (Medical Research Council (MRC) ²⁰ score less than 5/5 in at least one lower limb muscle group) and unable to walk independently at 1 week post-stroke (Functional Ambulation Categories (FAC) ²¹ score less than 4). Previous stroke was allowed if there was no residual lower limb weakness, and the patient was walking independently prior to the new stroke. Patients were excluded if they had cerebellar or bilateral stroke, used a walking frame pre-stroke, had a pre-stroke FAC score less than 4, had significant communication or cognitive deficits precluding supported informed consent, were medically unwell at the time of recruitment, or had a life expectancy less than 6 months. Patients who were transferred to an out-of-area hospital prior to assessments at 1 week post-stroke were also excluded. Patients with communication or cognitive impairments were provided with an aphasia-friendly participant information sheet, had family present at time of consent, and supported communication strategies were used to maximize participation of this cohort in the study.

The study was approved by the National Health and Disability Ethics Committee. All participants provided written informed consent in accordance with the Declaration of Helsinki. Anonymized data will be provided upon reasonable request. Target sample size was 100 participants based on the TWIST development sample size of 93 participants and allowing for attrition. ¹¹

Clinical and Demographic Data

Demographic and stroke characteristics were recorded at the time of consent (Table 1). Stroke severity was evaluated at the time of consent with the National Institutes of Health Stroke Scale (NIHSS). ²² The presence or absence of aphasia and unilateral neglect were recorded using the relevant NIHSS items. The following assessments were completed at 1 week post-stroke: Walking ability (FAC), ²¹ knee extension strength (MRC grades), ²⁰ global lower limb impairment (LL Motricity Index) and sitting and standing balance (Berg balance scale). ²³ Assessments were selected to enable calculation of the TWIST score ¹¹ and for characterization of the sample.

Table 1.

Demographic and Stroke Characteristics Development, Validation and Combined Cohorts.

Demographic characteristics n (%)	Development cohort	Validation cohort	Combined cohort
	n = 93 (100)	n = 89 (100)	n = 182 (100)
Age (years)
Median age (range)	71 (38-97)	69 (23-95)	70 (23-97)
Sex
Female	36 (39)	38 (43)	74 (41)
Ethnicity
European	65 (70)	42 (48)	107 (59)
Māori	8 (8)	9 (10)	17 (9)
Pacific	10 (11)	9 (10)	19 (10)
Asian	10 (11)	27 (30)	37 (20)
MELA (Middle East, Latino)	0 (0)	2 (2)	2 (1)
Co-morbidities (CCI)
Low (CCI < 2)	77 (83)	70 (81)	144 (81)
Stroke characteristics
Consent days post-stroke median (range)	3 (1-8)	4 (1-8)	4 (1-8)
Hemisphere
Right	53 (57)	51 (57)	103 (57)
Stroke type (Oxfordshire)
Total anterior circulation infarct (TACI)	17 (18)	15 (17)	32 (18)
Partial anterior circulation infarct (PACI)	14 (15)	20 (23)	34 (19)
Lacunar infarct (LACI)	34 (37)	36 (40)	70 (38)
Posterior circulation infarct (POCI)	14 (15)	6 (7)	20 (11)
Intracerebral hemorrhage	14 (15)	12 (13)	26 (14)
Stroke location
Subcortical only	62 (67)	44 (49)	106 (58)
Thrombolysis	16 (17)	15 (17)	31 (17)
Clot retrieval	3 (3)	13 (15)	16 (9)
Stroke severity (NIHSS)
NIHSS median (range)	8 (1-24)	7 (1-22)	7 (1-24)
Moderate to severe (NIHSS ≥ 7)	56 (60)	50 (56)	106 (58)
Visuospatial neglect
YES	29 (31)	34 (38)	63 (35)
Aphasia
YES	16 (17)	18 (20)	34 (19)
LL strength
MRC Knee extension median (range)	4 (0-5)	3 (0-5)	3 (0-5)
LL Motricity Index median (range)	59 (1-100)	47 (1-100)	53 (1-100)
Postural control
Berg balance score median (range)	7 (0-54)	15 (0-48)	9 (0-54)
1 week FAC score (0-5)
FAC median (range)	0 (0-3)	0 (0-3)	0 (0-3)
Non-ambulatory (FAC = 0)	69 (74)	51 (57)	120 (66)
Dependent ambulation (FAC 1, 2, 3)	24 (26)	38 (43)	62 (34)
Length of stay days median (range)	33 (7-93)	35 (2-129)	35 (2-129)
Therapy total minutes median (range)	607 (44-2830)	544 (10-2948)	587 (10-2948)
Therapy minutes per day median (range)	29 (9-66)	23 (3-97)	27 (3-97)
TWIST score n (%)
0	6 (7)	3 (3)	9 (5)
1	31 (33)	24 (27)	55 (30)
2	14 (15)	17 (19)	31 (17)
3	17 (18)	20 (23)	37 (20)
4	25 (27)	25 (28)	50 (28)
Timepoint of independent walking n (%)
4	30 (32)	28 (32)	58 (32)
6	15 (16)	17 (19)	32 (18)
9	12 (13)	12 (13)	24 (13)
16	13 (14)	10 (11)	23 (13)
26	5 (5)	5 (6)	10 (5)
Dependent	18 (20)	17 (19)	35 (19)

Abbreviations: CCI, Charlson Comorbidity Index; NIHSS, National Institutes of Health Stroke Scale; LL, lower limb; MRC, Medical Research Council grade; FAC, Functional Ambulation Category.

Assessment of Walking Outcomes

The FAC scale (0-5) was used to assess walking ability at 1 week post-stroke and repeated at 4, 6, 9, 16 and 26 weeks post-stroke to match the timepoints of the TWIST prediction tool. The use of FAC for the TWIST tool has been reported in detail previously.^11,12 Independent walking was defined as FAC ≥ 4, indicating that the participant can walk on a flat surface without physical or verbal assistance.^11,19 This definition identifies a person able to walk independently indoors at home, which is an important consideration for discharge. Participants were allowed to use a stick, quad stick or ankle support/splint. The use of a walking frame for FAC assessment was not allowed due to the requirement for bilateral upper limb use and potential bias created by individual physiotherapists’ preference for timing of walking frame use. ¹¹ If a participant was using a walking frame at the time of the assessment, they were asked to complete the assessment without it. The primary outcome was week post-stroke by which independent walking (FAC ≥ 4) was achieved.

Follow-up FAC assessments were conducted either in person or by phone using a standardized interview schedule and clearly defined parameters for FAC score. ¹¹ Information obtained from phone consultations about walking ability was verified with a family member, staff at the residential care facility, or the participant’s community rehabilitation physiotherapist. All follow-up assessments were completed by experienced research physiotherapists and occupational therapists who were blinded to the baseline assessment results and TWIST prediction score and were not involved in patient care. Therapists involved in routine clinical care were also blinded to the results of the research assessments.

Inpatient Therapy Measurement

Therapy dose and intensity were collected to characterize the rehabilitation environment for this cohort. The treating therapist recorded time spent in active rehabilitation immediately after each lower limb rehabilitation session. Therapy dose was defined as the total number of minutes of lower limb rehabilitation between inpatient admission and discharge. Therapy intensity was defined as minutes of lower limb rehabilitation per available therapy day (excluding weekends and holidays).

Data Analysis

Participants’ data were removed from the analysis if they died, withdrew, or had a significant medical event affecting rehabilitation or walking ability prior to achieving independent walking. If a follow-up assessment was missed after the participant had achieved independent walking (FAC ≥ 4), the FAC score from the previous timepoint was carried forward. If a follow-up assessment was missed prior to achieving independent walking and the participant remained dependent (FAC < 4) at the subsequent assessment timepoint, the participant was categorized as “dependent” for the missed assessment. The participant data in this study forms the validation cohort. The development cohort refers to the previously reported TWIST prediction tool study. ¹¹

A TWIST score out of 4 was calculated for each participant based on age, MRC knee extension strength grade and Berg Balance Test score at 1 week post-stroke (Figure 1). Each participant was categorized as independent (FAC ≥ 4) or dependent (FAC < 4) at each follow-up timepoint (4, 6, 9, 16 and 26 weeks post-stroke). The percentage of participants who were independent at each timepoint for each TWIST score was calculated. The probability of achieving independent walking for each timepoint for each TWIST score are reported previously ¹¹ and in Table 2.

Table 2.

Predicted and Actual Probabilities of Participants Achieving Independent Walking at Each Timepoint for Each TWIST Score.

TWIST score	4 wk (%)			6 wk (%)			9 wk (%)			16 wk (%)			26 wk (%)			Dependent (%)
	P	A	C	P	A	C	P	A	C	P	A	C	P	A	C	P	A
0	1	0		1	0		3	0		11	0	11	20	0	11*	80	100
1	3	8		7	12		21	17		49	46		62	67	64	38	33
2	13	12		35	35		68	65		88	77	87	92	77	87*	8	23
3	43	25	27*	81	65	68*	95	85	87*	98	90	92*	99	90	92*	1	10
4	79	76		97	92		99	100		100	100		100	100		0	0

Abbreviations: P, predicted; A, actual; C, outcome data from development and validation cohorts combined.

Values in bold have a 95% confidence interval that does not cross the calibration line in the validation cohort and represents significant over or under-prediction. Values with asterisk represent combined observed probabilities that are 5 or more percentage points from predicted probabilities and subsequently used in refined TWIST tool.

Overall fit of the prediction tool for each timepoint was calculated with the Brier Score which examines the mean squared difference between the observed outcomes and the predicted probabilities. ²⁴ Brier scores range from zero (perfect fit) to 1 with scores closer to zero representing better overall fit of the prediction tool. Receiver operating characteristics (ROC) curves were generated for each timepoint with a 95% confidence interval. Discrimination (C-statistic) of the TWIST prediction tool was determined using the area under the ROC curve, with a C-statistic of ≥ 0.8 considered “good” and ≥ 0.9 considered “excellent.” ²⁵

Calibration plots with 95% confidence intervals were generated to visualize the agreement between predicted ¹¹ and observed probabilities for each TWIST score at each timepoint post-stroke. If the 95% confidence interval did not cross the calibration line, the corresponding TWIST score was considered to either over- or under-predict outcomes for that timepoint. In this situation, the outcome data for the relevant TWIST score and timepoint were combined across both the development and validation cohorts. The TWIST tool was refined if the observed probability in the combined cohort differed by 5 or more percentage points from the predicted probability. ¹¹ The combined cohort was used to refine the TWIST tool rather than the validation cohort alone to reduce over-fitting of the refined tool. ²⁶

Discrimination and Brier Score of the refined tool were re-calculated and calibration plots updated. The suggested clinical interpretation of the TWIST tool was refined if required.

Results

Ninety-eight participants were recruited with data from 89 retained for analysis (43% women, median age 69 (23-95) years). Over half of participants (57%) were non-ambulatory at 1 week post-stroke (FAC = 0), and 56% had moderate-severe stroke based on NIHSS score (Table 1). Baseline assessments were completed for all participants at median 7 days post-stroke (4-9 days). Of the 9 participants excluded from analysis, 1 became independent between study enrollment and 1 week post-stroke, 6 died before achieving independent walking and 2 were withdrawn due to medical events (second stroke, brain surgery, (Supplemental Figure 1)). Six trained and blinded assessors conducted the follow-up assessments.

Fifty-seven participants (64%) achieved independent walking by 9 weeks post-stroke and 15 (17%) between 9 and 26 weeks post-stroke. The remaining 17 participants (19%) were still dependent at 26 weeks post-stroke (Table 1). Predicted and observed probabilities of independent walking at each timepoint for each TWIST score are reported in Table 2. Overall fit of the TWIST tool prediction for each timepoint was good with Brier Scores ranging from 0.11 to 0.15 (Table 3).

Table 3.

Discrimination and Calibration of the TWIST Prediction Tool in the Development, Validation and Combined Cohorts.

Timepoint post-stroke	Development cohort % (95% CI) 11	Validation cohort % (95% CI)	Combined cohort % (95% CI) Refined TWIST tool
4 wk
Calibration slope	0.98	0.88	0.92
Discrimination (C-statistic)	0.90 (0.83-0.96)	0.84 (0.75-0.93)	0.86 (0.78-0.94)
Brier score		0.14	0.13
6 wk
Calibration slope	0.97	0.86	0.89
Discrimination (C-statistic)	0.94 (0.89-0.98)	0.86 (0.79-0.94)	0.86 (0.78-0.94)
Brier score		0.15	0.14
9 wk
Calibration slope	1.02	0.99	1.04
Discrimination (C-statistic)	0.92 (0.87-0.98)	0.91 (0.85-0.97)	0.86 (0.78-0.94)
Brier score		0.11	0.11
16 wk
Calibration slope	0.98	1.04	1.05
Discrimination (C-statistic)	0.89 (0.82-0.96)	0.86 (0.78-0.94)	0.86 (0.78-0.94)
Brier score		0.13	0.13
26 wk
Calibration slope	1.05	1.11	0.97
Discrimination (C-statistic)	0.87 (0.80-0.95)	0.81 (0.71-0.91)	0.81 (0.71-0.91)
Brier score		0.12	0.12

Discrimination of TWIST

Discrimination of the TWIST tool was good for predicting independent walking by 4, 6, 16 and 26 weeks post-stroke with a C-statistic of >0.8, and excellent for 9 weeks post-stroke with a C-statistic of 0.91 (Table 3).

Overall Calibration of TWIST

Model calibration was good for timepoints 4, 6 and 9 weeks post-stroke for TWIST scores 0, 1, 2 and 4 (Figure 2). TWIST scores 0 and 2 were too optimistic for timepoints 16 and 26 weeks, and TWIST score 3 was too optimistic at all timepoints (Figure 2).

Figure 2.

Calibration plots. Calibration plots representing the predicted and observed probability of achieving independent walking at each time point for each TWIST score with a 95% confidence interval. The 5 data points represent TWIST score 0 to 4. The dotted line represents perfect calibration. If the 95% CI bar does not cross the dotted line, this indicates significant over- or underprediction. The second column of plots represents calibration of the refined TWIST tool.

Calibration and Refinement of TWIST Score 0

Three participants (3%) had a TWIST score of 0. None of these participants achieved independent walking within 26 weeks. The predicted and observed probabilities of participants walking independently were well calibrated at 4, 6 and 9 weeks post-stroke but predictions were too optimistic at 16 and 26 weeks (Figure 2, Table 2). Participants with a TWIST score of 0 in the development and validation cohorts were combined for a total of 9 participants. One participant (11%) was independent by 16 weeks, and no participants with a TWIST score of 0 achieved independent walking between 16 and 26 weeks. The predicted and observed probabilities of participants walking independently in the combined cohort were within 5% points at 16 weeks (predicted 10%, observed 11%) but not at 26 weeks (predicted 20%, observed 11%). Therefore, the predicted probability of a participant with a TWIST score of 0 achieving independent walking by 26 weeks was updated from 20% to 11% (Table 2, Figure 3). The clinical interpretation for a TWIST score of 0 was revised to include the following statement:

“About 9 out of 10 people will still need help to walk at 6 months.” (Figure 3)

Figure 3.

Refined TWIST tool and clinical interpretation. (A) Probability of achieving independent walking for each TWIST score and each timepoint post-stroke (%). Updated probabilities in bold. (B) Suggested clinical interpretation for each TWIST score.

Calibration of TWIST Score 1

Twenty-four participants (27%) had a TWIST score of 1. The predicted and observed probabilities of participants walking independently were well calibrated at all timepoints (Figure 2, Table 2). The TWIST tool is valid for people with TWIST score of 1 and no changes were made to the clinical interpretation of the TWIST tool (Figure 3).

Calibration and Refinement of TWIST Score 2

Seventeen participants (19%) had a TWIST score of 2. The predicted and observed probabilities of participants walking independently were well calibrated at 4, 6 and 9 weeks post-stroke but predictions were too optimistic at 16 and 26 weeks (Figure 2, Table 2).

Participants with a TWIST score of 2 in the development and validation cohorts were combined for a total of 31 participants. Twenty-seven participants in the combined cohort (87%) achieved independent walking by 16 weeks. No participants achieved independent walking between 16 and 26 weeks. The predicted and observed probabilities in the combined cohort were within 5 percentage points at 16 weeks (predicted 88%, observed 87%), but not at 26 weeks (predicted 92%, observed 87%). Therefore, the predicted probability of a participant with a TWIST score of 2 achieving independent walking by 26 weeks was revised from 92% to 87% (Table 2, Figure 3). No changes were required for the clinical interpretation of the TWIST tool (Figure 3).

Calibration and Refinement of TWIST Score 3

Twenty participants (23%) had a TWIST score of 3. The TWIST predictions were too optimistic at all timepoints (Table 2). Participants with a TWIST score of 3 in the development and validation cohorts were combined for a total of 37 participants. Ten participants (27%) achieved independent walking by 4 weeks, 25 (68%) by 6 weeks, 32 (87%) by 9 weeks and 34 (92%) by 16 weeks post-stroke. No participants with a TWIST score of 3 achieved independent walking between 16 and 26 weeks post-stroke. In the combined cohort, the TWIST predictions remained too optimistic at all timepoints (Table 2). The predicted probabilities of a participant with a TWIST score of 3 were revised to reflect the combined observed probabilities at each timepoint (Table 2, Figure 3). The clinical interpretation was refined for the TWIST score of 3 from the original interpretation of:

“Around half of people achieve independent walking within a month, and 4 out of 5 people achieve independent walking by 6 weeks post-stroke. Almost everyone with this score will achieve independent walking by 9 weeks post-stroke.” ¹¹

To the revised interpretation to reflect the probabilities in the combined cohort:

“Around 2 out of 3 people will achieve independent walking by 6 weeks post-stroke and 9 out of 10 will achieve independent walking by 9 weeks post-stroke.” (Figure 3).

Calibration of TWIST Score 4

Twenty-five participants (28%) had a TWIST score of 4. The predicted and observed probabilities of participants walking independently were well calibrated at all timepoints. The TWIST tool is valid for people with TWIST score of 4 and no changes were made to the clinical interpretation (Figure 3).

Discrimination and Calibration of the Refined TWIST Tool

The refined TWIST tool and clinical interpretation are presented in Figure 3. There was no change in model discrimination after the TWIST tool was refined, with “good” to “excellent discrimination (Table 3). Updated calibration plots are presented in Figure 2. The refined TWIST tool has good calibration for predictions of independent walking by 4, 6 and 9 weeks post-stroke for all TWIST scores. TWIST predictions err on the side of optimism for participants with a TWIST score of 2 at 16 and 26 weeks post-stroke. Overall fit of the revised TWIST tool at each timepoint was good with Brier scores ranging from 0.11 to 0.14.

Discussion

This is the first study to validate and refine the TWIST walking prediction tool. TWIST combines age, knee extension strength and Berg Balance Test score at 1 week post-stroke to generate a TWIST score which is then used to predict the likelihood of an individual patient achieving independent walking by 4, 6, 9, 16 or 26 weeks post-stroke. Discrimination of the TWIST tool is good to excellent at all timepoints (C-statistic > 0.8). The predictions were well calibrated for most TWIST scores at most timepoints (Figure 2). The TWIST tool was too optimistic for participants with a TWIST score of 0 and 2 at 16 and 26 weeks, and a TWIST score of 3 at all timepoints. The refined TWIST tool and clinical interpretation for these scores and timepoints reflect a more conservative prediction (Figure 3).

The refined TWIST tool had strong predictive performance at 4, 6 and 9 weeks post-stroke, with lower performance at 16 and 26 weeks post-stroke. It is possible that patients who take longer than 9 weeks to achieve independent walking have other factors influencing their walking recovery in addition to those used in the TWIST tool. ¹³ Factors such as cognition, visuospatial neglect, sensory or communication impairments may play a role in slower achievement of independent walking than baseline physical capacity may suggest. Further research could explore whether inclusion of additional baseline factors would improve predictive performance at 16 and 26 weeks post-stroke.

There are some notable differences between the development cohort reported previously ¹¹ and this validation cohort (Table 1). The median Berg Balance Test score was higher and the percentage of participants who were non-ambulatory with FAC 0 at 1-week post-stroke was lower in the validation cohort. The validation cohort also had a different ethnic composition with 48% European (70% in development cohort) and 30% Asian (11% in development cohort). This reflects changing regional demographics and may have contributed to a slightly lower median age. Despite these differences, the percentages of participants with each TWIST score, and those achieving independent walking at each timepoint, were similar between cohorts and to previous work.^13,27 The use of multiple blinded assessors to evaluate FAC at follow-up timepoints demonstrates the robustness of both the prediction tool and the FAC assessment.

Predicting not just whether a patient will achieve independent walking at a single timepoint post stroke but how long it will take to achieve independent walking provides important information for both clinicians and patients. In the current study, 32 participants (36%) had not yet achieved independent walking at 9 weeks post-stroke. Of these, almost half (n = 15, 47%) became independent between 9 weeks and 6 months and the rest remained dependent at 6 months post-stroke. These findings are consistent across both the development and validation cohorts and highlight the value of multiple outcome timepoints in walking prediction models. Prediction of independent walking at a single timepoint of 3 months post-stroke such as in the EPOS validation study ¹⁹ risks providing unnecessarily pessimistic prognoses for those patients who take longer than 3 months to achieve independent walking. Conversely, a single longer timeframe such as 6 months increases the likelihood of accurate predictions but the tool becomes less clinically useful.

Validation of the TWIST prediction tool is the first step toward implementation in clinical practice, and there are further steps to be taken. This study was a temporal validation of the tool with a new cohort at a later point in time, but in the same clinical setting as the development cohort.^25,26 A geographical external validation in a different clinical setting, preferably in a different country, would support the generalizability of the tool across different types of healthcare delivery settings.^25,26 The suggested clinical interpretation of the TWIST scores in Figure 3 remains relatively rudimentary. The prediction tool itself is simple, but skill is required for clinicians to use prediction information in a way that is beneficial and not harmful to patients and health services. Further work ought to include engaging with patients, families and clinicians to explore perspectives on prediction information, what format this information should be provided in, by whom and when.

Barriers and facilitators to implementing prediction tools into clinical practice have been explored for the upper limb^28,29 but not yet for walking prediction tools. In previous work, the implementation of the PREP2 upper limb prediction tool changed the type but not the amount of rehabilitation the participants received. ³⁰ This was an important finding as it identified that having a “poor” prediction did not result in rationing of care. To the best of our knowledge, no studies have explored the effects of walking prediction tools on clinical practice. This future work is important to ensure that prediction tools are used as intended, recognizing that recovery of independent walking is not the only important mobility milestone after stroke and patients with a “poor” prognosis may require more rehabilitation, not less.

This study had several limitations. Firstly, the validation cohort was in one of the same hospitals as the development cohort. Although the development cohort had 2 hospital sites, both were in the same city and there is likely to be considerable overlap in clinical practice. Future work ought to externally validate TWIST in a completely new setting. Secondly, although the sample size was similar to the development cohort, the participants in this validation study had fewer non-ambulatory participants (FAC 0) than the development cohort and only 3 participants with a TWIST score of 0. Patients with severe stroke are more likely to be excluded from research due to communication or cognitive impairments or being unwell. Additionally, participants withdrawn from analysis due to medical status or dying prior to study completion are more likely to be those with severe stroke (Supplemental Table 1). Future work could purposively sample participants with more severe stroke. Recruitment of more patients with severe stroke may require more comprehensive supported consent strategies and the inclusion of speech language therapists in the consent process. For these patients, recruiting as close as possible to day 7 post-stroke may increase the likelihood of medical stability and ability to provide informed consent.

Finally, the TWIST prediction tool is designed for use with patients who have lower limb weakness after stroke. Future work could explore prediction of walking recovery for patients unable to walk independently due to cerebellar stroke.

This study also had several strengths. Participants with previous stroke, those with hemorrhage or ischemic stroke and those who had endovascular interventions post-stroke were all eligible for participation. Extensive efforts were also made to include participants with communication impairments to improve generalisability of the findings to this population who are often excluded from clinical trials. Twenty percent of the cohort had aphasia documented during NIHSS assessment. Finally, the prediction of independent walking at multiple timepoints enables inclusion of the small group of participants who recover independent walking later than 3 months post-stroke. This improves prediction accuracy for more severely impaired patients post-stroke, who are arguably most in need of prediction information.

Conclusions

The TWIST prediction tool was temporally externally validated but was too optimistic for participants with a TWIST score of 3. The TWIST tool and interpretation was refined for participants with a TWIST score of 3 to reflect a more conservative prediction. The refined TWIST tool can predict, within 1 week of stroke, whether a patient will achieve independent walking by 4, 6, 9, 16 or 26 weeks post-stroke, or remain dependent at 26 weeks.