Step Monitor Accuracy During PostStroke Physical Therapy and Simulated Activities

Abstract

Introduction/Purpose:

The amount of stepping activity during rehabilitation post-stroke can predict walking outcomes, although the most accurate methods to evaluate stepping activity are uncertain with conflicting findings on available stepping monitors during walking assessments. Rehabilitation sessions also include non-stepping activities and the ability of activity monitors to differentiate these activities from stepping is unclear. The objective of this study was to examine the accuracy of different activity monitors worn by individuals post-stroke with variable walking speeds during clinical physical therapy (PT) and research interventions focused on walking.

Methods:

In Part I, 28 participants post-stroke wore a StepWatch, ActiGraph with and without a Low Frequency Extension (LFE) filter, and Fitbit on paretic and non-paretic distal shanks at or above the ankle during clinical PT or research interventions with steps simultaneously hand counted. Mean absolute percent errors were compared between limbs and tasks performed. In Part II, 12 healthy adults completed 8 walking and 9 non-walking tasks observed during clinical PT or research. Data were descriptively analyzed and used to assist interpretation of Part I results.

Results:

Part I results indicate most devices did not demonstrate an optimal limb configuration during research sessions focused on walking, with larger errors during clinical PT on the non-paretic limb. Using the limb that minimized errors for each device, the StepWatch had smaller errors than the ActiGraph and Fitbit (p<0.01), particularly in those who walked < 0.8 m/s. Conversely, errors from the ActiGraph-LFE demonstrated inconsistent differences in step counts between Fitbit and ActiGraph. Part II results indicate that errors observed during different stepping and non-stepping activities were often device-specific, with non-stepping tasks frequently detected as stepping.

Conclusions:

The StepWatch and ActiGraph-LFE had smaller errors than the Fitbit or ActiGraph, with greater errors in those walking at slower speeds. Inclusion of non-stepping activities affected step counts and should be considered when measuring stepping activity in individuals post-stroke to predict locomotor outcomes following rehabilitation.

Introduction

Approximately 80% of individuals surviving an initial stroke recover some locomotor function (1), although residual impairments in strength, coordination, and balance can limit functional mobility. Interventions aimed at improving these impairments are frequent components of clinical physical therapy (PT) (2, 3), although the efficacy of these strategies is uncertain. Rather, previous data suggests the amount of walking practice may be an important training parameter that maximizes gains in walking performance (4, 5). Accordingly, an increasing number of studies have attempted to evaluate the amount of walking practice during PT, with data suggesting the stepping amount can predict locomotor outcomes post-stroke (6–8). Recent data suggest routine integration of step monitoring during inpatient rehabilitation (9, 10), consistent with recent reviews extolling the potential utility of devices in widespread clinical use (11).

While many activity monitors can quantify stepping activity (12), accuracy during certain clinical conditions is uncertain. The StepWatch has been established as the gold standard across clinical populations (13), including post-stroke (14, 15). The ActiGraph has also been utilized in a number of studies (13), although accuracy may be compromised at slower speeds (16, 17). At lower speeds, however, accuracy is improved with a specific filter (Low Frequency Extension; LFE) (18, 19) and monitor placement on the ankle rather than waist (20, 21). The Fitbit One manufacturer recommends wearing the device on the user’s belt or pocket, although previous studies have similarly identified undercounting when walking at speeds < 0.9 m/s (22–24). Recent research suggest Fitbit placement on the ankle may also improve step count accuracy at speeds > 0.4 m/s (23–26).

A potential limitation of these and other studies is that determination of step count accuracy occurs only during forward, level walking. During clinical PT, however, patients perform both walking and non-walking activities (3, 27), with stepping activities often performed in variable contexts (e.g., multiple directions or on stairs) (19). Indeed, most studies assess the sensitivity of devices (i.e., detecting steps during stepping) without testing the device specificity, (i.e., absence of step detection with non-walking leg movements). Previous work evaluating steps counts during ‘free-living’ (18, 28, 29) and simulated tasks (19, 30) suggested accuracy varies depending on the device utilized, the movement performed, and placement on non-paretic (15, 25, 31) or paretic limbs (14, 26). The accuracy of these monitors during the varied activities occurring during clinical PT (i.e., different stepping and non-stepping tasks) is unknown, which limits conclusions regarding the contributions of stepping amount to locomotor gains.

The purpose of the current study was to evaluate step count errors of 4 activity monitor configurations (StepWatch, ActiGraph, ActiGraph-LFE, and Fitbit) during post-stroke rehabilitation. In Part I, participants post-stroke wore activity monitors on the distal shanks of their paretic and non-paretic limbs during clinical PT, during which walking and non-walking interventions often occur. A separate cohort also wore the monitors during a research trial where only walking tasks were performed. In Part II, individuals without neurological injury performed different walking and non-walking tasks frequently observed in clinical PT or research trial sessions as observed in Part I. For both Parts I and II, steps or repetitions of activities were hand counted for comparison with activity monitors. We hypothesized monitors would differ in their ability to discriminate between walking and non-walking activities due to variations in their step detection algorithms (32). We further hypothesized monitors placed on the paretic vs non-paretic limb would have smaller errors, specifically due to non-paretic limb movements during activities that did not involve walking, and that greater errors would be observed in participants who walked at slower speeds. Optimizing selection of device and limb placement to minimize errors in steps counts during physical therapy can contribute to improved predictions of locomotor outcomes post-stroke.

Methods

Participants

Research protocols were approved by the Indiana University IRB (#1608161738, 1608087541) and all participants provided written informed consent. For Part I, data were collected and analyzed from two separate studies. The first was an observational clinical physical therapy (PT) study in which step counting was measured on patients with subacute (< 6 months) stroke performing inpatient rehabilitation. Step counting was performed on patients during their scheduled physical therapy sessions, and treating therapists performed different exercise tasks during these 1-hr sessions without influence from the research team. The second study was a research trial in which individuals with chronic (> 6 months) stroke enrolled in an exercise training trial focused on practicing only walking tasks at different exercise intensities during the 1-hr sessions (33). Specific inclusion criteria for the clinical PT study were: 18–89 years old (i.e., 90 years old or greater considered identifier by local Institutional Review Board), ability to ambulate >50 m prior to most recent stroke, and no significant lower extremity orthopedic impairment (e.g., amputation, fracture) restricting participation in walking activities. Specific enrollment criterion for the research trial included: 18–85 years old, ability to walk > 10 m with assistive devices (e.g., cane) or bracing below the knee at speeds < 1.0 m/s, and no medical history limiting walking capacity.

For Part II, healthy individuals aged 18–35 years were recruited to evaluate the ability of the activity monitors to identify stepping activity during simulated therapy tasks observed in Part I.

Instrumentation

Specific monitors included in this study were the StepWatch 3 (Modus Health, Edmonds, WA), ActiGraph wGT3X-BT (ActiGraph Corp., Pensacola, FL), and Fitbit One (Fitbit Inc., San Francisco, CA). Prior to study initiation, the sensitivity of each device settings to participant’s height, weight, or age was tested by placing 3 identical monitors with a range of attribute values on the same distal shank of research staff and ambulating approximately 400 m. Only step counts reported by the StepWatch were sensitive to height, and individual participant’s data were programmed prior to each assessment.

The StepWatch is a research-grade, biaxial accelerometer designed to be worn on the lateral aspect of one ankle, directly superior to the malleoli and counts each stride taken by that limb. In addition to the participant’s height, each StepWatch was initialized using standardized settings characterizing walking impairments post-stroke (quick stepping: ‘no’, walking speed: ‘slow’, range of speeds: ‘rarely varies pace’, leg motion: ‘gentle and/or geriatric’). In Part I, the non-paretic StepWatch was set to collect data in 5-second epochs and the paretic StepWatch was set to collect data in 60-second epochs; the 60-second epochs used on the paretic leg were necessary as per the protocols for the research trial and clinical observation studies. In Part II, the StepWatch was initialized to capture data in 5-second epochs.

ActiGraph GT3X monitors are a research grade, triaxial accelerometer that capture bodily accelerations, which are converted into step counts using ActiLife software. In both parts of this study, ActiGraph monitors were initialized to collect data at 30 Hz with data reported in 5-second epochs, and then were affixed to each participant’s distal shank.

Fitbit One activity monitors are consumer-grade, triaxial accelerometers that can estimate daily step count, distance ambulated, and caloric expenditure. This monitor was designed to be worn on a belt, bra or in a pocket; however, in both parts of this study, they were affixed to the distal shank.

Protocol

In Part I, participants simultaneously wore three activity monitors on the lateral aspect of the distal shank during clinical PT or research trial sessions. The first monitor was placed superior to the lateral malleolus of both the paretic and non-paretic ankles, with the two others placed adjacent and superior to the first in random order (i.e., proximal to distal). Steps were manually counted by a single observer (CEH), a physical therapist and biomechanist, using separate hand tally counters for each lower limb and summed to determine a total step count. Manually counting steps using hand-tally counters is a widely accepted criterion (12) for healthy (32, 34, 35) and patient (26) (Treacy) populations and across a wide range of walking speeds (1 to 10 mph (36)), especially when privacy concerns do not allow for video-recorded steps. The criteria for stepping were derived and modified from training protocols developed in a previous observational study in which one author (TGH) participated (3). A step was defined as intended or unintended advancement of either limb in any direction during upright walking, including lateral steps performed during transfers, when subjects were standing or during standing balance adjustments, and dragging or shuffling of the foot for leg advancement in any direction during upright activities. Additionally, standing balance exercises resulting in raising the foot onto a surface (e.g., tapping a cone or step-ups to a platform) constituted a step in each direction, including attempts to advance the foot if the foot left the ground, even if it did not meet the intended target. Comfortable gait speeds were obtained within 3 days of step monitoring to investigate the effect of walking speed on device accuracy. Only a highly trained, single observer was utilized to ensure consistency of counting stepping according to the above definitions (see for example (34, 37)).

In Part II, 12 healthy individuals performed 8 stepping and 9 non-stepping activities frequently observed during Part I with all activity monitors on a random limb (Supplemental Table 1 has additional details). Tasks were performed for ~60 seconds except for lower extremity wheelchair propulsion (120 seconds). Stepping activities included walking forward at 0.4 and 1.0 m/s on a level treadmill, at 7.5⁰ and 15⁰ incline or decline, backward and side-stepping at 0.28 and 0.56 m/s, respectively, and completing 4 flights of stairs at self-selected pace. Non-stepping activities included short-arc quads, straight leg raises, seated marches and long-arc quads at 30 and 45 reps/min, recumbent and upright stepping or cycling machines at 60 and 90 steps/min, and lower extremity wheelchair propulsion at self-selected pace. The range of speeds of activities performed were reflective of what was observed in Part I, and steps or movement repetitions were hand-counted as above.

Data Processing

Data were extracted from the StepWatch and ActiGraph using Modus software (version 3.4) and ActiLife (version 6.13.3), respectively. ActiGraph data were analyzed both with and without the Low Frequency Extension (ActiGraph and ActiGraph-LFE, respectively). The ActiGraph LFE allows lower intensity acceleration data to enter algorithm processing by expanding the lower threshold of the bandpass filter, which has been shown to capture step counts with a higher degree of accuracy for individuals who ambulate slowly (20, 21). To report comparable epoch-length data, ActiGraph and StepWatch data were reduced from 5-second to 60-second epochs by summing across each 60-second period. While the ActiGraph and Fitbit collect data as “steps”, data from the StepWatch are reported in strides, and per-session step counts from the StepWatch were determined by doubling the total number of strides across the PT session. Fitbit step counts were calculated as differences between the displayed step counts at the start and end of activities (24).

Statistical Analyses

Statistical analyses were performed in SPSS v24 (IBM Corp., Armonk, NY) with alpha = 0.05. For Part I, activity monitors were evaluated for accuracy relative to summed bilateral hand counts with clinical PT and research trial sessions analyzed separately. Mean steps/session from summed hand counts were calculated for both clinical PT and the research trial and compared using unpaired t-tests. Mean device bias was calculated as hand count subtracted from the activity monitor count (i.e., positive values indicate activity monitor over-counting). In addition, absolute percent error per session (absolute mean bias divided by criterion steps) was also determined to allow calculation of mean absolute percentage error (MAPE). Bland-Altman plots examined the errors between the monitors and the manual step count for systematic trends (38). Intraclass correlation coefficients (ICC_2,1) (31) assessed agreement between monitors and manual step counts, with ICCs ≥ 0.75 classified as excellent, 0.60–0.74 as good, 0.40–0.59 as fair, and < 0.40 as poor (26, 39). Shapiro-Wilk tests indicated non-normal, positively skewed distribution of stepping data in Part I, although analyses using parametric versus non-parametric statistics led to very similar results. One participant (6 sessions) tapped his non-paretic foot repeatedly during clinical PT sessions, resulting in abnormally high step counts on the non-paretic limb. Removal of data from this participant also resulted in positively skewed data, and statistical analyses are presented both with and without inclusion of this participant’s data. In addition, attempts to improve normality by transforming positively skewed data with square-root transformations also resulted in similar findings. Therefore, parametric analyses of the original data were utilized for consistency between Parts I and II (for similar results, please see (40, 41)). Paired t-tests also determined whether placement on the paretic or non-paretic limb influenced the magnitude of errors of devices. Data from the limb that minimized the MAPEs for each device were used to compare errors between devices using a repeated measures ANOVA with post-hoc Tukey-Kramer tests. Finally, data were combined across the clinical PT and research trial with participants stratified by gait speed (< 0.4, 0.4–0.8, >0.8 m/s) (42) for comparison of device errors within each stratum.

For Part II, stepping and non-stepping activities were descriptively evaluated with mean step counts, bias, and MAPE reported. Conversely, as no step counts should be reported during the non-stepping activities, data are reported as mean device bias and percent non-stepping movement repetitions, which represent the number of events detected by the device divided by the number of movement repetitions.

Results

Part I.

Clinical PT (n=63) or research trial (n=37) sessions were observed in 28 participants post-stroke (Table 1). Most sessions (82%) were 60 minutes in duration. All research sessions focused on walking activities and all but 4 clinical PT sessions involved some walking practice. Across clinical PT sessions, 47% of the time was dedicated to walking practice with less time for other activities, including standing exercise (17%), outcome measures (12%), transfers (8.5%) and sitting exercises (6.8%). Steps/session ranged from 0–4669, with more steps in the research trial (mean [SD]: 2218 [1245]) vs clinical PT (481 [370]; p<0.001; Table 1).

Table 1.

Subject and session demographics. Values are mean [standard deviation] or mean [range] where appropriate.

Participants	Clinical PT (n=21)	Research trial (n=7)
Age (years; mean [SD])	64.0 [13.5]	65.5 [8.3]
Sex (N [%])
Male	10 [48%]	5 [71%]
Female	11 [52%]	2 [29%]
Paretic side (n [%])
Left	8 [38%]	4 [57%]
Right	13 [62%]	3 [43%]
Time since stroke (month; mean [SD])	0.7 [0.5]	17.3 [12.3]
Lesion type (N [%])
Ischemic	12 [57%]	7 [100%]
Hemorrhagic	9 [43%]	0 [0%]
Vascular distribution (cortical; n [%])	11 [52%]	3 [43%]
Observed PT Sessions	Clinical PT (n=63)	Research trial (n=37)
# sessions observed per subject (mean [SD])	3.0 [1.9]	5.3 [3.1]
Steps per session (mean [SD])	481 [370]	2219 [1245]
Sessions with gait speed available (n [%])	45 [71%]	22 [59%]
Gait speed (m/s; mean [range])	0.33 [0.00–1.21]	0.49 [0.16–1.08]
< 0.4 m/s / 0.4–0.8 m/s / > 0.8 m/s (n/n/n)	23/20/2	13/1/8

Across all sessions, ICC_2,1 analysis identified good to excellent agreement with observed step counts for all devices when placed on the paretic limb during clinical PT or the research trial (0.59–0.96; Table 2). While ICCs were also excellent for all devices on the non-paretic limb during the research trial, agreement decreased substantially during clinical PT (0.04–0.62) with inclusion of all data. Following removal of the participant who tapped their foot, ICCs improved on the non-paretic limb during clinical PT sessions, although the StepWatch (ICC=0.97) was much higher than the other three devices (ICC range: 0.62–0.78).

Table 2:

Intra-class correlation coefficients with 95% confidence intervals for clinical PT and research trial sessions, with exclusion of the individuals who repeated tapped their non-paretic limb during clinical sessions.

		Clinical PT (n=63 sessions)	Clinical PT without “tapper” (n=57 sessions)	Research trial (n=37 sessions) (n=37 sessions)
Paretic	StepWatch	0.96 (0.93–0.98)	0.96 (0.93–0.98)	0.92 (0.86–0.96)
	AG LFE	0.84 (0.71–0.92)	0.84 (0.69–0.92)	0.96 (0.93–0.98)
	AG	0.59 (0.06–0.81)	0.57 (0.03–0.80)	0.81 (−0.04–0.95)
	Fitbit	0.72 (0.25–0.88)	0.71 (0.24–0.87)	0.92 (0.36–0.98)
Non-Paretic	StepWatch	0.32 (0.09–0.52)	0.97 (0.91–0.99)	0.97 (0.94–0.98)
	AG LFE	0.04 (−0.18–0.26)	0.77 (0.59–0.87)	0.97 (0.94–0.98)
	AG	0.62 (0.29–0.79)	0.62 (0.16–0.82)	0.86 (0.07–0.96)
	Fitbit	0.59 (0.40–0.73)	0.78 (0.39–0.91)	0.92 (0.59–0.97)

Bland-Altman plots evaluating error rates between manual and device counts on paretic and non-paretic limbs are provided for clinical PT (Supplemental Figure 1) and the research trial (Supplemental Figure 2). Inspection of the plots from both clinical PT and research sessions identified systematic undercounting by the Fitbit and ActiGraph regardless of limb placement, whereas the StepWatch and ActiGraph-LFE tended to overestimate stepping activity. In addition, the participant that tapped their foot during clinical PT sessions is identified with very large errors (Supp Fig 1, gray markers) for the StepWatch and ActiGraph-LFE. During clinical PT, the StepWatch and ActiGraph-LFE placed on the paretic limb demonstrated the smallest mean bias (over-counting by 0.8 and 91 steps, respectively), with greater undercounting in the other devices (~ −200 steps). On the non-paretic limb, the ActiGraph-LFE and StepWatch had substantial overcounting (210 and 385 steps; Table 3), with smaller differences following removal of the participant that tapped their foot (50 and 106 steps). For the other devices, changes were smaller with exclusion of this individual. Conversely, during the research trial the non-paretic and paretic ActiGraph-LFE exhibited the smallest differences (−9 and 2 steps), with higher differences with the StepWatch (9–91 steps). Mean bias values for the ActiGraph and Fitbit were much larger (−383 to −695 steps).

Table 3:

Paretic and non-paretic device bias and absolute percent errors for step counts during clinical PT and research trial sessions.

		Clinical PT (n=63 sessions)
		Mean (SD) step count	Mean Bias	MAPE	MAPE 95% CI
					LL	UL
Paretic	StepWatch	482 (415)	1	27.0	19.6	34.3
	AG LFE	572 (362)	91	56.1	33.0	79.2
	AG	260 (240)	−221	65.0	54.1	75.9
	Fitbit	286 (380)	−195	72.8	58.4	87.2
Non-Paretic	StepWatch	691 (590)	210	178.8	30.4	327.2
	AG LFE	866 (950)	385	320.8	85.9	555.7
	AG	323 (231)	−158	56.7	41.0	72.4
	Fitbit	403 (442)	−77	123.7	59.9	187.6
		Clinical PT without “tapper” (n=57 sessions)
Paretic	StepWatch	523 (415)	6	24.1	16.8	31.3
	AG LFE	611 (357)	93	50.1	30.8	69.4
	AG	284 (241)	−234	63.5	51.5	75.4
	Fitbit	316 (388)	−202	70.4	54.6	86.2
Non-Paretic	StepWatch	567 (379)	50	26.8	15.5	38.2
	AG LFE	624 (372)	106	69.8	43.5	96.2
	AG	329 (237)	−189	49.1	36.7	61.5
	Fitbit	350 (405)	−167	62.3	49.1	75.5
		Research trial (n = 37 sessions)
Paretic	StepWatch	2228 (1424)	9	29.5	18.7	40.2
	AG LFE	2210 (1224)	−9	17.3	12.6	22.0
	AG	1525 (1127)	−695	36.6	31.2	41.9
	Fitbit	1781 (1459)	−438	32.4	21.1	43.7
Non-Paretic	StepWatch	2310 (1289)	91	14.9	9.1	20.7
	AG LFE	2222 (1286)	2	14.6	10.6	18.6
	AG	1649 (1219)	−570	32.3	25.3	39.3
	Fitbit	1836 (1506)	−383	33.0	21.5	44.5

Evaluation of MAPEs across devices and limbs in clinical PT and research sessions revealed substantial variability. For example, the MAPEs observed in research sessions varied from 14–37% across devices and limbs, whereas MAPEs in clinical PT sessions varied from 27–73% in the paretic limb to 56–321% in the non-paretic limb. These latter data were influenced by the participant who tapped their non-paretic foot; removal of these data revealed lower MAPEs for the StepWatch (26.8%), with higher values for other devices (49–70%) similar to paretic limb data. Comparison of MAPEs between limbs revealed smaller errors for the paretic vs non-paretic StepWatch (p<0.01) and ActiGraph-LFE (p=0.02) during clinical PT with all data, with between limb differences only for the ActiGraph-LFE following removal of the participant that tapped their foot. Conversely, the StepWatch on the non-paretic limb revealed smaller errors than the paretic limb (p<0.01) during the research trial, with no other between-limb differences for other devices.

Using data from each device on the limb that minimized errors (i.e., smallest MAPEs), the repeated measures ANOVA revealed significant (p<0.001) differences between devices within both clinical PT and research trial sessions (Figure 1A). Post-hoc analyses indicated the StepWatch and ActiGraph-LFE had significantly smaller errors than the ActiGraph. With removal of the participant that tapped their foot, the StepWatch demonstrated smaller errors than all other devices (p<0.001). During the research trial, both the StepWatch and ActiGraph-LFE demonstrated significantly less errors than the Fitbit or ActiGraph (p<0.001).

Figure 1.

Mean absolute percentage errors in each device during sessions separated by clinical PT or research trial sessions (A) and stratified by walking function (B). Error bars represent standard error. *p = 0.05, **p < 0.05

To evaluate differences in device errors across individuals with varying functional capacity, MAPEs from the optimal limb were combined from clinical PT and the research trial and stratified by participants’ walking speed (<0.4, 0.4–0.8, and >0.8 m/s; Figure 1B). The largest errors were observed in those who walked slowest; notably, the MAPE for the Fitbit was 129% for those who walked < 0.4 m/s, which was significantly greater than the StepWatch (33%), ActiGraph (61%) and ActiGraph-LFE (47%; all p<0.01). Differences between StepWatch and ActiGraph were also significant (p<0.01). In those with walking speeds between 0.4–0.8 m/s or > 0.8 m/s, MAPEs were significantly higher in the ActiGraph (50% and 25%, respectively) than the other devices, with ranges from 18–35% for those who walked 0.4–0.8 m/s and 4.2–7.1% for individuals who walked >0.8 m/s.

Part II.

Twelve adults [8 women; mean (SD); 29.9 (3.5) years old, 171.7 (8.4) cm, 69.0 (15.3) kg] performed 8 stepping and 9 non-stepping activities observed in Part I with all devices on a single limb. During stepping activities, mean device bias varied depending on the speed and activity performed, with examples provided in Figure 2A (full data in Supplemental Table 2). During treadmill walking at 0.4 m/s, the Fitbit and ActiGraph demonstrated the largest errors (39–45% MAPE), with improvements in the Fitbit with decline walking. At 1.0 m/s, all devices demonstrated reduced errors, except for the ActiGraph and ActiGraph-LFE with treadmill incline. Mean device bias and MAPEs during backward walking and side-stepping revealed similar trends of greater errors at slower speeds, with reduced errors at higher speeds except for the ActiGraph and ActiGraph-LFE during side-stepping (50–67% MAPE). Stair negotiation revealed the smallest errors for the StepWatch and Fitbit (1.8–7.7% MAPE) when compared with either the ActiGraph or ActiGraph-LFE (17–42% MAPE).

Figure 2.

Mean absolute percentage errors during simulated walking tasks (A) and non-stepping movements (B). Error bars represent standard error.

During non-stepping activities, ~50% of movement repetitions for short-arc quads (Figure 2B) and straight leg raises performed in supine were recorded as steps for the ActiGraph, ActiGraph-LFE, and Fitbit regardless of the speed of activity (full data in Supplemental Table 3). Conversely, the StepWatch reported zero steps with short-arc quads and straight leg raises, but registered nearly all repetitions during seated marches and long-arc quads. During recumbent and upright stepping, certain devices detected minimal activity counts at the slowest speeds while most devices detected > 50% of the non-stepping activity. Finally, lower extremity wheelchair propulsion was frequently identified as stepping activity by all devices.

Discussion

The present study evaluated the ability of specific activity monitors to quantify stepping behaviors during clinical PT or research sessions provided to individuals post-stroke. Agreement between device and manual counts were excellent for devices placed on paretic and non-paretic limbs during research sessions focused on walking, although ICCs were reduced during clinical PT with placement on the non-paretic limb. Bland-Altman plots, mean device bias, and MAPEs also confirm greater errors during clinical PT sessions particularly on the non-paretic limb, although removal of one participant who tapped their non-paretic limb reduced the errors. Comparisons between devices indicated the smallest errors with StepWatch during clinical PT, and relatively similar errors for the StepWatch and ActiGraph-LFE during research sessions. Larger errors were observed with the Fitbit and the ActiGraph, particularly in patients who walk at slower speeds.

The collective findings identify potential limitations of specific activity monitors during various exercise activities observed during stroke rehabilitation. While step counts reported during research sessions were fairly consistent with manual counts (i.e., high sensitivity), these sessions focused entirely on walking activities. Indeed, most device validation studies utilize walking assessments on level surfaces to evaluate stepping monitor accuracy (17, 23, 31). However, during clinical PT, walking activities comprise only a portion of rehabilitation interventions (27). During non-walking activities during clinical PT in Part I and simulated in Part II, all devices detect steps in specific situations, although some devices appear to minimize false-positive errors (i.e., higher specificity) during different tasks (e.g., short-arc quads, straight leg raises). In addition, when performing various walking activities, the speed and extent of locomotor impairments also influence the errors observed, as larger false-negatives are observed in those who move at slower speeds.

An important consideration is the accuracy of data collected from the non-paretic versus paretic limbs. Bland-Altman plots and ICC analyses suggest substantial differences in stepping errors or monitor agreement depending on limb placement. In research sessions, limited errors and excellent agreement were observed for all devices on each limb, which is likely due to the focus on walking tasks (14, 26). However, during clinical PT, greater errors and disagreement with manual counts on the non-paretic limb are likely related to use of that limb to assist with various functional tasks outside of walking, such as wheeled mobility tasks or other extraneous movements. The false-positive rate of certain devices was delineated in the Part I results with and without the one individual who tapped their foot, although removing these data still resulted in larger errors on the non-paretic limb in nearly all devices. These data suggest non-paretic limb placement in patients post-stroke results in aberrant step counts and is consistent with observations in previous studies (7, 30).

Another consideration is the choice of device to detect stepping activity in individuals post-stroke. Our study utilized activity monitors that have all been suggested to be accurate at the slower speeds typical of individuals post-stroke (15, 20, 21, 23, 24, 26). However, the present data suggest the Fitbit and the ActiGraph (without LFE) demonstrate significantly greater errors than ActiGraph-LFE (worn on the ankle) and StepWatch, particularly at reduced walking speeds. These findings are important for patients post-stroke undergoing inpatient rehabilitation, as the level of physical impairments are substantial and movement speeds are generally slower. Given the limitations in device placement on the non-paretic limb described above and between devices, the use of these the StepWatch or ActiGraph-LFE is essential for accurate step counts if these data are utilized to facilitate prediction of outcomes. Importantly, however, differences between the StepWatch and ActiGraph-LFE (e.g., Table 2–3) suggest the use of the StepWatch may be warranted. Indeed, determination of stepping activities with this device has been utilized to facilitate prediction of outcomes at discharge (7, 8) which can be valuable for discharge planning (i.e., going home or another facility). The collective findings are consistent with previous recommendations (12), and may be extended to conditions where multiple therapy activities are performed in patients post-stroke with activity monitors on the paretic limb.

Specific limitations of this study include analyses of clinical PT activities focused on stepping and non-stepping activities, or variable walking tasks in research interventions, in which practice of tasks were not controlled. This was purposely done to observe real-world activities that were performed during “typical” sessions; indeed, these observations were able to delineate specific behaviors that may not have been considered initially. For example, the observation of one individual tapping their non-paretic limb is not a task that has been considered previously in other studies determining the accuracy of various step monitors. Even with removal of this participant, selected measures of device accuracy (i.e., ICC) were still <0.80 for all monitors except the StepWatch. In preliminary, unpublished observations from a previous inpatient rehabilitation study (7), similar findings were observed and necessitated the placement of StepWatch on the paretic vs non-paretic limb. To help delineate how these non-stepping behaviors are detected with different devices, Part II was undertaken to evaluate the relative magnitude of errors in simulated stepping and non-stepping conditions.

Another limitation was the lack of another observer to track stepping activity, or the use of video recordings to ensure accuracy of observed stepping activity. The primary observer was a licensed physical therapist and biomechanist and previously trained on protocols for detecting stepping activity (3). While other studies have used single, trained observers to monitor clinical stepping activities (16, 18, 34, 37), the lack of another observer remains a limitation. Alternatively, video recordings have been utilized in other studies, although in the “free-living” clinical setting, video recordings are frequently not permissible in large rehabilitation gyms where other patients not enrolled in the study may have been identifiable (i.e., health-information privacy violation).

Additional limitations included the use of parametric statistics, although non-parametric analyses and transformation of data using square-roots of stepping data revealed similar outcomes. Further, a small number of individuals with gait speeds > 0.8 m/s were enrolled, limiting our ability to extend our findings to higher functional individuals post-stroke. However, individuals participating in inpatient rehabilitation post-stroke present with moderate to severe functional impairments, consistent with many of the subjects tested here. Further work is required to extend these findings to those patients with greater ambulatory capacity during a variety of exercise activities.

Despite these limitations, an important consideration related to the present results may be the clinical utility of the findings. While recent studies have begun to implement monitoring of stepping activities into clinical care (9, 10), widespread clinical integration of step monitoring has not yet occurred. A recent review has delineated some of the barriers related to routine clinical implementation of activity monitors (11), and provided a framework to mitigate these barriers such that clinicians could systematically evaluate intervention dose and facilitate prediction of outcomes. Such integration may occur rapidly, similar to the recent widespread adoption of specific outcome measures (10 m walk test, 6 min walk test, Berg Balance Scale) in neurological rehabilitation over the past 5–10 years (27, 43, 44). At that time, the selection of accurate activity monitors should be a priority to ensure valid assessments of dosage and prognosis.

In conclusion, the StepWatch and ActiGraph-LFE demonstrated greater consistency and smaller errors than the Fitbit or ActiGraph during PT sessions, particularly in individuals with slower walking speeds. Limb placement influenced the observed errors for specific devices in various conditions or at different speeds. Researchers and clinicians may wish to strongly consider the mobility limitations and specific activities to be performed when choosing activity monitors and their limb placements to measure stepping activity in individuals post-stroke.

Supplementary Material

Supplemental Table 1 (.doc, .tif, pdf, etc.)

Supplemental Table 1. (Supplemental Table 1.docx)

Supplemental Table 2 (.doc, .tif, pdf, etc.)

Supplemental Table 2. (Supplemental Table 2.docx)

Supplemental Table 3 (.doc, .tif, pdf, etc.)

Supplemental Table 3. (Supplemental Table 3.docx)

Supplemental Fig 1 (.doc, .tif, pdf, etc.)

Supplemental Figure 1.(Supplemental Figure 1.tif) Bland-Altman plots of step count errors of each device on the paretic and non-paretic limbs in each device (A-H) during clinical PT sessions; white markers indicate data from all participants with one exception (gray) of a patient who tapped their non-paretic limb repeatedly during sitting. Dashed lines represent upper and lower limits of agreement (95% CI).

Supplemental Fig 2 (.doc, .tif, pdf, etc.)

Supplemental Figure 2. (Supplemental Figure 2.tif) Bland-Altman plots of step count errors of each device on the paretic and non-paretic limbs in each device (A-H) during research trial sessions. Dashed lines represent upper and lower limits of agreement (95% CI)