Validation of the Fitbit Flex in an Acute Post–Cardiac Surgery Patient Population

Abstract

Purpose: This study examined the validity of the Fitbit Flex activity monitor for step count and distance walked among post–cardiac surgery patients. Method: Participants (n=20) from a major urban cardiac surgery centre were recruited 1–2 days before hospital discharge. The Fitbit Flex step count and distance walked outputs and video recording of each participant performing the 6-minute walk test were collected. Fitbit Flex output was compared with criterion measures of manual step count obtained from the video recording and manual measurement of distance walked. Statistical analysis compared the output and criterion measures using paired sample t-tests, Pearson correlation coefficients, Lin's concordance correlations, and Bland–Altman plots. Sub-analysis compared slower walking (<0.8 m/s; n=11) and faster walking (≥0.8 m/s; n=8) group speeds (1 participant was excluded from analysis). Results: Steps counted and distance walked were significantly different between the Fitbit Flex outputs and criterion measures (p<0.05). The Fitbit Flex steps counted and distance walked showed moderate association with manual measure steps counted (r=0.67) and distance walked (r=0.45). Lin's concordance coefficients revealed a lack of agreement between the Fitbit Flex and the criterion measurement of both steps counted (concordance correlation coefficient [CCC]=0.43) and distance walked (CCC=0.36). The percentage of relative error was −18.6 (SD 22.7) for steps counted and 25.4 (SD 45.8) for distance walked. Conclusions: The Fitbit Flex activity monitor was not a valid measure of step count and distance walked in this sample of post–cardiac surgery patients. The lack of agreement between outputs and criterion measures suggests the Fitbit Flex alone would not be an acceptable clinical outcome measure for monitoring walking progression in the early postoperative period.

Cardiac surgery is the most widely used treatment approach for individuals diagnosed with severe, multi-vessel, operable coronary artery disease or valvular heart disease.¹ After cardiac surgery, patients typically remain in hospital for 5–7 days,² and early recovery involves a daily, graduated walking programme. Physical activity (PA), in the form of a walking programme, is considered a protective intervention that may prevent early postoperative complications; it is also a primary component of cardiac rehabilitation (CR), which aims to reduce long-term cardiovascular morbidity and mortality.^3–6 Patient referral to CR upon discharge from hospital is considered the standard of care.^7,8 Outpatient CR is a comprehensive, exercise-based, risk-reduction programme; optimally, patients enrol in this community-based programme 6–8 weeks after being discharged. During the acute postoperative hospital stay, patients are provided with an activity schedule that provides guidance for a safe and independent walking programme. They are also provided with information about how to progress their activity during the transition period between discharge home and enrollment in outpatient CR.

Wearable activity monitors such as pedometers and accelerometers are promising tools because they can be used by both clinicians and patients to promote and objectively measure progression in terms of step count and distance walked.^9–12 These devices may also help facilitate patients' confidence in their ability to exercise, thereby allowing them to progress to higher levels of activity and actively self-manage their rehabilitation.⁹ The Fitbit Flex¹³ (Fitbit Inc., Boston, MA) is a relatively new triaxial accelerometer and a popular commercial device, marketed as having the ability to easily estimate steps taken and distance walked. It is conveniently worn on the non-dominant wrist, and activity data can be uploaded wirelessly to a user-friendly Web site that tracks activity levels over time.

Although the Fitbit Flex has not been validated in the acute post–cardiac surgery population, Fitbit and other commercially available activity monitors have been tested in populations with stroke, traumatic brain injury,¹⁴ and chronic obstructive pulmonary disease¹⁵ as well as in healthy individuals.^10,16,17 Recently, Alharbi and colleagues¹⁸ demonstrated the ability of the Fitbit Flex to correctly categorize activity levels (light, moderate, and vigorous) in free-living participants with coronary heart disease completing phase III CR. Despite demonstrating a strong association with the gold standard (ActiGraph) for measuring step count, the Fitbit Flex was reported to overestimate step count, with an average error rate of 13%, and the level of error increased as the number of steps taken increased. An important consideration, specifically in the acute post–cardiac surgery population, is the reported validity problems of some activity monitors in measuring step count and distance walked at slower walking speeds.^10,11

Monitoring activity by step count and distance walked is an ideal means of clinically measuring progress in the post–cardiac surgery population and thereby fostering self-management and behavioural change for PA. Considering that patients' post cardiac surgery walking speeds are likely to be slower than their pre-surgical gait speed and that progress in PA level will be minimal during early recovery, it is important to validate the Fitbit Flex in the post–cardiac surgery patient population before it is implemented in clinical care and self-monitoring. To our knowledge, a paucity of literature has been focused on the validation of devices in this population. Therefore, the purpose of this cross-sectional study was to determine the accuracy of the Fitbit Flex in the acute post–cardiac surgery population before being discharged from hospital.

Methods

We examined the accuracy of the Fitbit Flex during the 6-minute walk test (6MWT)¹⁹ in a convenience sample of patients post–coronary artery bypass graft (CABG) surgery, aortic valve repair, or mitral valve replacement at a large, urban cardiac surgery centre. Demographic inclusion criteria were patients aged ≥35 years, with an uncomplicated postoperative course (see exclusion criteria), and awaiting discharge home within 1–2 days. Activity-level inclusion criteria were ability to ambulate safely and independently without a walker for at least 6 minutes, which was determined by the physiotherapists directly involved in patient care.

Exclusion criteria were the presence of significant postoperative complications delaying discharge from hospital (e.g., bleeding requiring revision, stroke, arrhythmia), significant lower limb impairment, neurological disorder, cognitive disorder that would prevent the patient from being able to understand and provide consent, prior surgery affecting gait and mobility, physical motor function such as amputation, gait aid use, and inability to understand and provide informed consent.

Previous studies of the intra-class correlation coefficient (ICC) of activity monitor output to step count have found ranges from 0.5 to 1.0.¹¹ On the basis of a conservative estimate (ICC=0.5–1.0), α=0.05, and a power of 0.80, we calculated that a sample size of 20 participants would be needed. Study approval was obtained through the hospital ethics review board and the University of Toronto.

Procedures

Once informed consent was obtained, socio-demographic and clinical characteristics were collected from the participants' charts. Height and mass were important to collect before beginning the 6MWT for the Fitbit Flex setup. Other characteristics collected included age, sex, BMI, left ventricular ejection fraction, type of surgery, postoperative days since surgery at time of testing, and length of hospital stay.

Participants performed the 6MWT on the day of, or the day before, their planned discharge home. In preparation for the 6MWT, the Fitbit Flex wrist location and the patients' sex, height, and weight were input into the device. The 6MWT was conducted on the cardiac unit; patients walked continuously in one direction in a corridor more than 30 metres long and were video recorded from the waist down while walking. Data from the Fitbit Flex were compared with criterion measures of a manual count of steps and distance walked (in metres). Each 6MWT video step count was manually counted, independently, by two members of the research team using a handheld counter to ensure accuracy within one step between the two assessors. Distance was measured manually from the lateral malleolus at the beginning of the walk to the lateral malleolus of the most forward foot, to the nearest centimetre, at the end of the 6MWT.

Statistical analysis

Statistical analysis was performed using IBM SPSS Statistics, version 22.0 (IBM Corporation, Armonk, NY) and MedCalc statistical software, version 16.8.4 (MedCalc Software, Ostend, Belgium). Shapiro–Wilk tests were completed on the variables of steps counted and distance walked for both the criterion measurement and the Fitbit Flex output in the initial inspection of data to determine normality. After confirming normal distribution, manual step count and distance measured were compared with the Fitbit Flex output of step count and distance walked using paired sample t-tests. Pearson's correlation coefficient (r) was calculated to determine the association between the Fitbit Flex output and criterion measures. Associations of r=0.25–0.50 were considered fair; r = 0.50–0.75, moderate; and r>0.75, strong.²⁰

Finally, the decisive assessment of validity was performed by determining the agreement between the Fitbit Flex output and criterion measures using Lin's concordance correlation coefficient (CCC). The level of agreement was set at CCC greater than 0.95.²¹ Lin's CCC measures the actual agreement between the values reported by the Fitbit Flex and the criterion manual measures of interest and, therefore, reflects the accuracy of the Fitbit Flex. The percentage of relative error of the Fitbit Flex output of step counts and distance measured was calculated by

$$Percentage\mathrm{ }relative\mathrm{ }error=\left[\frac{\left|Fitbit\mathrm{ }Flex\mathrm{ }output-criterion\mathrm{ }measure\right|}{\mathrm{c}\mathrm{r}\mathrm{i}\mathrm{t}\mathrm{e}\mathrm{r}\mathrm{i}\mathrm{o}\mathrm{n}\mathrm{ }\mathrm{m}\mathrm{e}\mathrm{a}\mathrm{s}\mathrm{u}\mathrm{r}\mathrm{e}}\right]\times 100$$.

Sub-analysis was performed in two groups of differing gait speed: a slower walking group (<0.8 m/s; n=11) and a faster walking group (≥0.8 m/s; n=8; 1 participants was excluded). The differentiation of walking speed was based on a threshold identified in the previous literature, which examined the effect of varied walking speeds on the accuracy of differing activity monitors.¹⁸ Paired comparisons between the sub-groups were completed using Wilcoxon signed-rank tests for step count and distance walked.

Results

A total of 32 post–cardiac surgery patients were screened, and 20 (18 men, 2 women) who had undergone CABG and/or valve surgery consented to participate. The patients' characteristics are presented in Table 1. One participant was excluded from data analysis because testing did not record any measurement on the Fitbit Flex. Otherwise, no data were missing for this sample. Video of the excluded participant was reviewed, and minimal arm swing was noted, which may have affected logging data into the device.

Table 1.

Participant Characteristics and Surgical Procedures (n=20)

Characteristic	No. of participants*
Sex
Male	18
Female	2
Mean (SD) age, y	61.3 (10.2)
Mean (SD) height, m	1.7 (0.2)
Mean (SD) mass, kg	82.9 (12.7)
Mean (SD) BMI, kg/m2	29.0 (7.4)
Mean (SD) postoperative time of testing, d	7.4 (3.7)
Mean (SD) length of stay, d	7.7 (3.7)
Left ventricular ejection fraction
Grade 1, >50%	13
Grade 2, 40%–49%	3
Grade 3, 30%–39%	2
Grade 4, 20%–29%	1
Unknown	1
Surgical procedure
Coronary artery bypass graft surgery	13
Aortic valve repair	6
Mitral valve replacement	1
Other procedures	4

^*Unless otherwise indicated.

Table 2 presents the criterion measurements and Fitbit Flex output from the 6MWT. A Shapiro–Wilk test of normality indicated that the data were normally distributed for steps counted and distance walked for both the criterion measurements and the Fitbit Flex output. Both steps counted and distance walked were significantly different between Fitbit Flex output and criterion measures (p<0.05).

Table 2.

Criterion Measurements and Fitbit Flex Output

	Mean (SD)
Variable	Criterion measurement	Fitbit Flex output	Mean difference	95% CI	p-value
Distance walked, m	246.1 (70.0)	295.3 (95.7)	49.1	5.9, 92.3	0.03
Slower group	200.9 (51.6)	276.4 (74.5)	75.5	–	0.01
Faster group	308.3 (32.0)	321.3 (111.9)	12.9	–	0.48
Step count, no. steps	504.3 (77.6)	415.3 (135.9)	89.1	138.0, 40.1	0.01
Slower group	464.6 (68.3)	392.9 (111.2)	71.6	–	0.02
Faster group	559.0 (50.3)	446.0 (156.6)	113.0	–	0.01
Gait speed, m/s	0.7 (0.2)	–	–	–	–
Slower group	0.6 (0.1)	–	–	–	–
Faster group	0.9 (0.1)	–	–	–	–

Figure 1 illustrates the relationship between criterion measurements and Fitbit Flex output for both step count and distance walked. Pearson correlation coefficients revealed that the Fitbit Flex steps counted showed moderate association with manual count of steps (r=0.67), and distance walked showed fair association with manual measurement of distance walked (r=0.45). However, CCC revealed a lack of agreement between the Fitbit Flex and the criterion measurement of both steps counted (CCC=0.43) and distance walked (CCC = 0.36). The percentage of relative error of the Fitbit Flex steps counted compared with the criterion measurement was −18.6 (22.7), and the percentage of relative error of the Fitbit Flex distance was 25.4 (45.8).

Figure 1.

Scatter plots representing the comparisons between the criterion measure (manual measures) and the Fitbit Flex output for (a) steps counted and (b) distance walked. Black triangles (▾) represent participants with slower walking speeds, and open circles (○) represent participants with faster walking speeds. The line of equity is shown.

Participants grouped by slower walking speed

In the slower walking group (<0.8 m/s; n=11), the Fitbit Flex output and the criterion measures were significantly different for both distance walked and steps counted (Table 2; p<0.05). The association between Fitbit Flex output and criterion measures was moderate for steps counted (r=0.68) and fair for distance walked (r=0.35). However, Fitbit Flex output and criterion measures lacked agreement on measures of both steps counted (CCC=0.46) and distance walked (CCC=0.18). The percentage of relative error was −16.2 (SD 20.0) for steps counted and 42.2 (SD 45.2) for distance walked.

Participants grouped by faster walking speed

In the faster walking group (≥0.8 m/s; n=8), the Fitbit Flex output was significantly different from the criterion measure for steps counted (Table 2; p=0.01), but not for distance walked (Table 2; p=0.48). The Fitbit Flex and criterion measures demonstrated a strong association for steps counted (r=0.84) and a moderate association for distance walked (r=0.68). However, a lack of agreement between the Fitbit Flex output and criterion measures was seen for both steps walked (CCC=0.33) and distance walked (CCC=0.34). The percentage relative error was −21.9 (SD 27.2) for steps counted and 2.48 (SD 38.0) for distance walked.

Discussion

This study was conducted to determine whether the Fitbit Flex output for step count and distance walked was valid in post–cardiac surgery patients during the in-hospital phase of recovery. Walking is the most common form of PA used in rehabilitation after cardiac surgery, and step count and distance walked can be used as objective measures to assess progression.⁸ Our findings demonstrated a lack of agreement between the Fitbit Flex output and criterion measures of step count and distance walked in the current sample of patients, regardless of walking speed. This suggests that the Fitbit Flex alone would not be an acceptable clinical outcome measure for monitoring patient progression in walking programmes during the acute phase of recovery after cardiac surgery.

The growing popularity of wearable activity monitors in various clinical populations reinforces the idea that patients of all ages may be interested in devices such as the Fitbit Flex for motivation and self-monitoring, regardless of whether it is integrated into formal CR programming. Typically, activity monitoring after cardiac surgery uses self-measurements including time, distance walked, symptoms experienced, and rate of perceived exertion.⁸ Despite the lack of agreement required for its use as a valid clinical measurement device, the Fitbit Flex demonstrated moderate associations with criterion measures for steps walked in both the slower and the faster groups and strong associations for distance walked only in the faster group. This may make it possible for patients to use it at home as a gross measure of PA to complement, but not replace, current self-monitoring tools, particularly as their walking speed increases.

The Fitbit Flex was recently validated in a study that compared it with the ActiGraph monitor in coronary heart disease participants (n=28) attending phase III CR; the study also addressed the ability of the Fitbit Flex to identify patients who met specific PA guidelines (e.g., achieving the cut-off point guidelines of 7,000 and 10,000 steps per day).¹⁸ Although participants' walking speeds were not reported, it can be assumed that the walking speed of participants in outpatient CR programmes would be significantly faster that of the participants in this study. In contrast to the current study, Alharbi and colleagues¹⁸ established validity by stratifying how participants met their goals according to a range of activity levels. This demonstrates that the Fitbit Flex may be suitable only for capturing the gross measurement of PA, not for reflecting the more subtle improvements noted in the earlier phases of in-hospital CR.

Recognizing that, in addition to their possible clinical usefulness, activity monitors offer the potential benefit of assisting patients in self-monitoring and motivation while they work toward their postoperative activity goals,⁹ it is important for clinicians to educate their patients in how to use an activity monitor. If the Fitbit Flex is used in the early phases of CR, patients need to be informed of the potential level of error associated with it. The current study reveals that, during the acute recovery phase after cardiac surgery, the Fitbit Flex lacks agreement with criterion measures and, in general, tends to underestimate the number of steps taken, with a percentage of relative error of −18.6 (SD 22.7) and a percentage of relative error for distance walked of 25.4 (SD 45.8).

Previous studies have also demonstrated that, at slower walking speeds, some activity monitors tend to underestimate step counts.^11,12,22,23 Therefore, their output may not truly represent patient activity level and may lead to a misinterpretation of progress at this early stage in postoperative CR. Future studies need to examine the Fitbit Flex's ability to monitor change in a home walking programme as a patient transitions from hospital to formal outpatient CR. This approach may help determine its usefulness for independent self-monitoring and motivation while patients are in a stage of uncertainty regarding their activity progression without formal guidance or observation.

The placement of the activity monitor may also contribute to the poor validity of the Fitbit Flex noted in the current study. Although we placed the Fitbit Flex on the wrist, several validation studies used activity monitors attached to other body surfaces, including the waistband or hip,^10,11,23 ankle,^22,24 and anterior mid-thigh.²⁵ Post–cardiac surgery patients commonly experience sternal incision pain and tightness,^26,27 which have been associated with apprehension about, and fear of, movement after cardiac surgery.²⁸ These factors could result in decreased arm swing, which might alter the results of the Fitbit Flex. This finding may explain why, in the current study, the device failed to record any steps or distance for one participant with minimal arm swing. A potential solution would be to use an activity monitor that attaches to the ankle; this placement has been shown to accurately measure step counts, although not distance walked, at walking speeds similar to that of the participants in the current study.^22,24 However, it may not be suitable for all post–cardiac surgery patients; some might have physical limitations and pain with saphenous vein graft incision sites, ankle edema, and bending to secure the device after a sternotomy.^26,27

Occasionally, post–cardiac surgery patients require the use of a gait aid for safety when they are discharged from acute care. Although the Fitbit Flex manufacturer does not specifically indicate that it cannot be used with a walking aid, the use of an aid may affect acceleration patterns. This may pose a further problem for the accuracy of activity monitors such as the wrist-worn Fitbit Flex, which uses a threshold acceleration to calculate step count and distance.¹³ Fitbit, Inc., has acknowledged that step counts may be lower when one is pushing a stroller or shopping cart; however; it suggests that the device can still be used to track general progress in overall activity levels.¹³ Future research should explore the accuracy of the Fitbit Flex for participants using a gait aid.

We examined the validity of the Fitbit Flex during the 6MWT, a commonly used outcome measure in CR,¹⁹ in an acute-care hospital setting. This study adds to the body of literature because many previous studies have assessed activity monitors in either speed-controlled (e.g., treadmill)^10,11,25 or distance-controlled environments.²⁹ Activity monitors have the potential to be a useful tool for both clinicians and patients as objective measures of exercise performance,^9,10 and they can also aid in developing self-efficacy in self-monitoring and progression during a walking programme.⁹ This is especially important in this acute population because these tools may be useful for bridging the transition in care between surgery and enrolment in outpatient CR programming.

Although other studies have shown the Fitbit Flex to be a valid measure of step count and distance walked,^16–18 our findings suggest that it is not a valid measure of step count or distance walked in acute post–cardiac surgery patients. The discrepancy in these findings may be a result of participants in previous studies having faster walking speeds than those in our study.^16–18 Considering these results, the Fitbit Flex cannot be recommended to replace the clinical measures currently used to measure progression in post–cardiac surgery walking programmes. However, these study findings may be useful for educating patients about the possible limitations of self-monitoring devices if they express an interest in using them after they are discharged home.

This study has some limitations. To our knowledge, no normative data for walking speed after cardiac surgery exist in the current literature; therefore, we are unable to comment on the generalizability of findings overall to the acute post–cardiac surgery population. In addition, men may be overrepresented in our study; previous research has shown that women represent approximately 20%–30% of post–cardiac surgery patients,^30,31 whereas women constituted only 10% of our sample. This small number may further limit the generalizability of our findings to the women in this group.

Conclusion

The Fitbit Flex activity monitor is not considered an accurate device for measuring step counts or distance walked in this acute post–cardiac surgery population. Percentage of relative error was substantial for both step count and distance walked. Our findings suggest that this activity monitor may have limited accuracy when it is applied to patient monitoring and progression in a walking programme during early recovery after cardiac surgery. Patients planning to use the Fitbit Flex as part of their self-monitoring of activity levels after discharge should be educated about its potential error of measurement at this stage in their recovery.

Key Messages

What is already known on this topic

Clinicians and researchers have identified the usefulness of wearable physical activity (PA) monitors such as the Fitbit Flex as a tool to promote activity and self-management in both the healthy population and the population with chronic disease. Increasing PA is a key focus before patients are discharged from hospital and during early recovery from cardiac surgery, when they are waiting to be enrolled in cardiac rehabilitation (CR). Although accelerometer devices may facilitate patients' confidence in their ability to exercise as well as their motivation, research to date is limited regarding the validity of these devices after cardiac surgery and, specifically, at slower walking speeds.

What this study adds

To our knowledge, this study is the first to examine the accuracy of the Fitbit Flex in a sample of in-hospital post–cardiac surgery patients. The results suggest that the Fitbit Flex is not an accurate device for measuring step count or distance walked in the slower walking population. Despite our findings, devices such as the Fitbit Flex can complement other current clinical measures of activity progression while patients transition from hospital to home and subsequently into CR.