Using Commercial Physical Activity Trackers for Health Promotion Research: Four Case Studies

Abstract

Background

Wearable physical activity (PA) trackers are becoming increasingly popular for intervention and assessment in health promotion research and practice. The purpose of this article is to present lessons learned from four studies that used commercial PA tracking devices for PA intervention or assessment, present issues encountered with their use, and provide guidelines for determining which tools to use.

Method

Four case studies are presented that used PA tracking devices (iBitz, Zamzee, FitBit Flex and Zip, Omron Digital Pedometer, Sensewear Armband, and MisFit Flash) in the field—two used the tools for intervention and two used the tools as assessment methods.

Results

The four studies presented had varying levels of success with using PA devices and experienced several issues that impacted their studies, such as companies that went out of business, missing data, and lost devices. Percentage ranges for devices that were lost were 0% to 29% and was 0% to 87% for those devices that malfunctioned or lost data.

Conclusions

There is a need for low-cost, easy-to-use, accurate PA tracking devices to use as both intervention and assessment tools in health promotion research related to PA.

INTRODUCTION

The adoption, use, and sophistication of mobile health (mHealth) technology have greatly increased over the past decade (Adibi, 2015). These technologies have allowed health promotion initiatives to distribute health-related messages to users (Patrick et al., 2009; Turner-McGrievy et al., 2009; Turner-McGrievy & Tate, 2014) and facilitate self-monitoring of dietary (Turner-McGrievy et al., 2013) and other health-related behaviors (Kreuzer et al., 2014; Sheehy, Cohen, & Owen, 2014; Tran, Tran, & White, 2012). While some of these technologies have been mainly developed for research purposes, the commercial sector has been the primary source of growth for mHealth technology, particularly around wearable physical activity (PA) trackers (Rabbani, 2017). Technology companies are now selling PA trackers to consumers as well.

BACKGROUND/LITERATURE REVIEW

The Fitbit Tracker, released in 2008, was one of the earlier consumer PA trackers (Marshall, 2016). The original devices were wearable small rectangular device that could be clipped to a belt or other article of clothing. The original device counted steps, distance, kilocalories expended, and number of stairs climbed. One of the features that distinguished Fitbits from traditional pedometers was the ability to sync via Bluetooth to a computer, which allowed for automatic data capture and visualization via a Web browser or app. Later versions of the Fitbit included additional features, such as newform factors (e.g., wrist-worn devices), greater features (e.g., heart rate monitoring), automatic sync with a mobile phone app, and water resistance. Other wearable PA devices have followed (Ferguson, Rowlands, Olds, & Maher, 2015).

The proliferation of these devices represented exciting opportunities for researchers interested in promoting and assessing PA in research. These devices offer opportunities to collect objective PA data in a less obtrusive and expensive manner compared to traditional waist-worn accelerometry (Evenson, Goto, & Furberg, 2015). In addition, these PA trackers hold promise as intervention devices that could allow users to self-monitor exercise, promote weight loss and increases in PA, and provide real-time feedback to users (Gelman, Hill, & Yajima, 2012). Wearable trackers can also be used to detect falls, assist with proper posture and balance, and help with physical rehabilitation at home (Patel et al., 2010; Yang & Hsu, 2010).

Yet there are reasons that researchers should proceed with caution when relying on commercial devices for PA intervention and assessment. For example, while these devices often cost much less than research-grade accelerometers, they still have the same potential to be lost or damaged by users as accelerometers, necessitating the purchase of spare devices. Additionally, many of the companies that have developed these devices are new and may not survive the marketplace. New versions of the devices and regular software updates can pose challenges for studies currently in the field. Last, there may be concerns around validity of the devices, particularly around data not related to steps, such as sleep and energy expenditure (Evenson et al., 2015), or a general lack of reliability data for the vast majority of commercial devices (Ferguson et al., 2015).

The goal of this article is to present lessons learned from four studies that used several different commercial PA tracking devices for PA intervention and assessment. In addition, findings from this article set out to help other researchers by walking them through commercial PA device issues encountered by researchers in the field during previous studies. The information presented in this article aims to assist professionals and researchers engaged in the practice of developing, implementing, and evaluating PA-related health promotion and disease prevention programs that may include wearable technology components. The appeal of using these wearable tracking devices is partly informed by cognitive load theory and user control theory. Cognitive load theory states that the more cognitive burden, or mental concentration, users experience, the less able they will be to retain and act on what they learned (Brunken, Plass, & Leutner, 2003; Paas, Tuovinen, Tabbers, & Van Gerven, 2003). User control theory states that an increase in the variety of different ways to access information adds to the control a user feels and therefore increases learning (Eveland & Dunwoody, 2001). These wearable devices have the potential to simultaneously target both reductions in cognitive load, by use of easy interfaces and automatic tracking, and increases in user control, by allowing users to view feedback on their device, on their smartphone, or on a computer.

Table 1 presents an overview of the four research case studies that used commercial devices with a summary of the devices used. The goal of this article is to provide details about the challenges and opportunities that existed with each of the devices profiled and to provide health promotion professionals and researchers with guidance on what to consider when selecting commercial PA trackers for research and evaluation. All studies were approved by their university’s institutional review board, and all participants provided informed, written consent.

TABLE 1.

Information on Wearable Physical Activity (PA) Trackers Used in Four Different Research Studies

Study	Devices Used	Location of Device Wear	Data Output	Average Price of Devicea	Used for Intervention or Assessment in the Study?	Devices Lost During the Study, % (n)	Devices That Lost Data or Malfunctioned During The Study, % (n)
Mobile family research	iBitz	Waist (but can also be clipped to other locations)	• Steps	$35 (child); $50 (parent)	Intervention	0 (0)	40 (4)
	Zamzee	Waist (but can also be clipped to other locations)	• Minutes in “Zamzone” • Available on the back-end for researchers: moderate to vigorous PA (minutes)	$30		0 (0)	0 (0)
	FitBit Flex	Wrist	• Steps • Calories expended • Distance (miles) • Sleep (time)	$100		0 (0)	0 (0)
	Omron Pedometer	Waist	• Steps	$20		0 (0)	0 (0)
IMAGINE	Sensewear Armbands	Upper arm	Extensive data including: • Time spent wearing armband • Total and active energy expenditure • Total PA duration, including sedentary, moderate, vigorous, and very vigorous • Steps • Time spent lying down and in sleep • Average metabolic equivalents Additional data can be obtained from the minute-level data including: • Detailed data on sleep (e.g., efficiency, latency) • More specific PA data (e.g., 10-minute bouts) • Skin/body temperature and galvanic skin response	N/A—devices are no longer being made	Assessment	0 (0)	4.9 (2)
Vegan Bytes	MisFit Flash	Wrist (but can also be clipped to shoe)	• Number of points out of 1,000 (related to number of steps taken) • Steps • Calories expended • Distance (miles) • Total sleep (as well as light sleep and restful sleep)	$25	Assessment	12 (2)	87 (13)
HealthE-U	FitBit Zip	Waist (or other location clipped to clothing)	• Steps • Calories expended • Distance (miles)	$60	Intervention	29 (3)	3 (1)

^aPrices were obtained by the latest for what is currently available online (as of December 2015) or what the study paid for them.

METHOD

Study 1: Mobile Family Research

PA Devices Used: iBitz™, Zamzee™, Fitbit Flex™, and Omron™ Pedometer

Table 1 provides details on the devices used in the mobile family research study. The goal of this study was to explore the usability and acceptability of commercial apps and mobile monitoring devices for PA with parent–child dyads and to examine the impact of the technologies on family communication, cohesion, and social support. Parent–child dyads (children aged 9–12 years) were recruited for two phases of the study (n = 1 dyad, Phase 1; n = 2 dyads, Phase 2). Given the goal of the research to examine realworld use of PA devices and apps designed for children, the iBitz device (ibitz.com) was selected for Phase 1. This device offers a different version and app for children and parents that can sync together, such that the parent can see their child’s data. The initial phase involved formative research with one parent–child dyad and the iBitz device. During Phase 1, the parent–child dyad wore their iBitz devices and worked on increasing their PA during 4 weeks of the study. Actigraph GT3 accelerometers were used for PA assessment at pre- and posttest. After the 4 week study, the parent–child dyad participated in a structured interview to discuss their experiences with the devices and apps. Due to numerous difficulties with the iBitz devices (e.g., difficulty with syncing devices, missing data, frequent battery failure), the study protocol was adjusted for Phase 2.

The second phase was conducted over 4 weeks with two parent–child dyads and compared four PA devices: iBitz, Zamzee (hopelab.org), Fitbit Flex (fitbit.com), Omron Pedometer (omron-healthcare.com). Each week, the dyads wore a different device (order randomly determined) along with a GT3 accelerometer. At posttest, dyads participated in a structured interview to discuss their experiences with the devices and any associated tracking platforms (e.g., mobile apps). During data analysis for Phase 2 of the study, the Zamzee corporation stopped servicing commercial customers and was acquired by another corporation. Additionally, the support team at iBitz expressed concern about the use of their devices for pediatric obesity prevention and did not provide sufficient support during the research process, especially given all of the difficulties encountered with syncing the devices.

Despite the hypothesis that novel technology (e.g., iBitz, Zamzee, Fitbit Flex) would encourage greater levels of PA in short-term testing with parents and children, the only week with significantly increased steps was when children wore the Omron Pedometers. Structured qualitative interviews revealed that children were most motivated by seeing a readout of their PA progress (in this case steps) on the device itself, instead of having to rely on their parents’ devices to sync to an app.

This study experienced several PA tracker–related challenges. One of the companies was wary of supporting research with their device because of concern that the results of the study may be potentially negative toward the device branding. In addition, companies went out of business or were acquired during the study, which necessitated frequent downloading and backups of data and less reliance on the device memory or associated apps and tracking websites. Technical difficulties with battery life and device syncing occurred. This has the potential to lead to significant loss of PA data in the field, especially if participants do not regularly report to a research center in person. Backup batteries and data monitoring protocols are essential for catching technical issues early to prevent excessive data loss.

Lessons Learned

There is a need for reliable and valid wrist-worn PA device from an established company that is appropriate for children to use. Additionally, devices for children should include on-device readouts of steps or other easily interpreted PA metrics if being used as an intervention tool.

Study 2: The IMAGINE Study

PA Device Used: BodyMedia’s SenseWear™ Armband

The goal of the IMAGINE study is to develop and test a comprehensive diet, PA, and stress management intervention to reduce systemic inflammation. The study, which is currently underway, is a yearlong trial in which eligible participants can elect to participate in the intervention or an information-only control. The study enrollment is staggered, with three cohorts of intervention (n = 60 total) and three cohorts of control (n = 60 total) participants starting over the course of 5 months. As part of the PA assessment, which takes place at baseline, 3 months, and 12 months, participants are asked to wear a BodyMedia SenseWear Armband (SenseWear.bodymedia.com) for 10 days (Table 1 details information on the SenseWear device). BodyMedia has recently been acquired by Jawbone™, which decided to discontinue the production of the armband. Our team is in need of additional armbands and is still, after multiple unsuccessful attempts, working with Jawbone representatives to secure them. This has presented challenges with our staggered enrollment. Our team had to ensure that assessment periods do not overlap to allow for adequate armband supply for each assessment. This has also left little room for device loss or malfunction.

While SenseWear Armbands are relatively easy to use, they do require trained personnel to program and process data from the devices if being used for research purposes. In addition, the armbands have caused skin irritation with some participants who are sensitive to certain components of the device (e.g., nickel metal), and to avoid irritation, some participants will continuously move the armband around their arm every few hours to avoid what feels and looks like a burn. The armbands also are not water resistant and cannot be used while bathing or swimming. Devices worn on the arm, such as the SenseWear Armband, are more obtrusive than wrist-worn devices. Some participants have expressed concern using the armband when wearing short sleeves since they did not want to armband to be visible. However, the benefit of using the SenseWear Armband is the extensive validation previously done on the device (Shin, Swan, & Chow, 2015; Van Hoye, Boen, & Lefevre, 2015). In addition, it should be noted that SenseWear Armbands used for research do not completely reflect the commercial versions of these devices, as data output and options for data output may differ between commercial and research devices. Also, staff from BodyMedia, and now Jawbone, can “convert” commercial devices into research devices. However, the company has stopped manufacturing the armband at this time, presenting problems for continued application of the protocol as replacement armbands are required.

Lessons Learned

Health technology companies can be acquired by other companies during your research. If possible, before deciding on a certain PA tracking device, ensure the company can provide you with an adequate number of devices for the duration of the project and that you have trained personnel who can process the data obtained.

Study 3: Vegan Bytes

PA Device Used: MisFit Flash™

The objective of the Vegan Bytes study was to assess eating frequency, sleep, and PA differences between individuals following vegan and vegetarian diets or an omnivorous diet. A wrist-worn Bite Counter device (Scisco, Muth, & Hoover, 2013) was used to track-eating frequency. The MisFit Flash (misfit.com) was selected for PA assessment because it was low-cost (approximately $35), was waterproof (so it could be used during bathing and swimming), and tracked sleep duration in addition to PA (see Table 1). An additional, important feature of the MisFit Flash for this particular study was that it did not display steps to the user. A user could see number of daily steps and sleep only by syncing their device with a smartphone. Researchers for the study created accounts for each device, which allowed them to sync the devices before and after use by participants. In addition, MisFit has a robust, well-documented Application Programming Interface (API) available for programmers. This allowed our research team to track all data captured by each device via a study website that synced participant data automatically.

Currently 34 participants have completed the 1-week Vegan Bytes data collection period with a rotating supply of 17 MisFits. Participants are asked to wear the MisFit Flash at all times during the 1-week study.

While MisFit Flashes are inexpensive, easy to use, and unobtrusive devices for PA collection, there are some considerable challenges with using these devices. While the devices are advertised to be able to be worn at all times (including bathing and swimming), two of our study participants swam while wearing the device and reported that the device stopped working after that. The Flashes are the least expensive version in the MisFit line of products and have certain compromises compared to more expensive devices, such as the MisFit Shine. Part of the cost saving was the result of using a cheaper wristband. The band that comes with the Flash routinely broke or allowed the small Flash button to fall through, increasing the likelihood of loss. As a result, our team purchased bands made of more substantial material that included a backing to hold the Flash in more securely. This, in turn, added to the cost. In addition, there was a period of time during our study that the devices were recording only about 10% of activities. This resulted in extensive missing data with several days containing no data at all. There also were issues with nonresponse from customer service, which was unable to resolve the issue of missing data.

Lessons Learned

While these devices hold promise as an inexpensive way to track PA and collect all data through the API interface, these advantages were offset by missing data and high rate of lost devices.

Study 4: HealthE-U

PA Device Used: Fitbit Zip™

The goal of the HealthE-U study was to assess the feasibility and acceptability of a healthy weight intervention targeted to college students as compared to a control condition focused on human papillomavirus vaccination (West et al., 2016). The study randomized two classrooms to receive either a healthy weight or human papillomavirus vaccination awareness (control) intervention. Both groups received eight health promotion lessons via e-mail and participated in private Facebook groups over the course of 9 weeks. The participants in the healthy weight condition also received a Fitbit Aria™ scale to self-monitor body weight and a Fitbit Zip™ to self-monitor PA. Fitbit Zips were chosen because they cost less than other Fitbit devices and have been shown to provide an accurate measure of steps taken (Butryn, Webb, & Wadden, 2011; see Table 1 for details on the Fitbit Zip). In addition, the research team was able to objectively track Zip usage by having participants “friend” the study account, allowing the research team to view their progress and use of the Zip.

Our team experienced few problems with using the Zips during the study. A few individuals (3 out of 29 or 10%) lost their Zips during the 9-week intervention period and required a replacement. One participant (3%) had a device that malfunctioned from the onset (which was returned to Fitbit and a replacement that functioned properly was provided to the study) and we provided a replacement, and two additional Zips (7%) malfunctioned after being inadvertently run through the washer and dryer. Anecdotally, several more Zips were washed and dried but did not malfunction and therefore the participants continued to use them. Several participants (6 out of 29 or 21%) commented on the posttreatment program satisfaction inventory that they would have preferred one of the wrist-worn Fitbit versions over the Zip. No periods of missing data were experienced.

Lessons Learned

While the Zips’ price point allowed for our team to purchase enough devices for study participants, some participants would have preferred a wrist-worn device. The waist-clipped device resulted in some lost devices and some that ended up in the laundry. There were also advantages to using the Fitbits. There were no instances of missing data, and the customer support from Fitbit allowed for rapid replacement of the malfunctioning devices. Using mature technology that has been through several product revisions from a well-established company will provide more reliable technology and better support.

DISCUSSION

The goal of this article is to provide health promotion researchers and practitioners who may be interested in using wearable PA tracking devices for intervention or evaluation with an overview of some of the strengths and weaknesses of different mobile PA tracking methods. Researchers interested in using wearable PA tracking devices should think through the considerations outlined in Table 2 when deciding which device to use for their research.

TABLE 2.

Considerations and Questions to Assess When Deciding on Using Wearable Physical Activity (PA) Trackers in Public Health Research

Considerations	Questions to Ask
Population	What is the age of your study population? What is the technical ability level of your population?
PA outcomes	Is PA your primary or secondary outcome? If it is a primary outcome, there will be a greater need to have high levels of accuracy and flexibility with how the data are presented.
Intervention or assessment?	Are you using the device to assess PA or as an intervention device? If using for assessment, you should choose a device that is nonreactive (e.g., does not display activity to the participant on the device, which may influence PA levels).
Budget	How much can you afford to spend on devices, taking in consideration lost devices as well?
Data accuracy	What published research has demonstrated the validity and reliability of the device? What is the frequency of missing data? Are the device-specific collected measures and/or cut points appropriate for your research interests?
Place of wear	Will your participants tolerate wearing a device on their arm, waist, or wrist?
Waterproof	Do you need the device to be worn at all times? Would you like to be able to track swimming activities?
Sleep	Are you interested in objectively tracking sleep?
Data display	Do you want participants to be able to see their data on the device? Do you want to see their data after syncing to a computer or mobile phone? Do you want to prevent participants from seeing their data? Are you comfortable with participants viewing data through the app or website provided by the device vendor?
Application Programming Interface (API)	Do you need to capture, store, and/or display data to a local data collection system or website rather than through the vendor-provided app or website? How will you collect and prepare the data for analysis? Is the API publically available and well-documented?
Customer service and return policy	What is the refund policy if devices are lost or malfunctioning? What support is available from customer service to resolve technical issues?

Researchers assessing and intervening on PA-related behaviors are increasingly interested in low-cost ways to collect PA data and provide real-time feedback to users. Commercial, wearable, PA tracking devices hold promise to fulfill this need. As presented in this article, considerable challenges still exist in using these commercial devices for intervention and assessment. Consequently, a need still exists for research-grade devices that are low-cost, reliable, and unobtrusive for a user to wear, with the ability to be worn at all times, upload data seamlessly, provide good compliance rates for wear, and allow researchers to view and download the data.

There has been progress in this area. Companies, like Fitabase (Small Steps Labs; Diaz et al., 2015), have allowed researchers to harness all the data collected by participants wearing Fitbits. In addition, in-phone accelerometers on both Android and iOS devices have resulted in free smartphone apps that have potential to accurately track steps (Case, Burwick, Volpp, & Patel, 2015). As smartwatches decrease in cost and gain market share, new tracking options also may become available. Yet the need for easy-to-use, continuous wear, waterproof devices that can be used in research still exists.

Previous research using pedometers for activity tracking has found inaccuracies for step counting, particularly at slower rates of walking (Akerberg, Soderlund, & Linden, 2014; Beevi, Miranda, Pedersen, & Wagner, 2015; Melanson et al., 2004). Newer devices, such as wearable PA tracking devices and PA tracking apps, have been shown to be highly accurate in detecting number of steps and hours of sleeping as compared to a gold standard (Case et al., 2015; Ferguson et al., 2015). Devices like the Fitbit have already been successfully used as an intervention tool to assist people with increasing PA (Cadmus-Bertram, Marcus, Patterson, Parker, & Morey, 2015).

An additional benefit of using these devices is that they tend to be less burdensome to the research team to set up, monitor, download, and analyze than traditional PA monitoring methods (e.g., accelerometers). The devices come with comprehensive instructions meant for the average consumer, as opposed to the expertise and expensive software required to manage accelerometers. This is also a potential limitation as well since it limits the researcher’s ability to be flexible with the data if new PA cut points are recommended. The device companies also tend to report data back to the user in a partially analyzed format, such as steps or PA at different levels of intensity, meaning that the first steps of the analysis are already completed for the researcher. While this also holds some drawbacks in terms of what the researcher can do with the data, some companies will allow access to the full or partial raw data on request (e.g., Misfit and FitBit) via well-documented, public APIs.

While wearable PA tracking devices have been found to be accurate for some measures of PA, as was the case with the devices used in these case studies, there was limited accuracy when dealing with resistance exercises or addressing the added weight an individual may carry, such as carrying groceries or wearing a backpack. Cycling and incline walking were also concern with many of the devices used in the case studies. Additionally, there are challenges with the comparability of data collected across different types of commercial devices. All devices use their own cut points to determine level of PA intensity, and some do not offer standard measures of PA (e.g., Zamzee reported PA i only n their own “ZamZone” measure, not steps or other standard measures, e.g., energy expenditure or time). This limits the ability of researchers to compare data across devices and can be difficult for participants to interpret, if data are presented to them in unfamiliar units. There is work underway at present to develop standardized metrics to help alleviate this issue, but for now it remains a limitation of using these devices (Lanthrop, 2015).

Participants also may experience frustration due to issues with the devices. In the case studies, missing data and device malfunctions were reported 0% to 87% of the time, and these situations often created participant frustration due to the fact that they may have been diligently wearing the device as instructed and also may have been personally interested in the data that were lost. Another area of participant frustration came with the different battery configurations and capacities on the various devices. This was particularly an issue with the iBitz units, where batteries were frequently replaced but seemed to drain incredibly fast, leading to device shutdown and loss of data. There was a range of battery configurations observed across devices, with some using standard watch batteries (e.g., iBitz, Omron Pedometer, Misfit) and others using rechargeable lithium polymer batteries that charge while plugged into a computer or charging dock (e.g., Zamzee, Fitbit Flex and Zip, SenseWear). Researchers should carefully consider both the battery life and configuration of PA devices in their selection of a monitor to use, including how long they intend to deploy the device in the field, the likelihood that battery drain will be an issue, and the additional cost of purchasing new batteries.

CONCLUSION

As devices increase in sophistication, accuracy, usability, and decrease in cost, there is an opportunity for novel uses in larger research studies for PA assessment and to scale up PA-related health promotion interventions. Although there is a great deal of potential, researchers need to be aware of the inherent challenges and limitations they may encounter as they take on research in this area of emerging technology. Products that work very well as consumer devices are not always well-suited for research purposes, and companies can cease production of devices while researchers are in the field. While the intended us of PA devices and type of research questions asked in studies will drive the choice of device, it is likely that no device will meet all research needs. Therefore, knowing some of the lessons learned from other studies up front may help future researchers avoid lost data, lost devices, and high costs.