Usability of a Wrist-Worn Smartwatch in a Direct-to-Participant Randomized Pragmatic Clinical Trial

Abstract

Background

The availability of a wide range of innovative wearable sensor technologies today allows for the ability to capture and collect potentially important health-related data in ways not previously possible. These sensors can be adopted in digitalized clinical trials, i.e., clinical trials conducted outside the clinic to capture data about study participants in their day-to-day life. However, having participants activate, charge, and wear the digital sensors for long hours may prove to be a significant obstacle to the success of these trials.

Objective

This study explores a broad question of wrist-wearable sensor effectiveness in terms of data collection as well as data that are analyzable per individual. The individuals who had already consented to be part of an asymptomatic atrial fibrillation screening trial were directly sent a wrist-wearable activity and heart rate tracker device to be activated and used in a home-based setting.

Methods

A total of 230 participants with a median age of 71 years were asked to wear the wristband as frequently as possible, night and day, for at least a 4-month monitoring period, especially to track heart rhythm during sleep.

Results

Of the individuals who received the device, 43% never transmitted any data. Those who used the device wore it a median of ∼15 weeks (IQR 2–24) and for 5.3 days (IQR 3.2–6.5) per week. For rhythm detection purposes, only 5.6% of all recorded data from individuals were analyzable (with beat-to-beat intervals reported).

Conclusions

This study provides some important learnings. It showed that in an older population, despite initial enthusiasm to receive a consumer-quality wrist-based fitness device, a large proportion of individuals never activated the device. However, it also found that for a majority of participants it was possible to successfully collect wearable sensor data without clinical oversight inside a home environment, and that once used, ongoing wear time was high. This suggests that a critical barrier to overcome when incorporating a wearable device into clinical research is making its initiation of use as easy as possible for the participant.

Introduction

The availability of a wide range of innovative wearable sensor technologies today allows for the ability to capture and collect potentially important health-related data in ways not previously possible. These sensors can be adopted in digitalized clinical trials, i.e., clinical trials in which a participant is first informed about the details of the study, together with its risk and benefits, either in person or online, and is given a chance to ask questions about the trial. Then the participant is guided through an informed consent procedure and can become part of the study in a fully digital way [1]. These trials may be conducted outside the clinic to capture data about study participants in their day-to-day life [2]. An ever-increasing number of digitalized clinical trials are being developed that take advantage of these technologies in order to design trials that are more participant friendly and do not limit participation based on geographic barriers [1]. However, many studies to date incorporating a range of personal, digital data collection technologies have found limited usage beyond a month or two [3, 4, 5]. This along with some patients' views of wearable devices might explain, at least in part, the limited efficacy found in most studies of digital interventions to date [6, 7].

As part of the mHealth Screening to Prevent Strokes (mSToPS) trial, designed to primarily utilize an electrocardiography (ECG) sensor patch to screen at-risk individuals for asymptomatic atrial fibrillation (AF), we designed an exploratory substudy of a wrist-wearable device to better understand its capabilities for long-term heart rhythm screening [8, 9]. In this digital substudy, an Amiigo wrist-wearable device, which has been validated against a gold standard [10], was directly sent to individuals who would activate it and use it in a home-based setting. A wrist-wearable offers several potential advantages relative to other mechanisms of heart rhythm monitoring. First, rhythm monitoring can occur passively without any addition work required of the participant beyond wearing and occasionally charging the device. Second, by also tracking activity, sleep, heart rate, time, and more simultaneously participants are likely to derive greater individual value than they would from a wearable that tracks heart rhythm only. This additional value might enhance patient engagement, enabling long-term use of the device. Third, the device can be activated by the participant outside the clinic, thus allowing for continuous remote monitoring without the cost and limitations of supervised monitoring. These advantages make this technology suitable for passive heart rhythm monitoring in both healthy individuals and patients with chronic conditions.

At the time the mSToPS trial was designed, the validity of detecting AF via a photoplethysmography (PPG) signal relative to ECG was unknown, and this exploratory substudy was designed to provide early evidence of that potential. However, since that time substantial progress has been made in the field of heart rhythm sensors, and although there remain many unknowns, AF detection via a wrist wearable device has recently become mainstream, with its incorporation into several commercially available smartwatches [11, 12].

The goal of the study was to address the question of the effectiveness of study-provided wearable sensors in a population of older individuals who had already consented to be part of an AF screening trial. The RE-AIM framework was utilized as a practical way to contextualize this study so that through a better understanding of real-world experience with the uptake and usage in this digitalized substudy, future studies dependent upon the data from wearable sensors can be better designed [13, 14].

Materials and Methods

Study Population

The study population described here is a subset of 2,659 individuals who consented to take part in the mSToPS trial, whose recruitment period began on November 17, 2015 and was completed on October 4, 2016 [9]. In brief, participants in the primary mSToPS trial were randomized to either immediate or delayed monitoring with a single-lead ECG patch sensor. For the exploratory substudy, an Amiigo (now Wavelet Health) wrist-worn PPG device was to be provided to the first 500 mSToPS participants who confirmed that they routinely used a smartphone and consented to wanting to participate in the substudy [8]. However, enrolment in the substudy was stopped at 230 people due to logistical challenges with delivering the Amiigo devices to participants, technical support needed to assist with device activation issues, and the financial cost of each device being 100 dollars. Inclusion criteria for the mSToPS trial were age ≥75 years or older, or being male aged >55 years or female aged >65 years with one or more comorbidities, as detailed in the mSToPS primary manuscript [9]. Individuals with a prior diagnosis of AF, atrial flutter, or atrial tachycardia, individuals who were already prescribed anticoagulation therapy, or individuals with an implantable pacemaker were excluded from the study. The demographics (age and sex) of this substudy as well as the mSToPS trial are reported in Table 1.

Table 1.

Demographic characteristics of the study participants

	mSToPS	Substudy
Type of device	ECG sensor patch	PPG wrist-worn device
Number of participants	2,655	230
Participants with data
Number (%)	1,738 (65.5)	130 (56.5)
Age (SD)	73.7 (7.0)	70.6 (7.4)
Female (%)	704 (40.5)	54 (23.5)
Participants without data
Number (%)	917 (34.5)	100 (43.5)
Age (SD)	72.6 (7.7)	70.9 (7.7)
Female (%)	321 (35.0)	23 (23.0)

ECG, electrocardiography; mSToPS, mHealth Screening to Prevent Strokes; PPG, photoplethysmography.

Wearable Sensor Device

The Amiigo device used in this substudy weighs 23 g, has a 5-day battery life, and measures PPG among other vital signs. Additional information on how the device works, validation against gold standards, and additional functions needed to use the Amiigo device can be found in a device validation study [10]. During the consent process, interested participants were presented a picture of the Amiigo device and details of the objective of the substudy (Fig. 1). Following consent, the Amiigo wristband was sent to participants along with a charging device and written instructions of how to download the free mSToPS version of the smartphone app. This also included contact information for a study coordinator if the participant had any issues with setting up the device. Of note, approximately 60% of the participants who received the Amiigo device called or e-mailed for assistance as opposed to 15% of participants who received the iRhythm Zio XT Patch, which was used for the main mSToPS trial. The most common issue reported for the Amiigo device was difficulty setting up the Bluetooth with an individual's smartphone or tablet. Only routine fitness data, such as activity, calories burned, and sleep duration were made available to Amiigo participants via the app. Participants were asked to wear the Amiigo device as frequently as possible, night and day, for at least a 4-month monitoring period, especially to track heart rhythm during sleep.

Fig. 1.

Presentation of the Amiigo device to the study participants as shown on the mSToPS website. mSToPS, mHealth Screening to Prevent Strokes.

Data Collection

The wrist-worn device was programmed to collect PPG signals at regular intervals, in order to preserve battery life. The data were initially collected in the device manufacturer's servers through the app and then shared with the researchers. The authors of the paper used the data to produce the results in the paper. There are two data collection modes as defined by the manufacturer: a day mode, in which the device collects 30 s of data every 20 min (2.5% of a 12-h day), and a night mode, in which the device collects 30 s of data every 7 min (7.1% of a 12-h night). Theoretically, assuming 2 h of recharging per day, the total amount of data collected in a day could be up to 66 min and if the device is worn continuously for only 4 months (∼121 days), the total amount of data collected could be 7,986 min.

Data Analysis

The initial intent of this exploratory substudy was to assess the efficacy of the Amiigo wristband in detecting AF via a PPG signal relative to AF events confirmed by ECG. However, the sample size was too small to allow for any meaningful analysis of AF detected by PPG versus ECG (only 2 people were wearing the wrist monitor at the same time as having an episode of AF confirmed by the ECG sensor patch), so we decided to focus on a different analysis. Therefore, the analysis performed in this paper was intended to clarify the amount of data collected by the wrist-worn devices relative to the data that could have been collected if the wrist sensor had been worn as intended by all participants, and to describe observed usage characteristics. First, we calculated the fraction of participants who used the device and collected some data at least once. Then, among these individuals, we calculated the number of days of device usage. Since the device collects data intermittently, we calculated the average number of minutes of collected signal per day for each participant. Then we calculated for each participant the total amount of PPG signal collected. Since due to motion artifacts or signal transmissions errors only part of the signal is analyzable, we also calculated the total number of minutes of analyzable signal. All results are represented with a distribution of the number of people and detailed with median and IQR.

Results

The median age of the 230 substudy participants was 71 years, with an IQR of 66–76, and the percentage of females was 23.5%.

Device Usage

Among the 230 participants who were sent the wrist wearable, 130 (57%) used it at least once and successfully transmitted some PPG data. The age (median 70.5 years) and sex (23.8% female) of the participants who used the device at least once were not significantly different from the overall substudy population.

Participants with data wore the device over a median of 15 weeks (IQR 2–24) and for 5.3 days (IQR 3.2–6.5) per week. Therefore, the median total wear time was 78 days, with an IQR of 9–144, showing great variability in device usage (Fig. 2). All participants were asked to wear the device for at least a 4-month monitoring period. The number of participants who were compliant with ≥97 days of data (80% of the 4 months) was 57 (24.8%); 41 (17.8%) participants wore the device more than the 4-month period they were asked to. The average time of signal recorded per day of usage was 25.6 min, with an IQR of 8.9–32.5 (Fig. 3).

Fig. 2.

Distribution of the number of days of usage of the device for the participants who collected photoplethysmography data at least once. On the x axis we have different intervals of the number of days of usage, e.g., between 0 and 25 days, while on the y axis we represent the number of participants using the device for that specific number of days.

Fig. 3.

Average number of minutes of recorded signal per day for the participants wearing the device at least once. This value strictly depends on the time in which the device was worn by the participant (during the day, at night, or both). The distribution of this value is shown in the figure, where we divided the x axis into short intervals of 5 min, while on the y axis we represent the number of participants with average minutes per day included in that specific interval.

Signal Recorded

A total of ∼394,000 min of PPG signal was recorded by all participants as reported by the manufacturer. The median time of signal collected per participant was 1,307 min (IQR 101–4,578) (Fig. 4). Only 21,964 total minutes of data - 5.6% of all recorded data - were analyzable (with beat-to-beat intervals reported). The total amount of data collected and analyzable per participant is represented in Figure 5.

Fig. 4.

Distribution of the number of minutes recorded per participant. On the x axis we have intervals of number of minutes, represented in a logarithmic scale for clarity, while on the y axis we have the number of participants corresponding to each interval.

Fig. 5.

Total amount of collected and analyzable data per participant.

Discussion and Conclusions

This study sheds additional light on the effectiveness characteristics of wrist-worn devices for health-related data collection to help inform the use of similar devices in a wide range of future studies. The study provides several important contributions. It shows how collection of PPG signals is possible for older individuals, without clinical or technical supervision, and inside a home environment. Fifty-seven percent of participants were able to transmit a significant amount of PPG signal, while 5.6% of the signal was analyzable; the median per individual was 5.8% (IQR 3.4–6.4%). There are some possible reasons for this result. First, this is not a typical trial where there is full control on the environment in which participants use a device, so it is possible that the device was not used properly. Second, PPG collection is nontrivial compared to other types of data collection, such as heart rate or steps, since PPG sensors are highly affected by motion artifacts. The remaining 43% of participants who received the device were not able to transmit any signal for undisclosed reasons. Of note, approximately 60% of the participants who received the Amiigo device called or e-mailed our support team for assistance as opposed to 15% for the main mSToPS trial. The most common issue reported was difficulty setting up the Bluetooth with an individual's smartphone or tablet. As shown in Table 1, there was no significant difference between the 57% of participants with data and the 43% of participants without data. However, there was a difference in the characteristics of the participants with data for the substudy and the main mSToPS trial as the population of the substudy was younger on average (70.6 vs. 73.7). It also had a smaller percentage of females (23.5 vs. 40.5%). The participants previously expressed an interest in being selected to be part of this specific study, thus biasing the population towards people more interested in their health and possibly in technology.

The large number of participants without data contrasts with another clinical trial that utilized the same wearable device [15]. That particular study was conducted inside clinic walls where PPG signals from all patients were acquired by a single investigator at each clinical site, who underwent centralized training on a documented procedure. This suggests that a critical barrier to overcome when incorporating a wearable device into a digitalized clinical trial is making its initiation of use as easy as possible for the participant and to account for significant dropout and plan accordingly. While it is not possible to say whether doing so will assure long-term use, we at least observed that in this study those who did start wearing the device kept wearing it for a median of 78 days. There are also limited data to compare our study with other studies involving other wearable devices. In one Fitbit study, recruitment involved a younger demographic and a financial incentive [16], thus it cannot be compared directly with our study.

Another important lesson from the study was the small fraction of collected data that were of adequate quality to be analyzable for heart rhythm detection. Maximizing the quantity and quality of the data collected by minimizing motion artifacts or other issues in the data transmission from the sensor to the data collection point will be especially important when capturing heart rhythm data in the hope of identifying rare and limited episodes.

A limitation of this study is the limited communication with and information on the participants. This information could have illuminated barriers to successful Amiigo device implementation, such as why 43% of participants never successfully transmitted data. In a future study, it would be interesting to gather more information about the participants to confirm whether effectiveness is affected by a specific type of device, duration of follow-ups, seasonal implications, or the amount of data returned in real time to the participant.

Statement of Ethics

This study was reviewed by the Scripps Institutional Review Board and deemed exempt from human subjects' research. The de-identified data shared by the manufacturer contained only information not personally identifiable that was aggregated.

Disclosure Statement

Dr. S.R. Steinhubl reports receiving grants from Janssen, Qualcomm Foundation, and the National Institutes of Health/National Center for Advancing Translational Sciences (grant UL1TR001114) and serves as an advisor for DynoSense, EasyG, and Spry Health. No other disclosures are reported.

Funding Sources

This study was supported by a research grant from Janssen Pharmaceuticals to the Scripps Research Translational Institute. Additional support was provided through the National Institutes of Health/National Center for Advancing Translational Sciences (grant UL1TR001114).

Author Contributions

Concept and design: M. Galarnyk, G. Quer, S.R. Steinhubl. Acquisition, analysis, and interpretation of data: M. Galarnyk, K. McLaughlin, G. Quer. Drafting of the manuscript: M. Galarnyk, G. Quer, S.R. Steinhubl. Statistical analysis: M. Galarnyk, K. McLaughlin, G. Quer. Obtained funding: S.R. Steinhubl. Administrative, technical, and material support: L. Ariniello.