Feeling validated yet? A scoping review of the use of consumer-targeted wearable and mobile technology to measure and improve sleep

Summary

The objectives of this review were to evaluate the use of consumer-targeted wearable and mobile sleep monitoring technology, identify gaps in the literature and determine the potential for use in behavioral interventions. We undertook a scoping review of studies conducted in adult populations using consumer-targeted wearable technology or mobile devices designed to measure and/or improve sleep. After screening for inclusion/exclusion criteria, data were extracted from the articles by two co-authors. Articles included in the search were using wearable or mobile technology to estimate or evaluate sleep, published in English and conducted in adult populations. Our search returned 3,897 articles and 43 met our inclusion criteria. Results indicated that the majority of studies focused on validating technology to measure sleep (n=23) or were observational studies (n=10). Few studies were used to identify sleep disorders (n=2), evaluate response to interventions (n=3) or deliver interventions (n=5). In conclusion, the use of consumer-targeted wearable and mobile sleep monitoring technology has largely focused on validation of devices and applications compared with polysomnography but opportunities exist for observational research and for delivery of behavioral interventions. Multidisciplinary research is needed to determine the uses of these technologies in interventions as well as the use in more diverse populations including sleep disorders and other patient populations.

1. Introduction

Consumer-targeted fitness devices, most of them containing sleep monitoring capabilities, are rapidly growing in their popularity. Sales of these devices nearly doubled from 2014 to 2015 and grew by 67% in the first quarter of 2016, compared to the previous year [1]. In addition, 72% of U.S. adult consumers report owning a smartphone [2]. Given that sleep monitoring smartphone applications are a top selling category, [3] nearly 3 of 4 U.S. consumers have access to a device with the potential to record sleep. Despite most consumer-targeted wearable and mobile sleep tracking technologies being classified as low risk fitness devices rather than medical devices by the Federal Drug Administration (FDA) [4], surveys suggest that sleep is reported as the most interesting health measure to automatically track [5]. Previous reviews have comprehensively evaluated the types of devices, web and mobile applications aimed at consumer sleep [6] as well as the evidence for the validity for monitoring sleep [7] and applications for sleep disorders screening [8]. These reviews demonstrate the large number of devices aimed at estimating sleep and screening for sleep disorders and consistently found relatively sparse published validity data. In many cases, claims of these sleep tracking devices and applications outweigh the evidence to support them.

In this review, we investigate the research literature to determine how these technologies were being used and whether they used as intervention tools for sleep behavior change. We chose to use a scoping review methodology to map and summarize the evidence. Scoping reviews are a type of systematic qualitative review that consists of formalized methods for “mapping” the research in a field. The process includes a well-defined and in depth search strategy as well as processes to identify areas of emphasis and analytical interpretation [9]. This is an appropriate review methodology for a diverse area with insufficient studies to conduct a quantitative review.

2. Methods

2.1 Overview

We used the framework for conducting scoping reviews proposed by Arksey and O’Malley [10] and, consistent with the evolving standards for scoping reviews, initiated the review process by developing a protocol that defined our objectives and mapped out our methods. [11] The steps of the review included the following: 1. Defining the research question, 2. Identifying relevant studies, 3 Study selection, 4. Charting the data and 5. Collating, summarizing, and reporting results.

2.2 Research question

The main research question for this review was: How are consumer-targeted wearable devices and mobile sleep technologies being used and reported in the scientific literature to measure and improve sleep?

2.3 Identifying relevant studies

An experienced information specialist (MB) developed the database search strategies using subject headings and title/abstract keywords related to sleep, wearable devices, smartphones, and mobile applications. We searched the following databases: MEDLINE, EMBASE, PsycINFO, CINAHL, Science Citation Index Expanded, and Compendex (an engineering database). We ran our original searches on March 29, 2016 and later ran search updates in all databases on June 13, 2017. We developed the initial search strategy in MEDLINE using appropriate medical subject headings (MeSH) and title/abstract keywords and translated the MEDLINE search to the appropriate syntax for use in the other databases. Where possible, we limited search retrieval to articles published in English. Detailed search strategies, including all search terms, database platforms and dates of searches are available in Appendix 1. We exported records from each database into a master EndNote library and removed duplicates. [12] We also reviewed the reference lists of included studies to attempt to locate additional relevant papers that may have been missed in the database searches.

2.4 Study selection

Our database searches retrieved a total of 3,897 records. After de-duplication, a total of 2,633 records remained. We uploaded the de-duplicated records to the Covidence systematic review platform to facilitate the title/abstract screening and full-text review processes. [13] Two reviewers independently screened the title and abstract of each record to determine whether full-text review was warranted. A total of 2,268 records were excluded on the basis of title/abstract screening, leaving 365 records eligible for full-text review. After retrieving the full-text articles identified in the title/abstract screening, two authors independently reviewed each paper and applied the agreed-upon inclusion criteria. We included 43 studies that examined the use of consumer-targeted wearable devices or mobile applications to estimate sleep, were published in English, with adult subjects and included at least 10 participants. Studies that were published as abstract only and those examining only self-reported sleep diaries or questionnaires were excluded. We resolved disagreements among reviewers at both the title/abstract screening and full-text review steps through arbitration by a third reviewer. See Figure 1 for a diagram detailing the flow of studies into the review.

Figure 1.

PRISMA Flow Diagram

2.5 Charting the data

The data extraction fields were determined through an iterative process. The study team identified the main areas of interest (study date, location, authors and title) as well as relevant study characteristics (type of technology, study aim, comparison group, sample size, study outcome). After the team extracted a few test articles, additional fields were added for thematic analysis (e.g., study population). Information recorded in the final extraction form included the following: Publication year, study location, type of device, population studied, number of participants, study objective, intervention administered (yes or no) and study outcome. The authors identified themes through discussion. Four main themes were identified: validation studies, observational studies, sleep disorders screening and intervention studies. All included studies were extracted by two individuals. Disagreements were resolved by discussion among the co-authors.

3. Results

3.1 Study selection

A flow diagram of the scientific literature search and selection criteria is listed in Figure 1. The total number of studies included in the analysis, number and reasons for exclusion are included.

3.2 Study characteristics

The most common study location was the United States (n=20) but studies were also based in Canada (n=1), Europe (UK n=5, Belgium n=1, Finland n=1, Italy n=1, Australia (n=3), Israel n=1, Asia (Korea n=3, China n=1, Japan n=1, India n=1) and Argentina (n=1). There were also 3 studies with more than one study location listed (USA, Germany and China; China, USA and Italy, Demark and Sweden). The majority of studies included healthy populations (n=24) but many studies did not provide details on the screening procedures. Six studies included participants with sleep disorders, three studies included participants with medical disorders (one conducted in dementia, two conducted in hemodialysis patients) and one study included participants with major depressive disorder. Four studies researched sleep among participants in a registry of mobile application or device users. There were also studies conducted in special populations including college students, medical residents, employees enrolled in a wellness program and judo athletes. One study did not specify the population beyond stating the age criteria and one indicated “a range of sleep contexts”. In many cases, particularly articles from Engineering, details of participant characteristics were often unclear, such as inclusion/exclusion criteria, recruitment and screening methods.

3.3 Study themes

3.3.1 Validation Studies of Wearable/Mobile Technologies for sleep

Validation studies of wearable and mobile technologies for monitoring sleep were the largest category in our review with 23 studies (Table 1). The majority of studies compared the wearable or mobile sleep tracking technology against PSG or EEG (n=14)[14–26] or actigraphy (n=4) [21, 23, 27–29]. Wearable and mobile technologies were also compared with sleep logs (n=3) [30–32]. Four studies conducted a comparison between other consumer wearable devices. [33–36]

Table 1.

Validation studies of wearable and mobile sleep tracking technologies, n=23

Year	Author	Title	Location	Population	N of Study	Consumer Targeted Device(s)	Comparison
2016	Bellone et al.[27]	Comparative analysis of actigraphy performance in healthy young subjects	Argentina	Healthy adults	22	MicroMini Motionlogger Watch, Condor Act Trust, MisFit Flash, Fitbit Flex and Thermachron	None
2014	Bhagat et al.[14]	Clinical validation of a wrist actigraphy mobile health device for sleep efficiency analysis	USA	Sleep disorders	33	S-Band	PSG
2015	Bhat et al.[15]	Is There a Clinical Role For Smartphone Sleep Apps? Comparison of Sleep Cycle Detection by a Smartphone Application to Polysomnography	USA	Healthy adults	20	Sleep Time smartphone application	PSG
2017	Brooke et al.[30]	Concurrent Validity of Wearable Activity Trackers Under Free-Living Conditions	USA	Healthy adults	95	8 monitors: Nike+ FuelBand SE, Garmin VivoFit, Misfit Shine, Fitbit Flex, Jawbone UP, Polar Loop, Fitbit Charge HR, and SenseWear Armband Mini	Sleep log
2017	Cook et al.[16]	Utility of the Fitbit Flex to evaluate sleep in major depressive disorder: A comparison against polysomnography and wrist-worn actigraphy	USA	Psychiatric	21	Fitbit Flex	PSG
2017	Cuttone et al.[33]	Sensible sleep: A Bayesian model for learning sleep patterns from smart phone events	Denmark, Sweden	Sony mobile users	126 for model development, 324 for testing	Mobile phone device (usage data)	Consumer-targeted armband sleep trackers
2015	De Zambotti et al.[17]	Evaluation of a consumer fitness-tracking device to assess sleep in adults	USA	Healthy adults and Sleep disorder	28	Jawbone Up	PSG
2016	Dickinson et al.[34]	A practical validation study of a commercial accelerometer using good and poor sleepers	USA	No criteria listed	38	Fitbit Charge HR	Fitbit
2015	Ferguson et al.[28]	The validity of consumer-level, activity monitors in healthy adults worn in free-living conditions: a cross-sectional study	Australia	Healthy adults	21	Fitbit One, Fitbit Zip, Jawbone UP, Misfit Shine, Nike Fuelband, Striiv Smart Pedometer, Withings Pulse, Bodymedia Sensewear Pro	Actigraphy
2017	Gruwez et al.[18]	Reliability of commercially available sleep and activity trackers with manual switch-to-sleep mode activation in free-living healthy individuals	Belgium	Healthy adults	20	Jawbone Up, Withings Pulse O2, Bodymedia Sensewear Pro	Home PSG
2015	Jiang et al.[19]	An effective way to improve actigraphic algorithm by using tri-axial accelerometer in sleep detection	China	Healthy adults	21	Zeo	Tri-axial accelerometer
2010	Kameyama et al.[29]	The development of a system for sleep care and its applications	Japan	Healthy adults	45	Sleep-telecare system (actigraphy and heart rate variability)	Actigraphy
2017	Kang et al.[20]	Validity of a commercial wearable sleep tracker in adult insomnia disorder patients and good sleepers	Korea	Healthy adults, sleep disorder	50	Fitbit Flex	Portable PSG
2014	Lane et al.[31]	BeWell: Sensing Sleep, Physical Activities and Social Interactions to Promote Wellbeing	China, USA, Italy	Healthy adults	27	Mobile phone application: BeWell	Sleep log
2013	Lawson et al.[21]	Validating a mobile phone application for the everyday, unobtrusive, objective measurement of sleep	UK	Healthy adults	26	Mobile phone sensors	PSG, Actigraphy
2017	Lee at al.[35]	Comparison of Wearable Activity Tracker with Actigraphy for Sleep Evaluation and Circadian Rest-Activity Rhythm Measurement in Healthy Young Adults	Korea	Healthy adults	16	Fitbit Charge HR	Fitbit
2016	Mantua et al.[22]	Reliability of sleep measures from four personal health monitoring devices compared to research based actigraphy and polysomnography	USA	Healthy adults	40	Basis Health Tracker, Misfit Shine, Fitbit Flex, Withings Pulse O2	PSG
2016	Markwald et al.[23]	Performance of a Portable Sleep Monitoring Device in Individuals with High Versus Low Sleep Efficiency	USA	Healthy adults	29	Zeo	PSG, Actigraphy
2014	Min et al.[32]	Toss ‘N’ turn: Smartphone as sleep and sleep quality detector	Canada	Adults (healthy and psychiatric)	27	Mobile phone application Toss ‘N’ Turn	Sleep log
2012	Montgomery-Downs et al.[24]	Movement toward a novel activity monitoring device	USA	Healthy adults	24	Fitbit	PSG, Actigraphy
2014	Perez-Macias et al.[36]	Comparative assessment of sleep quality estimates using home monitoring technology	Finland	Healthy adults	23	Fitbit One, Beddit Pro	No gold standard comparison. Devices compared to each other.
2016	Rosenberger et al.[25]	Twenty-four Hours of Sleep, Sedentary Behavior, and Physical Activity with Nine Wearable Devices	USA	Healthy adults	40	Fitbit One, Jawbone Up, Nike Fuelband, GENEactiv, and LUMOback	Portable EEG (Z-machine)
2016	Singh et al.[26]	A method of REM– non-REM sleep distinction using EKG signal for unobtrusive personal monitoring	India	Healthy adults	20	Mobile phone + EKG signal analysis	PSG

The majority of studies (n=17) examined the validity of consumer-targeted wrist worn or arm band accelerometers (e.g. FitBit, Jawbone, Sensewear)[14, 16–20, 22–25, 27–30, 34–36]. Several studies were conducted to evaluate sleep tracking technology using components of mobile phones (n=5) [15, 21, 31–33]. Phone-based components used included accelerometers, microphones and phone usage patterns (app launches, screen interaction) along with statistical modeling techniques to determine sleep and wake times. One study evaluated a consumer-targeted EEG device (Zeo), [23] one measured REM/nREM using EKG [26]. Multiple studies published comparisons between devices, typically focusing on comparing different brands of consumer-targeted actigraphy [18, 22, 25, 27, 28, 30, 36].

Studies using smartphone sensors and applications demonstrate the poorest agreement with validated sleep measures. For example, Bhat and colleagues (2015) reported SleepTime smartphone application was not correlated with PSG measured sleep efficiency, light sleep, deep sleep or sleep latency [15]. Measures using wrist or armband warn consumer targeted actigraphy were correlated with PSG and research validated actigraphy but tended to overestimate sleep duration. An early study that compared Fitbit with PSG demonstrated that this device overestimated total sleep time by 63 minutes compared with PSG and by only 25 minutes compared with actigraphy [24]. A more recent measure demonstrated a more favorable comparison for the Fitbit Flex compared with PSG [20]. In this study, the Fitbit Flex overestimated sleep duration by 6.5 min and 1.75%, on average, in the good-sleepers group, and by 32.9 min and 7.9% in the insomnia group. This study also set a priori criteria for clinically acceptable agreement with home PSG as <30 min sleep duration and <5% sleep efficiency and reported 82% of healthy sleepers and 34% of participants with insomnia had recordings that met this criteria [20]. The greatest agreement with PSG was reported for consumer targeted EEG. Comparisons between sleep duration estimates of consumer targeted actigraphy demonstrated little difference between devices. Rosenberger and colleagues (2016) demonstrated error rates ranging from 8–16.92% for 9 consumer targeted devices compared with EEG based sleep duration [25]. In contrast to wrist or armband actigraphy, consumer targeted EEG measures demonstrated greater concordance with PSG. Markwald and colleagues (2016) reported Zeo overestimated total sleep time by 20 minutes compared with PSG and underestimated sleep by 21.6 min compared with actigraphy [23]. In general, all consumer targeted devices overestimated sleep compared with PSG due to lower sensitivity to detect wake epochs. Furthermore, measurement in healthy populations was more concordant with PSG than in sleep disorder populations [17, 20, 23]. Most articles compare and report multiple measures (sleep duration, number and length of awakenings) while others report sleep duration only.

3.3.2 Observational Studies

The second most common use of consumer-targeted sleep wearable and mobile technology was observational studies (n=10; Table 2). Of these, the majority used wrist or armband devices (6 studies) [37–42]. Three studies were conducted using Zeo wireless EEG [41, 43, 44], one study used a smartphone algorithm [45] and one used a fitness belt wearable plus a smartphone application [46]. The purposes for using consumer targeted devices varied from the device as a tool to gather epidemiologic data (e.g. norms for age, gender differences)[43, 45, 47] or measurement of sleep in populations in the home environment [39–41]. Other measurements were used to evaluate relationships between sleep with other behaviors in the home environment. [42, 43, 46] Two studies used registries of device users, which provides a unique opportunity for analysis of large populations [44, 46]. These observational studies suggest that wearable and mobile measurement of sleep can be used to non-intrusively estimate sleep in the free-living environment in large samples as well as in patient and other specific populations. Registries of users present a large opportunity but companies are not consistently publishing in scientific journals or working with researchers to study sleep epidemiology. Results are also limited by proprietary algorithms used in data processing, lack of access to data by independent scientists and non-random assignment of device use.

Table 2.

Studies using wearable and mobile sleep tracking technologies to measure sleep in the population, n=10

Year	Author	Title	Location	Population	N of Study	Device
2015	Baroni et al.[37]	Fitbit Flex: An unreliable device for longitudinal sleep measures in a non-clinical population	USA	Healthy adults	107	Fitbit Flex
2015	Ben-Zeev et al.[38]	Next-generation psychiatric assessment: Using smartphone sensors to monitor behavior and mental health	USA	Healthy adults	47	Mobile phone sensors
2013	Butt et al.[43]	Automatically captured sociability and sleep quality in healthy adults	USA	Healthy adults	20	Zeo, Mobile phone device-using bluetooth proximity sensing)
2016	Eyal et al.[46]	Mobile based study links insomnia and sympathovagal balance	Israel	Sleep disorder	348	SleepRate mobile phone application + heart rate belt
2016	Han et al.[39]	Quantifying Physical Activity Levels and Sleep in Hemodialysis Patients Using a Commercially Available Activity Tracker	USA	Medical disorders	29	Fitbit Flex
2017	Morhardt[40]	Determining resident sleep during and after call	USA	Residents	12	Fitbit Flex
2015	Murphy et al.[41]	The use of wearable technology to measure energy expenditure, physical activity, and sleep patterns in dementia	UK	Adults with Dementia	20	Bodymedia Sensewear Amrband
2010	Shambroom & Fabregas[44]	Objective sleep differences among men and women measured in the home	USA	Dozer registry	4235	Zeo
2016	Walch et al.[45]	A global quantification of “normal” sleep schedules using smartphone data	USA	Smartphone application users	5450	Mobile phone device (smartphone application)
2017	Williams et al.[42]	Physical Activity and Sleep Patterns in Hemodialysis Patients in a Suburban Environment	USA	Medical disorders, HD patients	29	Fitbit Flex

3.3.4 Sleep disorders screening

Our search found two studies that used wearable devices or smartphones to screen for sleep disorders (Table 3). Both studies tested sleep apnea screening applications that used a smartphone application connected via Bluetooth to a pulse oximeter and other smartphone features. Behar (2015) reported an application titled “SleepAp” that used body position, audio, actigraphy and polyplethysmography signals, and the STOP–BANG questionnaire. After a machine learning model was trained on a database of patients who completed home sleep apnea testing, the application was able to correctly classify patients with moderate to severe OSA with 92% accuracy [48]. Daly and colleagues (2014) tested SleepCare: a smartphone based OSA screening tool using a pulse oximeter, a smartphone and an online server. The screening algorithms demonstrated 85.6% accuracy in a test database [49]. These studies demonstrate preliminary evidence that screening can be completed with smartphone applications paired with external devices. Given the high prevalence of sleep disorders and the limited availability of sleep labs and sleep specialists in some areas of the United States, a widely available and inexpensive method of screening is needed. At this stage, both of these examples demonstrate good agreement with clinical classification but have not been tested in large samples.

Tables 3.

Other studies of consumer targeted mobile or wearable technology to measure sleep including sleep disorders screening (n=2), intervention outcome (n=3) and intervention delivery (n=5)

Year	Author	Title	Location	Population	N of Study	Device	Study objective
2015	Behar et al.[48]	SleepAp: an automated obstructive sleep apnea screening application for smartphones	UK	Healthy adults and sleep disorder	856, 735, 121 (three study segments)	Mobile phone application paired with an armband, external microphone, and pulse oximeter	Validation of a sleep apnea screening method to properly classify users (sleep apnea vs. no sleep apnea) compared to previous diagnosis by clinician using PSG
2016	Crowley et al.[53]	The Impact of Wearable Device Enabled Health Initiative on Physical Activity and Sleep	USA	Adults	565	Jawbone Up	Device used to deliver intervention for sleep
2014	Daly et al.[49]	SleepCare: Obstructive sleep apnea screening using mobile health technology	UK	Healthy adults and Sleep Disorder	1007	Mobile phone device (with SleepCare Smartphone Application)	Validation of OSA screening framework of pulse oximeter, mobile device and online server
2017	Dunican et al. [50]	The Effects of the Removal of Electronic Devices for 48 hours on Sleep in Elite Judo Athletes	Australia	Judo athletes	32	Readiband	Device used in measuring outcome of an intervention
2017	Kang et al.[54]	Cognitive Behavioral Therapy Using a Mobile Application Synchronizable With Wearable Devices for Insomnia Treatment: A Pilot Study	Korea	Sleep disorder	19	Fitbit Charge HR	Device used to deliver intervention for sleep
2016	Liang et al.[55]	Sleep explorer: a visualization tool	Australia	Healthy adults	12	Fitbit paired with web application	Device used to deliver intervention for sleep
2016	Melton et al.[56]	Wearable devices to improve physical activity and sleep: A randomized controlled trial of college-aged African American women	USA	Healthy adults	69	Jawbone Up	Device used to deliver intervention for sleep
2011	Rondanelli et al.[51]	The effect of melatonin, magnesium, and zinc on primary insomnia in long-term care facility residents in Italy: A double-blind, placebo-controlled clinical trial	Italy	Sleep disorder and Medical disorders	43	Bodymedia Sensewear Pro2 Armband	Device used in measuring outcome of an intervention and measure changes in sleep/sleep quality
2016	Smith et al.[52]	Changes in taste preference and steps taken after sleep curtailment	USA	Healthy adults	51	Fitbit (Flex or One)	Device used to measure a sleep outcome

3.3.5 Wearable and mobile sleep monitoring technology used in interventions

There were three studies identified that used wearable devices to measure outcomes of an intervention (Table 3) [50–52] All three studies used wrist accelerometer data to measure the objective changes associated with interventions. One study evaluated short term removal of electronics among Judo athletes and found no change [50]. The other study evaluated change in sleep with melatonin, magnesium and zinc supplement for insomnia in long term care residents and found an increase in total sleep time with the supplements, as measured by the wearable device [51]. The third study evaluated change in taste preference after experimental sleep restriction and the wearable was used to demonstrate adherence to the experimental manipulation [52].

There were 5 studies identified that used mobile phones or wearable devices in delivering an intervention for sleep [53–57]. Two studies focused on improving sleep in general populations [55, 57]. There were two studies that focused on both physical activity and sleep, one in an employee wellness program, [53] one in college students [56]. One study reported on a pilot study of CBT for insomnia using a mobile application paired with a wearable device [54]. The results of the intervention studies were variable. Use of the wearable device in an employee health program demonstrated increased sleep duration whereas the wearables intervention conducted in a college population did not increase sleep duration[53, 56]. Results of the use of a consumer-targeted sleep tracking wearable paired with a smartphone intervention for insomnia demonstrated a high response rate (94.7%) as well as improvement in ISI and PSQI scores and sleep latency, as measured by the sleep diary, and high ratings for acceptability. One study tested sleep as estimated by a mobile device in a multiple behavior change application. Lin and colleagues [57] developed BeWell+, an app that continuously monitors sleep, physical activity and social interactions and provides adaptive wellbeing feedback (automatically generated based on user variables such as age) in order to promote realistic user goals and prioritize the monitoring of and feedback around the wellbeing area (sleep, physical activity or social interaction) in which the user needs the most help. Through a 19-day field trial of 27 users, Lin and colleagues [57] concluded that BeWell+ could operate on standard smartphones and that 70% of users reported it was a helpful and enjoyable program.

4. Discussion

The purpose of this study was to survey the scientific literature to evaluate the potential for consumer-targeted wearable and mobile sleep monitoring technologies in sleep research and interventions. As wearable and mobile sleep monitoring technologies become more widely used, it is important to understand the opportunities and limitations in the use of these devices. Movements such as the “quantified self” and the “internet of things (IOT)” demonstrate an increasing ability to collect granular data as well as connect behaviors to places, things and social networks [58]. Advances in sensor technology, including reduction in the size and cost as well as connectivity between sensors and other aspects of daily living provide new opportunities to promote health and healthy lifestyles, including sleep. Our review identified four main themes: validation studies, observational studies, sleep disorders screening and intervention studies. The greatest number of studies focused on developing and testing devices and algorithms, with the majority of studies evaluating consumer-targeted wrist worn devices. These studies demonstrated that there is high agreement between the different wrist worn devices and moderate to poor agreement between sleep measured these devices compared to PSG. One critical issue in this area is lack of standardized criteria of what is acceptable in terms of agreement with PSG. Furthermore, use of proprietary algorithms as well as discontinuation of devices/entry of new devices makes it difficult to compare across studies.

Our review also identified a substantial number of studies that seek to use consumer-targeted wearable devices to unobtrusively collect sleep data in the home environment. Commercial device user registries provide an excellent opportunity to evaluate trends in everyday life. These registries, if open to greater scientific collaboration could result in greater understanding of sleep patterns in the population. The majority of this research published in non-peer review formats, such as industry blogs [59]. Furthermore, lack of transparency in the data processing and non-random selection of participants limits the generalizability of this data.

Our search also revealed few studies of consumer sleep monitoring trackers and mobile technology for the purpose of sleep disorders screening or treatment. Existing studies typically used the smartphone paired with other sensors (e.g. microphone) or external sensors (oximeter). Given the prevalence of smartphones, the opportunity to screen for sleep disorders at home could improve access to screening, but current technologies are far from that capability. Furthermore, our review identified one study of a mobile intervention paired with a consumer-targeted wearable device that demonstrated promise and was highly regarded by patients. There are now several effective internet-based treatments available for the treatment of insomnia (Shut-I, Sleepio, CBTi Coach)[60, 61]. Some programs allow for linkage between wearable consumer targeted sleep wearables with the online program, but currently no data is available on whether the use of these technologies affects outcomes or user experience. Few studies have explored the impact of automatic sleep tracking on sleep behavior, among individuals with or without sleep disorders. In one small qualitative study comparing the experience of sleep diaries to automatic passive monitoring with a wrist worn device, participants reported greater changes in behavior due to self-monitoring with sleep diaries compared to passive monitoring [62]. It is unknown whether removing the effort of completing sleep diaries in favor of automatic sleep sensing will impact treatment adherence or outcome in sleep disorders populations.

One of our main interests in this review was to examine whether any research was conducted using consumer-targeted wearable and mobile sleep technology to either evaluate or deliver interventions for sleep. We found that this technology is being in several populations including intervention to improve sleep in general populations (college students, employees), in insomnia and also bundling sleep with other behavior change, such as physical activity. Despite limitations to validity, these studies suggest there is the opportunity to involve users in their own behavior change and connect sleep to other health promoting behaviors. An example from the multiple health behavior literature is a study of “health mashups” where participants were given feedback from multiple areas and were able to draw conclusions [63]. However, using sleep monitors alone is unlikely to change behavior. As demonstrated in the weight loss literature, technology may provide opportunities for cost-effective intervention dissemination and brief coaching improves outcomes above technology alone [64]. One study has even suggested the addition of technology could even be deleterious to some health outcomes. Jakicic and colleagues [65], reported poorer weight loss maintenance at 24 months in a group that was assigned to armband activity monitors versus participants who were assigned to complete food records using a web platform only. This study demonstrates that the addition of technology does not always improve the outcome beyond traditional approaches.

4.1 Limitations

Our review was limited to English only publications, and may have excluded important studies published in other languages. We also did not review studies that published algorithms or described new technology without providing a comparison group, and therefore did not present all new and upcoming technologies. We did not review articles of smart shirts, wearable patches bed sensors, non-contact sensors or smart homes. We also did not include mobile sleep trackers or screening methods that only relied on self-report diaries or questionnaires. The strength of our study is that we provide a comprehensive review of the most popular currently (or recently) available consumer technology to measure sleep. We chose to include some devices currently not available, such as Zeo, because the research provides an important perspective on the possibilities of consumer sleep tracking technology.

4.2 Conclusions

Our review demonstrates there is substantial interest in developing more accurate, reliable and low-cost measurement of sleep for consumer, clinical and research purposes but these technologies have rarely been used in clinical research applications. Our review presents possibilities for use of consumer-targeted sleep tracking devices for public health observation, wide spread sleep disorders screening and for delivery of behavioral interventions. At this point, the majority of research on consumer-targeted wearable devices for sleep is focused on validation of such devices. Given that the information collected is less precise than PSG, there is an open question as to whether these devices are precise enough to measure trends in the population, changes in sleep over time or to deliver interventions for sleep disorders. There have been recent advances in the use of these devices in delivery of sleep and multiple behavior change, but these are still in the intervention development stage. Our review highlights the need for multi-disciplinary research teams that include both engineers and behavioral/medical scientists. As the sleep monitoring technology continues to improve, it will require expertise across fields for these technologies to fully provide the insight and intervention for sleep that is needed to improve public health.

Research Agenda.

Given the popularity and interest in consumer-targeted sleep tracking devices and mobile technology, further research is needed to understand how to use these technologies as research tools. The following research priorities are recommended as a result of this review:

• The research community needs to establish criteria for validation (excellent, good, poor) of these technologies in relation to the setting and population studied.

• We recommend encouraging partnership with technology companies to analyze big data and patient registries.

• Self-management research designs including process measures are needed to evaluate the impact of these technologies on sleep as well as feasibility/acceptability among different populations.

• More work is needed to develop and validate sleep disorders screening, there is little research in this area and pairing wearable and mobile sleep tracking devices with other devices, such as oximeters, may yield effective screening tools.

• There is a need to develop and test behavioral interventions using consumer-targeted sleep monitoring devices and understand the impact of consumer-targeted devices in behavioral interventions for sleep disorders.

• Research is needed to understand how consumer-targeted sleep tracking of sleep can improve other areas of functioning such as alertness, fatigue and mood management.

Practice Points.

• Clinicians are likely to observe more patients who use wearable and mobile sleep trackers in clinical practice. The rapid growth in the use of health and fitness mobile applications and wearable fitness trackers suggests that patient may have an increasing interest in “objective” sleep monitoring using these technologies in clinical settings.

• Despite the large amount of research attempting to validate these technologists, data suggests that most devices, with the exception of consumer-targeted EEG devices (which are no longer commercially available) overestimate sleep duration and perform more poorly in patient populations compared to healthy populations.

• Current data have limited support for use of mobile and wearable technology in evaluating for disorders such as sleep disordered breathing, although pairing these technologists with other sensors such as oximeters demonstrates having potential to be used in screening, based on data from a few small studies.

• There are few interventions using mobile or wearable sleep tracking technology to provide clinical intervention for disorders such as insomnia or circadian rhythm sleep wake disorders.

Abbreviations

PSG

Polysomnography

OSA

Obstructive sleep apnea

EEG

Electroencephalogram

EMG

Electromyogram

EOG

Electrooculogram

Appendix 1—Database Search Strategies

The searches of all databases were run on March 29, 2016 and June 13, 2017.

Ovid MEDLINE(R) In-Process & Other Non-Indexed Citations and Ovid MEDLINE(R) <1946 to Present>

1. (sleep* and (apple watch* or beddit or fitbit or fuelband* or fuel band* or garmin or jawbone or mondaine or moov or motorola or sleeprate or withings or xiaomi or zeo)).tw,kw. (19)

2. (sleep* and (wearable* or app or apps)).tw,kw. (164)

3. ((consumer* adj25 sleep) and (track* or monitor*)).tw,kw. (13)

4. wearable*.tw,kw. (3263)

5. Mobile Applications/(884)

6. (mobile application* or mobile app or mobile apps).tw,kw. (845)

7. exp Cell Phones/(6805)

8. ((cell* or smart or mobile) adj phone*).tw,kw. (6427)

9. (smartphone* or cellphone* or mobilephone* or smartwatch* or smart watch*).tw,kw. (2616)

10. (iphone* or ipod* or ipad* or android* or blackberr*).tw,kw. (2939)

11. ehealth.tw,kw. (1219)

12. e-health.tw,kw. (1453)

13. mobile health*.tw,kw. (1234)

14. mhealth.tw,kw. (808)

15. m-health.tw,kw. (165)

16. or/4–15 (20435)

17. exp Sleep/(65519)

18. sleep*.tw,kw. (131450)

19. exp Sleep Wake Disorders/(68276)

20. insomnia.tw,kw. (14214)

21. exp Sleep Apnea Syndromes/(26730)

22. sleep apnea.tw,kw. (21112)

23. or/17–22 (165324)

24. 16 and 23 (426)

25. or/1–3 (184)

26. 24 or 25 (485)

27. exp animals/not humans.sh. (4205567)

28. 26 not 27 (471)

29. limit 28 to english language (451)

EMBASE (Embase.com) (Includes Embase 1974–present; Embase Classic 1947–1973; Medline 1966–present)

Embase Session Results (29 Mar 2016)

No.	Query	Results
#24	#23 AND [english]/lim	932
#23	#21 NOT #22	952
#22	‘animal’/exp OR ‘nonhuman’/exp NOT ‘human’/exp	6085938
#21	#19 OR #20	984
#20	#1 OR #2 OR #3	352
#19	#12 AND #18	827
#18	#13 OR #14 OR #15 OR #16 OR #17	318234
#17	‘sleep apnea’:ab,ti	33336
#16	insomnia:ab,ti	24743
#15	‘sleep disorder’/exp	191384
#14	sleep*:ab,ti	197428
#13	‘sleep’/exp	187991
#12	#4 OR #5 OR #6 OR #7 OR #8 OR #9 OR #10 OR #11	26644
#11	ehealth:ab,ti OR ‘e-health’:ab,ti OR ‘mobile health*’:ab,ti OR mhealth:ab,ti OR ‘m-health’:ab,ti	4337
#10	iphone*:ab,ti OR ipod*:ab,ti OR ipad*:ab,ti OR android*:ab,ti OR blackberr*:ab,ti	4895
#9	smartphone*:ab,ti OR cellphone*:ab,ti OR mobilephone*:ab,ti OR smartwatch*:ab,ti OR ‘smart watch*’:ab,ti	3455
#8	((cell* OR smart OR mobile) NEXT/1 phone*):ab,ti	8432
#7	‘mobile phone’/exp	11197
#6	‘mobile application*’:ab,ti OR ‘mobile app’:ab,ti OR ‘mobile apps’:ab,ti	897
#5	‘mobile application’/de	1810
#4	wearable*:ab,ti	3535
#3	consumer* NEAR/25 sleep AND (track* OR monitor*)	26
#2	sleep*:ab,ti AND (wearable*:ab,ti OR app:ab,ti OR apps:ab,ti)	259
#1	sleep* AND (‘apple watch*’ OR beddit OR fitbit OR fuelband* OR ‘fuel band*’ OR garmin OR jawboneOR mondaine OR moov OR motorola OR sleeprate OR withings OR xiaomi OR zeo)	84

PsycINFO (EBSCOhost)

• S19 S4 OR S18

• S18 S12 AND S17

• S17 S13 OR S14 OR S15 OR S16

• S16 TI “sleep apnea” OR AB “sleep apnea” OR TI “insomnia” OR AB “insomnia”

• S15 DE “Sleep Disorders” OR DE “Hypersomnia” OR DE “Insomnia” OR DE “Kleine Levin Syndrome” OR DE “Narcolepsy” OR DE “Parasomnias” OR DE “Sleepwalking” OR DE “Sleep Apnea”

• S14 TI sleep* OR AB sleep*

• S13 DE “Sleep” OR DE “Napping” OR DE “NREM Sleep” OR DE “REM Sleep”

• S12 S5 OR S6 OR S7 OR S8 OR S9 OR S10 OR S11

• S11 TI (ehealth or “e-health” or “mobile health*” or mhealth or “m-health”) OR AB (ehealth or “ehealth” or “mobile health*” or mhealth or “m-health”)

• S10 TI (iphone* or ipod* or ipad* or android* or blackberr*) OR AB (iphone* or ipod* or ipad* or android* or blackberr*)

• S9 TI (smartphone* or cellphone* or mobilephone* or smartwatch* or “smart watch*”) OR AB (smartphone* or cellphone* or mobilephone* or smartwatch* or “smart watch*”)

• S8 TI (((cell* or smart or mobile) N1 phone*)) OR AB (((cell* or smart or mobile) N1 phone*))

• S7 TI (“mobile application*” or “mobile app” or “mobile apps”) OR AB (“mobile application*” or “mobile app” or “mobile apps”)

• S6 DE “Mobile Devices” OR DE “Cellular Phones”

• S5 TI wearable* OR AB wearable*

• S4 S1 OR S2 OR S3

• S3 TI (((consumer* N25 sleep) and (track* or monitor*))) OR AB (((consumer* N25 sleep) and (track* or monitor*)))

• S2 TI ((sleep* and (wearable* or app or apps))) OR AB ((sleep* and (wearable* or app or apps)))

• S1 TI ((sleep* and (“apple watch*” or beddit or fitbit or fuelband* or “fuel band*” or garmin or jawbone or mondaine or moov or motorola or sleeprate or withings or xiaomi or zeo))) OR AB ((sleep* and (“apple watch*” or beddit or fitbit or fuelband* or “fuel band*” or garmin or jawbone or mondaine or moov or motorola or sleeprate or withings or xiaomi or zeo)))

CINAHL with Full Text (EBSCOHost)

• S21 S4 OR S20

• S20 S13 AND S19

• S19 S14 OR S15 OR S16 OR S17 OR S18

• S18 TI “sleep apnea” OR AB “sleep apnea”

• S17 TI insomnia OR AB insomnia

• S16 (MH “Sleep Disorders”)

• S15 TI sleep* OR AB sleep*

• S14 (MH “Sleep+”)

• S13 S5 OR S6 OR S7 OR S8 OR S9 OR S10 OR S11 OR S12

• S12 TI (ehealth or “e-health” or “mobile health*” or mhealth or “m-health”) OR AB (ehealth or “ehealth” or “mobile health*” or mhealth or “m-health”)

• S11 TI (iphone* or ipod* or ipad* or android* or blackberr*) OR AB (iphone* or ipod* or ipad* or android* or blackberr*)

• S10 TI (smartphone* or cellphone* or mobilephone* or smartwatch* or “smart watch*”) OR AB (smartphone* or cellphone* or mobilephone* or smartwatch* or “smart watch*”)

• S9 TI (((cell* or smart or mobile) N1 phone*)) OR AB (((cell* or smart or mobile) N1 phone*))

• S8 TI (“mobile application*” or “mobile app” or “mobile apps”) OR AB (“mobile application*” or “mobile app” or “mobile apps”)

• S7 (MH “Mobile Applications”)

• S6 (MH “Cellular Phone+”) OR (MH “Smartphone+”)

• S5 TI wearable* OR AB wearable*

• S4 S1 OR S2 OR S3

• S3 TI (((consumer* N25 sleep) and (track* or monitor*))) OR AB (((consumer* N25 sleep) and (track* or monitor*)))

• S2 TI ((sleep* and (wearable* or app or apps))) OR AB ((sleep* and (wearable* or app or apps)))

• S1 TI ((sleep* and (“apple watch*” or beddit or fitbit or fuelband* or “fuel band*” or garmin or jawbone or mondaine or moov or motorola or sleeprate or withings or xiaomi or zeo))) OR AB ((sleep* and (“apple watch*” or beddit or fitbit or fuelband* or “fuel band*” or garmin or jawbone or mondaine or moov or motorola or sleeprate or withings or xiaomi or zeo)))

Science Citation Index Expanded (Web of Science)

• #11 #10 OR #9

• #10 TS=((consumer* NEAR/25 sleep) and (track* or monitor*))

• #9 #8 AND #1

• #8 #7 OR #6 OR #5 OR #4 OR #3 OR #2

• #7 TS=(ehealth or “e-health” or “mobile health*” or mhealth or “m-health”)

• #6 TS=(iphone* or ipod* or ipad* or android* or blackberr*)

• #5 TS=(smartphone* or cellphone* or mobilephone* or smartwatch* or “smart watch*”)

• #4 TS=((cell* or smart or mobile) NEAR/1 phone*)

• #3 TS=(wearable* or app or apps or “mobile application*” or “mobile app” or “mobile apps”)

• #2 TS=(“apple watch*” or beddit or fitbit or fuelband* or “fuel band*” or garmin or jawbone or mondaine or moov or motorola or sleeprate or withings or xiaomi or zeo)

• #1 TS=(sleep* or insomnia)

Compendex (Engineering Village-Elsevier)

((((consumer* NEAR/25 $sleep) WN TI) OR ((consumer* NEAR/25 $sleep) WN AB)) AND (1884–2016 WN YR)) OR (((((((($ehealth OR {e-health} OR {mobile health*} OR $mhealth OR {m-health}) WN TI) OR (($ehealth OR {e-health} OR {mobile health*} OR $mhealth OR {m-health}) WN AB)) AND (1884–2016 WN YR)) OR ((((iphone* OR ipod* OR ipad* OR android* OR blackberr*) WN TI) OR ((iphone* OR ipod* OR ipad* OR android* OR blackberr*) WN AB)) AND (1884–2016 WN YR)) OR ((((smartphone* OR cellphone* OR mobilephone* OR smartwatch* OR {smart watch} OR {smart watches}) WN TI) OR ((smartphone* OR cellphone* OR mobilephone* OR smartwatch* OR {smart watch} OR {smart watches}) WN AB)) AND (1884–2016 WN YR)) OR (((({cellular phone} OR {cellular phones} OR {smart phone} OR {smart phones} OR {mobile phone} OR {mobile phones}) WN TI) OR (({cellular phone} OR {cellular phones} OR {smart phone} OR {smart phones} OR {mobile phone} OR {mobile phones}) WN AB)) AND (1884–2016 WN YR)) OR ((((wearable* OR $app OR $apps OR {mobile application} OR {mobile applications} OR {mobile app} OR {mobile apps}) WN TI) OR ((wearable* OR $app OR $apps OR {mobile application} OR {mobile applications} OR {mobile app} OR {mobile apps}) WN AB)) AND (1884–2016 WN YR)) OR ((((apple ONENEAR/1 watch*) OR beddit OR fitbit OR fuelband* OR (fuel ONENEAR/1 band*) OR garmin OR jawbone OR mondaine OR moov OR motorola OR sleeprate OR withings OR xiaomi OR zeo) WN AB OR ((apple ONENEAR/1 watch*) OR beddit OR fitbit OR fuelband* OR (fuel ONENEAR/1 band*) OR garmin OR jawbone OR mondaine OR moov OR motorola OR sleeprate OR withings OR xiaomi OR zeo) WN TI) AND (1884–2016 WN YR)))) AND ((((((({Human engineering–Sleep studies*} WN CV) OR ({Sleep research} WN CV)))) AND (1884–2016 WN YR)) OR ((((sleep*) WN AB) OR ((sleep*) WN TI)) AND (1884–2016 WN YR))))))