Skip to main content
PMC home page
JMIR mHealth and uHealth logoLink to JMIR mHealth and uHealth
. 2018 Nov 8;6(11):e11066. doi: 10.2196/11066

Usability Study of Mainstream Wearable Fitness Devices: Feature Analysis and System Usability Scale Evaluation

Jun Liang 1,2, Deqiang Xian 3, Xingyu Liu 4, Jing Fu 5, Xingting Zhang 6, Buzhou Tang 7, Jianbo Lei 8,9,✉
Editor: Gunther Eysenbach
Reviewed by: Alycia Sullivan, Gabriel Signorelli, Juan Quiroz
PMCID: PMC6250954  PMID: 30409767

Abstract

Background

Wearable devices have the potential to promote a healthy lifestyle because of their real-time data monitoring capabilities. However, device usability is a critical factor that determines whether they will be adopted on a large scale. Usability studies on wearable devices are still scarce.

Objective

This study aims to compare the functions and attributes of seven mainstream wearable devices and to evaluate their usability.

Methods

The wearable devices selected were the Apple Watch, Samsung Gear S, Fitbit Surge, Jawbone Up3, Mi Band, Huawei Honor B2, and Misfit Shine. A mixed method of feature comparison and a System Usability Scale (SUS) evaluation based on 388 participants was applied; the higher the SUS score, the better the usability of the product.

Results

For features, all devices had step counting, an activity timer, and distance recording functions. The Samsung Gear S had a unique sports track recording feature and the Huawei Honor B2 had a unique wireless earphone. The Apple Watch, Samsung Gear S, Jawbone Up3, and Fitbit Surge could measure heart rate. All the devices were able to monitor sleep, except the Apple Watch. For product characteristics, including attributes such as weight, battery life, price, and 22 functions such as step counting, activity time, activity type identification, sleep monitoring, and expandable new features, we found a very weak negative correlation between the SUS scores and price (r=−.10, P=.03) and devices that support expandable new features (r=−.11, P=.02), and a very weak positive correlation between the SUS scores and devices that support the activity type identification function (r=.11, P=.02). The Huawei Honor B2 received the highest score of mean 67.6 (SD 16.1); the lowest Apple Watch score was only 61.4 (SD 14.7). No significant difference was observed among brands. The SUS score had a moderate positive correlation with the user’s experience (length of time the device was used) (r=.32, P<.001); participants in the medical and health care industries gave a significantly higher score (mean 61.1, SD 17.9 vs mean 68.7, SD 14.5, P=.03).

Conclusions

The functions of wearable devices tend to be homogeneous and usability is similar across various brands. Overall, Mi Band had the lowest price and the lightest weight. Misfit Shine had the longest battery life and most functions, and participants in the medical and health care industries had the best evaluation of wearable devices. The perceived usability of mainstream wearable devices is unsatisfactory and customer loyalty is not high. A consumer’s SUS rating for a wearable device is related to their personal situation instead of the device brand. Device manufacturers should put more effort into developing innovative functions and improving the usability of their products by integrating more cognitive behavior change techniques.

Keywords: wearable devices, usability, System Usability Scale, function comparison, fitness

Introduction

Background

In recent years, wearable devices have gained tremendous momentum and have become part of people’s daily lives. Wearable devices are portable mobile intelligent devices that can be worn directly or integrated into clothing [1]. The wearable devices in this study are wristbands and watches with step counting as their core function [2]. After combining IMS market research consulting with 2012 market research data from ABI Research, the US market research firm BI Intelligence believed that less than 40% of US consumers were familiar with wearable devices in 2012, whereas at least 54% of consumers developed a strong interest in them for health monitoring and fitness tracking in 2013. BI Intelligence expects the global wearable market to grow at a compound annual rate of 35% between 2015 and 2019 [3]. According to data from the International Data Corporation’s Worldwide Quarterly Wearable Device Tracker, worldwide shipments of wearable devices grew by 1.2% in the first quarter of 2018 [4]. In addition, global smartwatch shipments in the second quarter of 2018 grew by 56% year on year [5]. Meanwhile, based on their original wearable market forecast from 2013 and interviews with industry experts, BI Intelligence concluded that, based on current trends, at least 33% of the US population will own wearable devices by the second quarter of 2017 [6]. The Tencent ISUX User Research Center released a report (smart wearable equipment market white paper) in 2015 [7] that contained a large-scale questionnaire survey with a total of 8083 domestic Chinese Tencent Instant Messenger user respondents. This stated that approximately 60% of internet users were familiar with wearable devices and that the number of Chinese internet users who owned a wearable device increased from 2.9% to 8.4% between November 2014 and May 2015. Among these users, wristband ownership was 4.6%, which was significantly higher than the 3.1% ownership of intelligent watches. Awareness of intelligent watches increased from 48% to 52%, and awareness of intelligent wristbands increased from 35% to 40% over the same period.

Wearable devices have significant potential for health monitoring and they are being widely investigated [8-13]. As a very popular tool, the use of wearable devices in medical and fitness applications is rapidly increasing. These devices can monitor vital signs such as motion [14], nutrition status [15], and heart rate [16], as well as a diabetic patient’s blood sugar [17], cardiac disease status, apnea during sleep, and numerous other health parameters and statuses [18]. Luo and Fan [19] believed that the development of wearable devices may trigger innovation in medical treatment models. Kirk [20] believed that wearable medical technology may significantly improve the curative effect for patients, reduce physician time, and lower medical costs, representing future trends of precise personalized medical treatment. Appropriate technology, such as smart wearable devices, may create unprecedented opportunities for medical and health care service providers to obtain the accurate real-time data required to prevent, diagnose, and treat various chronic diseases in a more economical manner [21]. Recent studies have expressed concerns about the long-term use of wearable devices, emphasizing the need to combine behavioral change techniques such as target setting, feedback, and rewards with other evidence-based techniques in order to use them effectively to prompt behavior changes [22].

However, consumer acceptance of wearable devices has not been as positive as expected. Many important issues that affect the ultimate adoption of wearable devices by consumers have been investigated, including the reliability [23,24] and validity of the measures that they monitor [25-27]. In addition, studies have confirmed that the key to a product’s success is the acceptance of wearable products by ordinary consumers and the comfort level of these products; product usability is an important aspect of this [28,29]. According to a definition by the International Organization for Standardization [30], usability refers to the effectiveness, efficiency, and user satisfaction rating of a product in a specific environment by a specific user for a specific purpose. It includes three aspects: effectiveness (ie, the accuracy and completeness of a goal that is achieved by a product); efficiency (ie, the effort required for a user to complete a task); and satisfaction (ie, the comfort and acceptability of a product). Usability tests and evaluations aim to make medical equipment easier, safer, and more effective and pleasant for users. A usability evaluation helps wearable devices to satisfy the requirements of the market and consumers [31].

In traditional usability studies, a thematic analysis based on heuristic evaluations, cognitive walkthroughs, think-alouds, and interviews is used; the results are then integrated together [32-34]. Although these methods are very popular, they have some limitations because results obtained in a laboratory environment can be difficult to interpret and translate into practice [35]. Questionnaires are another commonly employed method. They have been widely accepted by clinical medical informatics and consumer health informatics researchers and practitioners, and they are also widely applied in electronic health records, computerized physician order entry, and health applications [36-39]. At the same time, as a reliable, easy-to-operate, and low-cost method of usability evaluation, they have been recommended by the Agency for Healthcare Research and Quality [40]. Common questionnaire scales include the System Usability Scale (SUS) [41]; Ages and Stages Questionnaire; Computer System Usability Questionnaire; Post-Study System Usability Questionnaire [42]; Use, Satisfaction, and Ease [43]; and Web Analysis and Measurement Inventory [44].

Research has continuously established that usability evaluation and the resulting improvements in wearable device usability are critical to the wide and rapid adoption of wearable devices [45]. Research on the usability of wearable devices for health monitoring is still in the early stages. Schenkenfelder et al [46] investigated the user experience during running for three types of wearable devices (sports wristband, intelligent watch, and intelligent glasses). By analyzing feedback from 18 participants they found that the wristband and watch received higher SUS scores than the glasses. However, no significant differences between the wristband and watch were observed. Kaewkannate and Kim [47] compared the functions and features of four wearable devices (Withings Pulse, Misfit Shine, Jawbone Up24, and Fitbit Flex) and recruited seven users for objective and subjective evaluations. The Withings Pulse received the highest user acceptance, followed by the Fitbit Flex, Jawbone Up24, and Misfit Shine.

Significance of This Study

This study has some advantages over other similar studies: it used a larger number of devices (seven different intelligent watches and wristbands), including mainstream international and domestic devices, and involved a significantly larger number of participants (388 persons, the largest sample size among similar studies). We compared the functions and features of various devices and evaluated the usability of wearable health tracking devices currently on the market using a mature and stable SUS scale. These results will help to assess the acceptance level of wearable devices and to identify the influencing factors. As there is a growing consensus that wearable devices should be developed for health applications, this study will help to examine the issues affecting their large-scale use and it will contribute to the research and development of better health applications.

Methods

Device Selection

The device selection was based on open market performance data from NPD (a leading global consumer and retail data supplier) and Canalys (a leading global technology market analyst with a distinct channel focus) [48-50]. Our inclusion criteria for the wearable activity trackers included (1) continuous monitoring of some kind of physical activity (eg, steps), and (2) provision of feedback via a separate mobile device or personal computer. We considered a device to be a wearable activity tracker if it contained an accelerometer and connected to a mobile platform. The device also had to be able to connect wirelessly with handheld or desktop computers, and be compatible with either Android 1.6+ or Apple’s operating system iOS 6.0+. Therefore, current mainstream wearable devices with distinct market performance were selected as the research objects. These included the Apple Watch, Samsung Gear S, Fitbit Surge, Jawbone Up3, Mi Band, Huawei Honor Band B2, and Misfit Shine. Among these devices, the Apple Watch and Samsung Gear S are flagship products from leading mobile digital gadget vendors. The Fitbit Surge, Jawbone Up3, and Misfit Shine are well-known international sports wristband brands representative of different price ranges. Finally, the Mi Band and Huawei Honor Band B2 are leading brands of sports wristband in the Chinese market.

Questionnaire Selection

Usability refers to the effectiveness, efficiency, and user satisfaction rating of a product in a specific environment by a specific user for a specific purpose [51]. We chose to use the SUS, proposed by John Brooke in 1986, as the usability test tool. It is a simple scale based on a questionnaire and has been widely adopted in product usability evaluations. The SUS has the following advantages: (1) it is versatile and can be used to evaluate websites [52], software [53], mobile devices [54], and medical systems [55]; (2) it is a short questionnaire that is quick to answer; (3) a final score is provided with interpretation based on a well-established reference standard [56]; (4) it is free; (5) it is suitable even when applied to small samples (N<14); and (6) it has excellent reliability (0.85). Overall, the SUS is a quick and simple method for usability evaluation.

The SUS contains 10 questions based on the Likert five-point scale; questions 1, 3, 5, 7, and 9 are positive and questions 2, 4, 6, 8, and 10 are negative. The 10 questions are closely related and are employed for the comprehensive evaluation of a product. A higher SUS score indicates better product usability.

Details about the questionnaire’s design (the 10 questions) and scoring method are given in Multimedia Appendix 1.

Subject Recruitment and Test Design

Recruitment was based on a convenience sampling method. It leveraged WeChat, the social media platform with the largest user population in China. Over a period from July 1, 2016 to October 31, 2016, the research team used their WeChat accounts to send the questionnaire link to 2180 friends (between 8 and 314 each) and various WeChat groups (approximately 20 groups) and invited their friends to forward the link. A total of 388 volunteers who had experience with the seven wearable devices were recruited.

Once the volunteers agreed to take part in the study, consent was obtained and they were invited to complete an evaluation questionnaire about their experiences with these products.

This research passed an audit of the Peking University Biomedical Ethics Committee #IRB00001052 - 16008 - Exampt, and consent was obtained from all participants.

Data Collection

The questionnaires were distributed and the results collected via “sojump.com,” the largest free questionnaire survey platform in the world, which satisfied all the requirements for the questionnaires in this research [57]. The questionnaires were sent and completed via personal computers and mobile terminals. The final questionnaire “Comprehensive Evaluation Questionnaire for Intelligent Wearable Devices” is accessible online [58].

Statistical Analysis

For a descriptive statistical analysis, basic information about the respondents was collected, including gender, age, education, profession, and monthly income. Then, the product attributes (eg, price, weight, and battery life) and specific functions (whether 22 functions, such as step counting, activity time, activity type identification, sleep monitoring, and expandable new features, were supported) were summarized for the seven wearable devices. Finally, the SUS scores for the seven wearable devices used by the respondents were calculated.

For the inferential statistical analysis, the relationships between the SUS scores and product attributes (product price, weight, and battery life) were explored using Pearson product moment correlation (since price data are continuous and have a normal distribution) and Spearman rank correlation coefficient (since the weight and battery life do not have a normal distribution). The correlations between the SUS score and the functions of the 22 products (eg, step counting, activity time, activity type identification, sleep monitoring, and expandable new features) were analyzed using point-biserial correlation. The strength of the correlation was assessed based on Cohen criteria: correlations less than .30 are considered small, correlations between .30 and .50 are considered medium, and correlations greater than .50 are considered strong [59,60]. Finally, to analyze the participants’ attitudes to the usability of different products, the analysis of variance (ANOVA) method was used with the brand and user demographic information. These analyses were all completed using IBM SPSS version 20 software (IBM Corp, Armonk, NY, USA).

Results

Comparison of Functions and Attributes of Devices

Based on the product specifications and official websites, the features and general attributes of the seven devices were summarized and compared (see Tables 1 and 2). Table 1 summarizes the functions of each device. All seven devices had very powerful functions in three major categories (activity, health, and miscellaneous) and 20 features. The Fitbit Surge had the most features, followed by the Apple Watch, Huawei Honor B2, Samsung Gear S, Jawbone UP3, Mi Band, and Misfit Shine. The Samsung Gear S had the most sports features, the Jawbone UP3 had the most health features, and the Apple Watch had the most other additional features. All the devices supported three basic features: step counting, activity timer, and distance record. The Fitbit Surge was the only device to record the number of floors climbed, the Samsung Gear S had a unique sports track recording feature, and the Huawei Honor B2 had a unique wireless earphone. The Apple Watch, Samsung Watch, Jawbone Up3, and Fitbit Surge could measure heart rate. All the devices could monitor sleep, with the exception of the Apple Watch. Table 2 lists some of the major attributes of each device. Among the seven devices, the Misfit Shine had the longest battery life of 3 months, followed by the Mi Band, Fitbit Surge, Jawbone Up3, Huawei Honor wristband, Samsung Gear S, and Apple Watch, which had a battery life of only 18 hours. The Fitbit Surge, which weighed 354 grams, was the heaviest device, followed by the Jawbone UP3, Misfit Shine, Apple Watch, Huawei Honor Band B2, Samsung Gear S, and Mi Band, which weighed only 5 grams.

Table 1.

Function summary of the seven wearable devices.

Items Apple Watch Samsung Gear S Jawbone Up3 Fitbit Surge Misfit Shine Huawei Honor B2 Mi Band
Activity







Step counting Yes Yes Yes Yes Yes Yes Yes

Activity time Yes Yes Yes Yes Yes Yes Yes

Activity type identification —a — Yes Yes Yes Yes Yes

Floor climbing — — — Yes — — —

Energy consumption Yes Yes Yes Yes Yes Yes Yes

Running track — Yes — — — — —

Activity goal Yes Yes Yes Yes Yes Yes Yes

Professional fitness record Yes Yes — Yes — — Yes
Health







Heart rate Yes Yes Yes Yes — — —

Sleep monitoring — Yes Yes Yes Yes Yes Yes

Diet record — — Yes Yes Yes — —

Inactivity notification Yes — Yes — Yes Yes Yes
Miscellaneous







Incoming call alert Yes Yes — Yes Yes Yes Yes

Vibrating alarm clock Yes Yes Yes Yes Yes Yes Yes

Text message display Yes Yes — Yes Yes — —

Time display Yes Yes — Yes Yes — —

Remote control photograph Yes — — — Yes — —

Sports achievement sharing Yes — Yes Yes Yes Yes Yes

Bluetooth synchronization Yes Yes Yes Yes — Yes Yes

Expandable new features Yes Yes — — — — —
Total 15 14 12 16 14 11 12
Open in a new tab

aDevice does not support this feature.

Table 2.

Major attributes of the seven wearable devices.

Device Country Date to market Price (US$) Weight (g) Battery life (days)
Apple Watch USA September 10, 2014 $402 40.00 0.75
Samsung Gear S Korea September 25, 2013 $404 122.50 2.00
Fitbit Surge USA October 27, 2014 $279 354.00 7.00
Jawbone Up3 USA November 5, 2014 $183 29.00 7.00
Mi Band China July 22, 2014 $12 5.00 30.00
Huawei Honor B2 China March 1, 2015 $155 40.00 5.00
Misfit Shine USA September 14, 2013 $77 9.40 180.00
Open in a new tab

System Usability Scale Questionnaire Respondents

A total of 388 volunteers completed the questionnaire for this research. The volunteers included 83 Apple Watch users, 36 Samsung Gear S users, 37 Fitbit Surge users, 32 Jawbone Up3 users, 122 Mi Band users, 47 Huawei Honor Band B2 users, and 31 Misfit Shine users. Of the volunteers, 257 (66.2%) were male and 131 (33.8%) were female. The demographic information of the participants is shown in Table 3.

Table 3.

Demographic information of the respondents (N=388).

Items n (%)
Device to be evaluated

Apple Watch 83 (21.4)

Samsung Gear S 36 (9.3)

Fitbit Surge 37 (9.5)

Jawbone Up3 32 (8.2)

Mi Band 122 (31.4)

Huawei Honor B2 47 (12.1)

Misfit Shine 31 (8.0)
Length of time the device was used

<1 week 49 (12.6)

1 week-1 month 96 (24.7)

1-3 months 66 (17.0)

3-6 months 53 (13.7)

6 months-1 year 69 (17.8)

1-2 year 49 (12.6)

>2 years 6 (1.5)
Gender

Male 257 (66.2)

Female 131 (33.8)
Age

Under 18 10 (2.6)

18~25 107 (27.6)

26~30 88 (22.7)

31~40 113 (29.1)

41~50 56 (14.4)

51~60 13 (3.4)

>60 1 (0.3)
Education

Under primary school 3 (0.8)

Primary school-high school 19 (4.9)

Junior college 46 (11.9)

Bachelor 185 (47.7)

Master of Science or Master of Arts 98 (25.3)

PhD 37 (9.5)
Profession

Internet 114 (29.4)

Finance 28 (7.2)

Health care 92 (23.7)

Manufacturing 19 (4.9)

Fast consumer goods 4 (1.0)

Education 29 (7.5)

Law 6 (1.5)

Trade 6 (1.5)

Service 20 (5.2)

Advertising and media 6 (1.5)

Agriculture, forestry, fishery 2 (0.5)

Miscellaneous 62 (16.0)
Monthly income (US$)

<145 27 (7.0)

145~290 34 (8.8)

291~580 45 (11.6)

581~1160 93 (24.0)

>1160 189 (48.7)
Open in a new tab

System Usability Scale Scores for Each Device and Each Question

Based on the SUS calculation formula, the SUS scores for each brand are shown in Table 4. The mean SUS score for each question was calculated using SPSS, as shown in Table 5 (for full results, see Multimedia Appendix 2). For further visualization, please see corresponding boxplot diagrams in Multimedia Appendixes 3 and 4.

Table 4.

Total System Usability Scale (SUS) score of each device (N=388).

Device n Mean (SD) High score Low score % of total sum
Apple Watch 83 61.36 (14.69) 95.00 27.50 20.40
Samsung Gear S 36 62.08 (19.29) 100.00 5.00 9.00
Fitbit Surge 37 63.85 (21.97) 100.00 0.00 9.50
Jawbone Up3 32 65.94 (17.53) 100.00 27.50 8.50
Mi Band 122 65.12 (14.73) 100.00 35.50 31.80
Huawei Honor B2 47 67.61 (16.12) 100.00 22.50 12.70
Misfit Shine 31 65.97 (20.21) 100.00 10.00 8.20
Open in a new tab

Table 5.

Mean System Usability Scale (SUS) score for each question and device.

Device SUS score, mean (SD)

1 2 3 4 5 6 7 8 9 10
Apple Watch (n=83) 2.86 (1.00) 1.71 (1.16) 2.90 (0.89) 2.30 (1.26) 2.60 (0.95) 2.11 (1.14) 2.84 (0.89) 2.20 (1.23) 2.78 (0.96) 2.23 (1.26)
Samsung Gear S (n=36) 2.72 (1.26) 1.61 (1.29) 3.08 (1.02) 2.31 (1.37) 2.78 (0.96) 1.92 (1.27) 2.86 (1.02) 2.14 (1.44) 3.06 (0.95) 2.36 (1.31)
Fitbit Surge (n=37) 2.64 (1.38) 2.00 (1.41) 3.00 (1.07) 2.56 (1.52) 2.53 (1.06) 2.06 (1.26) 2.81 (1.17) 2.44 (1.38) 2.92 (1.08) 2.67 (1.22)
Jawbone Up3 (n=32) 2.90 (1.14) 1.90 (1.40) 2.97 (0.91) 2.58 (1.41) 2.71 (1.01) 2.16 (1.21) 2.94 (1.06) 2.55 (1.29) 3.10 (0.91) 2.42 (1.52)
Mi Band (n=122) 2.65 (1.02) 1.87 (1.30) 2.94 (1.04) 2.80 (1.33) 2.43 (1.01) 2.27 (1.13) 2.93 (1.04) 2.82 (1.11) 2.50 (1.13) 2.84 (1.19)
Huawei Honor B2 (n=47) 2.70 (1.07) 1.72 (1.19) 2.96 (1.07) 2.85 (1.28) 2.78 (0.94) 2.46 (1.21) 2.83 (0.97) 2.76 (1.21) 3.13 (0.88) 2.80 (1.13)
Misfit Shine (n=31) 2.87 (1.01) 2.33 (1.45) 3.10 (1.09) 2.87 (1.36) 2.40 (1.13) 2.27 (1.28) 2.83 (1.21) 2.60 (1.25) 2.77 (1.14) 2.50 (1.36)
Total (N=388) 2.74 (1.09) 1.84 (1.09) 2.97 (1.00) 2.61 (1.35) 2.57 (1.00) 2.20 (1.19) 2.87 (1.02) 2.54 (1.25) 2.80 (1.05) 2.58 (1.27)
Open in a new tab

Correlation Between System Usability Scale Score and Product Characteristics

We first explored the relationship between the SUS score and product characteristics (see Table 2). Pearson product moment correlation was used to analyze the relationship between the SUS value and the price, and Spearman rank correlation coefficient was used to analyze the relationship between the SUS values and the weight and battery life. We found a very weak correlation between the SUS score and the price of the product (r=–.10, P=.03), and no significant correlation between the SUS scores and the weight (r=–.04, P=.39) or battery life (r=.09, P=.10).

We then explored the relationship between the SUS scores and the functions supported by the products (see Table 1) using point-biserial correlation. We found that the SUS score had a very weak positive correlation (r=.11, P=.02) with the devices that supported activity type identification, and a very weak negative correlation (r=–.11, P=.02) with devices that supported expandable new features, as shown in Table 6.

Table 6.

Point-biserial correlation analysis between the System Usability Scale (SUS) scores and product functions (N=388).

Product function Point-biserial correlation

r P value
Activity type identification .11 .02
Expandable new features −.11 .02
Floor climbing .03 .58
Running track −.04 .40
Professional fitness record −.09 .08
Heart rate −.09 .07
Sleep monitoring .09 .07
Diet record .04 .45
Inactivity notification .01 .81
Incoming call alert .01 .85
Text message display −.06 .22
Time display −.06 .22
Remote control photograph −.05 .35
Sports achievement sharing .04 .40
Bluetooth synchronization .08 .13
Open in a new tab

Correlation Between System Usability Scale Score, Device Brand, and Participants’ Demographic Information

To investigate a participant’s perception of the usability of each of the seven brands of wearable devices, seven ANOVA tests were performed. These were used to determine the correlation between the SUS scores, device brand, and participants’ demographic information (length of time the device was used, gender, age, education, profession, and monthly income) (see Table 7).

Table 7.

ANOVA tests of the System Usability Scale (SUS) scores of the seven devices and of the volunteer’s demographic information (profession, length of time the device was used).

Measures Sum of squares Degrees of freedom Mean square F (df1,df2) P value
SUS scores of devices





Between groups 1671.52 6 278.59 0.98 (6,381) .44

Within groups 108,057.91 381 283.62 —a —

Total 109,729.43 387 — — —
Profession





Between groups 5618.23 11 510.75 1.84 (11,376) .04

Within groups 104,111.19 376 276.89 — —

Total 109,729.43 387 — — —
Length of time device used





Between groups 12,581.19 6 2096.86 8.22 (6,381) .001

Within groups 97,148.24 381 254.98 — —

Total 109,729.43 387 — — —
Open in a new tab

aNot applicable.

First, we found that there was no statistically significant difference between the SUS scores of the different brands (see Table 7), but there were significant differences in the SUS scores according to occupation and user experience (length of time the device was used). No significant difference was found for other features such as gender, age, education, and monthly income, as detailed in Multimedia Appendix 5. Second, we further used independent sample t tests to compare the mean SUS scores of respondents in the health care and internet industries; this indicated that the former scores were significantly higher than the latter. The SUS scores for the health care industry (n=92) were mean 68.7 (SD 14.5, range 35-100); for the internet industry (n=114) they were mean 61.1 (SD 17.9, range 5-100; t204=–3.24, P=.001). Third, we used Pearson product moment correlation method to explore the relationship between the SUS scores and the length of time the device was used. The SUS score had a moderate positive correlation (r=.32, P<.001) with user experience. In other words, the SUS score was related to a volunteer’s user experience (how long they had used the product). Other factors had no significant influence on the SUS score.

Discussion

Low System Usability Scale Scores for Seven Wearable Devices

The mean SUS score for the seven devices was 64.3 (SD 16.8). Although the Huawei Honor Band B2 received the highest score (mean 67.6, SD 16.1), based on the ANOVA analysis, there was no statistically significant difference in the mean SUS score of the seven products. Based on the SUS score, we believe that these products have little difference in terms of the ease of use. Aaron et al from AT&T Labs [61] added one question to the end of the SUS questionnaire, “Overall, I would rate the user friendliness of this product as…” This allowed users to rate the user interface using adjectives such as poor, OK, and good. The purpose of this was to associate the SUS with these adjectives. According to the data in this research, the user interfaces rated as “good” by users received a mean SUS score of 71.4. Therefore, the usability of all seven wearable devices was “good” or “OK.” The Huawei Honor Band B2, which received the highest SUS score of 67.6 (SD 16.1), was also in the “good” range, whereas the Apple Watch received the lowest score of 61.3 (SD 14.6). Although the seven brands received different SUS scores, the ANOVA test of the SUS scores versus device brands revealed no significant correlation between them. We believe that the usability of a wearable health tracking device is moderate with no significant differences between brands. These products are immature and additional breakthroughs in the core technology are required. In this sector, there is no leading brand with an absolute competitive edge, and there is no significant difference between domestic and foreign brands.

Correlation Between System Usability Scale Score and Demographic Characteristics

The SPSS analysis indicated that the SUS scores for wearable devices are related to demographic characteristics. Factors such as user experience and profession may affect the scores. The results also revealed that user experience (the time length the device was used) could be an important factor affecting the SUS score. We found that there was a moderate positive correlation between the SUS scores and user experience (r=.32, P<.001); this matches the conclusions of MacDorman et al [62] and Kortum et al [63]. The participants in medical and health care industries gave a significantly higher score (mean 68.7, SD 14.5) than the participants in other industries, particularly those in the internet industry (mean 61.1, SD 17.9), with the former giving scores that approached the “good” level (71.4) in the acceptance diagram [61]. We conducted independent sample t tests to compare the mean SUS scores of participants in the health care and internet industries, which indicated that the former score was significantly better than the latter (P<.001). This suggests that in our sample set medical and health care employees evaluated the devices more highly and that their acceptance level of wearable device usability was higher than internet employees, indirectly showing that the demand for wearable devices is more urgent in health-related industries, a potential reason or phenomenon worthy of further research and analysis in the future. This may be due to the serious shortage of medical resources in China, especially in remote areas. The gap between supply and demand provides opportunities for mobile medical-based wearable equipment, whereas the rapid development of mobile internet and big data technologies provide the support required for the development of mobile medical treatments. In the future, patients with chronic diseases such as coronary heart disease, hypertension, and diabetes will not only receive drug treatment, but integrated disease management programs including remote monitoring, tele-treatment plan adjustments, and lifestyle management through wearable technology. Previous SUS research [61] indicated that no control was needed for gender or that gender was not a key influencing factor for consumer-grade products. Our result has validated this conclusion.

Correlation Between System Usability Scale Score and Product Characteristics

Some characteristics of wearable devices are related to the SUS scores. Our study found that the intelligent recognition of activity type had a very weak positive correlation with the SUS score, and that expandable new functions and price have a very weak negative correlation with the SUS score. We believe that these findings, to some extent, show a “user portrait” of Chinese wearable device consumers who are likely to prefer devices that are functional and easy to operate (a weak positive correlation between the SUS score and the intelligent recognition function of the device). The price was a favorable factor, although it was not strong (the SUS score had a weak negative correlation with the price of device). We found that this is in line with the current market strategy of the Mi Band bracelet which applies the lowest price (only US $12) and the most core functions (such as steps, exercise time, sleep time, and resting time) to maximize the market share. Using this strategy, Mi Technologies Inc sold more than 3.6 million Mi Band bracelets in China in the first quarter of 2017, surpassing the giant manufacturers Fitbit, Apple, and Samsung to become the world’s largest wearable device maker [64]. In addition, we found that Q3 and Q7 (see Multimedia Appendix 1) had the highest scores: 2.97 and 2.87, respectively (see Table 5 and Multimedia Appendix 2). These scores were related to the ease of use (accessibility) of the product and the fact that people are generally more inclined to use products that do not require experience or focus and do not produce, or produce less, cognitive stress [65]. We think that this may be one reason for the high short-term usage rate of wearable devices. Although the use of wearable devices can enhance the ability to monitor the user’s behavior, if they perform similar activities and movements every day then, over time, it becomes easier to stop using the device as the user’s perception of the link between the results of the behavior intervention and the device monitoring is lost [66]. In order to prolong the use-life cycle and long-term wear rate of wearable devices, we need to integrate more cognitive behavior change strategies and functions to promote and consolidate the changes in users’ habits. This includes the identification of behavioral disorders and adjustment of cognitive attitudes [67]. At the same time, we believe that the low SUS scores of the current fitness wearable devices are due to the users’ dissatisfaction with the function and effect of the equipment, or their lack of cognitive motivation to change their behavior for a healthy lifestyle [22]. Therefore, we suggest that developers focus on the core requirements of the user, combined with social support, environmental support factors, customizable self-monitoring schemes, and personalized feedback, which should be integrated into the wearable device’s regular monitoring functions. In this way, we can help to improve the user’s overall usability evaluation of such products, prolong their use time, increase product adhesion, and cultivate user loyalty.

Implications of Feature Analysis: Function Homogeneity, Awaiting Killer Function

Wearable devices have great potential in personal health management and clinical care. In future, the cost of wearable devices will decrease, such as the Mi Band which is sold for only US $12. Wearable devices can record sports data, help set exercise goals, and avoid excessive movement, making it easy to obtain sports performance data and maintaining and promoting the passion for sport. In addition, wearable devices can play a role in clinical practice. An obvious characteristic of wearable devices is the continuous monitoring of health data, which can provide clues for disease diagnosis, identify disease symptoms in a timely manner, and help doctors to make more comprehensive and accurate judgments.

Current mainstream wearable devices, including the seven brands in this study, are powerful; they not only track and record health data, such as sports data, sleep, and heart rate, but also provide numerous additional functions (including call notification, social media sharing, and Bluetooth). However, the function and feature analyses show that their functions are homogeneous and most concentrated on three functions: activity, reminders, and health monitoring. From this point of view, the homogenization of the product functions is very serious, and the functions of each product are lacking. There are no killer functions that solve major health management or clinical questions. Accordingly, the perceived usability of mainstream wearable devices is unsatisfactory and customer loyalty is not high. In fact, some recent studies have shown that many users use activity trackers for only a short time [68].

Characteristics and Limitation of This Research

In this research, the features of different devices were compared and their usability evaluated using the SUS. Compared to similar studies, we investigated the most extensive range of devices and used the largest group of participants. However, this research has its limitations. First, we adopted the snowball convenient randomization method; the participants were recruited based on the research teams’ circle of WeChat friends. Although the final respondents recruited were distributed across the country, this sample is not sufficient to represent consumers of wearable devices nationwide. Nonetheless, since our sample size was large and the responses were autonomous, we believe that our group still has a certain degree of representativeness. This is reinforced by the fact that the proportion of each brand in our study (Mi 31.4%, Huawei 12.1%, and Samsung 9.3%) is similar to the market share of intelligent wristbands in the 2014 report by Tencent ISUX [69] (Mi 34.3%, Huawei 12.5%, and Samsung 10.5%). Second, the survey was conducted using online questionnaires. Although this method is more convenient, faster, and more efficient than a conventional paper questionnaire in terms of distribution, collection, and analysis, many factors are uncontrollable and some factors, such as behavioral cognitive factors [70], social network factors [71], and environmental support factors [72], may affect the results.

Conclusions

The homogenization of wearable device functions is obvious, the usability of popular wearable devices on the market is unsatisfactory, and these devices have not been completely accepted by consumers. Usability is similar among the different brands with no absolute leader. No significant differences between domestic and international brands were observed. A consumer’s SUS score for wearable devices is related to their personal situation rather than the device brand. Device manufacturers should put more effort into developing innovative functions and improving the usability of their products.

Acknowledgments

We thank all volunteers who participated in this research. This study was sponsored by the National Natural Science Foundation of China (grants #81471756, #81771937, and #81871455) and the Medical and Health Planning Project of Zhejiang Province of China (grant #2017KY386).

Abbreviations

ANOVA

analysis of variance

SUS

System Usability Scale

Multimedia Appendix 1

System Usability Scale (SUS) questionnaire design and scoring method.

Multimedia Appendix 2

System Usability Scale (SUS) score of each question for each device.

Multimedia Appendix 3

System Usability Scale (SUS) scores for all devices.

Multimedia Appendix 4

System Usability Scale (SUS) scores for all questions.

Multimedia Appendix 5

ANOVA tests of the System Usability Scale (SUS) scores and the volunteer’s demographic information.

Footnotes

Authors' Contributions: XZ and DX were responsible for the questionnaire preparation and data collection; XL was responsible for literature review; JL was responsible for the data analysis and manuscript draft; JF and BT supervised the data analysis and interpretation; and JL was responsible for study design and revision of the manuscript. All authors reviewed this final manuscript and expressed agreement regarding its content.

Conflicts of Interest: None declared.

References

  • 1.Sungmee P, Jayaraman S. A transdisciplinary approach to wearables, big data and quality of life. Conf Proc IEEE Eng Med Biol Soc. 2014;2014:4155–4158. doi: 10.1109/EMBC.2014.6944539. [DOI] [PubMed] [Google Scholar]
  • 2.Bateman DR, Srinivas B, Emmett TW, Schleyer TK, Holden RJ, Hendrie HC, Callahan CM. Categorizing health outcomes and efficacy of mHealth apps for persons with cognitive impairment: a systematic review. J Med Internet Res. 2017 Aug 30;19(8):e301. doi: 10.2196/jmir.7814. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Danova T. Business Insider. 2015. Jan 07, [2018-03-05]. Apple Watch will control nearly half of the entire smartwatch market by 2017 http://www.businessinsider.com/wearable-computing-market-statistics-2015-1 .
  • 4.Shirer M, Liamas R, Ubrani J. IDC. 2018. Jun 04, [2018-08-21]. Wearable device shipments slow in Q1 2018 as consumers shift from basic wearables to smarter devices, according to IDC http://www.idc.com/getdoc.jsp?containerId=prUS43900918 .
  • 5.Waltzer S. Strategy Analytics. 2018. Aug 02, [2018-08-21]. Global smartwatch vendor market share by region: Q2 2018 https://tinyurl.com/y72zuydk .
  • 6.Danova T. Yahoo! Finance. 2014. Oct 29, [2018-03-05]. The Wearable Computing Market Report: growth trends, consumer attitudes, and why smartwatches will dominate https://tinyurl.com/y86hey85 .
  • 7.36kr. 2015. Jul 03, [2018-03-08]. Tencent ISUX released domestic wearable devices report, the user retention rate is low, professional is the main demand http://36kr.com/p/5035222.html .
  • 8.Brakenridge CL, Healy GN, Winkler EA, Fjeldsoe BS. Usage, acceptability, and effectiveness of an activity tracker in a randomized trial of a workplace sitting intervention: mixed-methods evaluation. Interact J Med Res. 2018 Mar 02;7(1):e5. doi: 10.2196/ijmr.9001. http://www.i-jmr.org/2018/1/e5/ v7i1e5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Koydemir HC, Ozcan A. Wearable and implantable sensors for biomedical applications. Annual Rev Anal Chem. 2018 Jun 12;11(1):127–146. doi: 10.1146/annurev-anchem-061417-125956. [DOI] [PubMed] [Google Scholar]
  • 10.Niwano S, Sekiguchi Y, Ishii Y, Iwasaki Y, Kato R, Okamura H, Takagi M, Kurita T, Nitta T. Clinical usefulness of wearable cardioverter defibrillator (WCD) and current understanding of its clinical indication in Japan. Circ J. 2018;82(6):1481–1486. doi: 10.1253/circj.CJ-17-1336. [DOI] [PubMed] [Google Scholar]
  • 11.Shen J, Naeim A. Telehealth in older adults with cancer in the United States: The emerging use of wearable sensors. J Geriatr Oncol. 2017 Nov;8(6):437–442. doi: 10.1016/j.jgo.2017.08.008.S1879-4068(17)30171-6 [DOI] [PubMed] [Google Scholar]
  • 12.Liu Y, Wang H, Zhao W, Zhang M, Qin H, Xie Y. Flexible, stretchable sensors for wearable health monitoring: sensing mechanisms, materials, fabrication strategies and features. Sensors (Basel) 2018 Feb 22;18(2) doi: 10.3390/s18020645. http://www.mdpi.com/resolver?pii=s18020645 .s18020645 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Nenggan Z, Zhaohui W, Man L, Yang L, Gang P. Infrastructure and reliability analysis of electric networks for e-Textiles. IEEE Trans Syst Man Cybern C. 2010 Jan;40(1):36–51. doi: 10.1109/TSMCC.2009.2031497. [DOI] [Google Scholar]
  • 14.Chen J, Lum PS. Pilot testing of the spring operated wearable enhancer for arm rehabilitation (SpringWear) J Neuroeng Rehabil. 2018 Mar 02;15(1):13. doi: 10.1186/s12984-018-0352-4. https://jneuroengrehab.biomedcentral.com/articles/10.1186/s12984-018-0352-4 .10.1186/s12984-018-0352-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Bell W, Colaiezzi BA, Prata CS, Coates JC. Scaling up dietary data for decision-making in low-income countries: new technological frontiers. Adv Nutr. 2017 Nov;8(6):916–932. doi: 10.3945/an.116.014308. http://europepmc.org/abstract/MED/29141974 .4772178 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Lee H, Lee H, Whang M. An enhanced method to estimate heart rate from seismocardiography via ensemble averaging of body movements at six degrees of freedom. Sensors (Basel) 2018 Jan 15;18(1) doi: 10.3390/s18010238. http://www.mdpi.com/resolver?pii=s18010238 .s18010238 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Lee H, Hong YJ, Baik S, Hyeon T, Kim D. Enzyme-based glucose sensor: from invasive to wearable device. Adv Healthc Mater. 2018 Apr;7(8):e1701150. doi: 10.1002/adhm.201701150. [DOI] [PubMed] [Google Scholar]
  • 18.Lee J, Lee DH, Oh TJ, Kim KM, Choi SH, Lim S, Park YJ, Park DJ, Jang HC, Moon JH. Clinical feasibility of continuously monitored data for heart rate, physical activity, and sleeping by wearable activity trackers in patients with thyrotoxicosis: protocol for a prospective longitudinal observational study. JMIR Res Protoc. 2018 Feb 21;7(2):e49. doi: 10.2196/resprot.8119. http://www.researchprotocols.org/2018/2/e49/ v7i2e49 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Luo X, Fan W. Medical wearable devices and the health care model innovation. Chinese J Health Manage. 2016;2016(1):70–75. doi: 10.3760/cma.j.issn.1674-0815.2016.01.016. [DOI] [Google Scholar]
  • 20.Kirk SR. Wearable medical technology-the future of personalized medicine is here. Chinese J Med Manage Sci. 2016;6(1):34–37. doi: 10.4072/j.issn.2095-7432.2016.01.006. [DOI] [Google Scholar]
  • 21.Phillips SM, Cadmus-Bertram L, Rosenberg D, Buman MP, Lynch BM. Wearable technology and physical activity in chronic disease: opportunities and challenges. Am J Prev Med. 2018 Jan;54(1):144–150. doi: 10.1016/j.amepre.2017.08.015.S0749-3797(17)30456-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Sullivan AN, Lachman ME. Behavior change with fitness technology in sedentary adults: a review of the evidence for increasing physical activity. Front Public Health. 2016;4:289. doi: 10.3389/fpubh.2016.00289. doi: 10.3389/fpubh.2016.00289. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Wen D, Zhang X, Liu X, Lei J. Evaluating the consistency of current mainstream wearable devices in health monitoring: a comparison under free-living conditions. J Med Internet Res. 2017 Dec 07;19(3):e68. doi: 10.2196/jmir.6874. http://www.jmir.org/2017/3/e68/ v19i3e68 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Gruwez A, Libert W, Ameye L, Bruyneel M. Reliability of commercially available sleep and activity trackers with manual switch-to-sleep mode activation in free-living healthy individuals. Int J Med Inform. 2017 Dec;102:87–92. doi: 10.1016/j.ijmedinf.2017.03.008.S1386-5056(17)30069-2 [DOI] [PubMed] [Google Scholar]
  • 25.Cadmus-Bertram L, Gangnon R, Wirkus EJ, Thraen-Borowski KM, Gorzelitz-Liebhauser J. The accuracy of heart rate monitoring by some wrist-worn activity trackers. Ann Intern Med. 2017 Apr 18;166(8):610–612. doi: 10.7326/L16-0353. http://europepmc.org/abstract/MED/28395305 .2618339 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Salisbury JP, Keshav NU, Sossong AD, Sahin NT. Concussion assessment with smartglasses: validation study of balance measurement toward a lightweight, multimodal, field-ready platform. JMIR Mhealth Uhealth. 2018 Jan 23;6(1):e15. doi: 10.2196/mhealth.8478. http://mhealth.jmir.org/2018/1/e15/ v6i1e15 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Xie J, Wen D, Liang L, Jia Y, Gao L, Lei J. Evaluating the validity of current mainstream wearable devices in fitness tracking under various physical activities: comparative study. JMIR Mhealth Uhealth. 2018 Apr 12;6(4):e94. doi: 10.2196/mhealth.9754. http://mhealth.jmir.org/2018/4/e94/ v6i4e94 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Wen D, Zhang X, Lei J. Consumers' perceived attitudes to wearable devices in health monitoring in China: a survey study. Comput Methods Programs Biomed. 2017 Mar;140:131–137. doi: 10.1016/j.cmpb.2016.12.009.S0169-2607(16)30873-2 [DOI] [PubMed] [Google Scholar]
  • 29.Mercer K, Giangregorio L, Schneider E, Chilana P, Li M, Grindrod K. Acceptance of commercially available wearable activity trackers among adults aged over 50 and with chronic illness: a mixed-methods evaluation. JMIR Mhealth Uhealth. 2016 Jan 27;4(1):e7. doi: 10.2196/mhealth.4225. http://mhealth.jmir.org/2016/1/e7/ v4i1e7 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Usability Partners. 1998. [2018-03-08]. ISO Standards in usability and user-centred design—ISO 9241-11 (1998) guidance in usability http://www.usabilitypartners.se/about-usability/iso-standards .
  • 31.Yan Y, Wang G, Liu S, Wu H, Zhang Q. Usability evaluation of infusion pump based on system usability scale. China Medical Devices. 2012;10(1):25–27. doi: 10.3969/j.issn.1674-1633.2012.10.005. [DOI] [Google Scholar]
  • 32.Khajouei R, Hajesmaeel Gohari S, Mirzaee M. Comparison of two heuristic evaluation methods for evaluating the usability of health information systems. J Biomed Inform. 2018 Apr;80:37–42. doi: 10.1016/j.jbi.2018.02.016.S1532-0464(18)30036-4 [DOI] [PubMed] [Google Scholar]
  • 33.Khajouei R, Zahiri Esfahani M, Jahani Y. Comparison of heuristic and cognitive walkthrough usability evaluation methods for evaluating health information systems. J Am Med Inform Assoc. 2017 Apr 01;24(e1):e55–e60. doi: 10.1093/jamia/ocw100.ocw100 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Hudson D, Kushniruk A, Borycki E, Zuege DJ. Physician satisfaction with a critical care clinical information system using a multimethod evaluation of usability. Int J Med Inform. 2018 Apr;112:131–136. doi: 10.1016/j.ijmedinf.2018.01.010.S1386-5056(18)30010-8 [DOI] [PubMed] [Google Scholar]
  • 35.Ho GW. Examining perceptions and attitudes. West J Nurs Res. 2017 Dec;39(5):674–689. doi: 10.1177/0193945916661302.0193945916661302 [DOI] [PubMed] [Google Scholar]
  • 36.Hoonakker PL, Carayon P, Walker JM. Measurement of CPOE end-user satisfaction among ICU physicians and nurses. Appl Clin Inform. 2010 Jan 01;1(3):268–285. doi: 10.4338/ACI-2010-03-RA-0020. http://europepmc.org/abstract/MED/21423791 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Alfonsson S, Maathz P, Hursti T. Interformat reliability of digital psychiatric self-report questionnaires: a systematic review. J Med Internet Res. 2014 Dec 03;16(12):e268. doi: 10.2196/jmir.3395. http://www.jmir.org/2014/12/e268/ v16i12e268 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Li J, Talaei-Khoei A, Seale H, Ray P, Macintyre CR. Health care provider adoption of eHealth: systematic literature review. Interact J Med Res. 2013 Apr 16;2(1):e7. doi: 10.2196/ijmr.2468. http://www.i-jmr.org/2013/1/e7/ v2i1e7 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Ritter P, Lorig K, Laurent D, Matthews K. Internet versus mailed questionnaires: a randomized comparison. J Med Internet Res. 2004 Sep 15;6(3):e29. doi: 10.2196/jmir.6.3.e29. http://www.jmir.org/2004/3/e29/ v6e29 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Johnson C, Johnston D, Crowley P, Culbertson H, Rippen H, Damico D. EHR Usability Toolkit: A Background Report on Usability and Electronic Health Records. Rockville, MD: Agency for Healthcare Research and Quality; 2011. [2018-08-01]. https://healthit.ahrq.gov/sites/default/files/docs/citation/EHR_Usability_Toolkit_Background_Report.pdf . [Google Scholar]
  • 41.Jordan P, Jordan JW, McClelland I. Usability Evaluation in Industry. London: Taylor & Francis; 1996. [Google Scholar]
  • 42.Lewis JR. IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. Int J Hum-Comput Int. 1995 Jan;7(1):57–78. doi: 10.1080/10447319509526110. [DOI] [Google Scholar]
  • 43.Lund A. ResearchGate. 2001. [2018-08-01]. Measuring usability with the USE questionnaire https://www.researchgate.net/profile/Arnold_Lund/publication/230786746_Measuring_Usability_with_the_USE_Questionnaire/links/56e5a90e08ae98445c21561c/Measuring-Usability-with-the-USE-Questionnaire.pdf .
  • 44.Kirakowski J, Cierlik B. Measuring the usability of web sites. Proceedings of the Human Factors and Ergonomics Society Annual Meeting; 42nd Human Factors and Ergonomics Society Annual Meeting; Oct 5-9, 1998; Chicago, IL. 1998. Oct 02, pp. 424–428. [DOI] [Google Scholar]
  • 45.McCallum C, Rooksby J, Gray CM. Evaluating the impact of physical activity apps and wearables: interdisciplinary review. JMIR Mhealth Uhealth. 2018 Mar 23;6(3):e58. doi: 10.2196/mhealth.9054. http://mhealth.jmir.org/2018/3/e58/ v6i3e58 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 46.Schenkenfelder R, Selinger S. A Comparison of Multiple Wearable Devices Regarding their User Experience During Running. [2018-03-08]. http://ffhoarep.fh-ooe.at/bitstream/123456789/686/1/125_218_Schenkenfelder_FullPaper_en_Final.pdf .
  • 47.Kaewkannate K, Kim S. A comparison of wearable fitness devices. BMC Public Health. 2016 Dec 24;16:433. doi: 10.1186/s12889-016-3059-0. https://bmcpublichealth.biomedcentral.com/articles/10.1186/s12889-016-3059-0 .10.1186/s12889-016-3059-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48.Canalys. 2015. May 20, [2018-03-08]. Media alert: Fitbit maintains leadership share of wearable band market before Apple Watch entrance https://tinyurl.com/y9bus445 .
  • 49.Canalys. 2014. May 21, [2018-03-08]. Fitbit accounted for nearly half of global wearable band shipments in Q1 2014 http://www.canalys.com/newsroom/fitbit-accounted-nearly-half-global-wearable-band-shipments-q1-2014 .
  • 50.Dolan B. MobiHealthNews. 2014. Jan 15, [2018-03-08]. Fitbit, Jawbone, Nike had 97 percent of fitness tracker retail sales in 2013 http://www.mobihealthnews.com/28825/fitbit-jawbone-nike-had-97-percent-of-fitness-tracker-retail-sales-in-2013 .
  • 51.Bevan N, Carter J, Harker S. ISO 9241-11 Revised: what have we learnt about usability since 1998? Lect Notes Comput Sci. 2015;9169(1):143–151. doi: 10.1007/978-3-319-20901-2_13. [DOI] [Google Scholar]
  • 52.Flavián C, Guinalíu M, Gurrea R. The role played by perceived usability, satisfaction and consumer trust on website loyalty. Inform Manage. 2006 Jan;43(1):1–14. doi: 10.1016/j.im.2005.01.002. [DOI] [Google Scholar]
  • 53.Greene K, Byrne M, Everett S. A comparison of usability between voting methods. EVT; USENIX/Accurate Electronic Voting Technology Workshop 2006 on Electronic Voting Technology Workshop; July 31-Aug 4, 2006; Vancouver, BC. 2006. [Google Scholar]
  • 54.Campbell BA, Tossell CC, Byrne MD, Kortum P. Voting on a smartphone: evaluating the usability of an optimized voting system for handheld mobile devices. Proc Hum Factors Ergon Soc; Human Factors and Ergonomics Society Annual Meeting; Sep 19-23, 2011; Las Vegas. 2011. Sep 01, pp. 1100–1104. [DOI] [Google Scholar]
  • 55.Trafton J, Martins S, Michel M, Lewis E, Wang D, Combs A, Scates N, Tu S, Goldstein MK. Evaluation of the acceptability and usability of a decision support system to encourage safe and effective use of opioid therapy for chronic, noncancer pain by primary care providers. Pain Med. 2010 Apr;11(4):575–585. doi: 10.1111/j.1526-4637.2010.00818.x.PME818 [DOI] [PubMed] [Google Scholar]
  • 56.Lewis J, Sauro J. The factor structure of the system usability scale. Lect Notes Comput Sc. 2009;5619(1):94–103. doi: 10.1007/978-3-642-02806-9_12. [DOI] [Google Scholar]
  • 57.WJX. [2018-03-08]. Introduction of Sojump http://www.wjx.cn/html/aboutus.aspx .
  • 58.Liang J, Xian D, Liu X, Fu J, Zhang X, Tang B, Lei J. Comprehensive evaluation questionnaire for intelligent wearable devices. 2018. [2018-11-02]. https://www.wjx.cn/jq/5733344.aspx .
  • 59.Wong WS, Chen PP, Chow YF, Wong S, Fielding R. A study of the reliability and concurrent validity of the Chinese version of the Pain Medication Attitude Questionnaire (ChPMAQ) in a sample of Chinese patients with chronic pain. Pain Med. 2016 Jun;17(6):1137–1144. doi: 10.1093/pm/pnv086.pnv086 [DOI] [PubMed] [Google Scholar]
  • 60.Cohen J. Statistical Power Analysis for the Behavioral Sciences. London: Taylor & Francis; 1988. [Google Scholar]
  • 61.Aaron B, Philip K, James M. Journal of Usability Studies. 2009. [2018-08-01]. Determining what individual SUS scores mean: adding an adjective rating scale https://tinyurl.com/yax6fx5n .
  • 62.MacDorman KF, Whalen TJ, Ho C, Patel H. An improved usability measure based on novice and expert performance. Int J Hum-Comput Int. 2011 Feb 08;27(3):280–302. doi: 10.1080/10447318.2011.540472. [DOI] [Google Scholar]
  • 63.Kortum PT, Bangor A. Usability ratings for everyday products measured with the system usability scale. Int J Hum-Comput Int. 2013 Jan;29(2):67–76. doi: 10.1080/10447318.2012.681221. [DOI] [Google Scholar]
  • 64.IDC. 2017. Oct, [2018-06-01]. Worldwide wearables market shares, 1H17: Xiaomi leads the market by volume, Apple leads by value http://www.idc.com/getdoc.jsp?containerId=US43141317 .
  • 65.Lidwell W, Holden K, Butler J. Universal Principles of Design, Revised and Updated 125 Ways to Enhance Usability, Influence Perception, Increase Appeal, Make Better Design Decisions. Beverly, MA: Rockport Publisher; 2010. [Google Scholar]
  • 66.Karapanos E, Gouveia R, Hassenzahl M, Forlizzi J. Wellbeing in the making: peoples' experiences with wearable activity trackers. Psychol Well Being. 2016;6:4. doi: 10.1186/s13612-016-0042-6. http://europepmc.org/abstract/MED/27376017 .42 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 67.Michie S, Richardson M, Johnston M, Abraham C, Francis J, Hardeman W, Eccles MP, Cane J, Wood CE. The behavior change technique taxonomy (v1) of 93 hierarchically clustered techniques: building an international consensus for the reporting of behavior change interventions. Ann Behav Med. 2013 Aug;46(1):81–95. doi: 10.1007/s12160-013-9486-6. [DOI] [PubMed] [Google Scholar]
  • 68.Patel MS, Asch DA, Volpp KG. Wearable devices as facilitators, not drivers, of health behavior change. JAMA. 2015 Feb 3;313(5):459–460. doi: 10.1001/jama.2014.14781.2089651 [DOI] [PubMed] [Google Scholar]
  • 69.Tencent ISUX. 2014. [2018-03-08]. 2014 smart wearable market white paper http://digi.tech.qq.com/a/20141231/032756.html .
  • 70.Herning MM, Cook JH, Schneider JK. Cognitive behavioral therapy to promote exercise behavior in older adults: implications for physical therapists. J Geriatr Phys Ther. 2005;28(2):34–38. doi: 10.1519/00139143-200508000-00001. [DOI] [PubMed] [Google Scholar]
  • 71.Rovniak LS, Kong L, Hovell MF, Ding D, Sallis JF, Ray CA, Kraschnewski JL, Matthews SA, Kiser E, Chinchilli VM, George DR, Sciamanna CN. Engineering online and in-person social networks for physical activity: a randomized trial. Ann Behav Med. 2016 Dec;50(6):885–897. doi: 10.1007/s12160-016-9814-8. http://europepmc.org/abstract/MED/27405724 .10.1007/s12160-016-9814-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 72.Sallis JF, Cerin E, Conway TL, Adams MA, Frank LD, Pratt M, Salvo D, Schipperijn J, Smith G, Cain KL, Davey R, Kerr J, Lai P, Mitáš J, Reis R, Sarmiento OL, Schofield G, Troelsen J, Van Dyck D, De Bourdeaudhuij I, Owen N. Physical activity in relation to urban environments in 14 cities worldwide: a cross-sectional study. Lancet. 2016 May 28;387(10034):2207–2217. doi: 10.1016/S0140-6736(15)01284-2.S0140-6736(15)01284-2 [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Multimedia Appendix 1

System Usability Scale (SUS) questionnaire design and scoring method.

mhealth_v6i11e11066_app1.pdf (33.4KB, pdf)
Multimedia Appendix 2

System Usability Scale (SUS) score of each question for each device.

mhealth_v6i11e11066_app2.pdf (30.8KB, pdf)
Multimedia Appendix 3

System Usability Scale (SUS) scores for all devices.

mhealth_v6i11e11066_app3.pdf (511.8KB, pdf)
Multimedia Appendix 4

System Usability Scale (SUS) scores for all questions.

mhealth_v6i11e11066_app4.pdf (436.6KB, pdf)
Multimedia Appendix 5

ANOVA tests of the System Usability Scale (SUS) scores and the volunteer’s demographic information.

mhealth_v6i11e11066_app5.pdf (32.9KB, pdf)

Articles from JMIR mHealth and uHealth are provided here courtesy of JMIR Publications Inc.

RESOURCES