Potential Applications of Mobile and Wearable Devices for Psychological Support During the COVID-19 Pandemic: A Review

Abstract

The coronavirus disease 19 (COVID-19) pandemic that has been raging in 2020 does affect not only the physical state but also the mental health of the general population, particularly, that of the healthcare workers. Given the unprecedented large-scale impacts of the COVID-19 pandemic, digital technology has gained momentum as invaluable social interaction and health tracking tools in this time of great turmoil, in part due to the imposed state-wide mobilization limitations to mitigate the risk of infection that might arise from in-person socialization or hospitalization. Over the last five years, there has been a notable increase in the demand and usage of mobile and wearable devices as well as their adoption in studies of mental fitness. The purposes of this scoping review are to summarize evidence on the sweeping impact of COVID-19 on mental health as well as to evaluate the merits of the devices for remote psychological support. We conclude that the COVID-19 pandemic has inflicted a significant toll on the mental health of the population, leading to an upsurge in reports of pathological stress, depression, anxiety, and insomnia. It is also clear that mobile and wearable devices (e.g., smartwatches and fitness trackers) are well placed for identifying and targeting individuals with these psychological burdens in need of intervention. However, we found that most of the previous studies used research-grade wearable devices that are difficult to afford for the normal consumer due to their high cost. Thus, the possibility of replacing the research-grade wearable devices with the current smartwatch is also discussed.

I. Introduction

The world is currently facing a public health calamity driven by the global pandemic of coronavirus disease 19 (COVID-19) [1]. Deriving a conclusion from an abundant collection of evidence, the World Health Organization (WHO) points out that the virus SARS-CoV-2 is primarily transmitted from one person to another via respiratory droplets [2]. As such, WHO recommends the general population to keep a safe physical distance and vigilantly maintain hand hygiene by sanitizing one’s hands with alcohol-based gel or solution, or washing them thoroughly with soap or detergent [3]. Despite several attempts by researchers to come up with the magic-bullet treatment for COVID-19, to date, no specific antiviral medication has yet been approved by the Food and Drug Administration (FDA) [4]. Currently, the first-line treatment for those contracting COVID-19 is symptomatic and supportive therapies [5]. There is a silver lining, given copious vaccine research projects that have been ongoing or are set to begin in the forthcoming months for the possibility that a cure will sooner, rather than later, be discovered [6]. The unfortunate news, however, involves the estimation by most experts that COVID-19 vaccines would take 12 to 18 months before reaching market readiness [7].

With no available vaccine nor specific treatment options, restricting social activities is the obvious route for policymakers to implement. In fact, this measure has been put in place in numerous countries crippled by the exponential rise in case numbers and other countries with less alarming increase alike. It has been reasoned that using nation-wide lockdowns to help flatten the spread of COVID-19 would provide some breathing room for the health system to handle the outbreak. Evidently, the lockdown policy in Wuhan, China, resulted in a significant reduction in the infection rate [8]. In fact, lockdowns have proven rather successful in slowing the spread of the virus. However, this decrease in case numbers was made to occur at the expense of a significant decline in economic activity [9]. Simply put, the COVID-19 pandemic has not only debilitated the health system but also continues to devastate the world economy [10].

It is now a foregone conclusion that the global economy will experience a recession in several years to come, although the degree of severity remains to be seen. The economic crisis is also expected to have multifarious knock-on impacts on the population’s mental health [11]. Both emerging diseases and recession can contribute to the disruption of market activities on a global scale. In the face of this catastrophic event, people are forced to rapidly adjust to a new way of working and staying at home by relying on the use of technology. It is apparent that the pandemic has had significant social and psychological effects on the general population, be that from the rising unemployment rate, separation from loved ones, domestic violence, and various other unexpected changes in the ordinary way we were accustomed to living.

Several reports documented that the number of people diagnosed with mental illness have risen considerably over the past decade. Notably, during COVID-19, many of these patients have been receiving sub-optimal care due to the overstretching health system, whose resources have been mostly allocated to deal with COVID-19. The global pandemic highlights the role of current digital tools, offering care in time of needs [12]. Digital devices present a wide range of opportunities for healthcare professionals, ranging from individual health to the common population. Digital devices, especially mobile or wearable devices, are increasingly capable of capturing various sources of real-time behavioral, physiological, and psychosocial data in a precise and confidential manner [13]–[15]. Examples of these technologies include smartphones [16]–[18] and smartwatches [19], [20]. Interestingly, how we use these technologies to improve mental well-being or mitigate mental illness in terms of emerging uncertainties is opened to discussion.

This situation has already created a large area of interest and research opportunities between mental health and existing wearable technology. It also calls for the need for more research to study its implication on mental health based on the data from previous literature. The aim of this scoping review is to gather evidence to determine the potential use of mobile or wearable devices for remote psychological support during the COVID-19 pandemic. Three specific issues that are focused on are as follows:

• 1)To understand the mental health impact of COVID-19 on the general population, at-risk population, and healthcare professionals.
• 2)To focus on the possibility of using the current mobile or wearable devices to early address any detectable mental health conditions.
• 3)To identify the issues of wearable devices in the previous study and evaluate the potential of using the current smartwatch toward psychological supports.

II. Scope of Work

A. Eligibility Criteria

The eligibility criteria for the research included in this scoping review following literature survey were empirical articles that examined mental health outcomes associated with COVID-19 pandemic and/or evaluated the mental health-related conditions by using smart devices; which could be categorized as follows:

• 1)Studies that reported the psychiatric symptoms in the general population, healthcare workers (HCWs), or patients with a prior or current diagnosis of COVID-19.
• 2)Studies that evaluated the application of smartphones, wearable devices, or smartwatches in mental health-related conditions.

B. Information Sources

The search covered various academic databases, including ACM digital library, IEEEXplore, ScienceDirect, and PubMed, with the objective of finding academic literature. Only articles published in English were included. The search was limited to the studies published within the past 5 years.

C. Literature Search

Literature searches were carried out using the keywords ‘mental health’, ‘psychological distress’, ‘behavioral problem’, ‘depression’, ‘anxiety’, ‘insomnia’, and cross-referencing them with ‘COVID-19’, ‘pandemic’, ‘smartphone’, ‘wearable devices’ or ‘smartwatch’. After screening the titles and online abstracts, articles were retrieved in full-text reports and further reviewed to determine compliance with inclusion criteria. The word cloud, as shown in Figure 2, displays the most frequently used keywords in the articles found in our selected literature search.

Fig. 2.

Word cloud of keywords presented in the studies included in this review.

D. Sections

This scoping review is organized into 6 sections (see Figure 1 for the overview). In Section I, the on-going state of the COVID-19 pandemic, its effect on mental health, and the possibility of wearable technology as a medium to improve the healthcare system during the inopportune time are addressed. The search method and how this review was structured are defined in Section II, the scope of work. Section III presents the current evidence from observational studies on the impact of COVID-19 on mental health. The potential applications of mobile and wearable devices to measure these physiological signals are emphasized in Section III. Section IV shows the limitations of wearable devices used in previous studies and presents the usage potential of the current smartwatch in the aspect of mental health monitoring. Discussion and conclusions are presented in Section V and Section VI, respectively.

Fig. 1.

Schematic layout of this scoping review: (I) Introduction, (II) Scope of work, (III) Mental health issues and the capability of mobile devices to detect them, (IV) Smartwatch toward psychological supports, (IV) Discussion, and (V) Conclusions.

III. Mental Health Issues and the Capability of Mobile and Wearable Devices to Detect Them

Stress, anxiety, depression, and insomnia are carrying mental risks to individuals globally. Indeed, the symptoms of stress can lead to alterations in physical, emotional, and behavioral constitutes. Universally, clinicians exercise face-to-face interactions and self-reported questionnaires for diagnosis and assessment. These methods are time-consuming, expensive, and often require the involvement of professionals. Moreover, data from self-reported questionnaires could be unreliable due to memory limitation and subjective recall issues. During the COVID-19 pandemic, the potential practicality of digital evaluation for mental health has become urgently apparent. Wearable devices could be used to monitor the psychological conditions [21] and suitable for monitoring the populations at risk as well as those in quarantine [13].

As evidenced by recent studies, smart devices (i.e., phone, watch, computer, and glasses) could be used to collect data on ambulatory activity, communication patterns, electrodermal activity (EDA), eye movement, facial expression, heart rate variability (HRV), location, mobility, social interaction, social media, speech patterns, and technological usage relevant to a diverse range of mental illnesses [22]. However, some devices, such as glasses equipped with electrooculography (EOG) components, are not widely used.

In 2020, the number of smartphone users in the world reaches up to 3.5 billion, which translates to 45.15% of the world’s population owning at least one smartphone [23]. There are several smartphone apps that were developed specifically for improving mental health care, such as Shine [24], Calm [25], Coa [26], Ginger [27] for anxiety, and Litesprite [28] and Sanvello [29] for depression. Moreover, the new models of smartphones are capable of detecting activity, communication patterns, location, mobility, and speech patterns, which have been demonstrated to be associated with mental health, as shown in Figure 3.

Fig. 3.

Psychological alterations, in addition to emotions, can have an impact on both mind and body, particularly in the context of systemic effects. Some of these changes, either physiological or behavioral signals, can be detected by smartphones and smartwatches. These wearable devices are equipped with the functions to detect basic personal health data, which are transferred via application programming interfaces (APIs) through the cloud system provided by the vendor’s application.

Figure 3 presents the technology equipped in smartwatches, displaying similar detection capabilities. Additionally, some recent models are capable of detecting EDA and HRV data. EDA is a measure of neuron-mediated effects on sweat gland permeability [30]. This can be observed as changes in the resistance of the skin to a small electrical current or as differences in the electrical potential between different elements of the skin. HRV is the fluctuation in the time intervals between adjacent heartbeats [31]. Advance in wearable technology allows manufacturers to incorporate Photoplethysmography (PPG), using light-based sensing technique comparable to the bio-potential measured by electrocardiogram (ECG), for comfort in assessing HRV [32], [33]. The usefulness of EDA can be regulated as a biomarker for mental health [34], whereas HRV is associated with stress and anxiety [32], [35]. According to Statista, a business data platform, the smartwatch sales in the United States of America (USA) are predicted to leap from 9 million units in 2016 to almost 23 million units by 2020 [23].

In this section, the impact on mental health as a consequence of the current pandemic and the monitoring of mental health using smart devices, i.e., mobile phones and smartwatches, are emphasized. We have constructed a method for collecting data on the assessment from questionnaires, study designs, countries, participants, COVID-19 exposure, and mental health outcomes. The data on the prevalence of mental health outcomes and key results (risk or protective factors) were extracted. However, we did not perform a quantitative analysis of the overall prevalence and correlations due to the heterogeneity of varieties of questionnaires and the lack of clinical diagnosis, as shown in Table I. The usage of sensors equipped on the devices, specifically for stress, depression, anxiety, and insomnia, corresponding to the effects of COVID-19 is described. Table II summarizes the related works, which aimed to employ the sensors on these devices for mental health.

TABLE I. Studies of Psychiatric Symptoms During the COVID-19 Pandemic Among General Population and Healthcare Workers.

Assessment	Participants(n)	Country	Stress	Depression	Anxiety	Insomnia	Key results	Ref.
A. General population
GAD-7	undergraduates (7143)	China	N/A	N/A	3.6 %	N/A	- Social support were negatively associated with anxiety symptom	[36]
DASS-21, PSQI > 5	young adults (1310)	Italy	50.1 %	24.4 %	32.6 %	52.4 %	–Sleep difficulty was associated with stress, anxiety and depression.	[37]
CES-D > 28, GAD-7, PSQI > 7	general public (7236)	China	N/A	20.1 %	35.1 %	18.2 %	–Young people had a higher risk of anxiety–HCWs were at high risk for poor sleep#x2013;Spending ≥ 3 hr./day thinking about COVID-19 were associated with anxiety.	[38]
PHQ-8 ≥ 10, GAD-7	young adults (898)	US	N/A	43.3 %	45.4 %	N/A	–Family support was associated with lower levels of depression and anxiety.	[39]
PHQ-9	people who visited the clinic (3974)	Vietnam	N/A	7.4 %	N/A	N/A	–People with suspected COVID-19 symptoms had a higher depression likelihood–Health literacy shows a protective effect on depression during the epidemic.	[40]
DASS-21	university member (2530)	Spain	40.3 %	48.1 %	35.2 %	N/A	–University staff had less DASS-21 scores compared to students.	[41]
DASS-21, ISI ≥ 15	workforce (673)	China	1.5 %	3.7 %	3.9 %	2.4 %	–returning to work had not caused a high level of psychiatric symptoms in the workforce.	[42]
PHQ-9	undergraduates (2485)	China	N/A	9 %	N/A	N/A	–Feeling extreme fear and short sleep duration were associated with depression.	[43]
PSQI > 5	general public (1630)	China	N/A	N/A	N/A	36.4 %	–Higher perceived stress was related to lower sleep quality.	[44]
GAD-7, PHQ-9	student (8079)	China	N/A	17.4 %	10.3 %	N/A	–Female gender and senior high school had higher risk for depressive and anxiety symptoms.	[45]
DASS-21	general public (1879)	Philippines	13.4 %	16.9 %	28.8 %	N/A	–Home quarantine is related to stress, depression, and anxiety.	[46]
DASS-21	general public (1210)	China	8.0 %	16.5 %	28.8 %	N/A	–Women, students and specific physical symptoms had higher risk for stress, depression, and anxiety.	[47]
B. Healthcare workers (HCWs)
DASS-21	HCWs (906)	India & Singapore	5.2 %	10.6 %	15.7 %	N/A	–HCWs with physical symptoms had higher rates of depression, anxiety, stress.	[48]
DASS-21	physician (442)	Turkey	31.0 %	47.1 %	35.2 %	N/A	–Woman, young and less experienced people were in the high-risk group of stress, depression and anxiety.	[49]
GAD-7, ISI ≥ 15, PHQ-9	physician (493) & nurse (764) (total = 1257)	China	N/A	14.8 % (13.8 vs 15.4)	12.3% (11.6 vs 12.7)	7.7 % (5.7 vs 9)	–HCWs exposed to COVID-19, women, nurses, those in Wuhan, and front-line had a high risk of adverse mental health outcomes.	[50]
HAMA ≥ 7, HAMD ≥ 7	HCWs (2299)	China	N/A	11.7 %	24.7 %	N/A	–Medical staff had greater psychological distress than the administrative staff.–Front line medical staff were twice more likely to suffer anxiety and depression.	[51]
GAD-7, ISI ≥ 8, PHQ-9,	HCWs (1563)	China	N/A	17.2 %	12.9 %	36.1 %	–Insomnia were associated with currently working in an isolation unit, worried about being infected, and perceived lack of mental support.	[52]
SAD, SDS	physician(79) & nurse (86) (total = 165)	China	N/A	44.2 % (45.6 vs 43.0)	20.0 % (11.4 vs 27.9)	N/A	–History of depression or anxiety was a risk factor for anxiety and depressive symptoms, while male was a protective factor for depression	[53]
C. General population and health care workers
GAD-2 ≥ 3, ISI > 8, PHQ −2 ≥ 3	HCW (927) & non-HCW (1255) (total = 2182)	China	N/A	10.6%	10.4%	33.9%	–HCWs, Living in rural areas, female, and at risk of contact with COVID-19 patients had higher risk for insomnia, anxiety, and depression	[54]
D. Patient with COVID-19
GAD-7, PHQ-9, PSS-10	patient (103) & normal (103) (total = 206)	China	N/A	17.5 % (patient)	7.0 % (patient)	N/A	–CRP levels correlated with the scores of PHQ-9 in patients with depressive symptom.	[55]
E. Patient with chronic disease
DASS-21, ISI ≥ 15	psychiatric patient (76)	China	17.1 %	22.4 %	23.7 %	26.3 %	–Stress, depression, anxiety, and insomnia were higher in psychiatric patients.	[56]
DASS-21	RA and SLE patient (512)	Philippines	12.3 %	27.7 %	38.7 %	N/A %	–Satisfaction with available health information was associated with low levels of stress and depression.	[57]

Abbreviations: CES-D; Center for Epidemiological Studies Depression Scale, CRP; C-Reactive Protein DASS-21; Depression Anxiety Stress Scale, GAD-7; General Anxiety Disorder 7-items scale, HAM-A; Hamilton Anxiety Rating Scale, HAM-D; Hamilton Depression Rating Scale, ISI; Insomnia Severity Index, PHQ-8; Patient Health Questionnaire-8, PHQ-9; Patient Health Questionnaire-9, PSQI; Pittsburg Sleep Quality Index, RA; Rheumatoid Arthritis, SAS; The Zung Self-Rating Anxiety Scale, SDS; The Zung Self-Rating Depression Scale, SLE; Systemic Lupus Erythematosus

TABLE II. Summary of Related Work for Mental Health Using Smart Devices.

Condition	Smart devices	Data	Sample size & type	Study length (days)	Algorithms	Ground truth	Accuracy	Year	Ref.
Stress	Phone	Accelerometer, WiFi, Cellular, GPS, Call, SMS, App usage, Microphone	30 Employees	56	DT	OLBI, POMS	67.57% and 71.73% after applying a semi- supervised	2020	[62]
	Wrist sensor	SC, HRV	16 Students	8	MLP, RF	PSS	HRV 80% with MLP and SC 73.59% with RF	2020	[63]
	Phone/Wrist sensor	Call, SMS, SC, GPS, Screen on/off	142 Students	30	LSTM	Stress score (0-100)	83.6% using the previous 7 days	2019	[64]
	Phone	Accelerometer	30 Employees	56	Naive Bays	OLBI	71% for user- specific models	2016	[65]
Depression	Phone	Call, SMS, App usage	412 Participants	14	RF	BDI-II	76.8% accuracy	2020	[76]
	Phone	Accelerometer, GPS	33 Participants	77	SVM	PHQ-9	87.2% in severe depression	2020	[75]
	Phone	GPS, WiFi	79 Colleges students	More than 200	SVM	PHQ-9	F-score up to 0.76	2018	[74]
	Phone	GPS	49 Bipolar patients	90	QDA	QIDS- SR	85% accuracy	2017	[73]
Anxiety	Phone/Watch	Accelerometer, Bluetooth, Call, SMS, GPS, WiFi, Battery, Screen on/off, HR, Sleep	60 Participants	30	N/A	STAI, BDI-II, AUDIT	N/A	2020	[87]
	Phone	Value of light, Microphone, Location, Call App usage, SMS Accelerometer	20 Participants	30	XG Boots	STAI	F-score of 74.2%	2019	[84]
	Phone	GPS	228 Participants	14	MLP	SIAS	85% accuracy	2018	[86]
	Phone	GPS, SMS, Call	54 Students	14	DT	SIAS	85% accuracy	2017	[85]
Insomnia	Phone/Wrist sensor	Accelerometer, ST, SMS, SC Location, Call Screen on/off	186 Students	30	LSTM	Sleep experts/Actigraphy	96.5% accuracy	2019	[89]
	Phone	Accelerometer, Earphone	9 Pariticipants	More than 180	SVM	NUELOG Respiration Monitor Logger Sensor	Over 80% true positive rate	2019	[18]
	Phone	Screening Questionnaires	50 Sleep difficulties	42	N/A	ISI PSQI PHQ-A GAD-7	N/A	2018	[90]
	Phone	Screen on/off	More than 400 Participants	28	Bayesian	Sleep tracker	89% accuracy	2016	[91]

A. Stress

1). Increased Stress Levels During COVID-19:

The definition of stress combines the physiological changes of the nervous system as well as the immune system in response to the exposure of external stressors, triggering the fight-or-flight response and impacting the mental health of an individual [58]. Long quarantine, boredom, inadequate resources, insufficient information, financial loss, unemployment, and stigma were the major stressors reported during the COVID-19 restrictions [59]. Nine articles measured the prevalence of perc-eived stress in different groups [37], [41], [42], [46]–[49], [56], [57]. Only Depression Anxiety Stress Scales (DASS)-21, a self-report stress questionnaire, was used to identify stress level in the study population. DASS-21 is a short version of DASS-41 [60], in which Antony et al. had chosen seven elements from each of the initial DASS subscales [61]. The cut-off for moderate symptomatology for stress used in the included articles was equal to or greater than 19 in the DASS-21 stress subscale. The range of the stress prevalence during COVID-19 period was 1.5 - 50.1 % (5.2 - 31.0 % in Healthcare workers). One review paper identified being university students, especially in the undergraduate level, as one of the risk factors for stress among the general population [41]. In addition, being women, being young, having fewer experiences, and suffering physical symptoms were associated with stress in HCWs [48], [49].

2). Smart Devices for Convenient Stress Detection:

Recently, sensing technologies have been used to detect or predict stress levels. Various studies demonstrated a significant correlation between data detected by smartphone (i.e., accelerometer, screen on-off) as well as other wearable devices (i.e., HRV and EDA) and stress level. In the last five years, many prominent works recruited participants with tertiary education and workers, given that the level of stress, especially in their respective environment, is found to be skyrocketed [62]–[65]. In 2017, Statista assembled different reasons for stress at work among employees in North America: 39% from workload, 31% from social issues, 19% from juggling work and personal life, and 6% lack of a job, while only 5% were not stressed. This might contribute further to the psychological effects instigated by COVID-19.

Tertiary level students, e.g., aged 18 to 22, experience a tremendous amount of stress due to the changes in their lives, such as leaving parents, meeting new peers, and the pressure from academic struggling [23]. Thus, they are the prime target participants for stress assessment. Can et al. developed an automatic stress detection system using smart bands for physiological data collection [63]. The stress level of 16 doctorate students was determined by the detected HRV and skin conductance (SC) during an 8-day period. The participants were asked to complete the Perceived Stress Scale (PSS) questionnaires as a baseline. The HRV signal achieved approximately 80% accuracy, using a Multilayer perceptron (MLP) model, whereas the SC signal showed a maximum of 73.59% accuracy with Random forest (RF). A work from MIT Media Lab showed that using long short-term memory neural network models (LSTM) improved the forecasting of stress [64]. The data were collected from 142 college students for 30 days. The results reached 83.6%, using the stress score (0-100) as ground truth, collected 7 days prior to the actual experimentation. The study also used only objective and passive data, collected from mobile phones and wearable devices, to forecast high/low binary stress levels on the next day. The results showed 81.4% accuracy. The stress level was shown to be forecasted more accurately using the objective features in combination with LSTM. Bedtime was one of the features which weighted higher on the forecasting. The data from smartphones (e.g., brightness), rather than self-reported sleep diaries, were collected to determine sleep behavior, which may contribute to the improvement of the accuracy.

Detection of work-related stress has been shown to be associated with various behaviors (i.e., nail-biting, pacing, and changes in appetite) [23]. Equipped sensors on smartphones have been contributing to the data collection associated with individual behaviors. Garcia-Ceja et al. paved the first step to use data obtained from only built-in accelerometer in the smartphone [65]. Combinations of different statistical models to classify stress level were performed. The overall accuracy achieved 71% for user-specific models, relying solely on data from a single accelerometer and validating with the Oldenburg Burnout Inventory (OLBI) questionnaire. Broadening the types of sensor used, Maxhuni et al. acquired physical activity level (accelerometer), location, social-interactions (cellular, Wi-Fi, and Google Maps), and social activity (application usage) during the workday and analyzed the changes in behavior [62]. The classification model included decision trees (DT) as well as the incorporation of semi-supervised learning methods, resulting in 67.57% accuracy and 71.73% after applying a semi-supervised method with Profile of Mood States (POMS) and OLBI as ground truth.

B. Depression

1). Increased Depression Levels During COVID-19:

Individuals with depression may display several clinical features such as negative cognitive contexts as well as psychomotor signs and symptoms, leading to difficulties with decision making, and eventually other psychological disorders such as major depression [66]. Seven different self-rating scales were employed in 20 articles assembled in this review. DASS-21 is the most common tool used in articles, followed by Patient Health Questionnaire-9 (PHQ-9) (9 and 6 articles, respectively) PHQ-9 is a nine-item questionnaire designed for depression screening in primary care and other medical settings [67]–[69]. The standard cut-off score for screening to identify possible major depression is 10 or greater. PHQ-9 appears to be sensitive, although less specific for young adults. However, it has been shown that a cut-off score can be used irrespective of age [70]. In the present review, the moderate or higher severity of depression, which is equivalent to the score of 10 or above in PHQ-9, was used to identify the prevalence of depression across different questionnaires. Regarding the nature of the studied population, the prevalence of depression found in studies carried out in the general population ranged from 3.7 - 48.1 % (3.7 - 48.1 % and 7.4 - 17.4 % in DASS-21 and PHQ-9, respectively). Although various self-reporting questionnaires were utilized, the prevalence of depression was higher than the global health estimation of 4.4 % assessed by WHO in 2017 [71]. Among the general public, women [45], [47], those who did not receive support [39], those with suspected COVID-19 symptoms [40], and/or those with sleeping problems [43] were identified as the population at risk of developing a depressive symptom. Moreover, the results of epidemiological studies showed that HCWs who were women [49], [54], nurses [50], frontline workers [49], [51], [54], those lacked support [52], and/or those with physical symptoms [48] were at higher risk of having depression. However, health literacy was demonstrated as a protective factor [40] of depression during COVID-19 pandemic.

2). Smart Devices for Convenient Depression:

Greater emphases on depression are demanding treatment and amelioration. Several studies have been successful in detecting and predicting the severity of depression using the data from smart devices. A significant correlation between the location data provided by mobile phone and depression severity was found [72]. To monitor depression, the team acquired the data from the sensors equipped on the smart device to analyze various behaviors, such as time spent being alone, duration of staying home, changes in sleep pattern, communication with family and friends, and time spent using social media platforms.

In several works, the assessment of depression fixated on physical activity and movement patterns as they are associated with the behaviors of individuals with depression, e.g., living alone and avoiding social interaction. Palmius and colleagues employed the GPS data for determining depression from an individual’s mobile device [73]. The GPS data revealed the movement patterns with an accuracy of 86%, using Quick Inventory of Depressive Symptomatology (QIDS-SR), a weekly questionnaire, as a baseline. Yue et al. collected data from two sources: GPS and WiFi association recorded by the phone [74]. The evaluation was performed using a dataset from 79 college students. The results showed a stronger correlation with PHQ-9 self-report. This led to depression prediction with much higher F1 scores; up to 0.76 in comparison to 0.5 prior to the application of data fusion. Similarly, Mohammed et al. assessed the depression level of an individual using acceleration sensor and GPS sensor on the smartphone for physical activity and movement patterns, respectively [75]. The implemented model was able to detect three different depression levels (i.e., absence, moderate, and severe) with 86.4% accuracy using the PHQ-9 as a baseline.

Not only the sensors but other minor aspects of mobile phone usage may not be so trivial in evaluating depression. Razavi et al. proposed a method of depression screening from mobile phone usage without the measurement of physical activity and movement patterns [76]. 412 participants reported a range of smartphone usage statistics, and Beck Depression Inventory—2^nd Ed (BDI-II) was used to measure the severity of depression among the participants. The team calculated the average number of daily calls and text messages (inbound and outbound) as well as the average time spent on conversing, social media, web browsing, and entertainment application data from the smartphone. The RF classification model showed a balanced accuracy of 0.768 and an AUC of 0.733 for classifying the two groups. The study demonstrated that the participants with depression were found to have fewer saved contacts on their devices, spend more time on their mobile devices to make and receive fewer and shorter calls, and send more text messages than participants without depression.

In addition to mobility and socialization, smart devices could detect certain symptoms of depression. Asare and colleagues presented scenarios where the triggered anomalous behaviors could be detected, e.g., extra-long duration of device usage, physical and social isolation, and changes in sleep patterns [77]. The results regarded sleep as an important feature to detect depression.

C. Anxiety

1). Increased Anxiety Levels During COVID-19:

Anxiety is a symptomatic milestone that occurs in the early stages of numerous mental illnesses. Some indications of depression and anxiety disorders may be overlapped as some patients may have a history of either one earlier in life before developing another [78]. Although slightly vague, anxiety can be defined as the adaptive feedback as a defensive coping in response to struggling encounters and personal uncertainty, in which the conflicts may result in behavioral alterations and various kinds of anxiety disorders [79]. Any uncertain situations can lead to panic-induced behavior [80]. The common behavioral signs and symptoms of anxiety disorders include nervousness, irritability, and problems with sleeping and concentrating [81].

Due to the restrictions during the COVID-19 pandemic, self-confinement and a lack of proper management have led to masses of panic and anxiety. A total of 19 studies reported anxiety as an indicator for psychological impact: among them, 7 recruited HCWs for the assessment. Different validated scales to measure anxiety are the Depression, Anxiety and Stress Scale (DASS)-21, Generalized Anxiety Disorder (GAD)-2/-7, Hamilton Anxiety Rating Scale (HAM-A), and self-rating anxiety scale (SAS). DASS-21, which follows the Patient Health Questionnaire-9, is the most popular tool used in articles. 9 studies used DASS-21, while 7 studies used GAD-7 to assess self-reported anxiety symptoms. GAD-7 was initially validated in 2,149 primary care patients, regarding a score of 10 or greater represents anxiety (sensitivity 89 %, specificity 82%) [82]. Also, the GAD-2, HAMA and SAS applied in 3 additional studies each. The overall prevalence was 3.6 - 45.4 % (10.6 - 47.1 % in HCWs), as presented in Table I. It was found that being female [45], [47], lacking support [36], experiencing sleep difficulty [37], or hovering over COVID-19 news [38] had higher risk of anxiety. Additionally, women [49], [50], young individuals [49], less experienced workers [50], nurses [50], frontline workers [50], [51], [54], or those with physical symptoms [48], were associated with anxiety in HCWs.

2). Smart Devices for Convenient Anxiety Detection:

Detecting anxiety remotely without access to psychological experts could be challenging in various ways. Studies have been attempting to measure anxiety using smart devices in order to address different anxiety disorders, such as generalized anxiety disorder (GAD) and social anxiety. Distinguished from previous studies using smartphone emphasizing only on a single feature [83], Fukuzawa et al. considered environmental, real-world, and online behavioral features to predict the stress level alteration of 20 young adults [84]. Data from different equipped sensors, such as acceleration, illuminance, volume, location, and smartphone orientation, as well as application usage and call log activity, were analyzed. The participants were asked to complete the monthly State-Trait Anxiety Inventory (STAI), a self-report questionnaire, for validation. The results of the model learned by RF and XGBoost yielded an F-score of 74.2%, which was shown to be higher than the results achieved by previous studies, analyzing only a single smartphone feature. Additionally, the variation of illuminance in specific events, e.g., using social network applications in a dark room while being motionless, may have contributed to the changes in anxiety level.

The decision trees (DT) with the combination of both mobility and communication to predict the social anxiety level among the tertiary level students has been examined and yielded 85% accuracy [85]. Boukhechba and colleagues analyzed the GPS location, text messages, and call data collected from 54 college students over a two-week period using the self-reported Social Interaction Anxiety Scale (SIAS). In another work by the same team, only GPS data were used to detect behavioral patterns associated with social anxiety [86]. 228 undergraduate participants were recruited to be assessed for 2 weeks using SIAS as ground truth. The results showed 85% accuracy using MLP. Pastor et al. developed a technique for digital phenotyping of patients with alcohol and anxiety symptoms, using data collected from smartphone and smartwatch to analyze the usability and patient satisfaction [87]. 60 participants were presented with STAI, the Beck Depression Inventory-II (BDI-II), and the Alcohol Use Disorders Identification Test (AUDIT) once per week through the HumanITcare app. The study demonstrated that monitoring patients using both smart devices is practical. Anxiety disorders have also been found to be associated with reduced HRV [35]. Moreover, the increase in galvanic skin response (GSR) has been shown to be related to different levels of anxiety [88], which could be detected using smartwatches.

D. Insomnia

1). Increased Insomnia Levels During COVID-19:

Preceding the limitation of health-related researches during the COVID-19, insomnia has been one of the major sleep problems among the general population. Insomnia is a common condition associated with impairments of daily performance and the quality of life as well as physical and mental morbidities [92]. Researchers from several countries, such as China [93], France [94], Greece [95], and Turkey [96], with the ability to conduct studies, have begun to assess the affected communities, especially in HCWs. To control for objectivity and validation, 8 studies employed different scales, including Insomnia Severity Index (ISI) and Pittsburgh Sleep Quality Index (PSQI) to measure sleep disturbances (Table I). While 5 studies used ISI with different cut-off scores to determine insomnia with a prevalence of 2.4 - 36.1 %, 3 studies used PSQI with the range of prevalence of 18.2 % - 52.4 %. Cellini et al. confirmed the associations between sleep difficulty, stress, anxiety, and depression in general public [37]. Moreover, in the healthcare profession, HCWs, who worked in an isolation unit, worried about being infected, or perceived the lack of mental support, had a higher risk of insomnia [52], [54].

2). Smart Devices for Convenient Insomnia Detection:

Insomnia has been shown to affect daily routines, such as performance at work or school, the lack of energy, and the increase of risk for car accidents. Insomnia is found to be caused by physical or mental health issues. The symptoms may include difficulty in falling asleep at night, waking up during the night, and waking up during unwanted time. A combination of physiological signals detection techniques, called polysomnography (PSG), is considered a gold standard for sleep detection and diagnosis. However, acquiring a PSG study can be expensive and uncomfortable for some individuals. Data on audio function and the equipped accelerometer on smartwatches and smartphones can be collected and combined to determine the basic sleep patterns, such as the number of hours of sleep, number of awakenings during the night, and occurrence of snoring [97].

Some smartphone features can be used to verify the possibility of insomnia. Andrea et al. adopted a Bayesian model to extract individual sleep patterns from screen on/off events on smartphones [91]. Data of the screen interaction sensor on smartphones were collected from more than 400 users to compare with the sleep tracking data extracted from fitness tracking armbands. The model performance reached 89% accuracy, inferring that the smartphone interaction could be an option in monitoring sleep patterns. Ren et al. presented a fine-grained sleep monitoring system to detect an individual’s breathing rate and sleep events by the microphone equipped on smartphones to capture the breathing sound for breathing rate monitoring [18]. To achieve higher detection accuracy, the team developed a body movement-assisted sleep event detection by capturing body movement patterns via an accelerometer embedded on smartphones. The experiments recruiting 9 subjects over a period of six months showed that their system could achieve high accuracy.

In clinical practice, cognitive behavioral therapy for insomnia (CBT-I) is one of two major approaches to treatment, the alternative being pharmacotherapy [98]. Additionally, internet-delivered CBT-I has been previously found to be effective and can be considered a viable insomnia treatment of choice [99], [100]. Besides Andrea et al., another team developed a smartphone application, Sleep Ninja, to tackle sleep difficulties in adolescents [90]. Sleep Ninja has been developed under the concept of internet-delivered CBT-I such as recommended bedtime based on sleep guidelines and a series of sleep tips as well as providing general information regarding sleep. 50 young teenagers with sleep difficulties were recruited for experiments. Significant improvements were observed on sleep variables including insomnia.

Detection of sleep/wake state and sleep onset/offset using a combination of features from wrist sensor (i.e., skin conductance (SC), skin temperature (ST), and acceleration) and smartphone (i.e., screen on-off, SMS, call, and location) has been shown to attain 96.5% accuracy with LSTM model in comparison to neural networks, LR and SVM [89]. In this study, sleep experts from Harvard Medical School scored the sleep stages from diaries with actigraphy data used as ground truth. Although the process was considered time-consuming, it was deemed more reliable than employing data from self-reports or actigraphy alone. The results demonstrated that the best performance was obtained from the combination of accelerometer, ST, and time features, whereas the smartphone features showed the lowest performance among the comparisons between different modalities. Some limitations of using the smartphone for sleep detection have been shown to involve the waking up times during the night and the possibility that the smartphones are not used.

E. Summary

Overall, studies have been demonstrating that sleep detection or prediction can be performed well using data from smart devices in comparison to self-reported questionnaires. Most studies still employed different questionnaires as ground truth as these are deemed simple and standardized. However, subjective answers and omitted truths, as well as limited memory recalls, might pose an issue for accuracy. Despite the global usages of smart devices, studies related to mental health using smartwatches can be found less than smartphones. Nevertheless, smartwatches are capable of detecting physiological data with more accurate details. Thus, the use of smartwatches will indisputably be of major interest in studies related to mental health and social interaction in the future [101].

IV. Smartwatch Toward Psychological Supports

In the previous section, the studies in the past five years illustrate the possibility of utilizing smart devices in monitoring mental health issue, including stress, depression, anxiety, and insomnia. The information solely from the smartphone hitherto is proven to be enough for predicting the mental health issue of the participants [62], [75], [84], [91]. However, the summary shown in Table II demonstrates that combining the information from the smartphone and another wearable device can provide higher prediction accuracy in comparison to relying only on the information from the smartphone (83.6% [64] compared to 71% [65] in the case of stress and 96.5% [103] compared to 89% [91] in the case of insomnia). This is because the wearable device can provide the physiological data (i.e., HRV and SC) to complement the data from the smartphone, which are limited only to the behavior (i.e., acceleration and GPS) and social interaction (i.e., call and SMS) of the participants.

Among various physical health data from the wearable devices, SC and HRV were scientifically proven to be related to mental health problems [32], [34], [35], [104]. The SC response of the patients with depression has been shown to be lower than that of the persons with strong mental health [104]. Additionally, patients with stress and anxiety symptoms were found to display lower HRV than that of the control group [32], [35]. In most of the previous studies, the wearable devices used in the researches are research-grade, in which they can provide one or both of these two information (i.e., Affectiva Q Sensor and Empatica E4) [63], [64], [103]. Affectiva Q Sensor can measure the acceleration, SC, and ST at the sampling frequency of 32 Hz [105]. Similarly, Empatica E4 can measure the acceleration at the sampling frequency of 32 Hz, SC and ST at 4 Hz, inter-beat interval (IBI), and blood volume pulse (BVP) at 64 Hz [106]. IBI refers to the interval between individual heartbeats and BVP refers to the volume of blood that passes through the sensor with each heartbeat. Both features can be further used to compute HR and HRV. These research-grade devices provide full support in extracting the raw data from the devices by using either their application programming interface (API) or exporting through their applications. However, the prices of the research-grade wearable devices are fairly expensive, as shown in Table III. From a financial feasibility standpoint, it could be more widely affordable if the research-grade wearable devices can be replaced by the smartwatch.

TABLE III. Comparison Between Specifications of Smartwatch and Research-Grade Wearable Device.

Name	Sensor	Extractable data	Unextractable data (currently export from official app only)	Communication	Battery / battery life	Current price	Ref.
Affectiva Q sensor	EDA, Accelerometer, Temperature sensor	Acceleration (8 Hz), SC (8 Hz), ST (8 Hz)	None	Bluetooth	Not show / Up to 24 hours	Discontinue (2013)	[105]
Empatica E4	PPG sensor (4 LEDs (Red, Green), 2 Photodiodes), EDA sensor, Accelerometer, Infrared thermopile	IBI (HRV, HR) (64 Hz), BVP (64Hz), SC (4Hz), ST (4Hz), Acceleration (32Hz)	None	Bluetooth Low Energy	Li-Po / 24 hours+ (streaming) /32 hours+ (recording)	1,690$	[106]
Apple Watch Series 6	PPG sensor (4 LED clusters (Red, Green, Infrared), 4 Photodiodes), ECG sensor, GPS, Compass, Altimeter, Accelerometer, Gyroscope, Ambient light sensor, Microphone	Acceleration (100 Hz), Angular acceleration (100 Hz), Compass, Altitude, Location, HR (1 Hz), SpO2, HRV, ECG* (Apple developer program is needed (99$/year))	None	Bluetooth 5.0, Wifi 802.11b/g/n, LTE and UMTS	Li-Ion 303.8 mAh / up to 18 hours (daily use) /7 hours (GPS on)	399$ (Released: Sep, 2020)	[109]
Fitbit Sense	PPG sensor (6 LEDs (Red, Green, Infrared), 4 Photodiodes), Temperature sensor, EDA sensor, ECG sensor, GPS, Altimeter, Accelerometer, Gyroscope, Ambient light sensor, Barometer, Microphone	Acceleration (100 Hz), Angular acceleration (200 Hz), HR (1Hz), Location, Atmospheric pressure (40 Hz)	SpO2, ECG*, HRV, ST, SC (instantaneous data by free user and trend over time by premium user (9.99$/month))	Bluetooth 5.0, NFC, Wifi 802.11 b/g/n	Li-Po / up to 6 days /12 hours (GPS on)	329.95$ (Released: Sep, 2020)	[110]
Samsung Galaxy watch 3	PPG sensor (8 LEDs (Red, Green, Infrared), 4 Photodiodes), ECG sensor, GPS, Accelerometer, Gyroscope, Ambient light sensor, Barometer, Microphone	Acceleration, Angular acceleration, HR, Location, Atmospheric pressure	SpO2 *, ECG*, Blood pressure*	Bluetooth 5.0, NFC, Wifi 802.11b/g/n, LTE and UMTS	340 mAh / up to 43 hours (daily use)	449.99$ (Released: Aug, 2020)	[111]
Huawei GT2	PPG sensor (2 LED clusters (Red, Green, Infrared), 2 Photodiodes), GPS, Compass, Accelerometer, Gyroscope, Ambient light sensor, Microphone	Acceleration, Angular acceleration, HR, Compass, Location, SpO2	None	Bluetooth: BT5.1, BLE / BR / EDR	Not show / up to 14 days /30 hours (GPS on)	155$ (Released: Oct, 2019)	[112]
Amazfit Stratos 3 (Xiaomi)	PPG sensor (2 LEDs (Green), 1 Photodiode), GPS, Accelerometer, Gyroscope, Compass, Microphone	Acceleration, Angular acceleration, HR, Location, Compass	None	Bluetooth 4.2/ BLE 5.0, Wifi 802.11/b/g/n	Li-Po 300 mAh / up to 14 days /35 hours (GPS on)	199.99$ (Released: Aug, 2019)	[113]

^*May not available in some countries depending on the approval of FDA

Figure 4 shows the worldwide shipments of the smartwatches of the top 5 vendors from 2016 to 2019. It could be seen that the demands of these smartwatches are increasing exponentially. Multiple studies have steered the attention to the utilization of smartwatch for health and wellness [107]. However, to the best of our knowledge, there are only a few studies that used the smartwatch for mental health-related applications, e.g., the case-control study by Pastor et al. [87]. One of the possible reasons for such a few investigations utilizing smartwatches could be due to their lack of capability to measure SC and HRV. Recently, however, some of the top smartwatch vendors (i.e., Fitbit [108]) have released a new smartwatch model with an application for stress tracking. This demonstrates the potential usage of the latest debut of these smartwatches toward mental health monitoring applications. Therefore, in this section, we will explore the possibility of using these smartwatches instead of the high-cost research-grade wearable devices in mental health monitoring applications. To this end, we surveyed the top model smartwatches from 5 vendors with the highest market share, including Apple Watch Series 6 [109], Fitbit Sense [110], Samsung Galaxy Watch 3 [111], Huawei GT 2 [112], and Amazfit Stratos 3 (Xiaomi) [113]. The surveyed smartwatches were compared with the research-grade wearable devices presented in this review from both consumer and developer perspectives (Table III).

Fig. 4.

Top 5 smartwatch unit shipments worldwide by different vendors from 2016 to 2019 (in millions) [102].

A. Sensor

1). PPG Sensor:

PPG is one of the most popular non-intrusive measuring technologies for collecting information related to vital signs [114]. Although PPG signal can be very susceptible to motion artifacts caused by hand movements and easily affected by environmental noise [115], PPG sensor can still be found in most of the smartwatch models due to its cost-effectiveness, comfortable to wear, and portability [116]. PPG’s working principle is based on light absorption and reflection of the vascular tissue and blood. PPG sensor illuminates light into the skin and uses photodiode to detect the light intensity [117]. The characteristics of the vital sign can be investigated through the fluctuation of light intensity caused by light absorption of the hemoglobin in blood [118]. In the past, most of the PPG sensors used in the smartwatch consisted of only green color LED [119]. This is because the shorter wavelength (530 nm) of green light is robust to the motion artifacts. However, it is also due to its short wavelength that it is difficult to penetrate deeper into the skin [120]. On the other hand, red and infrared light have longer wavelengths (660 nm and 850 nm, respectively) and can penetrate deeply into the skin. Nonetheless, they are reported to be highly susceptible to noise from motion artifacts [121]. Human skin has multiple layers with different types of blood vessels embedded in different layers. For example, capillaries can be found in the epidermis (outermost layer), while arteries are located deeply in the hypodermis layer [122]. By utilizing multiple light wavelengths in the PPG sensor, the blood pulsation information from different blood vessels can be measured. This result in better working performance and a wider range of application (e.g., blood pressure approximation) [123], [124]. Therefore, the later generation of PPG sensors in the smartwatch is equipped with multiple LEDs with different wavelength (generally green, red, and infrared LEDs) [109]–[112].

Each smartwatch vendor, as shown in Table III, have developed their respective technology for the PPG sensor. In Apple Watch Series 6, the new PPG sensor consists of 4 clusters of green, red, and infrared LEDs in combination with 4 photodiodes [109]. In Fitbit Sense, the newly debuted PurePulse 2.0 PPG technology, consisting of 6 LEDs (red, green, and infrared) and 4 photodiodes [110]. In Samsung Galaxy Watch 3, the PPG sensor consists of 8 LEDs (red, green, and infrared) and 4 photodiodes. In Huawei GT 2, the PPG sensor is equipped with 2 LED clusters (green, red, and infrared) and 2 photodiodes [112]. In Amazfit Stratos 3, the PPG sensor contains 2 green LEDs and 1 photodiode [113]. As all of the reviewed smartwatches in Table III were introduced into the market only recently, not many research articles are published to clarify the accuracy of these PPG sensors.

Aside from HR, the data from the PPG sensor can also provide information on blood oxygen saturation (SpO₂), HRV, and blood pressure [125]. SpO₂ refers to the oxygen level in blood where the optimal range is between 95-100% in a healthy person [126]. PPG sensor can measure SpO₂ by emitting two different wave-length lights (typically red and infrared) through the skin [127]. The ratio between light intensity of red and infrared lights sensed by the equipped photodiode can be used to analyze the oxygen level in blood given that the oxygenated hemoglobin can absorb infrared more than red light and the deoxygenated hemoglobin can absorb red light more than infrared [128]. SpO₂ measurement is just recently introduced to consumer-grade smartwatches as it can be used in the sleep tracking application [129]. Despite the featuring of SpO₂ in all of the smartwatches presented in Table III, with the exception of Amazfit Stratos 3, no research study is found investigating the accuracy of SpO₂ measurement in these models.

As previously mentioned, HRV is essential data for mental health analysis. Even though the best method that yields the highest accuracy in HRV measurement is via an ECG sensor [33], a PPG sensor can also be used as a surrogate. HRV can be derived from the PPG sensor’s raw data by observing the variation of an interval between two adjacent R-waves [130]. The concern over the measuring of HRV through PPG sensor involves the reliability of the sensor only in resting state (low motion artifacts) [131]. Only several smartwatches available in the market are catered with HRV feature. Amongst the reviewed smartwatches, only Apple Watch Series 6 and Fitbit Sense offer the HRV tracking feature. Although one study shows high reliability (more than 0.9) of HRV measurement from PPG sensor in the previous generation Apple Watch [132], further report on the reliability of Fitbit Sense HRV measurement is yet to be seen.

Blood pressure is related to the blood pulse transit time, which in turn, related to PPG signal pulse characteristics, such as rising time, falling time, and amplitude [133], [134]. Presently, not many smartwatches are furnished with technology to measure blood pressure. Within our reviewed smartwatches, only Samsung Galaxy Watch 3 provides the feature for reading the blood pressure [135].

2). EDA Sensor:

Electrodermal activity (EDA) can be easily referred to as the changes in electrical conductance of the skin caused by the modulation from the sweat gland [136]. EDA signal consists of two different components; skin conductance response (SCR) modulated by skin conductance level (SCL). SCR is the change in skin conductance when the sudomotor nerve is activated and is usually represented as a transient event in the EDA signal (0.05 - 1.5 Hz). On the other hand, SCL is a slow changing of skin conductance (0 - 0.05 Hz) responding to a tonic stimulus [137]. Currently, there are 3 methods available for measuring EDA: endosomatic methodology (ESM), exosomatic methodology with direct current (DC–EXM), and exosomatic methodology with alternating current (AC–EXM). ESM is a method to measure the EDA without applying an electric current. On the other hand, both DC–EXM and AC–EXM are measured by applying direct current and alternating current to the skin, respectively. Among the three methods, DC–EXM is the most widely utilized in wearable devices because the response from ESM is very difficult to interpret and the AC–EXM has no empirical demonstration of the superiority over DC–EXM yet [138]. EDA is best measured at the place with a high density of sweat glands, especially at hands and feet [139]. Interestingly, the only smartwatch that has the EDA sensor available at present is Fitbit Sense [108]. Fitbit Sense is capable of measuring the SC of the subject’s palm while the user needs to be seated and asked not to move. This is the optimal method that yields accurate results since the noise will be low; however, it is not practical for ambulatory use and 24/7 monitoring [140].

B. Data and Limitations

In order to replace the research-grade wearable device with the smartwatch, the feasibility of data extraction must be considered. All of the reviewed smartwatches and wearable devices can be connected to their companion device (i.e., smartphone) via Bluetooth for the data transfer. As research-grade wearable devices are designed to be used in the research, raw data can be easily extracted. For example, Empatica provides a web application for export the raw data (E4 Connect) and provides API and software development kit (SDK) for further research and development [106]. API is a computer interface used as a medium for interaction between multiple software. It can be a set of commands, functions, protocols, or objects [141]. SDK is a tool for developing an application and it usually contains API for the communication between the application and the target device. In smartwatches, some vendors provide official applications that can export the data for the normal consumers and also provide the API or SDK for the developer. The difference between exporting data from the official application and using API to extract the data is the obtained sampling rate. The sampling rate from the former method is unchangeable and usually low to reduce the power consumption, while the latter allows more flexibility for the setting or even the possibility to extract the raw data from the sensors. Unfortunately, the data that can be extracted by the provided API are limited depending on each vendor.

Apple Watch operating system (OS) is iOS and WatchKit SDK can be used for developing an application and extracting some raw data [142]. Apple provides CoreMotion framework for extracting raw acceleration (100 Hz), angular acceleration (100 Hz), compass data, and altitude data. This framework can adjust the sampling rate of the data. Apple also provides HealthKit API for extracting vital signs and other health-related data from Apple Health storage. However, some data extracted with HealthKit API are not the raw data but the already derived data, i.e., raw PPG signal is not available, but HR, HRV, and SpO₂ are available. Interestingly, there is still a method to export raw data from the PPG sensor by using Breath app developed by Apple [132]. In iOS 14, ECG data extraction is also available via HealthKit API [143]. In order to access any of these services, Apple developer program enrollment is necessary (cost 99 USD/year) [144].

Fitbit SDK can be used to develop an app for Fitbit’s smartwatch. Even though Fitbit Sense has the most number of sensors among the reviewed smartwatches, only acceleration (100 Hz), angular acceleration (200 Hz), HR (1 Hz), location, and atmospheric pressure (40 Hz) can be extracted using their Device API. Fitbit also provides Web API for accessing Fitbit cloud storage data such as activities data and HR, however, with the limit requesting rate of 150 requests/hour. There is no cost for developing with Fitbit SDK or API. At present, Fitbit still does not release the API for accessing SpO₂, HRV, ST, SC, and ECG data. The only method to extract these data is to subscribe to Fitbit premium user (9.99 USD/month) and export via Fitbit’s official application.

Samsung provides Tizen SDK for developing an application for their wearable devices. The company provides Native API for accessing some raw data from sensors with an adjustable sampling rate. Acceleration, angular acceleration, location, atmospheric pressure, and raw PPG data can be extracted through this API [145]. However, as the SpO₂, ECG, and blood pressure are just recently added, there is still no API for accessing these data. The only method to extract these data is by exporting from Samsung official application (Samsung Health for SpO₂ and Samsung Health Monitor for ECG and blood pressure). There is no cost for developing using Tizen SDK or exporting data from Samsung’s official application.

Huawei provides HiHealth Kit for developing an application related to health monitoring. However, HiHealth Kit cannot acquire the data directly from Huawei smartwatch and needs to procure the data through Huawei Health app [146]. Fortunately, the operating system in Huawei is based on Android, and it is possible to utilize Google Fit SDK and its API for extracting the raw data [147]. Through Google Fit, acceleration, angular acceleration, HR, compass, location, SpO₂, and raw PPG data can be extracted with an adjustable sampling rate.

Amazfit Stratos’s company (Xiaomi) has a subsidiary company named Huami, which is in control of developing and manufacturing Xiaomi’s wearable devices. Huami provides a Web API for extracting activity and health-related data from their smartwatch [148]. However, this API is not able to extract the raw data, such as accelerometer, gyroscope, and compass, from the sensors. Xiaomi also provides an official SDK for extracting the data from their smartwatch; however, it is supported only in the Chinese language [149]. Fortunately, Google Fit appears to be compatible with the Amazfit smartwatch. Therefore, the raw data can be extracted through Google Fit API, similar to Huawei.

C. Battery and Price

From a consumer perspective, battery life and price of the smartwatch are factors that need to be considered. First of all, in terms of battery life, Amazfit Stratos 3’s and Huawei GT 2’s battery lives last 35 and 30 hours, respectively, with continuous GPS usage. This could be due to their lower number of sensors comparing to the newer model from the other vendors released this year. Comparing to the research-grade wearable device, both of the mentioned batteries can last longer. On the other side, Fitbit Sense comes with a battery life of up to 6 days (no usage condition explanation) and 12 hours if continuously using GPS. Apple Watch Series 6 battery endures for the shortest time of 18 hours in daily life usage or 7 hours if continuously using GPS. Lastly, Samsung Galaxy Watch 3’s battery lasts up to 43 hours. However, it is difficult to compare Samsung Galaxy Watch 3 with the others since there is no usage condition explanation.

In terms of price, compared to the research-grade wearable device, the prices of all reviewed smartwatches are much more affordable. Even the most expensive one among them, Samsung Galaxy Watch 3, sets its price at least 3 times lower than Empatica E4. Three of the reviewed smartwatches (i.e., Apple Watch Series 6, Fitbit Sense, and Samsung Galaxy Watch 3) are new models released in Q3 and Q4 of this year. Among the three, Samsung Galaxy Watch 3 is the most expensive (449.99 USD). Apple Watch Series 6 comes in second with the price starting from 399 USD. Interestingly, Fitbit Sense is somewhat cheaper than the other two, despite the fact that Fitbit Sense introduces the highest number of sensors in all of the reviewed smartwatches. Huawei GT 2 and Amazfit Stratos 3 are both smartwatches released in 2019. Therefore, their prices are on another scale lower than the previous three.

V. Discussion on Smartwatch Candidates for Mental Health Monitoring

Various sensors are one of the main highlights in smart devices, offering variety in applications. All of the reviewed smartwatches have a PPG sensor. However, not even one smartwatch can provide all the features that the PPG sensor can measure. All of the smartwatches, with an exception of Amazfit Stratos 3, provide HR and SpO₂ features. However, HRV and blood pressure features are depending on the model. Apple Watch Series 6 and Fitbit Sense provide HRV feature but without blood pressure feature. On the other hand, Samsung Galaxy Watch 3 provides blood pressure feature but without HRV feature. Huawei GT 2 and Amazfit Stratos 3 provide none of these features. However, it is possible to extract raw PPG data from Huawei GT 2 and Amazfit Stratos 3 with Google Fit API to manually acquire the HRV and blood pressure. Another noteworthy point that should be considered is the accuracy of the PPG sensor. We have reviewed the accuracy of the previous generation smartwatches of these vendors; the accuracy of Apple Watch is the best compared to the other smartwatches, and, surprisingly, better than Empatica E4 [20], [150]. Additionally, Fitbit’s and Samsung’s smartwatch are comparable to Empatica E4 during the resting state [20], [150], [151]. Considering that all vendors upgrade their PPG technology in the new model smartwatches, higher PPG accuracy can be anticipated, although it has yet to be clarified. A study has demonstrated the use of a similar LED configuration as Apple Watch Series 6 (4 cluster LEDs (green, red, and infrared)). The result showed a reduction of error in active state HR estimation [123]. Fitbit’s PurePulse 2.0 introduces a PPG sensor with multi-wavelength LEDs, which should provide better accuracy and robustness compared to their previous generation PurePulseⓇ, equipped with only green LEDs [123], [124]. The accuracy validation of these newly debut PPG technologies is expected to stimulate the interest for further research.

For ECG sensor, Apple Watch Series 6, Fitbit Sense, and Samsung Galaxy Watch 3 contain 1-lead ECG that received the approval from the Food and Drug Administration (FDA) in some particular countries [152]–[154]. Although the ECG data from smartwatches are not suitable for precise health analysis, it is still useful for screening the health condition and raises the awareness of the public [155]. For the EDA sensor, only Fitbit Sense provides the sensor for measuring SC from the user’s palm. The output from Fitbit EDA app is not the SC raw data but the derived EDA response and stress analysis. All of the reviewed smartwatches are equipped with an accelerometer, gyroscope, GPS, and microphone as the fundamental sensors. Interestingly, all of these sensors can provide data that are necessary for mental health monitoring, as shown in Table II. In addition to the fact that some of the smartwatches can connect to the internet via Wifi and LTE, it might be possible to use the smartwatches for mental health monitoring without a mobile phone. Despite the enormous energy consumption for continuous processing and sending data, the changes in health-related behavior provided by these devices are remarkable.

In terms of data extraction, all of the smartwatches can be connected to the smartphone via Bluetooth for transferring the data. The fundamental data such as acceleration, angular acceleration, and location can be extracted by official API in all smartwatches. Apple’s, Samsung’s, Huawei’s, and Xiaomi’s smartwatches have their respective methods to extract raw PPG data with an adjustable sampling rate by using an official or third-party app (i.e., Breath app and Google Fit). On the other hand, Fitbit does not offer this option and only derived data are able to be extracted. Fitbit’s and Samsung’s smartwatches also contain data that are still unable to be extracted via API and need to be exported manually through their official application. According to these facts, smartwatches show obvious limitations in terms of data extraction as they were designed for consumer uses. Even though each vendor provides the official API/SDK for developers, it is still inflexible and not possible to access to all of the raw data. In addition to the several compatible platforms for each smartwatch (i.e., iOS, Android, and Window) [156], designing a common architecture that can be used to deploy in any smartwatches is a very challenging issue that needs to be researched further.

The smartwatch with the highest potential for substituting the research-grade wearable device in mental health monitoring application appears to be inkling towards Fitbit Sense. This is because Fitbit Sense possesses similar sensors as Empatica E4 (PPG, EDA, ST, and accelerometer) and comes with additional sensors that can be used instead of smartphone sensors. Fitbit’s smartwatch also has the highest number of studies, validating their reliability in comparison to the other vendors [144]. Currently, however, Fitbit still does not provide access to the PPG and EDA raw data (used in the reviewed study [63]) and only provides access to the derived HRV data and stress analysis data (from EDA) on their official app. It might be possible to use the derived data from Fitbit Sense instead of the raw data. However, the validation with the clinical level method as ground truth is necessary. In the future, if Fitbit offers open access to the PPG and EDA raw data or a third-party app provides a method to access these raw data, Fitbit Sense should pose as the best replacement for Empatica E4 in mental health monitoring application. Another choice for an alternative is Apple Watch Series 6. This is because Apple Watch’s PPG sensor is widely proven to be the most accurate PPG sensor in all of the smartwatches with the possibility to access the raw PPG data through Breath App. However, the downside of Apple Watch is that there are no ST sensor and EDA sensor. As described in Table II, no study has been found using only HRV for detecting mental health issues, hence the necessity for further validation to effectively utilize Apple Watch Series 6 in mental health monitoring application.

VI. Conclusion

The evidence from our scoping review confirms that the wearable devices and mobile technology have the potential to identify and target individuals with psychological burdens. Smartphones are able to provide communication data (i.e., call, SMS, and app usage) and behavior data (i.e., acceleration and GPS). Moreover, wearable devices can provide physiological health data (i.e., HRV and SC), which are associated with mental health, such as stress, depression, anxiety, and insomnia. From our findings, most of the studies use high-cost research-grade wearable devices that are difficult for the public to afford. However, we believe that smartwatches have the potential to replace research-grade wearable devices (or even smartphones). The use of smartwatches will indisputably be the major interest in future mental health research. To verify this assumption, we reviewed top model smartwatches from 5 vendors with the highest market shares from both developer and consumer perspectives. The result of our review shows similarity of sensors in the current models of smartwatch, the research-grade wearable device, and smartphone. Thus, we believe that the smartwatch has a high potential to substitute for the other devices in the application of mental health monitoring and psychological support during the COVID-19 pandemic.

Funding Statement

This work was supported in part by PTT Public Company Limited, in part by Siam Commercial Bank (SCB) Public Company Limited, in part by the Thailand Science Research and Innovation under Grant SRI62W1501, and in part by the Office of National Higher Education Science Research and Innovation Policy Council under Grant C10F630057.