Abstract
Background
Subclinical (ie, threshold) social anxiety can greatly affect young people’s lives, but existing solutions appear inadequate considering its rising prevalence. Wearable sensors may provide a novel way to detect social anxiety and result in new opportunities for monitoring and treatment, which would be greatly beneficial for persons with social anxiety, society, and health care services. Nevertheless, indicators such as skin temperature measured by wrist-worn sensors have not been used in prior work on physiological social anxiety detection.
Objective
This study aimed to investigate whether subclinical social anxiety in young adults can be detected using physiological data obtained from wearable sensors, including heart rate, skin temperature, and electrodermal activity (EDA).
Methods
Young adults (N=12) with self-reported subclinical social anxiety (measured using the widely used self-reported version of the Liebowitz Social Anxiety Scale) participated in an impromptu speech task. Physiological data were collected using an E4 Empatica wearable device. Using the preprocessed data and following a supervised machine learning approach, various classification algorithms such as Support Vector Machine, Decision Tree, Random Forest, and K-Nearest Neighbours (KNN) were used to develop models for 3 different contexts. Models were trained to differentiate (1) between baseline and socially anxious states, (2) among baseline, anticipation anxiety, and reactive anxiety states, and (3) social anxiety among individuals with social anxiety of differing severity. The predictive capability of the singular modalities was also explored in each of the 3 supervised learning experiments. The generalizability of the developed models was evaluated using 10-fold cross-validation as a performance index.
Results
With modalities combined, the developed models yielded accuracies between 97.54% and 99.48% when differentiating between baseline and socially anxious states. Models trained to differentiate among baseline, anticipation anxiety, and reactive anxiety states yielded accuracies between 95.18% and 98.10%. Furthermore, the models developed to differentiate between social anxiety experienced by individuals with anxiety of differing severity scores successfully classified with accuracies between 98.86% and 99.52%. Surprisingly, EDA was identified as the most effective singular modality when differentiating between baseline and social anxiety states, whereas ST was the most effective modality when differentiating anxiety among individuals with social anxiety of differing severity.
Conclusions
The results indicate that it is possible to accurately detect social anxiety as well as distinguish between levels of severity in young adults by leveraging physiological data collected from wearable sensors.
Keywords: social anxiety, wearable sensors, physiological measurement, machine learning, young adults, mental health, mHealth, new methods, anxiety, wearable, sensor, digital phenotyping, digital biomarkers
Introduction
Background
Social anxiety is a fear of social situations in which the individual is exposed to possible scrutiny by others [1], and high levels of social anxiety are associated with a low quality of life in various domains [2,3]. Even when not clinically diagnosable (ie, subclinical or threshold social anxiety), it can greatly affect young people’s lives. Fehm et al [4] showed that young adults with social anxiety who do not receive treatment are at risk of developing social anxiety disorder (SAD) and comorbid mental health problems such as depression, both of which cause further adverse life impairments [3,5]. SAD is one of the most common anxiety disorders [6]. One UK study in 2000 [7] revealed that the annual health care cost per person with SAD was £609 (US $834.59), with annual productivity losses and social security benefits adding to £1920 (US $2631.22) per person with SAD, whereas those with SAD and a comorbidity incurred even higher costs. Nevertheless, many individuals do not receive treatment owing to limited availability or lack of awareness of social anxiety among health care professionals [3,4,8]. Some may not even seek treatment owing to a fear of being negatively evaluated by health care professionals [8]. Thus, it is imperative to empower both individuals and health care professionals in early detection of social anxiety before it potentially escalates into SAD and other related problems.
Common methods for assessing social anxiety involve using subjective measures, usually in a clinical setting. Owing to the rising prevalence of social anxiety, however, it is becoming evident that traditional approaches are inadequate and unsustainable for health care services [4,5,9]. In recent years, increasing focus has been given to technological advances that might help in the early detection and subsequent intervention for anxiety-related problems. In terms of social anxiety, objective methods used to assess symptoms include monitoring physiological changes typically caused by anxiety such as an elevated heart rate (HR), increased electrodermal activity (EDA), variation in skin temperature (ST), and trembling [1,10-12].
Nevertheless, despite extensive and promising research into stress and emotion detection based on physiological indices applicable to social anxiety collected from wearable sensors (Table 1), there has been little effort to predict social anxiety particularly using this approach. This might be ascribed to the recent shift in attention toward social anxiety reported by Heimberg and Butler [13], owing to widening of the diagnostic criteria and leading to a rise in those who fit the criteria for social anxiety.
Table 1.
Study | Classification algorithms | Physiological data | Detection | Reported accuracy, % |
[16] | SVMa, Decision Tree, KNNb, Naïve Bayes, Random Forest, Neural Network, Zero K | HRc, STd, EDAe | Stress | 65.8-100% |
[18] | SVM | EDA, BVPf, PDg | Stress | 57.1-80% |
[19] | SVM, Decision Tree, KNN, Naïve Bayes | EDA, BVP, PZTh, EEGi, ECGj, EMGk | Emotion | 17-91.3% |
[20] | SVM | BVP, ST, EDA, PD | Stress | 61.5-90.1% |
[21] | KNN | HRVl | Stress | 79.2-94.6% |
aSVM: Support Vector Machine.
bKNN: K-Nearest Neighbours.
cHR: heart rate.
dST: skin temperature.
eEDA: electrodermal activity.
fBVP: blood volume pulse.
gPD: pupillary distance.
hPZT: piezoelectric response.
iEEG: electroencephalogram.
jECG: electrocardiogram.
kEMG: electromyography.
lHRV: heart rate variability.
Although not without its problems, detection via wearable sensors has the potential to underpin solutions addressing the growing needs of individuals with social anxiety and complement traditional therapeutic approaches. If subclinical social anxiety could reliably and validly be detected using wearable sensors, initial treatment could subsequently transition to digital self-help solutions to aid social anxiety at earlier stages when treatment is less extensive and costly [7]. Furthermore, self-help solutions may be a more appropriate method of treatment as individuals with social anxiety often feel nervous to seek treatment in clinical settings [8]. Detecting social anxiety using evidence-based objective methods could also complement current therapeutic approaches.
Prior Work
Emotion Detection Using Machine Learning
A rise in wearable devices has further enabled researchers to investigate methods for the detection of emotion and stress states [14,15], with many studies reporting high-accuracy detection levels (Table 1). To detect emotional states using physiological data, researchers have executed data collection experiments that invoke the state to be detected, with tasks including hyperventilation and watching emotional films [16,17].
After data collection, a supervised machine learning (ML) approach is commonly used owing to the classification nature of the investigations [16-18]. In supervised ML, the training data are labeled in accordance with the correct class as the classification algorithms learn by example. Table 1 shows an overview of ML approaches focusing on emotion and stress detection. The most dominant and successful algorithm in studies involving recognition of states using physiological data is Support Vector Machine (SVM). Classifiers such as Decision Tree, Random Forest, and K-Nearest Neighbours (KNN) have also been frequently used and are reportedly effective.
Physiological Indicators of Social Anxiety
Classical psychological experiments commonly use impromptu public speaking tasks to elicit a social anxiety response [1,11,22,23]. These experiments are often split into stages that measure three responses: baseline, anticipatory, and reactive anxiety (where the nature of the speaking task is announced beforehand to provoke an anticipatory anxiety response) [1,11,23]. In conjunction, respondents are typically asked about their anxiety levels through self-reports [24]. Although self-reports are an important way of gauging individual perceptions of social anxiety, this approach is not without its problems, including a high level of subjectivity.
A more objective way to measure social anxiety is using physiological indicators. Social anxiety activates the sympathetic nervous system (SNS) [25]. HR and ST are modulated by both the parasympathetic nervous system (PNS) and SNS divisions of the autonomic nervous system (ANS), whereas EDA is modulated by the SNS alone. Therefore, EDA, HR, and ST are seen as markers of SNS activation and can be considered as potential indicators of social anxiety [26-29].
Studies investigating physiological responses to social anxiety further illustrate the potential to use EDA, HR, and ST as indicators [10-12]. Despite the potential for these indicators, however, their responses are complex, and a few studies have indicated minor differences in SNS arousal for individuals with social anxiety compared to control groups [22,23,30]. Furthermore, although ST has been explored as a social anxiety marker [12], wrist ST measurements have not been explored systematically. To our knowledge, this is the first study to explore wrist ST as an indicator of experimentally induced social anxiety.
Research Aims and Objectives
This study aimed to investigate whether social anxiety in young people with subclinical social anxiety can be detected using physiological data (based on HR, ST, and EDA) recorded from an existing multi-sensor wearable. The study aims to explore if models can be trained to differentiate (1) between baseline and socially anxious states, (2) among baseline, anticipation anxiety, and reactive anxiety states, and (3) between social anxiety among individuals with social anxiety of differing severity. This study also aims to explore the predictive capability of the singular modalities.
Methods
Recruitment
Young adult participants were recruited using posters around Imperial College London. The initial sample comprised 13 individuals who self-identified as shy or socially fearful. An exclusion criterion was created to ensure that only young adults with subclinical social anxiety were recruited, as described in Figure 1. To assess participants’ social anxiety levels, the self-reported version of the Liebowitz Social Anxiety Scale (LSAS-SR) was initially used (mean 64.33, SD 13.12, range 38-80). In total, 13 individuals attended the experiment. One participant who showed up for the experiment was known to the experimenter and had their data subsequently excluded owing to likely bias. The final study sample thus comprised 12 participants (58% female; mean age, 19.75 years, SD 1.76 years; 67% Asian, 25% White, and 8% Mixed race).
Measures
LSAS-SR
We used the self-report version of the LSAS-SR owing to its well-established validity and reliability in a large amount of previous literature [24]. The LSAS-SR allows for the classification of individuals into differing severity groups, as a higher overall LSAS-SR score is seen to correspond with greater social anxiety severity [24,31]. Furthermore, the LSAS-SR examines both affective aspects (ie, quantifying how anxious participants feel) using the fear subscale and behavioral aspects (ie, gauging to what extent they avoid various social situations) using the avoidance subscale. Each subscale consists of 24 items, with response items ranging on a 4-point scale from “none (0)” to “severe (3)” for the fear subscale, and “never (0)” to “usually (3)” for the Avoidance subscale. Prior studies indicate a high level of reliability of the LSAS-SR (Cronbach α=.95 [24]). In this study, the Cronbach α values for the fear and avoidance subscales were .69 and .69, respectively, with an overall Cronbach α of .83.
Social Phobia Screening Questionnaire
To cross-validate the LSAS-SR, we also used the Social Phobia Screening Questionnaire (SPSQ), which comprises 8 questions about how much fear individuals feel in various social situations, including speaking in front of a group of people, going to a party, and being alone with someone unfamiliar [32]. This measure has shown good validity in prior research [32]. It can be used with or without additional questions that allow an estimation of whether individuals reach the clinical cut-off for SAD and has been used in previous research to indicate subclinical social anxiety levels [33]. The response items ranged from “none (1)” and “some (2)” to “a lot (3)” (Cronbach α=.74).
Ethics
The University Ethics Committee approved all the procedures and measures used in the study. Throughout the procedure, participants were reminded that their participation was voluntary and that they could withdraw their data at any time until used for statistical analysis. The collected data were anonymized and stored in a password-protected folder.
Data Collection
The data were collected using the E4 Empatica research-grade multi-sensor wristband wearable. The device was selected as it simultaneously monitors various types of physiological data at predetermined sampling rates [34]. However, only HR, EDA, and ST data were explored in this study as they could be considered social anxiety markers [26-29]. E4 Empatica has not yet been used in many studies of this nature, although other multi-sensor wrist-worn wearables have demonstrated effectiveness [16,17].
Using the default sampling rates of the E4 [35], EDA was measured in microSiemens (μS) at 4 Hz using stainless steel electrodes positioned on the inner side of the wrist. HR was measured in beats per minute (BPM) at 1 Hz using data derived from a photoplethysmography sensor. ST was measured in °C at 4 Hz using an infrared thermophile [35]. The data were collected throughout the duration of the experiment, an example of which is shown in Figure 2. The full data set and code needed to recreate the classification models and reproduce the results, as well as functions that enable further experimentation, is available in a designated GitHub repository (Multimedia Appendix 1).
Experimental Protocol
The experiments had an approximate duration of 30 minutes. Similar to previous studies [1,11,23], the experiment was split into 3 stages involving relaxation (baseline), task preparation (anticipation anxiety), and performance (reactive anxiety), as responses might differ across these stages [25]. Figure 3 illustrates the experimental stages. The timestamps for the stages were also recorded for labeling purposes. The experimental protocol is listed below.
First, the wearable was attached to the participant’s wrist. The procedure commenced with a 10-minute baseline period. During this time, participants were offered magazines and ocean sounds were played to create a calming effect.
Second, the nature of the task was then announced, and the participant was given 5 minutes to prepare a 3-minute speech on a selected subject from a choice of topics chosen on the basis of their anxiety-inducing potential. These included “Is Brexit good or bad, and why?,” “Intelligence is not enough,” and “The history of Western Europe until the 2000s.”
Third, a “judging” panel comprising experimenter confederates entered the room, and the participant performed the speech while being timed.
Finally, the participant was debriefed, and the wearable was removed.
Data Preprocessing
The HR data were first upsampled to 4 Hz, similar to ST and EDA. A Moving Average Filter (Equation 1) was then applied to the data to remove noise [17] and reduce the risk of model overfitting [36,37].
Where w refers to window size, Input[i] refers to original time series signal and Output[i] refers to processed time series signal.
An EDA range correction method (Equation 2) was applied to each participant’s EDA (E) data, see Figure 4 [38]. This removed inter-individual differences, particularly as physiological activation is believed to be better indicated by the variation within the EDA range rather than the range itself [39].
Following this, the labels were allocated on the basis of the experiment timestamps, assuming the suspected states were invoked. Classification investigation (1) examined whether models can be trained to classify baseline and socially anxious states. Therefore, the participants’ data were split into the respective classes and labeled using the experiment timestamps (Figure 5).
Classification investigation (2) focused on whether models can be trained to differentiate among baseline, anticipation anxiety, and reactive anxiety states. Therefore, the data were divided into the 3 respective classes using the timestamps and labeled as shown in Figure 6.
Finally, classification investigation (3) examined whether models could be trained to differentiate between anxiety experienced by individuals with differing levels of social anxiety. Therefore, the data were collected from participants within differing ranges of LSAS-SR scores, including anxiety category 1 (LSAS-SR:50-64) and anxiety category 2 (LSAS-SR:65-80) and was subsequently labeled (Figure 7).
The first 2 minutes from the baseline period were disregarded to account for acclimatization, and the recording was discarded after the task as it was not needed. All participant data were then combined.
The features were standardized to have zero mean and unit variance, which is a widely used scaling approach as algorithms such as Radial SVM assume features are centered around zero [36,40]. For each feature, the mean (µ) and the standard deviation (σ) were extracted from the raw training feature values. The training data were then standardized using equation (3), and the same transformation was applied to the test data [37].
Classification
The investigations were framed as supervised learning tasks owing to their classification nature. Four classification algorithms were explored: SVM, Random Forest, Decision Tree, and KNN. Furthermore, for classification investigation (2), a “One Vs. Rest” strategy was utilized as the investigation involved a multi-class data set.
The trained models were evaluated using 10-fold cross-validation. The method involves dividing the data set into k-folds with 1 fold for testing and the others for training. Confusion matrices were also utilized to calculate the average classification accuracy for each class.
Results
Study Descriptives
All study descriptives are shown in Table 2. In this sample, women had higher mean levels for all study variables than men (though the differences were nonsignificant, which is likely owing to the small sample). This is uncharacteristic, as women typically have a higher risk to develop anxiety and higher mean levels of social anxiety than men [41]. However, the self-reported LSAS scores were highly correlated with SPSQ scores (r=0.63; P=.05).
Table 2.
Gender | Participants, n | Score, mean (SD) | |
Liebowitz Social Anxiety Scale fear subscale | |||
|
Women | 7 | 1.3095 (0.33666) |
|
Men | 5 | 1.5917 (0.11562) |
Liebowitz Social Anxiety Scale avoidance subscale | |||
|
Women | 7 | 1.1845 (0.34766) |
|
Men | 5 | 1.3583 (0.25786) |
Liebowitz Social Anxiety Scale avoidance subscale overall score | |||
|
Women | 7 | 1.2470 (0.33190) |
|
Men | 5 | 1.4750 (0.15548) |
Social Phobia Screening Questionnaire | |||
|
Women | 7 | 1.3469 (0.36288) |
|
Men | 5 | 1.7429 (0.29277) |
Combined Modalities
For classification investigation (1), the yielded accuracies were between 97.54% and 99.48%, as shown in Table 3. For investigation (2), the accuracies were between 95.18% and 98.10%, as shown in Table 4. Additionally, for investigation (3) the yielded accuracies were between 98.86% and 99.52%, as shown in Table 5. In each classification investigation, Radial SVM outperformed other classifiers (Tables 3-5).
Table 3.
Classifier | Overall performance, % | Baseline state accuracy, % | Social anxiety state accuracy, % |
Radial Support Vector Machine | 99.48 | 99.40 | 99.52 |
K-Nearest Neighbours | 99.08 | 99.12 | 99.05 |
Decision Tree | 97.54 | 99.04 | 96.59 |
Random Forest | 97.96 | 99.38 | 97.13 |
Table 4.
Classifier | Overall performance, % | Baseline state accuracy, % | Anticipation anxiety state accuracy, % | Reactive anxiety state accuracy, % |
Radial Support Vector Machine | 98.10 | 99.30 | 98.37 | 95.52 |
K-Nearest Neighbours | 97.61 | 98.99 | 97.28 | 95.78 |
Decision Tree | 96.63 | 99.39 | 96.86 | 91.36 |
Random Forest | 95.18 | 99.27 | 95.99 | 85.99 |
Table 5.
Classifier | Overall performance, % | Anxiety category 1 accuracy, % | Anxiety category 2 accuracy, % |
Radial Support Vector Machine | 99.52 | 100 | 99.03 |
K-Nearest Neighbours | 98.86 | 99.35 | 98.36 |
Decision Tree | 99.04 | 100 | 98.09 |
Random Forest | 99.34 | 100 | 98.70 |
There were common class misclassification patterns among all classifiers. For investigation (1), the models were less able to classify anxious states (Table 3). For investigation (2), reactive anxiety was misclassified the most and often mistaken for anticipation anxiety (Figure 8). Additionally, in investigation (2), the baseline class was most accurately classified, as shown in Table 4. For investigation (3), the models were not as effective at classifying anxiety category 2 (Table 5).
Singular Modalities
The singular modality results are shown in Table 6 and Figure 9. In classification investigation (1) EDA yielded 80.46% and was shown to have the highest predictive capability. EDA was also shown to have the highest classification accuracy of 70.02% for investigation (2), whereas ST was the most effective modality for investigation (3) with an accuracy of 89.47%. For each classification investigation, HR was observed to be the least effective modality. Furthermore, KNN generally outperformed other classifiers (Table 6).
Table 6.
Modality | Classifier with the highest performance | Overall performance, % | |||
Classification investigation 1 | |||||
|
Heart rate | K-Nearest Neighbours | 68.18 | ||
|
Skin temperature | K-Nearest Neighbours | 76.30 | ||
|
Electrodermal activity | K-Nearest Neighbours | 80.46 | ||
Classification investigation 2 | |||||
|
Heart rate | K-Nearest Neighbours | 53.91 | ||
|
Skin temperature | K-Nearest Neighbours | 68.32 | ||
|
Electrodermal activity | K-Nearest Neighbours | 70.02 | ||
Classification investigation 3 | |||||
|
Heart rate | K-Nearest Neighbours | 72.00 | ||
|
Skin temperature | K-Nearest Neighbours | 89.47 | ||
|
Electrodermal activity | Random Forest and Decision Tree | 75.66 |
Discussion
Principal Findings
Combined Modalities
This study aimed to determine if ML models could be trained to (1) classify baseline and socially anxious states, (2) differentiate among baseline, anticipation anxiety, and reactive anxiety states, and (3) classify social anxiety with differing severity levels of social anxiety. High accuracies were obtained when differentiating between baseline and socially anxious states, suggesting that it is possible to detect social anxiety using HR, ST, and EDA. These high accuracies are likely due to physiological differences between baseline and socially anxious states and have also been shown in previous research [10-12].
The models also yielded high accuracies when classifying among baseline, anticipatory, and reactive states. The classifiers’ ability to differentiate between reactive and anticipatory anxiety might be due to the varying responses during these stages. It is, therefore, likely possible to detect the nature of social anxiety experienced on an individual basis.
The models also yielded high accuracies when differentiating between marked and moderate social anxiety. This demonstrates the possibility to identify social anxiety levels using physiological indices, implying that individuals with differing severity levels of social anxiety exhibit diverse physiological responses. This is in line with prior research indicating that individuals with greater social anxiety exhibit responses consistent with greater threat [10].
The results also indicated that higher modeling accuracies were yielded when all modalities were combined [42]. Research shows that models created using singular modalities may be underfit owing to lack of data [37]. This is likely because each physiological index contains varying information that enables classifiers to differentiate among certain classes, thus providing measurement granularity. Furthermore, when modalities were combined, Radial SVM outperformed the other classifiers in all investigations, which is possibly owing to the classifier’s ability to formulate complex decision boundaries [37,43].
Finally, certain classes were commonly misclassified, which could be explained by class imbalances owing to the different durations of each stage during the data collection sessions. Class imbalances can cause classifiers to bias toward larger classes [44].
Singular Modalities
Each modality had varying predictive capabilities, despite the complexity of the physiological indicators used in the study. EDA was the most effective singular modality when differentiating between baseline and social anxiety states (including anticipatory and reactive states). This is possibly because EDA comprises the sum of phasic and tonic components that change following stimuli, which is likely because sweat glands responsible for EDA variation are entirely controlled by the SNS, whereas HR and ST are mediated by both the PNS and SNS [26,27,29]. Thus, EDA represents an accumulation of information that could indicate social anxiety [27].
ST was the most effective modality when differentiating between anxiety experienced by individuals with differing severity levels of social anxiety. This might be because individuals with greater social anxiety exhibit differing amounts of blood flow to the skin. This surprising finding highlights the predictive capability of ST collected around the wrist and suggests that it could be viewed as a novel social anxiety marker.
HR showed the lowest effectiveness in all investigations, which might be explained by HR being mediated by the PNS and SNS [26,27]. The comparatively low recognition accuracies may also be a result of HR being sampled at the lowest rate.
Furthermore, KNN was the most effective classifier when the modalities were singular, which is likely because KNN can formulate complex decision boundaries between classes.
Limitations
Despite these promising results, these findings are preliminary. The sample size was small, with the COVID-19 pandemic preventing further data collection. Prior to the COVID-19 pandemic, we intended to collect test data in “real-world” settings to evaluate the models’ ability to detect social anxiety in practice. Instead, the models were evaluated using a subset of data from the experiment. Although this approach is often used in ML studies [16,17], it does not offer a realistic indication of model generalizability. Therefore, given the small sample size, our results need to be interpreted cautiously.
Additionally, classifiers may have been biased toward certain classes owing to the moderately differing class sizes (Multimedia Appendix 2). This may have accounted for the high accuracies but reduced model generalizability [44]. Like other studies of a similar nature (such as affect recognition studies using physiological data [45]), it was difficult to establish the ground truth of the data with respect to the presence and nature of social anxiety. Therefore, labeling was assumed to be aligned with the experimental protocol.
Furthermore, the physiological responses from the individuals could have been influenced by external factors such as caffeine and alcohol consumption [46,47], though this was not mitigated in the current study design. It is also important to note that EDA measurements can be affected by environmental conditions such as humidity and room temperature [27]. Although the experiments took place in the same room, these variables were not monitored and controlled. In sum, all of these limitations remain challenges for future research.
Comparison With Prior Work
Despite its limitations, this study has extended previous work and applications focusing on supervised machine learning in the field of physiological anxiety detection. This experiment was informed by existing study protocols, such as using an impromptu speech task, which is a cornerstone of experimental work invoking social anxiety [11,22,23]. Additionally, the study utilized the LSAS-SR measure, which is a widely used measure demonstrating good psychometric properties in previous research [24], and the social anxiety self-reports were cross-validated using another well-known indicator of subclinical social anxiety (SPSQ [32]). Overall, our findings also align with those of prior studies indicating that EDA is a “directed and undiluted” representation of the SNS [27]. Although prior work has focused on EDA as an indicator, physiological measurement from the anatomical site of the wrist had not been explored in a social anxiety context.
Conclusions
This study examined whether social anxiety could be detected in young adults using physiological data (HR, ST, and EDA) from wrist-worn sensors. The findings indicate that it is possible to detect social anxiety and its severity using this approach. Future work in this area has the potential to identify novel methods of detecting and monitoring subclinical social anxiety in young adults, which could help counteract development into SAD. As mental health provision is transitioning toward digital interventions, it is crucial that they are evidence-based and can target individuals with subclinical levels of social anxiety. The ability for future interventions to detect social anxiety before it escalates further could have great social and economic benefits for health care, society and those who experience its consequences.
Acknowledgments
We would like to thank all the individuals who volunteered to take part in this study.
Abbreviations
- ANS
autonomic nervous system
- BPM
beats per minute
- EDA
electrodermal activity
- HR
heart rate
- KNN
K-Nearest Neighbours
- LSAS-SR
self-reported version of the Liebowitz Social Anxiety Scale
- ML
machine learning
- PNS
parasympathetic nervous system
- SAD
social anxiety disorder
- SNS
sympathetic nervous system
- SPSQ
Social Phobia Screening Questionnaire
- ST
skin temperature
- SVM
Support Vector Machine
A Github repository was created to accompany this work. The repository contains the full dataset and code needed to recreate the classification models and reproduce the results, as well as functions that enable further experimentation.
The following tables illustrate the class data distributions for each classification investigation.
Footnotes
Conflicts of Interest: None declared.
References
- 1.Heiser NA, Turner SM, Beidel DC, Roberson-Nay R. Differentiating social phobia from shyness. J Anxiety Disord. 2009 May;23(4):469–76. doi: 10.1016/j.janxdis.2008.10.002. http://europepmc.org/abstract/MED/19028075 .S0887-6185(08)00181-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Eng W, Coles ME, Heimberg RG, Safren SA. Domains of life satisfaction in social anxiety disorder: relation to symptoms and response to cognitive-behavioral therapy. J Anxiety Disord. 2005;19(2):143–156. doi: 10.1016/j.janxdis.2004.01.007.S088761850400009X [DOI] [PubMed] [Google Scholar]
- 3.Wittchen H, Fuetsch M, Sonntag H, Müller N, Liebowitz M. Disability and quality of life in pure and comorbid social phobia--findings from a controlled study. Eur Psychiatry. 1999 Jun;14(3):118–131. doi: 10.1016/s0924-9338(99)80729-9.S0924-9338(99)80729-9 [DOI] [PubMed] [Google Scholar]
- 4.Fehm L, Beesdo K, Jacobi F, Fiedler A. Social anxiety disorder above and below the diagnostic threshold: prevalence, comorbidity and impairment in the general population. Soc Psychiatry Psychiatr Epidemiol. 2008 Apr;43(4):257–265. doi: 10.1007/s00127-007-0299-4. [DOI] [PubMed] [Google Scholar]
- 5.Spence SH, Rapee RM. The etiology of social anxiety disorder: An evidence-based model. Behav Res Ther. 2016 Nov;86:50–67. doi: 10.1016/j.brat.2016.06.007.S0005-7967(16)30111-5 [DOI] [PubMed] [Google Scholar]
- 6.Kessler RC, Chiu WT, Demler O, Merikangas KR, Walters EE. Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatry. 2005 Jun;62(6):617–627. doi: 10.1001/archpsyc.62.6.617. http://europepmc.org/abstract/MED/15939839 .62/6/617 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Patel A, Knapp M, Henderson J, Baldwin D. The economic consequences of social phobia. J Affect Disord. 2002 Apr;68(2-3):221–233. doi: 10.1016/s0165-0327(00)00323-2.S0165032700003232 [DOI] [PubMed] [Google Scholar]
- 8.National Collaborating Centre for Mental Health . Social Anxiety Disorder: Recognition, Assessment and Treatment. Leicester: British Psychological Society; 2013. [2021-09-29]. https://www.nice.org.uk/guidance/CG159 . [PubMed] [Google Scholar]
- 9.Mccrone P, Dhanasiri S, Patel A, Knapp M, Smith SL. Paying the Price: The cost of mental health care in England to 2026. King's Fund. 2008. [2021-09-27]. https://www.kingsfund.org.uk/sites/default/files/Paying-the-Price-the-cost-of-mental-health-care-England-2026-McCrone-Dhanasiri-Patel-Knapp-Lawton-Smith-Kings-Fund-May-2008_0.pdf .
- 10.Shimizu M, Seery MD, Weisbuch M, Lupien SP. Trait social anxiety and physiological activation: cardiovascular threat during social interaction. Pers Soc Psychol Bull. 2011 Jan;37(1):94–106. doi: 10.1177/0146167210391674.37/1/94 [DOI] [PubMed] [Google Scholar]
- 11.Thompson T, Kaminska M, Marshall C, Van Zalk N. Evaluation of the social phobia scale and social interaction anxiety scale as assessments of performance and interaction anxiety. Psychiatry Res. 2019 Mar;273:725–731. doi: 10.1016/j.psychres.2019.01.103.S0165-1781(18)32082-1 [DOI] [PubMed] [Google Scholar]
- 12.Edelmann RJ, Baker SR. Self-reported and actual physiological responses in social phobia. Br J Clin Psychol. 2002 Mar;41(Pt 1):1–14. doi: 10.1348/014466502163732. [DOI] [PubMed] [Google Scholar]
- 13.Heimberg RG, Butler RM. Research on social anxiety disorder continues to escalate: A commentary on Asmundson and Asmundson's analysis of publication trends in the anxiety disorders. J Anxiety Disord. 2018 May;56:8–10. doi: 10.1016/j.janxdis.2018.04.001.S0887-6185(18)30132-4 [DOI] [PubMed] [Google Scholar]
- 14.Peake JM, Kerr G, Sullivan JP. A Critical Review of Consumer Wearables, Mobile Applications, and Equipment for Providing Biofeedback, Monitoring Stress, and Sleep in Physically Active Populations. Front Physiol. 2018;9:743. doi: 10.3389/fphys.2018.00743. doi: 10.3389/fphys.2018.00743. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Kanjo E, Al-Husain L, Chamberlain A. Emotions in context: examining pervasive affective sensing systems, applications, and analyses. Pers Ubiquit Comput. 2015 Apr 25;19(7):1197–1212. doi: 10.1007/s00779-015-0842-3. doi: 10.1007/s00779-015-0842-3. [DOI] [Google Scholar]
- 16.de Arriba-Pérez F, Santos-Gago J, Caeiro-Rodríguez M, Ramos-Merino M. Study of stress detection and proposal of stress-related features using commercial-off-the-shelf wrist wearables. J Ambient Intell Human Comput. 2019 Jan 8;10(12):4925–4945. doi: 10.1007/s12652-019-01188-3. doi: 10.1007/s12652-019-01188-3. [DOI] [Google Scholar]
- 17.Zhang Z, Song Y, Cui L, Liu X, Zhu T. Emotion recognition based on customized smart bracelet with built-in accelerometer. PeerJ. 2016;4:e2258. doi: 10.7717/peerj.2258. doi: 10.7717/peerj.2258.2258 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Zhai J, Barreto AB, Chin C, Li C. Realization of stress detection using psychophysiological signals for improvement of human-computer interactions. Proceedings. IEEE SoutheastCon, 2005; April 8-10, 2005; Fort Lauderdale, FL. 2005. [DOI] [Google Scholar]
- 19.Xu Y, Hübener I, Seipp AK, Ohly S, David K. From the lab to the real-world: An investigation on the influence of human movement on Emotion Recognition using physiological signals. 2017 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops); March 13-17, 2017; Kona, HI. 2017. [DOI] [Google Scholar]
- 20.Zhai J, Barreto A. Stress Detection in Computer Users Based on Digital Signal Processing of Noninvasive Physiological Variables. 2006 International Conference of the IEEE Engineering in Medicine and Biology Society; August 30 to September 3, 2006; New York, NY. 2006. [DOI] [PubMed] [Google Scholar]
- 21.Karthikeyan P, Murugappan M, Yaacob S. Analysis of Stroop Color Word Test-Based Human Stress Detection using Electrocardiography and Heart Rate Variability Signals. Arab J Sci Eng. 2013 Sep 11;39(3):1835–1847. doi: 10.1007/s13369-013-0786-8. doi: 10.1007/s13369-013-0786-8. [DOI] [Google Scholar]
- 22.Anderson ER, Hope DA. The relationship among social phobia, objective and perceived physiological reactivity, and anxiety sensitivity in an adolescent population. J Anxiety Disord. 2009 Jan;23(1):18–26. doi: 10.1016/j.janxdis.2008.03.011. http://europepmc.org/abstract/MED/18436426 .S0887-6185(08)00083-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Knight ML, Borden RJ. Autonomic and affective reactions of high and low socially-anxious individuals awaiting public performance. Psychophysiology. 1979 May;16(3):209–213. doi: 10.1111/j.1469-8986.1979.tb02979.x. [DOI] [PubMed] [Google Scholar]
- 24.Fresco D, Coles M, Heimberg R, Liebowitz M, Hami S, Stein M, Goetz D. The Liebowitz Social Anxiety Scale: a comparison of the psychometric properties of self-report and clinician-administered formats. Psychol Med. 2001 Aug;31(6):1025–1035. doi: 10.1017/s0033291701004056. [DOI] [PubMed] [Google Scholar]
- 25.Leary MR, Kowalski RM. Social Anxiety. New York, NY: Guilford Press; 1995. pp. 83–131. [Google Scholar]
- 26.Michael-Titus A, Revest P, Shortland P. The Nervous System (2nd edition) New York, NY: Churchill Livingstone; 2010. pp. 1–30. [Google Scholar]
- 27.Cacioppo J, Tassinary LG, Berntson GG. Handbook of Psychophysiology. Cambridge: Cambridge University Press; 2007. [Google Scholar]
- 28.Craske MG, Waters AM, Lindsey Bergman R, Naliboff B, Lipp OV, Negoro H, Ornitz EM. Is aversive learning a marker of risk for anxiety disorders in children? Behav Res Ther. 2008 Aug;46(8):954–967. doi: 10.1016/j.brat.2008.04.011. http://europepmc.org/abstract/MED/18539262 .S0005-7967(08)00093-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Koeppen BM, Stanton BA. Berne & Levy Physiology. Philadelphia, PA: Elsevier; 2018. [Google Scholar]
- 30.Mauss IB, Wilhelm FH, Gross JJ. Autonomic recovery and habituation in social anxiety. Psychophysiology. 2003 Jul;40(4):648–653. doi: 10.1111/1469-8986.00066. [DOI] [PubMed] [Google Scholar]
- 31.Heimberg RG, Horner KJ, Juster HR, Safren SA, Brown EJ, Schneier FR, Liebowitz MR. Psychometric properties of the Liebowitz Social Anxiety Scale. Psychol Med. 1999 Jan 01;29(1):199–212. doi: 10.1017/s0033291798007879. [DOI] [PubMed] [Google Scholar]
- 32.Furmark T, Tillfors M, Everz P, Marteinsdottir I, Gefvert O, Fredrikson M. Social phobia in the general population: prevalence and sociodemographic profile. Soc Psychiatry Psychiatr Epidemiol. 1999 Aug;34(8):416–424. doi: 10.1007/s001270050163.90340416.127 [DOI] [PubMed] [Google Scholar]
- 33.Van Zalk N, Tillfors M, Mörtberg E. Social Anxiety-Impulsivity Subgroups and Links to Later Emotional Adjustment in Adolescence: A Latent Transition Analysis. J Early Adolesc. 2020 May 19;40(9):1397–1426. doi: 10.1177/0272431620919153. doi: 10.1177/0272431620919153. [DOI] [Google Scholar]
- 34.E4 wristband: Real-time physiological data streaming and visualization. [2021-09-27]. https://www.empatica.com/research/e4/
- 35.E4 wristband user's manual. [2021-09-27]. https://empatica.app.box.com/v/E4-User-Manual .
- 36.Géron A. Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems. Newton, MA: O'Reilly Media; 2019. [Google Scholar]
- 37.Dougherty G. Pattern Recognition and Classification. New York, NY: Springer; 2013. [Google Scholar]
- 38.Lykken DT, Rose R, Luther B, Maley M. Correcting psychophysiological measures for individual differences in range. Psychol Bull. 1966 Dec;66(6):481–484. doi: 10.1037/h0023922. [DOI] [PubMed] [Google Scholar]
- 39.Lykken DT, Venables PH. Direct measurement of skin conductance: a proposal for standardization. Psychophysiology. 1971 Sep;8(5):656–672. doi: 10.1111/j.1469-8986.1971.tb00501.x. [DOI] [PubMed] [Google Scholar]
- 40.Paper D. Hands-on Scikit-Learn for Machine Learning Applications. Berkeley, CA: Apress; 2020. [2021-09-29]. Introduction to Scikit-Learn. https://scikit-learn.org/stable/modules/preprocessing.html . [Google Scholar]
- 41.Kessler RC, Berglund P, Demler O, Jin R, Merikangas KR, Walters EE. Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatry. 2005 Jun;62(6):593–602. doi: 10.1001/archpsyc.62.6.593.62/6/593 [DOI] [PubMed] [Google Scholar]
- 42.Anusha A, Jose J, Preejith S, Jayaraj J, Mohanasankar S. Physiological signal based work stress detection using unobtrusive sensors. Biomed Phys Eng Express. 2018 Sep 07;4(6):065001. doi: 10.1088/2057-1976/aadbd4. doi: 10.1088/2057-1976/aadbd4. [DOI] [Google Scholar]
- 43.Kampouraki A, Manis G, Nikou C. Heartbeat time series classification with support vector machines. IEEE Trans Inf Technol Biomed. 2009 Jul;13(4):512–518. doi: 10.1109/TITB.2008.2003323. [DOI] [PubMed] [Google Scholar]
- 44.Muller A, Guido S. Introduction to Machine Learning with Python: A Guide for Data Scientists. Newton, MA: O'Reilly Media; 2017. pp. 76–284. [Google Scholar]
- 45.Picard R, Vyzas E, Healey J. Toward machine emotional intelligence: analysis of affective physiological state. IEEE Trans Pattern Anal Machine Intell. 2001;23(10):1175–1191. doi: 10.1109/34.954607. doi: 10.1109/34.954607. [DOI] [Google Scholar]
- 46.Barry RJ, Clarke AR, Johnstone SJ, Rushby JA. Timing of caffeine's impact on autonomic and central nervous system measures: clarification of arousal effects. Biol Psychol. 2008 Mar;77(3):304–316. doi: 10.1016/j.biopsycho.2007.11.002.S0301-0511(07)00189-5 [DOI] [PubMed] [Google Scholar]
- 47.Schrieks I, Stafleu A, Kallen V, Grootjen M, Witkamp R, Hendriks H. Moderate alcohol consumption, autonomic nervous system and mood. Appetite. 2013;71:485–485. doi: 10.1016/j.appet.2013.06.061. doi: 10.1016/j.appet.2013.06.061. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
A Github repository was created to accompany this work. The repository contains the full dataset and code needed to recreate the classification models and reproduce the results, as well as functions that enable further experimentation.
The following tables illustrate the class data distributions for each classification investigation.