Development of a Heart Rate Variability Based Ambulatory Stress Detection Model for Clinical Populations

Abstract

Biosensor-based, real-time stress detection has generated clinical interest for the purpose of driving just-in-time interventions that support recovery from mental disorders. Most stress detection models to date, however, have been trained with laboratory-based data from homogenous samples of healthy adults, and do not perform as well in clinical populations. As an initial step toward the development of a stress detection algorithm that functions well in clinical populations, we tested a series of stress-detection machine learning models on ambulatory electrocardiogram (ECG) and daily ecological momentary assessment (EMA) data collected from a sample of individuals in early recovery from alcohol use disorder (AUD). Forty-four individuals ages 18–65 in the first year of a current AUD recovery attempt wore an ECG monitor for 4 days, while concurrently completing 3-times-daily EMA of stress. Data were segmented and normalized. Target features were identified using unsupervised learning models (e.g., t-SNE, cluster analysis) and supervised learning models were tuned to optimize model performance. As a comparator, we also tested these models with laboratory-derived stress data from a sample of healthy young adults. Before accounting for individual characteristics, we achieved a modest accuracy of 63% in our clinical sample, which compared to 94% accuracy in the laboratory-derived healthy young adult sample. After accounting for age and body-mass-index (BMI) we increased model accuracy up to 80% in our clinical sample. Stress detection is challenging in clinical populations; however, better prediction is possible with data normalization and stratification considering age and BMI.

Introduction

The deleterious health effects of psychological stress are widely acknowledged (O’Connor et al., 2021). However, stress can be especially consequential for individuals with mental disorders, in that it may increase risk for engagement in maladaptive coping behaviors that undermine recovery efforts (e.g., Sinha, 2012).

Recent advances in wearable health technologies have generated much excitement because they have the potential to support real-time, objective stress detection through continuous monitoring of physiological arousal via embedded biosensors. Clinical interest in this technology is high because reliable real-time stress detection could be used to drive just-in-time interventions that prompt individuals to engage in adaptive coping strategies to mitigate stress-related risk. This kind of just-in-time intervention could be especially useful in populations where poor interoceptive awareness of affective states heightens risk. One population who may particularly benefit from a just-in-time stress mitigation is individuals seeking recovery from substance use disorder, as stress can be a trigger for substance use that bypasses individuals’ conscious intentions not to use alcohol and other drugs (Koob & Le Moal, 1997; Sinha, 2007).

Because affective states such as stress are regulated by integrated neurocardiac systems (Critchley, 2005; Eddie et al., 2022), heart rate (HR) and heart rate variability (HRV) may be leveraged as bio-indicants of stress states (Berntson & Cacioppo, 2004). Algorithms for detecting stress have utilized various combinations of HR, HRV, movement detection (accelerometry or actigraphy), respiration monitoring, and galvanic skin response (a.k.a. electrodermal activity); however, previous work has shown that stress can be detected using features derived from HRV alone (Plarre et al., 2011; Sarker et al., 2016; Shi et al., 2010). An added benefit of utilizing HR and HRV in stress detection algorithms is that these metrics are measurable with existing smartwatches and fitness trackers, making stress detection cost effective, accessible, and scalable.

Prior Work

Over the past decade, a variety of personalized ideographic models for stress detection have been demonstrated (e.g., Rozet et al., 2019). While these models can achieve fairly high accuracy (85–90%), this ideographic approach is of limited use in clinical practice because it is not yet logistically possible to construct a custom machine learning model for each end-user.

For practical reasons, the prevailing approach for developing stress detection algorithms has focused on nomothetic models, in which a sample dataset is used to train a general model that can be used for multiple users. To date, most nomothetic stress detection models have been trained with data from controlled laboratory stress reactivity paradigms, with a much smaller subset of studies training models with ambulatory, daily-life data (Can et al., 2019; Gedam & Paul, 2021). Additionally, the vast majority of these studies have trained models with data from healthy adults. While this aids stress detection accuracy in generally healthy populations, these models are likely to perform poorly for people with mental disorders, who as a group are psychologically and psychophysiologically heterogeneous (Wright & Woods, 2020), and require special considerations in psychophysiological research (Morris et al., 2015; Wright & Woods, 2020).

Kumar and colleagues’ cStress model (Plarre et al., 2011) is an exemplar stress detection approach that predicts the probability of stress from 1-minute segments of electrocardiogram (ECG) output from a laboratory stress paradigm with a small sample of college students. In initial testing, the cStress model achieved relatively high stress detection accuracy (86–88% accuracy and 4% false positive rate). This model, as well as subsequent laboratory-based stress prediction models have demonstrated accuracies in the range 80–99% when applied to data derived from laboratory stress paradigms (Can et al., 2019). However, when applied to ambulatory ECG data outside the laboratory setting (i.e. in natura), the cStress model trained on laboratory data exhibited poorer performance, with less than 72% accuracy (Plarre et al., 2011).

It is possible that to predict in natura stress with acceptable accuracy, stress prediction models need to be trained with in natura psychophysiological data. To date, however, the few studies using this approach have not produced robust results, with studies achieving 61–76% accuracy when training and testing models using ambulatory data from healthy adults (Can et al., 2019). Though in natura trained and tested models do not appear to fare as well as laboratory trained and tested models with homogenous samples of healthy adults, they may still have utility for stress prediction in more heterogenous clinical populations, especially if we account for individual characteristics. Instead of designing personalized ideographic models, a practical clinical implementation may be to create a small number of general nomothetic models which can be applied to stratified groups of patients.

In this study, we sought to develop a nomothetic model for stress detection that performs well in real-world settings with heterogenous clinical populations, with data from individuals in the first year of an alcohol use disorder (AUD) recovery attempt. As a comparative reference, we also show results for our models with data from a homogenous population of healthy graduate students exposed to a laboratory-based cue-reactivity paradigm (the Wearable Stress and Affect Detection Data Set; Schmidt et al., 2018a). Additionally, we explored different combinations of HR and HRV measures to increase stress detection accuracy, experimented with parameter normalization of these measures, and accounted for individual characteristics including age and body-mass index (BMI). All classification of data was based on a binary model of stress vs. non-stress as a necessary first step, mirroring the binary classification task of the original cStress model.

Methods

Clinical Data Collection

For our study, we recruited individuals in the first year of an AUD recovery attempt from the greater Boston area (Eddie et al., 2021a, 2023; Emery et al., 2022). Our inclusion criteria included the following individual requirements: (1) Meet DSM-5 AUD mild, moderate, or severe criteria (American Psychiatric Association, 2013), (2) must endorse a current goal of alcohol abstinence, (3) must be in the first year of a current AUD recovery attempt, and (4) must be participating in an outpatient treatment or mutual-help program for AUD. To minimize potential effects of acute alcohol withdrawal on physiological measures, participants were required to have at least two weeks of alcohol and other drug abstinence before beginning the study. Exclusion criteria for our study included having cardiac arrhythmias or serious medical conditions that may affect HRV, and using medications that directly influence HRV (e.g., beta-blockers). In order to avoid possible HRV effects due to different drugs that may have confounded the goals of the parent study, we also excluded participants that had past-year active substance use disorder related to drugs other than alcohol.

Participants were invited to Massachusetts General Hospital for an initial session at which they completed baseline questionnaire measures and were fitted with an eMotion Faros 180 ambulatory ECG monitor, which they wore for four days. The ECG device was affixed using an elastic belt embedded with two ECG contact points worn on the chest under the pectoral muscles. Participants were encouraged to wear the device throughout the day. Based on accepted data collection guidelines (Kwon et al., 2018), ECG was sampled at 250 Hz. The Faros device also included a three-axial accelerometer to record movement. We sampled accelerometry at 25 Hz (Small et al., 2021).

Subjective stress measurement was conducted using ecological momentary assessment (EMA; Shiffman et al., 2008) in the form of mobile phone questionnaires administered using the Metricwire smartphone application (Metricwire, 2023) installed on each participant’s personal smartphone. Over the four-day ambulatory monitoring period, participants completed three random surveys each day, prompting them to report their in-the-moment emotional states on a scale of 0–10 (e.g., stress, nervousness; sadness; happiness). Participants were also told to self-initiate a survey when experiencing high levels of stress, alcohol craving, or high risk for alcohol use. This research protocol was approved by the Mass General Brigham institutional review board (IRB# 2016P001178). Study data and code can be provided upon reasonable request and institutional review board approval.

The clinical sample was 61.90% male and 38.10% female (N = 42), with ages ranging from 18 to 65 (M = 41.59, SD = 12.60). The sample was 73.81% White/European American, 19.05% Black/African American, 4.76% Asian, and 2.38% Other race/Mixed race. Three participants were lost to follow-up and physiology data were missing for two participants (one due to a lost ECG device, and one due to ECG device failure).

Comparator Data Collection (WESAD)

As a comparator, we selected a well-known, publicly available affective dataset, the Wearable Stress and Affect Detection Dataset (WESAD; Schmidt et al., 2018a), which contains physiological data from 15 healthy university graduate students (80% male, 20% female) with an average age of 27.5 (SD = 2.4) years. In order to generate physiological data under different affective conditions, participants completed a laboratory-based cue-reactivity paradigm. A short, guided meditation was then employed to return participants to a neutral affective state. Participants were asked to avoid tobacco or coffee one hour prior to the test and the exclusion criteria for this study included: being pregnant, heavy smoking, having a psychological disorder, or chronic cardiovascular disease.

The WESAD cue-reactivity protocol included a resting baseline assessment, as well as additional tasks to elicit amusement, stress, and relaxation. The Trier Social Stress Test (Kirschbaum et al., 1993) was utilized in the WESAD study as the stress task. In the Trier stress task, participants were asked to deliver a five-minute speech focusing on the strengths and weaknesses of their personal traits in front of a three-person panel. Participants then had three minutes to prepare their speech but were not allowed to use their notes during the presentation. After the speech, participants were asked to count from 2023 to zero in steps of 17, starting over when making a mistake. Participants were given five minutes to complete each task.

Throughout all the test conditions, participants’ physiological data was recorded using a RespiBAN Professional biosensor strap worn around the chest that recorded ECG, electrodermal activity (i.e., skin conductance), electromyography, respiration, accelerometry, and body temperature with sampling at 700 Hz. Participants also wore an Empatica E4 wrist band (McCarthy et al., 2016) which recorded photoplethysmography blood volume pulse (BVP) at 64 Hz.

In addition to physiological data, subjective stress was also measured after each phase of the WESAD laboratory protocol using the Self-Assessment Manakin (SAM).

Analyses and Algorithm Development

Data Segmentation and Feature Extraction

We extracted a five-minute segment of ECG recording corresponding to each EMA stress survey. To avoid capturing potential psychophysiological reactivity associated with survey completion, the five minutes of ECG recording immediately prior to survey initiation was utilized (rather than the five minutes proceeding survey initiation). The ECG waveform from each five-minute epoch was manually inspected for issues such as missed, misplaced, or ectopic beats using an ECG analysis software application (Kubios, 2021), and irregularities were manually corrected. If necessary, minimal filtration was applied. On average, there were 8.13 (SD = 2.50; range = 2–12) EMA survey epochs per participant for which ECG recordings were available.

We then extracted a range of mathematical features from each 5-minute segment of data. The extracted features included HR, the standard deviation of normal-normal intervals (SDNN), root mean squared of successive differences (RMSSD), and the percent of adjacent normal-normal intervals differing by greater than 50 milliseconds (pNN50). Frequency domain features were also calculated, including high frequency HRV (HF HRV; 0.15–0.4 Hz) and low frequency HRV (LF HRV; 0.04–0.15 Hz).

For the WESAD dataset the physiological data was segmented and labelled based on each phase of the lab protocol (resting baseline, cognitive stress, and meditation (details can be found in Schmidt et al., 2018b). Several HRV indices were calculated, including mean RR-intervals, SDNN, RMSSD, pNN50, HF HRV, and LF HRV.

Initital cStress Model Testing

Initially, we sought to test the performance of the cStress model with our clinical data. The cStress model is detailed in Hovsepian et al. (2015), but in summary, the data were first annotated to mark each R peak. R-R intervals, the difference between consecutive R peaks, were normalized and used to compute aggregated HR and HRV features for every one-minute interval: R-R variance, quartile deviation, LF HRV, medium frequency HRV, HF HRV, LF/HF HRV ratio, and the mean, median, 20 th percentile, and 80 th percentile of HR. With the features extracted, the cStress model was applied to predict stress levels for each participant using code provided by Kumar and colleagues through the MD2 K Center of Excellence (Kumar et al., 2017).

Data Normalization

For clinical/AUD participants, the physiological data was labelled using the self-reported stress scores from the EMA surveys (see Fig. 1).

Fig. 1.

Box plot showing ambulatory ecological momentary assessment (EMA) measured stress responses for each participant, including median (orange line) and mean (green triangle) EMA scores with whiskers indicating minimum/maximum values

In order to normalize the self-reported level of stress across the clinical participants, we normalized EMA scores using the formula below:

$$\mathit{\text{Score}}=\frac{\mathit{\text{EMA}}-\mathit{\text{EM}}{A}_{\mathit{\text{min}}}}{\mathit{\text{EM}}{A}_{\mathit{\text{max}}}-\mathit{\text{EM}}{A}_{\mathit{\text{min}}}}$$ (1)

This normalization produced a stress score ranging from 0 to 1 for all participants. The resulting normalized EMA stress scores are shown in Fig. 2.

Fig. 2.

Box plot showing normalized ambulatory ecological momentary assessment (EMA) measured stress for each participant, including median (orange line) and mean (green triangle) EMA scores with whiskers indicating minimum/maximum values

For the purpose of assigning stress labels to each data point, a threshold of stress score = 0.2 was chosen to differentiate stress vs. non-stress states, which produced approximately 60% unstressed and 40% stressed data points, with a median stress score = 0.15.

To enable generalization of physiological data across multiple participants, for both the WESAD dataset and our clinical dataset, we created a normalized measure of HR by defining a heart rate frequency and using the formula below:

$$f^{*}=\frac{f-f_0}{f}$$ (2)

where f = HR/60 and f₀ is the resting heart rate frequency.

In order to reduce variability across participants, all HR and HRV statistics were also normalized using min-max scaling so that their values were restricted to the range [0, 1].

Unsupervised Learning

In order to examine the distribution of latent states in our data and identify our significant variables, we explored an unsupervised learning approach using t-distributed Stochastic Neighbor Embedding (t-SNE) two-dimensional visualization coupled with several cluster-analysis methods (spectral, Ward, BIRCH, and Gaussian mixture). As shown in Fig. 3, the resulting t-SNE visualizations enabled us to identify subtle clusters and gradients in the data, by comparing the distribution of ‘stressed’ vs ‘non-stressed’ data points.

Fig. 3.

Example showing t-Distributed Neighbor Embedding (t-SNE) to calculate the gradient path (green line) between the centroids (green dots) of each cluster (blue for non-stressed, orange for stressed)

In order to identify the most predictive features, we calculated the centroid of each data point cluster (stressed vs. non-stressed) and constructed a gradient path between the two centroids (Fig. 3). Using the t-SNE function, it was possible to calculate the variables that had the greatest differential change along this gradient. Based on this method, we identified the most predictive features, which are listed in Table 1.

Table 1.

Table showing normalized coefficients for the heart rate and heart rate variability features found to be most predictive of stress in our unsupervised learning models

Features	Normalized Coefficient
20 th percentile HR	0.982
Mean HR	0.943
80 th percentile HR	0.754
SDNN	−0.004
HF HRV	−1.321
LF/HF HRV	−1.354

HR = heart rate, SDNN = standard deviation of normal-to-normal intervals, HF HRV = high-frequency heart rate variability, LF = low-frequency heart rate variability, LF/HF = ratio of LF-HRV to HF-HRV

Supervised Learning

In order to develop a machine learning model to quantitatively predict the probability of stress in real-time, we then performed supervised learning using binary class logistic regression including the feature variables listed in Table 1. Since the sizes of the two classes labelled stressed and non-stressed were not equal, the class imbalance was taken into account during model training by adjusting class weights and data resampling. We then ran a grid search across a set of logistic regression hyperparameters to find hyperparameter values that optimized model performance. Confusion matrices and receiver operating characteristic (ROC) curves were then generated to evaluate optimized model performance.

Given the large degree of heterogeneity in the clinical data, we also explored stratifying the data into more similar demographic groups in order to improve stress prediction within each group. We stratified the data into four groups of roughly equal size based on age (younger/older; cut-point = 42.5 years) and BMI (cut-point = 25) and repeated the process of training a logistic regression model on each of the four groups.

Results

The cStress model performed poorly with our clinical sample (p >.05; see Fig. 4a), but well with the WESAD sample (p <.05; Fig. 4b).

Fig. 4.

(a) The performance of the stress detection using the cStress model was poor with our clinical data, with no significant relationship between self-reported stress derived from ecological momentary assessment (EMA) and cStress model stress probability (p >.05). (b) Conversely, the cStress model performance with WESAD data demonstrated good stress detection specificity with clear differentiation between stress probability at resting baseline and during laboratory-induced stress (p <.05)

The unsupervised learning model produced an area under the ROC curve (AUC) of 0.51 for the clinical dataset, and 0.68 for the WESAD dataset, with Fig. 5 showing t-SNE projection of the clinical and WESAD samples with clustering of stress versus non-stress. The following features were found to be most predictive of stress: (1) 20 th percentile HR, (2) mean HR, (3) 80 th percentile HR, (4) SDNN, (5) HF HRV, and (6) LF/HF HRV ratio (Table 1). These features were then utilized in our supervised learning models.

Fig. 5.

Scatterplots showing t-distributed Stochastic Neighbor Embedding (t-SNE) projection of clinical sample data (left), and Wearable Stress and Affect Detection (WESAD) data (right), with clustering of stress (orange) versus non-stress (blue). The unsupervised learning model produced an AUC (area under the ROC curve) of 0.51 for the clinical dataset, and 0.68 for the WESAD dataset

The results of the supervised logistic regression model with our unstratified clinical data and the WESAD data with restricted parameters are shown in Fig. 6. In our clinical sample, based on the ROC curve, model accuracy was AUC = 0.63 ± 0.18. For comparison, we also trained a logistic regression model using the WESAD dataset, which produced an AUC of 0.94 ± 0.03 when restricting the model to the six HR/HRV features identified in our unsupervised model and AUC = 0.95 ± 0.02 when using the full HR and HRV features provided in the WESAD dataset.

Fig. 6.

Left: Receiver operator curve (ROC) from the supervised learning model with the clinical data using restricted parameters showing area under curve (AUC) of 0.63. Right: ROC curve from supervised learning model with the Wearable Stress and Affect Detection (WESAD) data using restricted parameters showing AUC of 0.94

By stratifying the clinical data by age (above/below 42.5 years) and by BMI (above/below 25), we achieved better prediction within each demographic category, yielding the following AUC accuracies: Younger/Lower BMI = 0.65 ± 0.04, Younger/Higher BMI = 0.70 ± 0.03, Older/Lower BMI = 0.80 ± 0.04, Older/Higher BMI = 0.69 ± 0.03 (Fig. 7).

Fig. 7.

Receiver operator curves (ROC) curves for each clinical subgroup created after stratifying the clinical data by age and body-mass index (BMI). Area under the curve (AUC) accuracies for each stratum of age and BMI were: (Younger/Lower) = 0.65, (Younger/Higher) = 0.70, (Older/Lower) = 0.80, (Older/Higher) = 0.69

Discussion

The results presented here highlight how the accuracy of stress prediction models is affected by how models are trained and the homogeneity or heterogeneity of the sample. The high accuracy of the WESAD stress prediction model (AUC = 0.95) observed in this study affirms that very good stress-prediction accuracy is possible using laboratory data from a young, healthy homogenous sample. Conversely, we observed that a stress prediction model trained on a demographically diverse clinical sample of people with AUD produced relatively poor prediction accuracy, with AUC = 0.63. At the same time, our results show that if the data are properly normalized and if stratification is used to account for age and BMI, then markedly better prediction is possible in a model trained on ambulatory data with a clinical sample, with AUC accuracy ranging from 0.65 to 0.80 in various sample sub-groups (i.e., Younger/Lower BMI; Younger/Higher BMI; Older/Lower BMI; Older/Higher BMI). While higher degrees of stratification are certainly possible, a larger amount of training data would be required to enable the development of separate models for each subgroup. The tradeoff between model accuracy and practical clinical implementation of machine learning and artificial intelligence is an ongoing topic of discussion.

Although this supervised learning approach resulted in a marked improvement in stress detection accuracy, findings are notable for the disparity in stress detection in our clinical sample, which ranged a great deal between participant subgroups. While it appears that some of the heterogeneity found in clinical populations can be addressed through stratification and the use of partially-customized models, further work can be done to increase the accuracy of stress detection models across clinical populations by exploring different degrees of stratification and using a larger volume of training data.

Limitations and Future Directions

This study was not without limitations: (1) The design of this preliminary study did not allow us to tease out the unique influences of model training and testing with laboratory versus ambulatory data, and sample homogeneity/heterogeneity. Exploring the distinct effects of model training with laboratory versus ambulatory data in clinical populations will be an important next step and will be the subject of a forthcoming results from our ongoing National Institute of Alcohol Abuse and Alcoholism study K23 AA027577; (2) As a necessary first step, this study utilized foundational machine learning approaches. Newer, more sophisticated machine learning approaches employing neural nets should be explored in future studies; (3) We have focused on developing a binary stress classifier mirroring the binary classification task of the original cStress model. While a binary classifier is likely to have the greatest real-world utility, there may be value in developing a multi-tiered or continuous classifier. Future studies should explore these approaches.

Taken together, this work highlights the need for ongoing work to improve the accuracy of stress prediction algorithms in clinical samples, with real-time, wearable biosensor-based stress detection remaining an important clinical goal. This is a particularly important enterprise given clinical populations are most likely to benefit from the kinds of just-in-time interventions real-time stress detection makes possible (Eddie et al., 2021). Future studies may benefit from employing emerging machine learning approaches to address the inherent difficulty of identifying affective states in clinical samples. Future studies should also explore covariates not explored here such as accelerometry/physical activity, waist-to-hip ratio (vs. BMI), medications, and sympathovagal functioning, which may increase stress predication specificity in clinical samples.

Conclusions

We have demonstrated a stress prediction model trained on ambulatory self-report data from a clinical population and have compared its performance with laboratory data from a homogenous/healthy sample of young adults. Results indicate that the accuracy of stress prediction models is highly dependent on how the model is trained and the homogeneity of the data sample, though more work is needed to tease out the differential influences of training and testing models with laboratory versus ambulatory data, and sample homogeneity/heterogeneity. While nomothetic models generally have lower prediction accuracy than personalized ideographic models, this study demonstrates that nomothetic models trained with a minimal amount of stratification represent a practical path forward for stress detection in clinical populations. Findings show that it is possible to create relatively simple stress detection models by employing normalization across participants and using HR/HRV indices that are matched to a few basic demographic variables (i.e., age, BMI) to achieve better prediction accuracy in clinical samples.

Data Availability

Study data and code can be provided upon reasonable request and institutional review board approval.