ECG-based Emotion Prediction in Real Life: Tackling Heterogeneity Through a Hybrid Personalization Method

Abstract

Predicting people’s momentary emotions in real life remains an underexplored challenge in affective computing. Wearable electrocardiography (ECG) offers a promising tool for this purpose, given its psychological relevance to emotion and increasing affordability in recent years. However, a major obstacle lies in the substantial individual differences in the associations between emotional experiences and physiological responses, which is further amplified in real-world settings. Emerging personalization methods offer a potential solution, but their utility for real-life ECG-based emotion prediction has yet to be evaluated. To address this, the present study leverages the largest real-life ECG dataset to date (N = 363) to evaluate a hybrid personalization approach that tailors model training using a subset of the most informative participants. We compared three modeling strategies—idiographic personalization, hybrid personalization, and a general model—across binary classification tasks for positive affect (PA) and negative affect (NA). The hybrid personalization method outperformed both the idiographic personalization (PA: 76.6% vs. 71.5%; NA: 77.7% vs. 74.3%) and the general model (PA: 55.5%; NA: 49.3%), with the largest advantages observed under limited data conditions. Follow-up analyses revealed that data from individuals with higher emotional awareness contributed more to model performance. These findings highlight the utility of hybrid personalization for real-world ECG-based emotion prediction and suggest that emotional awareness may serve as a useful trait for identifying informative training data and enabling more efficient model development.

1. INTRODUCTION

Real-life emotion prediction is a well-recognized yet underexplored challenge in affective computing, despite its potential for real-world applications in health, education, and beyond. Recent advances in wearable technologies are facilitating progress toward this goal by enabling reliable and affordable monitoring of physiological signals in daily life [1] Electrocardiography (ECG) has emerged as one of the most widely used modalities for real-life emotion prediction, owing to its psychological relevance to emotion processes [1, 2] and feasibility for ambulatory measurement in daily life [3]. ECG signals can be used to derive a rich set of heart rate variability (HRV) metrics that indicate the physiological responses of the sympathetic and/or parasympathetic branches of the autonomic nervous system (ANS) [4, 5], which reflect core physiological components of emotion response systems [6]. In recent years, advances in wearable ECG devices have further fueled interest in leveraging ECG for emotion prediction in the context of daily life [3, 7].

Despite its promise, predicting emotions in real life is considerably more challenging than in laboratory settings. A central challenge lies in the individual differences in the associations between emotional experience and physiological responses [8]—a complexity that is also present in laboratory settings but becomes further amplified in real-world contexts for two main reasons. First, while emotional stimuli used in laboratory studies could be the same for all the participants, the emotional events people encounter in their daily lives are highly diverse. Even when reporting the same emotion scores, the specific emotional stimuli can vary markedly across individuals (e.g., a negative emotion rated as “10” might result from failing an exam for one person, but from the loss of a family member for another), leading to greater variability in the mapping between emotions and physiological responses. Second, individuals differ in both their tendency and ability to understand and label their own emotional states (i.e., emotional awareness) [9–11]. While laboratory-based studies can use standardized stimuli (i.e., a film clip validated to induce sadness) to pre-define emotional labels without relying on participants’ self-reports of their emotional states, it is nearly impossible to expose people to such controlled stimuli in their real lives. As a result, real-world affective computing typically relies on methods such as ecological momentary assessment (EMA), which requires individuals to report their emotional responses to naturally occurring events [12]. Differences in emotional awareness could introduce additional noise into the associations between physiology and self-reported emotion, further complicating the heterogeneity across individuals. Such complexity has long been acknowledged in psychological research [13, 14] but hardly addressed in affective computing studies.

While the sources of individual differences in emotional responding are not always explicitly addressed in affective computing research, numerous personalization strategies have nonetheless been proposed to manage their effects [8], which can be further grouped into the model-level strategies and data-level strategies [15]. Model-level strategies employ advanced machine-learning algorithms to construct models that can minimize the impact of the inherent heterogeneity in the data. For instance, machine-learning methods such as domain adaptation [16, 17], fine-tuning [18], and hypergraph learning [19] have been leveraged to accommodate the inherent heterogeneity in ECG data and thereby enhance model generalizability across individuals. Complementary to model-level strategies, data-level strategies address the source of heterogeneity at the data selection stage, proactively tailoring the training data to better match the target individual. One extreme version of this strategy is to train models exclusively on data from the target person [20, 21] While this approach theoretically eliminates any performance reduction due to inter-individual data variance [22], it may not be feasible for real-world applications due to the difficulty of collecting sufficient psychophysiological data from each person’s real life. Moreover, its potential performance advantages over one-size-fits-all models, as well as its sensitivity to training data size, have yet to be empirically evaluated in real-life settings.

A more feasible alternative could be a hybrid personalization approach using data from a selectively chosen subset of individuals most relevant to the target individual, which offers a middle ground between extreme personalization and broad generalizability [15, 23]. However, several key questions remain unresolved: How should such hybrid personalization models be constructed? And to what extent does their effectiveness depend on individual traits, such as emotional awareness? Progress in this line of work has been constrained in part by the lack of large-scale real-life ECG datasets for affective computing purposes [24]. To address this, the present study leverages what is, to our knowledge, the largest real-life ECG dataset (N = 363) with both emotion labels and trait-level information. We pursue three primary aims: (1) to evaluate how individual differences impact ECG-based emotion prediction in real-life settings; (2) to examine whether the hybrid personalization approach can mitigate these effects; and (3) to explore whether individual differences in emotional awareness contribute to model effectiveness and inform more efficient model training strategies.

2. Related Work

2.1. Emotion Prediction with ECG and its Real-life Implementation

The theoretical foundation of ECG-based emotion prediction is that emotional experiences are accompanied by autonomic nervous system (ANS) activities [25], which can be reflected by cardiovascular responses and captured by ECG measurement [4, 5]. This foundation has motivated numerous studies to extract ECG features that are well-established in psychophysiological research as sensitive biomarkers of ANS activity, using them as input for affective computing models. The most commonly used ECG-derived features are heart rate and various heart rate variability (HRV) metrics, including time-domain features such as root mean square of successive differences (RMSSD) and frequency-domain features such as high- and low-frequency power. These metrics are generally believed to reflect the activation of the sympathetic and/or parasympathetic nervous system. For example, high-frequency power is typically linked to parasympathetic activation, whereas low-frequency power is associated with both sympathetic and parasympathetic activation [26].

In addition to such predefined features, some studies have employed data-driven methods to extract oscillatory or frequency-related features from raw ECG signals. Techniques such as empirical mode decomposition [2, 27, 28] and wavelet transform [29, 30] have been used to derive features that may not have a clear psychological interpretation. Other studies have bypassed the feature extraction step altogether, adopting end-to-end deep learning approaches that learn emotion-relevant patterns directly from raw ECG signals [31] or from heartbeat interval series [32]. While existing research has not revealed clear performance differences between these approaches, HRV metrics remain the most widely used, appearing in over 60% of ECG-based affective computing studies [33]. This prevalence may be partly due to the added benefit of psychological interpretability, which enhances model explainability.

Another key theoretical consideration in affective computing is the choice of emotion model, which determines how emotional labels are defined and operationalized. A commonly adopted framework is the dimensional model of emotion, which includes several possible dimensions for representing affective space (e.g., [34, 35]). Among these, the most widely used one in affective computing is the arousal-valence model, which originates from the circumplex model of affect and posits that valence and arousal are the two fundamental, orthogonal dimensions of emotion [35]. Within this framework, emotions are typically categorized along each dimension (e.g., positive vs. negative valence, and high vs. low arousal). An alternative and also commonly used framework is the discrete emotion model, often grounded in Ekman’s theory of basic emotions [36], which assumes that specific emotions such as anger, fear, and sadness have distinct neurophysiological signatures. The arousal-valence dimensional model is more prevalent in ECG-based emotion prediction (see the summary of model usage in [33]), likely due in part to ongoing theoretical debates about whether discrete emotions indeed possess unique neurophysiological patterns [13, 25, 37]. From a practical standpoint, the limited availability of standardized stimuli that can reliably and specifically elicit discrete emotions may further constrain the application of the discrete emotion model in affective computing research.

Nonetheless, the arousal-valence dimensional model is not without limitations. It conceptualizes positive and negative emotions as opposite ends of a bipolar dimension, thereby treating them as mutually exclusive (Fig. 1a). However, research on mixed emotions [38, 39] suggests that individuals can experience both positive and negative emotions simultaneously (e.g., feeling both happy and sad at the graduation ceremony). EMA studies further support the prevalence of such mixed emotional states in daily life, with estimates indicating that about 10% to 50% of sampled experiences involve a blend of both positive and negative emotions [40–42]. Moreover, while positive and negative emotions tend to be strongly negatively correlated at the between-person level, their correlation could be moderate at the within-individual level [42]. In fact, there has been a longstanding theoretical debate regarding whether valence should be considered bipolar or bivariate (i.e., treating negative affect and positive affect as two separate dimensions; see more discussions in [34, 43–45]). In the context of real-world affective computing, modeling positive and negative emotions as two separate dimensions [46, 47] (Fig. 1b) may provide a more nuanced and ecologically valid framework.

Fig. 1.

Conceptual model of the bipolar vs. bivariate model of emotional valence

It is also important to note that most ECG-based affective computing studies have been conducted in controlled laboratory environments [1, 24, 33]. To the best of our knowledge, only one existing study has collected real-life ECG data with emotion labels to conduct affective computing (e.g., [48]), but it only involved a very small sample (N = 6). A more recent project has reported ongoing data collection aiming to build a real-life ECG dataset with 250 participants for affective computing purposes [49]; however, no affective computing results based on that dataset have been published to date. Although existing literature does not yet allow for a direct comparison between emotion prediction performance in laboratory versus real-life settings, one study has indirectly addressed this by comparing ECG-based emotion prediction accuracies under well-controlled lab conditions vs. in scenarios involving physical activity such as walking. The results showed that while models achieved over 50% accuracy in a four-class classification task under well-controlled laboratory conditions, the performance dropped below 20% when applied to data collected during physical movement [50]. These findings underscore the substantial challenges posed by real-world settings and indicate that models trained on laboratory data may not generalize well to everyday life, emphasizing the need for empirical validation of ECG-based emotion prediction with real-life data.

2.2. Personalization Methods for Emotion Prediction

Given the heightened data heterogeneity in people’s real lives, personalized modeling emerges as a promising direction with the potential to improve the performance of real-life emotion prediction. As previously discussed, personalization can be achieved through various strategies (see more detailed discussion in [15] and [8]). The current study focuses specifically on data-level personalization, aiming to understand how training data can be strategically selected to optimize model performance¹.

Accumulating studies have found that fully individualized models, which were exclusively trained on data from each individual, outperformed traditional general models trained on data from a broad sample [51–53]. However, such extreme personalized modeling requires a substantial amount of data from each individual, which is often impractical in real-world contexts where data collection is more resource-intensive [15, 54]. To address this limitation, several studies have adopted compromise strategies that involve training models on a subset or cluster of participants selected for their relevance to the target individual [51, 52, 55, 56]. The criteria for selecting these subsets vary considerably. Some studies set up clusters using variables that may be indirectly linked to the prediction target. For instance, Kathan and colleagues [56] trained separate models for males and females to predict depression levels; Adler and colleagues [55] employed a K-nearest-neighbors approach based on participants’ behavioral profiles to predict stress. Alternatively, some studies identify clusters directly using model-relevant features or outcome variables. For example, Tervonen and colleagues [52] grouped participants based on their physiological features; Can and colleagues [51] grouped participants by their baseline stress levels and trained models within each cluster to predict ambulatory stress states.

Despite variations in clustering criteria, these approaches share a common objective: to identify the most transferable data from other individuals in order to tailor a model for a given participant. In line with this goal, a more direct strategy could be to use a small portion of the target participant’s data to assess the transferability of models trained on others and select the most informative data accordingly. This approach has shown promise in improving cross-individual generalization in domain adaptation modeling (e.g., [57]). However, to our knowledge, it has not yet been applied to real-life emotion prediction tasks. Moreover, several key questions remain unanswered: How much data is sufficient? Which data are most “informative”? Resolving these questions is crucial for guiding efficient model design in practical applications.

2.3. Emotional Awareness and Its Role in Affective Computing

Although obtaining accurate emotion labels has been widely acknowledged as a key challenge in real-world affective computing [1, 15, 33], few (if any) empirical studies have explicitly investigated how the accuracy of emotion labels influences model performance. Clarifying this issue may also help address the question raised in the previous section: what types of data are most informative for model training? Yet, in affective computing, what constitutes an “accurate” emotion label remains an open question due to the subjective nature of emotional experience, where a “gold-standard criterion” is lacking. Insights from psychological research on emotional awareness may offer a valuable perspective in this regard.

Emotional awareness refers to an individual’s tendency to attend to and the capacity to understand their own emotional states [9–11, 58]. Emotional awareness is conceptualized as the expertise to identify and label one’s own experiential experiences [59] and plays an integral role in the use of affect labeling in everyday life [60]. The most commonly used tools to assess emotional awareness are the Levels of Emotional Awareness Scale (LEAS; [61]), which is a performance-based measure, and the Clarity and Attention subscales of the Trait Meta-Mood Scales (TMMS; [62]), which is a self-report measure.

Although prior literature suggests that emotional awareness may enhance the quality of emotion labels and, in turn, improve affective computing performance, this hypothesis has yet to be empirically tested in real-life settings. The present study will take an initial step toward addressing this gap, examining whether data from individuals with greater emotional awareness makes more contributions to model performance in real-life ECG-based emotion prediction.

3. Methods

3.1. Participants

As part of a larger study, 363 participants (ages 18–65; M = 34.63, SD = 13.67; 221 females, 142 males) recruited in Australia were included in the present analysis (Table 1). Exclusion criteria were a current manic episode, psychosis, or substance use disorder; regular use of medications known to influence HRV (e.g., ACE inhibitors, beta-blockers, diuretics, tricyclics, hypertension medications); pregnancy at the time of enrollment; and known cardiovascular disease. The larger project targeted at least 50% of the sample with elevated rumination and/or worry, to yield a diverse community sample spanning a broad range of distress symptoms (actual % = 54.24% with elevated rates of worry or rumination). This symptom heterogeneity was further supported by baseline assessment using Inventory of Depression and Anxiety Symptoms (IDAS), which indicated that 27.82% of participants met the diagnostic threshold for any internalizing disorders, 7.71% for panic disorders, 9.37% for social phobia, and 6.89% for post-traumatic stress disorder based on IDAS dysphoria, panic, social anxiety, and traumatic intrusions scores, respectively [63]. In addition, 23.20% of participants self-reported a history of receiving psychotherapy, and 21.49% reported a history of treatment with psychiatric medication. The study procedure was approved by the local Ethics Committee of the University of Western Australia. Informed consent was obtained from all the participants.

Table 1.

Demographic characteristics of participants

	N = 363
Age, year	34.63 ± 13.67
Female/male, n	221/142
* Race, %
White	65.56
Asian	28.37
Aboriginal and/or Torres Strait Islander	1.10
Native Hawaiian or Pacific Islander	1.10
Black or African	1.93
Hispanic or Latina/o, %	2.20
Highest Education Level, %
Some high school	4.41
High school diploma or equivalent	16.80
Some university	21.21
University degree	27.82
Some postgraduate	9.92
Postgraduate degree	19.83
* Marital Status, %
Single	52.62
Cohabitating	14.05
Married	28.93
Divorced	5.79
Household Income (AUD), %
$10,000–$20,000	27.27
$20,000–$40,000	16.25
$40,000–$60,000	8.54
$60,000–$80,000	9.09
$80,000–$100,000	10.74
More than $100,000	26.72
* Employment, %
Student full-time	36.09
Employed full-time	18.73
Unemployed	21.21
Employed part-time	32.51
Student part-time	7.16
Other	9.37

Note.

^*Multiple selections are allowed.

3.2. Procedure

All participants completed a 10-day ecological momentary assessment (EMA) protocol (Fig. 2), combined with continuous ECG monitoring during their waking hours using the Movisens EcgMove4 sensor (Movisens GmbH, Germany). Waking hours were self-defined by participants as a 12-hour window selected between their typical wake-up and bedtime (e.g., 8:00 a.m. to 8:00 p.m.). The EMA was conducted via the MovisensXS platform[64]. Each day, participants received eight prompts to answer a series of questions regarding their current emotional states, including an 8-item version of the Positive and Negative Affect Schedule (PANAS), which assessed positive affect (PA) and negative affect (NA) on 5-point scales. On average, each participant completed 57 prompts over the 10-day protocol.

Fig. 2.

Procedure for the 10-day EMA and ECG assessment

Before the 10-day ambulatory assessment, participants completed a battery of baseline questionnaires. For the purposes of the current study, only the Levels of Emotional Awareness Scale (LEAS) and Trait Meta-Mood Scale (TMMS) were analyzed. For LEAS, we calculated participants’ awareness level of their own emotions as their performance-based emotional awareness score. For TMMS, the Clarity and Attention subscales were summed to derive the self-reported emotional awareness score.

3.3. Data processing

ECG signals were processed using Movisens DataAnalyzer (version 1.15.1) software. ECG segmented into one-minute epochs for preprocessing, consistent with our prior ambulatory ECG validation study using the same device and preprocessing parameters, which demonstrated high reliability and validity [65]. For each EMA prompt, we used the 30-minute period preceding the prompt in the main analyses. We also varied the window length from 15 to 60 min and report the corresponding results in the Supplementary materials. From each epoch, six ECG metrics were computed: heart rate, standard deviation of normal-to-normal inter-beat intervals (SDNN), root mean square of successive differences (RMSSD), high-frequency (HF) power, low-frequency (LF) power, and the LF/HF ratio. As part of standardized preprocessing, Movisens DataAnalyzer (version 1.15.1) automatically assesses raw ECG signal quality and flags each epoch as valid or invalid. The valid data rate was calculated as the proportion of valid epochs among all epochs, and the valid data rate reached 73.96% across all participants. Epochs identified as invalid were excluded from subsequent analyses.

Negative affect (NA) and positive affect (PA) scores at each prompt were calculated as the sum of four negative emotion items (sad, anxious, upset, irritable) and four positive emotion items (strong, interested, excited, active), respectively. To build the binary classification model, high and low NA (or PA) labels were assigned based on the score from the EMA prompt closest in time to the ECG segment, using a threshold defined by the median NA (or PA) level across all participants.

3.4. Emotion prediction models

We developed three methods for daily emotion prediction based on ECG features, and their conceptual models are illustrated in Fig. 3.

Fig. 3.

The conceptual illustration of the three methods

1. Idiographic personalization: This approach involved training and testing the model exclusively on data from the same individual. The data were split as follows: the first one-third (1/3) of the participant’s data was used for training, the next one-sixth (1/6) for validation, and the final one-half (1/2) for testing.

2. Hybrid personalization: In this method, a model was trained for each participant, and the best-performing model was selected for the testing participant. Specifically, a model was trained using the data from each training participant, while the first one-half (1/2) of the testing participant’s data was used to identify the best K models. K was set as 20 throughout the main experiments. These selected models were then combined as the hybrid model and tested on the second half (1/2) of the testing participant’s data. Importantly, this second-half data was never used for model training, model selection, or any tuning decisions, thereby preventing data leakage. During testing, we aggregated the selected base models by averaging their predicted class probabilities. Specifically, each model produced a softmax probability vector, which was multiplied by its selection weight (equal to the accuracy on the first one-half of the testing participant), and the weighted probabilities were then averaged across the K models to obtain the final prediction. Conceptually, this hybrid approach models a practical “cold-start + calibration” scenario, in which a system collects a brief initial period of labeled data from a new user (e.g., the first few days) to identify which existing models generalize best to that user and then deploys the selected ensemble on subsequent data.

3. General model: This method trained a single model using data from all participants. The first one-third (1/3) of each participant’s data was used for training, the next one-sixth (1/6) for validation, and the final one-half (1/2) for testing.

For all three methods, a multilayer perceptron (MLP) with two hidden layers was employed. The two hidden layers consisted of 50 and 20 units, respectively. The learning rate was set to 0.001, and the Adam optimizer was used for optimization. Binary classification was performed using a cross-entropy loss function. The batch size was set as 128. The model was trained for 10 epochs in the idiographic personalization and hybrid personalization methods and for 100 epochs in the general method. The model that achieved the best performance on the validation data during training was subsequently used for testing. Additionally, we also conducted analyses with alternative MLP hyperparameter configurations (see details in Table S2) to more comprehensively assess whether the main conclusion holds under different parameter choices. Taken together, the full prediction pipeline is summarized in Table 2.

Table 2.

Summary of emotion prediction pipeline

Algorithm 1: Preprocessing and emotion prediction pipeline
Inputs: Raw ECG signals, EMA reports, top-K
1. Preprocessing
for each EMA prompt do
extract 30-min ECG before the prompt;
segment ECG into 1-min epochs;
discard invalid epochs;
extract HRV features (HR, SDNN, RMSSD, HF power, LF power, and LF/HF ratio);
assign binary NA/PA label from EMA by global median split;
end for
2.1 Hybrid personalization model
for each target participant t do
split target data into validation half and test half;
for each non-target participant s do
train one source MLP model Ms on participant s;
evaluate Ms on target validation half;
store validation accuracy Accs;
end for
select the top-K source models with highest Accs, denoted by TK;
for each target test sample x do
compute weighted average of softmax outputs:
$p_{\mathit{\text{final}}}=\frac{{\sum }_{s\in T_K}Ac{c}_s\cdot M_s(x)}{K}$
assign predicted label $\widehat{y}=\text{argmax}(p_{\mathit{\text{final}}})$
end for
end for
2.2 Idiographic personalization model
for each target participant t do
split target data into training (1/3), validation (1/6), and testing sets (1/2);
train one MLP model on the target participant’s training set;
select the best model based on validation performance;
generate predictions on the target participant’s testing set;
end for
2.3 General model
split the data of each participant into training (1/3), validation (1/6), and testing sets (1/2);
combine data from all participants;
train one MLP model on the pooled training set;
select the best model based on validation performance;
generate predictions on the pooled testing set.
Return: predicted labels for all samples in the testing set

We further validated the idiographic and the hybrid personalization methods in a predicting-the-following-day setting, which used the first N days of the testing participant for training or validation and the N+1 day for testing. For the idiographic method, we used the first N-1 days for training and the N-th day for validation (When N=1, the first half of the data was used for training and the second half was used for validation). For the hybrid method, we used the first N days for validation, i.e., we selected the best K models trained on other participants based on the validation data. To further assess whether the observed model performance differences were consistent across different machine learning algorithms, we also constructed models using Support Vector Machine (SVM), K-Nearest Neighbor (KNN), and Naïve Bayes (NB) algorithms, in addition to the MLP approach.

3.5. Performance analysis of the hybrid personalization model

To gain a deeper understanding of why the hybrid personalization model outperformed other approaches and to clarify the conditions under which this method achieves optimal performance, we conducted two follow-up performance analyses. First, we tested whether its performance was influenced by the number of individual classifiers included. To examine this, we varied the number of individual classifiers from 1 to 25 and assessed how model performance changed as a function of data size. Second, we evaluated whether the hybrid personalization model’s effectiveness depended on the emotional awareness of the individuals whose models were included. Specifically, for each individual classifier, we computed its average accuracy in predicting other participants’ PA or NA as an index of generalizability. To evaluate whether this generalizability was consistent across emotional valence, we calculated Pearson’s correlation between the two generalizability scores. Then, we averaged PA and NA generalizability scores to obtain an overall generalizability index. Based on this index, we identified the top 5% and bottom 5% of individuals and compared their emotional awareness levels using an independent samples t-test.

3.6. Feature attribution analysis of the hybrid personalization model

To interpret which ECG features drove the predictions for negative affect (NA) and positive affect (PA) in the hybrid models, we performed a post-hoc feature attribution analysis using SHAP (SHapley Additive exPlanations) values [66, 67]. SHAP values quantify, for each test instance, how much each feature contributes to the model output by averaging the feature’s marginal effect over all possible feature subsets. Positive (negative) values indicate that the feature increases (decreases) the model’s decision score toward the target class. SHAP values were computed for each target participant with respect to the hybrid personalization ensemble’s decision score, defined as the log-odds of the ensemble-predicted probability for the target class. For each participant, we used the first half of that participant’s data (the same segment used for model selection in the hybrid method) as the background set, and computed SHAP values on the second half of that participant’s data (the held-out test segment). For any feature subset, we evaluated the model on hybrid inputs where features in the subset were fixed to the test instance values, while features not in the subset were replaced by the corresponding values from a background sample. The subset output was obtained by averaging predictions over all background samples. We computed exact SHAP values by enumerating all 2⁶ feature subsets. The global importance was summarized as the mean absolute SHAP value per feature across all test instances for each participant, followed by aggregation across participants.

4. Results

NA and PA were moderately negatively correlated at the between-individual level (r = −0.258, p < 0.001; Fig. 4a). At the within-individual level, substantial variability was observed across participants, with an average correlation of r = −0.255 ± 0.261 (see three representative examples in Fig. 4b–d). These findings suggest that NA and PA function as relatively independent dimensions in daily life, supporting our decision to model them as separate prediction targets in subsequent analyses.

Fig. 4.

Between- and within-individual associations between negative affect (NA) and positive affect (PA).

(a) Scatterplot showing the between-individual association based on participants’ average NA and PA across all prompts. Each dot represents one participant. (b-d) Three examples showing participants with a positive within-individual NA–PA correlation, negative within-individual NA–PA correlation, and no significant within-individual correlation between NA and PA, respectively. Each dot represents one EMA report.

The hybrid personalization method achieved prediction performance of 77.7±1.3% for binary classification of NA and 76.6±1.2% for binary classification of PA (Table 3), which were significantly better than the idiographic method (74.3±1.3% for NA, t(362) = 3.27, p < 0.01, and 71.5±1.3% for PA, t(362) = 5.29, p < 0.001) and general method (49.3±1.2% for NA, t(362) = 15.40, p < 0.001, and 55.5±1.0% for PA, t(362) = 13.42, p < 0.001). We also evaluated the hybrid model using alternative window lengths (15, 45, and 60 min), and the results remained stable (Table S1). In addition, across a range of MLP hyperparameter configurations, the hybrid model consistently outperformed both the idiographic and general models (Table S2).

Table 3.

Prediction accuracies of the three methods (%)

	Idiographic	Hybrid	General
NA	74.3±1.3	77.7±1.3	49.3±1.2
PA	71.5±1.3	76.6±1.2	55.5±1.0

For the predicting-the-next-day setting, the hybrid personalization method consistently outperformed the idiographic personalization method (Fig. 5). The effect sizes were larger for training and validation days of only 1 day (Cohen’s d = 0.29 and 0.30 for NA and PA, respectively) or 2 days (Cohen’s d = 0.35 and 0.20 for NA and PA, respectively). The hybrid method achieved an accuracy of over 70% with only 1 day of validation data for both NA and PA, outperforming the idiographic method by approximately 16%. The results indicated that the hybrid personalization method showed more advantages when only limited training data were available. Similar patterns were observed when the models were constructed using SVM, KNN, and NB algorithms (see detailed results in Appendix Fig. S1).

Fig. 5.

Model performance when trained with limited data

Note: ** p < 0.01, *** p < 0.001; FDR corrected.

The performance of hybrid personalization models appeared largely unaffected by the individual classifiers selected in the hybrid model (Fig. 6). For NA, accuracy remained consistently around 78% regardless of the number of individual classifiers included. For PA, a slight increase was observed (from 75% to over 76%) when the number of individual classifiers included increased from 1 to 5, after which performance stabilized.

Fig. 6.

Hybrid personalization performance by the number of individual classifiers included. The shaded region indicates the 95% confidence interval.

A significant positive correlation (r = 0.36, p < 0.001) was observed between individual classifiers’ PA and NA generalizability scores. Particularly, individuals whose models demonstrated the highest generalizability for PA also tended to show high generalizability for NA. Participants whose classifiers ranked in the top 5% for predicting others’ NA or PA exhibited significantly higher self-reported emotional awareness (i.e., TMMS score) than those in the bottom 5% (p < 0.01 for NA; p = 0.04 for PA; Fig. 7)². The top 5% of participants also showed slightly higher performance-based emotional awareness (i.e., LEAS score) compared to the bottom 5%, but the differences were not statistically significant (ps > 0.05 for NA and PA).

Fig. 7.

Self-reported emotional awareness levels of participants ranking top 5% and bottom 5% in their classifier’s generalizability.

SHAP-based feature attribution analysis (Fig. 8) showed that the most important features (ranked by mean absolute SHAP value) for NA prediction were HF power, LF power, and HR (mean absolute SHAP value = 0.12, 0.09, and 0.04, respectively). For PA prediction, the top contributing features were HR, HF power, and LF power (mean absolute SHAP value = 0.17, 0.14, and 0.11, respectively).

Fig. 8.

Mean absolute SHAP values of each feature in the hybrid models.

5. Discussion

Predicting naturally occurring emotions from real-life physiological signals is a critical step toward making affective computing viable for real-world applications. One of the major challenges lies in the substantial cross-individual heterogeneity, which is further exacerbated in real-life settings. The present study introduced a hybrid personalization approach to address the heterogeneity challenge in real-life ECG-based emotion prediction. Using the largest available real-life ECG dataset with emotion labels, we found that the hybrid method outperformed both idiographic personalization (PA: 76.6% vs. 71.5%; NA: 77.7% vs. 74.3%) and the general model (PA: 55.5%; NA: 49.3%), with the most pronounced advantages observed under conditions of limited training data. Follow-up analyses further revealed that models trained on data from individuals with the highest emotional awareness (top 5%) exhibited significantly greater generalizability than those trained on individuals with the lowest emotional awareness (bottom 5%), for both PA and NA predictions.

This study provides the first empirical demonstration of the extent to which individual differences in real-life contexts affect the performance of emotion recognition models. When comparing personalized models with general models, we found that the latter exhibited approximately 25% lower average accuracy, indicating that the advantages of more training data do not necessarily outweigh the negative impact of cross-individual heterogeneity. In contrast, personalized modeling offers an effective strategy for mitigating this issue. Between the two personalized approaches, the hybrid model consistently outperformed the idiographic model. With five days of training data, the hybrid method achieved 5.1% and 3.4% higher accuracy for PA and NA, respectively. This advantage became even more pronounced under limited training conditions. For instance, with only one day of training data, the hybrid model outperformed the idiographic model by approximately 16%. These findings suggest that the hybrid approach strikes a more effective balance between data size and the inherent individual differences. One likely reason it outperformed the other approaches is that real-life emotion prediction requires balancing two competing demands: reducing cross-individual heterogeneity while retaining sufficient data to build a reliable model. In contrast, the general model may be more vulnerable to heterogeneous pooled data, whereas the idiographic model minimizes heterogeneity but is constrained by the limited data available from a single individual. This advantage makes the hybrid approach particularly well suited for real-world, low-data settings, where data-collection burden and user compliance are often the primary barriers to deployment. This is especially relevant when daily-life affect monitoring is intended to support clinical interventions or wellness-oriented applications, as target users may include clinical or subclinical populations for whom sustained self-reporting and long-term sensor adherence can be particularly challenging. In these contexts, reducing the number of days required to train a usable model may improve both feasibility and scalability. In addition, shortening the onboarding time can provide earlier, more tangible feedback, which may in turn enhance user engagement and sustained adherence.

We found that the performance of the hybrid model was relatively unaffected by the number of individual classifiers included in the ensemble, suggesting that overall model performance was largely driven by a small subset of highly informative individual classifiers. This highlights the importance of examining individual traits to better understand what contributes to an informative classifier. Specifically, classifiers trained on individuals with higher self-reported emotional awareness exhibited better generalizability. This is likely because these individuals are more accurate in identifying and reporting their emotions, thereby providing higher-quality labels. It is also worth noting that although participants with the most informative classifiers also exhibited slightly higher performance-based emotional awareness than those with less informative classifiers, this difference was not statistically significant. Further analysis showed that self-reported emotional awareness (TMMS scores) and performance-based emotional awareness (LEAS scores) were only weakly correlated (r = 0.18, p < .001). This is consistent with prior studies (e.g., [68]) and suggests that these two measures may capture partially distinct aspects of emotional awareness. One possible interpretation is that LEAS and TMMS differ not only in measurement format, but also in the response processes they engage. LEAS is a performance-based measure that estimates emotional awareness by coding how specifically and complexly individuals describe emotional experiences in structured scenarios, whereas TMMS assesses individuals’ self-perceptions of their emotional abilities (e.g., attention to and clarity of feelings). In this sense, LEAS may better reflect emotion conceptualization and labeling performance under task constraints, whereas TMMS may more strongly reflect perceived typical emotional functioning. From this perspective, divergence between TMMS- and LEAS-related findings should not necessarily be interpreted as inconsistency. Rather, the two measures may be differentially related to outcomes depending on which aspect of emotional awareness is most relevant in the specific context. Specifically, because our EMA labels were based on participants’ self-reported emotional experiences in daily life, TMMS may align more closely with this self-evaluative reporting process than LEAS. At the same time, LEAS may capture a related but distinct capacity that does not translate directly to EMA-based label quality. This interpretation is also consistent with evidence from related emotional awareness research showing weak overlap between self-report and performance-based measures [69]. Moreover, emotional awareness is only one of many factors that may influence reporting accuracy. For instance, people’s social desirability level may also play a role, resulting in underreported negative emotions [70]. Future studies could consider assessing such traits during data collection to more comprehensively capture the sources of labeling accuracy and improve model design accordingly.

We also observed that the correlation between PA and NA was modest both between and within individuals. Aligned with prior ambulatory findings [40, 42], this result supports the view that PA and NA should be treated as independent dimensions [46, 47], at least in the context of real-world emotion assessment. Importantly, modeling PA and NA separately enables the quantification of mixed emotional states. This approach not only allows for a more comprehensive description of complex emotional responses to real-world events but also facilitates a better understanding of the trajectories of mixed emotions over time. Given that mixed emotional experiences have been linked to mental health and well-being (e.g., [55]; [71]; [72]), such quantification may provide a more nuanced and accurate basis for predicting psychological outcomes in real-world settings.

The SHAP-based feature attribution analysis further suggested that the relative independence between NA and PA was evident not only at the subjective level, but also in their physiological representations. Specifically, HF power contributed most to NA prediction, whereas HR was the most important feature for PA prediction. HF power is commonly interpreted as an index of vagal (parasympathetic) control and has been linked to emotion regulation [73, 74]. Therefore, its contribution to NA prediction likely reflects the regulatory dynamics associated with negative emotional experiences. HR is often linked to emotional arousal [25], and its contribution to PA prediction is consistent with our PA measure emphasizing high-arousal positive states, which aligns with widely used operationalizations of PA in the literature [75]. Notably, the top three contributors were the same for NA and PA: HR (influenced by both sympathetic and parasympathetic activities [76]), HF power (primarily reflecting parasympathetic activity [77]), and LF power (influenced by both sympathetic and parasympathetic activities [77]), but their relative importance differed across NA and PA, pointing to differentiated physiological correlates. Together, converging behavioral and physiological evidence support the feasibility of modeling PA and NA as distinct targets in ambulatory affective computing applications.

While our primary focus is on affective computing strategies, we also hope it is helpful to briefly discuss the challenge of ambulatory ECG data collection itself, given that the current study is, to our knowledge, the first large-scale affective computing investigation to leverage daily-life ECG data. As described in the Methods, participants completed an average of ~5.7 EMA prompts per day (out of 8 scheduled), and the valid ECG data rate was ~74%, providing benchmarks that may guide expectations for EMA prompting frequency and feasibility in future ambulatory data collection. There are several protocol features likely contributed to the relatively high compliance observed here, including completion-contingent bonuses ($60 AUD for completing 80% of EMA prompts and $40 AUD for wearing the ECG sensor for at least 8 days) in addition to base compensation ($10 AUD/day for completing EMA prompts and wearing the ECG sensor), as well as routine monitoring of data quality by research assistants who proactively contacted participants to troubleshoot technical issues and provide support. These procedures may serve as practical strategies for improving compliance and data quality in similar ambulatory research, although they might also overrepresent individuals who are more willing to maintain high compliance under an intensive protocol. Building on our current experience, future work could test the feasibility of similar ambulatory data collection protocols under more application-oriented conditions with minimal or no compliance incentives, which may be particularly informative for real-world deployment.

There are also limitations in the current study. First, the present study focused exclusively on data-level personalization as an initial step toward adapting models to individual users. Future research could build on this foundation by integrating both data-level and model-level personalization strategies [15] to further improve model performance in real-life emotion recognition. However, implementing more advanced model-level personalization strategies (e.g., domain adaptation [16, 17], fine-tuning [18], and hypergraph learning [19]) also requires substantial computational resources. This trade-off becomes particularly relevant when moving toward real-world applications, where online/real-time processing may be needed. Accordingly, a key direction for future work is to examine how data- and model-level strategies interact. For example, future study could test whether data-level selection (e.g., identifying the most informative training samples) can reduce reliance on computationally intensive models while maintaining performance. Clarifying these trade-offs will be essential for developing “efficient personalization” approaches that balance accuracy with efficiency required for real-world deployment.

Second, we framed the prediction task as a binary classification using a median-split threshold, primarily because this simple reference point could facilitate cross-study comparisons even when emotion rating scales differ across datasets. A binary framework also maps naturally onto some decision-oriented applications (e.g., triggering feedback once a threshold is reached). The trade-off, however, is that dichotomization could discard meaningful variance and obscure the finer-grained structure of emotional experience. Future work could examine regression and ordinal formulations that better preserve this granularity and evaluate whether they improve performance, interpretability, and practical utility in specific real-world deployment.

At last, although our study leveraged the largest real-life ECG dataset currently available for affective computing and it includes a diverse range of ages and racial backgrounds, its generalizability to clinical populations remains to be tested. The current sample showed substantial heterogeneity in psychiatric symptoms, which we believe makes the sample well-suited for an initial feasibility-focused examination of everyday-life affective computing. However, our conclusions may not directly extend to more severely symptomatic clinical groups. For example, autistic individuals have elevated rates of alexithymia and often experience difficulties identifying and reporting emotional states [78]. Accordingly, whether our findings regarding emotional awareness generalize to such clinical populations remains an open question. In addition, some clinical populations (e.g., individuals with major depressive disorder) have been reported to exhibit altered physiological regulation [79], which may in turn affect model performance and feature contributions. In summary, the extent to which the current findings generalize to clinical populations should be carefully evaluated in future studies. Future work may also benefit from domain adaptation approaches to address cross-sample distribution shifts between non-clinical and clinical populations [80, 81]. For example, feature-level alignment methods and target-domain calibration/fine-tuning could be evaluated to improve model transferability. Importantly, because clinical populations may differ not only in the distribution of physiological features but also in the distribution of emotion labels (i.e., class priors), adaptation strategies should ideally consider both feature shift and label shifts as sources of transfer error.

6. Conclusion

For real-life ECG-based emotion recognition, substantial individual differences negatively impacted the performance of general models, leading to significantly lower prediction accuracy compared to personalized approaches. The hybrid personalization method, which leverages the most informative participants’ data, outperformed idiographic models trained exclusively on data from a single individual, especially when only limited training data were available. Moreover, models trained on data from individuals with higher self-reported emotional awareness demonstrated better generalizability than those trained on data from individuals with lower awareness. These findings underscore the potential of hybrid personalization as a practical and effective strategy for emotion prediction in real-world settings. Future work should examine how data-level selection can be integrated with model-level personalization strategies to further improve performance while maintaining computational efficiency. It will also be important to test whether these findings generalize to clinical populations and to more fine-grained prediction settings. Ultimately, such advances may pave the way for more scalable and adaptive affective computing systems, including just-in-time interventions that provide timely, personalized support in everyday life.

Biographies

Xin Hu received her Ph.D. degree in Psychology from Tsinghua University in 2022. She is currently a postdoctoral researcher in the Department of Psychiatry at the University of Pittsburgh School of Medicine. Her research focuses on affective neuroscience, affective computing, real-life psychophysiological assessment, and the emotional mechanisms of mental disorders.

Xinke Shen received his B.E. degree in Biomedical Engineering from Beihang University, Beijing, China, in 2017 and the Ph.D. degree in Biomedical Engineering at Tsinghua University, Beijing, China, in 2023. He has been a postdoctoral fellow at the Department of Biomedical Engineering, Southern University of Science and Technology from 2023 to 2025 and is currently a Research Assistant Professor there. His research interests include affective computing and affective neuroscience.

Kristin Naragon-Gainey received her PhD in Clinical Psychology from the University of Iowa in 2011. She is currently a Professor in the School of Psychological Science at the University of Western Australia and a licensed clinical psychologist. Her research focuses on emotional processes and well-being in daily life, as well as transdiagnostic models and assessment of psychopathology.

Lauren M. Bylsma received her PhD in Clinical Psychology from the University of South Florida in 2012. She is currently a licensed clinical psychologist and Associate Professor of Psychiatry and Psychology at the University of Pittsburgh. Her research focuses on characterizing emotional processes transdiagnostically across development in the laboratory and daily life, including the multi-modal assessment of psychophysiological, neural, behavioral, and experiential components of emotion.