Improvements in Mindfulness, Interoceptive and Emotional Awareness, Emotion Regulation, and Inter-Personal Emotion Management Following Completion of an Online Emotional Skills Training Program

Abstract

Socio-emotional skills, such as the ability to recognize, understand, and regulate the emotions of self and others, are associated with both physical and emotional health. The present study tested the effectiveness of a recently validated online training program for increasing these emotional skills in adults. In this study, 448 participants (323 female) were randomly assigned to complete this training program or a placebo control program. Among those who completed the training program or placebo (N = 326), the training program led to improved scores post-training on measures of interoceptive and emotional awareness, mindfulness, emotion recognition, and emotion regulation strategies (e.g., reduced emotion suppression and greater impulse control) relative to placebo. In a smaller group of participants who also completed a 6-month follow-up visit (N= 94), sustained improvements were observed on several measures in those who completed the training program, while the placebo group instead showed decreased performance. This suggested a potentially protective effect against emotional challenges associated with the COVID-19 pandemic occurring during this time. These results suggest that this online training program shows promise in improving emotional skills relevant to adaptive social and emotional functioning, and that it might be useful as an intervention within at-risk populations and those with emotional disorders associated with reduced application of these skills.

Introduction

Socio-emotional functioning is central to mental health (Thompson, 1993). It involves the ability to adaptatively recognize and regulate the emotions of self and others in order to maintain satisfying social relationships and personal well-being (Kemp et al., 2017). Social scientists from several subdisciplines have underscored the importance of effective socio-emotional functioning in these domains (Thompson, 1993). For example, such socio-emotional skills are known to contribute to daily functioning, mental health, and medical outcomes more broadly, such as reduced smoking, obesity, vulnerability to illness, and stress reactivity (Li et al., 2019; Smith, et al., 2019c; Uchino, 2009). In this paper, we describe an online emotional skills training program and investigate its efficacy at improving these socio-emotional skills. Below, we review relevant literature motivating the development and design of this program, and how it may represent an important contribution to existing research in this area.

Work within clinical psychology suggests that maintaining and/or improving socio-emotional functioning depends in part on the development of particular cognitive, emotional, and behavioral strategies or skills (Barlow et al., 2016; Hayes et al., 2003). In leading forms of cognitive-behavioral therapy (CBT; (Barlow et al., 2011; Sakiris & Berle, 2019)), such as the Unified Protocol (UP), these skills involve learning to: 1) attend to one’s own thoughts, feelings, and behaviors, 2) identify unhelpful automatic thoughts and consider other possible interpretations of events, 3) recognize automatic behaviors and identify healthier alternative actions, and 4) counter maladaptive avoidance tendencies by approaching difficult situations and allowing oneself to feel and process the uncomfortable feelings these situations provoke. Within acceptance and commitment therapy (ACT; (Hayes et al., 2003)), further strategies include learning to: 1) non-judgmentally notice one’s own thoughts and feelings without seeing them as true or false, 2) accept and allow negative thoughts and emotions to be present (as opposed to trying to suppress or avoid them), and 3) identify and seek valued directions in life whether or not negative thoughts and emotions persist.

Other leading approaches, such as emotion-focused therapy (EFT; (Greenberg, 2004)), further emphasize the importance of emotion-focused coping. Within EFT, therapists work with participants to achieve greater (1) emotional awareness, (2) emotion regulation, and (3) emotion transformation. This may be especially important, as the use of unhealthy (e.g., suppressive) emotion regulation practices has associated with several mental health issues, such as social anxiety disorder (Dryman & Heimberg, 2018), heightened arousal during negative emotional responses (Gross, 1998; Gross & Levenson, 1997), and interactions between heightened anxiety, depression, and anger (Hosogoshi et al., 2020; Rice et al., 2020). Emotional awareness and emotion regulation are also linked in important ways. For example, awareness of what one is feeling, and why, may provide important information in guiding selection of effective emotion regulation strategies (Barrett et al., 2001). Several studies have also found reduced emotional awareness in disorders characterized by emotion regulation difficulties, such as eating disorders (Vander Wal et al., 2020), depression and anxiety disorders (Kranzler et al., 2016; Novick-Kline et al., 2005; Sendzik et al., 2017), somatoform pain and related psychosomatic conditions (Lane et al., 2011; Stonnington et al., 2013; Zunhammer et al., 2015), and pain within various chronic illnesses (Baeza-Velasco et al., 2012; Lackner, 2005; Smith, et al., 2019a). There is also literature supporting the role of related emotion recognition abilities in vulnerability to depression (e.g., see (Collin et al., 2013; Nyquist & Luebbe, 2020)).

Developing emotion regulation skills also depends upon gaining some awareness and understanding of the relationship between thoughts, emotions, and behaviors – both in general and for how they play out in one’s own daily life – which is often addressed by psychoeducation components of the aforementioned (and many other) psychotherapeutic approaches (Burum & Goldfried, 2007; Wampold, 2015). Specific clinical interventions designed to improve emotional awareness have also been found effective in reducing emotion dysregulation (Neumann et al., 2017), and in improving somatic conditions associated with emotion dysregulation, such as irritable bowel syndrome (Farnam et al., 2014; Thakur et al., 2017) and chronic pain (Burger et al., 2016).

The socio-emotional skills reviewed above (e.g., identifying unhelpful thoughts, accepting one’s feelings without judgment, emotional awareness, and emotion regulation) are most frequently learned within the context of psychotherapeutic treatments for affective disorders, such as CBT, ACT, and EFT. However, these cognitive and emotional skills are beneficial to general emotional functioning and could be helpful to a broader population. For example, if one has learned effective ways of understanding and regulating emotions prior to experiencing a trauma or other serious hardship, this might help to attenuate the development of affective disorders or unhealthy coping mechanisms (e.g., as suggested by (Söndergaard & Theorell, 2004)). To date, however, there is a lack of widely available empirically supported resources for learning such skills outside the context of specific psychotherapies.

Of note, there have been some mobile or online interventions based on CBT, ACT, and mindfulness developed for use within specific populations. Online, self-guided therapy modules have been utilized among targeted groups, such as middle-aged women with weight concerns and youth exhibiting depressive symptoms (Boucher et al., 2016; Schleider et al., 2022; Schleider & Weisz, 2017). A handful of mobile app interventions employing CBT skills training have been found effective in college students (Oliveira et al., 2021). Similarly, a brief, online mindfulness-based intervention proved beneficial to college students (Cavanagh et al., 2013). Other mindfulness training programs have sparked a growing number of studies that find mixed evidence of effectiveness (e.g., see (Bartlett et al., 2019; Nadler et al., 2020; Schumer et al., 2018; Suleiman-Martos et al., 2020)). For example, a meta-analysis of workplace-delivered mindfulness trainings revealed that many programs deviate from evidence-based protocols; yet, most programs tend to be at least marginally effective (Bartlett et al., 2019). Some of these programs also tend to focus on a narrower range of skills (e.g., mindfulness or facial expression recognition alone; (Schumer et al., 2018)) than those learned in evidence-based psychotherapies. While there are clear strengths for brief intervention programs targeting individual skills, as is seen in the positive outcomes for individuals receiving online single-session interventions (SSIs; (McDanal et al., 2022; Schleider et al., 2020; Wasil, Osborn, et al., 2021; Wasil, Taylor, et al., 2021)), there could also be added benefits if multiple synergistic skills were learned simultaneously (e.g., using emotion regulation techniques to counter avoidance of difficult situations). Thus, multi-faceted training approaches offer additional promise.

One set of training programs aimed at increasing a more comprehensive set of abilities – jointly conceptualized as “emotional intelligence” (EI) – has also been the topic of many studies, focused largely on increasing effectiveness in academic and occupational settings (for a meta-analysis, see (Mattingly & Kraiger, 2019)). These abilities are often said to include recognizing, understanding, using, and managing the emotions of self and others (Mayer, 2004; Salovey & Mayer, 1990; Smith, et al., 2018a). However, there are different conceptualizations (and resulting measures) of EI, leading to difficulties in comparing the effectiveness of training efforts across studies. For example, “trait” EI refers to an individual’s self-reported beliefs about their competencies in various social and emotional domains (Bar-On, 2006; Petrides et al., 2016). In contrast, “ability” EI employs tests of emotion recognition, understanding, and management abilities, where scores are calculated based on similarity to test responses provided by the majority of test-takers (i.e., consensus scoring; (Mayer et al., 2003)). While the importance of such emotional skills is not in question, several authors have stressed the limitations of these specific models and measures (Conte, 2005; Fiori et al., 2014; Locke, 2005; Matthews et al., 2007; Maul, 2012; Roberts et al., 2006; Smith, et al., 2018a). Interventions focused on emotional intelligence have shown promising results, such as reducing aggression, stress, and personal distress, and increasing empathy, life satisfaction, and communication skills in a variety of different contexts, including schools, nursing programs, clinical populations, and sports teams (Castillo et al., 2013; Meng & Qi, 2018; Schutte et al., 2013). However, many of these EI training procedures are largely inaccessible to the general public (e.g., they typically offer educational materials or seminars within a specific workplace context, such as healthcare settings; (Karimi et al., 2020; Sharif et al., 2013)).

In summary, there have been many efforts to develop accessible training programs to improve social, emotional, and cognitive skills drawing from evidence-based psychotherapies and the construct of emotional intelligence. The literature reviewed above shows that programs focusing on specific skills can be successful interventions in unique populations. However, there is less research available on more comprehensive socio-emotional training programs. Additionally, few studies have explored how online programs of this sort could be used as a prevention effort in the general population to buffer future responses to stressful life events. Therefore, in this paper, we report on the efficacy of a recently validated and remotely accessible web-based emotional intelligence training program (Persich, et al., 2021a). This program was based in part on skills learned within evidence-based psychotherapies (e.g., CBT and ACT), as well as previous work in the EI literature. It was designed to help individuals develop generalizable skills in the domains of emotion recognition and interoceptive/emotional awareness for self and others, cognitive reappraisal and mindful acceptance skills, and socio-emotional decision-making skills. Each of these training modules were based on empirical research within social, emotional, and clinical psychology (Persich, et al., 2021a). An initial validation study of this program focused on improving scores on trait and ability measures of EI (Persich, et al., 2021a). Here, we test the effectiveness of this program at improving a wider range of socio-emotional skills using other established measures in affective science and clinical psychology known to be associated with mental health (e.g., (Burger et al., 2016; Lane et al., 2020; Lane et al., 2015; Preece et al., 2020; Thakur et al., 2017)). As this previously validated program has the potential to be easily and widely accessible, establishing its ability to improve this wider range of skills will be an important next step toward prevention efforts among the general population. In the following analyses, we therefore hypothesized that, relative to a placebo control program, this training program would improve scores on previously validated measures of each ability reviewed above, including interoceptive and emotional awareness, emotion recognition, emotion regulation, mindfulness, coping, and related socioemotional skills. We also hypothesized that these improvements would remain stable over a 6-month follow-up period. To inform future applications of the program, we also tested whether completion of the program within an efficient 1-week period would be equally efficacious relative to a more extended 3-week period that could potentially allow more opportunity for consolidation.

Methods

Participants

A convenience sample of students at the University of Arizona as well as individuals from the surrounding community was recruited from Tucson, AZ. To ensure that participants would have the ability to complete the study, and to reduce possible confounds, we required participants to be between 18 and 40 years of age, demonstrate a reading proficiency at an 8^th grade reading level or higher, and have no neurological, medical, or substance-related conditions that might impact task performance. A total of 448 participants took part in the baseline assessment (mean age = 23.7, SD = 5.5 years; 72.1% female; 60.9% White; 69.87% students; 81.5% with income level > $30K, 12.3% with income level $30-50K, 6.2% with income level < $50K). For more detailed information on demographics, see Supplementary Table S1. For a full consort diagram, see Figure 1.

Figure 1.

Consort diagram depicting participant recruitment, allocation to condition, and study completion.

Procedure

The protocol for this study was approved by the University of Arizona Institutional Review Board (IRB) and the U.S. Army Human Protections Office (HARPO). Participants gave informed consent and were compensated for their participation (see below for details). The study was supported by funding from the U.S. Army Medical Research and Development Command (W81XWH-16-1-0062).

Baseline and Post-Training

Participants reported to a laboratory where they completed baseline assessments of emotion recognition, emotional awareness, emotion regulation, and related socio-emotional skills (described below). They were then either assigned to an “internal awareness training” (treatment condition targeting emotional skills; n = 234) or an “external awareness training” (placebo control condition targeting non-emotional awareness of the external world; n = 214); this training component (described below) was completed entirely online and typically took 10-12 hours to complete. As mentioned above, one secondary question was whether duration of time to complete the program would have an influence on its efficacy. So, each program was set up to be completed over either a compressed (1-week) or distributed (3-week) schedule. In each condition, participants were given a specific schedule that told them which training program modules to complete on each day, divided evenly across either the seven or twenty-one day training periods (see Supplementary Tables S3 and S4 for exact schedule details). They were told to complete the assigned modules for each day by midnight, which was closely monitored. The first time a participant did not complete the assigned modules on a given day, they were emailed the following morning with instructions to complete the missed modules by the end of that day, along with any other modules that were assigned for that day. The second time a participant did not complete the assigned modules for a given day, they were removed from the study.

Therefore, participants were randomly assigned to one of four separate conditions (i.e., compressed placebo, distributed placebo, compressed emotional skills training, or distributed emotional skills training). No effects of training schedule (1-week vs. 3-week) were found for any measures in the initial validation study (Persich, et al., 2021a); however, we also assess potential effects of these different schedules on the outcome variables assessed here (see below). Upon finishing assigned trainings, participants returned to the laboratory to complete post-training assessments identical to those collected at baseline.

Follow-up

Approximately six months after the post-training assessments, participants were contacted to complete a long-term follow-up assessment. This was included for two reasons. First, improvements in emotional skills post-training may be gradual as participants continue to practice the skills they have learned. Second, we wished to assess whether post-training improvements were stable over time (and therefore more likely to promote long-term positive outcomes). Early in data collection, participants were inadvertently debriefed after the post-training assessments and made aware of their study condition. This was later corrected, but, as a result, the sample size at 6-month follow-up was unfortunately limited to those who did not receive debriefing and responded to later contact attempts. Specifically, of the 326 participants that completed post-training assessment (described below) and were eligible to return for the 6-month follow-up, 154 were not contacted due to accidental debriefing, 78 were contacted and did not respond, and 94 were successfully contacted and agreed to complete the 6-month follow-up. All participants completed pre and post visits before March 17, 2020. It is worth noting that the COVID-19 pandemic led to government initiated mandated “stay-at-home” orders to control the spread of the virus starting on March 19, 2020, between the last post-training assessment session and the first 6-month follow-up session. The 6-month follow-up assessments were initiated on March 1^st, 2020, with all but 4 participants completing the follow-up after April 21, 2020. These were completed entirely online to ensure the safety of participants. We emphasize this because it is important to keep in mind the emotional toll of the pandemic when interpreting changes in assessment scores relative to baseline. Additionally, due to time constraints and the need to administer assessments in an online format, only a subset of the original measures was given at the 6-month follow-up.

Compensation

Participants could earn up to $600 for completion of all study activities. This was prorated according to the following: completion of baseline measures only = $25 total; completion of baseline measures and all training modules = $100 total; completion of baseline measures, all training modules, and post-training assessments = $450 total; completion of baseline measures, all training modules, post-training assessments, and 6-month follow-up = $600 total. Adherence to the training program schedules (compressed vs. distributed) was strictly monitored virtually. Participants received reminders if they fell behind schedule and were removed if they were unable to maintain the necessary pace.

Emotional Skills Training Program

Overview

The program was introduced to participants as training “emotional intelligence” and defined emotional intelligence in terms of a set of adaptive social and emotional skills. The program structure is divided into three sequential portions, “tiers 1, 2, and 3”, that were designed to resemble moving up to more and more advanced levels in a video game (for details on the prior validation of this program, see (Persich, et al., 2021a)). Each tier has several modules, which focus on different content areas.

The content of each module was derived largely from components of evidence-based interventions within clinical psychology, including 1) psychoeducation, 2) motivational interviewing (Miller, 1996), 3) adaptive emotion regulation techniques (mindful acceptance, cognitive reappraisal; based on ACT and CBT; (Barlow et al., 2011; Hayes et al., 2003)), 4) identifying patterns of (and synergies between) thoughts, bodily sensations, emotions, and behaviors in common situations, 5) identifying alternative adaptive behaviors in difficult situations, 6) learning the role of context in emotion perception (Aviezer et al., 2008; Barrett et al., 2011), and 7) selecting adaptive goals and behaviors in social contexts. Content across several modules was based on the Unified Protocol (UP), an emotion-regulation based intervention program that uses a ‘shared-mechanisms of change’ approach (Sakiris & Berle, 2019). Example screenshots from the program are shown in Figure 2 and Supplementary Materials; also see (Persich, et al., 2021a). Content in other modules was drawn from current theories of emotion and emotional awareness (Barrett, 2017; Moors, 2013; Moors et al., 2013; Scherer, 2009; Siemer et al., 2007; Smith, 2020; Smith, et al., 2019b), studies of the protective physical and mental health benefits of emotional awareness (Kashdan et al., 2015; Smith, et al., 2018b), and studies of effective vs. ineffective emotion regulation strategies (Gross, 2007; Gyurak et al., 2011; Sakiris & Berle, 2019; Spaapen et al., 2014). Some exercises were further developed from common research paradigms used to assess individual differences in emotion recognition and socio-emotional management abilities (Allen et al., 2015; Austin & O’Donnell, 2013; Bänziger et al., 2009; Russell, 1994; Scherer et al., 2011; Smith, et al., 2018a; Wright et al., 2017).

Figure 2.

Example screenshots of different components of the program. The upper left is from a lesson that illustrates how interpretations of situations alter patterns of thoughts, feelings, and behavior. To the right of this is a tool in the program that allows individuals to choose fine-grained emotion labels to describe how they are/were feeling in different situations (each emotion label has an accessible definition). The upper right tool asks individuals to draw where they feel sensations in their body in different situations, and to describe the situation, their interpretation, what they wanted to do, and what they actually did. The bottom left is from a lesson that illustrates how the same facial expression can indicate different mental states in different contexts. To the right of this is a slide from a lesson teaching how to identify automatic unhelpful thoughts and to use strategies to consider alternative interpretations in the service of emotion regulation. The bottom right asks individuals to choose verbal responses to help other individuals feel better in simulated social situations. Note: Images of faces have been blurred to maintain anonymity.

Tier 1: Introduction, psychoeducation, and goal-setting.

This tier has 4 lessons that introduce the skills that will be trained and why they matter. It then focuses on psychoeducation about how thoughts, emotions, bodily sensations, and behaviors interact in bidirectional ways, how this can often be helpful, and how it can sometimes be harmful. It also introduces an “emotion tracking” tool used throughout the program for tracking one’s own thoughts, emotions, and behaviors in real-time. This includes identifying a situation, describing the automatic thoughts and emotions elicited by that situation, and indicating emotion-related bodily sensations felt in that situation (on an avatar, based on (Hietanen et al., 2016; Nummenmaa et al., 2014)), as well as identifying automatic motivations and behaviors and their consequences. Finally, it lays out the specific training goals of the program in terms of developing a concrete set of skills. It asks the participant to identify skills that matter most to them, specific benefits they expect from developing them, and to set their own goals. This was based on established motivational interviewing techniques and used to help maximize motivation and commitment to fully engage in the specific skills trained in the subsequent lessons (Miller, 1996).

Tier 2: Specific skills training.

This tier includes 5 lessons that introduce and provide practice at using specific emotional skills, including mindful acceptance, cognitive reappraisal, facial emotion recognition, and adaptive decision-making in emotional contexts by using these skills.

Tier 3: Extended Practice and Skill Generalization.

This tier includes 7 lessons that provided additional practice and extensions of the skills learned in Tier 2 (e.g., emotion recognition in voices, and identifying contexts where the same facial expression can mean different things; based on (Aviezer et al., 2008; Barrett et al., 2011)). It also included “challenges” where multiple skills needed to be used together in simulated situations. Finally, the program ended with a “self-investigation” where the participant could gain further awareness by looking back at each time they used the emotion tracking tool to identify patterns in their own thoughts, emotions and behavioral motivations, as well as to set goals for using what they’ve learned going forward. For more details on the training program, see Supplementary Materials (Figures S1- S9, Tables S2 and S3); also see (Persich, et al., 2021a).

Placebo Program

The matched placebo program took the same length of time to complete (10-12 hours) and taught non-emotional skills related to awareness of the external world (e.g., identifying a plant based on features of its leaves). For a full description of the placebo program, see Supplementary Materials (Figures S10-S12, Table S4). This program was designed to parallel the emotional skills training program where possible and was similarly organized into a tiered structure. Tier 1 focused on introducing the basics of the scientific process, tier 2 focused on teaching scientific practices and how they can be applied to the external world, and tier 3 provided participants with opportunities to apply knowledge learned in the previous lessons. Figure 3 shows example sections of the placebo program designed to be visually similar (and with similarly structured exercises) to the emotional skills training program.

Figure 3.

Example screenshots of different components of the placebo program. The left panel shows an example of a lesson that teaches students about food chains, and how the various components of the food chain interact (designed to match learning about emotional reaction cycles in the training program; see Figure 2, Top Left). The middle panel teaches students to identify plants and animals based on their features (designed to match learning about specific emotion categories and their situational and bodily features in the training program; see Figure 2, Top Middle/Right). The right panel asks students to help solve a problem using scientific reasoning skills (designed to match social reasoning practice in the training program; see Figure 2, Bottom Right).

Measures

To assess the ability of the training program to improve emotional skills and associated beliefs/dispositions, we selected a set of representative measures of each aspect of the training. This included two measures of interoceptive/emotional awareness, two measures of mindfulness tendencies, two measures of emotion regulation, and three measures of socio-emotional skills.

Interoceptive/Emotional Awareness

The Levels of Emotional Awareness Scale (LEAS) was used to assess changes in emotional awareness. The LEAS (Barchard et al., 2010; Lane et al., 1990) asks participants to describe what they believe they and another individual would feel in each of 10 hypothetical scenarios. Freely written responses (typed into a web-based interface; http://eleastest.net/) are scored automatically by computer software based on the specificity of the words chosen to describe feelings. Possible scores range from 0-40 for self and others and 0-50 for the total score; the LEAS demonstrates good internal reliability, with a Cronbach’s alpha for the total score of .88 (Lane et al., 2000). As a convergent measure of changes in emotional awareness, participants completed the Multi-Dimensional Interoceptive Awareness Scale, or MAIA (Mehling et al., 2012), which is a measure of eight facets of interoceptive/emotional awareness and includes a total of 32 items. Possible subscale scores range from 0-5 and may be added together for a total score ranging from 0-40. Cronbach’s alphas for the MAIA subscales range from .66 to .87 (Mehling et al., 2012).

Mindfulness

To assess changes in mindfulness, we used the Five-Facet Mindfulness Questionnaire (FFMQ) and the Multidimensional Experiential Avoidance Questionnaire (MEAQ). The FFMQ (Baer et al., 2006) has 39 items that measure tendencies to engage in mindfulness practices. Total scores range from 39-195. Most subscales include 8 questions and, therefore, have a possible range from 8-40 (the Non-reactivity subscale includes only 7 questions and has a possible range of 7-35). Cronbach’s alphas for the FFMQ subscales range from .75 to .91. As a convergent measure of changes in mindfulness, we collected the MEAQ (Gamez et al., 2011). This 62-item instrument measures self-reported tendencies to engage with distress in non-mindful ways (i.e., avoiding rather than accepting/observing distress). Possible subscale scores include the following ranges: 7-42 (procrastination, distraction/suppression), 11-66 (behavioral avoidance, distress endurance), and 13-78 (distress aversion, repression/denial). Total scores range from 62-372. Average Cronbach’s alphas for the MEAQ subscales is .85, and the MEAQ total score has an alpha of .94 (Gamez et al., 2011).

Emotion Regulation

The Emotional Regulation Questionnaire (ERQ) was used to assess changes in emotion regulation strategies. The ERQ (Gross & John, 2003) is a 10-item self-report scale which assesses tendencies to regulate emotions through reappraisal strategies or by suppressing their expression. Scores for the cognitive reappraisal and emotion suppression subscales range from 6-42 and 4-24, respectively. Both demonstrate good internal reliability, with Cronbach’s alphas of .89 for cognitive reappraisal and .76 for suppression (Preece et al., 2021). The Difficulties in Emotion Regulation Scale, or DERS (Gratz & Roemer, 2004), was used as a convergent measure of changes in emotion regulation. It is a 36-item self-report measure of an individual’s current difficulties regulating their emotions. Possible scores for subscales range from 5-25 (Goal Interference, Lack of Clarity), 6-30 (Nonacceptance, Impulse Control, Lack of Awareness), 8-40 (Lack of Strategies). Total score range from 36-180. Cronbach’s alphas for the subscales range from .80 to .89 (Gratz & Roemer, 2004).

Socio-Emotional Skills

As one representative socio-emotional skill, the Perception of Affect Task (PAT; (Lane et al., 1996; Wright et al., 2017)) was used to assess emotion recognition ability. It consists of four 35-item subtasks designed to assess seven emotion categories. These four subtasks vary the verbal vs. nonverbal nature of stimulus and response. The score for each subtask is the number of items for which the intended emotion is accurately recognized; the maximum score possible is 100. The PAT demonstrates good internal consistency across the four subtasks, with Cronbach’s alphas ranging from .74 to .93 (Wright et al., 2017). As a second, brief measure of socio-emotional abilities, we used the 58-item Managing Emotions of Others Scale, or MEOS (Austin & O’Donnell, 2013), and specifically its subscale for Poor Skills. This subscale measures the inability to motivate others or change their mood. Other MEOS subscales do not directly assess changes in skills (instead measuring strategic tendencies); therefore, we did not expect changes in these subscales and only analyzed them in exploratory fashion, correcting for multiple comparisons. Possible MEOS subscale scores include ranges between 5-25 and 15-75, depending on the number of items. The subscales show good internal reliability, with Cronbach’s alphas ranging from .68 to .91 (Austin & O’Donnell, 2013). As a final, performance-based measure of socio-emotional abilities, we collected the brief Situational Test of Emotion Management, or STEM-B (Allen et al., 2015). This instrument has 18 items, each describing an emotional situation. Using a multiple-choice format (4 choices per item), individuals are asked to select the response that would be most effective for managing both the emotions the person is feeling and the problems presented in each situation. Scores are calculated as a fraction of correct answers, ranging from 0.00-1.00. The STEM-B demonstrates good internal reliability, with a Cronbach’s alpha of .84 for total scores (Allen et al., 2015).

Additional information describing the subscales and scoring for each of the above measures can be found in Supplementary Materials.

Wechsler Abbreviated Scale of Intelligence (WASI-II)

As a measure of general intelligence (IQ), we used the two-subtest WASI-II (Wechsler, 2011) to account for training program effects that could be merely due to differences in general cognitive ability. This includes Vocabulary and Matrix Reasoning subtests, where the former assesses word knowledge and verbal concept formation, and the latter assesses different aspects of fluid intelligence (visuospatial and classification ability, simultaneous processing, perceptual organization).

Due to time constraints and other factors related to the COVID-19 pandemic, we were unfortunately unable to collect the LEAS, MAIA, FFMQ, MEAQ, or PAT at the 6-month follow-up visit. Data for these measures are therefore only available for baseline and post-training visits. However, the ERQ, DERS, MEOS, and STEM-B were collected at all 3 visits.

Analyses

Linear mixed effects models (LMEs) for each outcome variable were run in R (version 4.0.2), using the ‘lme4’ package (https://CRAN.R-project.org/package=lme4). These LMEs follow the “intention-to-treat” principle by adopting a likelihood-based method that includes all participants randomized to either group and provides unbiased estimates in cases with missing data due to dropout before post-intervention assessment. These models allowed for random intercept values in each participant and included fixed effects of age, sex, IQ, group, time, and schedule (i.e., compressed vs. distributed training period), as well as potential two-way interactions between group, time, sex, and schedule. Three-way interactions between sex, group, and time, and between schedule, group, and time, were also included. These models allowed us to assess whether changes in each variable from pre-to-post training were different in those who received the true vs. placebo training programs (i.e., a group by time interaction), and whether this was moderated by sex (i.e., a three-way interaction between sex, group, and time) or by compressed (1-week) vs. distributed (3-week) training schedules (i.e., a three-way interaction between schedule, group, and time). The interactions with sex were tested because multiple measures we used are known to show higher scores in females (e.g., on the PAT and LEAS; (Wright et al., 2017)), which raises the question of whether females might benefit less from the training program (e.g., due to ceiling effects) or perhaps benefit more due to greater interest/engagement with the program content.

We first examined the total score for each measure (and the MEOS scale for socio-emotional skills). If the hypothesized group by time interaction was significant, we carried out post-hoc analyses of subscales to provide insights about which subconstructs were driving the change in total scores. If the interaction was not significant for total scores, we examined subscales in an exploratory fashion and corrected for multiple comparisons (i.e., for the number of subscales). We tested for these interactions with the hypothesis that the training program, but not the placebo, would increase scores on measures of interoceptive/emotional awareness, mindfulness, and socio-emotional skills, and decrease scores on measures of poor emotion regulation. Effect sizes were estimated using the partial eta-squared statistic. We note that heuristic cutoffs for small, medium, and large effect sizes for this statistic are: .01, .06, and .14, respectively (Cohen, 1969).

The LMEs described above included scores on the first and second visits, allowing us to include data from all participants that completed the post-training assessment. For the subset of measures collected at 6-month follow-up, we ran analogous LMEs to those above including scores at all three timepoints (i.e., including the subset of participants that completed these measures at each timepoint). This allowed us to assess, for example, whether improvements seen post-training were retained over time, or whether measures that initially did not show improvement may have increased over time in the months post-training (e.g., after individuals had the opportunity to further practice the skills they learned in their daily lives). As a further test of the stability of improvements seen post-training, we also ran intraclass correlations, based on absolute agreement [ICC(2, 1)], between post-training scores and 6-month follow-up scores. These ICCs were calculated for any measures that showed a significant group by time interaction from pre-to-post training (i.e., indicating improvement attributable to the training program). In assessing ICCs, we apply the following established conventions (Cicchetti, 1994): ‘poor’ (ICC < .4) ‘fair’ (ICC ≥ .40) and ‘good’ (ICC ≥ .6), ‘excellent’ (ICC ≥ .75).

Transparency and Openness

We report all data exclusions, all manipulations, and all measures in the study, and we follow Journal Article Reporting Standards (Kazak, 2018). All study data and materials are available upon request without reservation. Analyses for this study were not pre-registered; data were analyzed using R (version 4.0.2) . For each analysis, participants were only included if complete data were available for those participants for all measures required for that analysis. As mentioned above, this study builds off of a prior validation study using data from the same sample (Persich, Smith, Cloonan, Woods-Lubbert, et al., 2021). Analyses presented here are unique, as we examine different outcome variables relating to socioemotional functioning and emotional awareness, whereas the previous study examined emotional intelligence measures. Another previously published study examined anxiety and depression in the smaller subset of participants who completed all three timepoints (Persich, et al., 2021b).

Results

Demographics and Attrition

Descriptive statistics for all demographic variables (i.e., age, sex, race, student status, education level, and income) at each study visit are reported in Supplementary Table S1. Two-sample t-tests and chi-squared analyses revealed no significant differences between the training group and placebo group for any demographic variable at any visit. Descriptive statistics for all other variables by group and timepoint are shown in Supplementary Table S5. Due to insufficient time to complete all measures, and/or removal of outlier data from participants that did not appear to perform tasks correctly, only a subset of data was available for certain measures. Table S5 therefore also lists the number of participants with usable data for each measure. Two-sample t-tests revealed no differences between groups at baseline for any of these measures, with the exception of the DERS Lack of Awareness subscale (t(443) = 2.04, p = .04, Cohen’s d = .19) and the STEM-B (t(438) = 1.99, p = .048, d = .19), each of which were higher in the training group.

Out of the initial baseline sample, 326 participants completed the program and returned for post-training assessment (Training: 168, Placebo: 158). Of those that did not return for post-training assessment, only 3 completed all training modules. Thus, nearly all of those who did not return for the second visit did so because they failed to complete the program. For reasons described above (see Procedures), only 94 participants also completed the 6-month follow-up (Training: 55, Placebo: 39). To assess potential determinants of this pattern of attrition, we conducted two-sample t-tests comparing baseline scores in those who did vs. did not complete post-training assessments. We then performed analogous t-tests comparing post-training scores for those who did vs. did not complete the 6-month follow-up. At baseline, results revealed significant differences between program completers and non-completers for several measures, including LEAS, FFMQ, MEAQ, ERQ Suppression, DERS, and PAT (for detailed results, see Supplementary Results). Briefly, results showed that, at baseline, completers scored higher than non-completers on emotional awareness, mindfulness, and emotion recognition, as well as lower on difficulties with emotion regulation and suppression. At post-training, results showed no significant differences between those that did vs. did not complete the 6-month follow-up.

As reported in the initial validation paper (Persich, et al., 2021a), participants in the distributed (3-week) condition also showed significantly greater attrition than those in the condensed (1-week) condition (reproduced here: chi-squared = 12.77, p = .004).

Pre-to-Post Training Program

Table 1 reports effects of group, time, and their interaction for pre-to-post training data on all primary outcome variables (i.e., total scores). To offer additional insights about specific effects underlying changes in primary outcomes, results for subscales with significant group by time interactions are also shown. For detailed presentation of all other main effects and interactions, as well as subscale scores that did not include a significant group by time interaction, see Table S6 in Supplementary Materials.

Table 1.

Results from linear mixed effects models (LMEs) baseline to post-training.

	F (df)	p	ηp2	B [CI]	Post-Hoc Contrasts for Group by Time Interaction (Post-Training - Baseline)
LEAS Total
Time	4.83 (1, 305)	0.029	0.02	0.37 [0.09, 0.66]	Training: 1.21, t(306) = 3.43, p < 0.001 Placebo: −0.16, t(305) = −0.42, p = 0.68
Group	3.6 (1, 405)	0.058	0.01	0.75 [0.28, 1.23]
Group by Time	7.2 (1, 305)	0.008	0.02	0.4 [0.11, 0.68]
LEAS Self
Time	9.78 (1, 321)	0.002	0.03	0.5 [0.22, 0.77]	Training: 1.27, t(323) = 3.75, p < 0.001 Placebo: 0.23, t(322) = 0.63, p = 0.53
Group	3.6 (1, 401)	0.058	0.01	0.66 [0.26, 1.06]
Group by Time	4.48 (1, 321)	0.035	0.01	0.31 [0.03, 0.58]
MAIA Total
Time	10.56 (1, 343)	0.001	0.03	0.35 [0.09, 0.6]	Training: 1.3, t(346) = 3.9, p < .0001 Placebo: 0.12, t(344) = 0.37, p = 0.71
Group	0.04 (1, 434)	0.843	0	0.16 [−0.38, 0.69]
Group by Time	6.05 (1, 344)	0.014	0.02	0.36 [0.11, 0.62]
MAIA Emotional Awareness
Time	8.41 (1, 352)	0.004	0.02	0.07 [0.02, 0.13]	Training: 0.26, t(355) = 3.87, p < 0.001 Placebo: 0.01, t(352) = 0.1, p = 0.92
Group	0.83 (1, 432)	0.362	0	0.11 [0.01, 0.2]
Group by Time	6.58 (1, 353)	0.011	0.02	0.06 [0.01, 0.12]
MAIA Not Distracting
Time	0.89 (1, 369)	0.345	0	0.02 [−0.04, 0.08]	Training: 0.17, t(373) = 2.21, p = 0.028 Placebo: −0.09, t(368) = −1.09, p = 0.278
Group	0.06 (1, 427)	0.808	0	−0.01 [−0.1, 0.08]
Group by Time	5.19 (1, 369)	0.023	0.01	0.09 [0.03, 0.15]
MAIA Self-Regulation
Time	18.41 (1, 351)	<0.001	0.05	0.09 [0.03, 0.14]	Training: 0.33, t(354) = 4.55, p < 0.001 Placebo: 0.1, t(352) = 1.38, p = 0.17
Group	0.05 (1, 432)	0.831	0	0.06 [−0.05, 0.16]
Group by Time	4.57 (1, 352)	0.033	0.01	0.06 [0.01, 0.12]
FFMQ Total
Time	22.97 (1, 339)	<0.001	0.06	1.93 [1.12, 2.75]	Training: 5.41, t(341) = 5.26, p < 0.001 Placebo: 1.34, t(339) = 1.26, p = 0.21
Group	0.79 (1, 435)	0.374	0	1.05 [−0.79, 2.9]
Group by Time	7.78 (1, 339)	0.006	0.02	1.15 [0.34, 1.97]
FFMQ Descriptions
Time	14 (1, 342)	<0.001	0.04	0.43 [0.17, 0.69]	Training: 1.43, t(344) = 4.35, p < 0.001 Placebo: 0.2, t(342) = 0.6, p = 0.55
Group	0.26 (1, 434)	0.607	0	0.17 [−0.38, 0.73]
Group by Time	7.3 (1, 342)	0.007	0.02	0.36 [0.1, 0.62]
MEAQ Total
Time	6.42 (1, 326)	0.012	0.02	−1.04 [−2.25, 0.17]	Training: −4.91, t(326) = −3.21, p = 0.001 Placebo: −0.23, t(326) = −0.15, p = 0.884
Group	0.29 (1, 435)	0.593	0	−0.41 [−4.04, 3.22]
Group by Time	4.7 (1, 326)	0.031	0.01	−1.23 [−2.44, −0.02]
ERQ Suppression
Time	1.48 (1, 342)	0.224	0	−0.04 [−0.3, 0.21]	Training: −0.78, t(344) = −2.44, p = 0.015 Placebo: 0.3, t(342) = 0.89, p = 0.376
Group	0 (1, 434)	0.975	0	−0.16 [−0.71, 0.38]
Group by Time	5.5 (1, 342)	0.02	0.02	−0.31 [−0.56, −0.05]
ERQ Reappraisal
Time	0.33 (1, 356)	0.565	0	0.01 [−0.32, 0.33]	Nonsignificant interaction
Group	0.11 (1, 431)	0.742	0	0.13 [−0.43, 0.69]
Group by Time	0.55 (1, 357)	0.458	0	0.08 [−0.25, 0.41]
DERS Total
Time	17.93 (1, 334)	<0.001	0.05	−1.65 [−2.54, −0.76]	Training: −5.26, t(336) = −4.67, p < 0.001 Placebo: −1.33, t(334) = −1.15, p = 0.252
Group	0.12 (1, 435)	0.725	0	−0.23 [−2.47, 2.01]
Group by Time	5.78 (1, 334)	0.017	0.02	−1.16 [−2.05, −0.27]
DERS Awareness
Time	2.55 (1, 343)	0.111	0.01	−0.08 [−0.3, 0.15]	Training: −0.68, t(345) = −2.42, p = 0.016 Placebo: 0.15, t(343) = 0.5, p = 0.617
Group	1.44 (1, 434)	0.231	0	0.07 [−0.4, 0.54]
Group by Time	4.35 (1, 343)	0.038	0.01	−0.26 [−0.48, −0.03]
DERS Clarity
Time	26.38 (1, 337)	<0.001	0.07	−0.28 [−0.43, −0.14]	Training: −1.04, t(338) = −5.66, p < 0.001 Placebo: −0.27, t(337) = −1.4, p = 0.162
Group	0.01 (1, 435)	0.911	0	−0.09 [−0.44, 0.26]
Group by Time	8.46 (1, 337)	0.004	0.02	−0.23 [−0.37, −0.08]
MEOS Poor Social Skills
Time	2.22 (1, 351)	0.137	0.01	−0.1 [−0.25, 0.05]	Training: −0.55, t(354) = −0.28, p = 0.005 Placebo: 0.15, t(351) = 0.74, p = 0.457
Group	0.23 (1, 432)	0.63	0	0.02 [−0.26, 0.3]
Group by Time	6.11 (1, 352)	0.014	0.02	−0.16 [−0.32, −0.01]
MEOS Conceal **
Time	32.61 (1, 334)	<0.001	0.09	−0.53 [−0.75, −0.31]	Training: −1.58, t(336) = −5.71, p < 0.001 Placebo: −0.56, t(334) = −2.02, p = 0.045
Group	0.04 (1, 435)	0.835	0	−0.09 [−0.65, 0.47]
Group by Time	6.56 (1, 334)	0.011	0.02	−0.32 [−0.54, −0.1]
MEOS Divert *
Time	11.52 (1, 345)	<0.001	0.03	−0.27 [−0.43, −0.11]	Training: −0.87, t(347) = −4.41, p < 0.001 Placebo: −0.09, t(345) = −0.49, p = 0.626
Group	0.03 (1, 434)	0.862	0	0.02 [−0.3, 0.34]
Group by Time	7 (1, 345)	0.009	0.02	−0.18 [−0.34, −0.03]
PAT Total
Time	54.79 (1, 183)	<0.001	0.23	1.85 [1.31, 2.38]	Nonsignificant interaction
Group	0.16 (1, 242)	0.686	0	0.3 [−0.84, 1.45]
Group by Time	2.83 (1, 183)	0.094	0.02	0.38 [−0.15, 0.91]
PAT Faces-Words **
Time	0.16 (1, 200)	0.693	0	−0.1 [−0.27, 0.08]	Training: 0.24, t(202) = 1.23, p = 0.222 Placebo: −0.41, t(200) = −1.92, p = 0.055
Group	0.1 (1, 238)	0.752	0	0.05 [−0.21, 0.31]
Group by Time	4.83 (1, 200)	0.029	0.02	0.21 [0.04, 0.39]
STEM-B Tota
Time	58.12 (1, 363)	<0.001	0.14	0.02 [0.01, 0.03]	Nonsignificant interaction
Group	5.39 (1, 423)	0.021	0.01	0.01 [0, 0.02]
Group by Time	0.28 (1, 363)	0.5999	0	0 [0, 0.01]

Note: Effects of time and group were treatment coded (i.e., T1/baseline = 0, T2/post-training = 1; placebo program = 0, training program = 1). Coefficients and significance tests for other predictors in each model are shown in Supplementary Materials. Age and IQ were centered. Sex and Schedule were sum coded (i.e., males = −1, females = 1; distributed = −1, compressed = 1), such that coefficients for other predictors were interpretable as main effects centered between levels of these variables.

^*Survived correction for multiple comparisons, as effects were not hypothesized a priori.

^**Did not survive correction for multiple comparisons.

Interoceptive/Emotional Awareness

For LEAS Total scores, we observed the hypothesized group by time interaction, indicating greater pre-to-post increases for the training program compared to the placebo program: F(1, 305) = 7.2, p = .008, η2=0.02 (see Figure 4). Post-hoc LMEs indicated that this interaction was driven by changes in the Self subscale scores (see Table 1 for details).

Figure 4.

Means and standard errors illustrating change from pre- to post-training in the emotional skills program vs. the placebo program. The measures shown here reflect emotional/interoceptive awareness and mindfulness tendencies, and include the LEAS, MAIA, FFMQ, and MEAQ total scores. Relative to the placebo program, those in the emotional skills programs showed adaptive improvements in each measure (i.e., increases in mindfulness/awareness and decreases in experiential avoidance on the MEAQ). Red stars denote statistically significant group by time interactions.

For MAIA Total scores, we observed the hypothesized group by time interaction, indicating greater pre-to-post increases for the training program compared to the placebo program: F(1, 344) = 6.05, p = .014, η2=0.02 (see Figure 4). Post-hoc LMEs indicated that this interaction was driven by changes in the Emotional Awareness, Not Distracting, and Self-Regulation subscale scores (see Table 1 for details).

Mindfulness

For FFMQ Total scores, we observed the hypothesized group by time interaction, indicating greater pre-to-post increases for the training program compared to the placebo program: F(1, 339) = 7.78, p = .006, η2= 0.02 (see Figure 4). Post-hoc LMEs indicated that this interaction was driven by changes in the Descriptions subscale scores (see Table 1 for details). Notably, there was also a significant schedule by group by time interaction for the FFMQ Descriptions subscale, indicating that, for the training program (relative to the placebo program), participants who completed the compressed (1-week) schedule showed significantly greater improvements over time than participants who completed the distributed (3-week) schedule (see Table S6 for details).

For MEAQ Total scores, we observed the hypothesized group by time interaction, indicating greater pre-to-post decreases for the training program compared to the placebo program: F(1, 326) = 4.7, p = .031, η2= 0.01 (see Figure 4). Post-hoc LMEs suggested that this was driven by changes in the Distraction Suppression and Distress Avoidance subscales, which showed qualitatively similar (but non-significant) effects for this interaction (ps of .095 and .08 respectively; see Table S6 for details). Notably, there was also a significant schedule by group by time interaction for the Distress Avoidance subscale, indicating that, for the training program (relative to the placebo program), participants who completed the compressed (1-week) schedule showed significantly greater improvements over time than participants who completed the distributed (3-week) schedule (see Table S6 for details).

Emotion Regulation

For ERQ Suppression scores, we observed the hypothesized group by time interaction, indicating greater pre-to-post decreases for the training program compared to the placebo program: F(1, 342) = 5.5, p = .02, η2= 0.02 (see Figure 5). There was not a significant group by time interaction for ERQ Reappraisal scores.

Figure 5.

Means and standard errors illustrating change from pre- to post-training in the emotional skills program vs. the placebo program. The measures shown here reflect emotion regulation and socio-emotional skills, and include the ERQ and DERS total scores, the MEOS scale measuring poor socio-emotional skills, and PAT facial emotion recognition (Faces-Words) scores. Relative to the placebo program, those in the emotional skills programs showed adaptive improvements in each measure. This includes reduced emotion regulation difficulties and suppressive strategies, reductions in measures of poor socio-emotional skills, and improved emotion recognition (although the latter finding did not survive correction for multiple comparisons). Red stars denote statistically significant group by time interactions.

For DERS Total scores, we observed the hypothesized group by time interaction, indicating greater pre-to-post decreases for the training program compared to the placebo program: F(1, 334) = 5.78, p = .017, η2= 0.02 (see Figure 5). Post-hoc LMEs indicated that this interaction was driven by changes in the Awareness and Clarity subscale scores (see Table 1 for details).

Socio-Emotional Skills

For MEOS Poor Skills, we observed the hypothesized group by time interaction, indicating greater pre-to-post decreases for the training program compared to the placebo program: F(1, 352) = 6.11, p = .014, η2 = 0.02 (see Figure 5). As only the Poor Skills subscale of the MEOS was hypothesized to reveal a significant group by time interaction, exploratory LMEs were conducted to examine the other five MEOS subscale scores, while correcting for multiple comparisons (p = .01). These revealed greater pre-to-post decreases for the Conceal and Divert subscale scores in the training program compared to the placebo program; however, only the Divert subscale results survived corrections for multiple comparisons (see Table 1 for details).

For PAT Total scores, we did not observe the hypothesized group by time interaction. Because PAT Total did not reveal a significant group by time interaction, we chose to conduct exploratory LMEs examining the four subscales while correcting for multiple comparisons (p = .0125). An LME of PAT Faces-Words showed a significant group by time interaction, indicating that scores increased pre-to-post to a greater degree in the training program than in the placebo program (see Figure 5); however, this did not survive corrections for multiple comparisons (see Table 1).

For STEM-B Total scores, we did not observe the hypothesized group by time interaction.

Comparison to 6-month Follow-up

Here, we conducted LMEs analogous to those in the previous section, but where the time factor included baseline, post-training, and 6-month follow-up. These analyses were restricted to the ERQ, DERS, MEOS, and STEM-B, as the other measures were unable to be collected during this third visit due to restraints related to the COVID-19 pandemic. Table 2 reports effects of group, time, and their interaction for all primary outcome variables. For detailed presentation of all other main effects and interactions, see Table S7 in Supplementary Materials. We also conducted intra-class correlation (ICC) analyses between post-training and follow-up to assess the stability of improvements over the 6 months following program completion.

Table 2.

Results from linear mixed effects models (LMEs) including baseline, post-training, and 6-month follow-up (in subset of individuals with available data at all time timepoints).

	F (df)	p	ηp2	B [CI]	Post-Hoc Contrasts for Group by Time Interaction
ERQ Suppression
Time	0.81 (2, 443)	0.444	0	T2 = −0.19 [−0.58, 0.2] T3 = 0.28 [−0.29, 0.84]	Nonsignificant interaction
Group	0 (1, 433)	0.996	0	−0.23 [−0.81, 0.35]
Group by Time*	2.64 (2, 44)	0.072	0.01	T2 = −0.24 [−0.63, 0.15] T3 = −0.14 [−0.71, 0.42]
ERQ Reappraisal
Time	1.03 (2, 468)	0.359	0	T2 = −0.21 [−0.71, 0.28] T3 = 0.44 [−0.27, 1.15]	Nonsignificant interaction
Group	0.22 (1, 428)	0.636	0	0.28 [−0.33, 0.9]
Group by Time*	0.7 (2, 470)	0.496	0	T2 = −0.08 [−0.57, 0.41] T3 = 0.31 [−0.4, 1.02]
DERS Total
Time	11.93 (2, 429)	<0.001	0.05	T2 = −3.2 [−4.57, −1.83] T3 = 3.05 [1.07, 5.04]	Training - Placebo T1: 2.12, t(529) = 0.9, p = 0.32 T2: −1.77, t(642) = −0.77, p = 0.44 T3: −7.55, t(798) = −2.28, p = 0.02
Group	0.01 (1, 435)	0.926	0	−1.88 [−4.24, 0.48]
Group by Time*	6.69 (2, 430)	0.001	0.03	T2 = 0.51 [−0.85, 1.88] T3 = −3.33 [−5.31, −1.34]
DERS Awareness **
Time	1.32 (2, 441)	0.268	0.01	T2 = −0.15 [−0.49, 0.18] T3 = 0.15 [−0.33, 0.64]	Training T2-T1: −0.68, t(439) = −2.39, p = 0.017 T3-T2: 0.07, t(437) = 0.16, p = 0.871 T3-T1: −0.61, t(452) =−1.35, p = 0.179 Placebo T2-T1: 0.15, t(436) = 0.49, p = 0.623 T3-T2: 0.57, t(444) = 1.03, p = 0.304 T3-T1: 0.71, t(453) = 1.3, p = 0.195
Group	1.07 (1, 433)	0.302	0	−0.17 [−0.67, 0.34]
Group by Time*	3.19 (2, 442)	0.042	0.01	T2 = −0.02 [−0.35, 0.32] T3 = −0.47 [−0.96, 0.01]
DERS Clarity **
Time	11.53 (2, 437)	<0.001	0.05	T2 = 0.1 [−0.24, 0.44] T3 = −0.34 [−0.57, −0.1]	Training T2-T1: −1.05, t(435) = −5.24, p < 0.001 T3-T2: 0.31, t(433) = 0.97, p = 0.33 T3-T1: −0.74, t(446) = −2.33, p = 0.02 Placebo T2-T1: −0.28, t(433) = −1.37, p = 0.173 T3-T2: 0.48, t(439) = 1.23, p = 0.219 T3-T1: 0.19, t(447) = 0.5, − = 0.614
Group	0.06 (1, 434)	0.813	0	−0.26 [−0.63, 0.11]
Group by Time*	4.32 (2, 437)	0.014	0.02	T2 = −0.04 [−0.28, 0.19] T3 = −0.36 [−0.7, −0.02]
DERS Impulse
Time	4.31 (2, 445)	0.014	0.02	T2 = −0.4 [−0.69, −0.11] T3 = 0.69 [0.26, 1.12]	Training - Placebo T1: 0.06, t(573) = 0.15, p = 0.882 T2: 0.02, t(700) = 0.05, p = 0.964 T3: −1.92, t(752) = −2.89, p = 0.004
Group	0.07 (1, 433)	0.788	0	−0.33 [−0.76, 0.1]
Group by Time*	5.75 (2, 446)	0.003	0.03	T2 = 0.33 [0.04, 0.63] T3 = −0.78 [−1.21, −0.36]
DERS Strategies
Time	9.1 (2, 430)	<0.001	0.04	T2 = −0.88 [−1.28, −0.49] T3 = 0.94 [0.36, 1.51]	Training - Placebo T1: 0.33, t(531) = 0.54, p = 0.592 T2: −0.38, t(645) = −0.57, p = 0.566 T3: −1.97, t(795) = −2.06, p = 0.039
Group	0.01 (1, 435)	0.943	0	−0.51 [−1.19, 0.17]
Group by Time*	4.02 (2, 431)	0.019	0.02	T2 = 0.24 [−0.15, 0.64] T3 = −0.9 [−1.47, −0.32]
MEOS Poor Social Skills
Time	3.42 (2, 462)	0.034	0.01	T2 = −0.22 [−0.46, 0.02] T3 = 0.24 [−0.1, 0.58]	Trainif T2-T1: −0.55, t(454) = −2.7, p = 0.007 T3-T2: 0.67, t(459) = 2.08, p = 0.038 T3-T1: 0.13, t(480) = 0.39, p = 0.695 Placebo T2-T1: 0.14, t(45) = 0.68, p = 0.494 T3-T2: 0.72, t(470) = 1.83, p = 0.068 T3-T1: 0.86, t(484) = 2.22, p = 0.027
Group	0.14 (1, 430)	0.712	0	−0.06 [−0.36, 0.25]
Group by Time*	3.09 (2, 463)	0.047	0.01	T2 = −0.08 [−0.32, 0.16] T3 = −0.16 [−0.5, 0.18]
MEOS Divert ***
Time	13.37 (2, 446)	<0.001	0.06	T2 = 0.09 [−0.15, 0.32] T3 = −0.71 [−1.05, −0.36]	Training T2-T1: −0.87, t(442) = −4.3, p < 0.001 T3-T2: −0.15, t(442) = −0.47, p = 0.641 T3-T1: −1.02, t(458) = −3.19, p = 0.002 Placebo T2-T1: −0.1, t(439) = −0.47, p = 0.641 T3-T2: −1.59, t(449) = −4.06, p < 0.001 T3-T1: −1.69, t(460) = −4.33, p < 0.001
Group	0.17 (1, 432)	0.676	0	0.21 [−0.13, 0.55]
Group by Time*	5.31 (2, 446)	0.005	0.02	T2 = −0.38 [−0.61, −0.14] T3 = 0.38 [0.04, 0.72]
STEM-B Total
Time	28.38 (2, 483)	<0.001	0.11	T2 = 0.02 [0.01, 0.03] T3 = −0.01 [−0.02, 0.01]	Training T2-T1: .04, t(463) = 5.71, p < .001 T3-T2: −.007, t(485) = −.58, p = .564 T3-T1: .03, t(514) = 3.07, p = .002 Placebo T2-T1: .035, t(457) = 4.75, p < .001 T3-T2: −.046, t(502) = −3.33, p < .001 T3-T1: −.011, t(520) = −.78, p = .436
Group	7.35 (1, 423)	0.007	0.02	0.02 [0.01, 0.03]
Group by Time*	3.01 (2, 484)	0.05	0.01	T2 = 0 [−0.01, 0] T3 = 0.02 [0, 0.03]

Note: Effects of time and group were treatment coded (i.e., T1/baseline = 0, T2/post-training = 1, T3/follow-up = 2; placebo program = 0, training program = 1). Coefficients and significance tests for other predictors in each model are shown in Supplementary Materials. Age and IQ were centered. Sex and Schedule were sum coded (i.e., males = −1, females = 1; distributed = −1, compressed = 1), such that coefficients for other predictors were interpretable as main effects centered between levels of these variables.

^*B [CI] for interaction effect refers to training program*time point.

^**All post-hoc contrasts by time point (i.e., Training - Placebo) were non-significant.

^***Survived correction for multiple comparisons, as effects were not hypothesized a priori.

Emotion Regulation

For ERQ Suppression scores, we did not observe the hypothesized group by time interaction. However, given that this effect was qualitatively similar to that observed in the larger sample above (but non-significant; p = .072), and that it was significant in analyses of pre- to post-training training effects in the larger sample, we also chose to examine the consistency of these scores over time. ICCs for scores between post-training and follow-up suggested that they remained stable after training for both groups (training program: ICC = 0.67, 95% CI [0.49, 0.79], p < .001; placebo program: ICC = .58, 95% CI [0.32, 0.76], p < .001). Similar to the results above (comparing pre- to post-training scores in the full sample), there was not a significant group by time interaction for ERQ Reappraisal scores.

For DERS Total scores, we observed the hypothesized group by time interaction, indicating that scores at follow-up were significantly lower for the training program when compared to the placebo program: F(2, 430) = 6.69, p = .001, η2= 0.03 (see Figure 6). Post-hoc LMEs indicated that this interaction was driven by changes in the Awareness, Clarity, Impulse, and Strategies subscale scores (see Table 2 for details). DERS Total scores between post-training and follow-up appeared stable over time in both groups (training program: ICC = 0.72, 95% CI [0.56, 0.83], p < .001; placebo program: ICC = 0.68, 95% CI [0.41, 0.83], p < .001). This consistency was also evident for the associated subscales with ICCs in either the excellent, good, or fair range: Awareness (training program: ICC = 0.70, 95% CI [0.48, 0.83], p < .001; placebo program: ICC = 0.70, 95% CI [0.48, 0.83], p < .001), Clarity (training program: ICC = 0.48, 95% CI [0.25, 0.66], p < .001; placebo program: ICC = 0.75, 95% CI [0.58, 0.86], p < .001), Impulse (training program: ICC = 0.65, 95% CI [0.46, 0.78], p < .001; placebo program: ICC = 0.58, 95% CI [0.32, 0.76], p < .001), and Strategies (training program: ICC = 0.57, 95% CI [0.37, 0.73], p < .001; placebo program: ICC = 0.68, 95% CI [0.44, 0.82], p < .001). Notably, there was a suggestive, but non-significant, sex by group by time interaction for the DERS Clarity subscale (p = .055), indicating that males in the training program trended toward lower scores than males in the placebo program at follow-up (i.e., while this pattern was not observed in females; see Table S6).

Figure 6.

Means and standard errors illustrating change from baseline to 6-month follow-up in the emotional skills program vs. the placebo program. The measures shown here reflect emotion regulation and socio-emotional skills, including the DERS and STEM-B total scores at 6-month follow-up for participants who completed all three visits. Relative to the placebo program, those in the emotional skills program maintained adaptive improvements in each measure (reductions in emotion regulation difficulties and increased abilities to manage others’ emotions). The DERS result reflected an increase in emotion regulation difficulties in the placebo group at 6-month follow-up, while such difficulties remained low in the emotion skills training group. The STEM-B result was present pre-to-post in both groups, but it declined to baseline levels at 6-month follow-up in the placebo group. As the 6-month follow-up occurred during the COVID-19 pandemic, these results could suggest protective effects of the training program. Asterisks indicate significant differences in post-hoc contrasts, although some are omitted for clarity (see Table 2 for further details).

Socio-Emotional Skills

For MEOS Poor Skills, we observed the hypothesized group by time interaction, indicating that, in the training program, there was a significant decrease in scores from baseline to post-training (but which subsequently increased from post-training to follow-up), while those in the placebo group showed consistent increases over time: F(2, 462) = 3.42, p = .034, η2 = 0.01. ICCs for MEOS Poor Skills also suggested that scores between post-training and follow-up were stable over time, although only at a fair level in the training group (training program: ICC = 0.40, 95% CI [0.15, 0.60], p = .001; placebo program: ICC = 0.77, 95% CI [0.6, 0.87], p < .001). As only the Poor Skills subscale of the MEOS was hypothesized to reveal a significant group by time interaction, exploratory LMEs were conducted to examine the other five MEOS subscale scores, while correcting for multiple comparisons (p = .01). These revealed a significant group by time interaction for the Divert subscale. This indicated a significant decrease from pre- to post-training for those in the training program that remained stable at follow-up, while no change from pre- to post-training was observed in the placebo group (however, there were significant decreases in the placebo group from post-training to follow-up; see Table 2 for details). These Divert scores also showed consistency between post-training and follow-up in ICC analyses (training program: ICC = 0.68, 95% CI [0.51, 0.80], p < .001; placebo program: ICC = 0.74, 95% CI [0.52, 0.86], p < .001). As MEOS Conceal scores did show the expected group by time interaction in the full sample, we note here that ICCs also reflected moderate stability in this measure over time for those with follow-up data (training program: ICC = 0.62, 95% CI [0.43, 0.76], p < .001; placebo program: ICC = 0.58, 95% CI [0.32, 0.76], p < .001).

For STEM-B Total scores, we observed the hypothesized group by time interaction, indicating that, in both programs, scores increased from pre- to post-training, but that scores decreased at 6-month follow-up in the placebo group while remaining stably higher in the training program group (see Figure 6): F(2, 484) = 3.01, p = 0.05, η2=0.01. STEM-B Total scores also showed fair ICCs between post-training and follow-up (training program: ICC = 0.49, 95% CI [0.26, 0.67], p < .001; placebo program: ICC = 0.53, 95% CI [0.26, 0.73], p < .001).

For the interested reader, correlations between all primary outcome variables at each of the three time points have been included in Supplementary Materials (Figure S13).

Discussion

In this paper, we have provided further evidence supporting the effectiveness of a previously validated web-based training program designed to develop and improve a range of socio-emotional skills that broadly fall under the umbrella concept of emotional intelligence. In a previous paper, we laid the foundations for this work by showing that the program led to significant increases in standard measures of trait and ability models of emotional intelligence (Persich, et al., 2021a). Here, we extended these findings to include a much broader range of emotional skills. Four primary areas of focus of this training program are improving interoceptive/emotional awareness, mindfulness, socio-emotional skills, and emotion regulation. We have shown that, compared to a placebo control program, this training program was successful in improving facets within each of these domains. This included increased scores on both a performance-based measure of emotional awareness and a self-report measure of interoceptive awareness, improved scores on two self-report measures of mindfulness, self-reported decreases in emotion suppression, and reductions in self-reported difficulties regulating emotions. Individuals who took the training program also self-reported improved abilities to regulate the emotions of others. We did not find evidence of increases in reappraisal strategies, and improvements in facial emotion recognition were no longer significant after correcting for multiple comparisons.

Post-hoc examination of subscale scores offered additional insights about the specific improvements associated with the training. For example, increases in emotional awareness were specific to the self, while other-focused emotional awareness did not appear to change. Mindfulness improvements appeared strongest for focusing on descriptive qualities of experiences and reduced distraction from attending to distress. These results were consistent with the improvements in interoceptive awareness, which were similarly linked to emotional awareness (better understanding the relation between bodily sensations and emotions) and reduced distraction. Improvements in emotion regulation appeared to mainly reflect reduced suppression strategies, greater acceptance and acknowledgment of negative emotions, and a clearer understanding of emotions. Thus, improvements across these domains had common themes of increases in acceptance, understanding, attention, and awareness.

A small subsample of participants also returned for a 6-month follow-up assessment, which (unintentionally) occurred after the start of the COVID-19 pandemic lockdown. While a number of factors prevented full reassessment of all measures, we were able to collect measures of emotion regulation and social abilities. Intraclass correlations between post-training and 6-month follow-up scores for these measures were significant and in the moderate range – suggesting at least partial retention of emotional skills acquired during the training. In contrast, there was a significant interaction in which the placebo group, but not the training group, showed increased emotion regulation difficulties and reduced socio-emotional management abilities at 6-month follow-up. This suggests that the training program may have conferred a protective effect in the face of emotional challenges resulting from the pandemic. Notably, our previous study with this sample also demonstrated protective effects against development of depression and anxiety symptoms after onset of the COVID-19 pandemic (Persich, et al., 2021b).

Constraints on Generality

One potential limitation with respect to interpretability is that there were significant (but low effect size) differences between the training group and the placebo group on two measures at baseline, indicating that the training group initially possessed somewhat greater difficulties with awareness-related emotion regulation abilities, but also greater skills at managing others’ emotions. As the training program improved these awareness-based emotion regulation abilities, it is possible that this was influenced by greater room for improvement. In contrast, the training program did not improve emotion management abilities pre- to post-training as expected (although it did appear beneficial at 6-month follow-up), which might suggest that potential for improvement was limited by these higher baseline scores. Future research will be necessary to examine these possibilities.

There was also significant attrition from baseline to post-training assessments (about 27%), and results showed that participants who didn’t drop out had greater baseline performance on specific measures of mindfulness, emotional awareness and regulation, and socio-emotional skills. It is possible that individuals with lower emotional skills had less motivation to complete a long online training program. However, it is possible that those individuals could be successful with the program if it were to be delivered in a different context. For example, as more participants completed the program in the compressed 1-week condition, this suggests we might successfully retain individuals with lower baseline emotional skills in this format. Improvements in some mindfulness measures were also greater in the 1-week condition. If such individuals benefited more from the training, this might also improve effect sizes. This will be an important direction for future work.

Another important limitation is that measures of mindfulness and emotional awareness could not be assessed at 6-month follow-up. As mindfulness skills in particular tend to emerge and strengthen with longer-term practice (Baer et al., 2008), it will be important to assess the potential long-term improvements and stability of improvements in these measures in future work. It is also worth noting the large number of analyses conducted in this study, which raises concerns related to multiple comparisons. A replication study will therefore be needed to identify consistent effects. With that said, the observed effects appear to indicate a convergent pattern of improvement across several related abilities targeted by the program, which affords some added confidence that they are not simply chance effects. Additionally, all effect sizes observed for the significant group by time interactions were either small or small/medium; thus, the magnitude of real-world benefit offered by this training program remains to be determined.

A further limitation of this study was the lack of diversity in our sample. The majority of participants were females (72.09%) and students (69.87%), making it unclear whether results will generalize to other populations. It is possible, for example, that moderating effects of sex could have been detected in a sample possessing a more even ratio of males to females. Additionally, participants were primarily white (60.71%); as such, generalizability to other racial and ethnic groups remains to be established.

With these limitations in mind, our findings may be of clinical significance. As reviewed in the introduction, deficits in emotional awareness, most often assessed using the LEAS, have been linked to multiple somatic and emotion-related disorders, as well as other problems with healthy socio-emotional functioning (for a recent review, see (Lane & Smith, 2021)). There is also literature supporting the role of emotion recognition abilities in vulnerability to depression (e.g., see (Collin et al., 2013; Nyquist & Luebbe, 2020)). In addition, psychoeducation tools promoting emotional awareness are an important component of most evidence-based psychotherapies (Burum & Goldfried, 2007), and specific clinical interventions designed to improve emotional awareness have been found effective in reducing emotion dysregulation and improving symptoms of chronic illness (Farnam et al., 2014; Thakur et al., 2017). Improvements in mindfulness due to other interventions have also been associated with improved physical and emotional functioning (e.g., see (Bartlett et al., 2019; Nadler et al., 2020; Schumer et al., 2018; Suleiman-Martos et al., 2020)). Finally, each of the skills targeted by our program are motivated by those taught within evidence-based psychotherapies shown to be of clinical benefit (Barlow et al., 2016; Hayes et al., 2003; Sakiris & Berle, 2019).

Conclusion

In summary, the ability of our training program to increase scores on standard measures of socio-emotional skills appears promising. Although the training program was not designed as a clinical intervention, it is informed by evidence-based psychotherapeutic approaches, and the web-based nature of the program means that it can be widely accessible to a broad population that may benefit from the training. In healthy individuals (who could still be at risk for developing psychopathology), improving these skills might be protective against, and reduce the chances of, developing emotional pathology when encountering difficult future life events. As mentioned above, we recently showed that this same program was protective against increases in depression, suicidal ideation, and anxiety during the early months of the COVID-19 pandemic (Persich, et al., 2021b). For those already suffering from symptoms of emotional disorders, the skills developed in the program might also aid in symptom reduction and offer tools for dealing with symptoms more adaptively. Future longitudinal studies in at-risk populations and in individuals with emotional disorders will be an important next step in further assessing the utility of this training program.