Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2024 Mar 1.
Published in final edited form as: Anxiety Stress Coping. 2022 Feb 8;36(2):214–228. doi: 10.1080/10615806.2022.2033973

Perceived Emotion Invalidation Predicts Daily Affect and Stressors

Melissa J Zielinski a,b, Jennifer C Veilleux b, Marley F Fradley a, Kayla D Skinner b
PMCID: PMC9357853  NIHMSID: NIHMS1774469  PMID: 35135399

Abstract

Background and Objectives:

Perceived emotion invalidation is linked to the development or worsening of a variety of emotional and physical health conditions. However, prior studies are largely cross-sectional and whether there are day-to-day effects of generally feeling invalidated is unknown.

Design:

We examined the relations between perceived emotion invalidation and momentary affect, average daily affect, and the experience of daily stressors among a sample of young adults using ecological momentary assessment (EMA).

Methods:

Participants (n = 86) completed measures of perceived emotion invalidation and emotional reactivity at baseline then completed one week of EMA including: 1) 7x/day reports of current affect and social context and 2) 1x/day index of experienced stressors and their intensity.

Results:

Higher perceived emotion invalidation predicted lower momentary positive affect. Perceived invalidation also interacted with social context such that higher emotion invalidation predicted greater negative affect when participants were with non-close others (i.e., co-workers, acquaintances). Only participants with high perceived emotional invalidation experienced increased stress alongside heightened daily negative affect.

Conclusions:

These results provide preliminary evidence that feeling emotionally invalidated may predict affective experiences, including how emotions are momentarily experienced and how life stressors are interpreted when they are later reflected on.

Keywords: emotion invalidation, ecological momentary assessment, daily stress, social context

Introduction

Although emotions are private, internal experiences, and emotion regulation can certainly occur via intrapersonal processes (Kappas, 2013; Shuman, 2013), emotions are also inherently social (Mesquita & Boiger, 2014; Niedenthal & Brauer, 2012; Shackman et al., 2018). Emotions are elicited and influenced by interactions with others (Mesquita & Boiger, 2014), and facilitate communication (Rothman & Magee, 2015). Emotions also lead people to seek social interactions (Carver et al., 1989; Williams et al., 2018)–often ones in which emotional experiences are shared and processed. Some people have social networks that are likely to respond to their emotions in ways that make them feel cared for, supported, and validated (Williams et al., 2018). Others have social networks that are much more likely to make them feel unsupported, ostracized, and ultimately invalidated.

Emotion invalidation is defined as interactions that communicate and/or are perceived to communicate that an individual’s emotions or affective experiences are unacceptable, wrong, or inappropriate (Zielinski & Veilleux, 2018). The experience of emotion invalidation is nested within the perceptions of each person. The verbal or nonverbal cues that one person finds invalidating another person might not; even the same person may interpret cues as invalidating or not depending on social context and/or their own current affective state. The degree of validation or invalidation perceived in a given interaction can also vary substantially and is a function of both external/observable behaviors and individual differences that influence appraisals (e.g., emotional reactivity). Over time, people develop a sense of what they can generally expect from their social networks if they share emotions (Zielinski & Veilleux, 2018). If emotion invalidation is characteristic, they may 1) change their emotional responses and behavior accordingly (e.g., increasing emotional intensity to get needs met, as is the case with the development of emotion dysregulation; Linehan, 1993) and/or 2) become increasingly depleted and unable to cope with new stressors by the presence of invalidation as a chronic stressor. Indeed, the overall degree to which one’s social network is perceived as characteristically invalidating is associated with emotion dysregulation and distress (Zielinski & Veilleux, 2018). Chronic stress, more generally, is associated with 1) exposure to even more stressors and 2) increased reactivity to and distress by daily stressors due to depletion of resources (Almeida, 2005). Chronic invalidation may operate similarly and also be associated with greater attention to and reactivity to social stress, in particular, given that its inherently social nature.

Together, there is a potential effect of the perceptions that one is chronically invalidated on emotional responding that goes beyond the effects that would be observable in a singular instance of invalidation; however, this has yet to be explored in existing research. Here, we extend existing research by examining one piece of this literature gap. Specifically, we investigated how the degree to which people perceive experiencing emotion invalidation predicts day-to-day experiences of emotions and stressors, while controlling for self-reported emotional reactivity as a relevant individual difference. We briefly describe the existing literature on emotion invalidation as background before describing the present study.

Existing Research on Emotion Invalidation

Despite its theorized importance in many health ailments, research on invalidation generally has been limited to 1) cross-sectional, observational studies of momentary responses to emotion invalidation in which whether invalidation occurred was defined by the observer – typically a research assistant and 2) studies of possible associations between retrospectively assessed childhood invalidation with indices of current emotional function. These observational and self-report studies found associations between emotion invalidation and a wide range of pathology including eating disorder symptoms (Haslam et al., 2008, 2012; Mountford et al., 2007), rheumatic diseases (Cano et al., 2012; Kool et al., 2010, 2013; Kool & Geenen, 2012), pain (Linton et al., 2012), marital dysfunction (Cano et al., 2008, 2009; Markman & Hahlweg, 1993), and possibly intergenerational transmission of emotion dysregulation (Buckholdt et al., 2014).

However, the results of studies that have examined the acute effects of experimentally manipulated and spontaneously observed invalidation have been more mixed. One study of chronic pain patients experimentally manipulated emotion invalidation during a pain-inducing task (Linton et al., 2012). The researchers found that participants who were invalidated had significantly lower positive affect and significantly higher worry after the series of pain trials compared to participants who had their experience validated. Negative affect decreased across the trials, regardless of the condition to which participants were assigned. A similar study, which experimentally manipulated responses to people seeking treatment for pain, found that invalidation was actually associated with decreases in some negative emotions (sadness, pain); however, invalidated participants also did not show the decreases in other negative emotions (anger, frustration) that participants who were validated experienced (Vangronsveld & Linton, 2012). Studies of spontaneous communication among couples and families have also been mixed, with some finding no relation between invalidating communication and internalizing symptoms (Leong et al., 2011; Shenk & Fruzzetti, 2013) and others finding significant associations (Cano et al., 2012; Cline et al., 2006) and/or associations with externalizing symptoms instead (Crowell et al., 2013; Shenk & Fruzzetti, 2013).

Emotion Invalidation as Emotional Vulnerability

Together, existing work provides evidence that perceived emotion invalidation can have a meaningful influence on one’s emotions—both in response to discrete instances of invalidation (as in laboratory studies) as well as when overall perceptions of the degree to which one experiences invalidation are considered as predictors of distress and psychopathology. However, the mixed literature on outcomes of momentary invalidation caused us to wonder whether a general sense that one’s environment is invalidating may be a predictor of one’s emotional experiences—possibly because perceiving one’s environment as generally emotionally invaliding could increase vulnerability to future emotions and events.

Research on the intersections of resilience/vulnerability factors and stress (e.g., Almeida, 2005) has similarly hypothesized that psychosocial vulnerability can stem from chronic stress and influence stressor exposure (characteristics and appraisals) and reactivity. Our hypothesis is also consistent with meta-analytic findings on related constructs such as rejection (Blackhart, Nelson, Knowles, & Baumeister, 2009) and in line with research on emotional inertia (Koval & Kuppens, 2012). For example, people who feel generally invalidated may experience their emotions as more negative and less positive, expecting that others around them will punish or dismiss how they feel. The experienced intensity of daily life events may also increase due to anxious anticipation of further invalidation. In other words, anticipating invalidation may preemptively raise stress levels—thereby increasing risk for greater overall negative affectivity and thus a greater likelihood of experiencing daily life events as more stressful. Moreover, dysfunctional responding may also be more likely to occur in social (versus non-social) situations among those likely to have experienced more chronic invalidation (e.g., people with borderline personality features; Dixon-Gordon, Fitzpatrick, & Haliczer, 2021)—thus raising the question of whether social context may influence the degree to which perceived emotion invalidation may influence day-to-day affect and stress.

The Current Study

Ecological momentary assessment (EMA), which can be used to collect nuanced assessments of momentary emotions and daily life events through repeated prompts (Shiffman, 2014; Trull & Ebner-Priemer, 2013), is well-suited for investigating fundamental questions about how overall perceptions of emotion invalidation are linked to how emotions and stressors are experienced on a day-to-day basis. Here, we use EMA technology to begin to examine if and how the degree to which people perceive experiencing emotion invalidation predicts emotional states and stressors in daily life. Each model was tested with and without including an index of self-reported trait emotional sensitivity given that it is possible that perceived invalidation could be heightened simply due to some people seeing themselves as more emotionally sensitive rather than explicitly feeling emotionally invalidated by others. Our primary research questions were:

  1. Does perceived emotion invalidation intensity predict positive and/or negative emotion intensity?

  2. Does perceived emotion invalidation intensity predict the number and intensity of daily stressors?

These research questions were examined at both the group average level (i.e., over the one-week measurement period) and at the momentary level (i.e., using nested models). We hypothesized that perceived emotion invalidation would predict daily negative affect intensity, as well as the intensity of stressors broadly. However, because emotion invalidation is a social process, we also examined whether perceived emotion invalidation predicted affective intensity during social stressors, in particular, as a secondary research question and expected that emotion invalidation would predict social stressor intensity more strongly than non-social stressors.

Methods

Sample size was determined a priori at a target of 80 participants. We recruited 100 with the idea that some participants would need to be excluded due to low response rate, consistent with other EMA studies (Shackman et al., 2018). With at least 80 participants completing responses for 7 days with 6-7 responses per day, we anticipated around 3500 total observations, which would allow for sufficient examination of variability over time and across people.

Participants

Participants for the current study were students from the undergraduate psychology subject pool. They were recruited via an initial pre-screening where they completed the Scale of Positive and Negative Experiences (SPANE; Diener et al., 2010). Scores on the SPANE affect balance scale (SPANE positive affect minus SPANE negative affect) were used to classify students into three strata to represent (a) large positive-to-negative balance (scores 10 or above), (b) moderately high positive balance (scores +1 to +9) and (c) equal or greater negative than positive affect (scores 0 or below). Recruitment efforts then focused on obtaining approximately equal number of students in each strata, with the intention of examining affect balance as a predictor of daily mood in a separate manuscript (Veilleux et al., 2020). This recruitment method ensured the sample had adequate variability in typical baseline affect.

In total 98 participants completed the study. However, after evaluating response rates for the daily prompts we excluded 13 people with a response rate of less than 50%, as these participants were not adherent to the EMA protocol. The final sample (n = 86) was 76.8% women, 78% White (9.8% Latino/a, 3.7% African American, 2.4% African-American, and 6.1 other), with an average age of 19.21 (SD = 1.95). There were no demographic differences between those excluded and those retained in the final analysis.

Measures

Perceived Emotion Invalidation.

The Perceived Invalidation of Emotion Scale (PIES; Zielinski & Veilleux, 2018) is a ten-item measure assessing the degree to which people feel that others judge their emotions as incorrect or inappropriate. An example item is “When I share how I’m feeling, others look down on me or judge me.” The PIES is given on a 1 (Almost Never; 0-10%) to 5 (Almost Always; 91-100%) Likert-type scale. The PIES has demonstrated good test-retest reliability, r = .67, p < .01 and concurrent validity with measures of psychopathology and health in previous studies among student and community participants (Zielinski & Veilleux, 2018). Perceived emotion invalidation, as measured by the PIES, was significantly correlated with higher levels of variables relating to psychopathology and lower levels of variables relating to health. In the current study, internal consistency of the PIES was excellent, α = 95.

Emotional Reactivity.

The Emotional Reactivity Scale (ERS; Nock, Wedig, Holmberg, & Hooley, 2008) is a 21-item measure assessing emotional sensitivity (i.e., how easily a person experiences emotion following emotional stimuli), emotional intensity, and how long emotions last. Together these assess the individual’s perception of their emotional vulnerability—the degree to which a person feels they have quick and strong reactions to emotional events. Items are given on a 4-point Likert scale, ranging from 0 (Not at all like me) to 4 (Completely like me). Emotional reactivity overlaps with neuroticism, such that higher scores reflect a tendency to experience emotions more often, more intensely and more persistently than people who score on the lower end. In this study, the internal consistency of the ERS was excellent, α = 95.

EMA Random Momentary Prompts.

At each random prompt, participants were first asked to rate their current mood state on 12 emotional adjectives (Joyful, Lonely, Scared, Numb, Calm, Angry, Nervous, Relaxed, Excited, Sad, Neutral, and Irritable) on a Likert-type scale from 0 (Not at all) to 6 (Extremely). For analysis, the average of negative emotion adjectives (lonely, scared, angry, nervous, sad, irritable; α = .81) and positive emotion adjectives (joyful, excited, relaxed, calm; α = .74) were calculated at each random session. We also calculated person-mean positive and negative affect scores for each person across the entire study. Neutral and numb were assessed but not included in either positive or negative.

Participants were also asked questions assessing their current context including: their current location (home, work, other’s home, bar or restaurant, school, in transit, or other), what they were doing (in class, working, traveling, internet/texting, housework, leisure [movies, tv, friends], exercising, interacting with others, nothing, and other), and whether they had experienced a positive or stressful event since the prior prompt (yes/no). Additional questions about distress tolerance, alcohol consumption, willpower, and emotion regulation strategies were also included but do not relate to the current investigation (see Veilleux et al., 2020).

We also asked participants to report on their social context (alone, spouse or romantic partner, friend, family members, acquaintances/classmates, coworker, and other). Initial frequencies revealed that just under half of the sessions occurred when people were alone (45.6%). Because there were too many categories to realistically analyze separately, and due to the low frequencies in some of the categories (e.g., only 3% of the instances occurred when people were with co-workers), the data were coded in two ways. We created a Closeness variable with three levels, (0) for alone, (1) with a close other (spouse, romantic partner, family members, friends), or (2) non-close others (acquaintances/classmates, coworkers and ‘other’). Initial descriptives suggested that 36.91% of the sessions occurred when participants were with close others, and 17.45% occurred in the presence of non-close others.

EMA Scheduled Nightly Prompts.

Every evening at 9:35pm, participants received a notification for a separate “nightly” session which asked them to reflect upon their day. They first completed a few brief questions asking about any difficulty they had responding to random prompts; these answers did not change their compensation rate but allowed us to monitor problems that individual participants encountered with the protocol. The brunt of the nightly session involved completing the Daily Stress Inventory (DSI; Brantley, Waggoner, Jones, & Rappaport, 1987), a 58-item measure which inquires about commonly occurring, potentially stressful events. Events range from mild (e.g., “misplaced something”) to more significantly stressful (e.g., “experienced narrow escape from danger”), and were rated as 0 (did not occur), 1 (occurred but was not very stressful), 2 (caused a little stress), 3 (caused some stress), 4 (caused much stress), or 5 (caused me to panic). The overall measure was scored in two ways: (a) a count of stressors that occurred (i.e., the number of items on which a person answered anything above 0) and (b) the average intensity from 1-5 for experienced stressors.

For analysis, we separated the DSI into items assessing social and non-social stressors. To ensure that the categorization of social and non-social stressors was validated empirically (rather than determined by the perceptions of the research team), we conducted a separate online study where 51 “workers” from Amazon’s Mechanical Turk were paid $0.65 to rate each of the 58 DSI items as social or not social. The sample (35.3% women, average age 30.96, 64.7% White) was given the definition of social as “involves an interpersonal interaction with another person” and then rated all DSI items as either social or not social. We made the a-priori decision to retain items as “social” when 80% or more of the rating sample indicated the stressor was social, and we likewise retained items as “not social” when 80% or more of the rating sample indicated that stressor was not social. In total, 21 of the items (26.21%) were identified as social, 19 (32.76%) were identified as not social, with the remainder (n = 18) not clearly falling in either category.

We then summed the count of both social and non-social stressors, and calculated average intensity of both social (α = .87) and non-social stressors α = .86) for every eligible nightly session. Note that the intensity scores include intensity only for experienced stressors (for each night, the average intensity rating across all stressors experienced, necessarily ignoring stressors that the person did not experience that day). In addition, we calculated person-mean variables (e.g., averages across the entire study for each person) for the average number of daily stressors on the entire DSI, average daily social stressors, and average daily non-social stressors. We also calculated person-mean intensity scores for the DSI overall, and separately for experienced social and non-social stressors.

Procedure

Participants completed one in-person laboratory session followed by 7 days of EMA on their own cell phone and a short debriefing session. The first portion of the laboratory session consisted of informed consent processes and completing a variety of individual difference measures (including the PIES) and behavioral tasks (see Veilleux et al., 2020 for a fuller description of these tasks, which are not used in the current investigation). The second portion of the laboratory session focused on orientation to EMA. Specifically, participants were guided to download the EMA application LifeData (http://lifedatacorp.com) and completed a brief app-based orientation session which overviewed the questions they would answer during the random and nightly prompts.

The week of EMA began immediately after the laboratory session. Participants received about 7 random notifications per day between 9:30 am and 9:30 pm, all spaced at least 45 minutes apart. At the conclusion of EMA, participants returned for a short debriefing session in which they were informed of the purposes of the study and received compensation (1.5 hours of research credit for the initial session, 2 hours of research credit for 80% or greater compliance on responding to notifications [with less credit for lower response rates], and 0.5 hours of research credit for the debriefing session).

Data for this study was collected between October 2016 and May 2017. All research procedures and measures were approved by the institutional review board at the University of Arkansas under protocol 16-07-005. All participants were consented to pre-screening and to study procedures.

Analytic Strategy

Analysis plan.

We first examined the relationship between PIES and person-mean variables calculated from the EMA sessions. These person-level scores do not account for the nested data structure, but allows a cursory view of the data at the between-subjects level. Because isolating the social and non-social stressors in the DSI was new to this study, we first evaluated differences between social and non-social stressors using paired t-tests. The person-level analyses were conducted in SPSS. We then examined both the between-person and the within-person correlations using the ‘rcorr’ package in R. Results are presented in Table 1.

Table 1.

Zero-order correlations for within- and between-person variables.

1. 2. 3. 4. 5. 6. 7. 8.
1. Emotion Invalidation (PIES) -- -- -- -- -- -- -- --
2. Negative Affect .22* -- −.53** .05* .06** .03 .07** --
3. Positive Affect −.44** −.35** -- −.05** −.05** −.04* −.12** --
4. Count Social Stressors (DSI) .31** .57** −.30** -- .56** .39** .27** --
5. Count Non-Social Stressors (DSI) .28** .56** −.34** .85** -- .32** .37** --
6. Intensity Social Stressors (DSI) .40** .28** −.28** .63** .61** -- .29** --
7. Intensity Non-Social Stressors (DSI) .38** .25* −.41** .44** .58** .78** -- --
8. Emotional Reactivity (ERS) .58** .33** −.26** .27** .28** .50** .50** --
Descriptives for Averaged Variables M (SD) 1.99 (0.99) 0.93 (0.71) 2.64 (0.67) 4.47 (3.35) 5.65 (3.27) 1.53 (0.79) 2.01 (0.80) 32.62 (19.58)
Open in a new tab

Note: Between person correlations below the diagonal; within-person correlations above the diagonal

*

p < .05

**

p < .01.

PIES = Perceived Invalidation of Emotion Scale; DSI = Daily Stress Inventory; ERS = Emotional Reactivity Scale

Then, to examine the role of PIES in predicting affect experienced in daily life, multi-level linear models (MLMs) were used to account for the fact that instances of emotion were nested within people. The primary data analytic strategy used mixed linear modeling via the ‘lme4’ package, with follow-up tests for interactions using the ‘reghelper’ package in R. Effects for linear mixed models are expressed in terms of unstandardized B’s and standard errors (SE). All variables and residuals were examined for normality. Skewness and kurtosis were within acceptable limits for each variable included in the models. Intercepts were free to vary across subjects, but slopes were not random as including them resulted in models failing to converge. Although our central predictions were about negative affect, for sake of inclusiveness we ran models predicting both negative and positive affect separately. All models controlled for time by including the session # within the overall study into the model. Initial models used social context as a Level 1 predictor with contrast coding, where the first contrast code depicted alone versus combined close and non-close others, essentially analyzing the effect of being with someone else or not (Shackman et al., 2018). The second contrast code compared close versus non-close others. These models also included PIES as a person-level predictor (grand mean-centered), and interactions between PIES and each of the contrast codes. After running the initial models, we then ran the same models with ERS included as a co-variate to determine whether any effects of invalidation on affect still held after controlling for self-reported emotional reactivity.

Additional MLM models predicted the experience of daily stress, measured at the nightly sessions. We hypothesized that PIES would predict the intensity of stressors, but not the count of stressors, and that PIES would be particularly useful in predicting intensity of social stress. Six models predicted (a) count of daily non-social stressors, (b) count of daily social stressors (c) count of daily social stressors controlling for count of daily non-social stressors (i.e., the same model as (b) but controlling for non-social stress, (d) intensity of non-social stress experienced, (e) intensity of social stress experienced, and (f) intensity of social stress controlling for intensity of non-social stress (i.e., the same model is (e) but controlling for non-social stress intensity.

In each model, PIES and ERS were entered as Level 2 grand mean-centered predictors. In addition, momentary daily negative affect was entered as a Level 1 predictor (person mean centered), to account for the fact that greater emotionality during the day may predict greater recognition of stressors or reporting of stress intensity assessed at night. Interaction terms between daily affect and PIES were also included in the models to assess if perceived emotion invalidation exacerbated the effect of affect on experience of daily stress. Significant interactions were teased apart by examining simple slopes which evaluated the effect of affect on daily stress at high (one standard deviation above) and below (one standard deviation below) the mean on PIES. All models predicting nightly variables included the day of study in the model to control for any changes in stress reports over the course of the study. In models (c) and (f), the non-social time-varying covariates were entered at Level 1.

Results

Response Rate

The exact number of random sessions varied slightly by person due to: (1) randomness built into the application, which determined the exact number of prompts (i.e., notifications to complete assessment) that were administered over the week of EMA and (2) participant behaviors, which determined the inclusion or exclusion of particular random sessions. We retained random sessions (defined as all of the items given following a prompt to complete the assessment) if the participant responded to the prompt within ten minutes of receiving the notification. A response rate was calculated by counting the number of random sessions retained by the total number of prompts received over the week. Nightly sessions, which did not factor into the response rate for the random sessions, were retained if completed within an hour of the prompt. The final sample of 85 participants had a response rate of 74.69% for random prompts and 85.81% for nightly prompts.

Person Level Data

We first examined the descriptive statistics of the person-level variables (see means and SDs in Table 1). Because the DSI has, to our knowledge, never been used to isolate social and non-social stressors, we examined whether people tended to experience more social or non-social stress using paired samples t-tests. In this study, people generally experienced more non-social stressors than social stressors, t(84) = 5.97, 95% CI [−1.16, −.94], p < .001, d = .65, and likewise found the non-social stressors more negative (i.e., more intense) than the social stressors, t(84) = 8.33, 95% CI [−.48, −.40], p < .001, d = .90.

Zero-order correlations examined the relationships among the study variables, calculated both between-subjects (PIES and average momentary variables) and within-subjects (momentary variables only; Table 1). People who tended to experience more social stressors also experienced more non-social stressors, and experiencing greater numbers of stressors was likewise associated with greater intensity of stress. Higher daily negative affect and lower positive affect was associated with experiencing more stress and higher intensity of stress. Finally, PIES and emotional reactivity were correlated with all of the emotion and stress variables, such that higher perceived emotion invalidation and higher emotional reactivity were associated with greater negative affect and lower positive affect, greater counts of stress and greater intensity of stress (both social and non-social).

Momentary Analyses

The function of these analyses was to examine if perceived emotion invalidation (i.e., PIES scores) predicted the experience of negative affect in daily life. We expected that greater PIES scores would predict higher momentary negative affect, and that this would be stronger for negative affect experienced when in social situations (i.e., around other people). Results (see Tables 2) revealed that people tended to have higher negative affect when alone compared to others, and that negative affect was lower when with close others compared with non-close others. There was an initial main effect of PIES on negative affect, but this effect was non-significant when self-reported emotional reactivity (e.g., ERS) was included in the model. Finally, there was an interaction between PIES and social context when predicting negative affect (see Figure 1). Specifically, higher negative affect was evident when people were with non-close others compared to close others only for those with heightened perceived invalidation (B = .19, SE = .05, t = 3.42, p < .001), not for those with low perceived invalidation (B = .02, SE = .06, t = .38, p = .70).

Table 2.

Multilevel models predicting momentary affect in social context

Outcome Covariate Predictor B (SE) t p d
Negative Affect Time Time .004 (.001) 3.91 <.001 .14
Alone vs Others −.18 (.03) −5.99 <.001 .26
Close vs Non-Close .11 (.04) 2.63 .008 .11
PIES .19 (.08) 2.54 .01 .51
PIES x Alone vs Others .01 (.03) .37 .45 .03
PIES x Close vs Non-Close .09 (.04) 2.08 .04 .05
Time & ERS Time .005 (.001) 3.93 <.001 .14
Alone vs Others −.18 (.03) −6.00 <.001 .27
Close vs Non-Close .11 (.04) 2.66 .008 .11
PIES .03 (.08) .32 .75 .04
PIES x Alone vs Others .01 (.03) .49 .63 .03
PIES x Close vs Non-Close .09 (.04) 2.15 .03 .05
ERS .01 (.004) 3.32 .001 .74
Positive Affect Time Time −.01 (.002) −5.03 <.001 .18
Alone vs Others .22 (.04) 5.51 <.001 .41
Close vs Non-Close −.53 (.06) −9.32 <.001 .03
PIES −.28 (.07) −4.18 <.001 .80
PIES x Alone vs Others <.001 .01 .91 .02
PIES x Close vs Non-Close −.07 (.05) −1.23 .22 .02
Time & ERS Time −.01 (.001) −5.03 <.001 .18
Alone vs Others .22 (.04) 5.52 <.001 .41
Close vs Non-Close −.53 (.06) −9.32 <.001 .03
PIES −.25 (.08) −3.06 .003 .60
PIES x Alone vs Others <.001 −.12 .90 .02
PIES x Close vs Non-Close −.07 (.06) −1.25 .21 .02
ERS −.002 (.004) −.56 .58 .14
Open in a new tab

Note: Contrasts coded as Alone (−1) vs Others (Close and Non-Close .5); Close (−1) vs Non-Close (1). PIES = Perceived Invalidation of Emotion Scale; ERS = Emotional Reactivity Scale.

Figure 1.

Figure 1.

Open in a new tab

Social Context and PIES Predicting Negative Affect

Note: PIES = Perceived Invalidation of Emotion Scale; ERS = Emotional Reactivity Scale. PIES (Level 2 variable) represented on X-axis in this figure because social context (Level 1 variable) is categorial. Although presented this way, we still consider social context the primary predictor with PIES as the moderator.

With regards to positive affect, people reported higher positive affect with others compared to when alone, and higher positive affect with close compared to non-close others. In addition, higher perceived emotion invalidation (PIES) predicted lower momentary positive affect. This effect remained significant even with self-reported emotional reactivity in the model. Moreover, there were no interactions between PIES and context in predicting momentary positive affect.

Predicting Nightly Stressors

Count.

Models predicting stress are reported in Table 3. The first model indicated that greater daily negative affect predicted a higher count of non-social stressors, but that PIES was not a significant predictor when controlling for self-reported emotional reactivity (ERS). Greater self-reported emotional reactivity did predict a greater count of non-social stressors. In addition, there was an interaction between negative affect and PIES in predicting non-social stress (see Figure 2; right panel), even controlling for emotional reactivity. People with greater perceived emotion invalidation experienced a greater count of non-social stressors when experiencing higher daily negative affect (B = .34, SE = .08, t = 4.67, p < .001), whereas there was no significant relationship between negative affect and count of stressors for people low in perceived emotion invalidation (B = .09, SE = .08, t = 1.00, p = .32).

Table 3.

Multilevel models predicting nightly stress reports from PIES and daily negative affect

Outcome Predictor B (SE) t p d
Count Non-Social Stress Time (day of study) −.37 (.02) −15.20 <.001 .60
Negative Affect .22 (.06) 3.56 .001 .14
PIES .43 (.40) 1.09 .28 .24
PIES x Negative Affect .13 (.05) 2.40 .02 .10
ERS .05 (.02) 2.69 .008 .60
Count Social Stress Time (day of study) −.39 (.03) −14.03 <.001 .56
Negative Affect .19 (.06) 2.79 .005 .11
PIES .45 (.38) 1.15 .25 .26
PIES x Negative Affect .13 (.06) 2.18 .03 .09
ERS .06 (.02) 2.95 .004 .66
Intensity Non-Social Stress Time (day of study) −.04 (.007) −5.49 <.001
Negative Affect .06 (.02) 3.54 <.001 .13
PIES .04 (.09) .55 .58 .11
PIES x Negative Affect .01 (.02) .45 .65 1.12
ERS .02 (.004) 4.93 <.001 .02
Intensity Social Stress Time (day of study) −.06 (.008) −7.42 <.001 .30
Negative Affect .03 (.02) 1.75 .08 .07
PIES .11 (.09) 1.18 .24 .26
PIES x Negative Affect .06 (.02) 3.32 <.001 .13
ERS .02 (.004) 4.17 <.001 .94
Open in a new tab

Note: PIES = Perceived Invalidation of Emotion Scale; ERS = Emotional Reactivity Scale.

Figure 2.

Figure 2.

Open in a new tab

Interactions of negative affect and PIES in predicting stressor count

Note: PIES = Perceived Invalidation of Emotion Scale; ERS = Emotional Reactivity Scale.

The model predicting count of social stress found no effect of negative affect or perceived invalidation. There was a significant effect of self-reported emotional reactivity. In addition, there was a similar interaction between negative affect and PIES on count of social stress (see Figure 2; left panel), even controlling for emotional reactivity, such that people with higher perceived emotion invalidation experienced a higher count of social stress alongside greater negative affect (B = .32, SE = .08, t, = 3.87, p < .001) , an effect that was not evident for people with low perceived emotion invalidation (B = .06, SE = .10, t = .60, p = .55). A final model predicting count of social stress added the count of non-social stress as a co-variate to isolate the count of social stress. In this model, where a higher count of non-social stress was a significant predictor of social stress (B = .63, SE = .02, t = 34.81, p = <.001, d = 1.41), self-reported emotional reactivity remained significant (B = .24, SE = .02, t = 2.03, p = .04, d = .46), but the interaction between PIES and negative affect was no longer significant (B = .05, SE = .05, t = 1.06, p = .29, d = .04), confirming that the relationship between greater perceived emotion invalidation and negative affect predicting stressors is not unique to social stress.

Intensity.

Higher negative affect predicted greater intensity of non-social stress (see Table 3), as did greater self-reported emotional reactivity. PIES did not predict greater intensity of non-social stress, and did not interact with negative affect to predict intensity of non-social stress. However, when predicting intensity of social stress, there was a significant interaction between PIES and negative affect (see Figure 3). People with high emotion invalidation experienced more intense social stress alongside greater negative affect (B = .09, SE = .02, t = 3.85, p <.001), whereas there was no relationship between negative affect and intensity of stress for people with low perceived emotion invalidation (B = −.02, SE = .03, t = −.82, p = .41).

Figure 3.

Figure 3.

Open in a new tab

Negative Affect and PIES Predicting Intensity of Social Stress

Note: PIES = Perceived Invalidation of Emotion Scale; ERS = Emotional Reactivity Scale.

The final model added the intensity of non-social stress to the prior model to predict intensity of social stress, thus isolating the intensity specific to social stress. Intensity of non-social stress was predictive of intensity of social stress (B = .32, SE = .02, t = 16.21, p = <.001, d = .64), self-reported emotional reactivity remained significant (B = .01, SE = .001, t = 3.17, p = .002, d = .71), and the interaction between PIES and negative affect also remained significant (B = .06, SE = .02, t = 3.24, p < .001, d = .13).

Discussion

Interpersonal environments that are perceived as emotionally invalidating are linked to long-term poor health outcomes and psychopathology (Crowell et al., 2009; Slavich & Cole, 2013). Here, we considered whether perceiving one’s environment as generally invalidating was a predictor of one’s proximal emotional experiences and stressors. We found evidence that feeling emotionally invalidated predicted affective experiences, including both how emotions are experienced in the moment and how life stressors are interpreted when reflecting on them later in the day. However, not all hypotheses were supported and modeling for self-reported emotional reactivity reduced the predictive power of perceived invalidation.

Relations between perceived emotion invalidation and momentary negative emotion were initially significant, but were more limited after controlling for self-reported emotional reactivity. Specifically, higher perceived emotion invalidation interacted with social context such that higher emotion invalidation predicted greater negative affect when participants were with non-close others only. However, people with greater perceived emotion invalidation reported a greater number of stressors (both social and non-social) and greater social stress intensity when experiencing higher daily negative affect as well. These relations may be explained by greater use of cognitive processes such as rumination, which have been strongly associated with the types of clinical concerns that have been associated with experiencing emotion invalidation (Ehring & Watkins, 2008; Nolen-Hoeksema et al., 2008), when distressed. People with higher levels of emotion invalidation may have also been sensitized to dwell on negative or ambiguous social interactions, anxiously anticipating invalidation, leading to the relation observed people higher perceived emotion invalidation and social stress intensity. Conversely, it is possible that people who experience higher perceived emotional reactivity and more frequent and/or intense emotional stressors may also strain their social relationships to a greater degree—requiring more social support from partners and perhaps more elicitation of emotion invalidation. More research is needed on this topic.

The examined relations between perceived emotion invalidation and positive affect were more robust. Higher perceived invalidation had a large “dampening” effect on positive emotion—regardless of context and while controlling for self-reported emotional reactivity. Maintaining a state of lower positive affectivity could be viewed as a more defensive emotional stance in which emotions are avoided to reduce the likelihood of emotion invalidation disrupting one’s emotional state (i.e., because negative emotions are by default more congruent with the expected results of momentary invalidation). This finding could potentially explain the co-occurrence of emotion invalidation, conditions associated with invalidation (e.g., borderline personality disorders), and depressive symptomatology (c.f. Yap, Allen, & Ladouceur, 2008).

Together, our results aligned with our prediction that perceiving one’s environment as invalidating can serve as an emotional vulnerability factor, particularly in predicting lower positive affect. Relations with negative affect are more complex but present. This study incrementally contributes to the literature through its focus on overall invalidation, instead of invalidation within specific interactions. Our significant findings should be used to inform existing frameworks of interpersonal emotion regulation, which currently emphasize immediate interactions between regulators and targets (Reeck et al., 2016; Zaki & Williams, 2013), also do not account for how social experiences broadly impact emotions in both social and non-social situations. Although we did not assess for specific instances of invalidation, our momentary results did not suggest greater negative affect in social situations thus it is unlikely that our findings are due to instances of acute invalidation (i.e., feeling greater negative affect after an experience of being invalidated by someone else). Importantly, the relation between perceived emotion invalidation, reported affect, and perceived stress were found even in a sample that was selected based on obtaining a distribution of affective balance and after controlling for self-reported emotional reactivity.

Limitations and Future Directions

Our study was limited by reliance on an entirely college student sample which had limited racial and ethnic diversity. Although college students are accessible and accepted participants in similar research studies, their typical social experiences may differ from a broader sample of adults and impact generalizability of findings. Follow-up studies could include a more diverse sample of adults to foster examination of how the consequences of emotion invalidation may differ by content (e.g., microaggressions vs. more overt invalidation) and by emotion beliefs/norms (e.g., acceptability of emotion express). It is also possible that the experiences of daily stressors differ for adults not in college (e.g., parenting, elder caretaking, financial obligations) compared to college students. Our study used a list of common stressors rather than having participants self-nominate particularly stressful and salient events (Almeida, 2005), where the latter may be particularly pertinent for social stress.

Our study design also limits our ability to speak to how emotion invalidation influences emotions and perceptions of daily events. Future research should examine the potential impact of both general perceptions of emotion invalidation and specific instances of occurrence on emotion dysregulation given that influential theories of emotion invalidation (Linehan,1993; Crowell et al., 2013) have hypothesized that emotion invalidation causes emotion dysregulation over time, possibly by making individuals begin to distrust their own emotions and self-invalidate. This extension of our work would be possible through additional EMA studies that capture discrete instances of invalidation and subsequent emotions as well as inquire about broad perceptions of invalidation.

Additionally, dyadic EMA studies could be used to begin to parse apart how behaviorally observed versus perceived emotion invalidation and/or emotional reactivity may converge or diverge in predicting both emotional and behavioral outcomes. Emotional reactivity has been operationalized and measured in many different ways and—while we prioritized individuals’ self-perceptions of their own emotionality—future research could build upon our work by examining emotion invalidation concurrently with other measures (e.g., event-based emotional responses and/or task-based indices). Our team has also explicitly prioritized investigation of perceived emotion invalidation in one’s current social environment given that whether an interaction is interpreted as invalidating (regardless of what can be observed by an outsider) will determine its internal impact on an individual. This perspective is consistent with appraisal theories of emotion (Moors, 2014), which suggest that interpretations of situations are vital in forming an emotional response. However, EMA technology would make it possible to study both concurrently and could help contextualize results of past laboratory studies which have often demonstrated mixed findings regarding the relations between invalidation and emotion dysregulation. Given the growing recognition of the role that interpersonal experiences play in shaping our momentary emotions and affective experiences, recent interpersonal regulation models would benefit from incorporating work that suggests that the regulatory influence of others is not always positive (Reeck et al., 2016; Zaki & Williams, 2013).

Conclusion

In summary, the current study demonstrated that recent perceived emotion invalidation is an individual difference that matters. When people feel that their emotions are judged as unacceptable, wrong or inappropriate, they experience the world differently. Daily emotions are less positive, and stressors are more common and more intense during negative affective states. As an emotional concept with both intrapersonal and interpersonal implications, further understanding of emotion invalidation and its relationship to emotional processes and psychopathology can help inform psychoeducational and treatment approaches that address the social nature of our emotional lives.

Financial Support:

Manuscript preparation was supported by K23DA048162 (PI: Zielinski).

Footnotes

Declaration of Interest: None

References

  1. Almeida DM (2005). Resilience and vulnerability to daily stressors assessed via diary methods. Current Directions in Psychological Science, 14(2), 64–68. 10.1111/j.0963-7214.2005.00336.x [DOI] [Google Scholar]
  2. Blackhart GC, Nelson BC, Knowles ML, & Baumeister RF (2009). Rejection elicits emotional reactions but neither causes immediate distress nor lowers self-esteem: A meta-analytic review of 192 studies on social exclusion. Personality and Social Psychology Review, 13(4), 269–309. 10.1177/1088868309346065 [DOI] [PubMed] [Google Scholar]
  3. Buckholdt KE, Parra GR, & Jobe-Shields L (2014). Intergenerational transmission of emotion dysregulation through parental invalidation of emotions: Implications for adolescent internalizing and externalizing behaviors. Journal of Child and Family Studies, 23(2), 324–332. 10.1007/s10826-013-9768-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Cano A, Barterian J. a, & Heller JB (2008). Empathic and nonempathic interaction in chronic pain couples. The Clinical Journal of Pain, 24(8), 678–684. 10.1097/AJP.0b013e31816753d8 [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Cano A, Leong LEM, Williams AM, May DKK, & Lutz JR (2012). Correlates and consequences of the disclosure of pain-related distress to one’s spouse. Pain, 153(12), 2441–2447. 10.1016/j.pain.2012.08.015 [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Cano A, Leong L, Heller JB, & Lutz JR (2009). Perceived entitlement to pain-related support and pain catastrophizing: associations with perceived and observed support. Pain, 147, 249–254. 10.1016/j.pain.2009.09.023 [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Carver C, Scheier M, & Weintraub J (1989). Assesing coping strategies: a theoretically based approach. Journal of Personality and Social Psychology, 56(2), 267–283. 10.1037//0022-3514.56.2.267 [DOI] [PubMed] [Google Scholar]
  8. Cline RJW, Harper FWK, Penner L. a, Peterson AM, Taub JW, & Albrecht TL (2006). Parent communication and child pain and distress during painful pediatric cancer treatments. Social Science & Medicine (1982), 63(4), 883–898. 10.1016/j.socscimed.2006.03.007 [DOI] [PubMed] [Google Scholar]
  9. Crowell SE, Baucom BR, McCauley E, Potapova NV, Fitelson M, Barth H, Smith CJ, & Beauchaine TP (2013). Mechanisms of contextual risk for adolescent self-injury: invalidation and conflict escalation in mother-child interactions. Journal of Clinical Child and Adolescent Psychology, 42(4), 467–480. 10.1080/15374416.2013.785360 [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Crowell SE, Beauchaine TP, & Linehan MM (2009). A biosocial developmental model of borderline personality: Elaborating and extending Linehan’s theory. Psychological Bulletin, 135(3), 495–510. 10.1037/a0015616 [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Dixon-Gordon KL, Fitzpatrick S, & Haliczer LA (2021). Emotion regulation and borderline personality features in daily life: The role of social context. Journal of Affective Disorders, 282, 677–685. 10.1016/j.jad.2020.12.125 [DOI] [PubMed] [Google Scholar]
  12. Ehring T, & Watkins ER (2008). Repetitive negative thinking as a transdiagnostic process. International Journal of Cognitive Therapy, 1(3), 192–205. 10.1521/ijct.2008.1.3.192 [DOI] [Google Scholar]
  13. Haslam M, Arcelus J, Farrow C, & Meyer C (2012). Attitudes towards emotional expression mediate the relationship between childhood invalidation and adult eating concern. European Eating Disorders Review, 20(6), 510–514. 10.1002/erv.2198 [DOI] [PubMed] [Google Scholar]
  14. Haslam M, Mountford V, Meyer C, & Waller G (2008). Invalidating childhood environments in anorexia and bulimia nervosa. In Eating behaviors (Vol. 9, Issue 3, pp. 313–318). 10.1016/j.eatbeh.2007.10.005 [DOI] [PubMed] [Google Scholar]
  15. Kappas A (2013). Social regulation of emotion: messy layers. Frontiers in Psychology, 4, 51. 10.3389/fpsyg.2013.00051 [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Kool MB, & Geenen R (2012). Loneliness in patients with rheumatic diseases: The significance of invalidation and lack of social support. The Journal of Psychology, 146(1–2), 229–241. 10.1080/00223980.2011.606434 [DOI] [PubMed] [Google Scholar]
  17. Kool MB, Middendorp H. Van, Lumley MA, Bijlsma JW, & Greenen R (2013). Social support and invalidation by others contribute uniquely to the understanding of physical and mental health of patients with rheumatic diseases. Journal of Health Psychology, 18(1), 86–95. 10.1177/1359105312436438 [DOI] [PubMed] [Google Scholar]
  18. Kool MB, van Middendorp H, Lumley MA, Schenk Y, Jacobs JWG, Bijlsma JWJ, & Geenen R (2010). Lack of understanding in fibromyalgia and rheumatoid arthritis: The Illness Invalidation Inventory (3*I). Annals of the Rheumatic Diseases, 69(11), 1990–1995. 10.1136/ard.2009.123224 [DOI] [PubMed] [Google Scholar]
  19. Koval P, & Kuppens P (2012). Changing emotion dynamics: individual differences in the effect of anticipatory social stress on emotional inertia. Emotion (Washington, D.C.), 12(2), 256–267. 10.1037/a0024756 [DOI] [PubMed] [Google Scholar]
  20. Leong LE, Cano A, & Johansen AB (2011). Sequential and base rate analysis of emotional validation and invalidation in chronic pain couples: Patient gender matters. The Journal of Pain, 12(11), 1140–1148. 10.1016/j.pain.2011.04.004 [DOI] [PubMed] [Google Scholar]
  21. Linton SJ, Boersma K, Vangronsveld K, & Fruzzetti A (2012). Painfully reassuring? The effects of validation on emotions and adherence in a pain test. European Journal of Pain, 16(4), 592–599. 10.1016/j.ejpain.2011.07.011 [DOI] [PubMed] [Google Scholar]
  22. Markman H, & Hahlweg K (1993). The prediction and prevention of marital distress: An international perspective. Clinical Psychology Review, 13, 29–43. 10.1016/0272-7358(93)90006-8 [DOI] [Google Scholar]
  23. Mesquita B, & Boiger M (2014). Emotions in context: A sociodynamic model of emotions. Emotion Review, 6(4), 298–302. 10.1177/1754073914534480 [DOI] [Google Scholar]
  24. Moors A (2014). Flavors of appraisal theories of emotion. Emotion Review, 6(4), 303–307. 10.1177/1754073914534477 [DOI] [Google Scholar]
  25. Mountford V, Corstorphine E, Tomlinson S, & Waller G (2007). Development of a measure to assess invalidating childhood environments in the eating disorders. Eating Behaviors, 8, 48–58. 10.1016/j.eatbeh.2006.01.003 [DOI] [PubMed] [Google Scholar]
  26. Niedenthal PM, & Brauer M (2012). Social functionality of human emotion. Annu. Rev. Psychol, 63, 259–285. 10.1146/annurev.psych.121208.131605 [DOI] [PubMed] [Google Scholar]
  27. Nolen-Hoeksema S, Wisco BE, & Lyubomirsky S (2008). Rethinking Rumination. Perspectives on Psychological Science, 3(5), 400–424. 10.1111/j.1745-6924.2008.00088.x [DOI] [PubMed] [Google Scholar]
  28. Reeck C, Ames DR, & Ochsner KN (2016). The social regulation of emotion: An integrative, cross-disciplinary model. Trends in Cognitive Sciences, 20(1), 47–63. 10.1016/j.tics.2015.09.003 [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Rothman NB, & Magee JC (2015). Affective expressions in groups and inferences about members’ relational well-being: The effects of socially engaging and disengaging emotions. 10.1080/02699931.2015.1020050 [DOI] [PubMed] [Google Scholar]
  30. Shackman AJ, Weinstein JS, Hudja SN, Bloomer CD, Barstead MG, Fox AS, & Lemay EP (2018). Dispositional negativity in the wild: Social environment governs momentary emotional experience. Emotion, 18(5), 707–724. 10.1037/emo0000339 [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Shenk C, & Fruzzetti A (2013). Parental Validating and Invalidating Responses and Adolescent Psychological Functioning: An Observational Study. The Family Journal, 22(1), 43–48. 10.1177/1066480713490900 [DOI] [Google Scholar]
  32. Shiffman S (2014). Conceptualizing analyses of ecological momentary assessment data. Nicotine and Tobacco Research. 10.1093/ntr/ntt195 [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Shuman V (2013). Studying the social dimension of emotion regulation. Frontiers in Psychology, 4, 922. 10.3389/fpsyg.2013.00922 [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Slavich GM, & Cole SW (2013). The emerging field of human social genomics. Clinical Psychological Science. 10.1177/2167702613478594 [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Trull TJ, & Ebner-Priemer U (2013). Ambulatory assessment. Annual Review of Clinical Psychology. 10.1146/annurev-clinpsy-050212-185510 [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Vangronsveld KL, & Linton SJ (2012). The effect of validating and invalidating communication on satisfaction, pain and affect in nurses suffering from low back pain during a semi-structured interview. European Journal of Pain, 16(2), 239–346. 10.1016/j.ejpain.2011.07.009 [DOI] [PubMed] [Google Scholar]
  37. Veilleux JC, Lankford NM, Hill MA, Skinner KD, Chamberlain KD, Baker DE, & Pollert GA (2020). Affect balance predicts daily emotional experience. Personality and Individual Differences, 154, 109683. 10.1016/J.PAID.2019.109683 [DOI] [Google Scholar]
  38. Williams WC, Morelli SA, Ong DC, & Zaki J (2018). Interpersonal emotion regulation: Implications for affiliation, perceived support, relationships, and well-being. Journal of Personality and Social Psychology. 10.1037/pspi0000132 [DOI] [PubMed] [Google Scholar]
  39. Yap MBH, Allen NB, & Ladouceur CD (2008). Maternal socialization of positive affect: the impact of invalidation on adolescent emotion regulation and depressive symptomatology. Child Development, 79(5), 1415–1431. 10.1111/j.1467-8624.2008.01196.x [DOI] [PubMed] [Google Scholar]
  40. Zaki J, & Williams WC (2013). Interpersonal emotion regulation. Emotion (Washington, D.C.), 13(5), 803–810. 10.1037/a0033839 [DOI] [PubMed] [Google Scholar]
  41. Zielinski MJ, & Veilleux JC (2018). The Perceived Invalidation of Emotion Scale (PIES): Development and psychometric properties of a novel measure of current emotion invalidation. Psychological Assessment. 10.1037/pas0000584 [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES