Abstract
Objectives:
The aim of the present study was to explore whether remote and on-site work stress during the COVID-19 pandemic was experienced with different severity. The second goal was to investigate stress conditions at both working modes.
Material and Methods:
The study involved 946 individuals working in the education system and BSS sector in different Polish organizations. The following tools were used: the Brief Scale of Vocational Stress by Dudek and Hauk, the Polish version of the scales to measure work–family conflicts by Grzywacz, Frone, Brewer and Kovner, Meyer and Allen's Affective, Continuance, and Normative Commitment Scales in the Polish adaptation by Bańka, Wołowska and Bazińska, the Satis faction with Job Scale by Zalewska.
Results:
The analysis of intergroup differences revealed that remote work stress severity was significantly lower than on-site work stress severity. The regression analyses proved that work–family conflict and job satisfaction were significant predictors of remote and on-site work stress. Continuance commitment positively predicted on-site work stress. Both models turned out to be statistically significant. The variables included in the models explained 39% and 35% of the variability of the remote work and on-site work stress, respectively.
Conclusions:
Remote work is associated with lower stress severity than on-site work. For both types of work, the higher the level of work–family conflict, the higher the level of stress severity, but the higher the job satisfaction, the lower the stress severity. Continuance commitment is positively related to on-site stress, which means that people who work for an organization and see no alternative feel more stressed. Such an effect was observed only in the case of on-site work. The study findings are discussed in light of previous research, and implications for organizational practice are considered. Int J Occup Med Environ Health. 2023;36(1):96–111
Keywords: job satisfaction, organizational commitment, COVID-19, remote work stress, on-site work stress, work–family conflicts
INTRODUCTION
The outbreak of the COVID-19 pandemic and the resulting restrictions forced employees from all over the world to suddenly change the way they work and to move from traditional work performed in the workplace to remote work. Poland was no exception, and in the second half of March 2020, when the epidemic situation affected the activities of many organizations, employers opened up to forms of employment allowing social distance, particularly remote work. Remote work, also known as telework, is a type of mental work carried out outside the traditional workplace. This type of work is entirely or partially free of personal contact with the employer, is provided remotely through electronic media such as the Internet or the telephone, and its partial or total effects are transmitted through these media. According to a report by the Polish Central Statistical Office (Główny Urząd Statystyczny – GUS), around 25% of all employees took advantage of the possibility of working remotely in 2020 [1].
Although remote work is not a new phenomenon, the circumstances surrounding the COVID-19 pandemic and the sudden and dynamic changes have become a source of new challenges for many employees, which could result in stress. According to the National Labor Inspectorate's definition [2], occupational stress occurs when employees experience psychological discomfort related to working conditions or demands. Stress is framed as harmful physical and emotional reactions that arise when the demands of the job do not match the employee's capabilities, resources, or needs and includes a situation in which, at any given time, the conditions and demands exceed the employee's ability to perform or cope effectively. Karasek and Theorell [3] argue that in the process of job stress, the level of demands and the ability to meet those demands are critical. According to classical stress theories [3,4], sudden and unpredictable changes at work could diminish workers' control and determine the occurrence and strength of stress.
During the pandemic, changes that occurred in a short period of time forced employees to behave adaptively. Control over the work situation and the predictability of what might come were low, and the employees did not always have direct influence over decisions, including the form of work, and could not always make free choices about work activities [5]. The situation may have been different for groups of employees working remotely and traditionally. Stress for these 2 groups of workers could also come from different sources. Remote workers have been confronted with numerous challenges closely related to working online. These include lack of appropriate hardware or software, poor quality of Internet connection, the feeling of loneliness and isolation, or the need to perform work-related and private life-related tasks at the same place and time. Furthermore, employees isolated from the workplace may have experienced a different type and intensity of work-home conflicts than employees working at the workplace. Their encounters with co-workers or superiors may have been of a different nature, they may have needed more support in various tasks, or they may have felt content to be free of certain obligations or duties, and so their assessment of commitment to the organization or job satisfaction during the pandemic period may have been different. In addition, the factors described above, differentiating the situation of remote and traditional workers, may have determined a different perception of the work situation, its conditions, and, therefore the characteristics of stress experienced by the 2 groups.
Occupational stress in traditional and remote work
There are many ways to study stress and vocational stress. The ways of measuring stress can be divided into physiological-biochemical, which usually include methods of measuring heart rate (blood pressure, rhythm), epinephrine, norepinephrine, cortisol levels secreted by the body, glycosylated hemoglobin levels, and interrogative, which involves asking people about their opinions and evaluations of work activities and conditions and work-related symptoms [6]. This study employs the latter – the psychological approach, where stress severity is based on a subjective evaluation of the work situation. It stems from the most widespread view of stress, the transactional approach proposed by Lazarus and Folkman [7], where it is assumed that stress is a dynamic transaction between a person and the environment when a situation occurs in which the demands of the environment exceed the person's capabilities. According to these authors, the main role in determining the formation and intensity of stress is played by the individual's assessment of the situation in relation to its characteristics and his or her ability to make changes in the environment. The same principles apply to occupational or vocational stress with the added notion that it is a phenomenon that occurs in the work environment [2].
Most research on occupational stress is devoted to problems arising from tasks performed in the workplace. However, the stress associated with remote work was also studied before the outbreak of the COVID-19 pandemic. The results of empirical work on the severity and determinants of stress in both types of work are inconclusive, and some are even contradictory. Concluding this topic is difficult due to the relatively small number of systematic comparisons of the stress of remote workers and those working in the workplace. In the pre-pandemic period, no consensus was reached on whether remote work generally brings more benefits than harms to workers and is associated with higher or lower stress levels than traditional work.
Remote working during the pandemic differs significantly from remote working performed before, as it is often not voluntary for the employer or the employee. The necessity to work in this way occurred suddenly, which did not allow for consideration of workers' skill levels, their ability to cope with social isolation, lack of information necessary to perform certain tasks, and other stressors. Workers were also forced to be with other household members, children, or the elderly while working, which may have exacerbated work–family conflicts. Research findings of studies investigating the impact of COVID-19 pandemic on work functioning suggest that remote working can result in various illnesses and disorders, both psychological [8] and physical [9]. Some of the recent studies also seem to confirm that the stress associated with remote working during a pandemic is high [10,11], partly due to feelings of isolation and loneliness and an increased sense of work–family conflict [11,12]. However, on the contrary, traditional work is associated with a number of stressors that are not present in remote work. First, performing work in the workplace involves an increased amount of interpersonal contact, and thus a greater risk of COVID-19 infection [13]. It also involves commuting, which implies extended time away from home and can exacerbate work–family conflict, but also requires the use of public transport, which is also associated with an increased risk of COVID-19 infection [14]. In fact, some studies conducted during the pandemic indicate that workers are apprehensive about going back to on-site working due to the existing health-related stressors [15]. In traditional work, employees may perceive that their autonomy is weaker, which may exacerbate their stress [16].
All this implies that stress resulting from remote work may be influenced by different factors than stress at traditional work. The relationship between work–family conflict intensity, organizational commitment, job satisfaction, and stress may differ between remote and on-site work. Hence, despite the lack of clear conclusions from the research conducted so far, it seems that traditional work may be associated with higher levels of stress than remote work. Thus, the main objective of this study is to compare the stress intensity of remote and traditionally performed work and investigate the relationships between the selected factors of stress in remote and traditional work.
Selected conditions of remote and on-site work stress
When an employee's commitment to one of their life roles, such as work or family life, makes it difficult for them to meet the other role's demands, conflicts referred to as work–family and family–work conflicts occur [17]. Such experiences hurt the employee and can cause mental and physical health deterioration. One of the key consequences of work–family conflicts is that they are positively related to the level of work and life stress [18]. These conflicts are bidirectional – on the one hand, there may be a situation where work-related demands and tensions hinder an employee's ability to meet family responsibilities, and on the other hand, a situation where family life demands negatively impact an employee's ability to complete work responsibilities [19]. Greenhaus and Beutell [20] distinguished 3 causes of family and work role mismatch. These encompass the limited amount of time that cannot be devoted to the tasks associated with both roles, the tension related to fulfilling the tasks of one role that affects performance in the other one, and specific patterns of behavior arising from one role that are incompatible with the demands placed on behavior in the other role.
Although work–family and family–work conflicts are different experiences for the individual, they are very often interrelated. Their experience is associated with stress and fatigue, irritability, various somatic complaints, reduced subjective psychological well-being, lower life satisfaction, reduced productivity, efficiency, distraction, reduced quality of tasks, and a higher risk of burnout [20].
During the pandemic, work–family conflicts may be exacerbated by the need to perform tasks from home, which may compound the experience of stress. Results from some studies indicate significantly higher levels of remote work stress compared to traditional work due to the experience of work–family conflict, increased isolation, work overload, role ambiguity, and insufficient information [21,22]. However, other studies indicate a significantly higher level of work–family balance and a reduction in conflict between these 2 areas of human functioning when working remotely, variables that constitute one of the main factors of work stress [23]. Therefore, the relationship of work–family conflicts during remote and traditional work and occupational stress is unclear and became the focus of this study.
Organizational commitment is a strong belief in and acceptance of the organization's goals and values, a willingness to put considerable effort into the organization, and a strong will to stay within the organization. Due to the lack of consensus and the multitude of measurement tools over the years, researchers tend to treat organizational commitment as a multidimensional construct that consists of 3 components [24]:
-
–
affective commitment, which includes the employee's emotional commitment to the organization, involvement, and identification with it – this commitment component encourages employees to stay with the organization because they want to;
-
–
continuance commitment, which stems from an awareness of the costs associated with the potential abandonment of the organization – this commitment causes employees to remain in the organization because they feel they have to;
-
–
normative commitment, which is the feeling of moral obligation to stay with the organization – due to this commitment, employees stay with the organization because they feel they should.
The consequences of the different commitment components vary. In general, employees with strong affective commitment work harder and perform better at work. A similar but weaker effect is found among employees with strong normative commitment, while employees with strong continuance commitment to the organization work less well, establish fewer relationships with colleagues, and even show dysfunctional or deviant behaviors. All 3 components of commitment are also associated with reduced employee turnover, although the strongest effect is found for affective commitment. A review of studies also indicates that organizational commitment, with all its components, is negatively associated with stress [25]. On this basis, it can be hypothesized that employees who commit to and stay with an organization may experience less work-related tension and lower stress. However, not too much evidence can support the unfolding of the possible relationships between commitment and occupational stress regarding remote and on-site work. Hence, this research aims to investigate the role of organizational commitment in predicting both types of occupational stress.
Job satisfaction is a component of subjective well-being relating to the work domain and representing a relatively stable attitude towards work, expressed in cognitive and affective reactions. Evaluative judgments in job satisfaction indicate whether an individual perceives the work environment as beneficial while the emotional component relates to feelings experienced at and toward work [26]. Research shows a fairly strong negative relationship between job satisfaction and perceived stress. Specifically, according to the job demands-resources model [4], job satisfaction is an adequate negative predictor of occupational stress. It is because, as noted by Jasiński et al. [27], job satisfaction is a general indicator of working conditions and physical, psychological, social, or organizational resources available to incumbents at work. These resources are crucial in coping with stress, and their absence – manifested by dissatisfaction – can be a source of stress [28] due to the inability to offset the demands of a work-related situation. Hence, job satisfaction acts as a health-protective factor that undermines the adverse effects of stressors and prevents job burnout [29]. Therefore, it was included in the present research.
Aim of the study and hypotheses
The main purpose of this study was to answer the following research questions:
-
–
Does the severity of stress resulting from remote work differ from the severity of stress associated with on-site work?
-
–
Do work–family conflict, employee commitment (i.e., affective, normative, and continuance commitment), and job satisfaction predict remote and on-site work stress?
Based on the results of previous studies regarding the stress intensity associated with remote and on-site work, the following hypothesis was formulated:
-
–
H1: On-site work stress is higher than remote work stress.
Concerning the second research question, 6 hypotheses were formulated regarding stress in remote and on-site work and its relationships with work–family and family–work conflict, organizational commitment, and job satisfaction. The first 3 of them concern the predictors of remote work stress, the following 3 relate to the predictors of on-site work stress.
-
–
H2: Experiencing work–family and family–work conflict is positively related to remote work stress.
-
–
H2a: Work–family conflict is a positive predictor of remote work stress.
-
–
H2b: Family–work conflict is a positive predictor of remote work stress.
-
–
H3: Employee commitment to the organization is negatively related to remote work stress.
-
–
H3a: Affective commitment is a negative predictor of re mote work stress.
-
–
H3b: Normative commitment is a negative predictor of re mote work stress.
-
–
H3c: Continuance commitment is a negative predictor of remote work stress.
-
–
H4: There is a negative relationship between job satisfaction and remote work stress. Job satisfaction is a negative predictor of remote work stress.
-
–
H5: Experiencing work–family and family–work conflicts are positively related to on-site work stress.
-
–
H5a: Work–family conflict is a positive predictor of on-site work stress.
-
–
H5b: Family–work conflict is a positive predictor of on-site work stress.
-
–
H6: Employee commitment to the organization is negatively related to on-site work stress.
-
–
H6a: Affective commitment is a negative predictor of on-site work stress.
-
–
H6b: Normative commitment is a negative predictor of on-site work stress.
-
–
H6c: Continuance commitment is a negative predictor of stress in on-site work.
-
–
H7: There is a negative relationship between job satisfaction and on-site job stress. Job satisfaction is a negative predictor of on-site work stress.
MATERIAL AND METHODS
The study sample
The sample in the study comprised 946 employees (766 women, 81% and 180 men, 19%) aged 18–75 years (mean (M) ± standard deviation (SD) = 31.36±10.13). They worked in the BSS sector as IT professionals and other occupations (N = 386, 40.8%), as teachers (N = 209, 22.1%), or in other professions (e.g., consulting, telecommunication, or restaurant business; N = 351, 37.1%). The majority of participants (N = 383, 40.5%) graduated with the title of MSc, 267 (28.2%) of them had secondary education and 218 (23%) had a bachelor's degree. The rest of the respondents had either a Doctor's degree (N = 67, 7.1%) or Polish National Vocational Qualification (N = 11, 1.2%). Respondents' general seniority ranged 1 month–55 years (M±SD 9.84±8.66), while their seniority in the current company spanned 1 month–46 years (M±SD 5.45±7.14). The respondents were divided into 2 groups, employees doing their work remotely (N = 487, 51.4%) and on-site (N = 459, 48.6%). The criterion for inclusion in the groups was the subjective declaration of the respondent about the time of performing work in a particular way. Following existing statistical suggestions [30], employees declaring that they worked remotely most of the time during the week were included in the group of remote workers, those declaring that they did it mostly at their workplace were included in the group of traditional workers.
The group of employees working remotely included 382 women (78.4%) and 105 men (21.6%). The average age of the respondents was almost 31 years (M±SD 30.85±9.89; Me = 28). The majority of participants (N = 186, 38.2%) graduated with the title of MSc, 133 (27.3%) of them had secondary education, and 117 (24%) had a bachelor's degree. The rest of the respondents had either a Doctor's degree (N = 48, 9.9%) or Polish National Vocational Qualification (N = 3, 0.6%). The average seniority in this group was slightly >9 years (M±SD 9.11±8.30). The group of stationary employees included 384 women (83.7%) and 75 men (16.3%). The average age of the respondents was almost 32 years (M±SD 31.88±10.43, Me = 28). The majority of participants (N = 197, 42.6%) graduated with the title of MSc, 134 (29.2%) of them had secondary education and 101 (22%) had a bachelor's degree. The rest of the respondents had either a Doctor's degree (N = 19, 4.1%) or Polish National Vocational Qualification (N = 8, 1.7%). The average seniority was >10 years (M±SD 10.37±9.03).
Procedure and measures
The study was conducted using an online questionnaire. Data were collected in April 2021–March 2022, during the third, fourth and fifth waves of the pandemic in Poland. Participation in the survey was voluntary, and the respondents were assured of anonymity, informed about the purpose of the study and the possibility of opting out. The research project was approved by the Research Ethics Committee of the Institute of Psychology at the SWPS University of Social Sciences and Humanities (approval No. WKEB74/12/2021).
Stress in remote and traditional work was measured using the Brief Scale of Vocational Stress by Dudek and Hauk [6]. The questionnaire contains 26 different job terms, e.g., nerve-wracking, pressured, relaxed, smooth-running. Possible variants of the respondent's answers are “Y” (yes) if the term refers to the work of the respondent, “N” (no) if the given feature does not refer to the work performed by the respondent and “?” (I don't know), in case of doubts and difficulties in making a decision. The Cronbach's α reliability coefficient for stress measurement in traditional and remote work was 0.91 and 0.90, respectively.
The work–family and family–work conflicts severity was measured with the Polish version of the tool developed by Grzywacz et al. [31], adapted to this study using a back-translation procedure. The tool consists of 3 questions about the work–family conflict (sample item “In the last month how often did your job or career interfere with your responsibilities at home, such as yard work, cooking, cleaning, repairs, shopping, paying the bills, or childcare?”) and 3 questions regarding family–work conflict (sample item “In the last month how often did your home life interfere with your responsibilities at work, such as getting to work on time, accomplishing daily tasks, or working overtime?”). The respondents indicate the frequency of experiencing family interference with work and vice versa on a 5-point Likert scale: from 1 – “never or less than once a month” to 5 – “five or more days a week.” Cronbach's α was 0.88 for work–family conflict and 0.89 for family–work conflict.
Organization commitment was measured using Meyer and Allen's Affective, Continuance, and Normative Com mitment Scales in the Polish adaptation by Bańka et al. [24]. The statements concern the feelings and beliefs describing the employee's relationship with their organization. The scale comprises 3 commitment subscales: affective (sample item “I feel like ‘part of the family’ at my organization”), continuance (“I feel that I have too few options to consider leaving this organization”), and normative (“If I got another offer for a better job elsewhere I would not feel it was right to leave my organization”). The participants answer 18 questions (6 items for each subscale) using a 7-point Likert scale, where 1 – “strongly disagree” and 7 – “strongly agree.” Cronbach's α ranged from 0.81 (continuance commitment) to 0.87 (affective and normative commitment).
Job satisfaction was measured with the Satisfaction with Job Scale by Zalewska [26], which indicates the cognitive aspect of job satisfaction. The tool consists of 5 questions (sample item “In many ways, my work is close to perfect”) with a response scale from 1 – “strongly disagree” to 7 – “strongly agree.” Cronbach's α was 0.86.
The authors also gathered data about individual (i.e., sex, age, and education level) and occupational (i.e., profession and seniority) demographic characteristics.
RESULTS
The statistical analyses were carried out in IBM SPSS Statistics 28 v. 28.0.1.0 on the data from 946 respondents. The analyses included calculating descriptive statistics, comparison of means, correlations, and hierarchical regression analysis with multiple predictors of stress experienced at work remotely and on-site.
To verify the hypothesis that traditional work is more stressful than remote work (H1), the authors compared the mean scores for 2 types of work using the independent samples t-test (Table 1). The analysis of intergroup differences showed that the average stress in traditional work was significantly higher than the average stress in remote work. Cohen's d indicated a weak relationship between the type of work and the stress experienced in it. In addition, as Table 1 shows, remote workers experience lower levels of work–family and family–work conflicts and manifest a lower intensity of continuance commitment to the organization. Cohen's d coefficient values show that these differences are small. Results also revealed small differences in normative attachment to the organization and job satisfaction. Remote workers show slightly lower normative attachment and slightly higher job satisfaction than stationary workers. Finally, affective commitment to the organization turned out to be the only variable that remote and traditionally working employees did not differ in.
Table 1.
Descriptive statistics, t tests and Cohen's d values – comparison of work on-site group (N = 459) and remote work group (N = 487) comprised of Polish employees in April 2021–March 2022, Poland
| Variable | M±SD | t (df) | Cohen's d |
|---|---|---|---|
| Stress at work | 3.00** (944) | 0.20 | |
| work on-site | 30.54±14.80 | ||
| work remotely | 27.72±14.17 | ||
| Work–family conflict | 4.09*** (944) | 0.27 | |
| work on-site | 8.78±3.60 | ||
| work remotely | 7.85±3.40 | ||
| Family–work conflict | 3.29** (944) | 0.21 | |
| work on-site | 7.10±3.37 | ||
| work remotely | 6.39±3.19 | ||
| Affective commitment | 1.58 (944) | 0.10 | |
| work on-site | 24.67±8.98 | ||
| work remotely | 23.76±8.75 | ||
| Continuance commitment | 3.29** (944) | 0.21 | |
| work on-site | 20.58±8.66 | ||
| work remotely | 18.77±8.30 | ||
| Normative commitment | 2.32* (944) | 0.15 | |
| work on-site | 20.00±9.27 | ||
| work remotely | 18.67±8.37 | ||
| Job satisfaction | –2.41* (944) | –0.16 | |
| work on-site | 22.08±7.13 | ||
| work remotely | 23.19±7.15 |
p < 0.05;
p < 0.01;
p < 0.001.
Stress experienced at work correlated positively with work–family and family–work conflicts but negatively with affective and normative commitment and job satisfaction. For both groups of remote and on-site work, the intercorrelations with other study variables were analogous. The correlation coefficients are shown in Table 2.
Table 2.
Spearman rank-order correlations between stress at work, work–family and family–work conflicts, organizational commitment and job satisfaction in a sample comprised of Polish employees in April 2021–March 2022, Poland
| Variable | Spearman correlation | |||||
|---|---|---|---|---|---|---|
| 1 | 2 | 3 | 4 | 5 | 6 | |
| Remote work (N = 487) | ||||||
| 1. Stress at work | – | |||||
| 2. Work–family conflict | 0.21*** | – | ||||
| 3. Family–work conflict | 0.10* | 0.62*** | – | |||
| 4. Affective commitment | –0.39*** | 0.07 | 0.11* | – | ||
| 5. Continuance commitment | 0.01 | 0.15* | 0.17*** | 0.34*** | – | |
| 6. Normative commitment | –0.22*** | 0.06 | 0.13* | 0.57*** | 0.36*** | – |
| 7. Job satisfaction | –0.58*** | –0.13** | –0.08 | 0.66*** | 0.11* | 0.37*** |
| Work on-site (N = 459) | ||||||
| 1. Stress at work | – | |||||
| 2. Work–family conflict | 0.27*** | – | ||||
| 3. Family–work conflict | 0.23*** | 0.62*** | – | |||
| 4. Affective commitment | –0.37*** | –0.06 | 0.05 | – | ||
| 5. Continuance commitment | –0.03 | 0.06 | 0.14** | 0.35*** | – | |
| 6. Normative commitment | –0.23*** | 0.03 | 0.13** | 0.59*** | 0.46*** | – |
| 7. Job satisfaction | –0.54*** | –0.18*** | –0.17*** | 0.69*** | 0.23*** | 0.47*** |
p < 0.05;
p < 0.01;
p < 0.001.
To verify the hypotheses, the authors performed a hierarchical regression analysis with control variables and predictors of stress at remote and on-site work groups. Con trol variables were entered (sex, age, and level of education) in the first model and independent variables of work–family/family–work conflict in the second model and organizational commitment (i.e., affective, continuance, and normative) and job satisfaction in the third model. Moreover, the authors calculated collinearity statistics for independent variables to verify if they could be entered simultaneously in the regression models. The results showed that VIF did not exceed 2.6 for any of the variables, and the tolerance was >0.4, indicating a very low degree of collinearity between the predictors of remote and on-site work stress.
Results of hierarchical regression for stress experienced at remote work (Table 3) indicated that all control variables were insignificant when the other predictors were included in the final regression model. Work–family conflict showed to be a significant and positive predictor of stress at remote work, explaining 8% of its variance. Thus, the H2a hypothesis was supported. The additional inclusion of job satisfaction, a significant and negative predictor of remote work stress, caused R2 to further increase significantly by 31%, which supported hypothesis H4. The significant predictors explained 39% of stress at remote work. Other independent variables (i.e., family–work conflict and affective, continuance, and normative commitment) showed no significant relationships with remote work stress. Thus, hypotheses H2b, H3a, H3b and H3c were not supported.
Table 3.
Hierarchical regression analysis of work–family/family–work conflict, organizational commitment and general job satisfaction as predictors of stress at remote work and work on-site in a sample of Polish employees in April 2021–March 2022, Poland
| Variable | Stress | |||||
|---|---|---|---|---|---|---|
| model 1 | model 2 | model 3 | ||||
| B (SE) | β | B (SE) | β | B (SE) | β | |
| Remote work (N = 487) | ||||||
| control variable | ||||||
| sex | –6.34 (1.54) | –0.18*** | –5.37 (1.53) | –0.16*** | –2.56 (1.26) | –0.07 |
| age | –0.04 (0.09) | –0.03 | –0.03 (0.09) | –0.02 | 0.02 (0.08) | 0.01 |
| education | 1.43 (0.66) | 0.14* | 1.34 (0.65) | 0.13* | 0.98 (0.53) | 0.09 |
| predictor | ||||||
| work–family conflict | 0.98 (0.23) | 0.24*** | 0.69 (0.19) | 0.17*** | ||
| family–work conflict | –0.25 (0.25) | –0.06 | –0.16 (0.2) | –0.04 | ||
| affective commitment | –0.15 (0.09) | –0.09 | ||||
| continuance commitment | 0.08 (0.07) | 0.05 | ||||
| normative commitment | 0.06 (0.08) | 0.03 | ||||
| job satisfaction | –1.03 (0.1) | –0.52*** | ||||
| F | 7.75*** | 9.18*** | 34.82*** | |||
| R2 | 0.05 | 0.09 | 0.40 | |||
| adjusted R2 | 0.04 | 0.08 | 0.39 | |||
| ΔR2 | 0.05 | 0.04 | 0.31 | |||
| ΔF | 7.75*** | 10.86*** | 61.13*** | |||
| Work on-site (N = 459) | ||||||
| control variable | ||||||
| sex | –1.02 (1.87) | –0.03 | 0.6 (1.83) | 0.01 | 1.54 (1.54) | 0.04 |
| age | 0.01 (0.03) | 0.02 | 0 (0.03) | 0.00 | 0.02 (0.03) | 0.03 |
| education | –0.36 (0.55) | –0.03 | –0.46 (0.53) | –0.04 | –0.19 (0.44) | –0.02 |
| predictor | ||||||
| work–family conflict | 0.98 (0.24) | 0.24*** | 0.7 (0.21) | 0.17*** | ||
| family–work conflict | 0.31 (0.26) | 0.07 | 0.05 (0.22) | 0.01 | ||
| affective commitment | –0.07 (0.1) | –0.04 | ||||
| continuance commitment | 0.16 (0.07) | 0.09* | ||||
| normative commitment | 0 (0.08) | 0.00 | ||||
| job satisfaction | –1.1 (0.11) | –0.53*** | ||||
| F | 0.25 | 8.15*** | 28.66*** | |||
| R2 | 0.00 | 0.08 | 0.36 | |||
| adjusted R2 | 0.00 | 0.07 | 0.35 | |||
| ΔR2 | 0.00 | 0.08 | 0.28 | |||
| ΔF | 0.25 | 19.97*** | 49.90*** | |||
Model 1 – multiple hierarchical regression with control variables (sex, age, education) as predictors of stress; Model 2 – multiple hierarchical regression with control variables (sex, age, education) and independent variables (work–family conflict, family–work conflict) as predictors of stress; Model 3 – multiple hierarchical regression with control variables (sex, age, education) and independent variables (work–family conflict, family–work conflict, affective, continuance, and normative commitment, and job satisfaction) as predictors of stress.
p < 0.05;
p < 0.01;
p < 0.001.
Regarding stress at work on-site, results of hierarchical regression (Table 3) indicated that all control variables were insignificant in the final regression model. Work–family conflict was a positive predictor of stress at work on-site, explaining 7% of its variance, which gave support to hypothesis H5a. The third model showed that job satisfaction negatively predicted on-site work stress. Thus, hypothesis H7 was also supported. Interestingly, the relationship between stress and continuance commitment was positive and thus opposite to what was expected in hypothesis H6c. However, this type of commitment is a very weak predictor of stress. Adding those independent variables to the model increased the explained variance of the dependent variable by 28%. Together, the postulated predictors explained 35% of variance in stress at work on-site. Simultaneously, independent variables such as family–work conflict and affective and normative commitment were not significantly related with on-site work stress, yielding no support for hypotheses H5b, H6a, and H6b.
DISCUSSION
The aim of the present study was 2-fold. First, the authors wanted to explore whether remote and on-site work stress during the COVID-19 pandemic was experienced with different severity. Second, the authors' goal was to investigate stress conditions in both working modes during the pandemic period. The pool of participants was intentionally chosen to include employees from educational institutions and the BSS sector since the COVID-19 pandemic impacted those workers significantly [32].
The research results revealed that employees experienced stress at remote work as less severe than at on-site work. This indicates that people staying at home perceived their work as performed under less pressure and risk and more attractive, likable, and calm [29]. One probable explanation is that individuals who worked from home were not exposed to contact with others, which was deemed risky and a potential threat to getting infected. Hence, staying at home meant being isolated from others and avoiding the risk of contracting the COVID-19 virus. The results are consistent with previous studies, which prove that the reduction of interpersonal contacts lowers the chance of infection [13] while the perspective of going back to work on-site during the pandemic is associated with the fear of contamination due to the uncertainty of the professional environment and virus-related precautionary measures [15]. Moreover, studies show that widespread adoption of remote work may remedy the public health challenges of a pandemic [14,33]. Grant et al. [34], who examined the impact of remote work on well-being and work-life balance, found several positive aspects of work performed from home, e.g., reducing commute time. Their study proved that daily driving time might significantly affect workers' well-being and stress severity as well as lower performance and mood throughout the workday. The interviewees pointed to integrating professional work and private life and improving social interactions at home as advantages of remote work. Some studies conducted during the pandemic have reached similar conclusions [35]. In line with these studies, this research shows that remote employees experienced lower levels of work–family and family–work conflicts and were more satisfied with their job than employees doing the work in the workplace.
Investigating work stress conditions proved that work–family conflict and job satisfaction were associated with remote and on-site work stress. In line with previous research [18], the work–family conflict turned out to be positively related to occupational stress. It suggests that during the COVID-19 pandemic, employees working in both modes (i.e., remote and on-site) experienced work-to-family spillover, which means that their functioning at work transferred to the family domain. The spillover, in turn, had a positive effect on stress experienced at work. It comes as no surprise that the changes the COVID-19 pandemic made included the disruption of balanced functioning at work and home, although a little more balance can be seen among remote workers.
Regarding on-site work, the pandemic forced many organizations to decrease the number of employees working on-site and introduce shift work for those in the offices. As such, on-site workers were potentially left with more tasks to do than usual, leading to the feeling of being overwhelmed with professional chores. Concerning remote work, the sudden need to use a private domain to fulfill professional obligations and oftentimes sharing the space with other people (e.g., significant others who also worked and children) could be a stress-evoking factor.
As hypothesized, job satisfaction was negatively related to occupational stress. This positive job attitude has long been recognized as an essential factor preventing workers from experiencing adverse work outcomes, including stress. Simultaneously, it is important to note that stress and job satisfaction are similar constructs related to employees' cognitive-emotional functioning at work with a certain theoretical overlap. However, there is sufficient evidence that a direct link between satisfaction and stress exists and should be further investigated [e.g., 27,36]. In fact, Fletcher and Payne [28] note that a high correlation between different psychological strains (states of being stressed) is an encouragement to predict stress with other measurements of one's subjective feelings as it may be an accurate clue to their level of stress.
With this in mind, the obtained results confirm that satisfaction might play a role in helping employees cope with stressful work conditions, both in the office and when delivering work at home. One possible explanation is that job satisfaction indicates an employee's working conditions and resources available in the organization [4,27]. Hence, when high, job satisfaction can signify that incumbents have the necessary physical, psychological, social, or organizational resources to cope with the negative effects of stress and offset work demands. In the opposite situation, when workers do not evaluate or feel that their organization takes satisfactory care of them, they might not feel well prepared to confront adverse circumstances and thus be more prone to stress. Nevertheless, due to the study's cross-sectional nature and employed regression analyses, a causal relationship between the variables cannot be determined. Thus, a 2-way association between job satisfaction and job stress should be considered when comprehending the results.
The authors' study was the first to show the relationships between job satisfaction and stress associated with remote and traditional work. It was done under the conditions of the COVID-19 pandemic. The pandemic situation might have significantly affected the possible relationship between these variables. In fact, Jasiński et al. [27] suggest that the pandemic reinforces the importance of job satisfaction in coping with occupational stress as it manifests the contentment with available resources in the face of stressors. Moreover, there are studies that illustrate the role of pandemic-related factors such as fear of being infected, dying and losing a loved one, and contact with people who might have been infected [37,38]. Although these variables were not covered by attention in this study, they might have altered the relationship between the 2 constructs of interest. To better understand the nature of the relationship, further studies, including measuring different aspects of those phenomena, should be conducted.
The authors also verified organizational commitment as a stress factor. Continuance commitment turned out to be weakly and positively associated with on-site work stress. It is consistent with research indicating that this type of commitment to the organization is associated with burnout [39]. Based on this result, it can be concluded that workers who feel the need to stay in the organization due to probable costs of changing the job find themselves in a dead-end professional situation. The authors presume that continuously committed employees might perceive the alternatives for employment as less available when working on-site. Simultaneously, they might feel less liberty due to more control exerted by the organization. This may lead to a conflictual situation where the workers are torn between longing for even more freedom and the subjective feeling of no possibilities to find it. Hence, they experience their work as more adverse.
Also, although affective commitment turned out to be an insignificant predictor of stress, it is worth noting that affective commitment to the organization is strongly associated with job satisfaction in both groups of employees, doing remote and on-site work. This type of commitment, through job satisfaction, can affect stress. Precisely, a positive relationship with the organization and job satisfaction together can be essential resources and buffers to mitigate the experience of stress. People who identify with their work, are more satisfied with it, and are willing to make sacrifices for the organization in which they are employed, are most likely to view even difficult tasks more in terms of a challenge than a threat.
Limitations of the study
This study has some limitations to acknowledge. The main constraint is that it was carried out on Polish employees only. More extended cross-cultural research involving other countries should be conducted to answer whether the result received is typical of Polish culture or applies to other countries. Investigating factors directly related to the pandemic would have deepened the knowledge of stress determinants, but the research plan did not include them.
It is also possible that the results are related to the fact that women prevailed in the group. According to some studies, employees' ability to reconcile work and family life is related to gender, perceptions of work–family conflicts, and coping skills. Women experience problems balancing child and home care responsibilities with work, lack of time, husbands' lack of involvement in household responsibilities, cultural norms, and gender biases [40]. All of these may be reflected in the severity of work-related stress and require further empirical exploration. Thus, in future surveys, ensuring a greater gender balance in the sample would be worthwhile.
Another limitation of this study is using a self-descriptive method. The authors employed the tools to measure the cognitive aspect of job satisfaction and the emotional elements of occupational stress. In both of them, the respondents' responses require making individual evaluations of various aspects of the job, and these assessments can be based on the respondent's emotional attitude toward the job features, as well as on a more rational assessment. Future research should utilize alternative measurement methods to get a better view of the studied phenomena, both their emotional and cognitive aspects, to better understand the relationships between them. Research methodology could be further improved by adding more qualitative methods, for instance, interviews with employees. Also, conducting a longitudinal study would provide insight into the possible processes of adaptation to the situation. This could give a deeper insight into incumbents' experiences and reasons for the stress of remote and on-site work.
Future study directions
The research presented in this paper did not take into account individual factors related to personality traits. However, previous studies suggest that stress severity is likely to be influenced by individual features. For example, continuance commitment was found to be negatively related to openness to experience and positively associated with neuroticism. It may indicate that employees who are committed to the organization in this way may be characterized by greater behavioral rigidity, less flexibility to change, and feeling more stressed by the recent need to work remotely [41]. Extending the research to include these aspects seems to be another goal. The authors believe that it would be desirable to supplement the area of research with the psychological features mentioned above. Combining these factors into a single coherent model would allow a complete picture of the significant determinants of work stress. Other conditions that could be considered in future research of stress are management styles, perception of fairness in the company, or motivation of employees. Generally, empirical comparisons of remote and traditionally performed work should be continued. The authors of other studies indicate that although the stress of remote work is lower, this type of work also has negative characteristics. Specifically, this type of work performed 5 days a week (full-time remote work) is associated with a decrease in work productivity [39].
CONCLUSIONS
The results of the study argue that working remotely during a pandemic may be less stressful than working on-site. The conducted analyses allowed the authors to look separately at the determinants of stress associated with remote and on-site work. The authors believe that knowledge of the relationships between the studied phenomena will help better understand the specificity of stress associated with both types of work during the COVID-19 pandemic, which made work more stressful overall. The obtained results can form the basis for building proposals for practical actions to reduce the stress experienced by employees in relation to remote working in the future, in the situation of a pandemic or other similar events. One suggestion is to encourage supervisors to provide social support to their subordinates regularly. This could be done by organizing online meetings a few times a week or at least once a month and discussing arising problems. Another idea is to conduct weekly or biweekly surveys to monitor employees' stress and react with agility when any disturbances are noted.
Due to the importance of job satisfaction, organizations should also take an interest in flourishing this particular attitude, for example, by collecting data on the employees' needs during the COVID-19 pandemic and then fulfilling them accordingly with available organizational resources. By doing so, companies can build important employee resources that help mitigate the adverse effects of the pandemic. It seems important to influence working conditions so that work–family conflicts are prevented, and the employee does not feel overwhelmed by an excess of difficult tasks. In order to prevent the negative consequences of overloading and problems with combining tasks associated with both spheres of life, it is worth considering planning work in a hybrid model, leaving employees the choice as to the mode of performing tasks, of course, where it is possible. Such measures can help increase both affective commitment to the organization and job satisfaction, which are significant negative predictors of stress experienced in remote and traditional work.
ACKNOWLEDGMENTS
The authors like to express their gratitude to all of the participants of the study.
Footnotes
Funding: this study was supported by the SWPS University of Social Sciences and Humanities, Warsaw, Poland (projects No. 27/2022/FRBN/E and 8-MGN-21/22, Scientific Research Development Fund, entitled “Conditions of remote and on-site work stress during the COVID-19 pandemic,” projects manager: Agata Chudzicka-Czupała, Ph.D.).
REFERENCES
- 1.Statistics Poland. Information society in Poland in 2020 [cited 2021 Jul 23]. Available from: https://stat.gov.pl/en/topics/science-and-technology/information-society/information-society-in-poland-in-2020,1,7.html.
- 2.Chief Labour Inspectorate. Czym jest stres [cited 2022 Oct 1]. Available from: https://www.pip.gov.pl/pl/bhp/stres-w-pracy/6421,czym-jest-stres-.html. Polish.
- 3.Karasek R, Theorell T.. Healthy Work Stress, Productivity, and the Reconstruction of Working Life. New York Basic Books. 1990. [Google Scholar]
- 4.Bakker AB, Demerouti E.. The Job Demands-Resources model: State of the art. J Manag. Psychol. 2007;22:309-328. [Google Scholar]
- 5.Orfei MD, Porcari DE, D'Arcangelo S, Maggi F, Russignaga D, Lattanzi N, Malizia AP, Ricciardi E.. COVID-19 and Stressful Adjustment to Work: A Long-Term Prospective Study About Homeworking for Bank Employees in Italy. Front Psychol. 2022;13:843095. 10.3389/fpsyg.2022.843095. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Dudek B, Hauk M.. Krótka skala do mierzenia stresu w pracy. Med Pr. 2010; 61(4):479-487. Polish. [PubMed] [Google Scholar]
- 7.Lazarus R, Folkman S.. Stress, Appraisal, and Coping. New York: Springer. 1984. [Google Scholar]
- 8.Chirkowska-Smolak T, Chumak M.. Job insecurity and emotional disturbance of Polish employees during pandemic COVID-19. Med Pr. 2021;72(6):645-652. 10.13075/mp.5893.01181. Polish. [DOI] [PubMed] [Google Scholar]
- 9.Zyznawska JM, Bartecka WM.. Wpływ pracy zdalnej wymuszonej pandemią COVID-19 na ból kręgosłupa w grupie zawodowej nauczycieli. Med Pr. 2021;72(6):677-684. 10.13075/mp.5893.01189. Polish. [DOI] [PubMed] [Google Scholar]
- 10.Gómez S, Mendoza O, Ramírez J, Olivas-Luján M.. Stress and myths related to the COVID-19 pandemic's effects on remote work. Management Research: Journal of the Iberoamerican Academy of Management. 2020;18(4):401–420. 10.1108/MRJIAM-06-2020-1065. [DOI] [Google Scholar]
- 11.Sandoval-Reyes J, Idrovo-Carlier S, Duque-Oliva E.. Remote Work, Work Stress, and Work–Life during Pandemic Times: A Latin America Situation. Int J Env Res Pub He. 2021;18(13):7069. 10.3390/ijerph18137069. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Galanti T, Guidetti G, Mazzei E, Zappalà S, Toscano F.. Work From Home During the COVID-19 Outbreak. The Impact on Employees' Remote Work Productivity, Engagement, and Stress. J Occup Environ. 2021;63(7):426–432. 10.1097/JOM.0000000000002236. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Toscano F, Zapalla S.. Social Isolation and Stress as Predictors of Productivity Perception and Remote Work Satisfaction during the COVID-19 Pandemic: The Role of Concern about the Virus in a Moderated Double Mediation. Sustainability. 2020;12(23):9804. 10.3390/su12239804. [DOI] [Google Scholar]
- 14.George TJ, Atwater LE, Maneethai D, Madera JM.. Supporting the productivity and wellbeing of remote workers: Lessons from COVID-19. Organ Dyn. 2021. 10.1016/j.orgdyn.2021.100869. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Orfei MD, Porcari DE, D'Arcangelo S, Maggi F, Russignaga D, Lattanzi N, Malizia AP, Ricciardi E.. COVID-19 and Stressful Adjustment to Work: A Long-Term Prospective Study About Homeworking for Bank Employees in Italy. Front Psychol. 2022;13:843095. 10.3389/fpsyg.2022.843095. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Lange M, Kayser I.. The Role of Self-Efficacy, Work-Related Autonomy and Work-Family Conflict on Employee's Stress Level during Home-Based Remote Work in Germany. Int J Environ Res Public Health. 2022;19(9):4955. 10.3390/ijerph19094955. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Hanson G, Hammer L, Colton C.. Development and validation of a multidimensional scale of perceived work–family positive spillover. J Occup Health Psychol. 2006;11(3):249–265. 10.1037/1076-8998.11.3.249. [DOI] [PubMed] [Google Scholar]
- 18.Rabenu E, Tziner A, Sharoni G.. The relationship between work–family conflict, stress, and work attitudes. Int J Manpower. 2017;38(8):1143–1156. 10.1108/IJM-01-2014-0014. [DOI] [Google Scholar]
- 19.Netemeyer R, Boles J, McMurrian R.. Development and validation of work–family conflict and family–work conflict scales. J Appl Psychol. 1996; 81(4): 400–410. 10.1037/0021-9010.81.4.400. [DOI] [Google Scholar]
- 20.Greenhaus JH, Beutell NJ.. Sources of Conflict between Work and Family Roles. The Acad Manage Rev. 1985;10(1):76. 10.5465/amr.1985.4277352. [DOI] [Google Scholar]
- 21.Russell H, O'Connell P, McGinnity F.. The Impact of Flexible Working Arrangements on Work-life Conflict and Work Pressure in Ireland. Gend Work Organ. 2009;16(1):73-97. 10.1111/j.1468-0432.2008.00431.x. [DOI] [Google Scholar]
- 22.Weinert C, Maier C, Laumer S.. Why are teleworkers stressed? An empirical analysis of the causes of teleworkenabled stress. Wirtschaftsinf. 2015:1407–1421. [Google Scholar]
- 23.Azarbouyeh A, Naini S.. A study on the effect of teleworking on quality of work life. Manag Sci Lett. 2014;4(6):1063–1068. 10.5267/j.msl.2014.5.027. [DOI] [Google Scholar]
- 24.Bańka A, Wołoska A, Bazińska R.. Polska wersja Meyera i Allen Skali Przywiązania do Organizacji. Czasopismo Psychol. 2002;8(0):65-74. Polish. [Google Scholar]
- 25.Cicei CC. Occupational stress and organizational commitment in Romanian public organizations. Procedia Soc Behav Sci. 2012;33:1077-1081. 10.1016/j.sbspro.2012.01.288. [DOI] [Google Scholar]
- 26.Zalewska A. Skala satysfakcji z pracy – pomiar poznawczego aspektu ogólnego zadowolenia z pracy. Acta Univ Lodz Folia Psychol. 2003;7:49-61. Polish. [Google Scholar]
- 27.Jasiński AM, Derbis R, Walczak R.. Workload, job satisfaction and occupational stress in Polish midwives before and during the COVID-19 pandemic. Med Pr. 2021;72(6): 623–632. 10.13075/mp.5893.01149. [DOI] [PubMed] [Google Scholar]
- 28.Fletcher BC, Payne RL.. Stress and Work: A Review and Theoretical Framework. Pers. Rev. 1980;9(1): 19-29. 10.1108/eb055400. [DOI] [Google Scholar]
- 29.Rogozińska-Pawełczyk A. Job satisfaction vs. occupational stress – Quantitative analysis of 3 organizational units of a public sector institution. Med Pr. 2018;69(3):301-315. 10.13075/mp.5893.00695. [DOI] [PubMed] [Google Scholar]
- 30.Bedyńska S, Książek M.. Statystyczny drogowskaz 3. Praktyczny przewodnik wykorzystania modeli regresji oraz równań strukturalnych. Warszawa: Wydawnictwo Akademickie Sedno. 2012. Polish. [Google Scholar]
- 31.Grzywacz JG, Frone MR, Brewer CS, Kovner CT.. Quantifying work–family conflict among registered nurses. Res Nurs Health. 2006; 29:414-426. 10.1002/nur.20133. [DOI] [PubMed] [Google Scholar]
- 32.Główny Urząd Statystyczny. Wpływ epidemii COVID-19 na wybrane elementy rynku pracy w Polsce w I kwartale 2021 r. 2021. [cited 2021 Jul 16]. Available from: https://stat.gov.pl/obszary-tematyczne/rynek-pracy/popyt-na-prace/wplyw-epidemii-covid-19-na-wybrane-elementy-rynku-pracy-w-polsce-w-czwartym-kwartale-2021-r-,4,8.html. Polish.
- 33.Ikegami K, Baba H, Ando H, Hino A, Tsuji M, Tateishi S, et al. Job stress among workers who telecommute during the coronavirus disease (COVID-19) pandemic in Japan: a cross-sectional study. Int J Occup Med Environ Health. 2022;35(3): 339–351. 10.13075/ijomeh.1896.01865. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Grant C, Wallace L, Spurgeon P.. An exploration of the psychological factors affecting remote e-worker's job effectiveness, well-being and work-life balance. Employee Relations. 2013; 35(5):527-546. 10.1108/ER-08-2012-0059. [DOI] [Google Scholar]
- 35.Shimura A, Yokoi K, Ishibashi Y, Akatsuka Y, Inoue T.. Remote Work Decreases Psychological and Physical Stress Responses, but Full-Remote Work Increases Presenteeism. Front Psychol. 2021;12:730969. 10.3389/fpsyg.2021.730969. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Akyurt N. Job satisfaction and perceived stress among radiology technicians: a questionnaire survey in relation to sociodemographic and occupational risk factors. Int Arch Occup Environ Health. 2021;94:1617–1626. 10.1007/s00420-021-01667-1. [DOI] [PubMed] [Google Scholar]
- 37.Dymecka J, Filipkowski J, Machnik-Czerwik A.. Fear of COVID-19: stress and job satisfaction among Polish doctors during the pandemic. Adv Psychiatry Neurol. 2021;30(4): 243-250, 10.5114/ppn.2021.111935. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Fardin MA. COVID-19 and anxiety: a review of psychological impacts of infectious disease outbreaks. Arch Clin Infect Dis. 2020. 10.5812/archcid.102779. [DOI] [Google Scholar]
- 39.Grabowski D, Chudzicka-Czupała A, Chrupała-Pniak M, Rachwaniec-Szczecińska Ż, Stasiła-Sieradzka M, Wojciechowska W.. Etyka pracy i przywiązanie organizacyjne a wypalenie zawodowe. Med Pr. 2019;70(3):305-316. 10.13075/mp.5893.00800. Polish. [DOI] [PubMed] [Google Scholar]
- 40.Karkoulian S, Srour J, Sinan T.. A gender perspective on work-life balance, perceived stress, and locus of control. J Bus Res. 2016;69(11):4918-4923. 10.1016/j.jbusres.2016.04.053. [DOI] [Google Scholar]
- 41.Choi D, Oh I, Colbert A.. Understanding Organizational Commitment: A Meta-Analytic Examination of the Roles of the Five-Factor Model of Personality and Culture. J Appl Psychol. 2015;100(5). 10.1037/apl0000014. [DOI] [PubMed] [Google Scholar]
