Table 2.
Main study limitations and risk of biases in the 83 studies reviewed
| References | CSS | PM | SSS | NPS | LRR | SIM | PMFS | PRS | TLSL | CGB | SSOWS | SVL | Other limitations | Years |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Manlove [111] | X | X | 1993 | |||||||||||
| Deary et al. [48] | X | The use of zero-order statistics for the evaluation of multifactorial constructs, such as psychological issues, has been recognized as being inappropriate | 1996 | |||||||||||
| Deary et al. [112] | 1996 | |||||||||||||
| Mills et al. [60] | X | X | X | The exclusion of the personality measure at Time 2 did not allow for a complete analysis of the effects of personality differences on burnout dimensions | 1998 | |||||||||
| Zellars et al. [113] | X | X | 2000 | |||||||||||
| Zellars et al. [53] | X | X | X | 2001 | ||||||||||
| De Vries et al. [114] | 2002 | |||||||||||||
| McManus et al. [50] | 2004 | |||||||||||||
| Zellars et al. [52] | X | X | X | X | 2004 | |||||||||
| Cano-García et al. [115] | 2005 | |||||||||||||
| Burke et al. [116] | X | X | X | 2006 | ||||||||||
| Goddard et al. [57] | X | X |
Participants may have different characteristics from those of those who did not participate No comparison with a control group |
2006 | ||||||||||
| Langelaan et al. [117] | X | X | 2006 | |||||||||||
| Mostert et al. [118] | X | X | 2006 | |||||||||||
| Bahner et al. [119] | X | X |
The focus of the study was on the male batterer intervention program in the context of heterosexual relationships, not that of same-sex relationships It is not known whether the results may reflect the different nature and structure of the working environments of the batterer intervention programs |
2007 | ||||||||||
| Ghorpade et al. [120] | 2007 | |||||||||||||
| Kim et al. [121] | 2007 | |||||||||||||
| Teven [122] | X | X | X |
The data comes from a sample with only the level of education university Respondents may have tried fatigue because many of the measures overlap conceptually |
2007 | |||||||||
| Leon et al. [123] | X | X | X |
There are missing data and differences between the samples for age and education No specific job characteristics to RTCs were explored that might be associated with the client presentations measured It is possible that this study's moderation findings will not generalize to other samples of front line staff |
2008 | |||||||||
| Chung et al. [124] |
Challenging behavior has been measured in terms of the level of challenging behavior of the clients with whom they worked most intensively, and this does not accurately or reliably reflect exposure to challenging behavior No data on staff views of their job, on assaults or witnessed violence by staff, or on the kind of recent significant life events of the staff |
2009 | ||||||||||||
| De Hoogh et al. [125] | X | X |
In Sample 1 ratings of independent and dependent variables were provided by the same subordinates In Sample 2 the selection of raters may have selected employees who share attitudes and opinions |
2009 | ||||||||||
| Gandoy-Crego et al. [126] | X | 2009 | ||||||||||||
| Kim et al. [127] | X | X | X | Focus limited to individual differences | 2009 | |||||||||
| Taormina et al. [128] | X | X | X | 2009 | ||||||||||
| Barford et al. [129] | X | X | X | X | Different methods of data collection were used | 2010 | ||||||||
| Perry et al. [51] | X | X | X | The distinct contexts across the two samples and lower variance across variables in Sample 1 may have affected the results | 2010 | |||||||||
| Ghorpade et al. [130] | 2011 | |||||||||||||
| Hudek-Knežević et al. [59] | X | X |
Burnout at Time 1 was not measured Sample attrition due to drop-out at Time 2 Significant differences (age, agreeableness, and continuance commitment) were found between the sample included and the one that in both Times and the one that dropped out |
2011 | ||||||||||
| Salami, [131] | X | X | 2011 | |||||||||||
| Sterud et al. [61] | 2011 | |||||||||||||
| Armon et al. [54] |
The findings could be biased because of the "healthy worker effect" Longitudinal design was based on only two waves of measurement The personality scale only return a total score and do not evaluate the influence of the personality variables on each of the burnout factors The possibility that the factors of the personality may interact in affecting burnout levels has not been tested |
2012 | ||||||||||||
| Zimmerman et al. [132] | X |
The sample was composed exclusively of high-level professionals Only neuroticism and extraversion and not all personality traits were evaluated |
2012 | |||||||||||
| De la Fuente Solana et al. [133] | X | 2013 | ||||||||||||
| Garbarino et al. [134] | X | X | X | 2013 | ||||||||||
| Hurt et al. [135] | 2013 | |||||||||||||
| Lin et al. [136] | X | X |
The sample was relatively young Questionnaires used are not popular therefore the comparison with other studies (also of other nationalities) may be limited The employees who did not respond to our questionnaire might have been those who were busier |
2013 | ||||||||||
| Gan et al. [56] | X | X | X | Could not examine the reciprocal relationship (burnout/engagement to demands/resources) using panel data | 2014 | |||||||||
| Reinke et al. [137] | X | X | X | 2014 | ||||||||||
| Taycan et al. [138] | X | Lack of an urban physician sample for comparison | 2014 | |||||||||||
| Yilmaz, [139] | 2014 | |||||||||||||
| Cañadas-De la Fuente et al. [140] | X | X | X | 2015 | ||||||||||
| Ang et al. [142] | X | X | X | 2015 | ||||||||||
| Iorga et al. [143] | X | X | 2016 | |||||||||||
| Vaulerin et al. [144] | X | 2016 | ||||||||||||
| Zhou et al. [145] | X | X |
Some scales have been adapted to the sample under study and their construct validity and temporal stability have not been confirmed The questionnaire was completed on workdays and may have caused additional stress for the firefighters |
2016 | ||||||||||
| De la Fuente-Solana et al. [146] | X | X | X | 2016 | ||||||||||
| Geuens et al. [147] | X | X | X | 2017 | ||||||||||
| Iorga et al. [148] | X | X | X | X |
The study was conducted from November 2012 to July 2013 and could lead to different results than a recent one The focus was only on individual factors, leaving out job-related and organizational factors |
2017 | ||||||||
| Lovell et al. [149] | X | X | X | The access to medical facilities and the small distance from patients who live in urban areas may have diminished the physicians' professional exhaustion | 2017 | |||||||||
| Ntantana et al. [150] | X | X | Methods based on measuring physiological parameters are not objective | 2017 | ||||||||||
| Al Shbail et al. [151] | X | X | X | Causal factors have not been investigated | 2017 | |||||||||
| Bergmüller et al. [152] | X | X |
Only companies that were part of the Amman Stock Exchange (ASE) were included in the study and not the others All types of shareholding companies were included in the study, regardless of sub-sectors |
2018 | ||||||||||
| Bianchi et al. [153] | 2018 | |||||||||||||
| Bianchi, [19] | X | X | X |
It was used only one measure of burnout The role of organizational and work-contextualized factors may have been overemphasized in burnout research, to the detriment of the role of generic dispositional factors Burnout's discriminant validity is unsatisfactory |
2018 | |||||||||
| Harizanova et al. [49] | X | X |
An exhaustion-centered measure of burnout was used Only neuroticism and extraversion and not all personality traits were evaluated |
2018 | ||||||||||
| Hildenbrand et al. [58] | X | 2018 | ||||||||||||
| Iorga et al. [154] | X | X | X |
The correlation between thriving and burnout at Time 1 and Time 2 was rather high Despite large groups of employees sharing the same supervisor, it was not possible to collect information on group composition and therefore take into account the variance due to team membership |
2018 | |||||||||
| Tang et al. [155] | X | X | 2018 | |||||||||||
| Tatalović Vorkapić et al. [156] | X | X | The psychological distress of the participants was not measured and also the fitting degrees of both logistic regression modes are low, suggesting that there are other unknown risk factors | 2018 | ||||||||||
| Yao et al. [157] | X | X | 2018 | |||||||||||
| Zaninotto et al. [158] | The participants of the sample, coming from the municipal hospitals, had heavy patient loads and therefore more exposed to burnout | 2018 | ||||||||||||
| Bahadori et al. [159] | X | X | A large number of correlations increases the risk of type 1 errors | 2018 | ||||||||||
| Brown et al. [160] | X | X | X | 2019 | ||||||||||
| Castillo-Gualda et al. [62] | X | X | X | 2019 | ||||||||||
| De la Fuente-Solana et al. [161] | X1 | X | X2 | X2 | X | 2019 | ||||||||
| De Looff et al. [55] | X | X | X | 2019 | ||||||||||
| Farfán et al. [162] | X | X | Methods based on measuring physiological parameters are not objective | 2019 | ||||||||||
| Khedhaouria et al. [163] | X | X | X | X | 2019 | |||||||||
| Pérez-Fuentes et al. [17] | X | X | Organizational factors that reduce stress, the effect of technostress, and personality traits on job burnout among employees, nor cases that could have specific antecedent conditions predisposing job burnout were not evaluated | 2019 | ||||||||||
| Ye et al. [164] | Some information is lost due to some scales, which are too short to adequately evaluate the personality construct, or they only return a total burnout score and do not evaluate the influence of the personality variables on each of the burnout factors | 2019 | ||||||||||||
| Banasiewicz et al. [165] | X | X | The Baron and Kenny method to examine the mechanism of the mediator is too simple | 2019 | ||||||||||
| Bhowmick et al. [166] | X |
The questionnaire contained closed-ended type questions The questionnaire required a long duration of completion Pregnancy terminations are performed only at several facilities in Poland |
2020 | |||||||||||
| De Vine et al. [167] | X | X | X | X | X | X | X | 2020 | ||||||
| Dionigi, [168] | X | X | 2020 | |||||||||||
| Farfán et al. [169] | X | X | X | The predictors evaluated accounted for only a part of the burnout dimensions | 2020 | |||||||||
| Liu et al. [170] | X | X | X | Sample with high concentration of people with a high education level | 2020 | |||||||||
| Mahoney et al. [171] | X | X |
The study focuses on negative workplace gossip and does not include positive workplace gossip The scale of negative workplace gossip is based on frequency, but the perception of negative workplace gossip may have different influences The mean score for negative workplace gossip is relatively lower than those of other studies with Chinese samples |
2020 | ||||||||||
| Malka et al. [172] | 2020 | |||||||||||||
| Tasic et al. [173] | X |
The issue of work supervisors who wanted to leave the job was not evaluated Being a quantitative study, it lacks the interpretive perspective of the participants It is not clear to what degree participants related to their role as fieldwork supervisor compared to their experiences workers in general |
2020 | |||||||||||
| Bianchi et al. [174] | 2020 | |||||||||||||
| Fuente-Solana et al. [175] | X | X | X | X | X | X | X | French sample was considerably larger than Spanish and Swiss samples | 2021 | |||||
| Fuente–Solana et al. [176] | X | X | X |
Methods based on measuring physiological parameters are not objective There was the impression that a sample with a certain level of burnout was more prone to participate |
2021 | |||||||||
| Manlove [111] | X | X | X | X | Methods based on measuring physiological parameters are not objective | 2021 |
CSS, Cross-Sectional; PM, Perceptual Measures; SSS, Small Sample Size; NPS, Non-Probabilistic Sampling; LRR, Low Response Rate; SIM, Single-Item Measures; PMFS, Predominantly Male/Female Sample; PRS, Poor reliability scale; TLSL, Time Lag too Short/Long; CGB, Cultural/Geographical Bias; SSOWS, Sample of a Single Organization/Work/State; SVL, Study Variables were Limited