Abstract
Background
Problem-solving appraisal is an important cognitive process that influences how individuals manage and cope with stress. Negative appraisals of problem-solving ability are associated with adverse mental health outcomes.
Methods
We conducted a meta-analysis using a random-effects model to synthesize the relationship between problem-solving appraisal and internalizing disorders. In addition, we examined the potential moderating role of age in this relationship and conducted subgroup analyses. To account for effect size dependencies, we used robust variance estimation (RVE). We examined publication bias with a funnel plot, Egger’s regression test, and weighting studies in terms of their probability of being published.
Results
We found a medium combined effect size (r = 0.40) for 66 coefficients from 49 studies. Age did not moderate the combined effect size. Subgroup analyses found no difference between the combined effect size of studies that used a translated version of the PSI and those that used the original English version, nor was there a difference between those who reported on the relationship between problem-solving appraisal and anxiety and those that focused on the relationship between problem-solving appraisal and depression. The combined effect size obtained with RVE did not significantly differ from the original combined effect size. A funnel plot, Egger’s regression test, and weighting of studies in terms of their probability of being published all evidenced publication bias. The trim-and-fill method imputed 16 studies, which adjusted the obtained combined effect size to a medium effect size of 0.38.
Conclusion
The findings underscore the moderate but robust relationship between problem-solving appraisal and internalizing disorders. Interventions to improve problem-solving skills may be beneficial across various demographics. Addressing publication bias is crucial to ensuring accurate meta-analytic results in mental health research.
Keywords: anxiety, depression, internalizing disorders, meta-analysis, problem-solving appraisal
1. Introduction
Mood and anxiety disorders are among the most prevalent mental health disorders globally. Depression is characterized by feelings of sadness and loss of interest and pleasure in daily activities, and especially cognitive (e.g., thoughts of suicide) and somatic (e.g., weight loss) alterations that adversely impact the individual’s social, occupational, and interpersonal functioning (American Psychiatric Association, 2022). Anxiety is associated with physical and cognitive symptoms such as irritability, difficulty controlling worry, disturbances of sleep and concentration, restlessness, and irritability (American Psychiatric Association, 2022). Research within longitudinal studies and nonclinical samples of adults and adolescents with depressive and anxiety disorders support a developmental progression from anxiety to depression (Danneel et al., 2020; Bendau et al., 2021; del-Valle et al., 2022; Kenntemich et al., 2024). Depression is associated with a mental focus on past experiences, particularly those involving the loss of important goals or aspirations. Individuals experiencing depression tend to dwell on these lost goals, which can result in a pattern of negative and biased cognitive attributions of the self and future. In contrast, anxiety is associated with a future orientation and is linked to the perception of a future threat (Beck et al., 1974; Kenntemich et al., 2024).
Although anxiety and depression frequently co-occur, the reasons for this comorbidity are unclear. Some researchers have suggested that anxiety may precipitate cognitive processes that lead to depressive symptoms, including ruminative thinking, brooding, and cognitions associated with hopelessness. Rumination entails repeatedly focusing on negative emotional experiences and their causes or consequences, whereas brooding involves a passive comparison of one’s current life situation or status with an unachieved and idealized standard (Starr and Davila, 2012; Eysenck and Fajkowska, 2018). The tripartite model (Clark and Watson, 1991) has been influential in accounting for the co-occurrence of anxiety and depression. This model proposes that anxiety and depression share a common valence (negative emotions and the experience of distress) that underlies their comorbidity. The model also proposes that there are three core components of these disorders: general distress (negative affect), which is specific to both anxiety and depression; physiological hyperarousal, which is more characteristic of anxiety than depression; and anhedonia, which is more typical of depression than anxiety. The presence of general distress or negative affect means that individuals who experience one of these conditions are also likely to suffer from the other (Clark and Watson, 1991).
A consistent finding that has emerged from research on mental health disorders is that the burden of mental illness is not equally distributed in the population. Rather, it is concentrated in a subset of individuals. This finding has prompted research into the factors underlying this differential vulnerability (Zimmermann et al., 2020; Padmanabhanunni and Wiid, 2022; Shoshani and Kor, 2022). According to Lazarus and Folkman’s transactional model of stress and coping, cognitive appraisals are central in influencing mental health outcomes. Their model differentiates between primary appraisals, which entail the initial determination of whether a stressor is perceived as benign or stressful, and secondary appraisals, which involve an assessment of the resources the individual has available for coping (Lazarus and Folkman, 1984).
The current study focuses on the relationship between problem-solving appraisal and internalizing disorders. Problem-solving appraisal refers to an individual’s subjective assessment of their ability to solve life problems effectively (Heppner, 1982; Heppner, 1988). This concept is similar to Lazarus and Folkman’s construct of secondary appraisal. Self-appraisals of competence and coping ability influence individuals’ emotional reactions to stressors and subsequent development of psychological distress. In a review of 20 years of research on the Problem-Solving Inventory (PSI), which assesses problem-solving appraisal, Heppner et al. (2004) concluded that problem-solving appraisal is a critical determinant of psychological adjustment and mental health outcomes. Their findings indicate that individuals who perceive themselves as more effective problem solvers tend to experience lower levels of stress and mental health disorders. Conversely, those with negative appraisals of their own problem-solving abilities are more likely to exhibit higher levels of internalizing disorders.
Suzuki and Ahluwalia (Suzuki and Ahluwalia, 2004) have noted several methodological concerns with Heppner et al. (2004)’ work. First, the quality of the reported data varied, particularly in terms of effect sizes (e.g., correlations) and sample descriptors (e.g., race/ethnicity, sample size, and gender). Suzuki and Ahluwalia argue that these inconsistencies highlight the need for a meta-analysis of the PSI to provide a more accurate and comprehensive understanding of the relationship between problem-solving appraisal and mental health outcomes. Such an analysis would address concerns regarding uneven reporting and offer robust conclusions about the impact of problem-solving appraisal across diverse populations. Hence, the current study is a meta-analysis that aims to synthesize the relationship between problem-solving appraisal, as measured by the PSI, and internalizing disorders, specifically depression and anxiety.
2. Method
2.1. Procedure
The meta-analysis followed the guidelines of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA: Moher et al., 2009). We aimed to find studies that examine the connection between problem-solving appraisal, assessed by the PSI, and internalizing disorders, specifically depression and anxiety. To this end, we searched PubMed, PsycArticles, ScienceDirect, Scopus, Taylor & Francis, and Web of Science for studies published from 1982, when the PSI was first published, to November 2023. The same search strategy was applied across databases, using the keywords “problem-solving appraisal” and “problem-solving inventory,” with minor database-specific formatting adjustments where necessary. The search was limited to English language records.
The two authors independently reviewed the abstracts of the identified studies. For inclusion, studies needed to be in English, utilize the PSI to evaluate problem-solving appraisal, and report on the relationship of problem-solving appraisal and internalizing disorders. We obtained full-text versions for all abstracts that were tentatively considered to meet the inclusion criteria. In cases of disagreement over an abstract, the full text was reviewed. Additionally, we used a snowballing technique to identify further studies from the reference lists of reviewed papers.
2.2. Data analysis
A random-effects model was used for the meta-analysis. The meta-analysis was conducted using the “metafor” (Viechtbauer, 2010) package in R (R Development Core Team, 2020). Heterogeneity of effect sizes was evaluated using I2, where 25%: = low heterogeneity, 50% = moderate heterogeneity, 75% = substantial heterogeneity (Higgins and Thompson, 2002). To evaluate publication bias, we utilized funnel plots and Egger’s regression test. A symmetrical distribution of effect sizes around the combined effect size in the funnel plot and a nonsignificant result from Egger’s test would suggest no publication bias. If effect sizes were asymmetrically distributed around the combined effect size (thus indicating publication bias), we used the trim-and-fill method to impute effect sizes and make the distribution symmetrical. In this instance, the trim-and-fill method provides an adjusted combined effect size to account for the missing effect sizes. We used Meta-Essentials (Suurmond et al., 2017) for the trim-and-fill method. However, funnel plots and Egger’s regression test are used primarily to detect small study bias, which can encompass publication bias (Quintana, 2015); therefore, we also assessed publication bias by applying weights to studies based on their publication probability, using the weight function in R. The combined effect size obtained when studies are weighted is then compared to the unweighted combined effect size using chi-square. A statistically significant chi-square would be indicative of publication bias.
To determine whether other study variables impacted the combined effect size, we examined the potential moderating role of age using meta-regression. We also conducted subgroup analyses of studies that focused on anxiety versus studies that focused on depression and studies that used the original English version of the PSI in comparison with studies that used a translated version of the PSI.
Meta-analysis assumes that effect sizes are independent. However, multiple studies in this analysis reported several effect sizes. To address the dependencies among these effect sizes, we employed robust variance estimation (RVE) with a small sample correction (Hedges et al., 2010). By comparing the combined effect size from RVE with the initial combined effect size, we determined the impact of effect size dependencies on the overall results. The RVE was conducted with the package “robumeta” (Fisher and Tipton, 2015) in the R software environment (R Development Core Team, 2020).
3. Results
3.1. Studies selection and characteristics
Figure 1 presents the PRISMA flow diagram for the study selection process. A total of 2,196 studies were initially identified. Of these, 366 used the PSI, while the remainder were excluded for reasons such as duplication and not being published in English. Among the 366 studies that used the PSI, 317 did not examine the relationship between problem-solving appraisal and internalizing disorders. The final meta-analysis therefore included 49 studies that specifically investigated this relationship. The characteristics of the included studies are presented in Table 1.
Figure 1.
PRISMA diagram of study selection for the meta-analysis of the relationship between problem-solving appraisal and internalizing disorders.
Table 1.
Characteristics of studies extracted for use in the meta-analysis of the relationship between the PSI and internalizing disorders.
| Author(s) & year | Population | n | Age | Language | Outcome | r |
|---|---|---|---|---|---|---|
| Pretorius et al. (2023) | Students | 322 | 26.0 | English | Depression | 0.47 |
| Teo et al. (2021) | Young males | 342 | 20.4 | English | Depression | 0.40 |
| Pretorius (2020) | Secondary school teachers | 207 | 33.0 | English | Anxiety | 0.52 |
| Korkmaz et al. (2020a) | Individuals who had attempted suicide | 50 | 35.3 | Translated | Depression | 0.68 |
| Korkmaz et al. (2020a) | Individuals who had attempted suicide | 50 | 35.3 | Translated | Anxiety | 0.65 |
| Korkmaz et al. (2020b) | Healthcare workers | 140 | 33.2 | Translated | Anxiety | 0.18 |
| Ji et al. (2018) | Caregivers of individuals with memory loss | 78 | 66.5 | English | Depression | 0.32 |
| Julal (2016) | Students | 175 | 20.5 | English | Depression | 0.22 |
| Tang et al. (2015) | Caregivers of adults with memory loss | 91 | 67.0 | English | Depression | 0.23 |
| Cha et al. (2015) | Young adults | 96 | 24.1 | English | Depression | 0.38 |
| Chow and Chan (2010) | Chinese migrant women | 68 | Translated | Depression | 0.43 | |
| Erdur-Baker (2009) | High school students | 250 | 15.4 | Translated | Depression | 0.22 |
| Eskin et al. (2008) | High school students | 805 | 14.8 | Translated | Depression | 0.30 |
| Pickett Jr et al. (2007) | Caregivers of persons with Huntington’s disease | 62 | 55.4 | English | Depression | 0.27 |
| Pickett Jr et al. (2007) | Persons with Huntington’s disease | 61 | 51.6 | English | Depression | 0.39 |
| Yeung et al. (2007) | Caregivers of stroke patients | 70 | N/A | Translated | Depression | 0.35 |
| Heppner et al. (2006) | Students | 2,889 | 20.7 | Translated | Depression | 0.32 |
| Heppner et al. (2006) | Students | 2,889 | 20.7 | Translated | Anxiety | 0.22 |
| Eremsoy et al. (2005) | Graduate students | 92 | 26.1 | Translated | Depression | 0.34 |
| Eremsoy et al. (2005) | Graduate students | 92 | 26.1 | Translated | Anxiety | 0.46 |
| Bastian et al. (2005) | Students | 246 | 19.9 | English | Anxiety | 0.29 |
| Good et al. (2004) | Students | 260 | N/A | English | Depression | 0.24 |
| Good et al. (2004) | Students | 260 | N/A | English | Anxiety | 0.31 |
| Dieserud et al. (2003) | Suicide attempters | 50 | 41.0 | Translated | Depression | 0.50 |
| Dieserud et al. (2002) | Individuals who had attempted suicide | 321 | 39.0 | Translated | Depression | 0.63 |
| Heppner et al. (2002) | Students | 234 | 23.3 | English | Depression | 0.23 |
| Heppner et al. (2002) | Students | 234 | 23.3 | English | Anxiety | 0.31 |
| Kerns et al. (2002) | Patients with chronic pain | 234 | 50.0 | English | Depression | 0.38 |
| Dieserud et al. (2001) | Patients with and without suicide attempts | 123 | 40.2 | Translated | Depression | 0.48 |
| Witty et al. (2001) | Persons with chronic low back pain | 78 | 48.4 | English | Depression | 0.42 |
| Chan (2001) | Students | 499 | 26.2 | English | Depression | 0.44 |
| Dixon (2000) | Students | 66 | 20.0 | English | Depression | 0.17 |
| Dixon (2000) | Students | 66 | 20.0 | English | Depression | 0.39 |
| Reid and Dixon (2000) | Students | 118 | 30.0 | English | Depression | 0.67 |
| Marcotte et al. (1999) | Adolescents | 306 | 15.3 | Translated | Depression | 0.53 |
| Hanson and Mintz (1997) | Older adults | 97 | 77.3 | English | Anxiety | 0.25 |
| Hanson and Mintz (1997) | Older adults | 97 | 77.3 | English | Depression | 0.20 |
| Perez et al. (1997) | Male delinquent adolescents | 146 | 16.5 | English | Depression | 0.41 |
| Miner and Dowd (1996) | Students | 278 | 35.2 | English | Depression | 0.41 |
| Miner and Dowd (1996) | Students | 278 | 35.2 | English | Anxiety | 0.42 |
| Mayo and Tanaka-Matsumi (1996) | Dysphoric and non-dysphoric students | 28 | N/A | English | Depression | 0.82 |
| Pretorius and Diedricks (1994) | Students | 450 | 24.1 | English | Depression | 0.32 |
| Dixon et al. (1993) | Students | 277 | 20.1 | English | Depression | 0.33 |
| Dixon et al. (1993) | Students | 277 | 20.1 | English | Depression | 0.43 |
| McGrath and Ratliff (1993) | Students | 269 | 21.5 | English | Depression | 0.35 |
| McGrath and Ratliff (1993) | Students | 269 | 21.5 | English | Depression | 0.52 |
| McGrath and Ratliff (1993) | Students | 269 | 21.5 | English | Anxiety | 0.49 |
| McGrath and Ratliff (1993) | Students | 269 | 21.5 | English | Anxiety | 0.41 |
| Priester and Clum (1993) | Students | 303 | N/A | English | Depression | 0.30 |
| Sahin et al. (1993) | Students | 224 | 19.8 | Translated | Depression | 0.33 |
| Sahin et al. (1993) | Students | 224 | 19.8 | Translated | Anxiety | 0.45 |
| Elliott et al. (1992) | Students | 104 | 25.0 | English | Depression | 0.41 |
| Elliott et al. (1992) | Participants with spinal cord injuries | 102 | 39.0 | English | Depression | 0.42 |
| Pretorius (1992) | Students | 450 | 24.0 | English | Depression | 0.31 |
| Bonner and Rich (1992) | Correctional inmates | 146 | 28.0 | English | Depression | 0.52 |
| Elliott et al. (1991) | Persons with spinal cord injury | 90 | 32.6 | English | Depression | 0.50 |
| Larson et al. (1990) | Students | 206 | N/A | English | Depression | 0.44 |
| Larson et al. (1990) | Students | 206 | N/A | English | Anxiety | 0.42 |
| Larson et al. (1990) | Students | 206 | N/A | English | Anxiety | 0.51 |
| Larson et al. (1990) | Students | 237 | N/A | English | Depression | 0.25 |
| Larson et al. (1990) | Students | 237 | N/A | English | Anxiety | 0.35 |
| Larson et al. (1990) | Students | 237 | N/A | English | Anxiety | 0.46 |
| Bonner and Rich (1988) | Students | 186 | N/A | English | Depression | 0.58 |
| Nezu (1986) | Students | 310 | 24.3 | English | Anxiety | 0.39 |
| Nezu (1986) | Students | 310 | 24.3 | English | Anxiety | 0.51 |
| Nezu et al. (1986) | Students | 128 | 22.7 | English | Depression | 0.41 |
N/A, data not reported.
Across the 49 studies, 66 correlation coefficients were available for analysis. The studies included a total of 18,834 participants, with a mean sample size of 285.4 (SD = 481.7; range = 28–2,889). The mean participant age across studies was 30.9 years (SD = 15.3; range = 14.8–77.3). The included effect sizes were geographically diverse, although they were concentrated primarily in the USA. Of the 66 coefficients, 24 (36.4%) were derived from studies conducted in the USA, 6 (9.1%) from Turkey, and 5 (7.6%) from South Africa. Smaller numbers of coefficients were contributed by studies from Hong Kong and Norway (n = 3 each, 4.5%), and from Australia, Canada, Singapore, Taiwan, and the UK (n = 1 each, 1.5%). Country information was not available for 20 coefficients (30.3%).
3.2. Combined effect size
Figure 2 presents the forest plot for the relationship between problem-solving appraisal and internalizing disorders. The pooled effect size across all included studies was 0.40 (95% CI [0.38, 0.41]), indicating a medium association between more negative problem-solving appraisal and higher levels of internalizing disorders. Study weights ranged from 0.59 to 2.11%, suggesting that no single study exerted undue influence on the combined effect size. However, I2 = 80.04% indicating substantial heterogeneity of effect sizes.
Figure 2.
Forest plot of the relationship between problem-solving appraisal and internalizing disorders.
3.3. Moderator analysis
Meta-regression was used to examine whether mean age potentially moderated the relationship between problem-solving appraisal and internalizing disorders. The results indicate that age was not a significant moderator of this association (B = −0.001, z = −1.11, p = 0.27).
3.4. Subgroup analyses
Subgroup analyses were conducted to determine whether the pooled effect size differed according to the language version of the PSI used or according to the specific internalizing disorder outcome examined. Figures 3 and 4 present the relevant forest plots, and Table 2 provides the statistical comparisons of subgroup differences.
Figure 3.
Forest plots of the relationship between problem-solving appraisal and internalizing disorders for studies that used a translated version versus studies that used an English version of the PSI.
Figure 4.
Forest plots of the relationship between problem-solving appraisal and internalizing disorders for studies that focused on anxiety versus studies that focused on depression.
Table 2.
Comparison of combined effect sizes (r and 95% confidence intervals for different subgroups.
| Subgroup | n | Combined effect | 95% CI | z | p | |
|---|---|---|---|---|---|---|
| LL | UL | |||||
| English version of PSI | 49 | 0.39 | 0.37 | 0.41 | −0.60 | 0.55 |
| Translated version of PSI | 17 | 0.42 | 0.38 | 0.45 | ||
| PSI and anxiety | 19 | 0.40 | 0.37 | 0.43 | 0.00 | 1.00 |
| PSI and depression | 47 | 0.40 | 0.38 | 0.42 | ||
For studies that used the original English version of the PSI, the pooled effect size was 0.39 (95% CI [0.37, 0.41]). For studies that used a translated version of the PSI, the pooled effect size was 0.42 (95% CI [0.38, 0.45]). This difference was not statistically significant (z = −0.60, p = 0.55), indicating that the observed association did not differ meaningfully by language version of the PSI.
A second subgroup analysis compared studies examining anxiety with those examining depression. The pooled effect size for studies focusing on anxiety was 0.40 (95% CI [0.37, 0.43]), while the pooled effect size for studies focusing on depression was also 0.40 (95% CI [0.38, 0.42]). The difference between these subgroup estimates was not statistically significant (z = 0.00, p = 1.00), suggesting that the magnitude of the association was similar across these two internalizing outcomes.
3.5. Accounting for effect size dependencies
Since multiple effect sizes were available from some studies, RVE was used to account for effect size dependencies. The RVE analysis yielded a pooled effect size of 0.40 (95% CI [0.36, 0.43]), which was substantively identical to the original pooled effect size of 0.40 (95% CI [0.38, 0.41]). The difference between the two estimates was not statistically significant (z = 0.00, p = 1.00), indicating that dependence among effect sizes did not materially affect the findings.
3.6. Publication Bias
Multiple analyses indicated the presence of publication bias. Egger’s regression test was significant (z = 2.33, p = 0.02), suggesting asymmetry in the distribution of effect sizes. In addition, weighting studies according to their probability of being published produced a pooled effect size of 0.38 (95% CI [0.35, 0.42]), which differed significantly from the unweighted pooled effect size of 0.40 (95% CI [0.38, 0.41]; χ2 = 4.57, p = 0.032).
The funnel plot is presented in Figure 5, with correlation coefficients expressed as Fisher’s z. Visual inspection suggested asymmetry in the distribution of studies around the pooled effect size when the imputed studies were not considered. The trim-and-fill procedure imputed 16 studies to restore symmetry. After adjustment, the pooled effect size was 0.38 (95% CI [0.34, 0.41]), which was lower than the original estimate but remained within the medium effect size range. Taken together, these findings suggest evidence of publication bias, although the adjusted estimate remained broadly consistent with the original conclusion regarding the magnitude of the association.
Figure 5.
Funnel plot for inspection of symmetry of effect sizes around the combined effect size.
4. Discussion
The present meta-analysis aimed to investigate the relationship between problem-solving appraisal, as assessed by the PSI, and internalizing disorders, specifically depression and anxiety. This analysis incorporated data from 49 studies, and there were several salient findings. First, the effect size of the relationship between problem-solving appraisal and internalizing disorders was found to be medium. This finding signifies that individuals who appraise their problem-solving abilities negatively tend to exhibit higher levels of internalizing disorders and supports cognitive model predictions that self-appraisals are a central factor in the onset and maintenance of mental health disorders.
The present study findings regarding self-appraisal are comparable to study findings regarding self-efficacy, as both constructs refer to an individual’s belief in their ability to manage and execute the actions necessary to achieve specific outcomes. Both problem-solving appraisal and self-efficacy involve self-perceptions of competence, which are critical in coping with stressors. Lonigro and colleagues (Lonigro et al., 2023) reported that emotional self-efficacy (i.e., appraisal of one’s emotional competence in regulating affective states) impacted internalizing behavior among adolescents, while Lau and colleagues (Lau et al., 2023) found self-appraisal biases among children with social anxiety. Karr and White (Karr and White, 2024) reported that lower academic self-efficacy was associated with higher levels of anxiety and depression among university students. Existing research has also highlighted the role of problem-solving appraisal in mental health outcomes (Chow and Chan, 2010; Padmanabhanunni and Pretorius, 2023).
The second major finding of this study is that age was not a significant moderator of the relationship between problem-solving appraisal and depression and anxiety, respectively. This finding suggests that the association between problem-solving appraisal and internalizing disorders is consistent across age groups. This finding is important because it implies that interventions aimed at improving problem-solving appraisal could be beneficial across a broad age range. Problem-solving therapy, which involves training in adaptive problem-solving attitudes and skills, has been effective in treating common mental health disorders (Zhang et al., 2018; Shang et al., 2021). Additional training in modifying dysfunctional appraisals of problem-solving competencies could be incorporated into such interventions. Interventions can specifically target these maladaptive appraisals to help individuals develop a more positive and realistic view of their problem-solving abilities. These positive self-appraisals can then lead to increased confidence and motivation in engaging in problem-solving activities, which may further reduce symptoms of depression and anxiety.
Third, the analysis found that the combined effect sizes were similar for studies that used the original English version of the PSI and those that used a translated version. Both groups of studies reported a medium effect size, which suggests a moderate relationship between problem-solving appraisal and internalizing disorders, regardless of the language of PSI administration. The lack of significant differences in effect sizes between studies that used translated contrasted with studies that used the original version of the PSI underscores the robustness of the PSI as a tool for assessing problem-solving appraisal across different languages and cultural contexts, potentially validating its cross-cultural applicability.
Fourth, the analysis found that the combined effect sizes were similar for studies that examined the relationship between problem-solving appraisal and anxiety and those that focused on the relationship between problem-solving appraisal and depression. There was no significant difference between these two combined effect sizes. This result indicates that problem-solving appraisal is equally associated with both anxiety and depression, reinforcing the notion that problem-solving appraisal is salient in the context of various internalizing disorders.
Fifth, after accounting for effect size dependencies the results indicated that there was no statistically significant difference between the adjusted and original combined effect sizes. This consistency in the combined effect size suggests that the overall findings of the meta-analysis are robust and the observed relationship between problem-solving appraisal and internalizing disorders is not influenced by dependencies among effect sizes within studies. This finding strengthens the reliability of the results and supports the conclusion that problem-solving appraisal has a moderate and consistent association with internalizing disorders.
Finally, the findings confirmed the presence of publication bias, such that studies with larger effect sizes were more likely to be published, potentially inflating the overall effect size reported in the meta-analysis. The issue of publication bias in meta-analytic research has been well documented, and its detection has important implications for interpreting the results of this meta-analysis (McShane et al., 2016). While the overall effect size remains in the medium range, the presence of bias suggests that the true effect size may be slightly smaller than initially reported. This finding highlights the need for cautious interpretation of meta-analytic findings and the importance of considering publication bias when evaluating the strength of the relationship between problem-solving appraisal and internalizing disorders.
An important contextual consideration is that a substantial proportion of the included studies were conducted with student samples. The observed association between problem-solving appraisal and internalizing disorders may therefore be influenced by the characteristics of younger and more educated populations, among whom the nature and consequences of negative problem-solving appraisals may differ from those observed in broader community or clinical groups.
This study has several limitations. The included studies exhibited significant heterogeneity in terms of sample characteristics, methodologies, and measures of internalizing disorders, and this variability may influence the generalizability of the findings. The majority of the studies included in the meta-analysis were cross-sectional, which limits the ability to draw causal inferences about the relationship between problem-solving appraisal and internalizing disorders. Longitudinal studies are needed to better understand the directionality and causality of this relationship. Most of the included studies relied on self-reported measures to assess problem-solving appraisal and internalizing disorders. Self-report instruments are subject to biases such as social desirability and response style, which can affect the accuracy of the data. A further limitation is that a large proportion of the included studies involved student populations. This may restrict the generalizability of the findings, as the relationship between problem-solving appraisal and internalizing disorders may differ across age groups, educational backgrounds, and clinical versus non-clinical populations.
5. Conclusion
In conclusion, this meta-analysis provides evidence for a moderate relationship between problem-solving appraisal and internalizing disorders. However, the presence of publication bias suggests that the observed effect size may be inflated. These findings underscore the importance of problem-solving appraisal in the context of mental health. It suggests that negative problem-solving appraisal may represent a useful intervention target in cognitive-behavioral and problem-solving-based approaches, particularly through strategies aimed at restructuring maladaptive self-appraisals and strengthening confidence in coping and problem resolution. The study also highlights the need for ongoing efforts to address publication bias in meta-analytic research.
Funding Statement
The author(s) declared that financial support was not received for this work and/or its publication.
Footnotes
Edited by: Adelinda Araujo Candeias, University of Evora, Portugal
Reviewed by: Flora Furente, University of Bari Aldo Moro, Italy
José Carlos Ramírez Cruz, University of Guadalajara, Mexico
Author contributions
TP: Writing – review & editing, Methodology, Writing – original draft, Conceptualization, Investigation, Visualization, Formal analysis. AP: Project administration, Conceptualization, Supervision, Writing – review & editing, Writing – original draft, Investigation.
Conflict of interest
The author(s) declared that this work was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Generative AI statement
The author(s) declared that Generative AI was not used in the creation of this manuscript.
Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
- American Psychiatric Association (2022). Diagnostic and Statistical Manual of mental Disorders: DSM-5-TR. Washington, DC: American Psychiatric Association Publishing. [Google Scholar]
- Bastian V. A., Burns N. R., Nettelbeck T. (2005). Emotional intelligence predicts life skills, but not as well as personality and cognitive abilities. Pers. Individ. Differ. 39, 1135–1145. doi: 10.1016/j.paid.2005.04.006 [DOI] [Google Scholar]
- Beck A. T., Weissman A., Lester D., Trexler L. (1974). The measurement of pessimism: the hopelessness scale. J. Consult. Clin. Psychol. 42, 861–865. doi: 10.1037/h0037562, [DOI] [PubMed] [Google Scholar]
- Bendau A., Kunas S. L., Wyka S., Petzold M. B., Plag J., Asselmann E., et al. (2021). Longitudinal changes of anxiety and depressive symptoms during the COVID-19 pandemic in Germany: the role of pre-existing anxiety, depressive, and other mental disorders. J. Anxiety Disord. 79, 102377–102377. doi: 10.1016/j.janxdis.2021.102377, [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bonner R. L., Rich A. (1988). Negative life stress, social problem-solving self-appraisal, and hopelessness: implications for suicide research. Cogn. Ther. Res. 12, 549–556. doi: 10.1007/BF01205009 [DOI] [Google Scholar]
- Bonner R. L., Rich A. R. (1992). Cognitive vulnerability and hopelessness among correctional inmates: a state of mind model. J. Offender Rehabil. 17, 113–122. doi: 10.1300/J076v17n03_08 [DOI] [Google Scholar]
- Cha E., Braxter B. J., Kim K. H., Lee H., Akazawa M. K., Talman M. S., et al. (2015). Preventive strategies to reduce depressive symptoms in overweight and obese young adults. Arch. Psychiatr. Nurs. 29, 258–264. doi: 10.1016/j.apnu.2015.04.002, [DOI] [PMC free article] [PubMed] [Google Scholar]
- Chan D. W. (2001). Dimensionality and correlates of problem solving: the use of the problem solving inventory in the Chinese context. Behav. Res. Ther. 39, 859–875. doi: 10.1016/S0005-7967(00)00082-6, [DOI] [PubMed] [Google Scholar]
- Chow S. K., Chan W. C. (2010). Depression: problem-solving appraisal and self-rated health among Hong Kong Chinese migrant women. Nurs. Health Sci. 12, 352–359. doi: 10.1111/j.1442-2018.2010.00537.x, [DOI] [PubMed] [Google Scholar]
- Clark L. A., Watson D. (1991). Tripartite model of anxiety and depression: psychometric evidence and taxonomic implications. J. Abnorm. Psychol. 100, 316–336. doi: 10.1037/0021-843X.100.3.316, [DOI] [PubMed] [Google Scholar]
- Danneel S., Geukens F., Maes M., Bastin M., Bijttebier P., Colpin H. (2020, 2020). Loneliness, social anxiety symptoms, and depressive symptoms in adolescence: longitudinal distinctiveness and correlated change. J. Youth. Adolesc. 49, 2246–2264. doi: 10.1007/s10964-020-01315-w [DOI] [PubMed] [Google Scholar]
- del-Valle M. V., López-Morales H., Gelpi-Trudo R., Poó F. M., García M. J., Yerro-Avincetto M., et al. (2022). More than a year of pandemic: longitudinal assessment of anxiety and depressive symptoms in the argentine general population during the COVID-19 outbreak. Stress. Health 38, 1070–1079. doi: 10.1002/smi.3163, [DOI] [PMC free article] [PubMed] [Google Scholar]
- Dieserud G., Forsén L., Braverman M. T., Røysamb E. (2002). Negative life events in childhood, psychological problems and suicide attempts in adulthood: a matched case-control study. Arch. Suicide Res. 6, 291–308. doi: 10.1080/13811110214525 [DOI] [Google Scholar]
- Dieserud G., Røysamb E., Braverman M. T., Dalgard O. S., Ekeberg Ø. (2003). Predicting repetition of suicide attempt: a prospective study of 50 suicide attempters. Arch. Suicide Res. 7, 1–15. doi: 10.1080/13811110301571 [DOI] [Google Scholar]
- Dieserud G., Røysamb E., Ekeberg Ø., Kraft P. (2001). Toward an integrative model of suicide attempt: a cognitive psychological approach. Suicide Life Threat. Behav. 31, 153–168. doi: 10.1521/suli.31.2.153.21511, [DOI] [PubMed] [Google Scholar]
- Dixon W. A. (2000). Problem-solving appraisal and depression: evidence for a recovery model. J. Couns. Dev. 78, 87–91. doi: 10.1002/j.1556-6676.2000.tb02564.x [DOI] [Google Scholar]
- Dixon W. A., Heppner P. P., Burnett J. W., Anderson W. P., Wood P. K. (1993). Distinguishing among antecedents, concomitants, and consequences of problem-solving appraisal and depressive symptoms. J. Couns. Psychol. 40, 357–364. doi: 10.1037/0022-0167.40.3.357 [DOI] [Google Scholar]
- Elliott T. R., Godshall F. J., Herrick S. M., Witty T. E., Spruell M. (1991). Problem-solving appraisal and psychological adjustment following spinal cord injury. Cogn. Ther. Res. 15, 387–398. doi: 10.1007/BF01173033 [DOI] [PubMed] [Google Scholar]
- Elliott T. R., Herrick S. M., Witty T. E. (1992). Problem-solving appraisal and the effects of social support among college students and persons with physical disabilities. J. Couns. Psychol. 39, 219–226. doi: 10.1037/0022-0167.39.2.219 [DOI] [Google Scholar]
- Erdur-Baker Ö. (2009). Peer victimization, rumination, and problem solving as risk contributors to adolescents' depressive symptoms. J. Psychol. 143, 78–90. doi: 10.3200/JRLP.143.1.78-90, [DOI] [PubMed] [Google Scholar]
- Eremsoy C. E., Çelimli Ş., Gençöz T. (2005). Students under academic stress in a Turkish university: variables associated with symptoms of depression and anxiety. Curr. Psychol. 24, 123–133. doi: 10.1007/s12144-005-1011-z [DOI] [Google Scholar]
- Eskin M., Ertekin K., Harlak H., Dereboy C. (2008). Prevalence of and factors related to depression in high school students. Turk Psikiyatri Derg. 19, 382–389. Available online at: https://pubmed.ncbi.nlm.nih.gov/19110980/, [PubMed] [Google Scholar]
- Eysenck M. W., Fajkowska M. (2018). Anxiety and depression: toward overlapping and distinctive features. Cognit. Emot. 32, 1391–1400. doi: 10.1080/02699931.2017.1330255, [DOI] [PubMed] [Google Scholar]
- Fisher Z., Tipton E. (2015). Robumeta: an R-package for robust variance estimation in meta-analysis. Availabe online at: 10.48550/arXiv.1503.02220 (Accessed March 7, 2015) [DOI]
- Good G. E., Heppner P. P., DeBord K. A., Fischer A. R. (2004). Understanding men's psychological distress: contributions of problem-solving appraisal and masculine role conflict. Psychol. Men Masculin. 5, 168–177. doi: 10.1037/1524-9220.5.2.168 [DOI] [Google Scholar]
- Hanson K. M., Mintz L. B. (1997). Psychological health and problem-solving self-appraisal in older adults. J. Couns. Psychol. 44, 433–441. doi: 10.1037/0022-0167.44.4.433 [DOI] [Google Scholar]
- Hedges L. V., Tipton E., Johnson M. C. (2010). Robust variance estimation in meta-regression with dependent effect size estimates. Res. Synth. Methods 1, 39–65. doi: 10.1002/jrsm.5, [DOI] [PubMed] [Google Scholar]
- Heppner P. P. (1982). Personal problem solving: a descriptive study of individual differences. J. Couns. Psychol. 29, 580–590. doi: 10.1037//0022-0167.29.6.580 [DOI] [Google Scholar]
- Heppner P. (1988). The Problem Solving Inventory. Palo Alto, CA: Consulting Psychologists Press. [Google Scholar]
- Heppner P. P., Baumgardner A. H., Larson L. M., Petty R. E. (1988). The utility of problem-solving training that emphasises self-management principles. Couns. Psychol. Q. 1, 129–143. doi: 10.1080/09515078808254196 [DOI] [Google Scholar]
- Heppner P. P., Heppner M. J., Lee D.-G., Wang Y.-W., Park H.-J., Wang L.-F. (2006). Development and validation of a collectivist coping styles inventory. J. Couns. Psychol. 53, 107–125. doi: 10.1037/0022-0167.53.1.107 [DOI] [Google Scholar]
- Heppner P. P., Pretorius T. B., Wei M., Lee D.-G., Wang Y.-W. (2002). Examining the generalizability of problem-solving appraisal in black south Africans. J. Couns. Psychol. 49, 484–498. doi: 10.1037/0022-0167.49.4.484 [DOI] [Google Scholar]
- Heppner P. P., Witty T. E., Dixon W. A. (2004). Problem-solving appraisal and human adjustment: a review of 20 years of research using the problem solving inventory. Couns. Psychol. 32, 344–428. doi: 10.1177/0011000003262793 [DOI] [Google Scholar]
- Higgins J. P. T., Thompson S. G. (2002). Quantifying heterogeneity in a meta-analysis. Stat. Med. 21, 1539–1558. doi: 10.1002/sim.1186, [DOI] [PubMed] [Google Scholar]
- Ji M., Sereika S. M., Rohay J. M., Erlen J. A. (2018). Psychometric properties of the problem solving inventory in caregivers of individuals with memory loss. J. Gerontol. Nurs. 44, 25–32. doi: 10.3928/00989134-20180509-05, [DOI] [PubMed] [Google Scholar]
- Julal F. S. (2016). Predictors of undergraduate students’ university support service use during the first year of university. Br. J. Guid. Couns. 44, 371–381. doi: 10.1080/03069885.2015.1119232 [DOI] [Google Scholar]
- Karr J. E., White A. E. (2024). Academic self-efficacy and cognitive strategy use in college students with and without depression or anxiety. J. Am. Coll. Heal. 72, 1387–1393. doi: 10.1080/07448481.2022.2076561, [DOI] [PubMed] [Google Scholar]
- Kenntemich L., von Hülsen L., Eggert L., Kriston L., Gallinat J., Schäfer I., et al. (2024). Trajectories of depressive and anxiety symptoms and associated risk factors during the COVID-19 pandemic in Germany: a longitudinal cohort study. J. Affect. Disord. 355, 136–146. doi: 10.1016/j.jad.2024.03.131, [DOI] [PubMed] [Google Scholar]
- Kerns R. D., Rosenberg R., Otis J. D. (2002). Self-appraised problem solving and pain-relevant social support as predictors of the experience of chronic pain. Ann. Behav. Med. 24, 100–105. doi: 10.1207/S15324796ABM2402_06, [DOI] [PubMed] [Google Scholar]
- Korkmaz S., Kazgan A., Çekiç S., Tartar A. S., Balcı H. N., Atmaca M. (2020b). The anxiety levels, quality of sleep and life and problem-solving skills in healthcare workers employed in COVID-19 services. J. Clin. Neurosci. 80, 131–136. doi: 10.1016/j.jocn.2020.07.073, [DOI] [PMC free article] [PubMed] [Google Scholar]
- Korkmaz S., Keleş D., Kazgan A., Baykara S., Gürkan Gürok M., Feyzi Demir C., et al. (2020a). Emotional intelligence and problem solving skills in individuals who attempted suicide. J. Clin. Neurosci. 74, 120–123. doi: 10.1016/j.jocn.2020.02.023, [DOI] [PubMed] [Google Scholar]
- Larson L. M., Piersel W. C., Imao R. A. K., Allen S. J. (1990). Significant predictors of problem-solving appraisal. J. Couns. Psychol. 37, 482–490. doi: 10.1037/0022-0167.37.4.482 [DOI] [Google Scholar]
- Lau N., Zhou A. M., Yuan A., Parigoris R., Rosenberg A. R., Weisz J. R. (2023). Social skills deficits and self-appraisal biases in children with social anxiety disorder. J. Child Fam. Stud. 32, 2889–2900. doi: 10.1007/s10826-021-02194-w, [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lazarus R. S., Folkman S. (1984). Stress, Appraisal, and Coping. Berlin: Springer. [Google Scholar]
- Lonigro A., Longobardi E., Laghi F. (2023). The interplay between expressive suppression, emotional self-efficacy and internalizing behavior in middle adolescence. Child Youth Care Forum 52, 253–265. doi: 10.1007/s10566-022-09685-x [DOI] [Google Scholar]
- Marcotte D., Alain M., Gosselin M. J. (1999). Gender differences in adolescent depression: gender-typed characteristics or problem-solving skills deficits? Sex Roles 41, 31–48. doi: 10.1023/A:1018833607815 [DOI] [Google Scholar]
- Mayo V. D., Tanaka-Matsumi J. (1996). Think aloud statements and solutions of dysphoric persons on a social problem-solving task. Cogn. Ther. Res. 20, 97–113. doi: 10.1007/BF02228029 [DOI] [Google Scholar]
- McGrath R. E., Ratliff K. G. (1993). Using self-report measures to corroborate theories of depression: the specificity problem. J. Pers. Assess. 61, 156–168. doi: 10.1207/s15327752jpa6101_12, [DOI] [PubMed] [Google Scholar]
- McShane B. B., Böckenholt U., Hansen K. T. (2016). Adjusting for publication Bias in Meta-analysis: an evaluation of selection methods and some cautionary notes. Perspect. Psychol. Sci. 11, 730–749. doi: 10.1177/1745691616662243, [DOI] [PubMed] [Google Scholar]
- Miner R. C., Dowd E. T. (1996). An empirical test of the problem solving model of depression and its application to the prediction of anxiety and anger. Couns. Psychol. Q. 9, 163–176. doi: 10.1080/09515079608256361 [DOI] [Google Scholar]
- Moher D., Liberati A., Tetzlaff J., Altman D. G. (2009). Reprint—preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Phys. Ther. 89, 873–880. doi: 10.1093/ptj/89.9.873, [DOI] [PubMed] [Google Scholar]
- Nezu A. M. (1986). Negative life stress and anxiety: problem solving as a moderator variable. Psychol. Rep. 58, 279–283. doi: 10.2466/pr0.1986.58.1.279, [DOI] [PubMed] [Google Scholar]
- Nezu A. M., Kalmar K., Ronan G. F., Clavijo A. (1986). Attributional correlates of depression: an interactional model including problem solving. Behav. Ther. 17, 50–56. doi: 10.1016/S0005-7894(86)80113-7 [DOI] [Google Scholar]
- Padmanabhanunni A., Pretorius T. B. (2023). Cognitive adaptation to stress and trauma: the role of self-appraised problem-solving in posttraumatic stress disorder. Chronic Stress 7:24705470231189980. doi: 10.1177/24705470231189980, [DOI] [PMC free article] [PubMed] [Google Scholar]
- Padmanabhanunni A., Wiid C. (2022). From fear to fortitude: differential vulnerability to PTSD among south African university students. Traumatology 28, 167–174. doi: 10.1037/trm0000312 [DOI] [Google Scholar]
- Perez R. M., Thoreson R. W., Pallon M. J., Heppner P. P. (1997). Problem solving appraisal and narcissism among delinquent adolescents. J. Offender Rehabil. 26, 19–44. doi: 10.1300/J076v26n01_02 [DOI] [Google Scholar]
- Pickett T., Jr., Altmaier E., Paulsen J. S. (2007). Caregiver burden in Huntington’s disease. Rehabil. Psychol. 52, 311–318. doi: 10.1037/0090-5550.52.3.311 [DOI] [Google Scholar]
- Pretorius T. B. (1992). Problem-solving appraisal in the association of life stress and depression: a south African study. Psychol. Rep. 71, 855–862. doi: 10.2466/pr0.1992.71.3.855, [DOI] [PubMed] [Google Scholar]
- Pretorius T. B. (2020). Pathways to health: conceptual clarification and appropriate statistical treatment of mediator, moderator, and indirect effects using examples from burnout research. S. Afr. J. Psychol. 50, 320–335. doi: 10.1177/0081246320943498 [DOI] [Google Scholar]
- Pretorius T. B., Diedricks M. (1994). Problem-solving appraisal, social support and stress-depression relationship. S. Afr. J. Psychol. 24, 86–90. doi: 10.1177/008124639402400206 [DOI] [Google Scholar]
- Pretorius T. B., Heppner P. P., Padmanabhanunni A., Isaacs S. A. (2023). The PSI-20: development of a viable short form alternative of the problem solving inventory using item response theory. SAGE Open 13:21582440231215633. doi: 10.1177/21582440231215633 [DOI] [Google Scholar]
- Priester M. J., Clum G. A. (1993). Perceived problem-solving ability as a predictor of depression, hopelessness, and suicide ideation in a college population. J. Couns. Psychol. 40, 79–85. doi: 10.1037/0022-0167.40.1.79 [DOI] [Google Scholar]
- Quintana D. S. (2015). From pre-registration to publication: a non-technical primer for conducting a meta-analysis to synthesize correlational data. Front. Psychol. 6, 1549–1549. doi: 10.3389/fpsyg.2015.01549, [DOI] [PMC free article] [PubMed] [Google Scholar]
- R Development Core Team (2020). R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing. [Google Scholar]
- Reid J. K., Dixon W. A. (2000). The relationships among grief experience, problem-solving appraisal, and depression: an exploratory study. J. Pers. Interpers. Loss 5, 77–93. doi: 10.1080/10811440008407848 [DOI] [Google Scholar]
- Sahin N., Sahin N. H., Heppner P. P. (1993). Psychometric properties of the problem solving inventory in a group of Turkish university students. Cogn. Ther. Res. 17, 379–396. doi: 10.1007/BF01177661 [DOI] [Google Scholar]
- Shang P., Cao X., You S., Feng X., Li N., Jia Y. (2021). Problem-solving therapy for major depressive disorders in older adults: an updated systematic review and meta-analysis of randomized controlled trials. Aging Clin. Exp. Res. 33, 1465–1475. doi: 10.1007/s40520-020-01672-3, [DOI] [PubMed] [Google Scholar]
- Shoshani A., Kor A. (2022). The mental health effects of the COVID-19 pandemic on children and adolescents: risk and protective factors. Psychol. Trauma 14, 1365–1373. doi: 10.1037/tra0001188, [DOI] [PubMed] [Google Scholar]
- Starr L. R., Davila J. (2012). Responding to anxiety with rumination and hopelessness: mechanism of anxiety-depression symptom co-occurrence? Cogn. Ther. Res. 36, 321–337. doi: 10.1007/s10608-011-9363-1, [DOI] [PMC free article] [PubMed] [Google Scholar]
- Suurmond R., Rhee H., Hak T. (2017). Introduction, comparison, and validation of Meta-essentials: a free and simple tool for meta-analysis. Res. Synth. Methods 8, 537–553. doi: 10.1002/jrsm.1260, [DOI] [PMC free article] [PubMed] [Google Scholar]
- Suzuki L. A., Ahluwalia M. K. (2004). Two decades of research on the problem solving inventory: a call for empirical clarity. Couns. Psychol. 32, 429–438. doi: 10.1177/0011000003262800 [DOI] [Google Scholar]
- Tang F., Jang H., Lingler J., Tamres L. K., Erlen J. A. (2015). Stressors and caregivers’ depression: multiple mediators of self-efficacy, social support, and problem-solving skill. Soc. Work Health Care 54, 651–668. doi: 10.1080/00981389.2015.1054058, [DOI] [PMC free article] [PubMed] [Google Scholar]
- Teo D. C. H., Suárez L., Oei T. P. S. (2021). Psychometric properties of the problem solving inventory in a Singapore young male adult sample. Curr. Psychol. 40, 1420–1428. doi: 10.1007/s12144-018-0073-7 [DOI] [Google Scholar]
- Viechtbauer W. (2010). Conducting meta-analyses in R with the metafor package. J. Stat. Softw. 36, 1–48. doi: 10.18637/jss.v036.i03 [DOI] [Google Scholar]
- Witty T. E., Heppner P. P., Bernard C. B., Thoreson R. W. (2001). Problem-solving appraisal and psychological adjustment of persons with chronic low-back pain. J. Clin. Psychol. Med. Settings 8, 149–160. doi: 10.1023/A:1011361418436 [DOI] [Google Scholar]
- Yeung S., Lui M. H., Ross F., Murrells T. (2007). Family carers in stroke care: examining the relationship between problem-solving, depression and general health. J. Clin. Nurs. 16, 344–352. doi: 10.1111/j.1365-2702.2005.01537.x, [DOI] [PubMed] [Google Scholar]
- Zhang A., Park S., Sullivan J. E., Jing S. (2018). The effectiveness of problem-solving therapy for primary care patients' depressive and/or anxiety disorders: a systematic review and meta-analysis. J. Am. Board Fam. Med. 31, 139–150. doi: 10.3122/jabfm.2018.01.170270, [DOI] [PubMed] [Google Scholar]
- Zimmermann M., Chong A. K., Vechiu C., Papa A. (2020). Modifiable risk and protective factors for anxiety disorders among adults: a systematic review. Psychiatry Res. 285, 112705–112705. doi: 10.1016/j.psychres.2019.112705, [DOI] [PubMed] [Google Scholar]





