Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2015 Sep 30.
Published in final edited form as: Psychiatry Res. 2015 Jun 29;229(0):440–446. doi: 10.1016/j.psychres.2015.06.033

The relationships between patients’ and caregivers’ beliefs about the causes of schizophrenia and clinical outcomes in Latin American countries

Alejandra Caqueo-Urízar a,b,*, Laurent Boyer c, Karine Baumstarck c, Stephen E Gilman b,d,e
PMCID: PMC4546864  NIHMSID: NIHMS702218  PMID: 26188641

Abstract

Beliefs about the causes of schizophrenia are thought to impact treatment outcomes. We investigated 3 theoretically opposing belief systems (biological, psychosocial, magical-religious) in relation to the severity of positive and negative symptoms and to attitudes towards medications. We recruited 253 patients with schizophrenia and their primary caregivers from public mental health clinics in Bolivia, Chile, and Peru. We assessed patients’ and caregivers’ beliefs about the causes of schizophrenia, which were used as predictors of symptom severity and medication attitudes in linear regression analyses. Patients’ scores on biological, psychosocial, and magical-religious beliefs were positively correlated with one another, indicating that these domains were not, as anticipated, “opposing.” Patients with higher levels of biological and psychosocial beliefs had significantly lower levels of positive and negative symptoms; in contrast, higher levels of magical-religious beliefs were associated with increased positive symptoms and less favorable attitudes towards medications. Patients’ belief systems are significant predictors of symptom severity and medication attitudes. Research is needed on the extent to which psychotherapeutic treatments for schizophrenia should bolster patients’ beliefs in the biological and psychosocial domains and weaken beliefs in the magical-religious domain; this research should also attend to the ethical considerations involved in intervening on belief systems cross-culturally.

Keywords: Schizophrenia, Causal beliefs, Caregivers, Health outcomes, Symptoms, Attitudes towards medication

1 INTRODUCTION

Patients’ and caregivers’ beliefs about the causes of schizophrenia (which we term causal beliefs) influence help-seeking behaviors, adherence to treatment, disease management and clinical outcomes (Phillips et al., 2000; Saravanan et al., 2007). Causal beliefs are usually classified along three dimensions: biological (schizophrenia is a brain disorder due in part to hereditary factors), psychosocial (schizophrenia is caused by external factors such as economic conditions, stressors or level of educational attainment) and magical-religious (schizophrenia originates from supernatural or spiritual forces). In patients with schizophrenia and their caregivers, studies reported a preference for attributing schizophrenia to psychosocial causes in diverse cultures, such as the United Kingdom (Pollock, 1988), Turkey (Karanci, 1995), China (Phillips et al., 2000) Germany (Holzinger et al., 2003) and Nigeria (Ilechukwu, 1988). Supernatural causes were common beliefs in patients with schizophrenia in India (Saravanan et al., 2007; Kate et al., 2012) and Israel (Al-Krenawi, 1999). In contrast, biological causes were mentioned rarely (Holzinger et al., 2003).

Studies show that patients’ and caregivers’ causal models of illness should match the biomedical and biological professionals' causal model (Angermeyer and Matschinger, 1996; Lobban and Barrowclough, 2005; Balhara and Yadav, 2012; Wiesjahn et al., 2014). The main assumption is that a biological causal model can help the patient understand the nature of the disease and the relevance of pharmacological treatment, improving engagement with psychiatric services and consequently clinical outcomes (Wiesjahn et al., 2014). Thus, some studies reported that patients with low-level symptoms and reasonably accurate insights into illness were more likely to accept biological causal beliefs, whereas patients with more severe symptoms more frequently held non-biomedical causal beliefs (e.g., magical-religious beliefs) (Saravanan et al., 2007). Patients who held magical-religious causal beliefs tended to replace or delay medical treatment (Saravanan et al., 2007; Borras et al., 2007). Similarly, studies of caregivers’ causal beliefs showed that magical-religious beliefs were strongly related to lack of psychiatric treatment for patients with schizophrenia (Kurihara et al., 2006). It has been also reported that knowing the caregivers’ beliefs may be fruitful for clinicians and help them form stronger therapeutic alliances with families (Brewin, 1994; Karanci, 1995; Phillips et al., 2000). In accordance with these findings, psychosocial treatments routinely focus on biological causal model of illness (e.g., cognitive behavioral therapy, psycho-education) (Sibitz et al., 2005; Kern et al., 2009).

However, several questions remain, the answers to which are needed to guide the optimal use of causal models by clinicians as they give more attention to cultural background in the therapeutic process.

First, despite the general agreement described above, the choice of the causal model to communicate has to be considered. Previous studies have mainly focused on pharmacological treatment. However, medications alone are not sufficient for recovery in schizophrenia, and psychosocial treatments are a necessary complement (Kern et al., 2009). Three recent studies suggested that a psychosocial causal model could be relevant for treatment success by promoting non-pharmacological treatment (Sanders, et al., 2011; Lüllmann et al., 2011; Freeman et al., 2012). Causal beliefs may also act in divergent ways across the spectrum of treatment. For example, if biomedical models of beliefs can be used to facilitate treatment adherence, psychosocial beliefs may in turn facilitate the acceptance of psychosocial interventions. It is therefore important to understand the relative influences of the different causal models on adherence to treatment and severity of symptoms. Second, findings about patient and caregiver beliefs as a determinant of clinical outcomes draw on a separate literature, and should now be studied together. Finally, there is cultural variability in the beliefs about the causes of schizophrenia between countries as well as their relationships with clinical outcomes (Saravanan et al., 2007; Zafar et al., 2008; Sanders et al., 2011). Previous findings should thus be extended to Latin American countries where no published studies have investigated this issue.

Cultural background provides a framework for understanding the origins of psychiatric illness as well as for interpreting the changes in mental functioning that family members observe in their relative experiencing a psychiatric disorder – for example, through shared “social representations” of disorder (Holzinger et al., 2003). Features of the mental health system itself may also influence family members’ causal beliefs (Kurihara et al., 2006), and are also important to consider in studying causal beliefs. Enhancing our understanding of the relationship of causal models held by patients and caregivers from this Latin-American context is therefore important for improving engagement with service providers and ultimately better psychiatric care (US Department of Health & Human Services (USDHHS), 2001).

Thus, the aim of this study was to determine whether biological, psychosocial and magical-religious causal beliefs in patients and caregivers were associated with attitudes towards medication and severity of symptoms in patients with schizophrenia from three countries in Latin America (Bolivia, Chile and Peru). We also investigated the influence of Aymara ethnicity on clinical outcomes. The Aymara culture, with a population of 2 million people, has lived in the Andes Mountains for centuries. However, recent generations of Aymara have undertaken a massive migration, moving from rural towns in the Highland to large cities. This rapid abandonment of rural settlements is one of the most difficult recent experiences of the Aymara (Köster 1992; Van Kessel 1996; Gundermann 2000; Zapata 2007; Gundermann & González 2009). Rapid social change such as that experienced by the Aymara has been shown in other cultures to be related to the expression of psychiatric symptoms (Becker, et al., 2005).

We tested the following hypotheses: 1. biological causal beliefs would be associated with positive attitudes towards medication and a low level of symptoms, whereas magical-religious causal beliefs would be associated with negative attitudes towards medication and high-level symptoms; 2. psychosocial causal beliefs would be associated with low-level symptoms, but unrelated to attitudes about medication (this hypothesis is based on the fact that psychosocial beliefs tend to be more strongly associated with participation in community-based treatments that strive to integrate patients into society); 3. finally, as far as caregivers’ causal beliefs were concerned, we expected to find the same type of relationships as for patients’ causal beliefs.

2 METHOD

2.1 Study participants

The process of recruitment of Aymara and non-Aymara patients and caregivers took place in public health sector clinics in each country. We selected the largest public health clinic in each region. The first author reviewed the lists of patients and caregivers who were attending each center, and the research team made assessments over a three month period in each country. Patients were invited to participate as they came to their monthly follow-up visits, usually accompanied by their key caregiver. Most of the people agreed to participate.

The study sample was comprised of patients with schizophrenia who were receiving services from mental health clinics in the Central-Southern Andean regions of northern Chile, southern Peru, and central-western Bolivia, and each patient's primary caregiver, defined as the person who fulfills the primary caring role and spends more time than anyone else with the patient in the task of caring. Diagnoses were made by treating psychiatrists based on the criteria established by the International Classification of Diseases, 10th version (WHO, 1992). The sample included both Aymara and non-Aymara patients and caregivers. Aymara patients and caregivers were identified by Aymara surnames as established by legislation regarding indigenous peoples in the three countries, or Aymara self-identification. Both the Aymara and non-Aymara patients live in the same urban areas, are served by the same mental health centers, and have roughly comparable socio-demographic characteristics.

We applied a small set of exclusion criteria to the patient (being in a state of psychotic crisis or having a sensory or cognitive type of disorder that prevents being evaluated) and caregiver (presence of organic symptomatology; having a psychoactive substance abuse disorder; having a sensory or cognitive type of disorder that prevents interviewing) groups to ensure ability to participate fully in the interviews. These criteria were applied by the treating psychiatrist. The final sample included 253 patients with an ICD-10 diagnosis of schizophrenia and 253 primary caregivers (33.6% from Chile, 33.6% from Peru, and 32.8% from Bolivia). Interviews were conducted between May 2012 and February 2013.

2.2 Measures

Survey for Relatives and Patients Regarding Causes of Schizophrenia (Appendix 1)

The scale was developed in two phases: first, a qualitative phase that featured interviews with anthropologists and people of Aymara descent; and second, a quantitative phase of questions based on the Causal Models Questionnaire for Schizophrenia in China (Phillips et al., 2000) and on the Questionnaire for Beliefs Regarding Schizophrenia (Kurihara et al., 2006). This instrument was developed in Bali; Kurihara et al. conducted a 7-day test-retest study and reported a mean kappa (a measure of chance-corrected agreement) of .82 for the 15 items in the scale. That study also asked participants to nominate what they regarded as the most important cause, which had a kappa of .90.

The Survey for Relatives and Patients Regarding Causes of Schizophrenia, developed specifically for this study, has 16 items that are recorded on a Likert scale where: 0 = no cause; 1 = possible cause; and 2 = cause. The items of the survey are divided into three domains, with a score derived for each: 1) Biological cause, with 4 items, score ranges from 0 to 8 and includes brain disorder, heredity, physical disease, and fatigue; 2) Psychosocial causes has 7 items, score ranges from 0 to 14 and includes stress related to persons outside the family, personality problems, work stress, financial problems, stress related to family members, low educational level, and school stress; and 3) Magical-religious causes has 5 items, score ranges from 0 to 10 and includes punishment by God, fright sickness, grab-from-land, witchcraft, and black magic. This subscale was developed from key informant interviews (anthropologists, an intercultural Aymara consultant) as well as from the literature review. In the current sample, Cronbach's alphas, a measure of internal consistency reliability, were .80 for the magical-religious domain; .73 for psychosocial domain and .58 for the biological domain for the patients’ survey, and .81 for the magical-religious domain; .63 for psychosocial domain and .49 for the biological domain for the caregivers’ survey. Scores on the three domains of beliefs about the causes of schizophrenia were standardized to a mean of 0 and standard deviation of 1.

Drug Attitude Inventory (DAI-10) (Hogan et al., 1983) (Appendix 2)

This 10-item self-report scale was developed to assess attitude, experience and beliefs about antipsychotic drugs. The DAI-10 is considered to be a good predictor of adherence to treatment in schizophrenia (Hogan et al. 1983; Nielsen et al. 2012). Scores ranged from −10 (very poor attitude) to +10 (best possible attitude). This is a simple and easy-to-use self-report instrument with good psychometric properties that assesses a unique clinical dimension relevant to nonadherence ((Nielsen et al., 2012). DAI-10 scores that are analyzed here were obtained from patients.

Positive and Negative Syndrome scale for Schizophrenia (PANSS) (Kay et al., 1987)

This is a 30-item, 7-point (1–7) rating scale that is specifically developed to assess psychotic symptoms in individuals with schizophrenia and which comprises three different subscales: positive, negative and general psychopathology. The PANSS has been translated and validated in Spain by Peralta and Cuesta (1994) and also Fresán, et al., (2005) examined the psychometric properties of this instrument in Mexico.

Demographic and clinical data

for patients, we collected sex, age, ethnicity (Aymara and non-Aymara), educational level (≥ 12 years or < 12), religion (Catholic vs. other religions such as Christian, Evangelical, Jehovah witness and other) employment status, family income (measure of the total salary per month for all members of the family, expressed in US dollars), receipt of government assistance (if or if not the patients receives financial support given by the government monthly) and duration of the disorder in years. For caregivers, we collected sex, age, religion and employment status.

2.3 Procedures

The study was approved by the Ethics Committee of the University of Tarapacá and the National Health Service of Chile. Two psychologists, who were part of the research team and supervised by the principal researcher, conducted the evaluations of family members and caregivers under the auspices of the mental health services of each country. The length of time of the evaluation of the caregivers was between 15 and 20 minutes; the evaluation of the patients was more extensive, between 20 and 30 minutes.

Before the start of the survey, informed consent was requested and received from the relative and the patient. The objectives of the study were explained as well as the voluntary nature of participation. No compensation was offered for participating in the study.

The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008.

2.4 Statistical analysis

Associations between biological, psychosocial and magical-religious causal belief scores and attitudes towards medications (DAI-10) and severity of symptoms (PANSS dimensions including positive, negative and psychopathology general scores and PANSS total score) were analyzed using Pearson's correlation coefficients. Associations between patients’ and caregivers’ causal beliefs were also explored using Pearson's correlation coefficients.

Multiple linear regressions were then performed to assess the association between biological, psychosocial and magical-religious causal belief scores and attitude towards medication and severity of symptoms, after adjustment for socio-demographic confounding factors. The DAI-10 score, the PANSS total score and each of its three dimensions were considered as separate dependent variables. In addition to patients’ and caregivers’ causal beliefs, the following variables were also included: age, gender, ethnicity, education level and annual family income. All continuous variables were standardized to a mean of 0 and standard deviation of 1; regression coefficients for these variables therefore indicate the expected differences (in standard deviation units) in the dependent variable associated 1-standard deviation changes in each predictor. The statistical analyses were performed using the SPSS version 18.0.

3 RESULTS

3.1 Sample characteristics

Two hundred and fifty-three patients with an International Classification of Diseases (ICD)-10 diagnosis of schizophrenia (World Health Organization (WHO), 1992) and their caregivers were enrolled in the present study. All the details are presented in Table 1. The mean age of patients was 35.6 years and 66.4% were men. The mean duration of illness was 14.6 years (±11.8 years). The patients show a moderate severity of symptoms with a total PANSS score of 71.3 (±28.2). In relation to ethnicity, 46.2% of the patients are Aymara.

Table 1.

Sample characteristics (N=253)

Patients M ± SD
Gender (men): N (%) 164 (66.4)
Age in years 35.6 ± 15.5
Aymara: N (%) Yes 117 (46.2)
No 136 (53.8)
Religion (%) Catholic 56.7
Other religions* 43,3
Educational level (≥ 12 years): N (%) 40 (15.8)
Monthly family income (US dollars)** 417.6 ± 429.4
Monthly financial support of government (%) Yes 26.2
No 73.8
Employment status (%) With employment 30.4
Without employment 69.6
Duration of disorder in years 14.6 ± 11.8
Attitudes towards medications– DAI-10 3.0 ± 4.8
Severity – PANSS Positive factor 15.5 ± 7.5
Negative factor 21.0 ± 9.9
Psychopathology general factor 35.3 ± 14.5
PANSS Total 71.3 ± 28.2
Causal belief Biological 1.5 ± 1.7
Psychosocial 2.7 ± 2.8
Magical-religious 1.4 ± 2.2

Caregivers

Gender (women): N (%) 170 (67)
Age in years 54.7 ± 14.4
Religion Catholic 64.2
Other religions* 35.8
Employment status (%) With employment 56.9
Without employment 43.1
Causal belief Biological 1.7 ± 1.7
Psychosocial 2.9 ± 2.6
Magical-religious 1.2 ± 1.9

M ± SD: mean ± standard deviation; N (%): effective (percentage).

Positive and Negative Syndrome scale for Schizophrenia (PANSS) severity: PANSS total score of 58 = “Mildly ill”; PANSS total score of 75 = “Moderately ill”; PANSS total score of 95 = “Markedly ill” and PANSS total score of 116 = “Severely ill”.

Drug Attitude Inventory (DAI-10): There are no cutoffs, the higher the score, the better the attitude towards medication.

*

Other religions: Christian, Evangelical, Jehovah Witness and other.

**

Monthly family income considered the economic contribution of all members of the family in this sample is low compare to the national average per person in each country.

Patients’ beliefs were higher for psychosocial than for magical-religious and biological causes. The majority of caregivers were women (67%) and their mean age was 54.7 years. As for patients, caregivers’ beliefs were higher for psychosocial than for magical-religious and biological causes.

3.2 Relationships between causal beliefs and attitudes towards medication and severity of symptoms

The correlations between causal belief scores and DAI-10 and PANSS scores are presented in Table 2. Patients with higher magical-religious causal beliefs had lower DAI-10 scores (r=−0.18). Patients with higher scores on the measures of biological and psychosocial causal beliefs had less severe symptoms according to the PANSS positive and negative factors (r=−0.21 and r=−0.19). In addition, caregivers’ scores on the scale of magical-religious beliefs were associated with patients’ DAI-10 scores (r=−0.13) and PANSS scores (r=0.22 and r=0.18 for the PANSS general psychopathology factor and total score). The correlation between the DAI-10 and PANSS was low but significant (r=0.14; p=.03).

Table 2.

Associations of causal belief and attitudes towards medications (DAI-10) and severity of symptoms (PANSS)

Pearson's correlation coefficients DAI-10 PANSS positive factor PANSS negative factor PANSS psychopathology general factor PANSS total
Patients
Biological causal belief 0.01 −0.18** −0.21** −0.04 −0.12
Psychosocial causal belief 0.04 −0.15* −0.19** 0.01 −0.09
Magical-religious causal belief −0.18** 0.07 −0.01 0.10 0.08

Caregivers
Biological causal belief −0.07 −0.05 −0.09 0.08 0.01
Psychosocial causal belief −0.03 0.01 −0.04 0.10 0.05
Magical-religious causal belief −0.13* 0.10 0.07 0.22** 0.18**

Statistically significant in bold:

*

p<0.05

**

p<0.01

Positive and Negative Syndrome scale for Schizophrenia (PANSS)

Drug Attitude Inventory (DAI-10)

Among patients, the correlations between biological, psychosocial, and magical-religious beliefs were positive and statistically significant (r, biological and psychosocial beliefs=0.61; r, biological and magical-religious beliefs=0.46; r, psychosocial and magical-religious beliefs=0.39). In addition, the correlations between patients’ and caregivers’ causal beliefs were all statistically significant and positive from 0.18 to 0.45. The highest correlations were between patients’ biological beliefs and caregivers’ biological beliefs. The lowest correlations were between patients’ magical-religious beliefs and caregivers’ psychosocial beliefs (Table 3).

Table 3.

Associations between patients’ and caregivers’ causal beliefs

Caregivers

Pearson's correlation coefficients Biological causal belief Psychological causal belief Magical-religious causal belief
Patients
Biological causal belief 0.45** 0.34** 0.29**
Psychological causal belief 0.25** 0.39** 0.30**
Magical-religious causal belief 0.26** 0.18** 0.30**

Statistically significant in bold:

*: p<0.05

**

p<0.01

The results of linear regression analyses of DAI-10 and PANSS scores are presented in Table 4. After adjustment for socio-demographic and clinical factors, patients with higher magical-religious causal beliefs had lower DAI-10 scores (β=−0.27; 95% confidence interval:−0.43, −0.10) and more severe symptoms according to the PANSS positive factor (β=0.17; 0.02, 0.33). Patients with higher biological causal beliefs had less severe symptoms according to the three PANSS factors (PANSS positive factor: β=−0.26 (−0.44,−0.07); PANSS negative factor: β=−0.27 (−0.47,−0.08); PANSS psychopathology factor: β= −0.26 (−0.44;−0.08)) and the PANSS total score with a β=−0.26 (−0.45,−0.07). We also found a significant association between higher psychosocial causal beliefs and less severe symptoms according to the PANSS positive factor with a β=−0.19 (−0.37,−0.01). The caregivers’ causal beliefs did not show associations with the DAI-10 or PANSS scores.

Table 4.

Factors associated with DAI-10, PANSS total and dimensions scores: multivariate analyses

Regression coefficient (95% CI) DAI-10 PANSS total PANSS Positive factor PANSS Negative factor PANSS Psychopathology general factor
Age# 0.25 (0.11;0.39) −0.04 (−0.18;0.10) −0.12 (−0.26;0.01) 0.00 (−0.14;0.14) −0.05 (−0.18;0.09)
Gender* Women −1.19 (−2.66;0.28) −2.52 (−10.85;5.81) −0.97 (−3.12;1.19) −1.84 (−4.82;1.14) −0.59 (−4.67;3.51)
Men (Ref)
Aymara* Yes −0.09 (−1.43;1.26) 0.92 (−6.80;8.63) 0.19 (−1.82;2.19) 0.51 (−2.22;3.25) 1.09 (−2.68;4.86)
No (ref)
Education* ≥ 12 years 0.72 (−1.15;2.58) −5.38 (−15.87;5.09) 0.86 (−1.85;3.57) −5.86 (−9.58;−2.13) −1.60 (−6.81;3.62)
<12 years (Ref)
Income# 0.09 (−0.05;0.23) −0.17 (−0.30;−0.04) −0.13 (−0.26;0.01) −0.17 (−0.31;−0.03) −0.17 (−0.30;−0.04)
Patients’ causal belief#
    Biological 0.14 (−0.06;0.33) −0.26 (−0.45;−0.07) −0.26 (−0.44;−0.07) −0.27 (−0.47;−0.08) −0.26 (−0.44;−0.08)
    Psychosocial 0.11 (−0.08;0.30) −0.14 (−0.32;0.04) −0.19 (−0.37;−0.01) −0.13 (−0.32;0.05) −0.05 (−0.22;0.12)
    Magical-religious −0.27 (−0.43;−0.10) 0.13 (−0.03;0.29) 0.17 (0.02;0.33) 0.11 (−0.05;0.27) 0.10 (−0.06;0.26)
Caregivers' causal belief#
    Biological −0.12 (−0.29;0.05) 0.06 (−0.11;0.22) −0.05 (−0.21;0.11) 0.07 (−0.09;0.24) 0.12 (−0.04;0.28)
    Psychosocial −0.01 (−0.18;0.16) 0.14 (−0.03;0.30) 0.12 (−0.04;0.28) 0.08 (−0.09;0.24) 0.13 (−0.03;0.28)
    Magical-religious −0.11 (−0.27;0.06) 0.14 (−0.02;0.30) 0.05 (−0.09;0.20) 0.08 (−0.05;0.26) 0.15 (−0.00;0.30)
Model R2 0.09 0.15 0.15 0.17 0.15
P 0.030 <10−3 <10−3 <10−3 <10−3
*

Unstandardized coefficients

#

β standardised beta coefficient (β represents the change of the standard deviation in DAI-10 and PANSS score resulting from a change of 1 standard deviation in the independent variable); Statistically significant in bold.

CI: Confidence interval; Ref: Reference group; Positive and Negative Syndrome scale for Schizophrenia (PANSS); Drug Attitude Inventory (DAI-10).

In addition, we found that several socio-demographic and clinical characteristics were associated with adherence and severity of symptoms. Older individuals had positive attitudes towards medication with standardized beta coefficient of 0.25 (0.11, 0.39). High education level was associated with fewer negative symptoms (β=−5.86; CI=−9.58,−2.13), and family income was associated with less severe symptoms (β for PANSS psychopathology factor=−0.17 (−0.30,−0.04); PANSS negative factor: β=−0.17 (−0.31,−0.03) and PANSS total: β=−0.17 (−0.30,−0.04). Finally, there were no differences between Aymara and Non-Aymara participants in clinical outcomes.

4 DISCUSSION

The present study sought to determine whether causal beliefs in patients and caregivers can be considered as potential treatment targets in patients with schizophrenia with longstanding illness in Latin American countries. This study explores this issue for the first time in Latin American countries, and provides several interesting results regarding the association between beliefs about the disorder and patients’ clinical outcomes that could have implications for enhancing psychotherapeutic treatments for schizophrenia.

The first important finding is that patients’ causal beliefs were associated with attitudes towards medications and severity of symptoms. However, these associations were moderate in strength and should be interpreted with caution. In line with our hypotheses, patients with higher biological causal beliefs had less severe symptoms according to the different PANSS dimensions and the PANSS total score, whereas patients with higher magical-religious causal beliefs had lower DAI-10 scores and more severe symptoms. These findings are consistent with results from previous studies on magical-religious causal beliefs (Kurihara et al., 2006; Saravanan et al., 2007). Also in this study the small but significant correlation between attitudes towards medications and scores on the PANSS illustrates the linkage between pharmacological treatment the level of the symptoms.

Our results are also consistent with the results of three recent studies on the importance of psychosocial causal beliefs and their link with better clinical outcomes (Sanders et al. 2011; Lüllmann et al. 2011; Freeman et al. 2012). Our study suggests that integrating psychosocial and biological aspects within a causal model in psychosocial intervention programs may be more efficacious than focusing only on the biological model, and basing treatment only on prescribing of antipsychotic drugs. However, two points in our results differ substantially from our hypotheses and findings from previous studies. We did not find any link between biological causal beliefs and attitudes towards medication, suggesting a balancing effect between two opposite trends: on the one hand, biological causal beliefs have been reported to be associated with patients’ preference for pharmacological treatment (Phelan et al., 2006; Schweizer et al., 2010). On the other hand, studies also have reported that patients with causal biological beliefs had a negative outlook on the disorder as being uncontrollable and untreatable, resulting in less motivation to engage in treatment (Lüllmann et al., 2011). This hypothesis should be further explored. The second novel finding concerned the absence of direct relationships between caregivers’ causal beliefs and clinical outcomes. Our study explored the direct relationships between patients' and caregivers' beliefs about the causes of schizophrenia and clinical outcomes. Future studies should explore the reciprocal relations between patients' and caregivers' beliefs, and their impact on clinical outcomes. This may be done, for example, by using techniques such as structural equation modeling (Boyer, et al., 2012) to directly estimate the simultaneous and synergic impact of patients' and caregivers' beliefs on clinical outcomes (Falissard, 2005)

Our findings also suggest a relationship between poorer socioeconomic status and worse clinical outcomes. Lower educational level and family income were associated with more severe psychotic symptoms, reinforcing prior findings that mental illness and poverty are closely linked (Wilton, 2004; Krieger et al., 2007; Marmot et al., 2008). Of note, we have shown in a previous work the absence of link between causal beliefs and poor socioeconomic status (Caqueo-Urízar et al., 2015). Two implications arise from these results. First, educational level and family income could be considered as risk factors by clinicians and greater attention to patients and caregivers of poor socioeconomic status is needed. Strengthening the ability to address the barriers to remission associated with poverty in Latin American countries is thus necessary to optimize treatment outcomes especially to vulnerable individuals (Boyer et al., 2014).

The second implication may be related to economic policies in Latin American countries. Recent studies have suggested that the use of first-generation antipsychotics and psychosocial interventions (i.e., family and psycho-educational) in schizophrenia could represent the most cost-effective interventions in developing regions and countries (Chisholm et al., 2008). However, people with mental disorders and their caregivers may also benefit from programs to reduce the level and chronicity of poverty (Cunningham et al., 2006). Indeed, growing international evidence shows that mental illness and poverty interact in a negative cycle: “poverty breeds ill-health, and ill-health keeps poor people poor” (Wagstaff, 2002; Lund et al., 2011). Poverty worsens the health of patients with schizophrenia, and increases the burden of caregivers, which in turn may affect the caregivers’ health and their ability to care for the patients (Karanci, 1995; Butzlaff et al., 1998; Caqueo-Urízar and Gutiérrez, 2006). In our study, a low percentage of patients were employed (30.4%) and only 26.2% of them received financial support from the government. It should also be noted that the monthly government assistance was small (145 USD), whereas the minimum wage in Chile is 408 USD (Ministerio de Economía, Fomento y Turismo de Chile 2013). In addition, caregivers have a similar situation, with only 56.9% employed and it is likely that most of them have stopped working to care for the patients, without assistance from the government. All these findings suggest the need for support from policymakers at the national and local levels for economically vulnerable populations.

It should be noted that caregivers and patients resided in urban areas. Future studies should consider the assessment of caregivers and patients who reside in the Andean highlands and who may causal beliefs not represented in urban populations. This study is limited by the fact that it used a cross-sectional rather than prospective in design, so the associations observed may be due to the effects of symptoms on beliefs or symptoms on economic status. No causal inference can be formally advanced, and our findings should be interpreted from an associational point of view.

The association between patients’ magical-religious beliefs and the PANSS positive factor was statistically significant only after adjustment for caregivers’ causal beliefs and socio-demographic factors. This suggests a complex set of relationships amongst patients’ and caregivers’ causal beliefs as well as demographic factors in the determination of clinical outcomes. Studies that could better understand these relationships are needed, particularly those that incorporate prospective and experimental designs.

In our study, we explored the association between patients' causal beliefs and health outcomes. However, we did not explore the influence of i) providers' causal model and ii) the congruence between the patients' and the providers' causal model on health outcomes. Greater congruence might be related to greater adherence to treatment and in turn less symptomatology. These issues should be explored in future studies. In addition, we hypothesized that biological models are the predominant model adhered to by practitioners, but this assumption should also be studied and confirmed in the future.

Our findings suggest that it could be beneficial for clinicians to address patients' causal beliefs as part of their treatment, and that poverty-based interventions may also needed, considering both the existing evidence base and also the economic reality of Latin America (Cunningham et al., 2006; Saravanan et al., 2007; Caldas de Almeida et al., 2010; Gallo and Thoen, 2010; Lora et al., 2012; Wiesjahn et al., 2014). However, our findings need to be replicated by prospective studies that can address the temporality of associations (i.e., primarily the possibility the observed associations are due to the effects of the disorder on both beliefs and poverty).

Finally, it is important to address the ethical considerations of incorporating patients’ or caregivers’ belief systems as part of treatment approaches. Should treatment providers explain to patients who hold magical and religious beliefs that are informed by their ethnic background that those beliefs are “wrong” and that biomedical models of illness are “right”?. Our finding that the three domains of beliefs were positively correlated with one another adds to the potential complexity of incorporating this study's findings into clinical practice, as it suggests that patients who hold strong beliefs in the biological or psychosocial domains also are likely to hold strong magical-religious beliefs. Our findings about magical-religious beliefs about health are thus complex and have as-yet uncertain relevance for clinical practice. Future studies should examine the link between magical-religious beliefs about health and religious involvement. Contrary to our findings, religious involvement has shown positive effects in reducing symptoms, improving coping and even recovery in patients with schizophrenia (Mohr et al., 2011). The knowledge of cultural background, including a patient's' religious involvement, has a positive impact on the therapeutic relationship between patients and health professionals (Conrad, 2007).

5. CONCLUSION

Patients’ causal beliefs were associated with attitudes towards medication and the severity of symptoms, suggesting that psychosocial programs may need to consider patients’ causal beliefs. Further studies are needed to evaluate whether these causal beliefs can be incorporated into treatment approaches for schizophrenia in Latin American countries.

Supplementary Material

1
02

ACKNOWLEDGEMENTS

This research was funded in part by CONICYT and BECAS-CHILE Postdoctorado en el Extranjero (74140004), the Convenio de Desempeño UTA-MINEDUC, and by the Intramural Research Program of the Eunice Kennedy Shriver National Institute of Child Health and Human Development.

We thank Felipe Ponce y Jorge Escudero for their assistance in the study and also the following people and facilities: Dr. Hugo Sánchez, Dr. Ricardo Alvites, Dr. Andrés Collado, Gladys Coaquira, Vilma Liendo and specially thanks to Dr. José Revilla from Hipólito Unanue Hospital in Tacna, Perú; Dr. Marcio Soto from Arequipa, Perú; Dr. Fernando Garitano, Dr. Mauricio Peredo and Dra. Mabel Romero from Centro de Rehabilitación y Salud Mental San Juan de Dios; and the Director of Hospital Psiquiátrico de la Caja Nacional de Salud from La Paz, Bolivia.

We also thank Dra. Magdalena Gardilcic, Ester López, and Alejandra Lagos from Servicio de Salud de Arica, Chile, and extend a special thanks to all the patients and caregivers who participated in this study.

Footnotes

CONTRIBUTORS

Conception and design: A. Caqueo-Urízar, and S. Gilman. Data collection and analysis of data: A. Caqueo-Urízar, L. Boyer and K. Baumstarck. Interpretation of data: S. Gilman, L. Boyer and K. Baumstarck. Drafting and writing the manuscript: A. Caqueo-Urízar, S. Gilman and L. Boyer.

CONFLICT OF INTEREST

The Authors have declared that there are no conflicts of interest in relation to the subject of this study.

Contributor Information

Laurent Boyer, Email: Laurent.BOYER@ap-hm.fr.

Karine Baumstarck, Email: karine.baumstarck@univ-amu.fr.

Stephen E. Gilman, Email: stephen.gilman@nih.gov.

REFERENCES

  1. Al-Krenawi A. Explanations of mental health symptoms by the Bedouin-Arabs of the Negev. International Journal of Social Psychiatry. 1999;45:56–64. doi: 10.1177/002076409904500107. [DOI] [PubMed] [Google Scholar]
  2. Angermeyer M, Matschinger H. Relatives’ beliefs about the cause of schizophrenia. Acta Psychiatrica Scandinavica. 1996;93:199–204. doi: 10.1111/j.1600-0447.1996.tb10632.x. [DOI] [PubMed] [Google Scholar]
  3. Balhara YP, Yadav T. A comparative study of beliefs, attitudes and behaviour of psychiatric patients and their care givers with regards to magico-religious and supernatural influences. Journal of Medical Sciences. 2012;12:10–17. [Google Scholar]
  4. Becker AE, Gilman SE, Burwell RA. Changes in prevalence of overweight and in body image among Fijian women between 1989 and 1998. Obesity Research. 2005;13(1):110–117. doi: 10.1038/oby.2005.14. [DOI] [PubMed] [Google Scholar]
  5. Borras L, Mohr S, Brandt P-Y, Gilliéron C, Eytan A, Huguelet P. Religious beliefs in schizophrenia: their relevance for adherence to treatment. Schizophrenia Bulletin. 2007;33(5):1238–1246. doi: 10.1093/schbul/sbl070. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Boyer L, Baumstarck K, Iordanova T, Fernández J, Jean P, Auquier P. A poverty-related quality of life questionnaire can help to detect health inequalities in emergency departments. Journal of Clinical Epidemiology. 2014;67(3):285–95. doi: 10.1016/j.jclinepi.2013.07.021. [DOI] [PubMed] [Google Scholar]
  7. Boyer L, Cermolacce M, Dassa D, Fernandez J, Boucekine M, Richieri R, Vaillant F, Dumas R, Auquier P, Lancon C. Neurocognition, insight and medication nonadherence in schizophrenia: a structural equation modeling approach. PLoS One. 2012;7(10):e47655. doi: 10.1371/journal.pone.0047655. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Brewin CR. Changes in attribution and expressed emotion among the relatives of patients with schizophrenia. Psychological Medicine. 1994;24:905–911. doi: 10.1017/s0033291700028993. [DOI] [PubMed] [Google Scholar]
  9. Butzlaff RL, Hooley JM. Expressed emotion and psychiatric relapse: a meta-analysis. Archives of General Psychiatry. 1998;55:547–551. doi: 10.1001/archpsyc.55.6.547. [DOI] [PubMed] [Google Scholar]
  10. Caldas de Almeida JM, Horvitz-Lennon M. Mental health care reforms in Latin America: An overview of mental health care reforms in Latin America and the Caribbean. Psychiatric Services. 2010;61(3):218–221. doi: 10.1176/ps.2010.61.3.218. [DOI] [PubMed] [Google Scholar]
  11. Caqueo-Urízar A, Breslau J, Gilman SE. Beliefs about the causes of schizophrenia among Aymara and non-Aymara patients and their primary caregivers in the Central-Southern Andes. Int J Soc Psychiatry. 2015;61(1):82–91. doi: 10.1177/0020764014544768. [DOI] [PubMed] [Google Scholar]
  12. Caqueo-Urízar A, Gutiérrez J. Burden in families of patients with schizophrenia. Quality of Life Research. 2006;15(4):719–724. doi: 10.1007/s11136-005-4629-2. [DOI] [PubMed] [Google Scholar]
  13. Chisholm D, Gureje O, Saldivia S, Villalón Calderón M, Wickremasinghe R, Mendis N, Ayuso-Mateos JL, Saxena S. Schizophrenia treatment in the developing world: an interregional and multinational cost-effectiveness analysis. Bulletin of World Health Organization. 2008;86(7):542–51. doi: 10.2471/BLT.07.045377. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Conrad R, Schilling G, Najjar D, Geiser F, Sharif M, Liedtke R. Cross-cultural comparison of explanatory models of illness in schizophrenic patients in Jordan and Germany. Psychological Reports. 2007;101(2):531–46. doi: 10.2466/pr0.101.2.531-546. [DOI] [PubMed] [Google Scholar]
  15. Cunningham P, McKenzie K, Taylor EF. The struggle to provide community-based care to low-income people with serious mental illnesses. Health Affairs. 2006;25(3):694–705. doi: 10.1377/hlthaff.25.3.694. [DOI] [PubMed] [Google Scholar]
  16. Falissard B. Modèles structuraux (Structural models). In: Falissard B, editor. Comprendre et utiliser les statistiques dans les sciences de la vie (Understanding and using statistics in life-related sciences) Masson; Paris: 2005. pp. 203–215. [Google Scholar]
  17. Freeman D, Dunn G, Garety P, Weinman J, Kuipers E, Fowler D, Jolley S, Bebbington P. Patients’ beliefs about the causes, persistence and control of psychotic experiences predict take-up of effective cognitive behaviour therapy for psychosis. Psychological Medicine. 2012;43:269–277. doi: 10.1017/S0033291712001225. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Fresán A, De la Fuente-Sandoval C, Loyzaga C, García-Anaya M, Meyenberg N, Nicolini H, Apiquian R. A forced five-dimensional factor analysis and concurrent validity of the Positive and Negative Syndrome Scale in Mexican schizophrenic patients. Schizophrenia Research. 2005;72:123–129. doi: 10.1016/j.schres.2004.03.021. [DOI] [PubMed] [Google Scholar]
  19. Gallo C, Tohen M. Current status and future perspectives for psychiatry/mental health research in Latin America. International Review of Psychiatry. 2010;22(4):382–393. doi: 10.3109/09540261.2010.500875. [DOI] [PubMed] [Google Scholar]
  20. Gundermann H. Las organizaciones étnicas y el discurso de la identidad en el norte de Chile, 1980–2000 [Ethnic organizations and the discourse of identity in the North of Chile, 1980-2000]. Estudios Atacameños. 2000;19:75–91. [Google Scholar]
  21. Gundermann H, González H, Vergara J. Vigencia y desplazamiento de la lengua aymara en Chile [Force and displacement of the aymara's language in Chile]. Estudio filológico. 2007;42:123–140. [Google Scholar]
  22. Hogan TP, Awad AG, Eastwood R. A self-report scale predictive of drug compliance in schizophrenics: reliability and discriminative validity. Psychological Medicine. 1983;13:177–183. doi: 10.1017/s0033291700050182. [DOI] [PubMed] [Google Scholar]
  23. Holzinger A, Kilian R, Lindebach I, Petscheleit A, Angermeyer M. Patients’ and caregivers’ causal explanation of schizophrenia. Social Psychiatry and Psychiatric Epidemiology. 2003;38:155–162. doi: 10.1007/s00127-003-0624-5. [DOI] [PubMed] [Google Scholar]
  24. Ilechukwu ST. Inter-relationships of beliefs about mental illness, psychiatric diagnoses and mental health care delivery among Africans. International Journal of Social Psychiatry. 1988;34:200–206. doi: 10.1177/002076408803400305. [DOI] [PubMed] [Google Scholar]
  25. Karanci AN. Caregivers of Turkish schizophrenic patients: causal attributions, burdens and attitudes to help from the health professional. Social Psychiatry and Psychiatric Epidemiology. 1995;30:261–268. doi: 10.1007/BF00805792. [DOI] [PubMed] [Google Scholar]
  26. Kate N, Grover S, Kulhara P, Nehra R. Supernatural beliefs, aetiological models and help seeking behaviour in patients with schizophrenia. Industrial Psychiatry Journal. 2012;21(1):49–54. doi: 10.4103/0972-6748.110951. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Kay SR, Fiszbein A, Opler L. The positive and negative syndrome scale (PANSS) for schizophrenia. Schizophrenia Bulletin. 1987;13:261–76. doi: 10.1093/schbul/13.2.261. [DOI] [PubMed] [Google Scholar]
  28. Kern RS, Glynn SM, Horan WP, Marder SR. Psychosocial treatments to promote functional recovery in schizophrenia. Schizophrenia Bulletin. 2009;35(2):347–61. doi: 10.1093/schbul/sbn177. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Köster G. Los Aymaras: Características demográficas de un grupo étnico indígena antiguo en los Andes centrales [The Aymara: Demographic characteristics of an ancient indigenous ethnic group in the Central Andes]. In: Van den Berg H, Schiffers N, editors. La cosmovisión Aymara. UCB/Hisbol; La Paz, Bolivia: 1992. pp. 81–111. [Google Scholar]
  30. Krieger N. Why epidemiologists cannot afford to ignore poverty. Epidemiology. 2007;18:658–663. doi: 10.1097/EDE.0b013e318156bfcd. [DOI] [PubMed] [Google Scholar]
  31. Kurihara T, Kato M, Reverger R, et al. Beliefs About Causes of Schizophrenia Among Family Members: A Community-Based Survey in Bali. Psychiatric Services. 2006;57(12):1795–9. doi: 10.1176/ps.2006.57.12.1795. [DOI] [PubMed] [Google Scholar]
  32. Lobban F, Barrowclough C. Common sense representations of schizophrenia in patients and their relatives. Clinical Pharmacological and Therapeutics. 2005;12(2):134–141. [Google Scholar]
  33. Lora A, Kohn R, Levav I, McBain R, Morris J, Saxena S. Service availability and utilization and treatment gap for schizophrenic disorders: a survey in 50 low- and middle-income countries. Bulletin of World Health Organization. 2012;90(1):47–54B. doi: 10.2471/BLT.11.089284. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Lüllmann E, Berendes S, Rief W, Lincoln TM. Benefits and harms of providing biological causal models in the treatment of psychosis–An experimental study. Journal of Behavior Therapy and Experimental Psychiatry. 2011;42:447–453. doi: 10.1016/j.jbtep.2011.03.003. [DOI] [PubMed] [Google Scholar]
  35. Lund C, De Silva M, Plagerson S, Cooper S, Chisholm D, Das J, Knapp M, Patel V. Poverty and mental disorders: breaking the cycle in low-income and middle-income countries. Lancet. 2011;378:1502–14. doi: 10.1016/S0140-6736(11)60754-X. [DOI] [PubMed] [Google Scholar]
  36. Marmot M, Friel S, Bell R, Houweling TA, Taylor S. Closing the gap in a generation: health equity through action on the social determinants of health. Lancet. 2008;372:1661–9. doi: 10.1016/S0140-6736(08)61690-6. [DOI] [PubMed] [Google Scholar]
  37. Ministerio de Economía, Fomento y Turismo de Chile [3 March 2013];2013 [ http://www.economia.gob.cl/].
  38. Mohr S, Perroud N, Gillieron C, Brandt PY, Rieben I, Borras L, Huguelet P. Spirituality and religiousness as predictive factors of outcome in schizophrenia and schizo-affective disorders. Psychiatry Research. 2011;186(2-3):177–82. doi: 10.1016/j.psychres.2010.08.012. [DOI] [PubMed] [Google Scholar]
  39. Nielsen RE, Lindström E, Nielsen J, Levander S. DAI-10 is as good as DAI-30 in schizophrenia. European Neuropsychopharmacology. 2012;22(10):747–50. doi: 10.1016/j.euroneuro.2012.02.008. [DOI] [PubMed] [Google Scholar]
  40. Peralta V, Cuesta MJ. Validación de la Escala de los Síndromes Positivo y Negativo (PANSS) en una muestra de esquizofrénicos españoles. [Validation of the positive and negative syndrome scale (PANSS) in a sample of Spanish schizophrenic]. Actas Luso-Española de Neurología, Psiquiatría y Ciencias afines. 1994;22(4):171–7. [PubMed] [Google Scholar]
  41. Phelan JC, Yang LH, Cruz-Rojas R. Effects of attributing serious mental illnesses to genetic causes on orientations to treatment. Psychiatric Services. 2006;57(3):382–387. doi: 10.1176/appi.ps.57.3.382. [DOI] [PubMed] [Google Scholar]
  42. Phillips M, Li Y, Stroup S, et al. Causes of schizophrenia reported by patients’ family members in China. British Journal of Psychology. 2000;177:20–22. doi: 10.1192/bjp.177.1.20. [DOI] [PubMed] [Google Scholar]
  43. Pollock K. On the nature of social stress: production of a modern mythology. Social Science and Medicine. 1988;26:381–392. doi: 10.1016/0277-9536(88)90404-2. [DOI] [PubMed] [Google Scholar]
  44. Sanders D, Kydd R, Morunga E, Broadbent E. Differences in patient's perceptions of Schizophrena between Maori and New Zealand Europeans. Australian & New Zealand Journal of Psychiatry. 2011;45:483–488. doi: 10.3109/00048674.2011.561479. [DOI] [PubMed] [Google Scholar]
  45. Saravanan B, Jacob KS, Johnson S, Prince M, Bhugra D, David A. Belief models in first episode schizophrenia in South India. Social Psychiatry and Psychiatric Epidemiology. 2007;42:446–451. doi: 10.1007/s00127-007-0186-z. [DOI] [PubMed] [Google Scholar]
  46. Schweizer S, Peeters F, Huibers M, Roelofs J, van Os J, Arntz A. Does illness attribution affect treatment assignment in depression? Clinical Psychology & Psychotherapy. 2010;17(5):418–426. doi: 10.1002/cpp.662. [DOI] [PubMed] [Google Scholar]
  47. Sibitz I, Katschnig H, Goessler R, Unger A, Amering M. Pharmacophilia and pharmacophobia: determinants of patients’ attitudes towards antipsychotic medication. Pharmacopsychiatry. 2005;38:107–12. doi: 10.1055/s-2005-864119. [DOI] [PubMed] [Google Scholar]
  48. Van Kessel J. La cosmovisión Aymara. In: Hidalgo J, Schiappacasse F, Niemeyer F, Aldunate C, Mege P, editors. Etnografía: Sociedades indígenas contemporáneas y su ideología [Ethnography: Contemporary indigenous societies and their ideology] Editorial Andrés Bello; Santiago, Chile: 1996. pp. 169–187. [Google Scholar]
  49. Wagstaff A. Poverty and health sector inequalities. Bulletin of World Health Organization. 2002;80(2):97–105. [PMC free article] [PubMed] [Google Scholar]
  50. Wiesjahn M, Jung E, Lamster F, Rief W, Lincoln T. Explaining Attitudes and Adherence to Antipsychotic Medication: The Development of a Process Model. Schizophrenia Research and Treatment. 2014;2014:1–11. doi: 10.1155/2014/341545. [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Wilton R. Putting policy into practice? Poverty and people with serious mental illness. Social Science and Medicine. 2004;58(1):25–39. doi: 10.1016/s0277-9536(03)00148-5. [DOI] [PubMed] [Google Scholar]
  52. World Health Organisation . ICD-10 Classifications of Mental and Behavioural Disorder: Clinical Descriptions and Diagnostic Guidelines. World Health Organization; Geneva: 1992. [Google Scholar]
  53. Zafar S, Syed R, Tehseen S, Gowan S, Waqar S, Zubair A, Yousaf W, Zubairi A, Naqvi H. Perception about the cause of schizophrenia and the subsequent help seeking behavior in Pakistani population –results of a cross-sectional survey. BMC Psychiatry. 2008;8:56. doi: 10.1186/1471-244X-8-56. [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. Zapata C. Memoria e historia: El proyecto de una identidad colectiva entre los aymaras de Chile [Memory and history: the project of a collective identity among the Aymara of Chile]. Chungara. 2007;39:171–183. [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

1
02

RESOURCES