Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2014 Sep 18.
Published in final edited form as: J Nerv Ment Dis. 2013 Oct;201(10):833–840. doi: 10.1097/NMD.0b013e3182a5bf1d

Perceptions of Efficacy, Expressed Emotion, and the Course of Schizophrenia

The Case of Emotional Overinvolvement

Nicholas JK Breitborde *, Steven R López †, Adrian Aguilera ‡, Alex Kopelowicz §
PMCID: PMC4167397  NIHMSID: NIHMS627325  PMID: 24080669

Abstract

Although it is clear that expressed emotion (EE) is associated with the course of schizophrenia, proposed models for this association have struggled to account for the relationship between the EE index of emotional overinvolvement (EOI) and relapse. To expand our understanding of the EOI-relapse association, we first attempted to replicate the finding that the EOI-relapse association is curvilinear among 55 Mexican-Americans with schizophrenia and their caregiving relatives. Second, we evaluated whether the caregivers’ perception of their ill relative’s efficacy may account for the EOI-relapse association. Our results comport with past findings with regard to the curvilinear nature of the EOI-relapse association among Mexican-Americans and suggest that EOI may only seem to be a risk factor of relapse because of its strong association with a true risk factor for relapse (i.e., caregivers’ perception of their ill relative’s efficacy).

Keywords: Schizophrenia, expressed emotion, emotional overinvolvement, Mexican-Americans

With 4 decades of confirmatory research, it is clear that the construct of expressed emotion (EE) is associated with the course of schizophrenia. More specifically, individuals with schizophrenia who are exposed to familial environments characterized by high levels of criticism, emotional overinvolvement (EOI), and/or hostility are more likely to experience a symptomatic relapse than individuals with schizophrenia who are exposed to familial environments in which the presence of these variables is low (Butzlaff and Hooley, 1998). The robustness of the EE-relapse association is further evidenced by replications of this association in studies across a variety of geographical locations and studies of individuals at different stages of illness (e.g., first-episode versus chronic schizophrenia [Butzlaff and Hooley, 1998]).

Recently, greater attention has been devoted toward clarifying the mechanisms underlying the EE-relapse association (Hooley, 2007; Hooley and Gotlib, 2000; Kavanagh, 1992; Kuipers et al., 2010). Unfortunately, such work has typically struggled to account for the association between the EE index of EOI and relapse (Singh et al., 2013). For example, drawing on Weiner’s (1995) attribution theory, the attribution model of EE suggests that caregivers who view their ill relative as responsible for the expression of past illness-related behavior are hypothesized to display more negative affective and behavioral reactions to their ill relative (i.e., more criticism and less supportive behavior). Exposure to these negative affective and behavioral reactions is then hypothesized to be a psychosocial stressor for the individual with schizophrenia, which, in turn, may activate the biological mechanisms that trigger the expression of psychotic symptoms (Hooley and Gotlib, 2000). However, studies have repeatedly found that the attributional stance of caregivers high in EOI is similar to that of caregivers low in EE, such that both view their ill relative as not responsible for his/her illness-related behavior (Barrowclough and Hooley, 2003). Ultimately, the inability of the various models of the EE-relapse association to account for EOI is particularly problematic given that EOI, and not criticism, seems to be the key factor underlying the EE-relapse association in certain populations (e.g., Mexican-Americans [Aguilera et al., 2010; Breitborde et al., 2007] and Japanese [Tanaka et al., 1995]).

There is growing evidence that the construct of human agency (i.e., the capacity to complete desired actions in the future) may be relevant in understanding the relationship between EOI and relapse in schizophrenia. For instance, research both within and outside the schizophrenia literature suggests that caregivers high in EOI tend to perceive their ill relative as less capable of completing tasks associated with the recovery process for his/her respective illness (Breitborde et al., 2005; Clarke et al., 1996; Coyne et al., 1988; Holmbeck et al., 2002; Kuijer et al., 2000). Theoretical investigations of the association between caregivers’ perceptions of their ill relative’s ability to complete recovery-oriented tasks and EOI have suggested that this association may stem from the perceived need to provide additional support/protection to a patient who is viewed as highly vulnerable to negative health outcomes (Anderson and Coyne, 1991; Thomasgard and Metz, 1997; Thomasgard et al., 1995).

In a recent study, Breitborde et al. (2009) attempted to clarify the nature of the interrelationships between relapse, caregivers’ perception of their ill relative’s agency, and EE, looking at both the EE indices of EOI and criticism. This study found that relatives high in EE—those high in EE due to both the index of criticism and EOI—perceived their ill relative as the agent in the behavioral expression of his/her psychiatric symptoms to a greater extent than did caregivers low in EE and that this attitude was associated with an increased risk for relapse of their ill relative. Moreover, after controlling for the caregivers’ perception of their ill relative’s agency, EE was no longer predictive of relapse in this sample. On the basis of these data, Breitborde et al. (2009) suggested that EE may not have a direct effect on the course of schizophrenia. Rather, because of its correlation with a true risk factor of relapse (i.e., caregivers’ perception of their ill relative’s agency), EE may seem to be a risk factor for relapse despite the lack of a direct effect of EE on the course of schizophrenia (i.e., EE may be a “proxy risk factor”). This type of relationship, which is shown in Figure 1, has been assigned several names within the psychological nomenclature, including the “illusory correlation” (Yule, 1922); “spurious correlation” (Simon, 1954); “third variable problem” (Kenny, 1975); and, more recently, proxy risk factor (Kraemer et al., 2001). In total, this hypothesized model comports with evidence outside the schizophrenia literature that suggests that exposure to negative perceptions with regard to one’s agency concerning the management/control of illness-related behavior is a psychosocial stressor (Coyne et al., 1988). Such stress-ors have been shown to produce the transient intermediate states (i.e., processing capacity overload [Schmader and Johns, 2003] and autonomic hyperarousal [Blascovich et al., 2001]) that are hypothesized to activate the physiological mechanisms underlying the expression of psychotic symptoms (Nuechterlein and Dawson, 1984).

FIGURE 1.

FIGURE 1

Open in a new tab

EE as a proxy risk factor of relapse.

To date, much research has highlighted the interplay between human agency and cultural factors (Emirbayer and Goodwin, 1994; Holland et al., 1998). For example, scholars from a variety of disciplines have noted the reciprocal relationship between agency and culture, with culture both shaping and being shaped by the expression of human agency (Dissanayake, 1996; Hays, 1994; Lehman et al., 2004; Ratner, 2000). Thus, a key limitation of the study of Breitborde et al. (2009) was its reliance on a sample composed solely of Euro-Americans. Although it is often problematic to assume that ethnicity (e.g., Euro-American) is an appropriate proxy for culture (López and Guarnaccia, 2000; López et al., 2009), investigation of the associations between agency, EE, and the course of schizophrenia in a single subpopulation may have obscured important cultural variation in the interrelationships between these variables.

Of note, there are a number of challenges in the study of human agency in psychological research. Most notably, there is a dearth of direct measures of this construct. Consequently, most studies to date have relied on proxy measures of human agency (Samman and Santos, 2009). One such proxy measure is self-efficacy (i.e., individuals’ confidence in their ability to complete prospective actions). Although the ability to complete desired actions (human agency) clearly extends beyond individuals’ confidence in their ability to complete desired actions (i.e., self-efficacy [Ahearn, 2001; Breitborde et al., 2009; Emirbayer and Mische, 1998]), available research indicates that self-efficacy clearly overlaps with the construct of human agency (Gecas, 1989; Kernis, 1995). In fact, Bandura (2000, 2006b) has suggested that self-efficacy may be the fundamental component of human agency.

Before exploring the relevance of caregivers’ perception of their ill relative’s efficacy to the EOI-relapse association among a Mexican-American sample, we will also examine the nature of the EOI-relapse association itself. Although there is mixed evidence with regard to whether the global EE construct (i.e., high EE versus low EE) predicts relapse for Mexican-Americans (Karno et al., 1987; Kopelowicz et al., 2006) or does not predict relapse for Mexican-Americans (Aguilera et al., 2010; Kopelowicz et al., 2002), the EE index of EOI is a reliable predictor of relapse among Mexican-Americans (Aguilera et al., 2010; Breitborde et al., 2007). Of note, the EE index of criticism does not predict relapse among Mexican-Americans (Aguilera et al., 2010; Kopelowicz et al., 2002; López et al., 2004). Previously, Breitborde et al. (2007) found that a curvilinear model best fits the relationship between EOI and relapse among Mexican-Americans, among whom high EOI was associated with a greater than average risk for relapse and moderate EOI was associated with a lower than average risk for relapse. This finding differs from the traditional assumption present within the EE literature that, when assessed as a continuous variable, higher levels of EOI are more problematic (i.e., associated with a greater risk for relapse) than lower levels of EOI and suggests that moderate levels of EOI may be ideal within a family caring for a relative with schizophrenia. In the current study, we will attempt to replicate this finding.

Thus, the goal of this study was to expand our understanding of the relationship between EOI and relapse among Mexican-Americans. First, we will attempt to replicate the finding that the relationship between EOI and relapse is curvilinear. Second, we will explore whether the relationship between EOI, relapse, and care-givers’ perception of their ill relative’s efficacy is best described by a mediator or a proxy risk factor model.

METHODS

Participants

Fifty-five Mexican-Americans with a schizophrenia-spectrum disorder and their caregiving relative were recruited from outpatient mental health clinics in southern California. The inclusion criteria for the caregivers and their ill relatives were a) being of Mexican origin and b) able to provide informed consent. Additional eligibility criteria for the ill relatives included a) diagnosis of a schizophrenia-spectrum disorder based on the Structured Clinical Interview for the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (First et al., 2002), and b) aged between 18 and 65 years.

The individuals with schizophrenia included 19 women and 36 men with a mean age of 39.44 years (SD, 10.99). Twenty-one individuals spoke primarily Spanish, and 34 spoke primarily English. Forty individuals were diagnosed with schizophrenia, and 15 were diagnosed with schizoaffective disorder.

The caregivers in this study were composed of 45 women and 10 men with a mean age of 54.63 years1 (SD, 16.74). Thirty-four caregivers spoke primarily Spanish, and 21 spoke primarily English. The caregivers in this study included 30 mothers, 6 sisters, 6 wives/girlfriends, 5 fathers, 3 daughters, 3 husbands, 1 brother, and 1 son.

Procedure

Upon enrollment in this study, both the caregivers and the ill relatives were administered a battery of measures. Of particular interest to this study, the caregivers completed an assessment of EOI and an assessment of their perception of their ill relative’s efficacy with regard to managing symptoms of schizophrenia. The ill relatives participated in monthly assessments of symptoms during the course of approximately 12.7 months (SD, 2.84; range, 8.9–22.7) to determine whether a relapse had occurred.2 The caregivers’ EOI and perception of their ill relative’s efficacy were reassessed at the end of this follow-up period.

Measures

Emotional Overinvolvement

The EE index of EOI was assessed using the Camberwell Family Interview (CFI; Vaughn and Leff, 1976). The CFI is a semistructured interview designed to assess the five indices of EE, including EOI. The Spanish version of the CFI used in this study was based on the translation used by Karno et al. (1987). Before scoring the CFI, all raters completed a training program in which they scored a minimum of 10 practice interviews and reached adequate to excellent levels of reliability as compared with master ratings with regard to scoring EOI (intraclass correlations [absolute agreement] = 0.69–0.95). All coders also participated in weekly rating meetings to reduce rater drift.

Caregiving Relatives’ Perceptions of Ill Relative’s Efficacy

A modified version of the Self-Efficacy Scale for Schizophrenia (SESS; McDermott, 1995) was used to assess the caregivers’ perceptions of their ill relative’s efficacy. The SESS is comprised of three subscales that assess individuals’ perception of their ability to control the prospective experience of positive and negative symptoms and exhibit appropriate social skills in the future. For the current study, we developed a modified version of the SESS that assesses the care-givers’ perception of their ill relative’s ability to complete these tasks. Items on this measure are scored from 0 to 100, with higher scores indicative of greater confidence in the ill relative’s capacity to execute the behavior. All three subscales possessed good to excellent internal consistency at baseline and follow-up (all Cronbach’s α ≥ 0.89).

The modified SESS was translated into Spanish by one member of our research team, and the translation was then reviewed by other members of the research team. For those items that were deemed to be poorly translated, alternative translations were offered, and a consensus was reached with regard to the proper translation.

Symptomatic Relapse

The Brief Psychiatric Rating Scale (BPRS; Lukoff et al., 1986) was administered to the ill relatives on a monthly basis. After the completion of data collection, we determined whether an individual had experienced a relapse during the follow-up period using the criteria proposed by Nuechterlein et al. (2006). On the basis of the longitudinal course of ratings for three items (i.e., hallucinations, unusual thought content, and conceptual disorganization) that are part of the positive symptom factor of the BPRS (Ventura et al., 2000), subjects can be classified as a) having relapsed, b) not having relapsed, or c) having experienced severe persistent symptoms. Of note, no subjects in this study met criteria for severe persistent symptoms. Additional detail with regard to the nine categories of symptom course that comprise the relapse, no relapse, and severe and persistent symptom classifications can be found in Nuechterlein et al. (2006). Consistent with the training program developed by Ventura et al. (1993) for the BPRS, before administering the BPRS, all raters scored six training tapes and attained a median intraclass correlation of 0.80 across all BPRS items with expert ratings.

The BPRS was translated into Spanish by one member of our research team, and the translation was then reviewed by other members of the research team. For those items that were deemed to be poorly translated, alternative translations were offered, and a consensus was reached with regard to the proper translation.

Statistical Analyses

To examine whether there was a quadratic relationship between EOI and relapse, we completed two logistic regression equations using relapse as the dependent variable. In the first equation, we examined the fit of the linear model (y = a + bx) by including EOI as the sole independent variable. In the second equation, we examined the fit of the quadratic model (y = a + bx + cx2) by including both the linear EOI term (x) and the quadratic EOI term (x2) as independent variables. To determine whether the quadratic model provided a better fit for the data, we compared the log-likelihood of each respective model using a chi-square test. For this final analysis, a one-tailed test is most appropriate given the specific question under investigation (i.e., Does the quadratic model fit the data better than the linear model?).

To test whether EOI was a proxy risk factor of relapse, we followed the protocol outlined by Breitborde et al. (2009). First, we examined whether EOI at baseline was associated with the care-givers’ perception of their ill relative’s efficacy at baseline using regression analysis. This analysis would confirm the existence of relationship A in Figure 1. Second, we evaluated whether the care-givers’ perception of their ill relative’s efficacy at baseline predicted the occurrence of a relapse. This analysis would confirm the existence of relationship B in Figure 1. Third, we determined whether the indirect effect (i.e., the combined effect of relationships A and B) was statistically significant using the distribution of the product analysis (z′: MacKinnon et al., 2002). Distribution of the product analysis has been shown to outperform other statistical tests of mediation with regard to both type I error and statistical power (MacKinnon et al., 2007, 2002). A statistically significant indirect effect would be consistent with the hypothesis that EOI is a proxy risk factor and does not directly influence the course of schizophrenia (relationship C in Figure 1).

Of note, the statistical tests for proxy risk factors are identical to those used to test for mediator variables despite the conceptual differences between these two relationships (MacKinnon et al., 2000). More specifically, whereas proxy risk factors are variables that only seem to be predictive of an outcome variable because of their strong association with a true risk factor, mediator variables are part of a hypothesized causal chain in which a predictor variable produces changes in the mediator variable, which, in turn, produces changes in the outcome variable (Kraemer et al., 2001). One key difference between mediator and proxy risk factor models is that mediational relationships require that the predictor variable temporally proceeds the mediator variable (i.e., changes in the predictor variable produce subsequent changes in the mediator variable), whereas proxy risk factor relationships do not (Breitborde et al., 2010; Kraemer et al., 2001). Thus, demonstrating that EOI scores are not associated with longitudinal changes in caregivers’ perception of their ill relative’s efficacy would suggest that the data are best described by a proxy risk factor model as opposed to a mediational model. Such an analysis deviates from traditional statistical analyses in that our goal was to “prove” the null hypothesis (i.e., the relationship between EOI and efficacy equals 0). Completing this analysis in the context of linear regression requires two steps (Dixon and Pechmann, 2005). First, one must define an equivalence region for the regression coefficient that encompasses the range of deviation from 0 that is considered negligible. Second, one completes two 1-sided tests that examine whether the regression coefficient falls outside the lower and the upper bound of the equivalence region. For both one-sided tests, the null hypothesis is that the regression coefficient falls outside the equivalence region; hence, rejecting both null hypotheses would suggest that the regression coefficient does differ significantly from 0.

To date, specific guidelines for determining the boundaries of an equivalence region are lacking (Blackwelder, 1982). As such, we defined our equivalence region for the regression coefficient of EOI at baseline as a predictor of each respective SESS subscale at follow-up, such that predicted SESS scores would not vary by more than 1 SD regardless of the EOI score at baseline.

RESULTS

Tests of skew and kurtosis revealed no significant departures from normality. Pre-analysis screening of the data revealed two multivariate outliers, which were dropped from the appropriate multivariate analyses. All variables were centered before the regression analyses to reduce multicollinearity between predictor variables.

Descriptive Statistics and Correlations

Means, standard deviations, and correlations between the EOI and SESS scores at baseline and relapse during the follow-up period3 are displayed in Table 1. The caregivers’ scores on the three sub-scales of the SESS clustered approximately in the middle of the scale range (i.e., 0–100), suggesting that the caregivers had moderate confidence in their ill relative’s efficacy with regard to the control of positive and negative symptoms as well as the display of appropriate social skills. The caregivers’ scores on the three sub-scales of the SESS were also positively correlated with each other (r’s = 0.62–0.78). Of note, only one baseline SESS subscale (i.e., positive symptoms) was correlated with the occurrence of a relapse during the follow-up period. More specifically, the caregivers who had less confidence in their ill relative’s ability to control future positive symptoms at baseline had ill relatives who were more likely to experience a relapse during the follow-up period.

TABLE 1.

Descriptive Statistics and Correlations Between Relapse, EOI, and Perceptions of Efficacy

Mean SD 1. 2. 3. 4. 5.
1. EOI 2.67 1.20 — −0.11 −0.13 −0.03 0.29*
2. SESS Positive Symptoms 59.26 23.12 — 0.62* 0.65* −0.33*
3. SESS Negative Symptoms 55.81 22.50 — 0.78* −0.17
4. SESS Social Skills 55.53 19.68 — −0.11
5. Relapse — — —
Open in a new tab
*

p < 0.05.

Is the Relationship Between EOI and Relapse Curvilinear?

To examine whether there was a quadratic relationship between EOI and relapse, we completed two logistic regression equations using relapse as the dependent variable. In the first equation, we examined the fit of the linear model (y = a + bx) by including EOI as the sole independent variable. In the second equation, we examined the fit of the quadratic model (y = a + bx + cx2) by including both the linear EOI term (x) and the quadratic EOI term (x2) as independent variables. To determine whether the quadratic model provided a better fit for the data, we compared the log-likelihood of each respective model using a chi-square test.

With regard to the relationship between EOI and relapse, both the linear model (χ21= 6.06; p < 0.01) and the quadratic model (χ22 = 9.10; p < 0.01) were statistically significant. Comparison of each model’s log-likelihood indicated that the quadratic model provided a superior fit for the data than did the linear model (χ21 = 3.04; p [one-sided] = 0.04). The curvilinear relationships between EOI and relapse among the current sample and the sample of Mexican-Americans in the study by Breitborde et al. (2007) are displayed in Figure 2. Of note, given that EOI scores of 0 and 5 were received only by one caregiver each, these points should not be overinterpreted.

FIGURE 2.

FIGURE 2

Open in a new tab

Predicted probability of relapse at different levels of EOI in the current sample and in the sample of Breitborde et al. (2007).

Is EOI a Proxy Risk Factor?

At baseline, only the caregivers’ scores on the SESS positive symptom subscale were associated with the occurrence of a relapse during the follow-up period (see Table 1). Of note, the lack of a relationship between relapse and the other SESS subscales indicates that these variables could not account for the relationship between EOI and relapse in this sample (Breitborde et al., 2010; Kraemer et al., 2001). As such, in our evaluation of whether EOI is a proxy risk factor, we limited our analyses solely to the caregivers’ scores on the SESS positive symptom subscale.

As noted earlier, three relationships would need to exist if the EOI were a proxy risk factor of relapse among Mexican-Americans. First, consistent with the curvilinear nature of the relationship between EOI and relapse, the quadratic EOI model was associated with the caregivers’ perception of their ill relative’s efficacy with regard to positive symptoms at baseline (r = 0.28; p = 0.04). This confirms the existence of relationship A in Figure 1.

There was a statistically significant negative relationship between the caregivers’ SESS positive symptom subscale scores at baseline and relapse during the follow-up period (B = 0.05; p = 0.02). This suggests that the caregivers who had less confidence in their ill relative’s ability to control future positive symptoms had ill relatives who were more likely to experience a relapse (relationship B in Figure 1).

Given that the relationship between EOI and relapse seems to be quadratic, the distribution of the product method cannot be applied to evaluate the statistical significance of the indirect effect. Specifically, this method (and all other methods of examining an indirect effect) assumes that the proxy risk factor is composed of a single factor. In the current study, the most accurate model of the relationship between EOI and relapse (y = a + bx + cx2) is composed of two factors: the linear EOI factor (bx) and the quadratic EOI factor (cx2). Given the curvilinear nature of the relationship between EOI and relapse, we completed our test of the indirect effect using a model that included only the quadratic EOI term (y = a + cx2). Among the current sample, this indirect effect was statistically significant (z′ = 1.06; p < 0.02).

To evaluate the alternative model that the caregivers’ perception of their ill relative’s efficacy may mediate the relationship between EOI and relapse, we tested whether the quadratic EOI factor was associated with the caregivers’ SESS positive symptom subscale scores at follow-up. Of note, we were able to obtain follow-up SESS ratings from only 61% of the caregivers4 (33/54). As per existing recommendations (Collins, 2006), missing data were replaced using multiple imputation. Of note, multiple imputation has been shown to produce reliable estimates in situations in which there was a similar or greater percentage of missing data as compared with our data set (e.g., Guyll et al., 2004; Royston, 2004). After controlling for the caregivers’ SESS positive symptom subscale scores at baseline, the quadratic EOI variable did not predict the caregivers’ SESS positive symptom subscale scores at follow-up (B = 0.10; p = 0.84). As per the guidelines outlined above, 0.96 to 0.96 was determined as the range of equivalence for this regression coefficient. Follow-up equivalence testing suggests that this effect was a negligible trend (tLower [52] = 2.11; p=0.02; tUpper [52] = 1.71; p = 0.046). These results are consistent with the proxy risk factor model of EOI and are inconsistent with a mediational relationship.

DISCUSSION

The goal of this study was to expand our understanding of the relationship between EOI and relapse among Mexican-Americans. Our results provide confirmatory evidence that the relationship between EOI and relapse among Mexican-Americans is curvilinear and provide preliminary evidence that caregivers’ perception of their ill relative’s efficacy may account for this relationship.

With regard to the relationship between EOI and relapse among Mexican-Americans, it is notable that the shape of the curvilinear model is similar in both the current sample and that of the previous study by Breitborde et al. (2007), with the lowest rate of relapse associated with moderate levels of EOI and the highest rate of relapse associated with high levels of EOI. This differs from the traditional assumption present within the EE literature that higher levels of EOI are more problematic (i.e., associated with a greater risk for relapse) than lower levels of EOI (Breitborde et al., 2007) and suggests that moderate levels of EOI may be ideal within a family caring for a relative with schizophrenia. Of note, however, among the current sample, the probability of relapse was lower at all levels of EOI as compared with the sample of the study of Breitborde et al. (2007; Figure 2). This overall reduced rate of relapse may stem from a key difference in the inclusion criteria across the two studies. Specifically, whereas individuals with schizophrenia included in the study of Breitborde et al. (2007) were required to have been hospitalized within the last month, most of the participants in the current sample had been living independently in the community for several years without hospitalization. Because the risk for relapse in schizophrenia may be greater among individuals who have recently been hospitalized (Hogarty and Ulrich, 1977), this difference in inclusion criteria may account for the differential rates of relapse across the two studies.

Similar to the previous study by Breitborde et al. (2009) of Euro-Americans with schizophrenia, the caregivers’ perception of their ill relative’s efficacy (i.e., a proxy for perceptions of agency) was shown to account for the relationship between EOI and relapse among the Mexican-Americans with schizophrenia in the current study. Specifically, EOI may only seem to have a direct effect on the course of schizophrenia because of its strong correlation with a true risk factor for relapse (i.e., caregivers’ perception of their ill relative’s efficacy). Moreover, in the current study, we were able to evaluate the possibility that the caregivers’ perception of their ill relative’s efficacy mediated the relationship between EOI and relapse—a possibility that was not supported by the data. In total, these findings provide further support for the conclusion that EOI is a proxy risk factor of relapse among individuals with schizophrenia and suggest that the applicability of this finding may extend beyond Euro-Americans.

The specific nature of the relationship between EOI, care-givers’ perception of their ill relative’s efficacy, and relapse in the current sample differs from that observed in the previous study carried out with Euro-Americans. In the previous study, Breitborde et al. (2009) found that greater perceptions of agency were associated with a greater likelihood of relapse. This runs counter to the current finding that greater perceptions of efficacy (i.e., a proxy measure for human agency) are associated with a lower likelihood of relapse. There are at least two possible explanations for this distinct pattern of association. First, the two samples differ with regard to clinical stability. The former study included persons who were recently hospitalized, whereas the current study included more stable patients. For persons who are not clinically stable, failing to recognize how one’s illness impinges on one’s agency may be particularly stressful. In contrast, for those who have been stable for quite some time, perceptions that they have little agency in their illness-related behavior is what may be most stressful. In this case, persons may find the illness role as problematic because they want to feel more responsible for their lives.

A second difference is that Euro-Americans comprised the sample in the past study and primarily immigrant Mexican-Americans comprised the sample in the current study. One could consider the cultural prototypes of Euro-Americans as being more individualistic and Mexican-Americans as being more collectivistic. Accordingly, Euro-American caregivers, when burdened with the stress of caring for an ill relative, may draw on their cultural orientation and view the expression of symptoms as stemming largely from their ill relative’s agency, with little recognition of the role of the illness (Jenkins, 1988a, 1988b; Weisman and López, 1997). This may be particularly stressful to the ill relatives who may be more apt to recognize the role of their illness in their everyday lives. In contrast, for more collectivistic Mexican-American caregivers, when stressed, they may rely on their cultural orientation of reaching out to their ill relatives, acknowledging that their ill relative is not responsible for his/her illness (Jenkins, 1988a, 1988b; Weisman and López, 1997). However, these actions may be excessive and, in turn, be stressful to the ill relatives because it communicates that they have little agency regarding their illness. Although this cultural explanation may be consistent with the data, more proximal measures of cultural processes are needed—individualism/collectivism and the distress felt by the ill relatives (Medina-Pradas et al., 2011). Future research will help to clarify the role of specific cultural processes as these relate to agency and clinical outcomes across sociocultural contexts.

It is noteworthy that the caregivers’ scores on only one subscale of the SESS (i.e., positive symptoms) were associated with relapse. However, this finding comports with available research. More specifically, Bandura (2006a) has noted that self-efficacy scales tend to be predictive of only the specific domain of functioning that they are designed to assess. Thus, given that criteria for relapse in this study are based solely on the recurrence of positive symptoms (Nuechterlein et al., 2006), it is not surprising that the positive symptom subscale would be predictive of relapse, whereas the negative symptom and social skills subscales would not. In fact, in a previous study that used a composite measure of self-efficacy composed only of the negative symptom and social skills subscales of the SESS, Pratt et al. (2005) found that this composite self-efficacy scale was associated with negative symptoms and social functioning among individuals with schizophrenia.

This study did have some limitations. For instance, the inability to evaluate the statistical significance of the indirect effect using all components of the quadratic EOI model (i.e., EOI and EOI × EOI) suggests that the results of this analysis may need to be interpreted tentatively. In addition, given that most of the individuals in this study had experienced symptoms of schizophrenia for several years, it is unclear whether the findings may be applicable to individuals with first-episode or prodromal psychosis. Likewise, although the translation of study measures was completed across multiple observers with multiple iterations, a formal translation back-translation approach was not used (Brislin, 1970). In addition, the SESS does not specify the time frame for which a subject is being queried about his/her perceived ability to complete a desired action in the future. Because the strength of the predictive association between self-efficacy and successful task performance decreases in magnitude as the temporal gap between the assessment of self-efficacy and the attempted performance of the task increases (Bandura, 1997), the lack of a specified time frame on the SESS may have resulted in an underestimation of the magnitude of the relationship between the caregivers’ perceptions of their ill relative’s efficacy and relapse. Finally, because of the significant challenges associated with quantifying human agency, we relied on a proxy measure of this construct (i.e., self-efficacy).

Although self-efficacy has been recognized as an important component of the recovery process in schizophrenia (Davidson, 2003; Davidson and Strauss, 1992; Deegan, 2001; Young and Ensing, 1999), there are few empirical studies that have sought to clarify the specific nature in which this construct matters for individuals with schizophrenia. The results of the current study contribute to other recent attempts to address this gap in the literature (Bentall et al., 2010; Corrigan et al., 2006; Pratt et al., 2005; Vauth et al., 2007) and also highlight the social aspect of efficacy (i.e., others’ perception of our efficacy may also be an influential factor in our lives). Further exploration of the perceptions of one’s efficacy held by others within one’s social world(s) may ultimately provide greater insight into the role of efficacy (and more broadly, human agency) in the relationship between family factors and the course of schizophrenia.

Acknowledgments

The authors thank Christina Chang, PhD; Mariana Cervantes Mendez; Perla Placencia, MSW; Susie Solano; and Roberto Zarate, PhD, for their assistance with this project.

Footnotes

1

Age for one caregiver was not obtained. As such, these figures are based on n =54.

2

Although there was variation in the amount of time for which each subject was followed as part of this study, follow-up time was not associated with the occurrence of a relapse during the course of this study (B < 0.01; p =0.98). As such, follow-up time was not included as a covariate in the longitudinal assessments of predictors of relapse.

3

Point biserial correlations were used to test the relationship between continuous variables and the dichotomous relapse variable (i.e., relapse versus no relapse).

4

The multivariate outlier was excluded from this multivariate analysis, hence, n =54 as opposed to n =55.

The authors declare no conflict of interest.

DISCLOSURES

This work was supported by the National Institute of Mental Health (MH01449 to S. R. L. and MH071054 to N. J. K. B.) and UCLA Life Sciences (to S.R.L.).

References

  1. Aguilera A, Lopez SR, Breitborde NJ, Kopelowicz A, Zarate R. Expressed emotion and sociocultural moderation in the course of schizophrenia. J Abnorm Psychol. 2010;119:875–885. doi: 10.1037/a0020908. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Ahearn LM. Language and agency. Annu Rev Anthropol. 2001;30:109–137. [Google Scholar]
  3. Anderson BJ, Coyne JC. “Miscarried helping” in the families of children and adolescents with chronic diseases. In: Johnson JHJ, Bennett S, editors. Advances in child health psychology. Gainesville: University of Florida Press; 1991. pp. 167–177. [Google Scholar]
  4. Bandura A. Self-efficacy: The exercise of control. New York: Freeman; 1997. [Google Scholar]
  5. Bandura A. Self-efficacy: The foundation of agency. In: Perrig WJG, Grob A, editors. Control of human behavior, mental processes, and consciousness: Essays in honor of the 60th birthday of August Flammer. Mahwah, NJ: Lawrence Erlbaum Associates; 2000. pp. 17–33. [Google Scholar]
  6. Bandura A. Guide for constructing self-efficacy scales. In: Pajares F, Urdan T, editors. Self-efficacy beliefs of adolescents. Greenwich, CT: Information Age Publishing; 2006a. pp. 307–337. [Google Scholar]
  7. Bandura A. Toward a psychology of human agency. Perspect Psychol Sci. 2006b;1:164–180. doi: 10.1111/j.1745-6916.2006.00011.x. [DOI] [PubMed] [Google Scholar]
  8. Barrowclough C, Hooley JM. Attributions and expressed emotion: A review. Clin Psychol Rev. 2003;23:849–880. doi: 10.1016/s0272-7358(03)00075-8. [DOI] [PubMed] [Google Scholar]
  9. Bentall RP, Simpson PW, Lee DA, Williams S, Elves S, Brabbins C, Morrison AP. Motivation and avolition in schizophrenia patients: The role of self-efficacy. Psychosis. 2010;2:12–22. [Google Scholar]
  10. Blackwelder WC. “Proving the null hypothesis” in clinical trials. Control Clin Trials. 1982;3:345–353. doi: 10.1016/0197-2456(82)90024-1. [DOI] [PubMed] [Google Scholar]
  11. Blascovish J, Spencer SJ, Quinn D, Steele C. African Americans and high blood pressure: The role of stereotype threat. Psychol Sci. 2001;12:225–229. doi: 10.1111/1467-9280.00340. [DOI] [PubMed] [Google Scholar]
  12. Breitborde NJ, Lopez SR, Ochs E, Nuechterlein KH. Emotional over-involvement, self efficacy, and recovery from schizophrenia: An understanding of an index of expressed emotion through narrative. Schizophr Bull. 2005;31:196. [Google Scholar]
  13. Breitborde NJ, Lopez SR, Wickens TD, Jenkins JH, Karno M. Toward specifying the nature of the relationship between expressed emotion and schizophrenic relapse: The utility of curvilinear models. Int J Methods Psychiatr Res. 2007;16:1–10. doi: 10.1002/mpr.194. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Breitborde NJK, Lopez SR, Nuechterlein KH. Expressed emotion, human agency, and schizophrenia: Toward a new model for the EE-relapse association. 2009;33:41–60. doi: 10.1007/s11013-008-9119-x. [DOI] [PubMed] [Google Scholar]
  15. Breitborde NJK, Srihari VH, Pollard JM, Addington DN, Woods SW. Mediators and moderators in early intervention research. Early Interv Psychiatry. 2010;4:143–152. doi: 10.1111/j.1751-7893.2010.00177.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Brislin RW. Back-translation for cross-cultural research. J Cross Cult Psychol. 1970;1:185–216. [Google Scholar]
  17. Butzlaff RL, Hooley JM. Expressed emotion and psychiatric relapse: A meta-analysis. Arch Gen Psychiatry. 1998;55:547–552. doi: 10.1001/archpsyc.55.6.547. [DOI] [PubMed] [Google Scholar]
  18. Clarke DE, Walker JR, Cuddy TE. The role of perceived overprotectiveness in recovery 3 months after myocardial infarction. J Cardiopulm Rehabil. 1996;16:372–377. doi: 10.1097/00008483-199611000-00006. [DOI] [PubMed] [Google Scholar]
  19. Collins LM. Analysis of longitudinal data: The integration of theoretical model, temporal design, and statistical model. Annu Rev Psychol. 2006;57:505–528. doi: 10.1146/annurev.psych.57.102904.190146. [DOI] [PubMed] [Google Scholar]
  20. Corrigan PW, Watson AC, Barr L. The self-stigma of mental illness: Implications for self-esteem and self-efficacy. J Soc Clin Psychol. 2006;25:875–884. [Google Scholar]
  21. Coyne JC, Wortman CB, Lehman DR. The other side of support: Emotional overinvolvement and miscarried helping. In: Gottlieb BH, editor. Marshaling social support: Formats, processes, and effects. Thousand Oaks, CA: Sage Publications; 1988. pp. 305–330. [Google Scholar]
  22. Davidson L. Living outside mental illness: Qualitative studies of recovery in schizophrenia. New York: New York University Press; 2003. [Google Scholar]
  23. Davidson L, Strauss JS. Sense of self in recovery from severe mental illness. Br J Med Psychol. 1992;65:131–145. doi: 10.1111/j.2044-8341.1992.tb01693.x. [DOI] [PubMed] [Google Scholar]
  24. Deegan PE. Recovery as a self-directed process of healing and transformation. In: Brown C, editor. Recovery and wellness: Models of hope and empowerment for people with mental illness. New York: Haworth Press; 2001. pp. 5–21. [Google Scholar]
  25. Dissanayake W. Introduction/agency and cultural understanding: Some preliminary remarks. In: Dissanayake W, editor. Narratievs of agency: Self-making in China, India, and Japan. Minneapolis: University of Minnesota Press; 1996. pp. ix–xxi. [Google Scholar]
  26. Dixon PM, Pechmann JHK. A statistical test to show negligible trend. Ecology. 2005;86:1751–1756. [Google Scholar]
  27. Emirbayer M, Goodwin J. Network analysis, culture, and the problem of agency. Am J Sociol. 1994;99:1411–1454. [Google Scholar]
  28. Emirbayer M, Mische A. What is agency? Am J Sociol. 1998;103:962–1023. [Google Scholar]
  29. First MB, Spitzer RL, Gibbon M, Williams JBW. Structured Clinical Interview for DSM-IV-TR Axis I Disorders, Research Version, Patient Edition With Psychotic Screen (SCID-I/P W/PSY SCREEN) New York: Biometrics Research, New York State Psychiatric Institute; 2002. [Google Scholar]
  30. Gecas V. The social psychology of self-efficacy. Annu Rev Sociol. 1989;15:291–316. [Google Scholar]
  31. Guyll M, Spoth RL, Chao W, Wickrama KA, Russell D. Family-focused preventive interventions: Evaluating parental risk moderation of substance use trajectories. J Fam Psychol. 2004;18:293–301. doi: 10.1037/0893-3200.18.2.293. [DOI] [PubMed] [Google Scholar]
  32. Hays S. Structure and agency and the sticky problem of culture. Sociol Theory. 1994;12:57–72. [Google Scholar]
  33. Hogarty GE, Ulrich RF. Temporal effects of drug and placebo in delaying relapse in schizophrenic outpatients. Arch Gen Psychiatry. 1977;34:297–301. doi: 10.1001/archpsyc.1977.01770150055005. [DOI] [PubMed] [Google Scholar]
  34. Holland D, Lachicotte WJ, Skinner D, Cain C. Identity and agency in cultural worlds. Cambridge, MA: Harvard University Press; 1998. [Google Scholar]
  35. Holmbeck GN, Johnson SZ, Wills KE, McKernon W, Rose B, Erklin S, Kemper T. Observed and perceived parental overprotection in relation to psychosocial adjustment in preadolescents with a physical disability: The mediational role of behavioral autonomy. J Consult Clin Psychol. 2002;70:96–110. doi: 10.1037//0022-006x.70.1.96. [DOI] [PubMed] [Google Scholar]
  36. Hooley JM. Expressed emotion and relapse of psychopathology. Annu Rev Clin Psychol. 2007;3:329–352. doi: 10.1146/annurev.clinpsy.2.022305.095236. [DOI] [PubMed] [Google Scholar]
  37. Hooley JM, Gotlib IH. A diathesis-stress conceptualization of expressed emotion and clinical outcome. Appl Prev Psychol. 2000;9:135–151. [Google Scholar]
  38. Jenkins JH. Conceptions of schizophrenia as a problem of nerves: A cross-cultural comparison of Mexican-Americans and Anglo-Americans. Soc Sci Med. 1988a;26:1233–1243. doi: 10.1016/0277-9536(88)90155-4. [DOI] [PubMed] [Google Scholar]
  39. Jenkins JH. Ethnopsychiatric interpretations of schizophrenic illness: The problem of nervios within Mexican-American families. Cult Med Psychiatry. 1988b;12:301–329. doi: 10.1007/BF00051972. [DOI] [PubMed] [Google Scholar]
  40. Karno M, Jenkins JH, De La Selva A, Santana F, Telles C, Lopez S, Mintz J. Expressed emotion and schizophrenic outcome among Mexican-American families. J Nerv Ment Dis. 1987;175:143–151. doi: 10.1097/00005053-198703000-00004. [DOI] [PubMed] [Google Scholar]
  41. Kavanagh DJ. Recent developments in expressed emotion and schizophrenia. Br J Psychiatry. 1992;160:601–620. doi: 10.1192/bjp.160.5.601. [DOI] [PubMed] [Google Scholar]
  42. Kenny DA. Cross-legged panel correlation: A test for spuriousness. Psychol Bull. 1975;82:887–903. [Google Scholar]
  43. Kernis MH. Efficacy, agency, and self-esteem. New York: Plenum Press; 1995. [Google Scholar]
  44. Kopelowicz A, Lopez SR, Zarate R, O’Brien M, Gordon J, Chang C, Gonzalez-Smith V. Expressed emotion and family interactions in Mexican Americans with schizophrenia. J Nerv Ment Dis. 2006;194:330–334. doi: 10.1097/01.nmd.0000217880.36581.6b. [DOI] [PubMed] [Google Scholar]
  45. Kopelowicz A, Zarate R, Gonzalez V, Lopez SR, Ortega P, Obregon N, Mintz J. Evaluation of expressed emotion in schizophrenia: A comparison of Caucasians and Mexican-Americans. Schizophr Res. 2002;55:179–186. doi: 10.1016/s0920-9964(01)00193-1. [DOI] [PubMed] [Google Scholar]
  46. Kraemer HC, Stice E, Kazdin A, Offord D, Kupfer D. How do risk factors work together? Mediators, moderators, and independent, overlapping, and proxy risk factors. Am J Psychiatry. 2001;158:848–856. doi: 10.1176/appi.ajp.158.6.848. [DOI] [PubMed] [Google Scholar]
  47. Kuijer RG, Ybema JF, Buunk BP, De Jong GM, Thijs-Boer F, Sanderman R. Active engagement, protective buffering, and overprotection: Three ways of giving support by intimate partners of patients with cancer. J Soc Clin Psychol. 2000;19:256–275. [Google Scholar]
  48. Kuipers E, Onwumere J, Bebbington P. Cognitive model of caregiving in psychosis. Br J Psychiatry. 2010;196:259–265. doi: 10.1192/bjp.bp.109.070466. [DOI] [PubMed] [Google Scholar]
  49. Lehman DR, Chiu C-Y, Schaller M. Psychology and culture. Annu Rev Psychol. 2004;55:689–714. doi: 10.1146/annurev.psych.55.090902.141927. [DOI] [PubMed] [Google Scholar]
  50. López SR, Guarnaccia PJJ. Cultural psychopathology: Uncovering the social world of mental illness. Annu Rev Psychol. 2000;51:571–598. doi: 10.1146/annurev.psych.51.1.571. [DOI] [PubMed] [Google Scholar]
  51. López SR, Nelson Hipke K, Polo AJ, Jenkins JH, Karno M, Vaughn C, Snyder KS. Ethnicity, expressed emotion, attributions, and course of schizophrenia: Family warmth matters. J Abnorm Psychol. 2004;113:428–439. doi: 10.1037/0021-843X.113.3.428. [DOI] [PubMed] [Google Scholar]
  52. López SR, Ramirez Garcia JI, Ullman JB, Kopelowicz A, Jenkins J, Breitborde NJK, Placencia P. Cultural variability in the manifestation of expressed emotion. Fam Process. 2009;48:179–194. doi: 10.1111/j.1545-5300.2009.01276.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Lukoff D, Nuechterlein KH, Ventura J. Appendix A. Manual for Expanded Brief Psychotic Rating Scale (BPRS) Schizophr Bull. 1986;12:594–602. [Google Scholar]
  54. MacKinnon DP, Fairchild AJ, Fritz MS. Mediation analysis. Annu Rev Psychol. 2007;58:593–614. doi: 10.1146/annurev.psych.58.110405.085542. [DOI] [PMC free article] [PubMed] [Google Scholar]
  55. MacKinnon DP, Krull JL, Lockwood CM. Equivalence of the mediation, confounding and suppression effect. Prev Sci. 2000;1:173–181. doi: 10.1023/a:1026595011371. [DOI] [PMC free article] [PubMed] [Google Scholar]
  56. MacKinnon DP, Lockwood CM, Hoffman JM, West SG, Sheets V. A comparison of methods to test mediation and other intervening variable effects. Psychol Methods. 2002;7:83–104. doi: 10.1037/1082-989x.7.1.83. [DOI] [PMC free article] [PubMed] [Google Scholar]
  57. McDermott BE. Development of an instrument for assessing self-efficacy in schizophrenic spectrum disorders. J Clin Psychol. 1995;51:320–331. [PubMed] [Google Scholar]
  58. Medina-Pradas C, Navarro JB, Lopez SR, Grau A, Obiols JE. Dyadic view of expressed emotion, stress, and eating disorder psychopathology. Appetite. 2011;57:743–748. doi: 10.1016/j.appet.2011.08.016. [DOI] [PubMed] [Google Scholar]
  59. Nuechterlein KH, Dawson ME. A heuristic vulnerability/stress model of schizophrenic episodes. Schizophr Bull. 1984;10:300–312. doi: 10.1093/schbul/10.2.300. [DOI] [PubMed] [Google Scholar]
  60. Nuechterlein KH, Miklowitz DJ, Ventura J, Gitlin MJ, Stoddard M, Lukoff D. Classifying episodes in schizophrenia and bipolar disorder: Criteria for relapse and remission applied to recent-onset samples. Psychiatry Res. 2006;144:153–166. doi: 10.1016/j.psychres.2004.04.018. [DOI] [PubMed] [Google Scholar]
  61. Pratt SI, Mueser KT, Smith TE, Lu W. Self-efficacy and psychosocial functioning in schizophrenia: A mediational analysis. Schizophr Res. 2005;78:187–97. doi: 10.1016/j.schres.2005.02.014. [DOI] [PubMed] [Google Scholar]
  62. Ratner C. Agency and culture. J Theory Soc Behav. 2000;30:413–434. [Google Scholar]
  63. Royston P. Multiple imputation for non-responses in surveys. Stata J. 2004;4:227–241. [Google Scholar]
  64. Samman E, Santos ME. The 2009 Human Development Report in Latin America and the Caribbean. Oxford Poverty and Human Development Initiative; 2009. Agency and empowerment: A review of concepts, indicators and empirical evidence. [Google Scholar]
  65. Schmader T, Johns M. Converging evidence that stereotype threat reduces working memory capacity. J Pers Soc Psychol. 2003;85:440–452. doi: 10.1037/0022-3514.85.3.440. [DOI] [PubMed] [Google Scholar]
  66. Simon HA. Spurious correlation: A causal interpretation. J Am Stat Assoc. 1954;49:467–479. [Google Scholar]
  67. Singh SP, Harley K, Suhail K. Cultural specificity of emotional over-involvement: A systematic review. Schizophr Bull. 2013;39:449–463. doi: 10.1093/schbul/sbr170. [DOI] [PMC free article] [PubMed] [Google Scholar]
  68. Tanaka S, Mino Y, Inoue S. Expressed emotion and the course of schizophrenia in Japan. Br J Psychiatry. 1995;167:794–798. doi: 10.1192/bjp.167.6.794. [DOI] [PubMed] [Google Scholar]
  69. Thomasgard M, Metz WP. Parental overprotection and its relation to perceived child vulnerability. Am J Orthopsychiatry. 1997;67:330–335. doi: 10.1037/h0080237. [DOI] [PubMed] [Google Scholar]
  70. Thomasgard M, Shonkoff JP, Metz WP, Edelbrock C. Parent-child relationship disorders: II. The vulnerable child syndrome and its relation to parental overprotection. J Dev Behav Pediatr. 1995;16:251–256. [PubMed] [Google Scholar]
  71. Vaughn C, Leff J. The measurement of expressed emotion in the families of psychiatric patients. Br J Soc Clin Psychol. 1976;15:157–165. doi: 10.1111/j.2044-8260.1976.tb00021.x. [DOI] [PubMed] [Google Scholar]
  72. Vauth R, Kleim B, Wirtz M, Corrigan PW. Self-efficacy and empowerment as outcomes of self-stigmatizing and coping in schizophrenia. Psychiatry Res. 2007;150:71–80. doi: 10.1016/j.psychres.2006.07.005. [DOI] [PubMed] [Google Scholar]
  73. Ventura J, Green MF, Shaner A, Liberman RP. Training and quality assurance with the Brief Psychiatric Rating Scale: ‘The drift busters’. Int J Methods Psychiatr Res. 1993;3:221–244. [Google Scholar]
  74. Ventura J, Nuechterlein KH, Subotnik KL, Gutkind D, Gilbert EA. Symptom dimensions in recent-onset schizophrenia and mania: A principal components analysis of the 24-item Brief Psychiatric Rating Scale. Psychiatry Res. 2000;97:129–135. doi: 10.1016/s0165-1781(00)00228-6. [DOI] [PubMed] [Google Scholar]
  75. Weiner B. Judgments of responsibility: A foundation for a theory of social conduct. New York: Guilford Press; 1995. [Google Scholar]
  76. Weisman AG, López SR. An attributional analysis of emotional reactions to schizophrenia in Mexican and Anglo American cultures. J Appl Soc Psychol. 1997;27:223–244. [Google Scholar]
  77. Young SL, Ensing DS. Exploring recovery from the perspective of people with psychiatric disabilities. Psychiatr Rehabil J. 1999;22:219–231. [Google Scholar]
  78. Yule GU. An introduction to the theory of statistics. 6. London: Griffin; 1922. [Google Scholar]

RESOURCES