Cross-National Comparisons: Problems in Interpretation When Studies Are Based on Prevalent Cases

Cohen et al challenge the belief, stemming largely from the World Health Organization (WHO) schizophrenia research program, that “schizophrenia has a better course and outcome in countries of the developing world” compared with developed countries. They thus examine findings on illness course, mortality, and social and occupational outcomes across an array of studies conducted in low- or middle-income countries throughout the world. They consider findings from all follow-up studies of schizophrenia conducted in these countries, without regard to case identification specification or length of follow-up. The outcome results across the studies identified by Cohen et al were variable, and in some cases, the samples fared very poorly indeed. Because none of the studies had control groups, cross-national disparities in the social and occupational trajectories of the patients with schizophrenia compared with counterparts from the same source populations are unknown.

Cohen et al suggest that there is a need for more outcome studies of “incident and prevalent cases.” Clearly, outcome studies of incident cases, defined as first episode, first contact with a provider, or first hospitalization, can potentially provide insights about the course and outcome of schizophrenia, and they are, unfortunately, fairly rare. On the other hand, follow-up studies of consecutive admissions to psychiatric inpatient and outpatient facilities, or prevalent cases, appear with great regularity. However, their findings are largely uninterpretable. As Cohen and Cohen¹ elegantly delineated nearly 25 years ago, many forms of selection bias creep into prevalent samples, particularly chronicity and comorbidity, creating an illusion about the nature and correlates of illness course. Studies with variable lengths of time between illness onset and outcome determination, such as those reviewed by Cohen et al, create ambiguity about the course of schizophrenia rather than shed light on the process and its mechanisms across time and place. Thus, additional prevalent studies in low- and middle-income countries are unlikely to improve our understanding of cross-national differences in the course of schizophrenia.

A prevalent sample is a mixture of recent onset and chronically ill patients. The specific composition of the mix will influence the findings and the inferences that are ultimately to be drawn. In our chapter on the epidemiology of psychosis, we showed that in a single mixed sample, the 5-year outcome was significantly better for the incident (first-admission) patients than for the prevalent (rehospitalized) patients in the group.² Among the incident cases, 22% experienced no relapse, 35% had one or more relapses with minimal or no impairment, and 43% were impaired throughout the 5 years. Among the prevalent cases, only 10% had no impairment, whereas 29% had multiple episodes with no/minimal impairment, and 60% were impaired with no return to normality. The more general point is that prevalent (consecutive admission) samples are biased by chronicity, thus overrepresenting the most severe and intractable forms of the illness. In contrast, incident samples, even when ascertained from clinical services, are less biased by chronicity although they too contain patients with variable durations of untreated or undertreated illness. Population-based incident samples identified from cohort studies, such as birth cohorts, are the most representative and contain the least bias.

A useful way to think about follow-up studies of schizophrenia is to conceptualize them as clinical cohort studies.³ The design requirements then become very clear. A cohort is a group with a common experience that is observed in a standardized fashion over time. Thus, a clinical cohort is a group with a uniform starting point (the closer to the actual onset, the better for minimizing lead-time bias), a transparent and systematic method of case ascertainment, and a reliable method of assessment and assignment of diagnosis. By their nature, well-designed clinical cohort studies yield findings that are readily interpretable. The WHO first-contact study was designed with these basic principles in mind.⁴ As Cohen et al point out, the cross-cultural application was not without its problems, but the ability to make comparisons was greatly enhanced by the design. Two recent North American studies were also designed with the same goals.^5,6 Thus, in countries where “ideal” epidemiologic samples, such as birth cohorts, are not available or feasible, the clinical cohort design offers the next best alternative.

The bottom line is that Cohen et al argue that the “better prognosis” hypothesis is not correct for all poor and middle-income countries because both good and bad outcomes are found in socioeconomically deprived regions of the world. In deciding about studies to include in their review, the distinction between incident and prevalent cases was noted but was not considered particularly germane. It is important to emphasize that the WHO investigators did not claim that all patients in developing countries had good outcome trajectories. Instead, they found that on average, the incident cohorts from developing countries fared better than those from richer countries. (As an aside, the World Mental Health Consortium, a series of population-based morbidity surveys, also found higher rates of psychiatric disorder in wealthy countries like the United States compared with poor countries like Nigeria.⁷) From an epidemiologic perspective, the biases in prevalent samples cannot be ignored when evaluating and comparing prognosis across studies. It is simply not possible to judge whether the differences in the outcomes reported in this review, including mortality, or the differences in the prognostic factors, represent methodological artifacts, true differences among the populations associated with culture or genetics, or a bit of both.

This commentary is not intended as a tutorial. Our textbook on psychiatric epidemiology has several chapters devoted to the design of clinical cohort studies and considerations in specifying the criteria for selecting cases and controls for clinical research.³ Rather, I would like to suggest a method for undertaking new clinical cohort studies in resource-poor environments so that we can extend our understanding of cross-national differences in illness course with relative expedience. Specifically, an economical method that can be implemented in poor settings with stable population bases is the historical cohort design, as exemplified by the classic studies of Ciompi,⁸ Huber,⁹ and Bleuler.¹⁰ This strategy requires that the initial presentation of the illness is carefully documented in a record, which is often the case, and also that the likelihood of locating the patients for follow-up is not biased by whether they remain in treatment. For example, one of the most influential studies in the field of psychiatry was Deviant Children Grown Up, a historical cohort study of children evaluated for behavior problems in 1924–1929 who were systematically assessed 30 years later, along with classmate controls.¹¹ The richness of the original records allowed the investigator to determine the childhood predictors of adult antisocial behavior and mood/anxiety disorders. Historical cohort studies from rich, middle-income, and poor countries of the world could provide important evidence about relative differences in the psychosocial trajectories of schizophrenia and their risk factors, particularly if they employ similar methodologies and measures.

I fully agree with the conclusion of Cohen et al that “clinical, epidemiological, and ethnographic” strategies are needed to identify the processes that promote good outcome in different settings. There is no debate among epidemiologists about how to design an informative clinical cohort study, and such designs can be implemented around the world. Hopefully, the next generation of schizophrenia outcomes research will fully integrate the 3 domains mentioned by Cohen et al as well as biologic, genetic, and treatment moderators.