Concordance between Chart Review and Structured Interview Assessments of Schizophrenic Symptoms

Abstract

Background

It is unclear whether direct structured interviews are able to capture the full range of psychopathology in schizophrenia, as is required in diagnostic assessments or clinical ratings. We examined agreement between symptom ratings derived from direct patient interviews and from review of casenotes.

Methods

The study sample comprised 1021 schizophrenic subjects collected as part of the Irish Case-Control Study of Schizophrenia (ICCSS). Diagnostic interviews utilized a modified version of the Structured Clinical Interview for DSM-III-R. Symptoms were rated by the interviewer. In addition, the Casenote Rating Scale was used to rate symptoms based on medical record information. For each negative and positive symptom, we calculated the Pearson correlation between the interview and the casenote rating. Using the mean of the interview and casenote rating for each symptom, exploratory factor analysis using Varimax rotation was performed.

Results

Three factors were extracted in factor analysis: positive, negative, and Schneiderian symptoms. The highest correlations between interview and casenote ratings were for negative symptoms, in which all symptoms were significantly correlated. Positive and Schneiderian symptoms were significantly correlated with the exception of thought insertion, thought withdrawal, voices speaking in sentences, and somatic hallucinations. Significant correlations were generally moderate (0.2–0.55)

Conclusion

Most schizophrenic symptoms, especially negative symptoms, can be assessed by direct interviews as the sole source of information with moderate reliability. However, the presence of some Schneiderian and possibly less prevalent positive symptoms may be difficult to determine without a review of records, which may include longitudinal observations and information from multiple observers.

Introduction

Schizophrenia has been conceptualized as a clinically multidimensional illness since its earliest modern descriptions[2]. The assessment of symptoms is a fundamental first step in making a reliable psychiatric diagnosis. It is also a crucial part of the process of determining drug response. Needless to say, primary studies of schizophrenia psychopathology, such as factor analyses of clinical features, presume an accurate and reliable assessment of signs and symptoms, on which the validity of any factor structure depends[18].

The assessment of schizophrenic psychopathology by clinicians is complicated by several factors. One of the most important is the unwillingness or inability of many patients to communicate their inner experiences. Some may be unwilling to express delusional and hallucinatory phenomena for fear of incurring involuntary commitment. This may be reality-based or a consequence of suspiciousness, itself an important positive symptom. Others may be unable to verbally provide information as a consequence of other symptoms. Negative symptoms such as apathy and alogia may reduce a patient's likelihood of speaking with caregivers at all. Severe conceptual disorganization may make it difficult for patients to clearly verbalize their internal experiences even if they attempt to. Finally, lack of insight may make it difficult for patients to identify symptoms as not reality-based [1;20], and so may lack the ability to judge that they are experiencing hallucinations or delusions.

These factors emphasize the importance of obtaining information from collateral sources such as family members and other more reliable informants. Such collateral information is often recorded in patients' medical records, which are important sources of information in conjunction with direct clinical interviews. The extent to which direct interviews adequately capture the potentially rich information inherent in clinical records, which may include multiple assessments over time from multiple different observers, is unknown. However, clinical records are typically produced by clinicians who may not have the time or interest in capturing all of the information required by structured interviews. Furthermore, although informants are able to provide detailed observations of patient behavior outside of the structured clinical setting, patients may have the same difficulties describing internal experiences to them as to their treating clinicians. Hence, there can be no “gold standard” to judge the presence of a symptom. To our knowledge, there is only one published study of the comparability of symptom assessment using direct interviews vs. other sources of information[10].

In this report, we examine in a study of schizophrenia the relationship between symptom measures rated on the basis of direct structured interviews and those obtained by a review of medical records. We hypothesized that negative symptoms would be more consistent across the two assessment modalities, as they are directly observable, and usually do not require responses from patients to verbal queries.

Materials and Methods

Subjects

The subjects analyzed in this study were derived from the Irish Case-Control Study of Schizophrenia (ICCSS). This sample was collected for genetic studies of schizophrenia, and was an extension of the Irish Study of High-Density Schizophrenia Families (ISHDSF)[13]. The goal of this collection was to have an independent sample in which to rapidly replicate genetic association results from the ISHDSF in the same population. Affected subjects were selected from inpatient and outpatient psychiatric facilities in the Republic of Ireland and Northern Ireland. Subjects were eligible for inclusion if they had a diagnosis of schizophrenia or poor-outcome schizoaffective disorder by DSM-III-R criteria, which was confirmed by a blind expert diagnostic review, and reported all four grandparents as being born in Ireland or the United Kingdom. DSM-III-R criteria were used for this sample to maintain consistency with our earlier collection of multiply-affected families for linkage studies, the Irish Study of High Density Schizophrenia Families (ISHDSF) collection {Kendler, 1996 770 /id}. The final sample of cases in the ICCSS totals 1021 affected individuals. All participating individuals gave appropriate informed consent for inclusion in the study. Information on subject sex was recorded and subsequently verified by molecular assessment of X and Y chromosome markers. All analyses described in this paper included only affected subjects.

Measures

All subjects were assessed using a modified version of the Structured Clinical Interview for DSM-III-R[22]. This was done in order to maintain consistency with our previous studies in this population[12;13]. Symptom ratings were derived directly from the interview itself, which required the interviewer to assess the severity of all symptoms. Family history was assessed using a modified version of the Family History Research Diagnostic Criteria. In addition, casenotes were thoroughly reviewed and rated using our Casenote Rating Scale. This scale, which was organized and selected in a manner parallel to that of the SCID interview, assessed the presence of a range of psychotic and affective symptoms, including positive, negative, and disorganization symptoms. Raters were trained to go through the case records, which were often extensive, and concentrate on the admission mental status and discharge summaries, which typically contained the most detailed data on symptoms and signs. In addition, they also scanned psychiatrists' and nurses' progress notes, as there is sometimes specific information about auditory hallucinations, delusional content, thought disorder or negative symptoms.

Statistical Analysis

We included the following items in the present analysis: delusions of reference, persecutory delusions, thought insertion, thought broadcasting, delusions of passivity, grandiose, somatic, and religious delusions, voices speaking in sentences, voices discussing, voices commenting, visual hallucinations, somatic hallucinations, affective flattening, alogia, avolition, anhedonia, attentional disturbance, inappropriate affect, and thought disorder. All items were recoded in order to standardize scoring, using a range of 0 to 1. For example, an item with possible responses of 1 – 4 in increments of 1 would be recoded using the levels 0, .25, .50, .75, and 1.0. To examine the comparability of symptom ratings based on casenotes vs. direct personal interview, Pearson product-moment correlations between the casenote-derived rating and the interview-derived rating for each clinical feature were calculated using Proc Corr in SAS. Although individual tests were highly correlated, in that most symptoms would be expected to comprise a factor along with other symptoms, we used a conservative Bonferroni correction, correcting for 21 tests.

To examine the factor structure of signs and symptoms of psychotic illness in this sample, we performed factor analysis in MPLUS[17] using VARIMAX rotation. Since there is no “gold standard” for determining whether casenotes or direct interviews are more valid sources of information for symptomatic ratings, we calculated the mean of each to produce a score for entry into factor analysis. We selected the number of factors by examining the scree plot, displayed in Figure 1. An item was included in factor analysis if its maximum loading was at least .4, as well as .2 units greater than the second highest.

Figure 1.

Scree plot in exploratory factor analysis of 21 signs and symptoms of psychotic illness.

Results

Relationship between symptom ratings from interviews vs. chart review

All symptoms were significantly correlated for method of assessment, with six exceptions: thought insertion, thought withdrawal, somatic delusions, voices speaking in sentences, visual hallucinations, and somatic hallucinations. All of these were “positive symptoms”. All Pearson product-moment correlations and their P-values are presented in Table 1. Significant correlations were typically in the moderate range (+.20 to +.55)

Table 1.

Factor Loadings and Inter-Measure Correlations for 21 Signs and Symptoms of Psychotic Illness

	Casenote-Interview Correlation			Factor Loadings
	r	R2	P	Schneiderian	Negative	Positive
Item
Delusions Of Reference	0.1973	0.039	0.0021a	0.375	0.228	0.106
Persecutory Delusions	0.23581	0.056	0.0021	0.26	0.086	0.13
Thought Insertion	0.08938	0.008	>1	0.72	−0.145	0.356
Thought Withdrawal	0.11898	0.014	>1	0.689	−0.11	0.348
Thought Broadcasting	0.23068	0.053	0.0021	0.745	−0.037	0.341
Delusions of Passivity	0.22717	0.052	0.0021	0.439	−0.12	0.562
Grandiose Delusions	0.42334	0.179	0.0021	0.104	0.041	0.682
Somatic Delusions	0.23266	0.054	0.0966	0.121	−0.035	0.591
Religious Delusions	0.54531	0.297	0.0021	0.042	0.069	0.629
Voices Speaking in Sentences	−0.06083	0.004	>1	0.413	0.268	−0.124
Voices Discussing	0.23141	0.054	0.0021	0.434	0.187	−0.049
Voices Commenting	0.28388	0.081	0.0021	0.56	0.045	−0.082
Visual Hallucinations	0.14183	0.020	0.4389	0.055	−0.056	0.536
Somatic Hallucinations	0.01584	0.000	>1	0.045	−0.193	0.776
Affective Flattening	0.34118	0.116	0.0021	0.009	0.873	−0.052
Alogia	0.44667	0.200	0.0021	0.025	0.855	−0.131
Avolition	0.35844	0.128	0.0021	0.06	0.795	−0.02
Anhedonia	0.34893	0.122	0.0021	0.099	0.67	0.002
Attentional Disturbance	0.36165	0.131	<.0001	0.131	0.841	−0.002
Inappropriate Affect	0.33913	0.115	<.0001	0.134	0.716	−0.064
Thought Disorder	0.19727	0.039	<.0001	0.05	0.76	−0.017

Factor Analysis of Combined Data

The scree plot revealed an apparent break between the three and four-factor solutions as seen in Figure 1. The three-factor solution was therefore retained. The three resultant factors were relatively straightforward to identify and included Positive Symptoms (grandiose delusions, somatic delusions, religious delusions, visual hallucinations, and somatic hallucinations), Negative Symptoms (affective flattening, alogia, avolition, anhedonia, attentional disturbance, inappropriate affect, and thought disorder), and Schneiderian Symptoms (thought insertion, thought withdrawal, thought broadcasting, voices discussing, and voices commenting). Four symptoms did not load strongly on any single factor, but had low to moderate loadings on more than one: delusions of reference, persecutory delusions, delusions of passivity, and voices speaking in sentences.

Of the three factors, negative symptoms clearly were most strongly correlated across method of assessment. All negative symptoms were significantly correlated, with a mean r of +.34. This was followed by Positive Symptoms, with two of five symptoms significantly correlated, and mean r of +.27. Least correlated across modes of assessment were Schniederian Symptoms, with three of five symptoms significantly correlated, and a mean r of +.19.

Discussion

Concordance between signs and symptoms assessed using direct interviews vs. casenotes

In this study, we examined the relationship between symptoms measured by trained raters using direct interviews vs. review of casenotes. This is a significant question as both sources of information are not always available in studies of schizophrenia and other psychotic illnesses. However, determining the presence or absence of a symptom is a fundamental first step in making a diagnosis, whether in clinical practice or research settings. It is currently not clear which is a more valid or reliable source of information. We are aware of only one prior paper that has examined this question [10]. That was a smaller study (n=55, compared to n=1021 in the present study) in which patient self-report, informant report, and consensus assessments were compared to each other. However, these were not necessarily independent, as consensus assessments were made on the basis of all sources of information, including patient and informant report. In that study, Ho et al. reported that negative symptom assessments were more likely to be concordant than those of other symptoms. They also reported that positive symptoms tended to be under-reported by patients.

We found that for most symptoms, there was reasonable concordance between the two assessment methods. This was strongest for negative symptoms. These are, for the most part, directly observable, so do not require a patient to divulge thought content to an interviewer. They are also more stable over time, especially later in the course of illness[15]. Therefore, they are less likely to change between the time at which casenotes were recorded and the time at which the interview was performed.

Six symptoms were not significantly correlated across assessment method. Two of these are classic Schneiderian delusions, both of which loaded on our Schneiderian factor: “thought insertion” and “thought withdrawal”. Voices speaking sentences, an auditory hallucination which did not load strongly on any single factor, was also not significantly correlated between interview and case-record ratings. Finally, “somatic hallucinations”, which loaded on our Positive factor, was also not correlated. Each of these symptoms is a “positive” symptom, reflecting delusional or hallucinatory experiences. As such, they require the patient to divulge their presence to the interviewer. Patients therefore need to feel a modicum of comfort with the interviewer, regardless of how skilled or experienced, in order to provide this information. They are therefore probably more likely to underreport such symptoms. Futhermore, they may also forget symptoms previously experienced – a common problem in survey research. Lastly, more specific symptoms may be less frequently probed by clinicians. For example, somatic hallucinations are not required for any diagnostic criteria, and therefore less likely to be asked about.

Schneiderian, or “first-rank” symptoms were originally described as such, in part because of the relative ease of their identification, which was thought to make them more reliable[21]. It is somewhat surprising, therefore, that chart review and direct interview ratings are less concordant for them than they are for other symptoms. Thought insertion and withdrawal are often probed using the queries “do people put thoughts in your head?” and “do people take thoughts out of your head?” respectively. It is possible that this phraseology does not adequately capture the experience of these symptoms to many patients. Verbal descriptions of mental phenomena may be particularly likely to cause confusion[23]. Furthermore, these symptoms as concepts are relatively abstract, and difficulty with abstract thinking is itself a considerable deficit in schizophrenia[3;9]. They may also not have been assessed by clinicians at hospitalization and therefore not recorded if they were absent.

Overall, our results suggest that single sources of information may have a significant risk of missing important positive symptoms such as Schneiderian delusions and hallucinations, as well as less prevalent positive symptoms such as somatic hallucinations. Although it is quite possible that patients underreport such symptoms in structured interviews, it is difficult to judge this as routine clinical records are only likely to even mention such symptoms if they in fact exist. This significant disparity in missing data probably presents a bias that makes a direct comparison of the positive and negative predictive values of structured interviews and casenote items quite difficult. The importance of combining information from chart reviews and informant interviews, as well as casenotes is therefore highlighted. Negative symptoms, on the other hand, are sensitively assessed in structured interviews, as evidenced by the higher correlations with casenote ratings.

Factor analysis of signs and symptoms

Factor analysis of symptomatic data was done using the mean of the interview-based rating and the casenote-based rating for all symptoms. We did not weight either, as there is no “gold standard”. This is because patient factors as well as rater factors may influence the under-reporting (or even over-reporting) of symptoms. Some symptoms may be less likely to be probed by a primary clinician writing casenotes, especially one without additional training in structured interviewing.

Exploratory factor analysis resulted in a three-factor solution. As with our previous factor analyses in the Irish population, a Schneiderian factor was distinguishable from a more general positive symptoms factor[16]. This has been reported in other samples as well[4–6;14]. Two classic positive symptoms common in schizophrenia did not load robustly onto any one factor: persecutory delusions and delusions of reference. However, they were highly significantly correlated across assessment method.

Negative symptoms had a considerably higher mean loading (.78) than the other two factors. This suggests that these symptoms are more likely to represent a stable underlying trait. Negative symptoms in fact have been reported to have greater longitudinal stability[15]. Two candidate gene studies (of dysbindin) suggest that they may be more genetically mediated than other symptoms[7;8]. One of the limitations of our assessments of negative symptoms is that we were unable, in our design, to distinguish primary from secondary negative symptoms. Secondary negative symptoms may result from depression, medication-induced parkinsonism, or chronicity[11;19] and may not be primary expressions of illness. Overall, our results support the usefulness of combining information from both direct interviews and medical records in the assessment of schizophrenic symptoms and in making diagnoses.