Validity of Chinese Version of the Composite International Diagnostic Interview-3.0 in Psychiatric Settings

Abstract

Background:

The Composite International Diagnostic Interview-3.0 (CIDI-3.0) is a fully structured lay-administered diagnostic interview for the assessment of mental disorders according to ICD-10 and Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV) criteria. The aim of the study was to investigate the concurrent validity of the Chinese CIDI in diagnosing mental disorders in psychiatric settings.

Methods:

We recruited 208 participants, of whom 148 were patients from two psychiatric hospitals and 60 healthy people from communities. These participants were administered with CIDI by six trained lay interviewers and the Structured Clinical Interview for DSM-IV Axis I Disorders (SCID-I, gold standard) by two psychiatrists. Agreement between CIDI and SCID-I was assessed with sensitivity, specificity, positive predictive value and negative predictive value. Individual-level CIDI-SCID diagnostic concordance was evaluated using the area under the receiver operator characteristic curve and Cohen's K.

Results:

Substantial to excellent CIDI to SCID concordance was found for any substance use disorder (area under the receiver operator characteristic curve [AUC] = 0.926), any anxiety disorder (AUC = 0.807) and any mood disorder (AUC = 0.806). The concordance between the CIDI and the SCID for psychotic and eating disorders is moderate. However, for individual mental disorders, the CIDI-SCID concordance for bipolar disorders (AUC = 0.55) and anorexia nervosa (AUC = 0.50) was insufficient.

Conclusions:

Overall, the Chinese version of CIDI-3.0 has acceptable validity in diagnosing the substance use disorder, anxiety disorder and mood disorder among Chinese adult population. However, we should be cautious when using it for bipolar disorders and anorexia nervosa.

INTRODUCTION

The Composite International Diagnostic Interview (CIDI)[1] is a comprehensive and fully standardized diagnostic interview, primarily for use in epidemiological surveys and mental health research. This instrument has not only been used worldwide in numerous epidemiological studies, but it was also psychometrically explored and reviewed in a considerable number of studies. Although standardized diagnostic interviews such as CIDI[2] have been shown to be quite reliable in various cultural settings and populations, very few studies have examined the clinical validity of these diagnostic interviews in Chinese population.[3,4,5] Therefore, there are still considerable concerns about the cross-cultural validity of diagnoses generated from these instruments when compared with careful and well-documented diagnoses assigned by clinical psychiatrists. Acceptable, but less promising estimates have also been obtained for some diagnoses in a few validity studies with quite variable designs in clinical samples and considerably less frequently in general population samples. Consistently poor validity estimates were obtained for several types of mental disorders,[3,6] whereas anxiety[7,8] and depressive disorders were demonstrated across studies to have good validity estimates, with kappa values of 0.5 or above.[3,4] Given the widespread use of the CIDI in Chinese epidemiological studies, there is an urgent need to determine the degree of concordance and the types of discrepancies between CIDI and clinical diagnoses made by mental health professionals. This paper presents the data from a systematic comparison of the translated CIDI version, known as the CIDI-3.0, in an unselected clinical sample of patients with mental disorders, by comparing CIDI/Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV) diagnoses with those assigned by the treating clinical psychiatrists. The aim of this study was to investigate the validity of the CIDI in relation to the Structured Clinical Interview for DSM-IV Axis I Disorders (SCID-I), a “gold standard” diagnostic tool that is administered by mental health professionals.[9,10]

METHODS

Ethnics

Every subject provided written informed consent after receiving a complete description of the study. The protocol was approved by the Institutional Reviewed Board of Institution of Mental Health, Peking University.

Study design

Translation of the Composite International Diagnostic Interview

There are several version of CIDI in China. CIDI-3.0 is the newest version.

With the original authors’ permission, the original English CIDI-3.0 was translated into Chinese. The standard procedure of back-translation was followed for the cross-cultural adaptation of an original English psychometric instrument. Namely, The CIDI-3.0 was translated into Chinese and back-translated. A team of experts completed the initial translation and a separate team then carried out an independent back-translation to confirm preservation of the meaning of the original English version. An expert panel of four academic psychiatrists evaluated its content validity, tested it with Chinese patients, and revised it to ensure its questions easily understandable and practically answerable by lay people.

Sample

In the pilot study of the validity of CIDI, the sensitivity ranged from 65.6% (mood disorders) to 87.0% (substance use disorder) and the specificity ranged from 83.3% (anxiety disorders) to 98.3% (substance use disorders). We used the following formula to calculate the sample size.[11]

(α = 0.05, δ = 0.15).

To achieve adequate representation of the major mental disorders and sufficient nonpatient controls, the study was designed to include at least 71 cases with major depressive disorder, 30 with mania, 40 with anxiety disorder, 20 with an alcohol dependence disorder, 20 with eating disorder, and 60 healthy community-residing adults.

All patients were consecutively recruited from two participating psychiatric institutions between September 2006 and February 2008. Exclusion criteria were age under 16 and having difficulties in completing CIDI interview because of functional and cognitive disabilities including hearing and reading impairments, dementia, mental retardation, serious medical illnesses, marked and persisting hallucinations, thought disorders and disorganized speech. A total of 194 patients, 46 were excluded, either because they satisfied the exclusion criteria or because they did not complete the interview, leaving 148 patients completed the study. The healthy controls were selected from communities nearby the two participating psychiatric institutions, and they voluntarily participated in the study. The overall response rate was 78%.

Interviewers

Our interviewers included two psychiatrists and six lay people. The two psychiatrists were fully licensed psychiatrists with at least 3 years of clinical experiences in psychiatry and had received an intensive training in the use of SCID-I. The six lay interviewers received 2-week training in the use of CIDI that was provide by the Beijing CIDI Training and Resource Center and completed at least 20 interviews under the supervision of experienced CIDI trainers.

Design and the clinical assessment procedure

Clinicians’ diagnoses

The two psychiatrists interviewed all subjects with SCID-I according to the definitions and criteria of DSM-IV, blinded to the results of CIDI interviews. Both were informed about aim and procedures of the study and the necessity of a complete and detailed documentation of the psychopathology, course of illness and final diagnostic decision for each participating patient; they were also encouraged to consult the DSM-IV manual[9] and use all information available (e.g., history of psychiatric illnesses and treatment) whenever necessary.

All probable and definite lifetime diagnoses recorded in SCID-I interview charts were used as the gold standard in the final analyses.

Discrepancy and case-by-case review

After analyzing the diagnostic concordance, one psychiatrist reviewed interview records of all discrepant cases, in an attempt to identify why either the CIDI did not pick up significant psychopathology or why the psychiatrist did not confirm the CIDI diagnoses.

Statistics

The validity of CIDI against SCID-I was assessed using the sensitivity (SN), specificity (SP), positive predictive value (PPV), and negative predictive value (NPV). The raw data were also cross tabulated.

Individual-level CIDI-SCID diagnostic concordance was evaluated using two different descriptive measures: The area under the receiver operator characteristic curve (AUC)[12] and Cohen's K.[13] Although K is the most widely used measure of concordance in validity studies of mental disorders, it has been criticized for its dependency on prevalence.[14,15] We should consider that the following implication is important. Various populations are different in prevalence when the populations do not differ in SN or SP. As SN and SP are considered to be fundamental parameters, this means that the comparisons across different populations cannot be used to evaluate the cross-population performance of a test. The odds ratio (OR) meets this requirement, as OR = (SN × SP)/([1 − SN] × [1 − SP]).[16] However, the upper end of the OR is unbounded, making it difficult to evaluate the extent to which CIDI diagnoses are consistent with clinical diagnoses. Yules Q has been proposed as an alternative measure to resolve this problem,[17] as Q is a bounded transformation of OR (Q = [ OR − 1]/[OR + 1]) that ranges between −1 and +1. Q can be interpreted as the difference in the probabilities of a clinical case and a clinical noncase that differ in their classification on the CIDI. The difficulty with Q is that “tied pairs” (clinical cases and noncases that have the same CIDI classification) are excluded, which means that Q does not tell about actual prediction accuracy. The AUC is a measure that resolves this problem, as AUC can be interpreted as the probability that a randomly selected clinical case will score higher on the CIDI than a randomly selected noncase.

RESULTS

Clinical and demographic characteristics of study sample

The mean age of all patient participants was 33.5 ± 14.2 years, 61.5% were females, and the mean educational year of the respondents was 10.1 years. The healthy controls and patients differed not significantly in terms of education, age group, and gender. Within the subjects with mental disorders, women were somewhat overrepresented in the anxiety depression and eating groups and underrepresented in the substance use disorder group. Inpatients represented 86% of the subjects with the mental disorder [Table 1]. Table 2 shows the frequency of DSM-IV lifetime diagnoses identified in the SCID-I and CIDI interviews. In total, SCID-I interviews recognized 46 cases with major depressive disorder, 20 with bipolar disorders, 59 with anxiety disorders, 20 with eating disorders, 33 with psychotic disorders, 20 with alcohol or drug disorders.

Table 1.

Demographic characteristics of patients with mental disorders and healthy people, n (%)

Variables	Patients	Healthy people
Age (years)
15–34	85 (57.4)	34 (56.7)
35–49	40 (27.0)	8 (13.3)
≥50	23 (15.6)	18 (30.0)
Gender
Male	57 (38.5)	28 (46.7)
Female	91 (61.5)	32 (53.3)
Education (years)
0–6	3 (2.1)	2 (3.3)
7–9	12 (8.3)	12 (20.0)
10–12	66 (45.1)	20 (33.3)
≥12	67 (45.8)	26 (43.3)

Table 2.

Frequency of DSM-IV lifetime diagnoses assigned by the clinician using SCID-P and by the CIDI 3.0 DSM-IV algorithms

DSM-IV diagnoses	CIDI diagnose (lifetime)	SCID diagnose (lifetime)
Mood disorder
Major depression	32	46
Bipolar disorder	2	20
Mood disorder	42	66
Anxiety disorder
General anxiety disorder	17	29
OCD	24	29
Other anxiety disorder†	27	41
Any anxiety disorder	46	59
Substance disorder	20	23
Psychosis screen	21	33*
Eating disorder
Anorexia nervosa	0	20
Bulimia nervosa	4	10
Binge eating disorder	4	6
Any eating disorder	8	20
Total diagnoses	151	223
1 diagnosis	30	50
2 diagnoses	25	38
3 diagnoses	17	23
4 diagnoses	5	7

*33 subjects was diagnosis as psychosis by clinicians; ^†Other anxiety disorders: Specific phobia disorder, social phobia, agoraphobia, panic disorder, PTSD. DSM-IV: Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition; SCID: Structured Clinical Interview for DSM-IV Axis I Disorders; CIDI-3.0: Composite International Diagnostic Interview-3.0; OCD: Obsessive and compulsive disorder.

Lifetime individual-level concordance between the Composite International Diagnostic Interview-3.0 and the Structured Clinical Interview for DSM-IV Axis I Disorder

Table 3 provides crosstab number of CIDI-SCID diagnose, AUC, OR, Q, SNs, SPs, PPVs, and NPVs for each diagnosis generated by the CIDI. The separate disorder-specific CIDI-SCID comparisons of lifetime prevalence were made for major depression, bipolar disorder, general anxiety disorder, obsessive and compulsive disorder (OCD), other anxiety disorders (including panic disorder, phobias, posttraumatic stress disorder [PTSD]), alcohol abuse with or without dependence, anorexia nervosa, bulimia nervosa, binge eating disorder, and psychosis. The individual-level CIDI-SCID lifetime concordance for DSM-IV diagnosis was substantial to excellent for three disorders: Substance use disorders (AUC = 0.926), anxiety disorders (AUC = 0.807), and mood disorders (AUC = 0.806). In detail, the concordance was moderate (AUC: 0.7–0.8) for general anxiety disorder, OCD, other anxiety disorder, and psychosis. The concordance was almost perfect (AUC ≥ 0.9) for alcohol and substance use disorders and major depression, but fairly inadequate (AUC in the range: 0.6–0.7) for the remaining disorders (bulimia nervosa). The concordance between the CIDI and the SCID for two individual disorders, bipolar disorders (AUC = 0.55) and anorexia nervosa (AUC = 0.50) was poor with very high SPs and very poor SNs. The majority of SCID cases were detected by the CIDI (SN) for major depression (69.6%), general anxiety disorders (58.6%), OCD (82.8%), substance dependence (87.0%), and psychosis screen (63.6%). The vast majority of CIDI cases, in comparison, were confirmed by the SCID (PPV), including 83.3% (85.0–100.0%) with anxiety disorder, 96.7–100% with mood disorder, and 98.3% with substance disorder. The SN values were particularly low for the diagnoses of bipolar disorder and eating disorder (0.0–40.0%).

Table 3.

Consistency of lifetime DSM-IV CIDI and SCID diagnoses with cross tabulation of findings (n = 148)

Diagnoses			SCID-P		AUC	OR	Q	SN%	SE%	SP%	SE%	PPV%	SE%	NPV%	SE%
			+   −
	CIDI	+	TN│FN
		−	FP│TP
Mood disorder
Major depression			32│2		83.1	16.2	0.9	69.6	7	96.7	2	94	4	81	5
			14│58
Bipolar disorder			2│54		55.0	15.4	0.9	10.0	7	100	0	100	0	77	5
			18│6
Mood disorder			42│2		80.6	14.4	0.9	63.6	6	96.7	2	95	3	72	5
			24│58
Anxiety disorder
General anxiety disorder			17│0		79.3	20.8	0.9	58.6	9	100	0	100	0	83	4
			12│60
OCD			24│2		72.4	30.7	0.9	82.8	7	96.7	2	92	5	92	3
			5│58
Other anxiety disorder*			27│9		74.5	10.9	0.83	65.9	7	85.0	5	61	10	89	4
			14│51
Any anxiety disorder			46│10		80.7	18.7	0.9	78.0	7	83.3	5	82	5	56	3
			13│50
Substance disorder			20│1		92.6	31.7	0.9	87.0	7	98.3	2	95	5	95	3
			3│59
Psychosis screen			21│5		77.7	16.4	0.9	63.6	8	91.7	4	81	8	82	5
			12│55
Eating disorder
Anorexia nervosa			0│0		50.0	15.4	0.9	0	0	100	0	0	0	77	5
			20│60
Bulimia nervosa			4│0		66.3	26.0	0.9	40.0	15	100	0	100	0	78	3
			6│60
Binge eating disorder			4│0		83.3	39.0	1.0	66.7	19	100	0	100	0	78	2
			2│60
Any eating disorder			8│0		70.0	20.8	0.9	40.0	11	100	0	100	0	83	4
			12│60

*Other anxiety disorders: Specific phobia disorder, social phobia, agoraphobia, panic disorder, PTSD. TN: True negatives; FN: False-negatives; FP: False-positives; TP: True-positives; OR: Odds ratio; AUC: Area under the ROC curve; ROC: Receiver operating characteristic; SN: Sensitivity; SP: Specificity; PPV: Positive predictive value; NPV: Negative predictive value. Q: Yules Q; SE: Standard error; PTSD: Posttraumatic stress disorder; OCD: Obsessive and compulsive disorder; DSM-IV: Diagnostic and Statistical Manual of Mental Disorders; Fourth Edition; SCID: Structured Clinical Interview for DSM-IV Axis I Disorders; CIDI: Composite International Diagnostic Interview.

DISCUSSION

The present study estimated the concordance of diagnoses based on the CIDI-3.0[1] with diagnoses based on SCID-I interviews. Overall, our data suggested that the CIDI-3.0 succeeds in validly eliciting diagnoses used in making DSM-IV diagnoses.

We found that the SNs is low for the bipolar disorders and eating disorders. There are several possible factors that may have contributed to the low concordance figures for the two disorders in this study. Similar results were found in a community sample.[18] One major reason is the difference between the information-collecting methods of the CIDI and SCID.[1,9] The patients with a diagnosis of bipolar disorder might refuse to confirm about some of their symptom. Some questions were often denied by the patients (e.g., SC24. Have you ever had a period lasting 4 days or longer when you became so happy or excited that you either got into trouble, people worried about you, or a doctor said you were manic?). Patients with anorexia nervosa were not more likely to endorse to some questions (EA1. This part of the interview is about problems you might have had either with eating or with your weight. Was a time in your life when you had a great deal of concern about or strongly feared being too fat or overweight?”). Because of the cultural differences in presentation of symptoms, some Chinese patients do not admit the symptom of “strongly feared being too fat or overweight,” instead, they often complain about their poor appetite and dyspepsia.

Several limitations should be mentioned in the findings. The study was used a clinical sample with the advantage of complete and comprehensive longitudinal symptom and diagnostic documentation. However, it is necessary to be cautious about generalizing the findings to nonclinical general population samples, for which one might expect more close-to-threshold conditions. Another possibility is that in the current study, the clinical samples are known to have a considerable degree of comorbidity. This has the advantage of allowing for more disorder-specific comparisons with small samples. However, this approach might also inflate the concordance estimations, due to the inclusions of the same subject in testing several diagnoses. Similar risk existed in our choice of not using the optional DSM-IV/CIDI-3.0 diagnostic hierarchy rules because it would considerably reduce the number of CIDI-3.0 diagnoses assigned. Second, both PPV and NPV depend on prevalence. Although SN and SP do not depend on prevalence, they still depend on the case mix. Finally, the small sample size did not allow for more extensive subgroup comparisons by characteristics of interviewers and subjects, but we could expect, based on similar previous studies, that the CIDI-3.0 is quite robust in empirical validity.[3,18,19,20,21] Further, we focused on lifetime mental disorders, therefore, the validity of CIDI in diagnosing 12-month or 1-month mental disorders needed to be examined in future studies.

In conclusion, the validity study has shown that CIDI-SCID concordance in DSM-IV diagnoses is generally good, however, for specific mental disorders, such as bipolar and eating disorders, we should be cautious in using CIDI.

Financial support and sponsorship

This work was supported by grants from Special Research Project of Ministry of Health (No. 201202022), the National 11^th 5-year Plan of Ministry of Science and Technology of China (No. 2007BAI17B01), and National Key Technology Research and Development Program of the Ministry of Science and Technology of China (No. 2015BAI13B01).

Conflicts of interest

There are no conflicts of interest.