Korean Version of the Delirium Rating Scale-Revised-98: Reliability and Validity

Abstract

Objective

The aims of the present study were 1) to standardize the validity and reliability of the Korean version of Delirium Rating Scale-Revised-98 (DRS-R98-K) and 2) to establish the optimum cut-off value, sensitivity, and specificity for discriminating delirium from other non-delirious psychiatric conditions.

Methods

Using DSM-IV criteria, 157 subjects (69 delirium, 29 dementia, 32 schizophrenia, and 27 other psychiatric patients) were enrolled. Subjects were evaluated using DRS-R98-K, DRS-K, Mini-Mental State Examination (MMSE-K), and Clinical Global Impression-Severity (CGI-S) scale.

Results

DRS-R98-K total and severity scores showed high correlations with DRS-K. They were significantly different across all groups (p=0.000). However, neither MMSE-K nor CGI-S distinguished delirium from dementia. All DRS-R98-K diagnostic items (#14-16) and items #1 and 2 significantly discriminated delirium from dementia. Cronbach's alpha coefficient revealed high internal consistency for DRS-R98-K total (r=0.91) and severity (r=0.89) scales. Interrater reliability (ICC between 0.96 and 1) was very high. Using receiver operating characteristic analysis, the area under the curve of DRS-R98-K total score was 0.948 between the delirium group and all other groups and 0.873 between the delirium and dementia groups. The best cut-off scores in DRS-R98-K total score were 18.5 and 19.5 between the delirium and the other three groups and 20.5 between the delirium and dementia groups.

Conclusion

We demonstrated that DRS-R98-K is a valid and reliable instrument for assessing delirium severity and diagnosis and discriminating delirium from dementia and other psychiatric disorders in Korean patients.

INTRODUCTION

Delirium is an acute psychiatric syndrome characterized with impairment of consciousness with a prominent disturbance of attention and deficits of other cognitive domains, perception, language, thought, motor behavior, sleep-wake cycle, and affective control. Delirium is a common problem among hospitalized medically ill patients, affecting roughly 25% of this population.¹ Delirium is associated with significant morbidity and increased mortality (between 11% and 65% during hospital admission), and length of hospitalization as compared to non-delirious patients.² Because it is frequently under- or misdiagnosed and associated with poor outcomes, a precise and reliable instrument for the diagnosis and repeated evaluation of delirium is needed.

The Delirium Rating Scale (DRS) objectively measures symptoms severity of delirium.³ The DRS is widely used and has been translated into eleven languages other than English (French, Chinese, Dutch, Spanish, Italian, Swedish, Japanese, German, Thai, Indian, and Korean).⁴ The DRS was validated to differentiate delirium from other psychiatric disorders, and compris-ed a broader range of symptoms of delirium than cognitive tests such as the Mini-Mental Status Examination (MMSE). Many studies have demonstrated that the DRS is a useful tool for assessment of delirium, showing high validity and reliability.^5,6 The Korean version of DRS (DRS-K) has also been demonstrated as a valid tool which significantly discriminates delirium from dementia and schizophrenia, with sensitivity and specificity similar to the original DRS. It is reliable with high internal correlation and interrater reliability.⁷

Limitations of the DRS include: certain items (e.g., temporal onset) are difficult to rate for repeated serial assessments during an episode of delirium; hypoactivity and hyperactivity are measured in the same item; and only one item rates all aspects of cognitive functions because the DRS was intended to be used in conjunction with bedside cognitive tests. Further, the DRS lacks items for language and thought process abnormalities. As a result, the Delirium Rating Scale-Revised-98 (DRS-R98) was created to compensate for the shortcomings of the DRS, especially for use in phenomenological and longitudinal research.⁴ The DRS-R98 has been translated into Dutch,⁸ Spanish,⁹ German, Portuguese,¹⁰ Japanese,¹¹ Danish, Hebrew, Lithuanian, Greek, Norwegian, both traditional¹² and modern Chinese, and Korean.¹³

The objectives of the present study were to evaluate the validity and reliability of the Korean version of Delirium Rating Scale-Revised-98 (DRS-R98-K) and to establish the optimal cut-off values, sensitivity, and specificity for discriminating delirium from other non-delirious psychiatric conditions.

METHODS

Subjects

All delirious patients were recruited from medical wards of two university hospitals. Patients for dementia, schizophrenia, and other psychiatric disorders groups were recruited from inpatient psychiatric and general medical units or outpatient facilities. Diagnoses were made according to the diagnostic criteria of Diagnostic and Statistical Manual of Mental Disorders 4th edition (DSM-IV).¹⁴ The consultation service psychiatrist referred patients diagnosed with delirium to the research team. Control patients were referred to the research team from their treating outpatient or inpatient service psychiatrist. Dementia and schizophrenia patients were chosen as control groups because of overlapping symptoms with delirium.^2,15 Other psychiatric disorder patients did not have significant cognitive dysfunction or psychotic symptoms and typically had mild to moderate depression or anxiety disorder. Following explanation of the procedures and rationale for the study, verbal consent was obtained from the patient or the proxy decision-maker. Because of the non-invasive nature of the study, ethics committee approved the use of verbal consent. Patients excluded if they refused evaluation or were on mechanical ventilation. This study was approved by the Institutional Review Board and Ethics Committee of the Kyungpook National University and Hospital.

DRS-R98 Translation

The original English version of the DRS-R98 scale was translated into Korean with permission from the copyright owner. This translation was performed by a native Korean who is a professor in the department of English literature who lived in the USA. for several years. Then the translation was reviewed and revised by three experts in psychiatry and delirium. This revised version was back-translated by a bilingual person who was fluent in both languages. The back-translated version was compared with the original version by the three above-mentioned experts. This Korean version was also reviewed and revised by a professor in the department of Korean to ensure that it was sensitive to the Korean culture and environment. Then, four expert psychiatrists who had vast experience with the diagnosis and treatment of delirium and were not involved with this research reviewed this Korean version from a user's point of view. Finally, taking into consideration the experts' opinions, the authors produced the final Korean version.

Measures

Korean version of Delirium Rating Scale-Revised-98

The DRS-R98 proved to be a valid and reliable instrument for the assessment of delirium severity and distinguish patients with delirium from non-delirious patients including dementia.⁴ It was originally developed for use by psychiatrically trained clinicians. Similarly, the DRS-R98-K is intended for these clinicians. The scale is composed of two sections. The first section is the severity scale, which consists of 13 items that are scored from 0 to 3 according to a Likert scale that is anchored by descriptions of phe-nomenology (Table 3). The second section consists of three diagnostic items that are scored from 0 to 2 or 3. These items include temporal onset of symptoms, fluctuation of symptom severity and physical disorder. The total scale is comprised of 16 items. Copies of this scale can be obtained from Dr. Trzepacz.

Table 3.

Alpha coefficients for DRS-R98-K total and severity scales when each item is removed from the scale

DRS-R98-K: Korean version of Delirium Rating Scale-Revised-98, N/A: not applicable

Korean version of Delirium Rating Scale

The DRS-K was performed in all groups in order to compare with DRS-98R-K for validity. The scale is commonly used for the evaluation of delirium in Korea.⁷ The DRS-K has shown significant differences in the total score between delirium, dementia, and schizophrenia patients as well as high sensitivity and specificity. Cronbach's alpha coefficient of this scale was 0.88 and interrater reliability was 0.98 (p<0.0001). The original English version correlated highly with the DRS-R98 English version.⁴

Korean version of Mini-Mental State Examination and Clinical Global Impression-Severity scale

The Korean version of Mini-Mental State Examination (MMSE-K), which was standardized by Park et al,¹⁶ is a commonly used instrument for the bedside evaluation of cognitive dysfunction in Korea. The CGI-S¹⁷ is a commonly used global rating of disease severity based on a clinician's total experience in evaluating patients with a particular disorder and is rated as a Likert scale from 1 (normal, not at all ill) to 7 (among the most extremely ill patients) points. The CGI-S was used for the comparison of disease severity across groups.

Procedures

All subjects were rated by one of the two primary raters (JR, JL) who are extensive experienced psychiatrists with administering these scales. Primary raters were each compared to another trained psychiatrist (YL) who independently scored a subgroup of the delirium and control patients (i.e., blinded to each other's ratings) to calculate interrater reliability between pairs of raters (JR-YL, JL-YL). Some of the delirious subjects (n=41) were retested by the same rater after the delirious symptoms resolved to determine whether the DRS-R98-K was sensitive to change in delirium status per DSM-IV criteria. All available information including medical records, nursing records, and interviews with care-givers or family members were reviewed to obtain data for proper rating. The rating of DRS-R98-K was based on a 24-hour period due to the fluctuating nature of delirium symptoms.

Data analysis

Data were analysed using SPSS software version 14. All rating scale scores by the primary raters (JR, JL) were used for statistical analysis. The scores of the secondary rater (YL) were only used for interrater reliability analysis. All demographic data and scale scores are expressed in means and standard deviations. Statistical significance was set as alpha level 0.05. Chi-square tests were used for the comparison of the categorical variables.

Concurrent validity was established by calculating Spearman's correlation coefficients between the DRS-R98-K and other tests (DRS-K, MMSE-K, and CGI-S) in delirium patients as well as for the whole sample. Predictive validity was assessed by comparing total and severity scale scores of DRS-R98-K with the scores of DRS-K, MMSE-K, and CGI-S among groups with post hoc (LSD) pairwise comparisons to determine where the differences existed. Boxplots were graphed for the distribution of scale scores, quartiles, and median scores with Kruskal-Wallis test for comparison.

Normality test was done for all variables using the Shapiro-Wilk p value. One-way ANOVA was used to compare the scale score between groups when the data was normally distributed (DRS-K, Shapiro-Wilk p=0.056-0.650). For the variables that were not normally distributed, Kruskal-Wallis test was used (DRS-R98-K total, Shapiro-Wilk p=0.000-0.514; DRS-R98-K severity, Shapiro-Wilk p=0.000-0.477; CGI-S, Shapiro-Wilk p=0.000-0.041; MMSE-K, Shapiro-Wilk p=0.002-0.123).

Clinically, delirium and dementia are difficult to differentiate and commonly co-morbid.^18,19 It is well known that mortality risk is increased in the comorbidity.²⁰ Therefore, discriminant analysis was conducted to determine which items of the DRS-R98-K could be helpful for differential diagnosis between delirium and dementia and then, to determine how well the current classification functions of discriminant analysis using these items of DRS-R98-K could predict clinical group classification between the two groups by DSM-IV. Optimal cut-off scores for the DRS-R98-K were determined from the receiver operating characteristic (ROC) curve and sensitivity and specificity.

Internal consistency of DRS-R98-K was assessed using Cr-onbach's alpha coefficient, including for after each item had been deleted. Interrater reliability was calculated using the intraclass correlation (ICC) between pairs of raters. The DRS-R98-K scor-es before and after treatment were compared by paired t-test to evaluate change.

RESULTS

Subjects

All 157 subjects were enrolled and comprised four groups: 69 delirium, 29 dementia, 32 schizophrenia, and 27 other psychiatric patients (e.g., depression). Half (n=75, 47.8%) of all patients in the study were men, and the mean age was 59.9 years (range 16-91; SD=15.7). Demographic data of each group are summarized in Table 1.

Table 1.

Demographic and rating scale scores in four diagnostic groups, mean±SD (range)

^a17 Alzheimer's, 7 vascular, 2 mixed, 3 dementia not otherwise specified (NOS), ^b18 depressive disorder, 1 bipolar disorder, 2 panic disorder, 2 generalized anxiety disorder, 2 adjustment disorder, 1 undifferentiated somatoform disorder, 1 alcohol dependence, ^†MMSE-K was assessed in 65 of the 69 delirium patients because of uncooperativeness, ^‡CGI-S score was calculated in 66 of the 69 delirium patients because of incomplete data. ^*Kruskal-Wallis Test, p=0.000 among groups, ^**One-way analysis of variance, p=0.000 among groups, ^***Kruskal-Wallis Test, p=0.009 among groups. DRS-R98-K: Korean version of Delirium Rating Scale-Revised-98, DRS-K: Korean version of Delirium Rating Scale, MMSE-K: Korean version of Mini Mental Status Examination, CGI-S: Clinical Global Impression-Severity

There was a significant difference in sex of the groups (χ²=28.09, df=3, p<0.0001). The majority of patients were female in each group, except delirium. However, there were no significant differences in DRS-K score and DRS-R98-K severity and total scores between sexes, except in the other psychiatric group where scores are well into the non-delirious range (The p values of DRS-R98-K total, DRS-R98-K severity, and DRS-K scores in each group, respectively; In delirium group, p=0.208, p=0.246, p=0.194; In dementia group, p=0.631, p=0.487, p=0.805; In schizophrenia group, p=0.169, p=0.185, p=0.276; In other group, p=0.007, p=0.004, p=0.003). There was also a significant difference in mean age of the groups where dementia patients were significantly older than the other groups and schizophrenia patients were the youngest (F=27.26, df=3,153, p<0.0001), though the age ranges were similarly broad in each group. The age of the delirium group was also significantly different from the other three groups, while there was no difference between other psychiatric and schizophrenia groups. Age was not considered to influence the rating scores because there was no correlation between the rating score and age for any of the scales. In addition, analysis of covariance (ANCOVA) was conducted to evaluate the possibility of an age effect on the comparison of rating scores between groups. Age was not a significant covariate for the rating scale scores (DRS-R98-K total score, p=0.379; DRS-R98-K severity score, p=0.301; DRS-K, p=0.433); therefore, it is not considered to have a large effect on group comparisons of the rating scores.

Concurrent validity

In the delirium group, the strongest correlations were between the DRS-R98-K total and severity scales and the DRS-K scale (r=0.831, r=0.811, respectively). Correlations between the DRS-R98-K total and severity and the MMSE-K scales were strong but in the moderate range (r=-0.657, r=-0.672, respectively) while with the CGI-S was also moderate but the lowest of the scale comparisons (r=0.532, r=0.552, respectively). All were statistically significant (p=0.000). These results demonstrate that the DRS-R98-K rating scale is a validated instrument for delirium. In the all-subjects comparison, all correlations were also significant. Strengths of correlation paralleled the pattern for the delirium only correlations.

Predictive validity

Mean scores for all rating scales were significantly different across groups at the p=0.000 or p=0.009 level (Table 1). Post hoc analysis demonstrated that DRS-R98-K total score was significantly higher in the delirium group compared to all other groups. The dementia group was also significantly different from the other three groups, though mean rating scale scores did not differ between the schizophrenia and other psychiatric group. The DRS-R98-K severity score was also significantly different among groups (p<0.001) where the delirium group had the highest mean score, followed by dementia, schizophrenia, and other psychiatric groups. DRS-K scores were significantly different across groups (p<0.001) where delirium group was significantly higher than in any other group, where the dementia group was significantly higher than the schizophrenia and other psychiatric groups that were similar to each other.

MMSE-K score was significantly different among groups, with the lowest scores recorded in delirium and dementia groups (no significant differences between them), whereas mean scores were in the mildly impaired/borderline normal range for the schizophrenia and other psychiatric groups. Mean CGI-S ratings, used to compare illness severity, scored in the "4" point range (moderately ill) for all groups suggesting good clinical comparability despite statistical significant differences among groups, largely accounted for by the delirium group. Nonetheless, the CGI-S score was used as a covariate to explore the possibility of impact of overall disease severity on DRS-R98-K differences among groups using the ANCOVA. CGI-S was a statistically significant covariate because it significantly correlated with all scale scores but there was no interaction between it and the other scores. After adjustment of the covariate, rating scale scores were again compared. DRS-R98-K total and severity scores were statistically significantly different among groups, and pairwise comparisons revealed significant differences for both among all groups. Findings were essentially the same on all scales for group differences as described above.

Boxplots were graphed to show the scale score distribution with medians and quartiles (Figure 1). Medians and middle quartiles of DRS-R98-K total score of the delirium group did not overlap with those of the other groups. However, DRS-R98-K severity distribution of the delirium group slightly overlapped with that of the dementia group. MMSE-K score showed substantial overlap between the delirium and dementia group and between the schizophrenia and other psychiatric group. CGI-S scores were significantly different but there was substantial overlap across groups.

Figure 1.

Boxplots of Korean version of Delirium Rating Scale-Revised-98 (DRS-R98-K) total, DRS-R98-K severity, Korean version of Mini-Mental State Examination (MMSE-K), and Global Impressio-Severity (CGI-S) scores for each of the four diagnostic groups. Median scores are denoted by the horizontal lines within the 50th percentile boxes; tails denotes the 25th percentiles and circles and asterisks are outliers. Significant differences between delirium and each other group were noted for the DRS-R98-K and DRS-K.

Sensitivity and specificity

ROC curve analysis was used to identify the threshold of DRS-R98-K total and severity score and DRS-K scores that could distinguish patients with delirium from those with all other diagnoses, and particularly from those with dementia (Figure 2). ROC curve analysis showed that the area under the curve (AUC) of DRS-R98-K total score between the delirium group and all other groups was 0.948 (SE=0.016), and that the AUC between the delirium and dementia group was 0.873 (SE=0.037). The best cut-off scores were 18.5 and 19.5 in DRS-R98-K total score between the delirium and other three groups, and 20.5 in those between the delirium and dementia group (Table 2).

Figure 2.

ROC curve of DRS-R98-K total and severity score between the delirium group and other three groups and between the delirium and dementia groups. ROC: receiver operating characteristic, DRS-R98-K: Korean version of the Delirium Rating Scale-Revised-98.

Table 2.

Sensitivity and specificity for DSM-IV delirium diagnosis at various cut-off scores (delirium N=69; dementia N=29; all other groups N=88)

DRS-R98-K: Korean version of Delirium Rating Scale-Revised-98, DRS-K: Korean version of Delirium Rating Scale

The AUC of DRS-R98-K severity score between the delirium and other three groups was 0.894 (SE=0.024), and the AUC between the delirium and dementia group 0.763 (SE=0.05). The best cut-off points for the DRS-R98-K severity scale were 14.5 and 15.5 between delirium and all other groups, and 19.5 between the delirium and dementia group.

The AUC of DRS-K recorded the highest value, 0.999 (SE=0.001) between the delirium and other three groups, and 0.997 (SE=0.003) between delirium and dementia group. The cut-off score of DRS-K was 12.5 between the delirium and other three groups, and 14.5 between the delirium and dementia group.

Discriminant analysis

Discriminant analysis was applied to determine which items were the most effective at differentially diagnosing between delirium and dementia. Five of the 16 items were statistically significant for discriminating delirium from dementia. Item 15 ('fluctuation of symptom severity', F=112.66, p=0.000, Canonical coefficient=0.633) showed the greatest discriminating power, followed by item 16 ('physical disorder', F=111.67, p=0.000, Canonical coefficient=0.408), item 2 ('perceptual disturbances and hallucinations', F=99.84, p=0.000, Canonical coefficient=0.468), item 14 ('temporal onset of symptoms', F=84.43, p=0.000, Canonical coefficient=0.349), and item 1 ('sleep-wake cycle disturbance', F=72.16, p=0.000, Canonical coefficient=0.282). And also, it is possible to set discriminant function as D=0.282×i-tem1+0.468×item2+0.349×item14+0.633×item15+0.468×i-tem16) with statistical significance (Wilks' Lambda=0.203, p=0.000). The concordance rate between the DSM-IV diagnosed delirium group and the group ascertained by applying these five items from discriminant analysis was 98%, showing that those five items can be enough to distinguish delirium from dementia with high discriminant power.

Scale reliability

Cronbach's alpha coefficient was 0.91 for the DRS-R98-K total scale and 0.89 for the severity scale, indicating good internal consistency. After the correction for the effect of each item removal from the scale, the alpha coefficients ranged from 0.90 to 0.92 for the DRS-R98-K total scale and from 0.87 to 0.91 for the DRS-R98-K severity scale, indicating high internal consistency and reliability of this scale (Table 3).

Interrater reliability

Interrater reliability was measured using the ICC between two primary raters and one secondary rater. When the primary rater JR was compared with the secondary rater YL for the DRS-R98-K total score (n=20), ICC=0.987; for the DRS-R98-K severity score (n=20), ICC=0.986; and for the DRS-K score (n=20), ICC=0.978. When the other primary rater JL was compared with the secondary rater YL for the DRS-R98-K total score (n=18), ICC=0.969; for the DRS-R98-K severity score (n=18), ICC=0.965; for the DRS-K score (n=18), ICC=0.965. These data indicate excellent interrater reliability.

DRS-R98-K pre and post treatment

Forty-four of the delirious subjects were reassessed after delirium treatment and 41 of them no longer met DSM-IV criteria for delirium. Mean scores for DRS-R98-K total (pre, 27.7±7.2; post, 9.2±4.0), DRS-R98-K severity (pre, 22.2±7.2; post, 6.5±3.6), and DRS-K (pre, 21.4±3.7; post, 6.1±2.9) during pre- and post-treatment were significantly improved (p=0.000) and well below cut-off scores from ROC analyses. Mean MMSE-K scores also improved (pre, 16.6±6.7; post, 23.8±4.4; p=0.000). The DRS-R98-K is responsive to change in clinical status as anchored by DSM-IV.

DISCUSSION

This study demonstrates that the DRS-R98-K is a reliable and valid instrument for diagnosis and severity assessment of delirium in Korean patients. Both DRS-R98-K total and severity scores showed high correlations with Korean versions of validated instruments that assess some or many delirium symptoms, the DRS-K and MMSE-K. DRS-R98-K total and severity scores showed higher correlations with the DRS-K than with MMSE-K and CGI-S scores. This finding is sup-ported by the report that the admission MMSE score have 12% influence of the variance on the DRS-R98 score.²¹ This result demonstrates that this instrument is specific to delirium and that it also shares common characteristics with MMSE-K, in measuring cognitive functions and with the CGI-S, in assessing disease severity. Our findings for validity, reliability, internal consistency and response to change are consistent with other validation studies of the DRS-R98 in other countries.

DRS-R98-K significantly distinguishes delirium patients from dementia, schizophrenia, and other psychiatric groups based on mean and median score comparisons. However, there is some overlap of DRS-R98-K scores distributions between the delirium and dementia groups as expected from overlapping phenomenology. The score distribution in the delirium group of our study is similar with the results of the original study,⁴ whereas the score distribution in our dementia group was somewhat higher than theirs [DRS-R98 total score 13.9±4.2 (range 9-22), DRS-R98 severity score 12.4±3.5 (range 8-20)]. The greater overlap in score distribution in dementia in our study may have resulted from inclusion of some end-stage dementia patients that raised the mean more than the median. The overlap between the two groups did not affect the diagnosis of each disease by discriminant analysis. As expected, the diagnostic three items (items 14, 15, 16) as well as item 1 ('sleep-wake cycle disturbance') and item 2 ('perceptual disturbances and hallucinations') were revealed as the important items for the differential diagnosis between delirium and dementia. Other studies have found similar high differentiation for the diagnostic items, though a different pattern for the other items and different statistical methods were employed.²²

CGI-S was clinically comparable across groups though statistically different. When data for DRS-K and DRS-R98-K total and severity scales were reanalyzed using CGI-S as a covariate, the findings did not differ. Thus, slight difference in overall illness severity among groups could not explain the group differences in DRS-K and DRS-R98-K severity and total scale findings. Unlike the delirium specific scales, the MMSE-K did not discriminate between delirium and dementia groups nor between the schizophrenia and other psychiatric group.

Our optimum cut-off scores (18.5 and 19.5) on the DRS-R98-K total scale between delirium and the other three groups were slightly higher than scores found by Trzepacz et al (15.25 & 17.75). Similarly, our cut-off score between delirium and de mentia groups was 20.5 whereas theirs was 17.75. The cut-off score of 14.5 and 15.5 for DRS-R98-K severity between delirium and other groups in our study was similar to the score of 15.25 in Trzepacz et al'.s study, while the cut-off score of 19.5 between delirium and dementia groups in our study was higher than their score of 15.25. These findings are likely because the mean scores of DRS-R98-K total and severity scale in our dementia group were high due to the enrollment of some end-stage dementia patients into the group, demonstrating that the severity of dementia might affect the cut-off points. Our cut-off scores are more similar to those reported in the Portuguese DRS-R98 validation (20.1) where more severe dementia cases were included than in some other studies.¹⁰ The optimum cut-off scores do vary somewhat across countries (Table 4) and may be affected by cultural differences in addition to dementia severity range.

Table 4.

Cut-off scores based on ROC analysis in studies across countries

ROC: receiver operating characteristic, DRS-R-98: Korean version of Delirium Rating Scale-Revised-98

One limitation of our study was that the sex distribution am-ong groups was not even, though the comparison of scale scores between sexes did not differ, except in the other psychiatric group where scores were in the non-delirious range. Another limitation was the age difference among groups. It was predicted because of the characteristics of the peak age of onset of each disease. It was not considered to affect the score of the rating scales due to the lack of correlation between age and score within groups. Next obstacle was the possibility that the test-retest results might be counterbalanced by the rater's expectation or the fact of clinical improvement.

This study included completely different subjects from another validation study¹³ of DRS-R98 carried out by other researchers in Korea. Compared to the previous study, this study enrolled more patients and end-stage dementia patients whom we could meet in the clinical field. Furthermore, our study put some great efforts to define the cut-off score in order to differentiate delirium and dementia, so these cut-off scores can be used in bedside effectively and easily. The most influential items for making a distinction between delirium and dementia were revealed by discriminant analysis in the study. Taken together our data reveal that the DRS-R98-K is a valid and reliable instrument for diagnosing and evaluating the severity of delirium.