The Standardization of the Clock Drawing Test (CDT) for People with Stroke Using Rasch Analysis

Abstract

[Purpose] The aim of this study was to standardize the clock drawing test (CDT) for people with stroke using Rasch analysis. [Subjects and Methods] Seventeen items of the CDT identified through a literature review were performed by 159 stroke patients. The data was analyzed with Winstep version 3.57 using the Rasch model to examine the unidimensionality of the items’ fit, the distribution of the items’ difficulty, and the reliability and appropriateness of the rating scale. [Result] Ten out of the 159 participations (6.2%) were considered misfit subjects, and one item of the CDT was determined to be a misfit item based on Rasch analysis. The rating scales were judged as suitable because the observed average showed an array of vertical orders and MNSQ values < 2. The separate index and reliability of the subject (1.98, 0.80) and item (6.45, 0.97) showed relatively high values. [Conclusion] This study is the first to examine the CDT scale in stroke patients by Rasch analysis. The CDT is expected to be useful for screening stroke patients with cognitive problems.

INTRODUCTION

Cognitive screening tests need to detect and manage the problems associated with deteriorated cognitive functions caused by stroke, traumatic brain injury or neurological psychiatric disorders, such as dementia or delirium^{1, 2)}. MMSE (Mini-Mental Status Examination) and NCSE (Neuro-behavior Cognitive Status Examination) have been used mainly as tools for screening the cognitive function in clinical practice.

Stroke-induced cognitive impairments in patients of various ages are expected to increase in an aging society³⁾. However, cognitive screening tests are being performed mainly for Alzheimer-type dementia (AD). Few studies have described the characteristics of dementia caused by stroke, and their findings are inconsistent^{4, 5)}. Therefore, more study of vascular dementia and the development of an appropriate screening tool is needed.

Because of the ease of use, the clock drawing test (CDT) has been widely used in differential diagnosis of early dementia^{6, 7)}. In particular, the CDT can discriminate patients with early dementia by their visual perception impairment. In recent years, studies of the CDT, which measures semantic memory and executive function of the damage, have been reported^{8, 9)}. The CDT in the differential diagnosis of stroke caused by damaged brain function has attracted increasing attention as an assessment tool¹⁰⁾. In addition, the CDT in a short period of time can relieve the pain of patients with cognitive impairment caused by stroke.

As a type of dementia, an early diagnosis and treatment of vascular dementia (VD) after a stroke can prevent further progress and can allow improvements in subjects’ conditions^{4, 11)}. Therefore, simple and sensitive test tools are needed to determine the cognitive function after stroke. Rasch analysis is widely used in development of rating scale to accurately analyze the level of individual potential and estimate the items of performance according to the characteristics of stroke¹²⁾. As Rasch analysis can track the ability of patients, regardless of the difficulty of the items, it is useful in the development of cognitive function screening¹³⁾. Therefore, the aim of this study was to standardize the CDT for people with stroke by Rasch analysis.

SUBJECTS AND METHODS

Seventeen items of the CDT identified through a literature review were performed by 159 stroke patients. All subjects gave their written informed consent to participation in the experiment in accordance with the ethical standards of the Declaration of Helsinki.

The seventeen items with six different classifications were selected by considering previous studies^{14,15,16,17,18,19,20)}. We organized all six categories of the seventeen items by adding a circle, hands, numbers of the clock categories proposed by Freedman (1994) to the categories consisting of the response, memory and time. Table 1 lists the references related to the configuration of the items.

Table 1. Item classification through a literature review.

Item of the CDT	Sunderland (1989)	Wolf-Klein (1989)	Mendez (1992)	Freedman (1994)	Shua-Haim (1996)	Lam (1998)	Woodford (2007)
1. Behavioral response	*	*	*			*
2. Language response	*	*
3. Size of circle	*		*	*
4. Closed of circle	*		*	*
5. Symmetry of circle	*		*	*
6. Twelve numbers	*		*	*	*	*
7. Numbers other than			*	*		*
8. Location (3, 6, 9, 12)	*	*	*	*	*	*
9. Location (1, 2, 4, 5)	*	*	*	*	*	*
10. Location (7, 8, 10, 11)	*	*	*	*	*	*
11. Memory of the time (11:10)							+
12. Memory of the procedures							+
13. Hour & minute hands	*		*	*	*
14. Length of hands			*	*		*
15. Exact time	*		*	*	*	*
16. Central point			*	*
17. The time to perform							+

* Items used in each study, + suggestion item

The statistical software, Winstep (Chicago, IL, USA) 3.57.1 version, was used for statistical analysis of the data. Rasch analysis is a one dimensional model that asserts that an item response is the result of an interaction between the respondent’s ability and the parameters of the item scale²¹⁾. The Rasch model is sometimes referred to as a rating scale. The rating scale model is appropriate for modeling Likert-type response data. The rating scale model was used because the CDT consists of a Likert scale and uses the same rating scale for all items^{22, 23)}.

The infit mean square statistic (infit MNSQ) and the outfit mean square statistic (outfit MNSQ) were used to confirm the unidimensionality as well as to identify the misfit items and subjects²⁴⁾. In this study, an item or subject was considered an appropriate model fit if the infit and outfit MNSQ had a range of 0.6–1.4, as well as a Z score between −2 and 2.

In Rasch analysis, both the subject’s ability and item difficulty are expressed as logits. Logits with a greater positive magnitude represent increasing item difficulty. The subject separation index (SI) and item SI represent the ability of a given test to separate cognition. SI must exceed 2 to achieve the desired level of separation reliability (SR), at least 0.80, and SI must exceed 3 to achieve an SR of at least 0.90²²⁾.

Each item was characterized by a series of threshold parameters that define the difficulty or probability of the response categories in Rasch analysis. The rating scale analysis includes the category frequencies, average measure, threshold estimates, probability curves and category fit. The item was considered an appropriate rating scale if the rating scale had an outfit MNSQ of less than 2, as well as a range of 1.0–5.0 logits for its threshold estimates²²⁾.

RESULTS

Ten out of the 159 participants (6.2%) were considered misfit subjects because their standard infit values exceeded 2. After further analysis, these participants were excluded from the analysis.

Table 2 lists the results of Rasch analysis. Overall, one item of the CDT was determined to be a misfit item based on Rasch analysis. The misfitting item was ‘Behavioral response’.

Table 2. Item fit statistics: entry order .

(N)Items	Measure	S.E.	Infit		Outfit
			MNSQ	Z-value	MNSQ	Z-value
1. Behavioral response*	−6.66	1.83	Minimum estimated measure
2. Language response	−3.28	0.42	1.28	0.90	2.12	1.30
3. Size of circle	−1.78	0.27	1.21	1.30	2.43	2.30
4. Closed of circle	−1.23	0.25	1.17	1.20	0.96	0.00
5. Symmetry of circle	−1.11	0.24	1.27	1.90	1.26	0.80
6. Twelve numbers	0.62	0.20	1.07	0.80	1.07	0.50
7. Numbers other than	−0.05	0.21	1.06	0.60	1.00	0.10
8. Location (3, 6, 9, 12)	0.94	0.20	0.81	−2.20	0.80	−1.20
9. Location (1, 2, 4, 5)	2.07	0.21	0.79	−2.30	0.59	−1.70
10. Location (7, 8, 10, 11)	1.90	0.20	0.89	−1.20	0.77	−0.90
11. Memory of the time (11:10)	1.41	0.20	1.33	3.40	1.45	2.00
12. Memory of the procedures	0.04	0.21	0.88	−1.10	0.80	−1.10
13. Hour & minute hands	−1.00	0.24	0.62	−3.20	0.39	−2.50
14. Length of hands	1.18	0.20	1.07	0.90	0.99	0.00
15. Exact time	0.78	0.20	0.72	−3.40	0.56	−3.10
16. Central point	−0.05	0.21	1.01	0.10	1.13	0.70
17. The time to perform	−0.42	0.22	1.02	0.20	1.10	0.50

*Item did not fit, that is, infit MNSQ outside 0.6–1.4 range or Z-value outside −2–2 range

Figure 1 shows the distribution of both the item location and subject measure plotted along the same ability levels and hierarchical order of the 16 items of the Winstep output.

Fig. 1.

Subject ability/item difficulty map of the 16 items of CDT. The subject ability is on the left and item difficulty is on the right. More able persons and more difficult items are near the top of the diagram. #=two persons, .=one person, M=Mean, S=1SD, T=2SD

Table 3 summarizes the rating scale analysis. The rating scales were judged as suitable, because the observed average showed an array of vertical order and MNSQ < 2.

Table 3. Summary of the rating scale analysis.

Category label	Observed count	Observed average	Infit MNSQ	Outfit MNSQ
0	913	−0.83	1.01	1.23
1	1487	1.85	0.97	0.86

The SI and the SR of the subjects (1.98, 0.80) and items (6.45, 0.97) showed a relatively high value. Generally, the SI increases with increasing number of subjects or items. In addition, the SR is the same as KR-20 or Cronbach’s alpha.

DISCUSSION

The main aim of this study was to standardize the CDT for stroke patients using Rasch analysis. Its unidimensionality was examined through an item fit, distribution of the item difficulty, and the reliability and the appropriateness of the rating scale.

A review of the literature showed that the reliability of the 17 items of the CDT was satisfactory, and the rating scale of the CDT was appropriate for stroke patients. The usefulness of the CDT can vary depending on whether a scoring system is used. This study used a screening tool developed by adding questions appropriate for stroke patients.

In previous studies, the CDT was used to assess patients with dementia, psychiatric disorders, and mild cognitive impairment. These studies reported that the CDT has high sensitivity and specificity, and is a good assessment tool^{10, 16, 25, 26)}. In the present study, appropriate items were selected according to the characteristics of stroke patients, and then used in actual evaluation. Therefore, the CDT will be useful for stroke patients with cognitive problems.

In this study, the misfit item was ‘behavioral response’. Because the questions for this item were considered too easy, one item was removed from the CDT on account of the results of Rasch analysis. On the other hand, the item of ‘verbal response’ was a fit. These results prove the existence of cognitive-response processes^{27, 28)}. In particular, the results indicate that the language cognitive response is more difficult than that of the behavioral response.

‘Verbal response’ was the easiest item among the sixteen items of the CDT and the ‘behavioral response’ was the most difficult. The distribution of both item locations and subjects appeared to be an appropriate fit statistic. A direct comparison was possible of the potential ability of the individuals and the difficulty of the items in Figure 1. Therefore, the range of difficulty with an individual’s ability to measure the distribution of the items properly occurred²⁹⁾. On the other hand, in the present study, 26 participants (17.4%) were found to show more than 2 logits. This suggests that they had a normal cognitive function. Subjects without cognitive impairment show a high score in the CDT.

The rating scale of the CDT was determined to be appropriate for stroke patients³⁰⁾. Rasch analysis is more useful because it can provide a more comprehensive understanding of the latent structure^{21, 31)}. This study focused on obtaining objective data using Rasch analysis. Overall, a CDT scale suitable for stroke patients was developed.

The results of this study may have limited generalization to stroke patients because of the lack of consensus among scholars and researchers’ in its development. Therefore, further studies on stroke patients with cognitive problems will be needed.

Despite these limitations, this study is the first to examine the CDT scale in stroke patients using Rasch analysis. In future studies, the CDT will be used to screen stroke patients with cognitive problems.