Clinical Utility and Cut-Off Scores of the Korean Adult Attention-Deficit/Hyperactivity Disorder Rating Scale

Abstract

Objectives:

This study was conducted to re-validate the clinical efficacy of the Korean Adult Attention-Deficit/Hyperactivity Disorder (ADHD) Rating Scale (K-AARS), which is a self-report scale for ADHD in adults, and to determine the clinical utility and cut-off scores of K-AARS.

Methods:

The participants were 135 drug naïve adults with ADHD and 144 healthy controls. To diagnose ADHD based on the diagnostic criteria of the Diagnostic and Statistical Manual of Mental Disorders, 5th Edition, two board-certified pediatric psychiatrists interviewed the participants and completed the Mini International Neuropsychiatric Interview. K-AARS was applied to all participants. K-AARS comprises six clinical subscales, one impairment subscale, and one driving behavior subscale. The receiver operating characteristic analysis was conducted to calculate the cut-off scores of K-AARS.

Results:

All subscale scores, including six clinical subscale, impairment subscale, and driving behavior subscale scores, were found to be significant in distinguishing adults with ADHD from healthy controls. The sensitivity and specificity of the six clinical subscales were 63.0–77.0% and 66.7–79.9%, respectively. The combined total score of the six clinical subscales, had a sensitivity of 80.0% and specificity of 79.9%.

Conclusion:

The discriminative power of K-AARS for the diagnosis of ADHD in adults was excellent, and K-AARS and the empirical diagnosis of adults can be useful in diagnosing ADHD in adulthood.

INTRODUCTION

In the past, the general population and physicians were less aware of attention-deficit/hyperactivity disorder (ADHD) in adulthood. Pediatric psychiatrists may most often encounter patients with ADHD as there is a high chance that the parents or other relatives of children treated for ADHD also present with such condition [1]. However, although physicians are experienced, they need adjunctive tools, such as psychological tests, to make an objective assessment. To diagnose ADHD in adults, only few psychological tools are available, which include Conners Adult ADHD Rating Scale [2], Brown Attention-Deficit Disorder Scales [3], Wender Utah Rating Scale [4], Barkley’s Adult ADHD Rating Scale-IV [5], and Adult ADHD Self-report Scale (ASRS) [6]. However, such tools are limited by copyright issue and by the effort to standardize procedure from English to Korean language for clinical use in Korea.

To overcome these obstacles, the Korean Academy of Child and Adolescent Psychiatry (KACAP) had organized a task force team to develop the Korean Adult ADHD Rating Scale (K-AARS) in 2009, and professor Jae Won Kim was appointed as the principal investigator [1]. Through discussions on whether the new scale will be a self-report form or a physician rate form and whether it can be used as a screening or diagnostic tool, the team members developed a self-report scale to diagnose ADHD in adults. The initial draft of the scale included 86 questions, and the scale was used on 136 adults with ADHD and 408 healthy controls. Via a validity and reliability analysis, the final draft included 73 questions and comprised eight subscales: six clinical subscales, one impairment subscale (IMP), and one subscale for driving behavior (DR) [1]. K-AARS was highly reliable in terms of internal consistency (Cronbach’s alpha: 0.77–0.95) and correlation between factors (0.57–0.86). Concurrent validity and discriminant validity were statistically significant [1].

The need for the assessment and diagnostic tools for ADHD in adults was emphasized to prepare for the application of the national health insurance to adults with ADHD since September 2016 in Korea [7]. KACAP had already published the revised version of the Korean practice parameter for ADHD [8]; however, it focused on children and adolescents. Thus, a practice parameter targeting ADHD in adults must be developed. This study was conducted to determine the cut-off scores for the clinical application of K-AARS as a follow-up to the previous study [1] and as a part of the preparation for the publication of the practice parameters.

METHODS

In the previous K-AARS study [1], the participants were aged between 20 and 50 years. In the current study, the target age was expanded from 19 to 65 years. The participants were recruited from March 2017 to February 2019 at eight university hospitals in Seoul and other metropolitan areas.

After two board-certified pediatric psychiatrists conducted clinical interviews in accordance with the Diagnostic and Statistical Manual of Mental Disorders, 5th edition (DSM-5) [9], patients who visited the psychiatric outpatient clinic and were diagnosed with ADHD were classified under the clinical group with their consent. The diagnosis was made only if there is an agreement between the two psychiatrists. Otherwise, the participants were not registered in the study. In case of inconsistency in the presentation of specifiers between psychiatrists, the participant was registered. The Mini International Neuropsychiatry Interview (MINI) Korean version [10] was performed to exclude other mental disorders. The healthy controls were recruited from the hospital, and the participants visited the hospital after seeing the research advertisement. After the pediatric psychiatrists conducted the diagnostic interview that focused on ADHD and MINI [10], the healthy controls were registered in the study if they did not present with mental disorders. K-AARS was used in all participants in both groups.

K-AARS is a self-report scale and five-point Likert-type scale with the following responses: never, rarely, sometimes, often, and always. The never response is scored 1 (lowest score) for all questions, and the always response is scored 5 (highest score) for all questions. The respondents answered each question about how often they present with a symptom over the past 6 months. K-AARS has 73 questions, which comprised three parts: six clinical subscales with 55 questions, IMP with 6 questions, and a subscale for DR with 12 questions. The six clinical subscales are inattention (IA), hyperactivity (HYP), impulsivity (IM), antisocial personality/ conduct behavior/oppositional defiant behavior (ACO), emotional dysregulation (ED), and disorganization (DO). If the score of the six clinical subscales and IMP subscale were higher, the symptoms were more severe. When the score of the DR subscale was higher, the driving habit is healthier, and this result is attributed to the reverse scoring of the DR subscale.

The exclusion criteria of the study were as follows:

• (1)Mental disorders

• (2)Congenital or hereditary genetic disorders

• (3)Organic brain disorders, including epilepsy, stroke, and cerebral palsy

• (4)Continuous treatment required due to serious physical illness

• (5)Drug treatment for ADHD within the last 3 months

Statistical analysis

Data were analyzed using descriptive methods, independent t-test, and χ². The SPSS software version 21.0 for Windows (IBM Corp., Armonk, NY, USA) was used for analysis. Moreover, receiver operating characteristic (ROC) analysis was conducted to calculate the cut-off points of K-AARS. Via the ROC curve analysis, the sensitivity and specificity of all subscales and the total score of the clinical subscales were obtained. The maximum point at which the sum of sensitivity and specificity is determined was considered the cut-off score. If the area under the ROC curve of a test is above 0.70, then the test can be categorized as fair, good, or very good. A p value <0.05 was considered statistically significant.

Research ethics

The study protocol was approved by the Institutional Review Board of the eight university hospitals, including Kyung Hee University Hospital (KMC IRB 2017-02-054), of which the principal investigator was Geon Ho Bahn. Informed consent was obtained from all participants.

RESULTS

A total of 135 adults with ADHD were included in the clinical group and 144 healthy adults in the control group. In the clinical group, 92 (68.1%) were male, with a mean age of 26.7±9.0 years. In the control group, 82 (58.2%) were men with an average age of 27.9±7.0 years. No significant differences were observed in terms of age and sex between the two groups (Table 1). The six clinical subscale and IMP subscale scores of the clinical group were significantly higher than those of the control group (Table 2). The DR subscale scores of the clinical group were significantly lower than those of the control group due to reverse scoring. According to the ROC analysis, the sensitivity and specificity of the six clinical subscales (IA, HYP, IM, ACO, ED, and DO) were 63.0–77.0% and 66.7–79.9%, respectively (Table 3, Fig. 1). The total score of the six clinical subscales had a sensitivity of 80.0% and a specificity of 79.9% (Fig. 1). The sensitivity and specificity of the IMP subscale were 70.5% and 72.5%, respectively (Fig. 2). As the DR subscales were negatively correlated, the sensitivity and specificity were 32.4% and 35.5%, respectively (Fig. 2).

Table 1.

Demographic characteristics of the subjects

	Adults with ADHD (n=135)	Healthy controls (n=144)	χ2 (p)
Age (years)	26.7±9.0	27.9±7.0	45.680 (0.107)
Sex (male)	92 (68.1)	82 (58.2)	3.726 (0.054)

Data are mean±SD or n (%) values. ADHD: attention-deficit/ hyperactivity disorder

Table 2.

The comparison of subscale scores of K-AARS between ADHD and control group

Subscales	Adults with ADHD (n=135)		Healthy controls (n=144)		t*
	Mean	SD	Mean	SD
IA	52.99	14.55	32.84	12.87	12.269
HYP	12.37	4.63	9.26	3.59	6.238
IM	23.07	8.76	15.06	6.20	8.748
ACO	12.96	5.19	9.15	3.47	7.162
ED	40.79	11.50	28.17	14.68	7.958
DIS	17.58	5.78	10.24	4.69	11.600
Subtotal	159.76	41.97	104.72	36.44	11.714
	(n=122)		(n=131)
IMP	14.11	7.03	7.70	4.61	8.503
	(n=68)		(n=62)
DR	29.31	22.91	44.39	17.70	-4.219

^*p=0.000. ACO: antisocial personality/conduct behavior/oppositional defiant behavior, ADHD: attention-deficit/hyperactivity disorder, DIS: disorganization, DR: driving habits, ED: emotional dysregulation, HYP: hyperactivity, IA: inattention, IM: impulsivity, IMP: impairment, K-AARS: Korean Adult ADHD Rating Scale

Table 3.

AUC analysis for subscales of K-AARS to examine sensitivity and specificity

Subscales	AUC	p	Cut-off value	Sensitivity (%)	Specificity (%)
IA	0.848	0.000	42.50	77.0	77.8
HYP	0.708	0.000	10.50	64.4	66.7
IM	0.781	0.000	18.50	71.1	72.9
ACO	0.745	0.000	10.50	63.0	72.9
ED	0.801	0.000	35.50	74.1	75.0
DO	0.844	0.000	13.50	74.8	79.9
Subtotal of clinical subscales	0.845	0.000	131.50	80.0	79.9
IMP	0.785	0.000	9.50	70.5	72.5
DR	0.288	0.000	46.50	32.35	35.48

ACO: antisocial personality/conduct behavior/oppositional defiant behavior, AUC: area under the curve, DO: disorganization, DR: driving habits, ED: emotional dysregulation, HYP: hyperactivity, IA: inattention, IM: impulsivity, IMP: impairment, K-AARS: Korean Adult ADHD Rating Scale

Fig. 1.

ROC curve of clinical subscales of K-AARS. ACO: antisocial personality/conduct behavior/oppositional defiant behavior, DO: disorganization, ED: emotional dysregulation, HYP: hyperactivity, IA: inattention, IM: impulsivity, K-AARS: Korean Adult ADHD Rating Scale, ROC: receiver operating characteristic.

Fig. 2.

ROC curve of impairment and driving behavior subscales from K-AARS. AARS: Adult Attention-Deficit/Hyperactivity Disorder Rating Scale, DR: driving, IMP: impairment, K-AARS: Korean Adult ADHD Rating Scale, ROC: receiver operating characteristic.

DISCUSSION

In Korea, although data about ADHD in adulthood are limited compared to those of ADHD in childhood [11], there is increasing knowledge about the fact that ADHD is a condition that affects not only children but also adults, and that such condition continues from childhood to adulthood. Moreover, the coverage of the national health insurance for adults with ADHD, which started in September 2016, can help obtain more clinical data of adult ADHD cases [7]. However, clinicians find it challenging to diagnose ADHD in adults who have different characteristics; thereby, leaving numerous adult patients without proper diagnosis and treatment. As presented in this study, each subscale score and the sum of the six clinical subscale scores of K-AARS could be used as a diagnostic cut-off for adult patients in clinical practice. Based on this study, a total score greater than or equal to 132 for the six clinical subscales will be considered the cut-off score for diagnosing ADHD with the use of K-AARS: the true positive rate (sensitivity) of diagnosis was 80.0%, and the false positive rate (1-specificity) was less than 20.1%. Meanwhile, Kessler et al. [12] have reported that the six-question ASRS had a sensitivity of 68.7% and specificity of 99.5%, and Heo et al. [13] have revealed that the ASRS had a sensitivity of 0.627 and 1-specificity of 0.196 for diagnosing ADHD in the Korean adult population. In Brazilians, the Brown Attention-deficit Disorder Scale presented a fair sensitivity with 72% accuracy and fair specificity with 88% accuracy [14]. Ustun et al. [15] have reported that the revised ASRS had a higher sensitivity at 91.4% and specificity at 96.0%.

Some of the subscales of K-AARS (ACO, ED, and DO) differ from the diagnostic criteria of ADHD in the DSM-5 [9]. From the outset of the development of K-AARS, authors aimed to identify the symptoms and characteristics of ADHD in adults because the diagnostic criteria of DSM-5 were mainly based on childhood symptoms. Since ADHD has been overlapped with a variety of psychiatric comorbidities, comorbid psychiatric disorders, particularly behavioral or personality disorders, must be identified [16]. Of the patients with cluster B personality disorders, 33–65% have a history of ADHD in childhood [16]. Oppositional defiant disorder and conduct disorder as well as anxiety disorders are observed in 25–33% of children with ADHD [17]. Although the ACO subscale itself cannot distinguish such diagnoses, it could help physicians diagnose and treat ADHD associated with antisocial personality, oppositional defiant disorder, or conduct disorder. In the Utah criteria for adult ADHD [4], temper, affective lability, emotional over-reactivity, and disorganization have been emphasized in addition to attention difficulties, hyperactivity/restlessness, and impulsivity. Moreover, Pinzone et al. [18] have emphasized temperament traits, such as lability, irritability, and excessiveness of emotional responses in adult ADHD. The ED and DO subscales can reflect such symptoms and characteristics in ADHD. As emotional regulation deficits are evident in about 34–70% of adults, the specific subgroups of ADHD must be differentiated from the initial stage of the diagnosis [19]. The efficacy of using ACO, ED, and DO subscales should be monitored and reevaluated using the K-AARS in clinical practice.

The IMP score could be used to assess the influence of ADHD on daily life as the diagnostic criteria D of the DSM-5 have stated that “clear evidence that the symptoms inter-fere with, or reduce the quality of social, academic, or occupational functioning” [9]. In future studies, authors should assess the possible use of IMP score as an indicator of the current severity of ADHD based on the DSM-5 [9]. Adults with ADHD present with impairments in cognitive, emotional, and social functioning, and difficulty in driving is one of these impairments [20]. However, not all individuals with ADHD are affected similarly. Ferro and Leatherdale [21] have reported the lack of association between ADHD and past-year traffic violations. However, they have found three factors that increase the odds for past-year traffic violations among individuals with ADHD: aged 20–29 years, male sex, and white ethnicity. Therefore, in this study, although the DR subscale score was a significantly sensitive subscale for discriminating between the ADHD and healthy control groups, it may not be the answer to whether all individuals with ADHD present with higher level of unsafe driving behaviors [20]. The DR score must also consider differences depending on regions and cultures, such as big and small cities as well as urban and rural areas.

The participants were included in the study based on the assessment of two psychiatrists using the DSM-5 but without the use of quantitative psychological measures, and this is considered a limitation of this study. Meanwhile, two psychiatrists diagnosed the participants with ADHD only when a consensus is made, and this is considered the strength of the study. Another limitation is that this study was conducted at eight university hospitals located in metropolitan areas, including Seoul. These limitations should be considered when generalizing results. In the future, further studies must be conducted to collect and analyze clinical data about the use of K-AARS according to region and type of psychiatric clinics. In addition, K-AARS is a self-report scale; thus, a clinical interview with physicians and other adjunctive tests, such as computerized comprehensive attention test [22] and Diagnostic Interview for ADHD in adults [23], must be conducted to diagnose adults with ADHD in clinical practice. Although the current study had some limitations, we expect that K-AARS can reflect the culture and characteristics of adults with ADHD in the Korean population.

CONCLUSIONS

The subscales of K-AARS were designed by pediatric psychiatrists to assess the clinical symptoms of ADHD in adults and to suit the culture in Korea. K-AARS differs from the existing adult ADHD scales due its subscales (ACO, ED, and DO), and a follow-up evaluation of the utilization of these subscales in the future must be carried out. Since this study showed that K-AARS had high sensitivity and specificity in the clinical group with ADHD, the cut-off scores presented were found to be a reasonable self-reporting measure in adults with ADHD.