Utility of an abbreviated questionnaire to identify individuals with ADHD at risk for functional impairments

Abstract

Objective

To discern whether a subset of items from the 99-item Current Behavior Scale (CBS) of behaviorally defined executive function deficits (EFDs) in adults with attention-deficit/hyperactivity disorder (ADHD) can identify a group at risk for poor outcome.

Methods

Subjects were 200 adults with ADHD participating in a family study of ADHD in adults. Factor analysis was used to reduce the number of items in the 99-item CBS.

Results

The one factor solution provided 8 items with factor loadings above 0.70. This abbreviated set of items was highly correlated with the 99-item CBS (0.91) and was similarly related to functional outcomes compared to the 99-item CBS (average correlation of 0.30 versus 0.32).

Conclusion

For adults with ADHD, a set of 8 empirically derived from the CBS similarly correlated with negative outcomes compared to the 99-item CBS, raising the possibility of utilization as a mechanism for identification of EFDs in adults with ADHD.

INTRODUCTION

As described in the DSM-IV (American Psychiatric Association, 2000), individuals diagnosed with attention-deficit/hyperactivity disorder (ADHD) have problems organizing tasks and activities, persisting with tasks until completion, controlling impulsivity and being forgetful in daily activities. Many of these symptoms are consistent with the description of a group of neuropsychological functions known as executive functions (EFs) (Barkley, 1997b, Willcutt et al., 2005). A large empirical literature has documented the presence of neuropsychological impairments in individuals with ADHD (Pennington et al., 1996).

Biederman et al. (Biederman et al., in press) recently examined whether questionnaires aimed at assessing behavioral concomitants of executive function deficits (EFDs) may be useful in the identification of adults with ADHD who show functional impairments. Using the Current Behavior Scale (CBS), a 99-item questionnaire developed by Russell Barkley (Barkley, 1997a), this study documented that ADHD individuals with high scores on this scale had reduced social class, educational and occupational attainments, as well as impairments in adaptive social and leisure functioning compared to subjects with ADHD who had low scores.

Although these findings suggested that the CBS can help identify a subgroup of ADHD individuals at risk for additional functional morbidity beyond that conferred by the diagnosis of ADHD alone, the large number of items that comprise the CBS makes it cumbersome for clinical use. Thus, examining whether a small set of items from the CBS could be identified can have important practical implications. Such a tool could assist clinicians and researchers in identifying adults with ADHD at high risk for functional impairments and help develop appropriate interventions to address them.

AIMS OF THE STUDY

The main aim of this study was to examine whether a smaller set of items from the CBS can be empirically derived. To this end, we used data from a large sample of comprehensively assessed adults with ADHD recruited for a family study of adults with ADHD that completed the 99 item CBS. We used factor analytic techniques to reduce the number of items in the CBS and compared the utility of the empirically derived abridged scale with the full CBS scale to predict functional outcomes.

METHODS

Subjects

Subjects (N=200, 53.5% male) with DSM-IV diagnosis of ADHD were recruited for participation in family-genetic study of adults with this disorder (Faraone et al., 2006a, Faraone et al., 2006b). Males and females between the ages of 18 and 55 were eligible for this study. We excluded potential subjects if they had major sensorimotor handicaps (e.g. deafness, blindness), psychosis, autism, inadequate command of the English language, or a Full Scale IQ less than 70. No ethnic or racial group was excluded. ADHD subjects were ascertained from referrals to a psychiatric clinic at a major university general hospital and media advertisements. A three-stage ascertainment procedure was used to select all participants. The first stage was the subject’s referral or response to advertisements. The second stage confirmed the diagnosis of ADHD by using a telephone questionnaire. The questionnaire asked about the symptoms of ADHD, as well as questions regarding study inclusion and exclusion criteria. The third stage confirmed the diagnosis of ADHD with face-to-face structured interviews with the individuals. Only subjects who received a positive diagnosis at all three stages were accepted into the study.

Psychiatric Assessments

We interviewed all subjects with the Structured Clinical Interview for DSM-IV (SCID) (First et al., 1997) to assess psychopathology supplemented with modules from the Kiddie-SADS-E (KSAD-E; Epidemiologic Version adapted for DSM-IV) (Orvaschel, 1994) to cover ADHD and other disruptive behavior disorders. The structured interview also included questions regarding academic tutoring, repeating grades, and placement in special academic classes.

The interviewers had undergraduate degrees in psychology, and they were trained to high levels of inter-rater reliability for the assessment of psychiatric diagnosis. We computed kappa coefficients of agreement by having experienced, board certified child and adult psychiatrists and licensed clinical psychologists diagnose subjects from audiotaped interviews made by the assessment staff. Based on 500 assessments from interviews of children and adults, the median kappa coefficient was 0.98.

A committee of board-certified child and adult psychiatrists and licensed psychologists resolved all diagnostic uncertainties. The committee members were blind to the subjects' ascertainment group, ascertainment source, and all non-diagnostic data (e.g., neuropsychological tests). Diagnoses were considered positive if, based on the interview results, DSM-IV criteria were unequivocally met to a clinically meaningful degree. We estimated the reliability of the diagnostic review process by computing kappa coefficients of agreement between clinician reviewers. For these clinical diagnoses, the median reliability between individual clinicians and the diagnoses assigned by the review committee was 0.87. The kappa coefficient for the diagnosis of ADHD was 1.0.

Psychosocial Assessments

Social functioning was assessed with the Social Adjustment Scale (SAS) (Weissman et al., 1976). As a measure of overall functioning, we used the Global Assessment of Functioning (GAF) (American Psychiatric Association, 2000), a summary score assigned by the interviewers based on information gathered during the diagnostic structured interview. Socioeconomic status (SES) was assessed with the Hollingshead scale (Hollingshead, 1975). The scale determines a total SES score from occupational and educational scores.

Current Behavior Scale (CBS)

We used the Current Behavior Scale (CBS) to assess behavioral concomitants of executive function deficits. This is a 99-item (see Appendix A) self-report questionnaire developed by Barkley (Barkley, 1997a). Responses to each item ranged from 0 (Never or Rarely) to 3 (Very Often). Therefore, a total score on the CBS could range from 0 to 297.

Statistical Analysis

We used principal-components factor analysis to identify redundant items from the 99-item CBS. For each retained factor, any items with a factor loading over 0.70 were used for the abridged scale. Pearson correlations were used to test how strongly the abridged scale and 99-item CBS were related to functional outcomes that were previously shown to be significantly related to the full 99-item CBS (Biederman et al., in press). Cronbach’s alpha was calculated as a measure of internal consistency for the abridged scale.

RESULTS

Principal-component factor analysis revealed one factor that explained 31% of the variance. As shown in Table 1, 8 items from the 99 item CBS had factor loadings over 0.70. The correlation between the 8-item abridged scale and the 99-item CBS was 0.91. The 8-item abridged scale had a Cronbach’s alpha of 0.90. Cronbach’s alpha for the entire CBS was 0.97.

Table 1.

Current Behavior Scale items with a factor loading above 0.70

Item	Number and Description	Factor Loading
22.	Have trouble planning ahead or preparing for upcoming events	0.712
27.	Can’t seem to accomplish goals I set for myself	0.704
31.	Can’t seem to hold in mind things I need to remember to do	0.722
44.	Easily frustrated	0.712
46.	Have difficulty motivating myself to stick with my work and get it done	0.720
64.	Have trouble doing what I tell myself to do	0.746
66.	Lack self-discipline	0.718
70.	Have trouble organizing my thoughts	0.704

Table 2 shows the percent of variance explained by the 99-item CBS and the 8-item abridged scale. The 8-item abridged scale was on average as correlated with the functional outcomes evaluated as the 99-item CBS. In other words, a 92% reduction in items (99 items versus 8 items) resulted in only a 6% lower correlation between the CBS and the functional outcomes (correlation 0.32 versus 0.30).

Table 2.

Comparison of correlations between functional outcomes and the 99-item Current Behavior Scale and abridged 8-item Current Behavior Scale

	99-Item Current Behavior Scale	Empirically Derived 8-Item Abridged Current Behavior Scale
Functional Outcome	Correlation	Correlation
Global Assessment of Functioning	0.44	0.37
Social Adjustment Scale	0.55	0.53
Number of Comorbid Disorders	0.30	0.25
Socioeconomic Status
Overall	0.21	0.18
Education	0.21	0.19
Occupation	0.23	0.25
Average Correlation	0.32	0.30

DISCUSSION

The purpose of this analysis was to examine whether a smaller set of items from a 99-item scale (CBS), previously shown to capture behavioral manifestations of executive function deficits (EFDs), will be useful to identify adults with ADHD at high risk for functional dysfunction. Results showed that 8 empirically derived items from the CBS were highly correlated with the 99 items and were as predictive of negative functional outcomes in adults with ADHD as the larger scale. These results indicate that a small set of self -reported items indexing behavioral manifestations of executive functions deficits can help identify adults with ADHD at high risk for functional morbidity.

Although in our previous work utilizing all 99 items (Biederman et al., in press) we documented that the CBS could help identify a subgroup of ADHD individuals at significant risk for functional morbidity, the large number of items made this scale cumbersome to use. Thus, the current finding that a small set of 8 empirically derived items were as predictive of functional impairments in adults with ADHD as the entire set of 99 items contained in the CBS can greatly facilitate its use in clinical practice and research settings.

The empirically derived 8 items represent a heterogeneous group of behaviors and cognitive functions reflecting difficulties with planning/ organization, working memory, initiation, inhibition and emotional regulation. For example, the items “can’t seem to accomplish goals I set for myself” and “having trouble doing what I tell myself to do” fall in the area of deficits with the neuropsychological construct of “initiation.” Poor initiation can often prevent individuals who wish to succeed from achieving their goals. Other items reflected difficulties with planning/organization (having trouble planning ahead; having trouble organizing my thoughts). The abbreviated scale, not surprisingly, includes items of Working Memory (“Can’t seem to hold in mind things I need to remember to do”). Being “easily frustrated” relates to emotional control and addresses the manifestation of executive functions within the emotional realm. The inability to modulate emotional responses is considered a regulatory function and part of the executive functions (Giola et al., 2000).

Although specific definitions of executive functions vary, it is widely agreed that such functions are involved in higher order cognitive processes that include self-control and the successful regulation of goal-directed behavior (Loring, 1999). Lyon (1996) describes these areas as deficient in ADHD patients who may not present as having difficulties on psychometric measures, but who are displaying failures in everyday life.

As it was the case with the larger CBS (Biederman et al., in press), the empirically-derived 8 item scale also predicted lower educational status and lower employment opportunity. Lower educational status can have a serious impact on future employment opportunities, as many jobs may be unattainable due to limited schooling (Biederman et al., 2004). Likewise, the detrimental impact of behavioral concomitants of EFDs in adults with ADHD on occupational outcomes may account for high rates of under and unemployment associated with adult ADHD in the community (Biederman et al., 2006a, Biederman et al., 2006b).

Our findings need to be viewed in light of some methodological limitations. Because our results were analyzed on referred adults with ADHD, they may not generalize to other clinical or non-clinical populations. Additionally, since the majority of our subjects were Caucasians, our results may not generalize to other ethnic groups. The significant relationships between the CBS and functional impairments are based on cross-sectional data. Prospective studies will be needed to determine the predictive validity of the CBS behavioral symptoms.

Despite these considerations, our results show that a small set of 8 empirically-derived items measuring behavioral manifestations of executive function deficits can help identify a sizeable number of individuals with ADHD at high risk for functional deficits in educational, occupational, and interpersonal functioning.

Appendix

CURRENT BEHAVIOR SCALE - SELF-REPORT
Copyright by Russell A. Barkley, Ph.D., University of Massachusetts Medical Center
Name ____________________			Date __________
Instructions
Please circle the number next to each item that best describes your behavior DURING THE PAST 6 MONTHS.
Items:	Never or Rarely	Some-times	Often	Very Often
1. Find it difficult to tolerate waiting; impatient	0	1	2	3
2. Make decisions impulsively	0	1	2	3
3. Unable to inhibit my reactions or responses to events or others	0	1	2	3
4. Have difficulty stopping my activities or behavior when I should do so	0	1	2	3
5. Have difficulty changing my behavior when I am given feedback about my mistakes	0	1	2	3
6. Easily distracted by irrelevant thoughts when I must concentrate on something	0	1	2	3
7. Prone to daydreaming when I should be concentrating on something	0	1	2	3
8. Procrastinate or put off doing things until the last minute	0	1	2	3
9. Make impulsive comments to others	0	1	2	3
10. Likely to take short cuts in my work and not do all that I am supposed to do	0	1	2	3
11. Likely to skip out on work early if its boring or unpleasant to do	0	1	2	3
12. Can’t seem to defer gratification or to put off doing things that are rewarding now so as to work for a later goal	0	1	2	3
13. Likely to do things without considering the consequences for doing them	0	1	2	3
14. Change my plans at the last minute on a whim or last minute impulse	0	1	2	3
15. Start a project or task without reading or listening to directions carefully	0	1	2	3
16. Poor sense of time	0	1	2	3
17. Waste or mismanage my time	0	1	2	3
18. Fail to consider past relevant events or past personal experiences before responding to situations	0	1	2	3
19. Do not think about the future as much as others of my age seem to do	0	1	2	3
20. Not prepared for work or assigned tasks	0	1	2	3
21. Fail to meet deadlines for assignments	0	1	2	3
22. Have trouble planning ahead or preparing for upcoming events	0	1	2	3
23. Forget to do things I am supposed to do	0	1	2	3
24. Have difficulties with mental arithmetic	0	1	2	3
25. Not able to comprehend what I read as well as I should be able to do; have to re-read material to get its meaning	0	1	2	3
26. Can’t seem to remember what I previously heard or read about	0	1	2	3
27. Can’t seem to accomplish the goals I set for myself	0	1	2	3
28. Late for work or scheduled appointments	0	1	2	3
29. Trouble organizing my thoughts or thinking clearly	0	1	2	3
30. Not aware of things I say or do	0	1	2	3
31. Can’t seem to hold in mind things I need to remember to do	0	1	2	3
32. Have difficulty being objective about things that affect me	0	1	2	3
33. Find it hard to take other people’s perspectives about a problem or situation	0	1	2	3
34. Have difficulty keeping in mind the purpose or goal of my activities	0	1	2	3
35. Forget the point I was trying to make when talking to others	0	1	2	3
36. When shown something complicated to do, cannot keep the information in mind so as to imitate or do it correctly	0	1	2	3
37. Give poor attention to details in my work	0	1	2	3
38. Find it difficult to keep track of several activities at once	0	1	2	3
39. Can’t seem to get things done unless there is an immediate deadline	0	1	2	3
40. Dislike work or school activities where I must think more than usual	0	1	2	3
41. Have difficulty judging how much time it will take to do something or get somewhere	0	1	2	3
42. Have trouble motivating myself to start work	0	1	2	3
43. Quick to get angry or become upset	0	1	2	3
44. Easily frustrated	0	1	2	3
45. Over-react emotionally	0	1	2	3
46. Have difficulty motivating myself to stick with my work and get it done	0	1	2	3
47. Can’t seem to persist at things I do not find interesting	0	1	2	3
48. Do not put as much effort into my work as I should or than others are able to do	0	1	2	3
49. Have trouble staying alert or awake in boring situations	0	1	2	3
50. Easily excited by activities going on around me	0	1	2	3
51. Not motivated to prepare in advance for things I know I am supposed to do	0	1	2	3
52. Can’t seem to sustain my concentration on reading, paperwork, lectures, or work	0	1	2	3
53. Easily bored	0	1	2	3
54. Others tell me I am lazy or unmotivated	0	1	2	3
55. Have to depend on others to help me get my work done	0	1	2	3
56. Things must have an immediate payoff for me or I do not seem to get them done	0	1	2	3
57. Have trouble completing one activity before starting into a new one	0	1	2	3
58. Have difficulty resisting the urge to do something fun or more interesting when I am supposed to be working	0	1	2	3
59. Can’t seem to sustain friendships or close relationships as long as other people	0	1	2	3
60. Inconsistent in the quality or quantity of my work performance	0	1	2	3
61. Don’t seem to worry about future events as much as others	0	1	2	3
62. Don’t think about or talk things over with myself before doing something	0	1	2	3
63. Unable to work as well as others without supervision or frequent instruction	0	1	2	3
64. Have trouble doing what I tell myself to do	0	1	2	3
65. Poor follow through on promises or committments I may make to others	0	1	2	3
66. Lack self-discipline	0	1	2	3
67. Have difficulty using sound judgement in problem situations or when under stress	0	1	2	3
68. Trouble following the rules in a situation	0	1	2	3
69. Not very flexible in my behavior or approach to a situation; overly rigid in how I like things done	0	1	2	3
70. Have trouble organizing my thoughts	0	1	2	3
71. Have difficulties saying what I want to say	0	1	2	3
72. Unable to come up with or invent as many solutions to problems as others seem to do	0	1	2	3
73. Often at a loss for words when I want to explain something to others	0	1	2	3
74. Have trouble putting my thoughts down in writing as well or as quickly as others	0	1	2	3
75. Feel I am not as creative or inventive as others of my level of intelligence	0	1	2	3
76. In trying to accomplish goals or assignments, find I am not able to think of as many ways of doing things as others	0	1	2	3
77. Have trouble learning new or complex activities as well as others	0	1	2	3
78. Have difficulty explaining things in their proper order or sequence	0	1	2	3
79. Can’t seem to get to the point of my explanations as quickly as others	0	1	2	3
80. Have trouble doing things in their proper order or sequence	0	1	2	3
81. Unable to “think on my feet” or respond as effectively as others to unexpected events	0	1	2	3
82. Clumsy; not as coordinated in my movements as others	0	1	2	3
83. Poor or sloppy handwriting	0	1	2	3
84. Have difficulty arranging or doing my work by its priority or importance; can’t “prioritize” well	0	1	2	3
85. Slower to react to unexpected events	0	1	2	3
86. Get silly, clown around, or act foolishly when I should be serious	0	1	2	3
87. Can’t seem to remember things I have done or places I have been as well as others seem to do	0	1	2	3
88. Accident prone	0	1	2	3
89. More likely to drive a motor vehicle much faster than others (Excessive speeding)	0	1	2	3
90. Have difficulties managing my money or credit cards	0	1	2	3
91. I am less able to recall events from my childhood compared to others	0	1	2	3
92. Lose my temper	0	1	2	3
93. Argue with others	0	1	2	3
94. Actively defy or refuse to comply with others’ requests or rules	0	1	2	3
95. Deliberately annoy people	0	1	2	3
96. Blame others for my own mistakes or misbehavior	0	1	2	3
97. Am touchy or easily annoyed by others	0	1	2	3
98. Am angry or resentful	0	1	2	3
99. Am spiteful or vindictive	0	1	2	3