ADHD: Is Objective Diagnosis Possible?

Abstract

Although attention deficit/hyperactivity disorder (ADHD) is one of the most common cognitive disorders, the usual diagnostic procedures pursued by psychiatrists, neurologists, pediatricians, and family practitioners are based largely, if not exclusively, on subjective assessments of perceived behavior. The recommended approaches to ADHD diagnosis are reviewed, first from the perspective of the various expert panels, and then from the research literature upon which those recommendations are based. The authors agree that ADHD is a clinical diagnosis, and that the assessment of subjective reports can be systematic. But they propose that objective data should also contribute to the clinical diagnosis of ADHD; and that new computerized assessment technology can generate objective cognitive data in an efficient and cost-effective way. Computerized tests can also improve the assessment of treatment response over time.

Introduction

Attention deficit/hyperactivity disorder (ADHD) is very much in the public eye. Increasing numbers of schoolchildren are diagnosed with the condition and millions of North American schoolchildren take ADHD medications. Members of college and graduate programs and test administrators for tests such as the Scholastic Achievement Test (SATs) are petitioned to make the necessary accommodations for people who are diagnosed with ADHD. What is a reasonable accommodation to one person may seem like “jumping the queue” to another.¹ Some adult patients are not shy about visiting their new family practitioner and announcing they have ADHD and requesting amphetamines. Thirty years ago, on the heels of amphetamine epidemics and diet-pill excess, such a patient would likely get thrown out on his ear. It is no small measure of how things have changed that today this patient usually gets what he asks for.

The reported prevalence of ADHD is likely to excite a sense of urgency among the converted and skepticism among the dubious. Thirty years ago, the prevalence of ADHD was said to be 1 to 3 percent of all schoolchildren in North America;² then it was 3 to 5 percent.³ In 1997, 3 to 10 percent of schoolchildren in North American were said to have ADHD.^4,5 By 2002, the cumulative prevalence was said to be 16 percent.⁶ That same year, the US Centers for Disease Control (CDC) estimated that 1.6 million elementary school children in the US had ADHD, a rate of only seven percent (www.cdc.gov/nchs).

The increase is probably due to changes in diagnostic emphasis. The diagnosis has been broadened to include adults as well as children and patients whose sole complaint is inattention and distractibility at school or work. The diagnosis is certainly in vogue. There have been a rash of popular books on the subject, and several companies are marketing new drugs for ADHD or new formulations of old drugs. Special services, including extra time on standardized tests, are afforded to children and adolescents diagnosed with ADHD. This sort of thing increases the likelihood of an ADHD diagnosis.

It is appropriate, then, to address the question of ADHD diagnosis with a critical eye.

What is ADHD?

That’s easy. ADHD is a psychiatric disorder. Why? Because it resides in the Diagnostic and Statistical Manual of Mental Disorders (DSM). And how do you make the diagnosis? Another easy one—If the patient meets the DSM criteria.

Therein, of course, lies the problem. How do you know that the patient meets the DSM criteria, beyond the patient’s subjective reports or the subjective opinions of the patient’s parents, spouse, teachers, etc.? The DSM criteria are perfectly sound, in the author’s opinion. The problem is, though, that the criteria are based on nothing more than reported symptoms. The various elements that comprise the syndrome—locomotor hyperactivity, impulsive behavior, excitability, emotional immaturity, short attention span, distractibility, and inefficiency at school or work—are seldom, if ever, observed directly, let alone measured, by the diagnosing physician. The criteria are a way to systematize diagnosis, but, by their very nature, they are subjective.

The symptoms of ADHD are very common. In one school survey, for example, no less than half of the boys were rated by their mothers as overactive.⁷ In the Isle of Wight study, 75 percent of the dull children were rated by teachers as inattentive, but 30 to 50 percent of the brighter children were also so described.⁸

The clinical presentation of ADHD is variable. Some ADHD kids were hyperactive in utero and some are said to “run since they learned to walk.” Others are not hyperactive or impulsive at all, but only inattentive and distractible. Hyperactive kids may turn into lazy, hypoactive adolescents. Girls with ADHD are often shy and self-effacing. Some ADHD patients are referred by their pre-school teachers. Others are only diagnosed in college or graduate school.

The symptoms are situational. Douglas is credited with the idea that the symptoms of ADHD are evoked only under certain circumstances.⁹ A deficit in sustained attention, after all, is only meaningful in situations where sustained attention is required. If children who can’t sit still and pay attention were simply excused from going to school—as, indeed, they are in many poor countries—then there won’t be any ADHD children left to disrupt class and bedevil their teachers. On the other hand, if every child’s education were strictly individualized, and if normative expectations were removed from the classroom altogether, then minor deficits in learning style would hardly be important. They would simply be the basis for individualizing the child’s learning experience. Indeed, it is arguable that the increased prevalence of ADHD is the consequence of changes we have made in the classroom environment and the high expectations we have of students to perform —from the first grade through college.

So, for all of these reasons, the DSM criteria for ADHD diagnosis are necessary but not sufficient. They are systematic, but not objective. The problem, though, is not the DSM. The problem is what is ADHD really?

ADHD may be a psychiatric disorder but it is not a mental illness, in the sense that depression and schizophrenia are mental illnesses. It is a constellation of personality traits and cognitive styles that cluster, in pure form, in a relatively small number of people and in various combinations and permutations in large numbers of people. It is typical of a large class of neuropsychiatric conditions that afflict large numbers of people to a mild degree and small numbers of people to a severe degree. It is a mild aberration of the regulatory apparatus of the brain, in particular the complex functional systems that are identified with the corpus striatum and the prefrontal cortex.¹⁰ It is a relative weakness in one’s ability to regulate attention, behavior, and emotional responding.¹¹ Like most personality traits, it runs in families and tends to persist over the life span. Like most mild impairments, people for the most part learn to adjust to it. Because the central elements of the disorder are functions that are outer-directed (attention, behavioral, and emotional responding), the difficulties that patients experience are usually in their adaptation to external events. For the same reason, certain environments evoke or aggravate the symptoms and other kinds of environment do not.^9,12–14

Because ADHD is a condition that exists in equilibrium with a social and educational milieu, a special responsibility devolves upon the physicians and psychologists who diagnose the disorder. ADHD has a variable threshold of expression: The DSM advises that “there has to be clear evidence of clinically significant impairment in social, academic, or occupational functioning” to warrant the diagnosis (DSM-IV-TR, 2000). The arbiters of where, precisely, that threshold lies are the doctors who make the diagnosis. One expects them to be objective, not relativistic, in exercising that responsibility. ADHD is, after all, a medical diagnosis that commits schools to additional expense, affords citizens special privileges, and releases, into a vulnerable population, yet one more class of drugs with potential for abuse.

Diagnosis is one of the perennial problems of the ADHD “movement;” is it possible to make an objective diagnosis when everything about the disorder seems to be subjective? Psychiatrists are accustomed to making informed judgments about what represents a “clinically significant impairment.” But primary care physicians, to whom increasing numbers of ADHD patients and would-be ADHD patients present, would probably prefer to have a more objective standard. The symptoms of ADHD are non-specific. That is, they may be met, alone or together, with a host of other conditions, including psychotic disorders, mood disorders, anxiety disorders, personality disorders, and developmental disabilities. In fact, the proper diagnosis of ADHD requires the clinician to exclude these other conditions. One is not always able to do that with confidence; physicians who are not trained in child psychiatry often find the task daunting.

Recommended Approaches to Diagnosis

This is from the Council on Scientific Affairs, American Medical Association, in 1998: The overall approach to diagnosis may involve (1) a comprehensive interview with the child’s adult caregivers; (2) a mental status examination of the child; (3) a medical examination for general health and neurological status; (4) a cognitive assessment of ability and achievement; (5) use of ADHD-focused parent and teacher rating scales; and (6) school reports and other adjunctive evaluations if necessary (speech, language assessment, etc.).¹⁵

The economic impact of applying this method to a disorder that afflicts perhaps five percent of schoolchildren is well worth contemplating. Indeed, “diagnostic complexity” and “time constraints” were among the reasons cited by general practitioners for their “low level of interest” in ADHD management.¹⁶ The American Academy of Pediatrics suggests a less ambitious, but more realistic approach: Rating scales from parents and teachers, exclusion of alternative diagnoses, and developmental testing, if necessary. Family practitioners are advised to use rating scales, to review the child’s school progress, to screen for vision or hearing impairment and comorbid psychiatric conditions, and to use “objective measures of cognitive function.”¹⁷ Child psychiatrists are urged to interview the child and to utilize direct observations (Table 1).¹⁸

Table 1.

Recommendations for ADHD diagnosis

Rating Scales

The expert panels are as strong in support of rating scales as they are of the clinical history. It is fair to say that rating scales are the sine qua non of ADHD diagnosis. ADHD rating scales have been used for almost 50 years. If a pediatrician has the Vanderbilt Rating Scale¹⁹ in hand, he or she feels that the standard of care has been met. Many universities will qualify students for learning disabilities services if they have the Brown Rating Scale.^20–22

In fact, there is no reason not to use rating scales to evaluate ADHD symptoms in children. There is even a standardized instrument, the Wender Utah Rating Scale, which captures childhood symptoms of ADHD in patients who present as adults.²³ Rating scales are good for delineating symptoms and their perceived severity. Ideally, several informants should fill out a rating scale, including parents, teachers, patient, and spouse.

Like the DSM-IV, rating scales are systematic but not objective. They are prone to “halo” effects—if the informant thinks a kid is hyper and needs Ritalin®, he or she is likely to score every item as “a severe problem” or “all the time.” On the other hand, if one is opposed to drug treatment, every item may be marked “not at all” or “never.” Agreement among different raters—even two parents—is seldom very good;²⁴ the expert panels have never advised practitioners how to reconcile disparate ratings.

Like the DSM criteria, rating scales are systematic and quantitative, but not objective. They are a necessary component of the diagnostic process, but not sufficient.

Psychological Testing

Faced with an ambiguous clinical picture or disparate ratings from parents and teachers or discouraged by the subjectivity of the whole process, the practitioner is tempted to punt—refer the child to a specialist. Availability usually favors the clinical child psychologist, who is not only well trained to diagnose ADHD but can also provide a “cognitive assessment of ability and achievement.”

In the hands of an experienced examiner, conventional psychological tests can indeed capture the elements of inattention and impulsivity that characterize patients with the disorder. Psychological testing is also capable of identifying cognitive disabilities that may compound the problem and complicate management. An ADHD patient with a borderline IQ score, a memory problem, or a specific learning disability will necessarily require a more comprehensive treatment program.

A conventional IQ test, like the WISC or the WAIS, affords the examiner an opportunity to observe the patient’s relative strengths and weaknesses in different kinds of tasks. ADHD patients may do poorly on auditory immediate memory tasks that tend to be rote and lacking in context, like Digit Span or Number-Letter Sequencing. Their deficits may show up in index scores like Working Memory and Processing Speed.

There are two problems with (neuro)psychological testing as a routine approach to ADHD diagnosis. One is expense. In our neighborhood, comprehensive psychological evaluations for ADHD can cost between $800 and $2,000. After the evaluation is done, the patient has to seek out a physician to get a prescription for medication. The second problem is that the ordinary psychoeducational battery is only an indirect measure of ADHD. Attention deficits, locomotor hyperactivity, and cognitive impulsivity may or may not be evidenced when a child is in a small room, one-on-one with a single adult. In fact, the “freedom from distractibility” factor of the WISC-III does not appear to be “a reliable or a valid index of attention or a diagnostic screening measure for identifying children with ADHD.”²⁵

For these reasons, most psychologists who evaluate large numbers of ADHD children tend to complement their testing battery with computerized tests of attention. After all, what could be more objective for ADHD diagnosis than a direct measure of the child’s attentional ability?

Computerized Tests of Attention

Computerized neurocognitive testing has a long history.²⁶ Computerized tests are used routinely in aerospace and military medicine, in sports medicine for concussion management, and in industrial medicine to evaluate the effects of toxic exposure. Beyond these highly specialized areas, however, tests of vigilance or sustained attention are the most popular computerized tests because they are used for evaluating ADHD. No one has ever maintained that a computerized test is sufficient for establishing the diagnosis of ADHD, but one can argue that it is inappropriate to make the diagnosis of ADHD without using at least one such test.²⁷

Continuous Performance Tests. The Continuous Performance Test (CPT) is a venerable test of vigilance or sustained attention. Versions of the CPT have been used in research with brain injured patients, epileptics, and ADHD children for 40 years. It is an easy test to computerize; in fact, the only way one can administer the test now is on a computer. Several free-standing CPTs are commercially available at this time (Table 2).

Table 1.

Commercially available CPTs

The Conners CPT (www.mhs.com): Visual stimuli (letters), 14 minutes long. $495 for unlimited use (norms for children >;6 yrs); $675 for a version normed for younger children.

IVA—Integrated Visual & Auditory CPT (www.braintrain.com): Auditory and visual stimuli, 13’ long. Normed from age 6–96 (sic). $295 for the first ten tests, $99 for each subsequent set of 10 tests; or $598 for the first 10 tests and $89 for each subsequent set of 10 tests).

ACPT—Auditory CPT (www.psychcorp.com): Auditory stimuli, 10’ long. Norms, age 6–11. $109 for the kit and 12 forms, $16 for 12 additional forms.

TOVA (Tests of Variables of Attention) (www.ADHDwarehouse.com): Visual stimuli (geometric patterns), 21.8’ long. Norms for children >6 yrs. $395 (plus $100 annual fee or $10 per test).

Gordon Diagnostic System (www.gsi-ADHD.com): Visual stimuli (numbers) on a free-standing, microprocessor-based machine that is portable. Cost $1595, unlimited administrations.

One would expect precise measures of sustained attention, like the CPT, to be the “gold standard” for ADHD diagnosis, but this is not the case. The correlation between CPT performance and parent or teacher rating scales is modest at best.^28–31 The Conners CPT and the TOVA, two of the more commonly used CPTs, are equally sensitive (85%) in demonstrating attention dysfunction in children who have been diagnosed with ADHD.^32,33 The TOVA, however, generates unacceptably high false positive rates (30%) in normal controls and children with other psychiatric disorders (28%).^28,34 Because the CPT is such a sensitive measure of CNS dysfunction, there are always multiple reasons why someone’s performance on the test is impaired.³⁵

The visual CPT using letters as stimuli was the original test developed by Rosvold and Mirsky.³⁶ Auditory CPTs are less easy to administer but more appropriate for children with suspected auditory attention difficulties. The results of visual and auditory CPTs, however, do not necessarily correlate.³¹

Another problem with the CPT is its unreliability. Even the commercially available CPTs fail to report acceptable levels of test-retest reliability. The TOVA, for example, does not even report test-retest reliability, but rather, split-half reliability coefficients.³⁷ Split-half reliability may be appropriate for a test that is given to a strong practice effect, but that is hardly true of the TOVA. Test-retest reliability is essential to consider in a test that is likely to be used serially (e.g., to evaluate an ADD patient’s response to stimulant medications). The Conners CPT-II does report test-retest reliability, but only on 23 normal subjects, and not on raw scores, but on derived scores.³⁸

The CPT is thought to be useful as an adjunct to clinical diagnosis, as its popularity attests. But it is not a “diagnostic” instrument, and, in the absence of acceptable levels of reliability, it may not even be appropriate for monitoring treatment effects.³⁹

In the opinion of one expert panel:¹⁸ Computerized tests of attention and vigilance (CPTs)^33,40–42 are not generally useful in diagnosis because they suffer from low specificity and sensitivity.^43,44 They are useful, however, as research tools. Behavioral observations while performing the CPT discriminate ADHD children from other groups as well as or better than the CPT scores.⁴⁵ The correspondence between impulsive errors on the CPT and behavioral impulsivity has not been established.⁴⁶ When used for assessment of medication efficacy, the applicability of results to the patient’s natural environment is unproven^47,48 or even absent.⁴⁹ CPTs are not consistently sensitive to stimulant effects.⁵⁰ Also, task and contextual factors, such as the presence or absence of an adult, the instructions given to the patient, and the nature of feedback and contingencies, can substantially affect scores.^51,52 Concerns have been expressed regarding commercial CPT products.⁵³

Computerized Neurocognitive Batteries

The CPT is also found in a number of computerized neurocognitive batteries, including the MicroCOG,⁵⁴ CogTest (www.cogtest.com), the NES2,⁵⁵ and CNS Vital Signs.⁵⁶ The batteries mentioned have been used, in research and in the clinic, to evaluate ADHD patients. But in addition to the CPT, which is simply a measure of vigilance of sustained attention, they also measure complex attention, visual and/or verbal memory, reaction time and information processing speed, psychomotor speed, and executive control.

The rationale for using a more comprehensive neurocognitive screening battery is that ADHD is not simply a disorder of sustained attention. Indeed, impairment in sustained attention is common to a certain extent to all children with psychiatric disorders.⁵⁷ Neuropsychological studies of ADHD children and adults reveal subtle but clear impairments in several complex functional systems: Selective attention;⁵⁸ memory;^59,60 reaction time⁶¹ and information processing speed;⁶² motor speed⁶³ and visuomotor ability;⁶⁴ and executive control functions, like set-shifting,⁶⁵ inhibitory control,⁶⁶ and working memory.⁶⁷

What is also clear from the neuropsychological literature is that although many tests indicate impairment in ADHD patients, no one test is sufficient to make the diagnosis on its own. Neuropsychologically, ADHD patients are a diverse group; individuals may demonstrate deficits in one test, or in one cognitive domain, but not in another. For that reason, combinations of tests are more likely to yield useful information than any one single test.^68–71 For that reason, theories of the cognitive basis of ADHD are likely to speak in terms of overarching principles (like “dysregulation,” “behavioral inhibition,”¹¹ and “cognitive resource allocation”)⁷² rather than in terms of one specific cognitive domain.

The utility of a combined neurocognitive screening approach, employing a computerized battery, was demonstrated by Gualtieri, et al.⁵⁶ Using a computerized battery with measures of sustained and selective attention, cognitive flexibility, memory, information processing speed, and psychomotor speed, the authors found that poor performance on the CPT scores was the least sensitive and specific measure of ADHD, but that measures of executive function/cognitive flexibility increased both sensitivity and specificity to a considerable degree. A summary score, comprised of all the tests in the battery, was the most effective predictor of ADHD status.

Computerized screening batteries can also be used, like the computerized CPT, as a way to evaluate drug effects in ADHD patients.⁷³ For example, is the medication still effective? Is a long-acting stimulant still effective late in the afternoon? How does drug “A” compare to drug “B”? And if a child on stimulants is doing poorly in school, but does well on computerized testing, then referral for a psychological evaluation may be warranted.

Technical Approaches of Limited Utility

The foregoing discussion does not, by any means, exhaust the wide range of medical and psychological tests that have been brought to bear on the problem of ADHD. “Actigraphs” (or accelerometers) are motion sensors that the patient wears like a wristwatch.^74–76 They seem to be reliable indicators of a child’s level of activity, but the results do not correlate with other measures and they are not diagnostic instruments.^77,78 They are of limited clinical utility, since hyperactivity per se is not typically the source of the most significant impairment.⁷⁹ Usually, the important variable is not the total amount of activity, but its situational appropriateness.¹⁸

Polysomnography, evoked potentials, quantitative EEG, and functional brain imaging (e.g., SPECT) are used in research settings and even in some specialized ADHD clinics, but have no clearly defined clinical utility.^80–83

The proper diagnosis of ADHD has been problematic for a long time. During the 1960s, when the condition was known as the “hyperkinetic syndrome,” research psychologists recommended direct observations of the child in the classroom. A psychologist would sit in the back row of the child’s class, trying to be unobtrusive, and actually count instances of “out of seat behavior” or “blurting out answers” during a set period of time.

Another approach was the “Activity Room,” a playroom with observation windows and four identical tables in four quadrants. Each of the four tables had a set of identical toys. As the child played in the playroom, for 30 minutes or so, a psychologist behind a one-way mirror would count how many times the child crossed from one quadrant of the room to another.

Some clinicians still use this method: A double-blind trial of placebo and two doses of a stimulant, each condition lasting a week or so, with parent and teacher ratings done every week or in some cases every day. Presumably, if the child was not markedly better on drug compared to placebo, he or she wasn’t really ADHD after all. (The whole process is a waste of time, though, if the clinician happened to choose the wrong drug or the wrong dose range for that particular patient.)

Summary and Conclusions

ADHD remains a clinical diagnosis. There is no frog test for ADHD. The diagnosis is made by taking a history and performing an examination, by reviewing school data, and ruling out alternative disorders. The DSM criteria and parent/teacher rating scales are essential elements. Occasionally, patients must be referred to a psychologist for psycho-educational testing, or to a child psychiatrist or a pediatric neurologist. But, as long as primary care physicians exercise a degree of skill, there is no reason why the vast majority of ADHD patients cannot be diagnosed accurately and treated appropriately in the primary care setting. If they are able to use objective measures in their offices, they can make the diagnosis and monitor treatment with more confidence.

The “gold standard” for ADHD diagnosis includes a comprehensive clinical history and examination, rating scales, direct behavioral observations, neuropsychological testing, and objective, comparative analysis of different drug effects. The economic burden of such an approach is, as we have pointed out, prohibitive.

The sine qua non for ADHD diagnosis is the history/examination and rating scales. School visits and direct behavioral observations are desirable, and a masters-level educational specialist can make them at a typical cost of $60 to $120 per hour. They will never be deemed reimbursable procedures, however, and will likely remain the prerequisite of the privileged class. Serial computerized neurocognitive testing, however, can be done efficiently and cheaply. The availability of inexpensive computer-based neurocognitive screening batteries places this technology in the hands of the physicians who need it most.

Even the most objective and comprehensive evaluation is subject to occasional error. A patient who has a typical history of ADHD, no indication of alternative pathology, poor performance on the CPT, and a beneficial response to a stimulant probably has ADHD. But he or she may have something else. One always hopes that the initial evaluation has managed to exclude problems like depression, bipolar disorder, anxiety disorders, incipient psychosis, obsessive-compulsive disorder, substance abuse, hypothyroidism, seizure disorders, or neurodegenerative disease. If it hasn’t, then the truth will come out over time.

It is necessary, then, to follow up with the patient over time. In our clinic, it has been sufficient to follow stable ADHD patients at quarterly intervals. In so doing, one is keen to detect problems with medication treatment. One is also alert to the possibility of an alternative diagnosis. Serial neurocognitive testing is not a bad idea, if it can be done cheaply and efficiently.

This is a simple but effective approach to the problem of ADHD diagnosis. It is not unduly expensive or oppressive to patients, and does not unduly tax the resources of a general medical office. Objective diagnosis is possible if the practitioner has the capacity for differential diagnosis, a couple of good rating scales, and a computer program that can administer neurocognitive tests.