Sensitivity of Scales to Evaluate Change in Symptomatology with Psychostimulants in Different ADHD Subtypes

Abstract

Objective:

To assess the sensitivity of scales (Conners’ Global Index Parent and Teacher form [CGI-P, CGI-T], Clinical Global Impression Scale [CGI], Continuous Performance Test [CPT], and Restricted Academic Situation Scale [RASS]) in evaluating improvement in symptomatology with methylphenidate in different Attention Deficit Hyperactivity Disorder (ADHD) subtypes.

Method:

Four hundred and ninety children (309 with ADHD Combined/Hyperactive [ADHD-CH] and 181 with ADHD Inattentive subtype [ADHD-I]) participated in a two week double-blind placebo-controlled crossover methylphenidate trial.

Results:

CGI-P showed small effect size for ADHD-I and medium effect size for the ADHD-CH subtype. CGI-T showed medium effect size for ADHD-I and large effect size for ADHD-CH subtype. CGI and RASS showed large effect size while CPT showed medium effect size for both subtypes.

Conclusion:

Acute behavioural assessments by clinicians (CGI, RASS) are better at detecting improvement with medication in all subtypes than parent or teacher reports (CGI-P, CGI-T). CGI-T is better than CGI-P for ADHD-I in detecting change in symptomatology as there is a greater demand for attention at school.

Introduction

Attention-Deficit Hyperactivity Disorder (ADHD) affects 5–10% of children (Faraone, Sergeant, Gillberg, & Biederman, 2003). Symptoms of ADHD include short attention span, impulsivity and motor hyperactivity (Biederman & Faraone, 2005). Three subtypes have been delineated in DSM-IV and are, by order of highest to lowest prevalence, Combined Type (ADHD-C), Predominantly Inattentive Type (ADHD-I) and Predominantly Hyperactive-Impulsive Type (ADHD-H). Over the last decade, there has been an important debate as to whether ADHD-I is a separate disorder instead of a subtype of ADHD. For example, ADHD-I has been shown to be different from ADHD-C and ADHD-H on many parameters, including age and gender distribution, severity of symptoms, comorbidities, medication response and possible etiological factors (Grizenko, Paci, & Joober, 2009). All subtypes, though, of ADHD can result in academic impairment with slightly higher risks for ADHD-C and ADHD-I than ADHD-H (Escobar et al., 2008).

Several assessment tools have been developed as a means to evaluate ADHD severity and to measure improvement with treatment. For example, the Conners’ Global Index parent and teacher forms (CGI-P and CGI-T) are reliable scales that provide an ecological assessment of children with ADHD and how they respond to treatment (Conners, 1998). Scales, such as the Clinical Global Impression Scale (CGI), Continuous Performance Task (CPT) and Restricted Academic Situation Scale (RASS), have also been used to assess symptom severity and acute response to medication (Greenhill, Findling, Swanson, & ADHD Study Group 2002; Egeland, Johansen, & Ueland, 2009; Fischer & Newby, 1998). These scales are not without limitations, for example, the CGI-P and CGI-T rely on subjective evaluations of the parents and teachers. The RASS and CPI require training to be administered and are based on the evaluation of the functioning of the child over a brief period in a controlled environment. Therefore, interpretation of these instruments may not be generalizable to other settings. Furthermore, all scales are designed to assess specific behaviours; some scales assess in detail inattention while others might focus more on the symptoms of hyperactivity, restlessness and fidgeting.

There are very few studies that compare the sensitivity of tests to assess improvements in ADHD symptomatology with psychostimulants. Grizenko et al. (2004), in a sample of 147 children with ADHD (all subtypes), showed that there was a large difference in effect size of scales to assess behavioural response to MPH. Effect sizes varied from 0.41 to 1.4 for the following tests: CGI-P, CGI-T, CPT, RASS and CGI. In a similar design of a double-blind placebo-controlled study of modified release MPH, Greenhill et al. (2002) showed that in a sample of 321 children with combined/hyperactive subtype, the effect size to detect improvement on the CGI-T was 0.8 and on the CGI-P was 0.4. There are no studies, to our knowledge, that allow for comparison of the sensitivity of different scales to detect improvement in ADHD symptomatology with psychostimulants in different subtypes.

Psychostimulants are effective in improving the symptoms of ADHD in 75% of children with few side effects. Other medications such as clonidine, atomoxetine, risperidone and bupropion have been used in ADHD but tend to have a greater frequency of side effects and are overall less effective (Biederman & Faraone, 2005). Methylphenidate (MPH) is the most commonly used medication in Canada for ADHD, as both an immediate-release (Ritalin) or long-term release formulation (for example, methylphenidate HCL [Concerta, OROS] and multi-layer release methylphenidate [Biphentin]). Therefore it is important to properly assess the degrees of improvement with psychostimulant treatment before further increasing medication or considering alternative treatments. Psychostimulant treatments are effective in all three subtypes, although there are variations according to subtypes. For example, Escobar et al. (2008), showed that ADHD-I subgroup did not respond as well as the ADHD-C, but when the ADHD-I subgroup did respond they required lower dosages. In addition, another limiting factor in interpretation is the developmental trajectory of ADHD itself. Children tend to have less hyperactive-impulsive symptoms as they age (Biederman, Mick, & Faraone, 2000) and this could lead to possible changes in subtype diagnosis. For example, Lahey et al. (Lahey, Pelham, Loney, Lee, & Willcutt, 2005) describe that 76% of children diagnosed with ADHD-H changed to ADHD-C eight years later in life, 16.6% of children with ADHD-C were diagnosed as ADHD-I at eight years follow-up and 25% of children with ADHD-I had a different diagnosis of either ADHD-H or ADHD-C at least once at the seven or eight year follow-ups. In conclusion, because different scales measure different behaviours, some scales might not be as effective as others in detecting improvement with medication in specific age groups or ADHD subtypes. It is thus very important to provide the busy clinician with information as to which scales, that are cheap and simple to use, will be most appropriate to help monitor their patients’ care.

In summary, to date, there are no studies that compare the ability of different scales to measure both ecological and acute behavioural change with medication across ADHD subtypes. The purpose of this study is to examine the sensitivity of different scales in detecting the improvement in symptomatology with methylphenidate versus placebo in the combined/hyperactive versus inattentive subtype of ADHD.

Methods

Participants

The present study is based on a sample of 490 children (382 male, 108 female) ages 6–12 years (mean 9.02, SD 1.92), who participated in a double-blind placebo-controlled crossover methylphenidate trial. All children were diagnosed with ADHD and their subtypes were determined using DSM-IV criteria. The diagnosis was based on a clinical interview by a child psychiatrist and supplemented by school reports and the Diagnostic Interview Schedule for Children-IV (DISC-IV; Shaffer, Fisher, Lucas, Dulcan, & Schwab-Stone, 2000). Exclusion criteria consisted of a history of pervasive developmental disorder, psychosis, Tourette’s syndrome or an IQ of less than 70. Children were sequentially recruited from the Disruptive Behaviour Disorder Program and from the outpatient clinic at the Douglas Institute, a psychiatric university teaching hospital. All children were enrolled in the study after initial assessment and did not receive any significant amount of psychosocial intervention prior to or during the medication trial. Ninety-five percent of eligible children’s parents agreed to participate in the study. The trial was approved by the ethics board of the Douglas Institute. Written informed consent was given by the parents. All children also agreed to participate. Four hundred ninety out of 494 children who started the medication trial completed it. Three children in the ADHD-Combined/Hyperactive (ADHD-CH) group and one in the ADHD-I group dropped out of the trial due to side effects or reluctance to complete the testing.

Procedure

One week before the medication trial, children completed the baseline assessment that examined the degree of behavioural problems, academic performance, severity of illness and IQ. During the baseline the children were assessed using the CPT and the 10-item CGI-P and CGI-T. Children were then randomly assigned to receive either one week of active medication followed by one week of placebo or vice versa. The dosage given was 0.5mg/kg/day of methylphenidate in a divided BID dose, which is the best documented dose prescribed in clinical and research settings (Sprafkin & Dadow, 1996). Methylphenidate was used because long-acting formulations are available only in fixed doses which would not have allowed us to administer the specific mg/kg dosing. Methylphenidate takes effect shortly after its administration. However, to make sure that slower treatment responses would not be missed, the children received medication for seven consecutive days. Both medication and placebo were prepared by a pharmacist in identical coloured gelatine capsules. On the third day of each treatment week (testing day), before taking the morning medication, the child was evaluated using the CPT, the CGI scale and the RASS and then re-evaluated on the same measures one hour later to assess the acute effect of MPH versus placebo. CGI-P and CGI-T were completed at the end of each week reflecting the child’s overall performance during the preceding week. Parents were called by the research assistant on Sunday after carefully observing their children on the weekend to assess their child’s overall performance and complete the CGI-P.

Measures

10-Item Conners’ Global Index Parent and Teacher Form (Conners, Sitarenios, Parker, & Epsteins, 1998) – Both CGI-P and CGI-T are composed of ten items used to evaluate the frequency and severity in the last week of the child’s impulsivity, emotional outbursts and motor hyperactivity. Scoring is age and gender specific. Both the CGI-P and CGI-T have an internal reliability coefficient of 0.94. Test-retest reliability coefficients over a six to eight weeks interval were 0.8 and 0.72 for CGI-T and CGI-P respectively. Scores over 65 are in the clinical range.

Clinical Global Impression Scale (Rapoport, 1985) – Evaluates the severity of symptoms and global improvement of the child as assessed by a research psychologist while the child is undergoing testing. The CGI ranges from one to seven, with one representing the absence of symptoms and seven extreme symptomatology. Change in symptomatology was assessed by the research psychologist as he/she watches how the child’s performance prepill is different from postpill. In our trial, one week was placebo and one week methylphenidate. Symptoms were rated on a 7-point scale from very much improved (1) to very much deteriorated (7). The scale has good inter-rate reliability.

Continuous Performance Task (Conners, 1995) – CPT is a computerized test that measures sustained attention, response inhibition and impulse control. For 14 minutes the child is instructed to press the space bar when all letters appear except for the letter X. The CPT overall index was used in the trial. It is a weighted measure of different parameters, including omission errors, commission errors and time of response. The CPT has been shown to provide a good means for monitoring the effectiveness of treatment (Conners, 1995). As described in its manual, there is a clear positive linear effect of dose of MPH on reaction time (F=9.81, p<.01).

Restricted Academic Situation Scale (Barkley, 1990) – This task provides information about the frequency and severity of ADHD symptoms during performance of independent academic work. The child is left alone in a room with a set of math problems adapted to his/her academic level and told to do as many as he/she can in 15 minutes. The child’s behaviour is scored by a researcher through a oneway mirror over consecutive 15-second intervals on five behavioural categories: off-task, fidgeting, vocalizing, playing with objects and out of seat. The RASS has been shown to significantly discriminate children with ADHD from normal children (Milich, Loney, & Landau, 1982). Previous research has also shown that the RASS is sensitive to improvement in scores with dosages as low as 0.2mg/kg/day of MPH. There was good inter-rater-reliability.

Statistical Analysis

Children were divided into two groups: ADHD-I, N = 181, and ADHD-CH, N= 309. We chose to group ADHD-H and ADHD-C due to the low frequency of ADHD-H (N=47) and its high percentage of crossover to ADHD-C.

Two-tailed paired t-tests were used to determine statistically significant change between active and placebo weeks. To assess acute effects of MPH, we analysed change scores in the RASS and CPT during the testing day.

The effect size of a test represents the difference between the means of the patients on placebo and active medication over the SD of placebo. Cohen (1998) has described an effect size of 0.2 as being small, 0.5 as medium and 0.8 as large.

Placebo effects on the CGI-P and the CGI-T were determined by comparing changes in scores from baseline to placebo for those children who received placebo in the first week of the trial (N=230) versus changes in scores for children who received MPH the first week of the trial.

Results

Children with ADHD-CH versus ADHD-I (Table 1) were more commonly male, younger and had prior medication treatment. The ADHD-CH group also had parents with less education, more single mothers, and lower income. There was no difference between the two groups with respect to I.Q. and age appropriate grade level. Comorbidity was present in 76.8% of children with ADHD-CH but only 64.2% of ADHD-I group. Conduct disorder was the only comorbidity that was different in the two groups; it was significantly more common in the ADHD-CH than ADHD-I group.

Table 1.

Demographics and clinical characteristics of participants

	ADHD-I N=181	ADHD-CH N=309
Gender (male/female), %	70/30	82/18
Mean baseline age	9.6	8.6
Mean IQ	96.1	96.7
Ethnicity (white/black/other), %	87.9/6.0/6.1	84.5/10.7/4.8
Income group [$CAN], %
<20,000	18.5	37.7
20 to 40,000	25.6	26.4
>40,000	55.9	35.9
Child ever/never on medication, %	28/72	46/54
Father’s education, years	13.0	11.9
Mother’s education, years	13.7	12.6
Single mothers, %	21	26.8
Comorbidity with conduct disorder, %	33	13

All scales, except the CGI-P for children with ADHD-I (p=0.2), were able to detect improvement between placebo and active medication weeks (p=0.000). Improvements on all scales were greater for the ADHD-CH group then the ADHD-I group (Table 2).

Table 2.

Comparison of means scores on MPH and placebo week

Subtype	Test	Mean (SD)		N	t	Sig
		Placebo	Active
Combined/hyperactive	CGI-P	65.81(13.9)	59.20(12.8)	264	7.01	,000
	CGI-T	68.49(12.8)	57.37(12.2)	261	13.01	,000
	CGI	4.66(0.9)	3.07(1.1)	261	18.50	,000
	CPTd	1.80(7.15)	−3.85(9.10)	254	8.37	,000
	RASSd	7.24(26.9)	−21.45(20.2)	303	15.54	,000
Inattentive	CGI-P	57.25(12.8)	55.94(12.5)	158	1.28	,202
	CGI-T	62.43(14.0)	53.96(11.2)	148	8.08	,000
	CGI	4.50(0.8)	3.32(1.0)	164	11.19	,000
	CPTd	2.73(7.8)	−1.57(8.8)	152	4.29	,000
	RASSd	7.80(18.7)	−15.91(22.8)	174	10.50	,000

CGI-P = Conners’ Global Index parent form; CGI-T = Conners’ Global Index teacher form; CGI = Clinical Global Impression global improvement; CPTd = Continuous Performance Test change score (time2-time1); RASSd = Restricted Academic Situation Scale change score (time2- time1)

The effect size (Table 3) of the scales to detect change in symptoms between MPH and placebo week was quite variable. For children with ADHD-I, CGI-P had a low effect size, CGI-T and CPT had medium effect size and CGI and RASS had a large effect size. For children with ADHD-CH, CGI-P and CPT showed a medium effect size, and CGI-T, CGI and RASS had a large effect size.

Table 3.

Effect sizes of tests to detect change in symptomatology with MPH in different subtypes

Test	Type of measure	Inattentive	Combined/hyperactive
CGI-P	Ecological	0.08	0.44
CGI-T	Ecological	0.57	0.86
CGI	Acute behavioural response	1.45	1.75
CPT	Acute behavioural response	0.39	0.61
RASS	Acute behavioural response	0.84	0.91

CGI-P= Conners’ Global Index parent form; CGI-T= Conners’ Global Index teacher form; CGI= Clinical Global Impression global improvement; CPT=Continuous Performance Test; RASS =Restricted Academic Situation Scale

There was a placebo effect for both the CGI-P and CGI-T in both subgroups, but the relative improvements for the CGI-T and CGI-P were higher in the inattentive group (Table 4). It is also important to note that parents using the CGI-P reported much more of a placebo effect than teachers with the CGI-T. In the ADHD-CH group there was a 14.4% improvement on the CGI-P and 4.3% improvement on the CGI-T during the placebo week. In the ADHD-I group, there was a 19.4% improvement on the CGI-P and 6% improvement on the CGI-T during the placebo week.

Table 4.

Comparison of the effects of placebo vs. MPH on test scores

Subtypes	Week 1 assignment	N	Test	Mean Baseline	Mean Week 1	Improvement (Relative %)	Sig.
Combined/Hyperactive	Placebo	125	CGI-P	75.8	64.5	14.4%	.000
		125	CGI-T	70.9	67.7	4.3%	.000
	Active medication	130	CGI-P	76.7	58.6	23.6%	.000
		130	CGI-T	72.7	58.1	20.0%	.000
Inattentive	Placebo	93	CGI-P	69.4	56.0	19.4%	.000
		93	CGI-T	66.9	62.9	6.0%	.000
	Active medication	61	CGI-P	68.7	56.5	17.8%	.000
		61	CGI-T	65.4	54.7	16.4%	.000

CGI-P = Conners’ Global Index parent form; CGI-T = Conners’ Global Index teacher form

Discussion

This is the first study, to our knowledge, that compares the sensitivity of various scales to detect improvement in symptomatology in children with different ADHD subtypes who were administered psychostimulants. All the scales except the CGI-P, for the inattentive subgroup, were able to detect improvement with MPH. The reason for this could be that in the home environment children are frequently not required to concentrate and pay attention as much as in school. Children who have the inattentive subtype of ADHD have difficulties focusing and completing tasks which are most detectable in the school environment. Furthermore, the combined/hyperactive subgroup had higher overall scores than the inattentive subtype on all the scales and so there was more possibility of improvement.

The effect size of the different scales for the two ADHD subtypes also varied. For the inattentive ADHD subtype, CGI-P was shown to have a small effect size and hence is not the best scale to detect improvement in symptomatology. The CGI-T, that assesses how well the children focus in the classroom, and the CPT, that detects errors of both commission (impulsivity) and omission (inability to focus), had a medium effect size in inattentive children. While the CGI and RASS, which are both based on trained observers rating the performance of children, was shown to have a large effect size. Therefore, it is important to emphasize that for inattentive children the clinician, when assessing for a response to medication, should not solely rely on parental reports, which often miss changes in attention of children with psychostimulants. It is very important to also get reports from teachers, such as the CGI-T or if possible, complete testing using CGI or RASS.

Overall for the ADHD-CH subtype, all scales showed medium to large effect size with the rater evaluated scales (RASS and CGI), that examined acute response, showing greatest effect size. This is clinically very relevant in that for both subtypes a trained clinician can assess accurately the response to medication of a child based on performance on the RASS and through direct observation (CGI). The CGI-T completed by the teachers was also very effective detecting change in the combined subtype.

What was also interesting in the study is the very high placebo response observed by parents; 14.4% of parents in the ADHD-CH group and 19.4% of parents in the ADHD-I subgroup described improvement on placebo. Teachers also noticed a placebo response but to a lesser extend (4.3% in the ADHD-CH group versus 6.0% in the ADHD-I group). Therefore, clinicians must be aware of the extent of placebo response documented by parents on the CGI-P before concluding that the medication is effective or not, based only on parental reports. It is important to note that placebo response rates in trials of stimulants for children are between 4 and 20%. For example, 13% of the children in the MTA showed a placebo response. However, more than 90% of children with ADHD and who had a placebo response in the MTA study relapsed within 15 to 60 days (Greenhill et al., 2002).

The strengths of this study are multiple. First, it was completed in a very large sample size of 490 patients referred both from the general outpatient and the severe disruptive behaviour disorders programs. Hence we were able to explore the ability of scales to detect improvement with psychostimulants across the whole spectrum of ADHD. Furthermore the diagnosis of subtypes was very rigorously completed: subtype was determined through a clinical interview by a child psychiatrist and supplemented by school reports, interview with the parents and the DISC. Thirdly, almost all the children who started the double-blind placebo-controlled MPH trial completed the trial. Only three of the Combined/Hyperactive and one of the Inattentive sub-type did not complete the trial.

The limitations of our study include that we only examined five scales and only one dose of methylphenidate was used. Sensitivity of the various tests in detecting change in symptomatology in relation to multidosing or use of psychostimulants other than MPH was not done. We utilised MPH in order to be able to administer specific mg/kg dosing. Long-acting formulations are only available in a limited number of fixed doses. Results may have been different had we used long-acting formulations. Another limitation was that for some children we were unable to get teachers to complete the CGI-T and certain children were too oppositional to properly complete the CPT. The final limitation of the study is that it would be incorrect to compare the effectiveness to detect change of the different scales across sub-types given that the ADHD-CH subtype as a whole had a better response to psychostimulants than the ADHD-I sub-type. Nonetheless, comparing the effectiveness of different scales to detect improvement of symptomatology within the subtypes should help the clinician decide what scales are most useful to adjust medication in the different subtypes.

Overall, the goal of our study was not to compare how ADHD-CH versus ADHD-I responded to psychostimulants but to demonstrate the strikingly large variability of responses depending on the scale that is used. Children with ADHDCH improved more than children with ADHD-I on methylphenidate but different scales were able to detect changes in different constellations of symptoms. One must utilize scales that are specific to these constellations of symptoms to detect change. Therefore, if a scale is utilised that is not sensitive to detecting change on specific variables targeted, the clinician may unwittingly underestimate the impact of medication and hence not advise parents to continue to administer medication needed or it may lead to an escalation in medication dose or change to another class of medication that may have more side effects.

In summary, it is very important for clinicians to have tools to assess the extent to which medication is effective, allowing for proper titration, and ability to decide when there is a need for change in classes of medication. And lastly, by having clearer measures, the clinician is able to demonstrate to the parents and the child the degree of changes in child’s performance on and off medication and thereby greatly increasing compliance.