Abstract
Objective
We conducted a cost-analysis of the behavioral, pharmacological, and combined interventions employed in a sequential, multiple assignment randomized, and adaptive trial investigating the sequencing and enhancement of treatment for ADHD children (Pelham et al., under review; N=152, 76% male, 80% Caucasian).
Methods
The quantity of resources expended on each child’s treatment was determined from records that listed the type, date, location, persons present, and duration of all services provided. The inputs considered were the amount of physician time, clinician time, paraprofessional time, teacher time, parent time, medication, and gasoline. Quantities of these inputs were converted into costs in 2013 USD using national wage estimates from the Bureau of Labor Statistics, the prices of 30-day supplies of prescription drugs from the national Express Scripts service, and mean fuel prices from the Energy Information Administration.
Results
Beginning treatment with a low-intensity regimen of behavior modification (group parent training) was less costly for a school-year of treatment ($961) than beginning treatment with a low dose of stimulant medication ($1689), regardless of whether the initial treatment was intensified with a higher “dose” or if the other modality was added.
Conclusions
Outcome data from the parent study (Pelham et al., under review) found equivalent or superior outcomes for treatments beginning with low-intensity behavior modification compared to intervention beginning with medication. Combined with the present analyses, these findings document that initiating treatment with behavior modification rather than medication is the more cost-effective option for children with ADHD.
Keywords: ADHD treatment, cost, cost-effectiveness
Attention deficit hyperactivity disorder (ADHD) is one of the most prevalent (CDC, 2010), impairing (Fabiano et al., 2006) costly (Pelham, Foster, & Robb, 2007), and refractory (Barkley, Murphy & Fisher, 2008) mental health disorders of childhood. Over the past 50 years, a multitude of studies have been conducted on the etiology, nature, treatment, and outcomes of ADHD. One of the most central questions regarding ADHD has long been what constitutes effective treatment. For fifty years, the most common intervention has been medication with a central nervous system stimulant, and many hundreds of studies have documented its short-term effectiveness (Paykina, Greenhill, and Gorman, 2007). Over the past twenty-five years, behavioral treatment has become the second most commonly used intervention for ADHD, and more than 150 studies have documented its short-term effectiveness (Pelham, Wheeler, & Chronis, 1998; Pelham & Fabiano, 2008; Evans, Owens, & Bunford, 2013). These interventions and especially the combination of the two have become the most often recommended treatments for ADHD by leading associations and governmental entities (e.g., Subcommittee on Attention-Deficit/Hyperactivity Disorder, Steering Committee on Quality Improvement and Management, 2011; http://www.chadd.org/; http://www.nimh.nih.gov/health/topics/attention-deficit-hyperactivity-disorder-adhd/index.shtml).
Given that there is widespread agreement that medication and behavioral treatment are effective, it is surprising that comparative studies of treatment effectiveness (e.g., pharmacological vs. behavioral vs. combined) are relatively few in number (cf. MTA Cooperative Group, 1999, for one prominent example, and Fabiano et al., 2007, and Pelham et al., 2014, for more recent examples). Even more important are studies that mimic the adaptive process that clinicians employ in treating ADHD—that is, deciding on a starting treatment, monitoring progress, and adding other treatment when necessary (e.g., more intensive dose of initial intervention or the other modality). Studies of this type are necessary to make decisions about treatments that are effective and useful in the practice world (Murphy, 2005). We employed a sequential, multiple assignment, randomized trial of behavioral and pharmacological treatment for ADHD, investigating whether it is better to initiate treatment with behavioral intervention or medication, and whether, in the case of insufficient response, it is more effective to intensify the initial intervention modality or to add the other modality to yield a combined intervention. The methods and results of that paper are reported elsewhere (Pelham et al., under review). Herein, we evaluate the economic costs and cost-effectiveness of the interventions employed in that trial—the first to our knowledge using this design.
Economic evaluation of treatment costs is important in healthcare. Effective interventions must not only be evidence-based but also must be sustainable in the settings in which they are implemented (Glasgow, Lichtenstein, and Marcus, 2003). To be sustainable, interventions must be affordable to those funding them. Determining the affordability of an intervention requires measurement of intervention delivery costs. An additional aim of economic evaluation is to identify interventions that represent worthwhile investments for society. Cost-benefit and cost-effectiveness analyses are used to determine which interventions produce the greatest outcome change for the resources spent on them (Gold, Siegel, Russell, & Weinstein, 1996; Drummond, O’Brien, Stoddard, & Torrance, 1997). If one intervention is more effective than the other and also more costly, this indicates a trade-off between the additional outcome improvement and additional costs. If two interventions are equally effective, the less costly one is preferred. If one intervention is less costly and also more effective, it strictly dominates the alternative (Gold et al. 1996; Van Hout, Gordon, and Rutten 1994). The Patient Protection and Affordable Care Act of 2010 has placed greater emphasis for federal funding on comparative effectiveness research in evaluating the outcomes of interventions and a new agency has been created to fund this type of research, the Patient Centered Outcome Research Institute (http://www.PCORI.org). Treatment costs have been identified as an important outcome to study as part of the comparative effectiveness research process (Garber & Sox, 2010).
In a review of the economic costs of ADHD, Pelham et al. (2007) found 13 studies with information on the cost of ADHD, with only a small subset including information on costs of treatment. The small number of available studies on treatment costs were from large databases (e.g., Medicaid) that tied clinic-recorded diagnoses to medical costs—typically those of medication. No information was available regarding psychosocial treatment costs, no comparative costs were available, and no information was available on cost-effectiveness.
To our knowledge, the only study that has provided information on comparative costs/cost-effectiveness of treatment for ADHD is the MTA study (Foster et al., 2007; Jensen et al., 2005; MTA Cooperative Group, 1999). Adopting a societal perspective (excluding possible time lost for work by parents), Jensen et al. found that over the 14-month duration of the study, the per-child cost (in 2000 dollars) of medication management was least expensive ($1,180), followed by intensive behavioral treatment ($6,988), followed by these two treatments combined ($7,827). Normalization rates in the study had previously been reported to favor medication (Swanson et al., 2001). Putting the two reports together, medication management was found to be more cost-effective than intensive behavioral treatment (i.e., it was less expensive and more effective). Combination treatment was found to be more expensive and marginally more effective than medication management. Foster et al. (2007) examined the same data to determine if the presence of a comorbid disorder affected the cost-effectiveness results. Adopting a payer perspective, the authors concluded that for patients with only an ADHD diagnosis (32% of the sample), medical management appeared to be the most cost-effective treatment. However, they concluded that for patients with a comorbid disorder, the remaining 68%, when willingness to pay is high, combined treatment might be most cost-effective.
The MTA cost study has several limitations. First, the MTA employed a highly intensive and costly behavioral treatment that is not typical of community practice. Second, since the time of that study, far more costly long-acting stimulant medication preparations have become the norm rather than the inexpensive, immediate-release, generic methylphenidate used in 1980’s and 1990’s, including the MTA study. Third, the combined treatment arm began both treatments simultaneously and all children received the same high dose/high intensity pharmacological and behavioral interventions regardless of need. An updated study of comparative costs for treatments of ADHD is needed to address these limitations.
The purpose of this study was to compare the treatment costs of a sequential, multiple assignment, randomized trial of study of medication and behavioral interventions for children with ADHD (Pelham et al., under review). These cost estimates can then be combined with outcome measures reported in the original study to determine the most cost-effective treatment strategy: (1) beginning treatment with a low “dose” of behavior modification or with a low dose of medication and (2) in the case of insufficient response, increasing the dosage of the starting modality or augmenting treatment by adding the other modality.
Methods
Participants
One hundred, fifty-two children with ADHD (76% male, 80% Caucasian), between the ages of 5 and 12, from western New York participated in this investigation. Participants were recruited in three annual cohorts of approximately 50 each for treatment lasting one school year. Diagnoses were made using a set of instruments standardly employed for making DSM IV diagnoses of ADHD (i.e., parent and teacher rating scales; parent structured and clinical interviews). A description of the study sample is available in Pelham et al. (under review). A total of 146 children completed the study assessments.
Design and Procedure of Treatment Study
This study employed an adaptive treatments design with multiple randomization points to evaluate different treatment strategies in a way that mimics clinical practice. The design is described in detail in the main outcome paper (Pelham et al., under review), but we review it here briefly (Figure 1).
Figure 1.
Study Design
Treatment algorithm
Figure 1 illustrates the study design. Initially, participants were randomly assigned to one of two initial treatments for 8 weeks: low-dose medication (an 8-hour stimulant for school hours equivalent to .15 mg/kg immediate release methylphenidate b.i.d.; Medication First [MedFirst]) or low-intensity clinical behavioral intervention consisting of 8 weekly sessions of group behavioral parent training and three consultation meetings with each child’s primary teacher at school to establish a school-home daily report card (Behavior First [BehFirst]). These conditions were selected based on controlled studies (Fabiano et al., 2007; Pelham et al., 2005; Pelham et al., 2014) showing that a substantial number of children demonstrated response to these low-intensity/low dose treatments. After eight weeks, if response to treatment was insufficient, children were re-randomized to one of two treatment strategies: (1) increase the dose/intensity of the initial treatment or (2) add the other treatment modality to yield a combined treatment condition. After this second randomization, children’s progress was evaluated monthly and each child’s additional treatment was tailored adaptively according to specific domains and settings of impairment. Table 1 specifies the components that were available in each treatment modality. However, treatment was provided on an as-needed, domain-specific basis, so the components listed represent the menu of treatment options, rather than what every child received.
Table 1.
Intervention Description
| Modality | Initial Treatment | Secondary/Adaptive Treatment |
|---|---|---|
| Medication |
|
|
| Behavior Modification |
|
|
Note. The adaptive components listed represent those offered or recommended as needed based on individual areas of impairment. Not every child received every treatment.
Assessing response to treatment
Teachers maintained a daily individualized target behavior evaluation (ITBE; Pelham, Fabiano, & Massetti, 2005) to serve as an objective measure of treatment response. In addition, each month parents and teachers completed a modified version of the Impairment Rating Scale (IRS; Fabiano et al., 2006) that asked the rater whether, given the treatment currently in place, he or she believed the child needed additional treatment. These data were reviewed monthly by the clinical team (Pelham et al., under review). Two clinicians who were not clinically involved in the case and were uninformed of the initial treatment condition were required to agree regarding the need for additional treatment. Children who showed sufficient response to the existing treatment were maintained on that treatment and monitored monthly; whenever deterioration occurred, additional treatment was provided according to the child’s group assignment.
Economic Analysis
Procedure
The quantity of intervention inputs over the 10-month (school-year) course of treatment was determined from extensive records of medication and psychosocial services delivered for each child, and the costs of these inputs were computed and summed for each child. These logs contained information on every instance of treatment each child received, including type, date, location, persons present, and duration. Only treatments that were actually delivered were counted; that is, if a treatment was prescribed but parents did not attend a session or medication visit, the visit was not counted. The value of each input, such as medication costs and hourly wages of personnel used for delivering the intervention, were obtained from publicly available data sources. These records were combined with the addresses of participants’ homes, schools, and clinic to calculate the time that clinicians, paraprofessionals, and parents spent commuting to and from treatment, as well as the miles traveled. Thus, for each child the amount of physician time, clinician time, paraprofessional time, teacher time, parent time, medication, and gasoline expended were computed.
The possible treatment program components were group, individual, and booster parent training; group, individual, and booster social skills training; school and home visits by the therapist; physician visits; tutoring sessions; a Saturday social skills program; direct, child-based interventions at school (e.g., the Good Behavior Game; Barrish, Saunders, & Wolf, 1969); and treatment planning. In the case of services delivered in a group setting, the time spent per child was adjusted to account for this. For example, group behavioral parent training was typically delivered to 16 parents at once, and the time of the clinician leading each session was divided up based on attendance. Thus, if a 90-minute session was attended by two families, each family received 45 minutes of clinician time; if the same session was attended by 10 families, each family received nine minutes of clinician time. If both parents attended a session, therapist time was allocated per family while the time of both parents was counted separately. In the case of a Saturday program for social skills training, a component of the high-intensity psychosocial treatment, the Center for Children and Families’ per-child enrollment cost of $200 was added for each of the subjects that participated at any point.
Providers involved in delivering the intervention included physicians, Ph.D.- and MA-level clinicians, paraprofessionals, teachers, and parents. The mean hourly wages of psychiatrists, psychologists, mental health counselors, paraprofessionals, elementary school teachers, and parents were taken from the Bureau of Labor Statistics’ (BLS) Occupational Employment Statistics (OES) 2011 survey (U.S. Department of Labor, 2012). This survey reports only the annual salary of teachers, so teacher salaries were adjusted to hourly wages assuming 1,394 hours worked per year (U.S. Department of Labor, 2011). The paraprofessional salary was approximated using the salary paid to research assistants who worked on the grant. This process resulted in the following hourly wages in 2013 USD: psychiatrist ($85.72), doctoral psychologist ($35.97), mental health counselor ($20.97), paraprofessional ($11.83), teacher ($40.92), and parent ($22.26). It should be noted that although a child psychiatrist provided treatment in this study, the hourly wage for a pediatrician would be nearly identical ($83.01). These figures were then scaled up to reflect full compensation including fringe benefits, where nominal wages were assumed to represent 69.2% of total compensation, per the Bureau of Labor Statistics’ economy-wide estimate (U.S. Department of Labor, 2013). The parent wage was exempted from this adjustment, as missed time from work does not entail a loss of fringe benefits. To obtain a nationally-representative estimate of medication costs, medication was valued using 30-day supply prices from a large, national pharmacy benefit manager, Express Scripts (http://www.express-scripts.com).
Lastly, the number of miles traveled for clinician visits to schools and parent visits to the clinic was converted to gallons of gasoline using an average fuel efficiency of 23.0 mpg (U.S. Department of Transportation, 2012). Gasoline was valued at $3.14 per gallon after adjusting for inflation the average cost of gas in New York State of $2.94 over the duration of the study (Energy Information Administration, 2013). Using these data, total intervention costs were computed for each child.
Cost Analyses
The analyses were performed two ways. The primary set of estimates included only direct costs attributable to the interventions. However, given that parent training and physician visits involved sessions in the clinic and required parental time, secondary analyses included the implicit costs to parents, including the value of their time, calculated using the “all occupation” wage estimate from the Bureau of Labor Statistics (2011). Parent training groups were always held on weekday evenings. Consistent with the analytic plan outlined in Pelham et al. (under review; see also Nahum-Shani et al., 2012 for details and examples of the analysis strategy), treatment costs were compared for initial randomized condition (Behavior Modification First or Medication First) and various combinations of rerandomized (contingent upon response to initial treatment) conditions. The average treatment cost was simply the sum of enumerated cost categories described above divided by the number of children.
First, average treatment costs per child were compared according to their initial random assignment into the MedFirst or BehFirst condition. This analysis evaluated whether it is less costly to begin treatment with medication or behavior modification. Next, comparisons were made for the two adaptive treatment strategies between original arms: increasing the amount of the initial treatment or adding the alternative modality, as described in the main outcomes paper (Pelham et al., under review). Although adaptive treatments were determined and administered separately for home and school settings, costs could not be easily separated by domain (e.g., additional parent training sessions may be held if teachers report impairment related to homework completion), so for these analyses the only subjects labeled as “responders” were those who never required additional treatment in either home or school.
Analysis Overview
Our analyses largely parallel those described by Nahum-Shani and colleagues (2012); we direct readers to that article and to the main outcomes article (Pelham et al., under review) for more details. The analysis of costs included a series of comparisons to test different treatment paths and protocols. Each comparison is described below.
Main effect of first-stage treatment
First, treatment costs for those that started with medication (MedFirst group) and those that started with behavioral treatment (BehFirst group) were compared. Regressions with group membership as predictors yielded comparisons of MedFirst and BehFirst on each cost measure.
Pairwise comparisons among SMART-embedded treatment protocols
Second, costs were compared across each of the four treatment protocols naturally embedded in the SMART design—BB, BM, MB, and MM. The first letter denotes that protocol’s first-stage treatment and the second letter denotes that protocol’s second-stage treatment, to be implemented in the event of nonresponse to the first-stage treatment. For example, the BM protocol entailed starting the participant with behavioral treatment and then adding medication if and only if there was nonresponse. It is important to note that the protocols do not reflect the actual treatment received, but rather the set of rules followed. For example, a participant that began with behavioral treatment, responded, and was never re-randomized is consistent with both the BM and BB protocols, and thus should be included in both groups for the analyses. This idea of being consistent with a particular embedded protocol is a subtle but important aspect of the SMART design that is discussed in detail elsewhere. We used an effects coding scheme and generalized estimating equations to achieve all the pairwise comparisons of protocols in a single model using SAS PROC GENMOD, as described in the appendices of Nahum-Shani et al. (2012). We also gave weights of 2 to the responders to first-stage treatment and weights of 4 to the insufficient responders in order to account for the systematic undersampling of the latter in each protocol due to the second re-randomization (see Nahum-Shani et al., 2012).
Best response given insufficient response to first-stage treatment
Third, supplemental comparisons were performed within each of the initial treatment arms to determine if, given insufficient response to an initial intervention, it is less costly to augment (i.e., increase the dose of) that treatment or add the other treatment. These analyses consisted of regressions with group membership dummy codes that compared (1) BB versus BM and (2) MM versus MB. Responders to the first-stage treatment were excluded from these supplemental comparisons.
Sensitivity
The purpose of a sensitivity analysis is to determine the extent to which the conclusions reached in the baseline analyses are sensitive to variability in model parameters, assumptions, or outliers (Drummond et al. 2005). Two sets of sensitivity analyses were thus performed. First, the top 10 percent in terms of total cost were dropped from the sample in order to determine if group differences were being driven by a small number of high-cost cases. Given the adaptive nature of the treatment upon second randomization, a small number of children received a very large amount of services relative to the initial dose of treatment (e.g., a large number of individual parent-training sessions; more frequent physician visits and movement to higher-cost, 12-hour medication formulations). Second, the wage of a doctoral psychologist ($35.97 per hour) was used for all clinician time instead of the wage for a mental health counselor. The assigned clinicians in the study were predominately masters-level clinicians, but the only previous study of comparative costs for ADHD, the MTA, employed exclusively doctoral-level clinicians, so this sensitivity analysis facilitates a comparison between the results of the two studies. For all of the comparisons discussed above, two-group, t-tests were performed to determine the statistical significance of observed differences in treatment costs.
Results
Comparison of Costs Based on Initial Assignment
Cost estimates based on initial treatment assignment are reported in Table 2. As shown in the Table, beginning treatment in behavior modification was significantly less costly than beginning treatment with medication—with and without implicit parental costs included. Predictably, medication and physician costs were higher for those starting with medication, and clinician, teacher, parent, and gasoline costs were higher for those starting with behavior modification.
Table 2.
Costs Based on Initial Treatment Assignment
| Cost | Medication (N = 73) | Behavior (N = 72) |
|---|---|---|
| Medication*** | 1,063 (465 [0, 2111]) | 300 (465 [0, 1593]) |
| Physician*** | 249 (117 [0, 567]) | 87 (109 [0, 381]) |
| Clinician*** | 203 (201 [0, 1018]) | 321 (158 [109, 737]) |
| Paraprofessional | 54 (200 [0, 1148]) | 75 (186 [0, 1116)] |
| Teacher* | 121 (115 [0, 675]) | 178 (147 [46, 817]) |
| Parents*** | 296 (330 [0, 2031] | 694 (315 [63, 1409]) |
| Gasoline* | 37 (41 [3, 283]) | 51 (43 [4, 207]) |
|
| ||
| Total Cost*** | 1,689 (720 [134, 4891]) | 961 (641 [191, 2895]) |
| Total Cost IPG* | 2,022 (959 [168, 6138]) | 1,706 (836 [609, 4435]) |
Note. IPG = including parent and gasoline costs. In 2013 USD. Standard deviation and range are reported in parentheses.
p < .05,
p < .01,
p < .001.
Differences Between Treatment Protocols Followed
Pairwise tests were used to compare costs for the various adaptive treatment strategies that were included in the study, according to the procedures outlined above. Results are illustrated in Table 3. Costs for individual intervention components were as expected, with physician and medication costs higher for the strategies including medication and clinician, paraprofessional, teacher, parents, and gasoline higher for the strategies employing more intensive behavioral treatments. Pairwise tests further illustrate that strategies beginning with behavioral treatments were less costly when parent costs were not included (Table 3). That is, BB was significantly less costly than MM or MB, and BM was significantly less costly than MB or MM. These differences were driven mostly by medication costs.
Table 3.
Costs by Treatment Protocol Followed
| Outcome | BB protocol | BM protocol | MB protocol | MM protocol | Significant Pairwise Tests |
|---|---|---|---|---|---|
| Medication | 73 (34) | 527 (86) | 921 (61) | 1,205 (83) | BM<MB*** BB<MM*** BB<BM*** BM<MM*** BB<MB*** MB<MM** |
| Physician | 21 (8.7) | 153 (17) | 215 (17) | 283 (20) | BM<MB** BB<MM*** BB<BM*** BM<MM*** BB<MB*** MB<MM** |
| Clinician | 402 (27) | 241 (15) | 317 (37) | 89 (7.4) | BB>MM*** BB>BM*** BM>MM*** MB>MM*** |
| Paraprofessional | 135 (41) | 15 (5.0) | 108 (45) | 0 (0) | BM<MB* BB>MM*** BB>BM** BM>MM** MB>MM* |
| Teacher | 232 (27) | 124 (15) | 164 (23) | 78 (7.4) | BB>MM*** BB>BM*** BM>MM** MB>MM*** |
| Parents | 783 (49) | 605 (48) | 430 (68) | 161 (13) | BM>MB* BB>MM*** BB>BM** BM>MM*** BB>MB*** MB>MM*** |
| Gasoline | 58 (7.6) | 45 (6.2) | 50 (8.2) | 24 (3.0) | BB>MM*** BM>MM*** MB>MM** |
|
| |||||
| Total Cost | 862 (95) | 1,060 (109) | 1,725 (135) | 1,654 (98) | BM<MB*** BB<MM*** BM<MM*** BB<MB*** |
| Total Cost IPG | 1,702 (130) | 1,710 (140) | 2,205 (189) | 1,840 (108) | BM<MB* BB<MB* |
Note. Values are estimated weighted means with robust standard errors in parentheses. IPG = including parent costs and cost of gasoline. All figures are in 2013 USD.
p < .05,
p < .01,
p < .001.
When indirect costs to parents were included, fewer comparisons were significantly different, presumably due to the increased parental time required to attend behavioral parent training classes. However, the BM strategy remained significantly less costly than the MB strategy, while the MB strategy was more costly than BB.
Comparisons of Costs for Insufficient Responders to First-Stage Treatments
Cost comparisons between the four adaptive treatment endpoint treatments given insufficient response to initial treatment are reported in Table 4. Although expected differences were found in individual cost categories, total costs were not significantly different for the second-stage treatments, regardless of whether parent costs were included.
Table 4.
Costs for Second-Stage Treatments Given Non-Response to Initial Treatment
| Nonresponse to Initial Behavior Modification
| ||
|---|---|---|
| Cost | B-then-B (N = 32) | B-then M (N = 32) |
| Medication*** | 82 (215 [0, 832]) | 593 (537 [0, 1593]) |
| Physician*** | 23 (56 [0, 237]) | 172 (102 [0, 381]) |
| Clinician*** | 417 (170 [185, 737]) | 236 (97 [109, 479]) |
| Paraprofessional** | 150 (260 [0, 1116]) | 15 (31 [0, 131]) |
| Teacher*** | 249 (173 [47, 817]) | 127 (99 [46, 529]) |
| Parents* | 811 (307 [290, 1407]) | 611 (303 [63, 1409]) |
| Gasoline | 61 (48 [9, 207]) | 46 (39 [4, 151]) |
|
| ||
| Total Cost | 920 (597 [240, 2639]) | 1,144 (681 [191, 2895]) |
| Total Cost IPG | 1,793 (811 [696, 3952]) | 1,801 (878 [609, 4435]) |
| Nonresponse to Initial Medication
| ||
|---|---|---|
| Cost | M-then-M (N = 34) | M-then-B (N = 34) |
| Medication** | 1,221 (523 [0, 2111]) | 917 (388 [0, 1794]) |
| Physician* | 286 (125 [0, 556]) | 213 (107 [0, 567]) |
| Clinician*** | 83 (33 [0, 163]) | 328 (230 [41, 1018]) |
| Paraprofessional* | 0 (0 [0,0]) | 116 (283 [0, 1148]) |
| Teacher*** | 79 (47 [0, 267]) | 171 (147 [50, 675]) |
| Parents*** | 158 (79 [0, 339]) | 446 (431 [90, 2031]) |
| Gasoline** | 22 (16 [4, 78]) | 50 (52 [3, 283]) |
|
| ||
| Total Cost | 1,669 (620 [114, 2801]) | 1,745 (853 [351, 4891]) |
| Total Cost IPG | 1,850 (687 [168, 3172]) | 2,242 (1196 [575, 6138]) |
Note. M = medication, B = behavior modification, IPG = including parent and gasoline costs. In 2013 USD. Standard deviation and range are reported in parentheses.
p < .05,
p < .01,
p < .001.
Costs for Responders to Initial Treatments
Behavior modification was significantly less costly than medication for the small number of children who responded to initial treatment and were not re-randomized at any point based on either home or school functioning due to insufficient response (N = 5 for medication and N = 8 for behavior modification). Excluding parent costs, average treatment cost per child was $392 for BehFirst and $1,448 for MedFirst (p < 0.001). When parent costs were included in secondary analyses, the difference remained statistically significant. Per child costs were $976 for BehFirst and $1,701 for MedFirst (p< 0.001).
Sensitivity Analyses
Excluding top 10% of total costs
As Tables 2, 3, and 4 show, the ranges of cost are very large, with some children having very high costs of treatment. To determine whether the results were being driven by a small number of children who received a large amount of services, the analyses were repeated excluding children in the highest 10 percent of total costs (Tables 5–6). This results in 15 children being excluded from the sample. Of these, seven were located in the MB condition, four were located in the BB condition, three were located in the BM condition, and one was located in the MM condition. In all of these analyses, the general pattern of results was similar to the initial analyses but the costs for behavioral treatments were reduced, magnifying the differences obtained in the original comparisons and increasing the significance levels of the cost advantage of BehFirst. This reflects the fact that a small number of children received extensive, adaptive behavior modification services, driving up average costs for the entire group.
Table 5.
Sensitivity Analyses for Initial Treatment Assignment.
| Medication First (N=73) | Behavior First (N=72) | ||
|---|---|---|---|
| Excluding top 10% | Total cost*** | 1,546 (551 [2478]) | 824 (497 [2025]) |
| Total cost IPG* | 1,788 (622 [2906]) | 1,521 (624 [2870]) | |
| Psychologist Wage | Total cost*** | 1,833 (788 [5390]) | 1,188 (682 [3233]) |
| Total cost IPG | 2,165 (1054 [6637]) | 1,933 (893 [4772]) |
Note. IPG = including parent and gasoline costs. In 2013 USD. Standard deviation and maximum are reported in parentheses.
p < .05,
p < .01,
p < .001
Table 6.
Sensitivity Analyses for Pairwise Comparisons of Treatment Protocols
| Outcome | BB protocol | BM protocol | MB protocol | MM protocol | Significant Pairwise Tests |
|---|---|---|---|---|---|
| Excluding Top 10% | 701 (63) | 945 (96) | 1,454 (92) | 1,622 (95) | BM<MB*** BB<MM*** BB<BM* BM<MM*** BB<MB*** |
| Excluding Top 10% IPG | 1,494 (96) | 1,547 (114) | 1,772 (111) | 1,802 (105) | BB<MM* BB<MB* |
| Psychologist Wage | 1,145 (107) | 1,230 (113) | 1,949 (152) | 1,717 (98) | BM<MB*** BB<MM*** BM<MM*** BB<MB*** |
| Psychologist Wage IPG | 1,985 (143) | 1,880 (145) | 2,428 (210) | 1,902 (108) | BM<MB* MB<MM* |
Note. Values are estimated weighted means with robust standard errors in parentheses. IPG = including parent costs and cost of gasoline. All figures are in 2013 USD.
p < .05,
p < .01,
p < .001.
Employing doctoral psychologists vs. masters-level clinicians
A final set of sensitivity analyses assumed that a psychologist was used to deliver all clinician services rather than a less expensive mental health counselor. Because clinicians were the providers in the conditions involving behavior modification, costs for treatment endpoints that included more extensive behavior modification increased by more than those that relied less on behavior modification. Few differences in significance emerged in these analyses, although costs were relatively higher; however, treatment costs based on initial assignment were no longer significantly different when parent costs were included (costs were $2,165 for the medication arm and $1,933 for the behavior modification arm, Table 5).
Discussion
The objective of this study was to measure the costs of different treatment strategies and sequences for children with ADHD. Overall, beginning treatment with a low intensity regimen of behavior modification was less costly than beginning treatment with a low dose of stimulant medication, regardless of whether adaptive additions to intervention for inadequate responders to initial intervention involved increasing the dose of initial modality or adding the other modality and regardless of whether implicit parental costs were considered. Using pairwise tests of the different treatment protocols, significant results always favored the treatment protocols beginning with behavioral treatment when implicit costs to parents were excluded. Sensitivity analyses were performed to determine the impact of the most costly children. The majority of those in the top 10 percent of costs were located in a treatment endpoint that involved adaptive and often intensive behavior modification for insufficient responders. Analyses were also repeated assuming that a doctoral level psychologist, rather than a masters level mental health counselor, delivered clinical services. Although costs increased more for treatment options that used behavior modification, treatments that began with behavior modification in a relatively low standard “dose” were nonetheless less costly than those that began with medication. The implications of these findings are discussed in detail below.
The finding that treatments involving behavior modification were either less costly or not significantly different from those involving medication may be surprising to some, particularly when compared with the cost results of the MTA (Jensen et al., 2007). The MTA concluded that medication was far less costly and more cost-effective than behavioral treatments, with combined intervention having higher costs than either but only marginally better results than medication alone. However, the present study differed from the MTA in several ways that affect the cost outcomes. First, at the time of the MTA, the current generation of extended-release ADHD medications had not yet been developed, and thus immediate-release, generic methylphenidate was the medication typically prescribed and utilized in the MTA study. In contrast, today and at the time of the present study, nearly all children with a stimulant prescription take 8–12 hour extended-release formulations, which we employed in the study and which are at current prices (http://www.express-scripts.com) 7–11 times more expensive than the immediate release methylphenidate prescribed in the MTA study. In addition, the behavioral treatment provided in the MTA was far more intensive than that employed in the present study and employed doctoral level clinicians. For example, it involved 35 sessions of behavioral parent training—27 group sessions (with six families per group) and 8 individual parent sessions, an average of 14 consultations per child with the child’s regular school teacher and a half time classroom aide in the child’s regular school classroom for 12 weeks, and an intensive 8-week summer treatment program focused on improving peer relationships (MTA Cooperative Group, 1999). All subjects assigned to behavioral treatment—even those who concurrently took medication—received the intensive intervention regardless of need. In contrast, in the current treatment study, initial behavioral treatment involved a low dose of behavior modification—8 group parenting sessions with concurrent social skills groups for the children (with an average of 16 parents per group) and 3 teacher consultations to establish a school-home Daily Report Card, with interventions provided by masters level clinicians. Additional, more intensive behavioral treatment was provided only adaptively where need was indicated due to insufficient response to the initial dose of treatment. These differences in treatment protocol and medication costs resulted in far lower costs for behavioral treatment and far higher costs for medication in the present study compared to the MTA, especially, as the sensitivity analyses showed, when the children in the top 10% of costs were excluded. Further, our analysis of normalization rates for the different treatments (see companion paper) used procedures comparable to those employed in the MTA, and showed comparable rates of normalization sequences and costs of all conditions that involved behavioral intervention, including initial treatment assignment as well as protocols that involved adding medication as the second stage treatment. Thus, from multiple perspectives we believe that we may conclude for the type of behavioral and pharmacological treatments employed in the current study, those beginning with behavior modification or including behavior modification as a second stage treatment were less costly and equally or more effective than those beginning with medication (see following discussion of implicit parental costs).
When the cost of parents’ time and gasoline were included, BB and BM protocols were no longer less costly than MM. Whether or not parent costs should be included in an analysis of treatment costs depends on the perspective from which the analysis is being conducted. For example, if costs were being measured to determine how much an insurer would have to reimburse for treatment, parent costs would be excluded, as the value of parent time is not reimbursed by insurance companies. However, if a societal perspective is adopted that considers the overall cost of treatment, including explicit and implicit costs, the value of parent time might be important to include when treatments differ in the amount of parent time required. That is, if two treatments produced equal outcomes, but one required substantially more effort from the parents, the treatment that did not put high demands on the parents’ time might be preferred. On the other hand, if parents participate in parent training on a voluntary basis and if parent training leads to more positive outcomes, parents may be willing to give their time to the endeavor. If parent training leads to more positive longer term outcomes, an insurer might be more willing to pay for intervention that includes parent training (cf. Foster et al., 2007).
The results reported in Pelham et al. (under review) document that treatments beginning with behavior modification were equally or more effective depending on the outcome measure than treatments beginning with medication. MedFirst was not superior on any measures. These results have implications for the cost-effectiveness of each treatment option. A detailed cost-effectiveness analysis would be needed if treatments beginning with behavior modification are more costly than treatments beginning with medication. In this case, a cost-effectiveness analysis would compute the incremental cost-effectiveness ratio (ICER) to determine the additional cost per unit of outcome improvement (e.g., additional cost to normalize a child) for the more expensive treatment. However, if treatments beginning with behavior modification are less costly than treatments beginning with medication, then they would by definition be more cost-effective given that outcomes were equivalent or better. In this case, the ICER would be negative, indicating that treatments beginning with behavior modification are cost-saving and therefore strictly dominate treatments beginning with medication. In this case, detailed analysis of cost-effectiveness is not needed to make a recommendation of the preferred treatment approach (Gold et al. 1996; Van Hout, Gordon, and Rutten 1994).
A number of cost-saving strategies for treatment of ADHD were revealed by the analysis of costs in this study. Among initial responders, costs were significantly lower for those who began treatment with behavior modification. This suggests that a cost saving strategy would be to begin treatment for ADHD children with a low dose of behavior modification, a treatment algorithm that we have recommended previously (Pelham, 2008) and that has been recommended for all young children with ADHD by the American Academy of Pediatrics (Subcommittee on Attention-Deficit/Hyperactivity Disorder, Steering Committee on Quality Improvement and Management, 2011). This contrasts with the current practice guideline of the American Academy of Child and Adolescent Psychiatry (2007), which recommends multiple trials of medication as first lines of treatment, with behavioral interventions added only late in the recommended treatment strategy—an approach that our analysis suggests would maximize costs for ADHD treatments without improving outcomes. Second, providing large-group, parent training instead of individual parent training as the first line intervention, will result in lower costs for ADHD treatment with no apparent loss in effectiveness. Third, using lower cost personnel, such as masters level mental health counselors instead of doctoral level psychologists, to deliver the behavioral interventions results in lower costs for similar services. Mental health counselors and licensed clinical social workers provide most of the mental health services in the North America, and it has long been known that they produce effective results when implementing evidence-based interventions (e.g., Henggler, Melton, & Smith, 1992). This practice in mental health is consistent with the shift towards lower cost physician extenders, such as physician assistants and nurse practitioners, in the delivery of primary care (Everett et al., 2013), and our data add to other data documenting that trend is one that lowers healthcare costs for ADHD children without a loss of effectiveness.
Fourth, the sensitivity analyses revealed that adaptive treatment for insufficient responders to the initial dose was expensive and necessary for only a subset of the children. Reserving these more intensive interventions for children who have failed to show sufficient response to less intensive treatment should result in substantially lower total costs for treating ADHD at a population level (cf. MTA Cooperative Group, 1999). Although this was typically a relatively small increase in parent sessions or teacher contacts, many of the children with costs in the top 10 percent were located in a protocol that included adaptive behavior modification. More intensive adaptive behavioral treatments provided to insufficient responders typically included paraprofessional costs (e.g., daily visits to the classroom for several weeks, Saturday treatment program) or frequent individual sessions with parents that are far more costly than typical group parent training sessions. For example, one case included more than $1,100 in paraprofessional costs and $1,150 in parent individual sessions. However, the top 10 percent of cases were split evenly between MedFirst and BehFirst. The severity of child behavior, comorbid conditions, or family situations were more related to eventual costs of these high-cost individuals than was the initial treatment approach. Of note is that despite the high costs for these individuals, the overall costs for the BB protocol were still significantly lower than for the MM protocol.
Finally, with regard to medication treatment, it is notable that many children were maintained on a low dose of medication during school hours only for the duration of the school year. Carefully assessing medication needs across settings and providing medication only for the hours and doses needed will also reduce costs associated with pharmacological treatments.
This study had limitations—some related to the cost analysis and some related to the design and content of the interventions in the parent study. Hourly wages for personnel were national estimates obtained from publicly available sources. In the real world, wages, salaries, and fringe benefits vary across employers and geographic areas. In addition, the design included monthly physician visits, whereas families may not see their primary care providers with the same frequency (although physician time was only a small portion of the cost of the pharmacotherapy). Overhead costs were not considered, because the interventions employed in this study used existing facilities and therefore did not result in any additional overhead costs.
Our results are necessarily limited to the low dose of initial behavioral intervention that we employed and the targeted but relatively unlimited access to more intensive behavioral treatment for participants who needed more. A higher initial dose (e.g., a few individual parenting sessions, a higher starting dose of medication, more booster parenting sessions), more rapid inclusion or modifications of the most intensive behavioral interventions, or beginning with combined treatment might have led to lower overall costs and/or different degrees of effectiveness and conclusions. Reliance on parent ratings of need for additional treatment in the home setting may have overestimated the need for treatment in that setting, as a single rating of impairment may have triggered a decision to increase treatment. On the other hand, our approach approximated the information clinicians have available to them when making decisions regarding treatment response. As such, our results have clear implications for clinical practice in treatment of ADHD.
These results have important implications for the relative cost-effectiveness of the treatment approaches studied for ADHD. Similar studies are needed of other major childhood disorders (cf. Domino et al., 2008, 2009). Pelham et al. (under review) found that starting with behavioral treatment for ADHD produced more favorable outcomes on the primary outcome measure and more favorable or comparable outcomes on other measures. Given the fact that beginning treatment with behavior modification was also less costly, beginning treatment with a low dose of behavior modification is, by definition, the more cost-effective approach in treating ADHD children. Further, providing adaptive behavioral treatment was never more costly than medication and sometimes less costly. Since we employed a design that modeled how a clinician would have to select, sequence, and titrate treatments, these results more closely mimic the cost in community settings than previous comparative cost analyses of ADHD treatments.
Currently, medication is far more likely than behavioral treatment to be recommended and used as a first line treatment for ADHD, despite its greater cost, and this approach maximizes the societal cost of ADHD treatments. Our findings together suggest strongly that if current treatment guidelines and parameters for childhood ADHD recommended behavioral treatments as first- and even second-line interventions, the societal cost of treating ADHD could be reduced without a loss of effectiveness. Given the high prevalence and high societal cost of ADHD, as well as the current need to reduce the cost of healthcare in the U.S.A., such a shift should be welcomed.
Acknowledgments
This research was funded by a grant from the Institute of Education Sciences (R324B060045). Dr. Pelham was also supported in part by grants from the National Institute of Mental Health (MH069614, MH069434, MH092466, MH53554, MH065899, MH62988), the Institute of Education Sciences (R324J060024, LO30000665A), the National Institute of Alcohol Abuse and Alcoholism (AA11873), and the National Institute on Drug Abuse (DA12414, DA12986).
Contributor Information
Timothy F. Page, Email: tpage@fiu.edu.
Gregory A. Fabiano, Email: fabiano@buffalo.edu.
Andrew R. Greiner, Email: agreiner@fiu.edu.
Elizabeth M. Gnagy, Email: egnagy@fiu.edu.
William E. Pelham, III, Email: wpelhami@fiu.edu.
Katie Hart, Email: khart@fiu.edu.
Stefany Coxe, Email: scoxe@fiu.edu.
James G. Waxmonsky, Email: jwaxmonsky@hmc.psu.edu.
William E. Pelham, Jr., Email: wpelham@fiu.edu.
References
- Barkley RA, Murphy KA, Fischer M. ADHD in Adults: What the Science Says. New York, NY: The Guilford Press; 2008. [Google Scholar]
- Barrish HH, Saunders M, Wolf MM. Good behavior game: Effects of individual contingencies for group consequences on disruptive behavior in a classroom. Journal of Applied Behavior Analysis. 1969;2:119–24. doi: 10.1901/jaba.1969.2-119. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Centers for Disease Control and Prevention. Increasing prevalence of parent-reported attention-deficit/hyperactivity disorder among children—United States, 2003 and 2007. Morbidity and Mortality Weekly Report. 2010 Nov 12;59(44):1439–1443. [PubMed] [Google Scholar]
- Domino M, Burns B, Silva S, Kratochvil C, Vitiello B, Reinecke M, … March J. Cost-effectiveness of treatments for adolescent depression: results from TADS. American Journal of Psychiatry. 2008;165(5):588–596. doi: 10.1176/appi.ajp.2008.07101610. [DOI] [PubMed] [Google Scholar]
- Domino ME, Foster EM, Vitiello B, Kratochvil CJ, Burns BJ, Silva SG, … March JS. Relative cost-effectiveness of treatments for adolescent depression: 36-week results from the TADS randomized trial. Journal of the American Academy of Child & Adolescent Psychiatry. 2009;48(7):711–720. doi: 10.1097/CHI.0b013e3181a2b319. [DOI] [PubMed] [Google Scholar]
- Drummond M, Sculper MJ, Torrance G, O’Brien BJ, Stoddard GL. Methods for the Economic Evaluation of Health Care Programmes. 3. Oxford: Oxford University Press; 2005. [Google Scholar]
- Energy Information Administration. Gasoline and Diesel Fuel Update. 2013 Retrieved February 30, 2013, from http://www.eia.gov/petroleum/gasdiesel/
- Evans SW, Owens JS, Bunford N. Evidence-Based Psychosocial Treatments for Children and Adolescents with Attention-Deficit/Hyperactivity Disorder. Journal of Clinical Child & Adolescent Psychology. 2013;43:256–269. doi: 10.1080/15374416.2013.850700. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Everett CM, Thorpe CT, Palta M, Carayon P, Gilchrist VJ, Smith MA. Division of primary care services between physicians, physician assistants, and nurse practitioners for older patients with diabetes. Medical Care Research and Review. 2013;70:531–541. doi: 10.1177/1077558713495453. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fabiano GA, Pelham WE, Gnagy EM, Burrows-MacLean L, Coles EK, Chacko A, … Robb JA. The single and combined effects of multiple intensities of behavior modification and multiple intensities of methylphenidate in a classroom setting. School Psychology Review. 2007;36:195–216. [Google Scholar]
- Fabiano GA, Pelham WE, Waschbusch D, Gnagy EM, Lahey BB, Chronis AM, … Burrows-MacLean L. A practical impairment measure: Psychometric properties of the Impairment Rating Scale in samples of children with attention-deficit/hyperactivity disorder and two school-based samples. Journal of Clinical Child and Adolescent Psychology. 2006;35:369–385. doi: 10.1207/s15374424jccp3503_3. [DOI] [PubMed] [Google Scholar]
- Foster EM, Jensen PS, Schlander M, Pelham WE, Hechtman L, Arnold LE, … Wigal T. Treatment for ADHD: is more complex treatment cost-effective for more complex cases? Health Services Research. 2007;42(1p1):165–182. doi: 10.1111/j.1475-6773.2006.00599.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Garber AM, Sox HC. The role of costs in comparative effectiveness research. Health Affairs. 2010;29(10):1805–1811. doi: 10.1377/hlthaff.2010.0647. [DOI] [PubMed] [Google Scholar]
- Glasgow RE, Lichtenstein E, Marcus AC. Why don’t we see more translation of health promotion research to practice? Rethinking the efficacy-to-effectiveness transition. American Journal of Public Health. 2003;93:1261–1267. doi: 10.2105/AJPH.93.8.1261. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Gold MR, Siegel JE, Russell LB, Weinstein MC. Cost-effectiveness in Health and Medicine. New York: Oxford University Press; 1996. [Google Scholar]
- Henggler SW, Melton GB, Smith LA. Family preservation using Multisystemic Therapy: An effective alternative to incarcerating serious juvenile offenders. Journal of Consulting and Clinical Psychology. 1992;60(6):953–961. doi: 10.1037//0022-006x.60.6.953. [DOI] [PubMed] [Google Scholar]
- Jensen PS, Garcia JA, Glied S, Crowe M, Foster M, Schlander M, … Wells K. Cost-effectiveness of ADHD treatments: findings from the multimodal treatment study of children with ADHD. American Journal of Psychiatry. 2005;162(9):1628–1636. doi: 10.1176/appi.ajp.162.9.1628. [DOI] [PubMed] [Google Scholar]
- MTA Cooperative Group. 14-month randomized clinical trial of treatment strategies for attention deficit hyperactivity disorder. Archives of General Psychiatry. 1999;56:1073–1086. doi: 10.1001/archpsyc.56.12.1073. [DOI] [PubMed] [Google Scholar]
- Murphy SA. An experimental design for the development of adaptive treatment strategies. Statistics in Medicine. 2005;24:1455–1481. doi: 10.1002/sim.2022. [DOI] [PubMed] [Google Scholar]
- Nahum-Shani I, Qian M, Almirall D, Pelham WE, Gnagy B, Fabiano GA, … Murphy S. Experimental design and primary data analysis methods for comparing adaptive interventions. Psychological Methods. 2012;17:457–477. doi: 10.1037/a0029372. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Paykina NL, Greenhill LL, Gorman JM. Pharmacological treatments for Attention-Deficit/Hyperactivity Disorder. In: Nathan PE, Gorman JM, editors. A Guide to Treatments That Work. 3. New York, NY: Oxford University Press; 2007. pp. 29–70. [Google Scholar]
- Pelham WE. Against the grain: A proposal for a psychosocial-first approach to treating ADHD – the Buffalo treatment algorithm. In: McBurnett K, Pfiffner LJ, editors. Attention Deficit/Hyperactivity Disorder: Concepts, Controversies, New Directions. New York: Informa Healthcare; 2008. pp. 301–316. [Google Scholar]
- Pelham WE, Burrows-MacLean L, Gnagy EM, Fabiano GA, Coles EK, Wymbs BT, … Waschbusch DA. A dose-ranging study of behavioral and pharmacological treatment for children with ADHD. Journal of Abnormal Child Psychology. 2014 doi: 10.1007/s10802-013-9843-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pelham WE, Fabiano GA. Evidence-based psychosocial treatments for attention-deficit/hyperactivity disorder. Journal of Clinical Child and Adolescent Psychology. 2008;37(1):184–214. doi: 10.1080/15374410701818681. [DOI] [PubMed] [Google Scholar]
- Pelham WE, Fabiano GA, Massetti GM. Evidence-based assessment of attention deficit hyperactivity disorder in children and adolescents. Journal of Clinical Child and Adolescent Psychology. 2005;34(3):449–476. doi: 10.1207/s15374424jccp3403_5. [DOI] [PubMed] [Google Scholar]
- Pelham WE, Fabiano GA, Waxmonsky JG, Greiner AR, Gnagy EM, Pelham WE, III, Coxe S, Verley J, Bhatia I, Hart K, Karch K, Konijnendijk E, Tresco K, Nahum-Shani I, Murphy SA. Treatment Sequencing for ADHD: An Adaptive, Multiple-Randomization Study of Stimulant Medication and Behavioral Interventions. doi: 10.1080/15374416.2015.1105138. under review. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pelham WE, Foster EM, Robb JA. The economic impact of attention-deficit/hyperactivity disorder in children and adolescents. Ambulatory Pediatrics. 2007;7(1):121–131. doi: 10.1016/j.ambp.2006.08.002. [DOI] [PubMed] [Google Scholar]
- Pelham WE, Wheeler T, Chronis A. Empirically supported psychosocial treatments for attention deficit hyperactivity disorder. Journal of Clinical Child Psychology. 1998;27(2):190–205. doi: 10.1207/s15374424jccp2702_6. [DOI] [PubMed] [Google Scholar]
- Practice Parameter for the Assessment and Treatment of Children and Adolescents with Attention Deficit/Hyperactivity Disorder. Journal of the American Academy of Child and Adolescent Psychiatry. 2007;46(7):994–921. doi: 10.1097/chi.0b013e318054e724. [DOI] [PubMed] [Google Scholar]
- Subcommittee on Attention-Deficit/Hyperactivity Disorder, Steering Committee on Quality Improvement and Management. ADHD: Clinical practice guideline for the diagnosis, evaluation, and treatment of Attention-Deficit/Hyperactivity Disorder in children and adolescents. Pediatrics. 2011;128:2011–2654. doi: 10.1542/peds.2011-2654. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Swanson JM, Kraemer HC, Hinshaw SP, Arnold LE, Conners CK, Abikoff HB, … Wu M. Clinical relevance of the primary findings of the MTA: Success rates based on severity of ADHD and ODD symptoms at the end of treatment. Journal of the American Academy of Child and Adolescent Psychiatry. 2001;40(2):168–179. doi: 10.1097/00004583-200102000-00011. [DOI] [PubMed] [Google Scholar]
- U.S. Department of Labor Bureau of Labor Statistics. National Compensation Survey: Occupational Earnings in the United States, 2010. 2011 Retrieved February 25, 2013 from http://www.bls.gov/ncs/ncswage2010.htm.
- U.S. Department of Labor Bureau of Labor Statistics. 2011 National Occupational Employment and Wage Estimates. 2012 May; Retrieved February 25, 2013 from http://www.bls.gov/oes/2011/may/oes_nat.htm.
- U.S. Department of Labor Bureau of Labor Statistics. Employer Costs for Employee Compensation News Release, December 2012. 2013 Retrieved March 20, 2013, from http://www.bls.gov/news.release/archives/ecec_03122013.htm.
- U.S. Department of Transportation Bureau of Transportation Statistics. National Transportation Statistics, 2011. 2012 Retrieved March 20, 2013, from http://www.rita.dot.gov/bts/sites/rita.dot.gov.bts/files/publications/national_transportation_statistics/2011/html/table_04_23.html.
- Van Hout BA, Al MJ, Gordon GS, Rutten FF. Costs, effects, and C/E ratios alongside a clinical trial. Health Economics. 1994;3(5):301–319. doi: 10.1002/hec.4730030505. [DOI] [PubMed] [Google Scholar]