Effectiveness of a Telehealth Service Delivery Model for Treating Attention-Deficit/Hyperactivity Disorder: A Community-Based Randomized Controlled Trial

Abstract

Objective

To test the effectiveness of a telehealth service delivery model for the treatment of children with attention-deficit/hyperactivity disorder (ADHD) that provided pharmacological treatment and caregiver behavior training.

Method

The Children’s ADHD Telemental Health Treatment Study (CATTS) was a randomized controlled trial with 223 children referred by 88 primary care providers (PCPs) in 7 communities. Children randomized to the experimental telehealth service model received 6 sessions over 22 weeks of combined pharmacotherapy, delivered by child psychiatrists through videoconferencing, and caregiver behavior training, provided in person by community therapists who were supervised remotely. Children randomized to the control service delivery model received treatment with their PCPs augmented with a telepsychiatry consultation. Outcomes were diagnostic criteria for ADHD and oppositional defiant disorder (ODD) and role performance on the Vanderbilt ADHD Rating Scale (VADRS) completed by caregivers (VADRS-Caregivers) and teachers (VADRS-Teachers) and impairment on the Columbia Impairment Scale-Parent Version (CIS-P). Measures were completed at 5 assessments over 25 weeks.

Results

Children in both service models improved. Children assigned to the telehealth service model improved significantly more than children in the augmented primary care arm for VADRS-Caregiver criteria for inattention (χ²[4]=19.47, p<.001), hyperactivity (χ²[4]=11.91, p=0.02), combined ADHD (χ²[4]=14.90, p=0.005), ODD (χ²[4]=10.05, p=0.04), and VADRS-Caregiver role performance (χ² [4]=12.40, p=0.01) and CIS-P impairment (χ²[4]=20.52, p<.001). For the VADRS-Teacher diagnostic criteria, children in the telehealth service model had significantly more improvement in hyperactivity (χ²[4]=11.28, p=0.02) and combined ADHD (χ²[4]=9.72, p=0.045).

Conclusion

The CATTS trial demonstrated the effectiveness of a telehealth service model to treat ADHD in communities with limited access to specialty mental health services.

Clinical trial registration information

Children’s Attention Deficit Disorder With Hyperactivity (ADHD) Telemental Health Treatment Study; http://clinicaltrials.gov; NCT00830700.

Introduction

Children who live outside of metropolitan areas experience disproportionately poor access to the expert mental health workforce and to evidence-based mental health treatment.^1–5 Federal mandates have promoted the use of technology to address such disparities. The American Recovery and Reinvestment Act (ARRA, February 17, 2009; http://www.recovery.gov/arra/About/Pages/The_Act.aspx) emphasized the use of information technologies to improve healthcare delivery, and the Patient Protection and Affordable Care Act (ACA; Public Law 111-148; March 23, 2010; http://www.hhs.gov/strategic-plan/goal1.html) specifically proposed the meaningful use of telehealth technologies to improve health care and population health for all citizens. The Health Resources and Services Administration (HRSA) defines telehealth as “The use of electronic information and telecommunications technologies to support and promote long-distance clinical health care, patient and professional health-related education, public health, and health administration (http://www.hrsa.gov/ruralhealth/about/telehealth/glossary.html). When telehealth relies on synchronous (interactive) technologies, such as videoconferencing or telephony to deliver medical care to patients, the Center for Medicare and Medicaid (CMS; (http://www.cms.gov/Telemedicine/) uses the term “telemedicine”; and when that care specifically involves mental health or psychiatric services, the terms “telemental health” (TMH) and “telepsychiatry,” respectively, are generally used.⁶ Asynchronous, or delayed, telehealth technologies promote the dissemination of evidence-based care by viewing recordings of clinical care or sharing information through the use of patient portals, websites, and social media.

In response to federal mandates,⁷ telehealth programs have developed rapidly across the country, but the evidence base supporting their effectiveness is emerging gradually. Several studies have shown that synchronous TMH that delivers services directly to children is feasible and acceptable to primary care providers (PCPs),^8–9 parents,¹⁰ and youth,¹¹ and that TMH can be used reliably to establish diagnoses.^12–13 A few preliminary studies have suggested that care provided through synchronous telehealth is effective in improving outcomes for children with mental health conditions, but these studies are limited by pre- to post-intervention study designs and/or very small samples.^14–17 Asynchronous telehealth technologies show promise for training clinicians in the delivery of evidence-based mental health care. One study examined the impact of technology on improving pediatricians’ adherence to guideline-based care for the assessment and treatment of children with attention-deficit/hyperactivity disorder (ADHD).¹⁸ Pediatric practices were randomized to a quality improvement model, including a Web-based ADHD registry to track patients’ care and to manage data collected from standardized rating scales versus practice as usual. The intervention group significantly improved their adherence to guideline-based care, sustained these gains over six months, and showed further continuous quality improvements at two years.¹⁹

To bring telehealth service delivery models for children into mainstream mental healthcare, additional well-designed effectiveness studies are needed. ADHD is a disorder that is well suited to test the effectiveness of telehealth service models, as it is a prevalent, impairing, and treatable condition with well-established treatment guidelines. ^20–22 Most children with ADHD living in non-metropolitan areas are treated by their PCPs,²³ for whom the delivery of empirically supported mental health care continues to be challenging,^24–26 particularly as most children with ADHD described in community studies have comorbid mental health conditions.²⁷- ²⁹ Telehealth can provide access to the expert mental health workforce to assist PCPs and patients with low mental health service capacity.^6–7; ^30–33 Within this context, we present findings from a community-based effectiveness trial designed to determine whether children who received treatment through a telehealth service model demonstrated better outcomes than children who received treatment in primary care augmented with a single telepsychiatry consultation.

METHODS

Study Design

The Children’s ADHD Telehealth Treatment Study (CATTS) was a community-based randomized controlled trial (RCT) that compared the effects of two service delivery models on outcomes for children with ADHD. The RCT was approved by the institutional review board of Seattle Children’s Research Institute and monitored by a data safety and monitoring board. The methodology for the trial has been described in detail elsewhere ^34–35 and is summarized below.

Study Sample and Setting

Between November 2009 and August 2012, we recruited children (5.5 to 12 years old) from the practices of PCPs in seven underserved communities distributed over a 40,000 square mile geographic area that spanned from western to central Washington and north central Washington to north central Oregon. Any PCP practicing in proximity to the study sites could refer children with possible ADHD to the trial regardless of whether they had participated in our telepsychiatry service.³⁶ All study services were provided at community clinics that had high bandwidth videoconferencing connections (384 kbits/sec to 1.0 MB/sec). Sites included outreach clinics of Seattle Children’s Hospital (SCH) (n=4), an SCH-affiliated specialty clinic (n=1), a large pediatric practice (n=1), and a frontier community mental health center (n=1). Children were eligible if they met diagnostic criteria for ADHD, were between ages 5.5 and 12.9 years, resided with English-speaking caregivers, and were enrolled in school Exclusionary criteria included being in state custody, unavailability of the legal guardian, or having medical, developmental, or psychiatric disorders that required interventions beyond the scope of the study. Based on experience in our telepsychiatry clinic.³⁶ we anticipated that PCPs would refer mainly children with ADHD and comorbid mental health conditions to the trial.

Procedures

Clinical eligibility was determined in a three-step process. We reviewed records submitted by referring PCPs to determine exclusionary conditions. Caregivers then completed the Child Behavior Checklist (CBCL)³⁷ online. If the CBCL ADHD diagnostic subscale T-score was ≥ 65, the child was eligible for recruitment. Caregivers then met in person at the community clinic with a CATTS therapist who administered informed consent and three modules of the Computerized Diagnostic Interview Schedule for Children based on the DSM-IV ²⁰ (CDISC-IV) ³⁸ to confirm that the child had a diagnosis of ADHD during the past month and to establish the presence of comorbid generalized anxiety disorder (GAD) and/or oppositional defiant disorder (ODD). Consistent with prior approaches,^{29, 39} to establish baseline comorbid disorders, we considered an anxiety disorder (AD) present if caregivers endorsed criteria of the CDISC-IV module for GAD or if the child’s T-score on the CBCL DSM-IV-Oriented Anxiety Problem Subscale was ≥70. Similarly, we considered ODD present if the child met diagnostic criteria on the CDISC-IV ODD module or if the child’s T-score on the CBCL DSM-IV-Oriented Oppositional-Defiant Problems Subscale was ≥ 70.

Children were then administered assent; caregivers and teachers completed a baseline assessment.

We carried out block randomization with stratification by age group (5.5–9.9 vs. 10.0–12.9 years) and site (n=7). A statistician generated a set of random numbers with an allocation ratio of 1:1 for assignment to one of two service models: 1) the CATTS telehealth service delivery model and 2) the primary care service delivery model with teleconsultation. The statistician used the random numbers to prepare a logbook with assignments listed in consecutive order.³⁴ Participants and clinicians were not blinded to intervention assignment, but teachers and research assistants who managed data collection were blinded.

Caregivers and teachers completed follow-up assessments online at four time points: 4, 10, 19, and 25 weeks post-randomization.³⁴ The 25-week assessment was timed to be administered three weeks after the final treatment session. The same caregiver completed each assessment and designated one teacher to complete all teacher assessments. However, because enrollment occurred throughout the year, some children advanced to the next grade or moved classrooms during the trial. Thus, for 14% of participants in each arm, two teachers were identified to complete assessments. Of the five research assessments, caregivers completed a mean of 4.8 ± 0.7 and teachers completed a mean of 3.3 ± 0.7 assessments; 91.0% and 94.7% of caregivers in the CATTS telehealth service delivery model and augmented primary care arms, respectively, completed at least four assessments. The Consolidated Standards of Reporting Trials (CONSORT) diagram is shown in Figure 1.

Figure 1.

Consolidated Standards of Reporting Trials (CONSORT) diagram. Note: ADHD = attention-deficit/hyperactivity disorder; CATTS = Children’s ADHD Telemental Health Treatment Study; CBCL = Child Behavior Checklist; CDISC-IV = Computerized Diagnostic Interview Schedule for Children, Based on the DSM-IV; PCP = primary care providers.

Interventions: CATTS Telehealth Service Delivery and Augmented Primary Care

The CATTS trial adopted a telehealth service delivery model appropriate for time-limited collaboration with PCPs, focusing on reduction of children’s ADHD-related symptoms and improved caregiver management of children’s behaviors. The telehealth service delivery model utilized both synchronous and asynchronous technologies. We used videoconferencing to provide children with the scarce resource of child psychiatrists who were located at Seattle Children’s Hospital. We used multiple asynchronous telehealth technologies to train master’s-level community therapists in an evidence-based caregiver behavior training intervention for children with ADHD.^{40, 41} Trained therapists would then serve as an enduring resource for communities. The two service-delivery arms are summarized below. ^{34, 40}

CATTS Telehealth Service Delivery

The CATTS telehealth service delivery provided six sessions spaced three to four weeks apart over 22 weeks consisting of combined pharmacotherapy and caregiver behavior training. Our decision to provide combined treatment was based on findings of high rates of comorbid psychiatric conditions, particularly oppositional defiant behaviors and anxiety symptoms, among community samples of children with ADHD, ^27–29 and on the increasing expectation for PCPs to manage routine cases of ADHD in their practices.^{18, 23–26} Empirical evidence ^{29, 42–43} and the practice guidelines for ADHD ^21–22 indicate that a combined treatment model yields optimal outcomes for children with ADHD and comorbid oppositional defiant behaviors and anxiety symptoms.

Telepsychiatry

The principal investigator (K.M.) developed a manual to train the child psychiatrists to deliver evidence-based pharmacological care for ADHD through videoconferencing. Each of the six sessions included pharmacotherapy based on the five ADHD algorithms in the Texas Children’s Medication Algorithm Project (TCMAP) ⁴⁴ and a session-specific module of psychoeducation based on the neurobiology of ADHD.⁴⁵ The modules included: 1) Role of Medication in ADHD Treatment; 2) Central Nervous System (CNS) Involvement: ADHD Symptoms and Treatment Focus; 3) CNS Involvement: The Prefrontal Cortex and Executive Functioning; 4) Conditions Comorbid with ADHD; 5) Long Term Course and Potential Consequences of ADHD; 6) Review and Implications for the Individual Child.

Caregiver Behavior Training

The study’s research psychologist (C.M.) developed and recorded a series of behavior training sessions for caregivers.⁴⁰ Master’s-trained community therapists hired for the study accessed the digitized recordings through a secure website, discussed the training cases with the psychologist, and practiced the intervention with two volunteer families. They then provided the caregiver behavior training in person.

The caregiver behavior training intervention and manual were developed based on key elements of evidence-based parent training programs for elementary school-aged children.^46–48 The manual included a description of the sequentially delivered core elements for each session with sample scripts for teaching skills to caregivers. The six modules included: 1) Understanding ADHD and Your Child: Understanding Antecedents and Consequences of Behavior; 2) School Advocacy; 3) Praising and Ignoring Skills; 4) Giving Clear Instructions and Follow-Through; 5a) Time-Out and Other Consequences (5.5–9 years old) and 5b) Point System for Behavior (10–12 years old); and 6) Putting It All Together. Therapists gave caregivers handouts and assigned skills to practice between sessions. Just prior to the end of each telepsychiatry session, the community therapist joined the session at the patient site and transitioned to the caregiver behavior training. We conducted team meetings weekly with the therapists at their sites and staff at the research hub through videoconferencing. The psychologist carried out small group supervision via telephone. Between sessions, therapists exchanged documents with the research staff through an asynchronous Web portal; during sessions, they shared clinical data with the telepsychiatrists through a synchronous website. ^{34, 41}

Primary Care Service Delivery Augmented With a Teleconsultation

Children randomly assigned to the control condition remained in the care of their PCPs and received a single consultation with a telepsychiatrist, who shared treatment recommendations with the referring PCP. The telepsychiatrists were instructed to provide a comprehensive consultation consistent with their outpatient practice, including medication alternatives and any relevant recommendations for other mental health specialists and school programs. PCPs were not restricted from referring to other resources, including other services at Seattle Children’s Hospital. We included an active control arm for several reasons. As telehealth programs develop, clinical, organizational, and financial stakeholders such as third party payers must decide whether to provide consultation to PCPs, a model that efficiently utilizes the expert mental health workforce, or to provide direct service to youth and families, a model that approximates parity with traditional mental health care. Comparing results of these two service delivery models would help stakeholders in their decision-making. As PCPs in non-metropolitan communities usually do not have the opportunity to participate in research,⁴⁹ we also thought that an active control arm would incentivize them to refer to the trial. Finally, as many of the participating communities had telepsychiatry services, we deemed that ethically we could not proscribe access to an established community resource.

In both arms, telepsychiatrists and therapists used checklists that outlined the essential treatment components to address during each session. All sessions were recorded, and sessions were randomly selected to rate the clinicians’ coverage of these components. Telepsychiatrists and therapists showed high fidelity to their protocols.^{34–35, 40}

Outcome Measures

The outcomes were caregiver- and teacher-rated improvement in ADHD-related symptoms and behaviors, role performance, and functional impairment across the 25-week follow-up period (http://clinicaltrials.gov/show/NTC00830700). ADHD, ODD, and role performance were assessed with the Vanderbilt ADHD Rating Scales (VADRS) by caregivers (VADRS-Caregivers) and teachers (VADRS-Teachers).⁵⁰ The VADRS scales include subscales of inattention (9 items), hyperactivity/impulsivity (9 items), oppositional defiant behaviors (8 items), and role performance related to academic, classroom, and interpersonal functioning (8 items). VADRS diagnosis-based items are based upon the DSM-IV ²⁰ criteria for ADHD and are rated on a 4-point Likert scale. The VADRS shows a 2-factor model of inattention and hyperactivity/impulsivity consistent with a diagnosis of ADHD and has demonstrated solid psychometric properties.^51–52 In the CATTS trial, the concurrent validity of the item total of the VADRS-Caregiver and the ADHD symptom scores on the CDISC-IV was high (r = .79). We scored the VADRS forms using their diagnosis-based algorithms for inattention, hyperactivity, combined ADHD, and ODD, thereby producing binary outcomes (diagnosis yes/no). The continuous role performance scale was scored by summing the item severity ratings.

The child’s level of functional impairment was assessed with the Columbia Impairment Scale, Parent-Report Version (CIS-P).⁵³ The CIS-P has 13 items that tap impairment across domains of interpersonal relations, psychopathology, schoolwork or job, and use of leisure time and map onto a one-dimensional factor. Its reported internal consistency reliability and validity are very good.⁵³ In the CATTS trial, internal consistency at baseline was high (Chronbach’s alpha = .84). The CIS-P is scored by summing the item severity ratings.

Statistical Analysis

We conducted intention-to-treat analyses. To evaluate the robustness of the randomization, we tested whether there were differences in demographic characteristics and baseline clinical status using chi-square test of proportions for diagnostic/binary variables and T-test for continuous variables. We found the number of comorbid conditions was unbalanced between the two arms and, therefore, controlled for comorbidity in subsequent regression analyses.

To test overall intervention effects, we applied Wald Chi-square tests to determine whether all four interaction terms were zero simultaneously or whether significantly more improvement in one arm occurred at least once during the four assessments. Then we evaluated individual interaction coefficients to determine when the significant differences occurred during the longitudinal assessments.⁵⁴

In the longitudinal data analyses, we applied logistic mixed effects models to the binary outcomes and linear mixed effects models to the continuous outcomes. A patient-specific random intercept was included in all models to account for the within-subject correlations due to repeated assessments. In the mixed effects models, we included main effects for treatment arms, time (week of assessment), comorbid conditions, and interaction terms between treatment arms and time. We modeled time as a discrete predictor with five levels corresponding to baseline, week 4, 10, 19, and 25. Then we evaluated the coefficients on the interaction terms to capture the differences in rates of improvement from baseline to each follow-up assessment between the two study arms. The coefficient on the main group effect tested differences between study arms at baseline. The coefficients on the main time effect tested the differences between each follow-up assessment and baseline for the augmented primary care arm. The coefficients on the group-by-time interaction tested whether children in the CATTS telehealth service delivery achieved more improvement from baseline than children in the augmented primary care arm. For these binary outcomes, the odds ratio estimates on the interaction terms or their logarithmic transformations provide adjusted effect size estimates.⁵⁵

For the continuous outcomes measuring functioning, we reported beta coefficients from linear mixed effects models. In addition, we reported Cohen’s d based on between-study arm comparisons of within-individual changes in scale scores from baseline to 25-week follow-up. In a study such as ours, where adjustments are warranted for uneven distribution of confounding variables across study arms and for features of the study design (longitudinal, nested structure), the Cohen’s d statistic provides a rough indication of effect size.

Analyses were conducted using Stata 12.1.⁵⁶ We set the significance level at 0.05, a priori.

RESULTS

Participants, Engagement, and Characteristics of the Study Arms

Eighty-eight PCPs made successful referrals to the trial. The sample consisted of 223 children with mean age 9.25 (± 2.0) years, predominantly of European/white ancestry (91.5%), with a median annual family income of $35,000 to $75,000. Seventy-five percent of children (n=168) had at least one comorbid disorder. Forty-two percent (n=93) met criteria for ODD, and 6% (n=13) met criteria for AD; 28% (n=62) met criteria for ODD and AD.

Of the six possible sessions, participants randomized to the CATTS telehealth service delivery arm (n=111) attended an average of 5.2 (range 0 to 6) telepsychiatry sessions and an average of 5.1 (range 0 to 6) caregiver behavior training sessions. Of participants randomized to the augmented primary care arm (n=112), 94.6% attended their telepsychiatry consultation session.

As shown in Table 1, the demographic characteristics of children in the CATTS telehealth service delivery and augmented primary care arms were comparable with no statistically significant differences between the groups. Clinical characteristics were also well balanced with one exception. A higher proportion of children in the CATTS telehealth service delivery arm had ADHD with both comorbid disorders; therefore, we adjusted for comorbidity status (0,1, 2 comorbidities) in multivariate analyses.

Table 1.

Sample Demographic and Clinical Characteristics

Demographic Variables		CATTS Telehealth Service Delivery (n=111)	Augmented Primary Care (n=112)
Child Age, M (SD)		9.2 (2.0)	9.3 (2.0)
Child Gender, n (%)
Male		76 (68.5)	87 (77.7)
Female		35 (31.5)	23 (22.3)
Child Race, n (%)
White		104 (93.7)	100 (89.3)
Black		1 (0.9)	1 (0.9)
Asian		0 (0)	2 (1.8)
NH/PI		0 (0)	4 (3.6)
AI/AN		4 (3.6)	2 (2.7)
Other		2 (1.8)	2 (1.8)
Child Ethnicity, n (%)
Hispanic		10 (9.0)	19 (17.0)
Non-Hispanic		101 (91.0)	93 (83.0)
Marital Status, n (%)
Married/Cohabitating		76 (68.5)	83 (74.8)
Other		35 (31.5)	28 (25.2)
Household Income, n (%)
<35k		41 (36.9)	36 (32.4)
35k-75k		28 (25.2)	42 (37.8)
75k-100k		22 (19.8)	13 (11.7)
>100k		20 (18.0)	20 (18.0)
Comorbid Disorders, n (%)*
ADHD alone		20 (18.0)	35 (31.3)
ADHD+ODD		44 (39.6)	49 (43.8)
ADHD+AD		7 (6.3)	6 (5.4)
ADHD+ODD+AD		40 (36.0)	22 (19.6)
Clinical Measures
Caregiver Ratings
Vanderbilt ADHD Rating Scale, Diagnostic Scoring
Inattention diagnosis, n (%)		92 (83)	92 (82)
Hyperactivity/Impulsivity diagnosis, n (%)		74 (67)	65 (58)
ADHD-combined diagnosis, n (%)		67 (60)	58 (52)
ODD diagnosis, n (%)		68 (61)	57 (51)
Measures of Functioning
VADRS-Role Performance Score, M (sd)		3.51 (0.59)	3.52 (0.55)
CIS-P Functioning Score, M (sd)		24.71 (9.18)	22.89 (8.95)
Teacher Ratings
Vanderbilt ADHD Rating Scale, Diagnostic Scoring
Diagnostic Criteria per Algorithmic Scoring
Inattention Diagnosis, n (%)		56 (50)	64 (57)
Hyperactivity/Impulsivity diagnosis, n (%)		42 (38)	32 (29)
ADHD-combined Diagnosis, n (%)		26 (23)	19(17)
ODD Diagnosis, n (%)		34 (31)	35 (31)
Measure of Functioning
VADRS-Role Performance Score, M (SD)		3.75 (0.72)	3.90 (0.66)

Note: AD = anxiety disorder; ADHD = attention-deficit/hyperactivity disorder; AI/AN = American Indian/Alaska Native; CATTS = Children’s ADHD Telemental Health Treatment Study; CIS-P = Columbia Impairment Scale-Parent Version; NH/PI = Native Hawaiian/Pacific Islander; ODD = oppositional defiant disorder; VADRS = Vanderbilt ADHD Rating Scales.

^*p < 0.05

Caregiver-Rated Outcomes by Intervention Status

Figure 2 graphically shows the unadjusted proportions of participants in the CATTS telehealth service delivery and augmented primary care arms who met VADRS-Caregiver-based diagnostic criteria at baseline and at 4, 10, 19, and 25-week follow-up. At baseline, participants in the two arms had similar occurrence of meeting VADRS-Caregiver diagnostic criteria for inattention (intervention vs. control: 83% vs. 82%), hyperactivity (67% vs. 58%), combined ADHD (60% vs. 52%), and ODD (61% vs. 51%). Children in both study arms improved over time. At 25 weeks, compared with children in the augmented primary care arm, lower proportions of children in the CATTS intervention arm met diagnostic criteria on the VADRS-Caregiver: inattention (23% vs. 48%), hyperactivity (16% vs. 31%), combined ADHD (12% vs. 26), and ODD (16% vs. 26%).

Figure 2.

Proportions (unadjusted) of children at each assessment meeting caregiver-reported diagnostic criteria on subscales of the Vanderbilt Attention-Deficit/Hyperactivity Disorder (ADHD) Rating Scale. Note: ODD = oppositional defiant disorder.

The steeper decline in the unadjusted proportions of children in the CATTS telehealth service delivery arm meeting diagnostic criteria shown in Figure 2 were formally tested. According to the tests of overall intervention effects, the CATTS telehealth service delivery arm demonstrated greater improvements at least once during follow-up assessment in all four VADRS-Caregiver diagnostic outcomes: inattention (Wald χ²=19.47, df=4, p<.001), hyperactivity/impulsivity (Wald χ²=11.91, df=4, p=0.02), combined ADHD (Wald χ²=14.90, df=4, p=.005), and ODD (Wald χ²=10.05, df=4, p=0.04).

Table 2 shows the course of the VADRS-caregiver ratings. The coefficients on the main time effect indicate the significant improvement for the augmented primary care arm, and the coefficients on the group-by-time interaction indicate the greater improvement for the CATTS telehealth service delivery arm, which were evident by week 10 for hyperactivity and combined ADHD, and by week 19 for inattention and ODD.

Table 2.

Treatment Outcomes: Meeting Diagnostic Criteria Based on Vanderbilt Attention-Deficit/Hyperactivity Disorder (ADHD) Rating Scales

Caregiver Ratings (N = 223)		Inattention OR (95% CI)	Hyperactivity OR (95% CI)	ADHD-Combined OR (95% CI)	ODD OR (95% CI)
Group
Intervention vs. Control		.822 (0.276–2.446)	1.354 (0.516–3.556)	1.338 (0.552–3.244)	1.220 (0.431–3.452)
Time (in weeks)
4		.228 (0.100–0.522)***	.524 (0.252–1.092)	.471 (0.233–0.951)*	.512 (0.232–1.129)
10		.085 (0.036–0.200)***	.190 (0.087–0.412)***	.185 (0.088–0.390)***	.137 (0.058–0.324)***
19		.059 (0.025–0.141)***	.090 (0.039–0.206)***	.100 (0.045–0.222)***	.183 (0.079–0.427)***
25		.058 (0.024–0.139)***	.120 (0.053–0.272)***	.149 (0.070–0.318)***	.103 (0.042–0.251)***
Group × Time
1 × 4		1.410 (0.439–4.528)	.368 (0.128–1.058)	.486 (0.179–1.316)	.535 (0.172–1.658)
1 × 10		.604 (0.184–1.984)	.213 (0.069–0.664)**	.296 (0.101–0.874)*	.618 (0.186–2.051)
1 × 19		.211 (0.060–0.745)*	.236 (0.070–0.794)*	.149 (0.044–0.512)**	.247 (0.072–0.853)*
1 × 25		.140 (0.038–0.508)**	.137 (0.040–0.466)**	.132 (0.040–0.432)**	.148 (0.039–0.562)**
Teacher Ratings (n=149)		Inattention OR (95% CI)	Hyperactivity OR (95% CI)	ADHD–Combined OR (95% CI)	ODD OR (95% CI)
Group
		.518 (0.174–1.542)	2.135 (0.646–7.052)	1.746 (.511–5.967)	.834 (0.239–2.908)
Intervention vs. Control
Time (in weeks)
4		.234 (0.096–0.568)***	.614 (0.232–1.629)	.337 (0.117–0.968)*	.841 (0.320–2.216)
10		.272 (0.114–0.649)**	.785 (0.304–2.028)	.910 (0.315–2.628)	.354 (0.128–0.983)*
19		.227 (0.095–0.541)***	.164 (0.056–0.480)***	.191 (0.060–0.608)**	.179 (0.062–0.520)**
25		.254 (0.101–0.637)**	.171 (0.054–0.546)**	.153 (0.046–0.510)**	.332 (0.117–0.945)*
Group × Time
1 × 4		2.046 (0.568–7.373)	.699 (0.169–2.884)	.731 (0.173–3.084)	.490 (0.113–2.132)
1 × 10		.493 (0.133–1.822)	.073 (0.014–0.371)**	.077 (0.013–0.451)**	.673 (0.141–3.212)
1 × 19		.504 (0.136–1.863)	.749 (0.164–3.417)	.443 (0.087–2.267)	.441 (0.084–2.312)
1 × 25		.320 (0.079–1.300)	.311 (0.056–1.732)	.178 (0.027–1.174)	.108 (0.017–0.704)*

Note: Analyses per logistic mixed effects regression models; all models adjusted for baseline comorbid conditions; coefficients on the group × time interaction terms are the logarithmic ORs. Note: ODD = oppositional defiant disorder; VADRS = Vanderbilt ADHD Rating Scale.

^*p< 0.05;

^**p < 0.01;

^***p < 0.001

Functional improvements also differed by intervention status. At baseline, the average VADRS-Caregiver role performance scores were comparable between the CATTS telehealth service delivery and augmented primary care arms.

According to the tests of overall differences, children in the CATTS telehealth service delivery demonstrated significantly greater improvement from baseline to follow-up assessments in VADRS-Caregiver role performance (Wald χ²=12.40, df=4, p=0.01) and CIS-P impairment scores (Wald χ²=20.52, df=4, p<.001). As noted in Table 3, these significant intervention differences were noted at 19 weeks for CIS-P impairment (12.80 ± 6.95 vs. 15.11 ± 9.58, p=0.05) and at 25 weeks for VADRS-Caregiver role performance (2.96 ± 0.67 vs. 3.21 ± 0.77, p=0.01). Looking at within-participant changes between baseline and 25-week follow-up, we estimated that the CATTS telehealth service delivery achieved an effect size of 0.38 (Cohen’s d) on the VADRS-Caregiver rated role performance, and 0.44 on the CIS-P.

TABLE 3.

Treatment Outcomes: Role Performance and Impairment Scale Scores

		Caregiver VADRS-Role Performance Scale β (95% CI) (N=223)	Caregiver-Report CIS-P Scale β (95% CI) (N=223)	Teacher VADRS-Role Performance Scale β (95% CI) (n=149)
Group
Intervention vs. Control		−.052 (−.195 – .091)	.447 (−1.617 – 2.512)	−.159 (−.352 – .034)
Time (in weeks)
	4	−.180 (−.274 – −.085)	−4.555 (−5.797 – −3.313)	−.176 (−.278 – −.074)
	10	−.245 (−.357 – −.132)	−5.973 (−7.419 – −4.529)	−.253 (−.381 – −.126)
	19	−.339 (−.447 – −.232)	−7.040 (−8.666 – −5.413)	−.405 (−.543 – −.267)
	25	−.317 (−.434 – −.200)	−8.111 (−9.720 – −6.502)	−.245 (−.395 – −.096)
Group × Time
	1 × 4	.055 (−.064 – .175)	1.047 (−.642 – 2.737)	.045 (−.100 – .191)
	1 × 10	.023 (−.135 – .182)	−1.386 (−3.546 – .774)	−.149 (−.349 – .050)
	1 × 19	−.140 (−.306 – .025)	−3.593 (−5.885 – −1.301)**	−.113 (−.318 – .092)
	1 × 25	−.220 (−.389 – −.051)*	−3.678 (−5.870 – −1.485)**	−.223 (−.462 – .016)

Note: Analyses per linear mixed effects models; all models adjusted for baseline comorbid conditions. CIS-P = Columbia Impairment Scale, parent-report version; VADRS = Vanderbilt Attention-Deficit/Hyperactivity Disorder (ADHD) Rating Scale.

^*p< 0.05;

^**p < 0.01

Teacher-Rated Outcomes by Intervention Status

Figure 3 graphically depicts the unadjusted proportions of participants in the CATTS telehealth service delivery and augmented primary care arms who met VADRS-Teacher-based diagnostic criteria at baseline and at 4-, 10-, 19-, and 25-week follow-up. At baseline, participants in the two arms had a similar occurrence of meeting VADRS-Teacher diagnostic criteria for inattention, hyperactivity, combined ADHD, and ODD, and both groups improved over time.

Figure 3.

Proportions (unadjusted) of children at each assessment meeting teacher-reported diagnostic criteria on subscales of the Vanderbilt Attention-Deficit/Hyperactivity Disorder (ADHD) Rating Scale. Note: ODD = oppositional defiant disorder.

According to the tests of overall intervention effects, the VADRS-Teacher scores demonstrated significantly greater improvements at least once for the CATTS telehealth service delivery arm for hyperactivity (Wald χ²=11.28, df=4, p=0.02) and total ADHD (Wald χ²=9.72, df=4, p=0.045). As indicated in Table 2, results of logistic mixed effects regression models show that these intervention effects reflected significant differences only at week 10. There was no difference in outcomes between the intervention arms for inattention and ODD.

As shown in Table 3, results of linear mixed effects models indicate that improvements in VADRS-Teacher-reported role performance scores did not differ significantly between the two study arms. Evaluation of the within-subject changes between baseline and 25-week follow-up yields an estimated effect size of 0.37 (Cohen’s d) for the CATTS telehealth service delivery arm, similar to caregiver-reported role performance.

We conducted sensitivity analyses with additional demographics (age, gender, and parental education) and site indicators in the models. There were no significant changes in the intervention effects estimates, consistent with the balance among these factors achieved in our randomization.

DISCUSSION

The CATTS trial is the first community-based investigation testing the effectiveness of a telehealth service delivery model to improve mental health outcomes for children, and the only trial comparing a telehealth service delivery model with a primary care service model. The trial demonstrated that both the short-term CATTS telehealth service delivery and the augmented primary care treatment arms were associated with reductions in caregiver-reported ADHD and ODD diagnoses and improvement in role performance and impairment over 25 weeks, and that the telehealth service model performed better than the augmented primary care model. These results are broadly consistent with prior findings, that children with ADHD and comorbid disorders who were treated in person over a longer duration with a combined intervention improved more than those treated in primary care.⁵⁷ They support the “added value” of providing short-term expert care through telehealth over treatment in primary care.

For most measures, significant differences between the CATTS telehealth service delivery and the augmented primary care arms did not emerge until 10- to 19-weeks post-randomization, possibly reflecting the CATTS short-term intervention model with fewer and less frequent caregiver training sessions than delivered in prior trials conducted in person, or the lack of an initial washout and titration of medication to an individual optimal dose.^{27, 28, 42, 48, 57–58} Alternatively, the single teleconsultation may have led PCPs to initiate successful treatment, consistent with other telehealth studies, ^59–61 thereby delaying the emergence of differences between the treatment arms, but a teleconsultation may have been insufficient to sustain benefits, consistent with adult studies that have shown inadequate management of chronic conditions in primary care. ^62–63 By the 10-week assessment, families had participated in three telepsychiatry and three caregiver behavior training sessions, and by the 19-week assessment, they had participated in 5 combined sessions. There may be a critical minimal number of sessions needed to detect group differences, and/or the caregiver training sessions may need to be conducted more closely together to detect differences over a shorter timeframe.

The more limited benefits for the CATTS intervention for teacher-rated outcomes reflect several factors. Behavioral treatments yield the largest effects in the setting in which they are implemented,^{48, 58} and interventions provided directly to teachers have shown improved ADHDrelated behaviors per both teacher ratings and observations.^{57, 64–65} The CATTS service delivery model may have benefitted with the addition of a school component in which teachers provided more frequent feedback during medication adjustment through an asynchronous website and received consultation regarding classroom behavior management through videoconferencing.⁶⁵ Methodological issues may have contributed to the teacher-reported outcomes. Eligibility was determined on the basis of caregiver ratings, but caregivers and teachers only correlate moderately in their descriptions of youth characteristics. Consistent with prior studies,^{65, 67} teachers rated lower levels of ADHD symptoms, ODD, and role performance than caregivers on the same scales leaving less room for improvement over the course of the trial and attenuating observed effect sizes. Future work should consider both caregiver and teacher reports in determining study eligibility and use telehealth technologies to include teachers in intervention trials. Outcomes should include objective school outcomes measures, such as behavioral observations or homework completion.

There are several implications of the CATTS trial for future work. Investigators may be interested in determining the relative contributions of the pharmacologic and behavioral treatment components of the CATTS telehealth service delivery, particularly for subgroups of children with ADHD, for example those with and without comorbid disorders or learning disabilities, or for families that have access to one but not both of the treatment components. Of note in the current study, children in the CATTS telehealth service delivery showed improvements in both ADHD symptoms, which may preferentially respond to pharmacologic interventions,^68–69 as well as ODD, role performance, and impairment outcomes, which may preferentially respond to behavioral interventions. ^70–71

Sustainability of the two models is important to further development of telehealth programs. Increasingly, states are mandating coverage for telehealth,^72–73 and third party payers are reimbursing synchronous telehealth services ^74–76 using specific Current Procedural Terminology (CPT) codes.^77–78 Telepsychiatry appears to be reimbursed more predictably than teletherapies, particularly by federal and state programs.^{74–76, 78} Further work to establish the effectiveness of delivering caregiver behavior training through videoconferencing will promote the sustainability of teletherapies that can help to reduce the burden of mental illness for families who are not well served by usual models of care.⁷⁹

Telehealth is well poised to contribute to efforts by Accountable Care Organizations (ACOs) to provide mental health care to children living in underserved areas,⁷⁸ and future work should examine the cost to benefit of a range of telehealth models of care for ADHD implemented at the population level⁸⁰ and determine the optimal mix of intervention components and number of sessions needed to effectively and efficiently improve children’s mental health.

The CATTS trial had several limitations. The augmentation of primary care treatment with a teleconsultation may have yielded smaller group differences than would have been evident if we had used it as a treatment-as-usual control. Methodological issues may have contributed to the teacher-reported outcomes. Compared to caregivers, teachers completed fewer assessments with inconsistency in teacher reporters for some children, which introduced greater variance for teacher-reported outcomes. Overall, the trial may have been underpowered to detect group differences for teacher-rated outcomes. The sample may have been biased by referral from PCPs who were willing to use telehealth services and/or who referred children with more complicated symptomatology, and by the exclusion of children with complex medical, developmental, and psychiatric disorders.

CLINICAL GUIDANCE.

• The CATTS trial supports the effectiveness of treating children with ADHD using telehealth.

• A direct telehealth service model offers undeserved communities potential parity with metropolitan communities in accessing guideline-based interventions.

• An augmented primary care model may optimally redistribute the child psychiatry workforce and support PCPs’ efforts to treat children’s mental health disorders..

• Asynchronous telehealth technologies offer therapists an opportunity to upgrade their skills in treating children and to strengthen the child mental health workforce in underserved communities.

• Stakeholders who are considering telehealth should determine whether a short-term direct service model or an augmented primary care model best meets their community’s needs and resources.^81–82

• Child and adolescent psychiatrists interested in a telehealth practice should consider the evidence base for providing care through telehealth technologies, collaborative models for working with clinicians at the patient site, supports needed at both the provider and the patient sites, and financial models to sustain a successful practice.