The Children’s Attention-deficit Hyperactivity Disorder (ADHD) Telemental Health Treatment Study: Methodology for Conducting a Trial of Telemental Health in Multiple Underserved Communities

Abstract

Background

Children who live in non-metropolitan communities are underserved by evidence-based mental health care and underrepresented in clinical trials of mental health services. Telemental Health (TMH), the use of videoteleconferencing (VTC) to provide care that is usually delivered in person, shows promise for helping to rectify these service disparities.

Purpose

The Children’s ADHD Telemental Health Treatment Study (CATTS) is a randomized controlled trial designed to test the effectiveness of TMH in providing treatment to children diagnosed with attention-deficit hyperactivity disorder (ADHD) who are living in underserved communities. In this paper we describe the methodologies we developed for the trial and lessons learned.

Methods

Children ages 5.5-12 years of age with ADHD were referred to CATTS by their primary care physicians (PCP’s). The test intervention group (Group A) received six telepsychiatry sessions followed by in-person caregiver behavioral training delivered by a local therapist who was trained and supervised remotely. A secure website was used to support decision-making by the telepsychiatrists, to facilitate real-time collaboration between the telepsychiatrists and community therapists, and communication with the PCP’s. The control group (Group B) received a single telepsychiatry consultation followed by treatment with their PCP’s who implemented the telepsychiatrists’ recommendations at their discretion.

Caregivers completed five sets of questionnaires about children’s symptoms and functioning and their own levels of distress. Older children (aged 10-12 years) completed questionnaires about their symptoms and functioning. Teachers completed ADHD rating scales. Questionnaires were completed online through a secure portal from personal computers.

Results

Eighty-eight PCP’s in seven communities referred the 223 children who participated in the trial. Attrition was low (3%). Children in Group A completed an average of 5.3 of 6 scheduled sessions; 96% of children in Group B completed their telepsychiatry consultation. Parents in both groups completed an average of 4.8 of 5 assessments. Telepsychiatrists and therapists showed high adherence to treatment protocols.

Lessons Learned

TMH proved to be a viable means of providing evidence-based pharmacological services to children and of training local therapists in evidence-based caregiver behavioral management. Recruitment was enhanced by offering the control group a telepsychiatry consultation. To meet recruitment targets across multiple dispersed sites, we developed community-specific strategies. A dedicated scheduler was a critical staff role to coordinate the multiple sites, sessions, and clinicians. Trial implementation was easier with sites that shared an electronic medical record system with our research hub.

Conclusions

The CATTS study used methods and procedures to optimize inclusion of children living in multiple dispersed and underserved areas. These experiences should advance the development of technologies needed to recruit underserved populations into research projects with the goal of reducing disparities in access to quality mental health care.

Introduction and Background

Children living outside of metropolitan areas account for more than 20% of the United States population under the age of 18.¹ Psychiatric disorders affect 7% to 20% of the nation’s children, regardless of geographic location,² but most children with mental health needs who reside outside of metropolitan areas receive inadequate or no treatment due to the chronic shortage of child mental health specialists.³ Even when services are available, the quality of care is often inadequate, as empirically supported mental health treatments are not well disseminated beyond academic centers where federal grant funding to support their development and initial dissemination is concentrated.

Many families rely on their primary care providers (PCP’s) to provide child psychiatric care. Recognizing this critical role, professional organizations have developed guidelines for PCP’s to use to diagnose and treat psychiatric disorders of childhood,^4-6 but these guidelines have not been widely implemented.⁷ Furthermore, many children have complex needs for which their PCP’s need additional support. While the need for collaborative models to help PCP’s provide guideline-driven care has been recognized for two decades, federal, state, local and private funding to address this need has been lacking.

Despite efforts to encourage the recruitment of representative population samples in order to improve the generalizability of research findings⁸ and, ultimately, to correct disparities in access to evidence-based health care, disparities persist.⁹ People living outside of metropolitan areas, including children, are underrepresented in research.¹⁰ Multiple barriers discourage investigators from conducting research in non-metropolitan areas; these include low population densities and weak research infrastructure.¹¹ Recruiting samples of adequate size and diversity is particularly challenging when studies focus on mental illnesses that are under-diagnosed.

The dual needs of developing new models to provide mental health services to children living outside of metropolitan communities and of representing them in clinical trials have converged with the call to use telecommunications technologies to rectify disparities in children’s access to mental health care.^{12, 13} Telemental health is a promising model for redistributing access to the mental health workforce, disseminating evidence-based interventions to the community, and collaborating with local providers.^14-17 Telemental health (TMH) refers to the use of videoteleconferencing (VTC) to render psychiatric and mental health care that usually is provided in-person. Published work on the use of TMH with children and adolescents is limited and focuses predominantly on program descriptions and reports of satisfaction of providers,^{18, 19} parents,²⁰ youth.²¹ Two small pre- to post- intervention outcome studies have been conducted to date.^{22, 23}

Over the past 15 years, a body of research has accumulated to support the use of TMH with adults. These papers describe development and implementation of innovative programs, satisfaction of providers and patients, therapeutic alliances, and reliability of outcome assessments.^{24, 25} Several early studies with small samples that used early technologies reported patient outcomes in terms of rates of hospitalization,²⁶ global assessment of functioning,²⁷ and general health status.²⁸ In a study of 495 outpatients with a variety of psychiatric disorders, O’Reilly and colleagues compared outcomes of those randomly assigned to consultation in person or by videoconferencing.²⁹ Both groups showed comparable decreases in general ratings of symptomatology and distress, levels of “caseness,” and rates of hospitalization and reported moderate satisfaction with care.

Randomized controlled trials of depressed military veterans have incorporated advanced technologies, larger sample sizes, and more sophisticated designs and shown equivalent or non-inferior outcomes for telepsychiatry versus in-person psychopharmacological treatment or psychotherapy.^{16, 30, 31} One superiority trial conducted in multiple remote sites showed that depressed veterans randomly assigned to a telepsychiatry-mediated Collaborative Care Model demonstrated better adherence to treatment plans, satisfaction, quality of life, and clinical outcomes than those randomized to treatment as usual in primary care.³²

To date no systematic, controlled telepsychiatry studies with large samples of children or adolescents have been conducted. Such studies are needed to establish an empirical foundation for the viability of telepsychiatry as a model of care to bring evidence-based services to families living outside of major metropolitan areas.

The Children’s Attention-Deficit/Hyperactivity (ADHD) Telemental Health Treatment Study (CATTS) was a randomized controlled trial (RCT) conducted to evaluate a new mental health service delivery model for children living in multiple underserved communities. CATTS was designed to determine whether an evidence-based intervention for ADHD could be implemented using VTC and could improve outcomes of reduced symptoms and caregiver distress compared to treatment in primary care. Enrollment of study participants, delivery of clinical interventions, and data collection for children and their parents began in November 2009. Clinical interventions were completed in February 2013 and data collection was completed in April 2013. In this paper we relate our experience in conducting the CATTS trial to guide future investigations aimed at reducing geographic disparities in children’s access to mental health care and utilizing technology in service delivery models.

Methods

The CATTS study utilized a “hub and spoke” administrative model. The research “hub” was located at Seattle Children’s Research Institute, the research arm of Seattle Children’s Hospital (SCH). SCH is the major pediatric training center for the University of Washington Departments of Pediatrics and Psychiatry. The University and SCH provide tertiary care for a four-state region of the Pacific Northwest. With a mission to extend the reach of its resources across a broad geographic area, SCH operated a successful telepsychiatry service for several years prior to implementation of CATTS. The research hub was responsible for all study coordination and implementation of activities, including interventionist hiring, training and supervision and management of participant referral, enrollment, treatment, and research assessment. Telepsychiatry services were delivered from the research hub with support of the SCH Audiovisual Department.

The CATTS investigators recruited PCP’s and families from seven “spoke” sites with low access to local child mental health specialty resources. The CATTS administrative model required development of strategies to support communication and project coordination and to engage providers as research partners and their patients as research participants. PCP’s in Washington and Oregon referred boys and girls 5.5 to 12.9 years old with suspected ADHD to CATTS. Providers faxed to the hub a referral form, medical records, and contact information for potential participants.

Eligibility for participation was determined in a three-step process. Medical records were reviewed to identify any exclusionary medical, developmental, or psychiatric comorbidities, e.g., cardiac disease, autism, or psychosis that required interventions beyond the scope of the study. When a child met initial eligibility criteria, caregivers completed the Child Behavior Checklist, a widely used assessment that measures a broad range of symptoms and indicates the probability of a diagnosis of ADHD and other common psychiatric disorders. When the Child Behavior Checklist suggested a diagnosis of ADHD, the child and caregiver were asked to provide assent and consent, respectively, to participate in CATTS. Consenting caregivers completed the Computerized Diagnostic Interview Schedule for Children³⁴ in person with the community-based therapist to confirm a diagnosis of ADHD. Children who met criteria for ADHD were randomly assigned to test intervention or control groups.

The test intervention was based on findings from the Multimodal Treatment of ADHD Study which demonstrated that children diagnosed with ADHD who received combined medication and behavioral interventions had better outcomes than children who received treatment as usual in the community.^36-38 In CATTS, participants randomized to the test intervention, Group A, received six two-component sessions spaced 3-4 weeks apart over the course of 22 weeks. The first component, provided by a psychiatrist via VTC , was an algorithm-driven pharmacological treatment for ADHD ³⁹ and included psychoeducation on the neurobiology of ADHD. The second component, provided in-person by a community therapist, was evidence-based caregiver behavior training. ⁴⁰ We developed a web-based electronic tool to aid telepsychiatrists’ decision-making regarding medications, to facilitate collaboration between the telepsychiatrists and therapists, to document and track intervention activities, and to communicate with referring PCP’s.

Children randomized to the control intervention, Group B, remained in the care of their PCP and received a single consultation session with a telepsychiatrist. The telepsychiatrist discussed the child’s care with the referring PCP by telephone and sent a typical clinic report with treatment recommendations. PCP’s implemented these recommendations at their discretion and scheduled follow-up appointments as they deemed appropriate. We refer to this management strategy as “augmented treatment as usual.”

Outcome assessments were obtained at baseline and at four additional time points during the trial with the last one at 25 weeks post-randomization. Assessments consisted of caregiver-completed questionnaires and questionnaires completed by older children (≥10 years old) regarding children’s symptoms and functioning. Caregivers also reported on their own level of distress. Teachers reported on children’s ADHD symptoms.

All telepsychiatry sessions and caregiver training sessions per Group A participants and all telepsychiatry consultation sessions per Group B participants were recorded. Evidence-based intervention guidelines were summarized in a checklist of topics for the clinicians to cover during each session. Two recordings of telepsychiatry sessions and one recording of caregiver training sessions per family were randomly selected to rate fidelity to the intervention protocol as itemized in the session checklists. Half of the telepsychiatry sessions for the control participants were randomly selected to rate fidelity to the consultation guidelines. Research assistants were trained to score checklists after attaining over 90% inter-rater reliability with the investigators.

Challenges in Conducting a Clinical Trial in Multiple Under-served Communities and Lessons Learned

Challenges in conducting mental health services research with children living in multiple underserved communities are considered in four arenas: 1) designing the research and intervention approaches; 2) recruiting and retaining participants; 3) providing evidence-based interventions to distant sites; and 4) coordinating research activities.

Designing the Research and Intervention Approaches

There is little in the literature to guide the design of a study to test the effectiveness of an innovative mental health service model delivered to a dispersed non-metropolitan sample.⁴¹ Study designs that emphasize feasibility, proof of concept, or non-inferiority of outcomes are not adequate to guide policy decisions in anticipation of health care reform. An RCT of a new service delivery model versus usual care was the optimal design to meet our objective.

CATTS investigators selected ADHD as a model disorder for the trial, as it is a chronic condition that is prevalent across geographical areas.^{35, 42} ADHD is encountered frequently in primary care,^{4, 5} and primary providers often need assistance with complex cases.⁴³ The trial investigators endeavored to recruit children with a broad spectrum of ADHD so that inferences could be extended to children with the range of ADHD severity and complexity seen clinically.

General practice guidelines for ADHD recommend pharmacotherapy as the core treatment supplemented by caregiver behavioral training, particularly for children who are diagnosed with comorbid disorders.^{4, 5, 44} Therefore, a primary challenge was to determine the optimal model for supporting PCP’s. We considered several models, including training PCP’s in the management of ADHD,⁴⁵ providing case consultation to networks of community providers,⁴⁶ and giving on-demand telephone consultation to providers.⁴⁷ These approaches, however, did not address the needs for expert evaluation and treatment of children with complex needs for which PCP’s most often request assistance. Therefore, we chose a model in which child psychiatrists and community therapists assumed treatment to implement a time-limited intervention for ADHD aimed at symptom reduction and behavior management and then returned children to their referring PCP’s with recommendations for continuing care.

As the SCH telepsychiatry service was well-established in the region, the ethics of using as the control arm “treatment as usual” by the referring care provider without a “usual dose” of telepsychiatry was questioned. Therefore, Group B participants were provided a single telepsychiatry consultation, consistent with common practice for telepsychiatry programs nationally⁴⁸ and utilized by local psychiatrists who were participating in the SCH Telemental Health Clinic. Providing an active control management strategy was expected to encourage providers to refer their patients to the trial and to reduce attrition, but at the same time to diminish ability to detect differences in treatment outcomes between trial arms.

Recruiting Participants

A major challenge was to recruit our target of 210 families living in multiple, non-metropolitan communities for treatment delivered through a novel technological medium in which participants did not meet in-person with the treating psychiatrist. Another challenge was to retain families through the final 25-week assessment, particularly families randomized to the control condition after they completed their single telepsychiatry consultation.

Multiple strategies are often necessary to meet recruitment goals when conducting research in non-metropolitan areas.⁴⁹ While conducting clinical trial activities at the offices of participating PCP’s in non-metropolitan communities would have offered the advantage of using an existing infrastructure and keeping the trial in the forefront of providers’ awareness, disadvantages included strain on staff and interruptions of office routines. CATTS had the additional need for VTC equipment requiring bandwidth that exceeded the capability of most community practices. We opted to establish in each community a central clinic site with VTC equipment and staff that could trouble-shoot minor clinical and technical problems. Features of the seven central clinics are summarized in Table 1.

TABLE 1.

SITES PARTICIPATING IN THE CATTS STUDY, CHARACTERISTICS, AND ENROLLMENT

Site	Population Size1	Racial/Ethnic Composition 2	Distance from Hub	Economic Base	Date Joined CATTS	Enrollment
Site 1 SCH satellite	46,478 city 256,591 county	83.7% White 2.0% Black 6.3% Hispanic	60 miles	Government, real estate, insurance, and college	September 2009	72
Site 2 SCH satellite	182,000 three adjacent cities 262,000 two counties	56.0% to 87.0% White 1.4% to 1.9% Black 8.0% to 56,0% Hispanic	216 miles	Agriculture, technology, biotechnology, manufacturing, service industry and government.	October 2009	29
Site 3	31,925 city 72,453 county	76.7% White 0.4% Black 29.4% Hispanic	138 miles	Agriculture, forestry, and ranching	September 2009	37
Site 4	36,648 city 102,498 county	86% White 0.9% Black 9.7% Hispanic	126 miles	Medical, marine, timber, and recreational	January 2009	57
Site 5	20,245 two towns 87,894 two counties	74.2% White 0.8% Black 34.9% Hispanic	268 miles	Agriculture, livestock, food processing, and forest products	January 2009	15
Site 6 SCH satellite	103,019 city 722,400 county	74.6% White 4.1% Black 14.2% Hispanic	30 miles	Marine, naval, manufacturing and retail, and technology	February 2011	10
Site 7 SCH satellite	122,363 city 1,969,722 county	62.63% White 2.3% Black 7.0%Hispanic	11 miles	Services, retail, commerce technology, biotechnology	February 2011	3

¹All information from the United States Census Quick Facts: (http://quickfacts.census.gov/qfd/states/53/5335275.html)

^2Percentages may exceed 100% as individuals could have endorsed more than racial and ethnic heritage

Recruitment strategies targeted PCP’s who referred their patients to the trial and resumed care after completion of the treatment provided in the trial. We initially considered recruiting the 30 to 50 providers who had participated in our telepsychiatry clinic. Instead, to include providers and patients from a larger geographic area, we opened recruitment to all PCP’s practicing near the seven central clinic sites. This approach optimized the representation of providers both with and without experience with telepsychiatry and of children in need of mental health services.

To foster partnerships with community providers, study investigators visited clinic sites and practices of PCP’s prior to initiation of the trial, posted flyers in clinics, provided informational packets to providers’ offices and published quarterly newsletters. We hosted meetings with continuing education presentations. When PCP’s referred children who did not meet eligibility criteria, the CATTS principal investigator sent a letter suggesting alternative treatment options. Telepsychiatrists sent referring PCP’s a clinic note after each treatment session and arranged transfer of each child’s care back to their PCP at the end of their participation in the trial. A total of 150 PCP’s referred patients to the trial. Of these, 88 referred one or more children who met all eligibility criteria.

Because families could express to their provider an interest in referral to CATTS, we also developed recruitment strategies geared to families. We built a CATTS website, established a toll-free telephone number, posted flyers in and sent information packets to schools and other key community organizations. We advertised in or provided interviews to small community newspapers and radio stations. CATTS therapists acted as community liaisons by giving presentations to schools and organizations. Additionally, we collaborated with community “champions” who encouraged local families to speak with their providers about the trial.

A total of 530 children were referred to CATTS, of whom 223 met study criteria, gave consent/assent to participate, and were randomized. The randomized sample included 163 boys (73%); mean age was 9.25 years (± 1.99; range of 5.5 to 12.9 years). Most participants (93.3%) were identified by caregivers as of European ancestry; median family income was $50,000-$74,999. There were no statistically significant differences between children in Groups A and B with respect to demographic characteristics, baseline diagnoses, or symptom severity.

CATTS employed a 1.0 full time equivalent of telepsychiatrist services from the Department of Psychiatry and Behavioral Medicine at SCH. Over the life of the trial, eight child and adolescent psychiatrists participated, of whom four were active at any time. The principal investigator trained telepsychiatrists to provide algorithm-driven pharmacotherapy³⁹ and psychoeducation for ADHD through VTC.

We considered the use of VTC to provide caregiver behavioral training from SCH using academic psychologists. However, our communities had master’s level therapists employed at local agencies or as independent practitioners; we hired these local therapists to serve as part-time CATTS therapists. Training and supervision of local therapists was provided remotely via digital recordings, VTC and telephone by a clinical psychologist at the research hub. Thus, CATTS was able to provide a sustainable asset to the community beyond the life of the trial.

Retaining Participants

Retention was a major concern with our novel model of service delivery and coordination. The first retention issue concerned continued participation by the central clinic sites. CATTS retained six of the seven sites in the trial. As noted in Table 1, enrollment varied among the sites, varied in part reflecting population base, duration of participation in the study, and experience with our telepsychiatry service. The small number of referrals from Site 5 was expected given its designation as a frontier community. Site 2 was a relatively new SCH satellite clinic that opened part time in 2007. It seemed to be a good choice for a clinical trial of children’s mental health services, although the lack of a longstanding relationship of the community PCP’s with SCH and the telepsychiatry clinic made it an uncertainty. Site 6 was a long time satellite clinic of SCH that had no experience with telemedicine services but did have VTC equipment that was used for administrative and educational purposes. Due to a lack of child psychiatric services in the community, the largest clinics had hired behavioral pediatricians to try to meet the growing need for behavioral health services. We were unable to recruit a “clinical champion” there, but through vigorous efforts, recruitment was increasing as the study was winding down. Site 7 was not retained. It is located in a city near Seattle that serves a large surrounding area of small towns and agricultural areas. We surmised, incorrectly, that PCP’s and families would prefer quick access to treatment through the trial over placement on the wait-list for access to community psychiatrists. We discontinued recruitment attempts after receiving few referrals in six months.

One factor on which retention of families in the CATTS intervention was predicated was whether they perceived a therapeutic alliance with a “virtual” psychiatrist and whether they perceived the services as effective. Practical factors were also important, including scheduling appointments around school and work schedules and distance from home to the central clinic site. In general, retention in the CATTS test intervention was facilitated by adhering to principles of patient-centered care. As each family in Group A enrolled, we scheduled all six sessions 3-4 weeks apart to enable families to coordinate their schedules. We honored participants’ need to reschedule appointments, as long as all sessions were completed prior to the final (25-week) outcome assessment. Cancellations were followed-up vigorously. For a few families, we provided reimbursement for fuel for travel and cell phone minutes to coordinate care.

Parents, children, and teachers received incentives for completing questionnaires; parents received a $25 gift card, children 10 to 12 years old received a $10 gift card for completing baseline and four follow-up assessments and a bonus of an additional gift card if they completed all assessments. Younger children did not complete assessments but received a $10 gift card when their parents completed their assessments. Teachers received a $20 gift card for each of five assessments they completed.

Participants in Group A completed a mean of 5.3 of 6.0 scheduled sessions; 95.1 % of Group B participants completed their single telepsychiatry consultation. The groups were comparable in their completion of the five outcome assessments with a mean of 4.8 (± 0.65) for Group A and 4.8 (± 0.61) for Group B.

Providing Evidence-based Interventions to Distant Sites

To provide care from different clinicians across multiple distant sites, CATTS employed various telecommunications technologies.

Psychiatry Intervention

The CATTS telepsychiatry intervention focused on guideline-based pharmacological treatment of ADHD and psychoeducation regarding the neurobiological roots of ADHD. Telepsychiatrists followed the medication guidelines developed by the Texas Children’s Medication Algorithm Project³⁹ which were included on an electronic decision-making and treatment documentation tool, termed WebCATTS.⁵⁰ Telepsychiatrists’ were provided guidelines of care to cover in each session. Fidelity to these guidelines was high across telepsychiatrists with a mean fidelity rating of 91.6% (± 9.5%) for Group A sessions and 89.3% (± 9.6%) for the Group B consultation session.

Behavior Training for Caregivers

The therapists’ intervention focused on helping caregivers develop knowledge and skills to manage behaviors of young children with ADHD. CATTS investigators adapted an evidence-based behavioral intervention for ADHD to a six-session format. Caregivers were given materials to guide practice between sessions. Therapist fidelity to protocols for the six sessions was high across therapists with a mean fidelity rating of 94.2% (± 9.7%) per session.

Coordination of Telepsychiatrists’ and Therapists’ Interventions

Treatment by therapists and telepsychiatrists had to be coordinated. Therapists greeted families, obtained vital signs, administered rating scales and quizzes, and distributed psychoeducation materials for review during telepsychiatry sessions. They logged relevant information onto WebCATTS for telepsychiatrists to review remotely and aid their decision-making. The therapist remained in the initial telepsychiatry sessions to learn about the child’s history and caregiver’s needs. After completion of the telepsychiatry session, therapists conducted caregiver training in front of the VTC unit to record sessions.

After sessions, telepsychiatrists and therapists documented treatment activities in WebCATTS.⁵⁰ The information was integrated electronically into an ADHD Management Plan that was sent to the referring providers to apprise them of the child’s progress and to families to prepare them for the next session. The final ADHD Management Plan included recommended follow-up steps for PCP’s and family to complete over the ensuing two months. WebCATTS was an effective tool to document clinical care across sites. Vital signs, ADHD rating scale scores, quiz scores, session content, medication choices, families’ adherence to medication recommendations and behavioral assignments were recorded for 97% of sessions.

Telepsychiatrists provided services through VTC in real time utilizing high bandwidth (384 kbits/sec to 1.0 MB/sec) connections through a secure T1 line. Therapists used the equipment to set up sessions for the telepsychiatrists and to record caregiver training sessions. The staff in the SCH Audiovisual Department trained the telepsychiatrists and therapists in the technical use of the equipment. The CATTS principal investigator trained the telepsychiatrists to use the equipment to provide treatment. Study clinicians were provided a telephone number for a rapid response from the Audiovisual Department whenever problems developed. During the trial, two of 650 scheduled sessions had to be canceled due to technical problems; all other sessions were completed. All sessions were recorded, and 217 sessions were randomly selected for independent rating by research assistants of technical difficulties. Of these, 158 (73.0%) were rated as having no difficulties and five (2.4%) sessions were rated as having severe technical difficulties although the sessions were completed.

Coordinating Research Activities

1. Scheduling Intervention Sessions

Scheduling participants at seven clinics for six sessions 3-4 weeks apart proved a major challenge. The trial honored both site staff requests to limit study appointments to selected blocks of time and family requests to schedule appointments around school, work, and other obligations. The part-time telepsychiatrists and therapists also had limited hours to contribute to the trial. To manage complexities in coordinating appointments, CATTS hired a dedicated scheduler located at the research hub; each site designated a staff member to work with the CATTS scheduler.

At the four SCH satellite clinics, the scheduler logged appointments and reserved rooms directly via the shared electronic medical record. At the other three clinics, the scheduler coordinated the appointments with designated clinic staff.

2. Communication between Therapists at the Spoke Sites and Staff at the Research Hub

Utilizing community therapists in research trials requires training in and ongoing supervision of adherence to research requirements such as protecting human research subjects, consenting and assenting participants, maintaining documentation, and representing the trial locally. Initial research training was conducted with therapists through VTC, email, websites, DVDs, and conference calls. Weekly team meetings were held through VTC with all community therapists and hub-based research staff including the supervising clinical psychologist and principal investigators for the duration of the trial. These meetings were the main mechanism for team-building and problem-solving for therapists and research staff who never met in-person. The agenda reviewed general topics, such as protocol, recruitment, and use of data-gathering forms, and site-specific issues, such as interactions with the community clinic staff or local event planning. Therapists provided feedback to the research team that prompted problem-solving issues that might be relevant to another site or to all sites. For example, when a therapist reported that the color printers that we had purchased to print educational materials were unreliable and inefficient, the hub staff prepared a binder of educational materials for each participant and mailed them to therapists to distribute to families. One therapist reported difficulties when participants checked in with the site clinic’s front desk staff. Another therapist recommended her approach for bypassing front desk staff which relieved confusion for participants, decreased burden on the clinic staff, and created greater efficiency for the therapist.

Therapists at the spoke sites maintained communication with research staff through a secure portal,⁵¹ HubCATTS, that allowed research and clinical staff to exchange records and forms for completion during each session. For example, diagnostic interview results that were documented electronically during in-person interviews were transmitted to the research staff. Copies of the consent and assent forms were downloaded from the portal, reviewed and signed onsite, and copies were returned to the research team through HubCATTS portal.

3. Data Collection and Management: DataCATTS

We had limited information to guide our planning for data collection from participants in multiple distant sites. No research staff was located at the sites to supervise completion of assessment questionnaires. Mailing questionnaires to families risked delays. Telephone administration of questionnaires was staff-intensive and expensive. We developed a secure website, DataCATTS, on which families completed assessments from their personal computers at their convenience.

To collect self-administered questionnaires remotely, Catalyst WebQ, a Secure Sockets Layer encrypted web questionnaire system, was selected.⁵¹ Questionnaires were stored in a database until requested by a participant. The WebQ software was used to build the questionnaire and to display it in the participant’s browser. Upon completion, the participant submitted the responses via a secure internet connection directly into the outcomes database where they were stored in a secure database with a study identifier.

Most CATTS participants (96%) in both Group A and Group B completed questionnaires online through DataCATTS. Some families without personal internet access completed the questionnaires at the workplace or from a relative’s computer. The remaining 8 families (4%) completed their assessments by telephone. Teachers in each trial arm completed an average of 3.3 out of five questionnaires. All teachers (100%) completed assessments online.

Discussion

The CATTS study used telecommunications technologies to provide evidence-based mental health interventions to children living in underserved communities and to include these children in a clinical trial of the interventions.^{8, 9, 12, 13} The telepsychiatrists’ and therapists’ adherence to treatment protocols and participants’ high rate of completion of intervention sessions strongly support the viability of VTC as a means of providing evidence-based care. Further, VTC was an effective tool for training and supervising therapists in distant sites and should be considered a means to disseminate evidence-based psychotherapies beyond academic settings.

The trial also demonstrated the ability to use VTC to engage families and their PCP’s in research. Many PCP’s across multiple geographic areas referred patients to the trial; many of these providers had no prior experience with our telepsychiatry service. High rates of participant completion of questionnaires in both trial arms provided evidence of families’ willingness to complete research requirements and of convenience to families of using personal computers to complete online questionnaires. VTC and secure portals made it possible to extend representation of participants in mental health services research to children and parents in non-metropolitan areas.

CATTS also provided cautionary lessons for future mental health trials with referred samples of children living in underserved communities. While retention turned out to be less challenging than we anticipated, recruitment was a challenge. We encountered challenges in obtaining access to larger group physicians’ practices. In retrospect, focusing recruitment efforts on small practices may have been more fruitful. Absence of a “clinical champion” and lack of experience with telepsychiatry were factors associated with poor accrual at some sites.

Over half of the referred children were not eligible for the trial. The growing ability of PCP’s to manage “uncomplicated” cases of ADHD in their own practices ^{4, 5} likely resulted in referral of complicated cases. Many patients who were referred did not satisfy the enrollment criteria of this clinical trial. Furthermore, many children were ineligible on the basis of criteria that were outlined clearly on recruitment materials, e.g., age or custody status. We learned that investigators must maintain active communication with referral sources and adapt recruitment strategies to community context.

Finally, in CATTS we worked with distant community sites that were not reimbursed for their participation in a clinical trial. We strove to minimize burden to the sites and their staff, and they were gracious in their commitment to the trial. But problems and confusions were inevitable, especially for those sites outside of the SCH network with which we did not share an electronic medical record or clinical scheduling system. We did not anticipate hiring a dedicated scheduler at the research hub and had to request an administrative supplement to cover the added expense. Once the scheduler was hired, research staff burden was decreased, and staff at the clinic sites appreciated having a designated “go to” person.

In conclusion, CATTS was implemented using methods and procedures intended to facilitate participation in a clinical trial by children living in multiple dispersed sites. Study findings will be used to make service and policy recommendations for providing evidence-based care to traditionally underserved populations. Experiences from CATTS and other such trials are expected to advance development of the technologies needed to allow diverse populations to participate in research with the ultimate goal of disseminating services that meet the mental health care needs of all the nation’s children.