Improving Access to Child and Adolescent Mental Health Care: The Choice and Partnership Approach

Abstract

Objective

The Choice and Partnership Approach (CAPA) is designed to improve access and quality of pediatric mental health care. We tested whether CAPA improved access in an academic pediatric hospital.

Method

We used de-identified administrative data to compare access pre- (2011) and post-CAPA (2013).

Results

Wait time to first appointment in 2011 was 225.3 days (95% CI = [211.0, 239.6], N = 364), compared to 93.0 days (95% CI = [89.2, 96.8], N = 838) in 2013 (p<.001). Mean wait time between the first and second appointments was 59.2 days (95% CI = [46.5, 71.9], N = 86) in 2011, compared to 95.9 days (95% CI = [90.3, 101.5], N = 487) in 2013 (p < .001). However, overall mean wait time from referral to second appointment decreased from 271.2 days (95% CI = [236.5, 305.9], N = 86) in 2011 to 168.9 days (95% CI = [161.6, 176.2], N = 487) in 2013 (p < .001). Provider productivity increased from 32.6 to 57.0 first appointments/FTE/year. Depending on the question, 65 to 95% of parents and children gave positive answers about CAPA.

Conclusions

CAPA implementation was associated with more patients served, decreased waiting time to first appointment, and higher productivity.

Background

About one in five North American children or youth meets lifetime criteria for a mental disorder, and over half of adult disorders begin by adolescence (Davidson, 2010). Untreated pediatric mental illness leads to complications including lifelong disability, substance abuse, and suicidal behaviour (Kessler, Berglund, Demler, Jin, & Walters, 2005; Kessler, Chiu, Demler, & Walters, 2005; Wang et al., 2005). Nevertheless, 75% of children and youth suffering from a mental disorder do not receive needed care (Waddell, McEwan, Shepherd, Offord, & Hua, 2005).

One barrier to treatment is long wait times (Davidson, Kutcher, Manion, McGrath, & Reynolds, 2010). In Canada, two-thirds of urgent cases wait longer than 14 days and nine of ten regular cases wait longer than 30 days (Kowalewski, McLennan, & McGrath, 2011). Long wait times are harmful: children’s symptoms get worse and their functioning declines. Families lose hope and become less likely to engage in care (Schraeder & Reid, 2014). Those who have waited more than 90 days are less likely to attend appointments when they eventually get them. Unfilled slots in clinicians’ schedules are inefficient (Rawlinson & Williams, 2000; Sherman, Barnum, Buhman-Wiggs, & Nyberg, 2008).

However, reducing wait times is difficult. Vallerand and McLennan surveyed 379 child and adolescent mental health agencies about wait lists (Vallerand & McLennan, 2013). Forty-one management strategies were identified, but none were routinely successful. Research on wait time interventions suffers from a lack of rigorous evaluations (Kowalewski et al., 2011) and variability in how wait times are measured (Davidson et al., 2010).

The Choice and Partnership Approach

The Choice and Partnership Approach (CAPA, http://www.capa.co.uk/) is a model of care delivery to improve mental health services for children and adolescents (York & Kingsbury, 2009). Patient and family needs drive the organization of services. CAPA is a shared decision-making model that treats the patient and family as experts and supports them in becoming stewards of their mental health.

CAPA reduces wait times using several components. It is grounded in demand/capacity, queuing theories, and other system improvement methods, including Lean (Kim, Spahlinger, Kin, & Billi, 2006). Strategies include care pathways, full booking processes, reallocation of FTEs based on patient demand, quantifying care providers’ capacity, streamlining referral, smoothing patient flow through the system, ensuring that services match population needs, and matching clinicians’ skills to patients’ problems.

Following referral and match to service eligibility criteria, the first patient and/or family (hereafter, the patient/family) contact is the Choice appointment (see Figure 1). During this 60–90-minute session, a clinician and the patient/family identify the main problem(s) and the family’s strengths. At the Izaak Walton Killam (IWK) Health Centre, allied health professionals provided Choice appointments, but physicians can also play this role. They assess risk and explore how the family can ameliorate the problem. The clinician and patient/family decide together whether treatment is necessary and jointly select the next step in care.

Figure 1.

The CAPA Model

Note. Dashed lines reflect that in Partnership, Specific Partnership work may be pulled in as needed.

The patient/family receives education and resources at Choice and, if indicated, treatment options to use at home. If the best treatment is not available within the system, the patient is referred to community-based resources. A patient/family designated for treatment within the system is matched with a clinician with the skills and an open appointment, on a calendar accessible to all providers on the team. This “Partnership” treatment appointment is booked at the end of the Choice appointment. This full-booking process promotes patient/family engagement and improves patient flow.

“Partnership” conveys CAPA’s commitment to collaboration and agreement between the clinician and the patient/family about treatment goals. The first session in Partnership builds on the case formulation established in Choice, shifting into specific treatment goals and initiating that treatment. Partnership has two layers: Core and Specific. Core Partnership is work that includes the standard assessment and treatment skills that comprise most of child mental health care (e.g. Cognitive Behavioural Therapy (CBT) for anxiety disorder). Specific Partnership is a specialized assessment or treatment to augment Core Partnership, e.g., medication to augment CBT for obsessive-compulsive disorder.

A full mental health diagnostic assessment for diagnostic clarification is layered into care when the diagnostic profile is unclear and treatment requires a comprehensive assessment to move ahead. This type of assessment is offered within Specific Partnership time by a clinician or physician with the required skills and builds on the assessment information gathered in the Choice appointment but is more targeted in its intention than the Choice appointment.

Most cases begin treatment in Core Partnership rather than a specialized clinic focused on a single diagnosis. Over-commitment of system personnel to specialized clinics causes three harms. First, personnel are fixed in specialist roles. Specialists lose general care skills and non-specialists lose skills to manage problems such as eating disorders, psychosis, or obsessive compulsive disorders. Second, some clinics will have excessive demand, causing bottlenecks, while others will have insufficient demand and use staff inefficiently. Third, directing a patient to a specialized clinic can encourage staff to reduce the patient to their diagnosis, rather than treating the whole person.

CAPA is widely used in the UK, Australia, New Zealand, Belgium, and in several Canadian sites. Anecdotal and unpublished reports about CAPA have been positive, but only three published studies have evaluated CAPA.

A pre-post design study of a rural Australian mental health clinic used mixed methods (qualitative evaluation of meeting minutes and administrative data). The goals were to determine how well the agency had complied with the 11 Key Components of CAPA (CAMHS Network, 2013) and to assess whether wait times changed with implementation of CAPA (Naughton, Basu, O’Dowd, Carroll, & Maybery, 2015). The investigators did not score their adherence to the 11 Key Components, but presented qualitative data suggesting they had moderately good compliance. The wait time between referral and the first face-to-face contact decreased from 64 to 11 days.

In a London inner city clinic, a pre-post study was conducted with an embedded qualitative longitudinal study (Fuggle et al., 2015). Administrative data comprising wait times and discharge Clinical Global Impression (CGI) Scale scores (Busner & Targum, 2007) were analyzed for the six months pre-CAPA, six months of CAPA implementation, and six months post-implementation. Fifty-four patients seen in a Choice appointment were chosen three months after CAPA implementation had begun and examined for satisfaction with the Choice appointment and goal-setting for the 28 who went on to Partnership care.

The number of first appointments increased significantly by 47%. Wait times to the first appointment decreased from 82 days to 71 days (non-significant). There was no prepost difference in mean discharge CGI scores, indicating no decline in the quality of care. Ninety-three percent of the longitudinal group were satisfied with the Choice appointment. All Partnership patients set treatment goals and experienced significant improvement on these goals.

A mixed methods (survey and focus group) study of CAPA surveyed all U.K. child and adolescent mental health agencies about use of CAPA (Robotham, 2009; Robotham, James, & Cyhlarova, 2010). Forty-eight percent responded, with 97 agencies using CAPA. Responding agencies were then surveyed about adherence to the 11 CAPA Key Components. Fifty-two percent responded, with eighteen self-categorized as High Utilizers, twenty-eight as Medium Utilizers, and seven Low Utilizers.

Four High and two Medium Utilizer agencies were then interviewed in focus groups. Agency personnel reported that the success of CAPA depended on adherence to the 11 Key Components. Reported challenges with CAPA included that specialist providers can feel devalued, CAPA does not work well without strong administrative support, and that post-Choice bottlenecks will continue if CAPA Key Components are not followed. Finally, the CAPA principle of discharge when the main treatment goals are accomplished is difficult for many providers.

In summary, unpublished data and three studies suggest that CAPA can reduce wait times. However, evaluations using larger quantitative datasets are needed. Moreover, previous studies have focused on community and secondary care level sites. There is no information about the effect of CAPA on service delivery in academic, tertiary care settings, where wait times are often long and the complexity and severity of the disorders is high. Finally, no study has examined how

Methods

Setting and Description of CAPA

The IWK is a tertiary care academic women and children’s hospital in Halifax, NS. It has a complex pediatric mental health mandate: secondary and tertiary care for the surrounding metropolitan area, tertiary care for the province (e.g., IWK has the only child and adolescent inpatient psychiatry unit), and quartenary care for select mental health services (e.g., eating disorders, inpatient concurrent substance abuse/mental health disorders) for the Maritime provinces and Newfoundland and Labrador. There is a single entry for non-emergency referrals: the Central Referral service. The NS Department of Health and Wellness allows physician, care provider, and self-referrals for mental health care. In 2011, 1111 unique children and youth received care through the IWK Outpatient Mental Health Service in over 8500 appointments.

IWK’s implementation of CAPA was, to our knowledge, the first in a tertiary care, academic medical centre. CAPA was implemented in the Outpatient Mental Health Service in April 2012. The first step was a “wait-list blitz” between January and March 2012. All families on the wait list were called or contacted by mail to ask if they still needed services. Any who did not or who did not reply after six weeks were removed from the list. Those still needing care were booked for a Choice appointment starting in January 2012.

Central Referral was redesigned to support CAPA. Access navigators were trained to utilize screening tools and standardized clinical criteria to categorize referrals. This allowed them to redirect referrals not appropriate for the Outpatient Mental Health Service to community or other health services.

Seven full time equivalents (FTEs) of allied health professionals and 0.5 FTEs of psychiatry were added over the period of study, although no new personnel were hired. Instead, we redistributed personnel from specialist clinics to newly created multi-disciplinary teams at each site. Every team included clinicians and physicians with a range of Core and Specific Partnership skill sets. CAPA enables users to structure the delivery system to meet the unique needs of a mental healthcare setting. Therefore, at the IWK, we used the scarce resource of psychiatry for Specific Partnership and allied health clinicians did the Choice appointments, as well as both kinds of Partnership.

For patients who were severely ill or required a specialized skill set, we created new Specific Care Clinics. The Specific Care Clinics had two purposes. First, they saw patients who required specialized care due to the complexity or acuity of their presentation. Second, they supported clinical capacity development by providing clinicians with opportunities to learn new skills (i.e., Maudsley treatment for Eating Disorders). Clinicians with these specialized skills did not belong exclusively to a Specific Care Clinic, but instead they spent most of their time with a variety of cases on the outpatient multi-disciplinary teams. In addition, the Central Referral team stopped referring every patient labeled with a disorder to the corresponding specialized clinic. Instead, providers pulled in specialized care offered by Specific Care Clinic physicians and clinicians to augment their Core Partnership treatments. Patients were seen by Specific Care Clinic providers if: 1) their Core Partnership provider requested special care; 2) diagnostic clarification was required because of complicated co-morbidities; or, 3) one of our problembased care paths included the need for Specific Care Clinic involvement (e.g., the Care Path for Eating Disorders indicated that symptoms of medical instability required the Specific Care Clinic providers). Specific Care Clinics also became internal training hubs. Other clinicians could learn the skills to treat moderate to severe patients previously managed exclusively by the specialist clinics, (e.g., Maudsley approach for Eating Disorder treatment).

The founders of the CAPA model trained our clinicians and physicians. CAPA’s long-term success requires an environment of continuous quality improvement. Therefore, daily operations of the Outpatient Service were reorganized to provide such an environment. Changes were made based on the 11 Key Components of CAPA (http://capa.co.uk/11-key-components/). These include: Choice training, job planning for providers based on clinical demand, and detailed assessments of providers’ skills and competency. There are also weekly Team Meetings (to review Choice appointments, evaluate progress on individual providers’ Partnerships with patients, provide peer support), weekly patient flow and quality-focused meetings of clinical team leaders, and quarterly “Team Away Days” for the entire Outpatient Service (providing education on a topic, discussion of operational, and quality issues arising in the previous quarter). We adhered as closely as possible to the 11 Components, and prior to the 2013 data point, in September 2012 scored 14/19 points on the list defined by CAPA founders Kingsley and York (http://capa.co.uk/11-key-components/).

Design

Wait times and staff productivity were studied in a pre-post design. Pre-CAPA was defined as January 1–June 30, 2011 and post-CAPA was January 1–June 30, 2013. Patient and family experiences of Choice appointments were studied with a cross-sectional survey administered after the CAPA implementation. The IWK Research Ethics Board (REB) judged that the use of de-identified administrative data was not human research and did not require REB review.

Procedures

Dates of patients’ first and second visits were obtained from the IWK’s Central Referral and Decision Support Offices records and from chart reviews. However, some charts were in long term storage and could not be retrieved. 154 (of N = 364) charts were available and reviewed for the pre-CAPA period and 794 charts (of N = 838) were reviewed for the post-CAPA period.

Measures

Wait Times from Referral to the First Appointment

Time in days from when the patient’s referral was received at Central Referral until the patient’s first appointment.

Wait Times from First to Second Appointment

Days from the first until the second appointment.

Productivity

Clinician productivity was measured by dividing the number of first visits during each period by the number of clinician FTEs. Because of the limitations imposed by hand review of charts, we only examined changes in productivity for the first appointments.

Experience of Choice Survey

Patients and families seen for Choice appointments between April and August of 2013 were asked to fill out the Commission for Health Improvement Experience of Services Questionnaire (CHI-ESQ). This version of the CHI-ESQ was adapted by the developers of CAPA (Brown, Ford, Deighton, & Wolpert, 2014; East Herts CAMHS, 2010). It has been used in other audits (CAMHS Network, 2013) and by Fuggle and colleagues (Fuggle et al., 2015). A typical item is: “I feel that the people who have seen my child listened to me.” Response choices are three levels, ranging from “Certainly True”, “Partly True”, “Not True” and “Don’t Know”. This study used a youth self-report version of the CHI-ESQ for youths nine years and older and a parent-report version for children younger than nine.

Statistical Analyses

Chi-squared and t-tests were used to test statistical significance in the pre- versus post-CAPA comparison. Analyses were done using IBM-SPSS V.21.

Results

Patient Characteristics

Figure 3 shows the flow of patients from the first appointment to the second appointment, including those not available for chart review. The number of referrals received in Central Referral increased 175% during the study period (424 in 2011 to 1165 in 2013). The number of referred patients receiving first visits increased by 130% (364 to 838). The proportion of patients referred to Central Referral who did not attend a first visit increased from 14% in 2011 to 28% in 2013 (p < .001). The proportion of patients who attended first visits and went on to attend second visits increased from 2011 to 2013, although the difference was not statistically significant (p = .24). Table 1 reports the characteristics of the patients whose charts we reviewed in the two study periods.

Figure 3.

Patient flow pre- and post-CAPA

Table 1.

Reviewed charts: characteristics of patients

	2011	2013	p
Charts reviewed	154	794	-
Mean Age (SD)	11.9 (3.6)	12.5 (3.8)	.071
Female	54.5%	54.8%	> .9

Wait Times

Figure 4 displays wait times in 2011 and 2013, calculated from patients with reviewed charts. The mean wait time to the first appointment in 2011 was 225.3 days (95% CI = [211.0, 239.6]), compared to a mean of 93.0 days (95% CI = [89.2, 96.8]) in 2013 (p<.001). The CAPA manual indicates that approximately two-thirds of the Choice appointments will go on to Partnership and in our service, 62% did so (York & Kingsbury, 2009).

Figure 4.

Changes in Wait Times Pre- and Post-CAPA

The mean wait time between the first and second appointment was 59.2 days (95% CI = [46.5, 71.9]) in 2011, which rose to 95.9 days (95% CI = [90.3, 101.5]) in 2013 (p < .001). However, the fall in the wait time from referral to the first appointment was substantially less than the rise from the first to the second appointment. Therefore, the wait time from referral to the second appointment decreased from 271.2 days (95% CI = [236.5, 305.9]) in 2011 to 168.9 days (95% CI = [161.6, 176.2]) in 2013 (p < .001).

Productivity

The number of FTEs providing care in the outpatient clinics increased from 22.3 to 29.4 over the period of the study. In 2011, care provider productivity was 32.6 first appointments per FTE per year. By 2013, care provider productivity was 57.0 first appointments per FTE per year, a 75% improvement.

Experience of Choice

Table 2 presents the results of the CHI-ESQ Choice data. Data were collapsed into dichotomous outcomes: “Certainly true” and “Partly True” vs. “Not True”. Most parents and youth reported positive experiences with the Choice appointments on all items, with percentages of parents and youth feeling positive ranging from 96% (Parents feeling listened to) to 65% (Youth feeling Choice appointment made them more willing to talk). More parents felt positively than youths, but nearly all youths felt listened to and understood the services being offered.

Table 2.

Experience of Choice Questionnaire

	% Responding All or Partly True
Experience of Choice Questionnaire	Parent (n=125)	Youth (n=81)
The first appointment was a one off: did this make you feel more willing to talk? (compared to more cautious)	85	62
Did you feel that people here listened to your concerns?	96	92
Was today’s session helpful for you?	91	75
Did you feel involved in deciding what would happen next?	85	84
Do you understand what is being offered by the service and the reasons for it?	93	90
Did the appointment give you new ideas that might help your situation?	82	78
Do you have clear ideas about what to try at home now to help the problem?	75	76

Discussion

Implementation of CAPA significantly improved mental health care access by reducing waits from referral to first visit for child and adolescent outpatient services and from referral to second visit, and by increasing the number of patients treated. These results are consistent with the findings from others (Naughton et al., 2015; Robotham et al., 2010) and the results of numerous local audits available on the CAPA website.

We are the first to test whether wait times from the first to the second appointment were affected by CAPA. Although we predicted no change, the mean wait from the first to the second appointment increased. We believe that the main reason this happened was an inability to accurately predict our patient flow. CAPA requires quantitative estimates of patient demand for Choice appointments, which then informs estimates of Partnership need. Because of limited data management capabilities (e.g., no electronic health record), the estimated numbers of Choice and Partnership appointments required were not as accurate as desired.

The increase in first visit to second visit wait times likely resulted from a 175% increase in referrals between 2011 and 2013. Although clinical FTEs increased during the same period, the increase in visits was greater than the increase in FTEs. Thus under CAPA clinician productivity increased 75%. To the best of our knowledge this is the first report of how CAPA affects clinician productivity.

The increase in referrals was likely due to the “woodwork phenomenon”, i.e., when the community heard that there was a new, accessible system in place, more families sought care and more clinicians referred them. The developers of CAPA predicted this increase would occur and those wanting to implement CAPA should plan for it. However, despite the increase in referrals, wait times from referral to the first and second appointments significantly decreased.

The proportion of patients referred to Central Referral who did not attend a first appointment unexpectedly increased from 14% to 28%. We speculate that this was due to the reorganization of the Central Referral office, which now screens out referrals not meeting criteria for care at the Outpatient Service, e.g., requests for custody assessments. We have since improved our method of matching families to care at the point of Central Referral and our no-show rate for Choice has fallen to 8% for the first five months of 2017.

Patient and family experiences reported on the CHI-ESQ were positive. Our results are consistent with those reported by others (Brown et al., 2014; Fuggle et al., 2015). Although we only sampled a sub-group of our patients, our study and the previous studies suggest that Choice appointments are well-received by patients and families. Future research should include measuring patient/family satisfaction with the Partnership process.

What mechanisms could explain CAPA’s effect on service delivery? We believe that there are four possible explanations. First, the structure of the Choice appointment increases system capacity for the delivery of treatment, increasing patient flow through the system and decreasing the bottleneck at the first visit. Second, CAPA’s method of using a patient’s needs to “pull in” highly specialized services, rather than the Outpatient Service “pushing” such services towards all patients at all times, also increases providers’ capacity to deliver treatment and layers in required skills. Third, Partnership care, patient/family goal-based treatment plans, and discharge when goals have been accomplished further increases patient flow. Fourth, reorganization of the clinics and changes in operational practices, including full-booking also decreased wait times by decreasing internal delays in obtaining care. Productivity was increased through more efficient care, such as centralized control of clinician’s schedules that modified each provider’s schedule on a quarterly basis based on patient demand, rather than each provider organizing their own schedule without knowledge or attention to system demand.

Limitations

Our study used a pre-post design and investigated only one site. With this design, we cannot be certain that CAPA was the cause of the increased access and productivity. Future studies should use a design which provides more information about the causal effects of a CAPA implementation.

We could not review charts for all the patients seen during the study periods. This means we have incomplete data on the wait times from the first to the second visit. Moreover, our sample may have been biased if patients whose records were sent to long term storage were systematically different from other patients. Similarly, we did not measure clinician satisfaction before and after CAPA. Because we had no pre-CAPA baseline for patient/family experiences, we cannot conclude that Choice improved their satisfaction, compared to how they experienced appointments in the original model.

Conclusions

Implementing CAPA in an academic mental health system increased the ability of the system to handle higher demand and reduced the wait time from referral to first visit, and increased clinician productivity. Many gaps remain in our knowledge of CAPA. In addition to addressing the limitations previously discussed, future research should measure the effects of CAPA on the cost of care, replicate our finding of increased productivity, and study whether CAPA improves patient-reported outcomes, decreases symptoms, and/or increases child and family functioning.

Figure 2.

Definitions of Wait Times