Abstract
We investigated whether a service-planning document outlining recommendations for what providers should address in treatment (i.e., targets) and the associated clinical techniques they should employ (i.e., practices) influenced the targets and practices that providers reported actually implementing during the subsequent treatment episode. Participants included 94 youths ages 4 to 17 (M = 13.57, SD = 3.59) who received community-based mental health services from the Hawai'i Child and Adolescent Mental Health Division. Data on targets and practices were compared across initial Mental Health Treatment Plans and Monthly Treatment and Progress Summaries. Data were analyzed using two-level, generalized mixed effects models with two-way cross-classification or linear mixed effects models. Providers were more likely to report the use of targets and practices in treatment if they were included within the treatment plan. In addition, the more closely targets addressed during treatment followed the recommended targets from the treatment plan, the more closely implemented practices followed the recommended practices listed in the treatment plan. Furthermore, as providers shifted their focus to different targets, a shift in their use of practices was also evident over time. Last, practices for which there is demonstrated efficacy for particular targets were more likely to be used. Service planning documents appear to help organize care; however, results also suggest possible limitations to the current system. These findings highlight potential areas for improvement in planning and care delivery.
The Child and Adolescent Service System Program was created to promote the development and implementation of comprehensive, coordinated, community-based systems of care (SOC) for youth with serious mental health needs (Stroul & Friedman, 1986). Such networks have since been shown to effectively improve access to and quality of care; however, these system-level improvements have not necessarily translated to better client-level outcomes (Bickman, Summerfelt, & Noser, 1997; Burns, Farmer, Angold, Costello, & Behar, 1996). Bickman and colleagues (1999) hypothesized that system-level reforms may not be able to strongly influence client-level outcomes due to the multitude of steps that take place between these two processes and the diluted impact of system-level changes at each subsequent step. For instance, system-level reform may be linked to changes in treatment, which is then linked to the way in which providers and clients interact, which may then finally be linked to client-level outcomes. This suggests a need to target reform at multiple steps along this continuum in order to positively impact client-level outcomes and to connect everyday service activities with strategic system initiatives.
The Hawai'i Child and Adolescent Mental Health Division (CAMHD) is one SOC that employs a three-step treatment planning process that aims to provide a comprehensive, high-quality, coordinated package of services that are tailored to the distinct needs of each client (Hawai'i Department of Education and Department of Health, 2006). In this system, mental health professionals first conduct comprehensive assessments with the family, which result in treatment recommendations that are summarized in a Mental Health Assessment. Next, a Coordinated Service Plan maps the recommendations onto a myriad of services across varying agencies to create a multifaceted system of care that targets all indicated concerns. Finally, Coordinated Service Plans guide service procurement and are then translated by service providers into mental health treatment plans (Treatment Plans), which specifies the intended targets of treatment (e.g., depressed mood) as well as the practices or clinical technique (e.g., relaxation) that provider(s) will use to address those targets. Each step essentially produces a document that guides the development of a subsequently more detailed set of recommendations. Unfortunately, in assessing the stability of service recommendations across the three service-planning documents, findings revealed an average retention rate falling below 50% for both targets and practices, indicating low levels of stability of codes (Young, Daleiden, Chorpita, Schiffman, & Mueller, 2007). This suggested a potential weakness in the coordination of services across the treatment planning process and raised concerns about the extent to which these planning documents actually influence service provision.
The ultimate goal of this complex individualized planning process is to help inform a provider’s service implementation (i.e., selection of targets and implementation of practices) in order to improve the quality of mental health care for youths. In considering factors associated with higher rates of execution of specific service behaviors, research indicates the importance of developing action plans that provide individuals with concrete instructions on when, where, and how they can engage in the desired behavior (e.g., Schwarzer, 2008). This is supported by research on general practitioners, which indicates that use of plans may increase the likelihood of a practitioner carrying out a recommended intervention procedure (Verbiest et al., 2014). However, further research is needed to understand how service planning is linked to actual service provision among mental health providers. To this end, this study explored whether use of Treatment Plans developed by CAMHD service agencies were meaningfully connected to the services their providers delivered to the youth within the year following the development of the Treatment Plan.
Given that the Treatment Plan contained the most refined and definitive treatment recommendations, we investigated whether recommendations of treatment targets and practices within this document affected the odds of those targets being addressed and those practices being implemented during treatment. Furthermore, we examined whether these effects changed over time. Because the Treatment Plan was developed by the agency whose providers were reporting the delivered services during the treatment episode, we hypothesized a significant relationship between the two. However, we expected the effects of the Treatment Plan to diminish over time as the client developed and circumstances evolved.
Although Treatment Plans should in theory inform service provision, simple adherence to a plan does not necessarily indicate “quality” treatment. Theory suggests that certain practices (e.g., exposure) may possess a better “clinical fit” with certain target problems (e.g., anxiety) than with others (e.g., conduct; Chorpita & Daleiden, 2009; Chorpita, Daleiden, & Weisz, 2005). Thus, if providers focused care delivery on the targets indicated in the Treatment Plan, they should also theoretically adhere closely to the practices outlined in the Treatment Plan. Therefore, we expected to see a shift in the implemented practices as providers shifted their targets and hypothesized that the selection of planned targets and planned practices would be linked to one another over time. Similarly, we expected providers to change the practices they implemented as they shifted the targets they addressed over time. Thus, we investigated whether changes in practice use were associated with changes in targets. Last, given the surge in focus on evidence-based care in community-based settings, we specifically examined whether the “clinical fit” of practices with targets as demonstrated by research (Chorpita et al., 2011) influenced the odds of that practice being implemented.
METHOD
Participants
Archival records of youth registered in the CAMHD system were utilized. Specifically, a previous study (Young et al., 2007) identified a random selection of cases (n = 200) who were registered for at least 30 days. The present study utilized data from this sample given the availability of target and practice data for one or more Treatment Plans and data from providers regarding reported endorsement of targets and practices during the treatment episode from Monthly Treatment and Progress Summaries (Treatment Summary) collected within 12 months following the completion of the coded treatment plan(s). Treatment Plan data were available for 111 cases. Of these, five cases did not have a treatment episode during the targeted time interval, and 12 had no available Treatment Summary data, resulting in a sample of 94 cases.
Youths averaged 13.57 years of age (SD = 3.59), and the majority were male (55%); 44% were Mixed Ethnicity, 19% were Caucasian, 11% were Asian, 10%were Native Hawaiian/Other Pacific Islander, and 1% were Other. Information on ethnicity was not available for 16% of youths. Diagnostic information was available for 52% of the sample. Of these, the diagnostic breakdown included 65% disruptive behavior, 53%substance use, 38.8% mood, 25%attention/hyperactivity, 25% anxiety, 16.3% adjustment, 6% personality, 6% learning, and 4% other (e.g., dissociative identity) disorders.
Measures
Monthly Treatment and Progress Summary (Treatment Summary; Child and Adolescent Mental Health Division, 2003)
The Treatment Summary is a provider report form developed to measure service format and setting, treatment targets, clinical progress, and practice elements utilized with individual clients on a monthly basis. Providers were asked to indicate the amount of progress made toward each target, as well as the practice elements (e.g., “relaxation,” “rewards”; see Chorpita et al., 2005) used to address those targets. This study examined Treatment Summary data collected up to 12 months following the development of the Treatment Plan.
Previous studies indicated good levels of reliability for coding of the treatment target and practice elements (Daleiden, Lee, & Tolman, 2004; Nakamura, Daleiden, & Mueller, 2007) and support for target convergent and discriminant validity when compared with Diagnostic and Statistical Manual of Mental Disorders (DSM-IV; American Psychiatric Association, 1994) diagnostic categories (Daleiden et al., 2004) and criterion and discriminant validity regarding rates of target-related improvement compared with other measures of functioning (Nakamura et al., 2007). Borntrager and colleagues (2013) found adequate concordance between therapists and independently trained coders on commonly reported practices.
Service Guidance Review Form (SGRF; Young et al., 2007)
The SGRF codes for the presence/absence of treatment targets and practice elements in treatment planning documents. For the purposes of this study, we examined SGRF data that stemmed from coding each youth’s Treatment Plan. The SGRF adopted codes from the Treatment Summary coding system and incorporated additional codes relevant to treatment planning and contextual information (e.g., educational background of document author). The SGRF gathers information on: (a) service allocation information and diagnostic impression, (b) treatment targets, and (c) practice elements. ICC scores demonstrated a high level of reliability across all document types by all raters for both treatment targets and practice elements.
Analytic Approach
SGRF data were reviewed to identify cases for which one or more Treatment Plans had been previously coded. In the event that multiple Treatment Plans were created (three cases), targets and practices were considered present if they occurred in any of the case’s plan. Three of the Treatment Plans were missing the document date necessary to identify the target time frame for Treatment Summary data collection. The document dates for these three plans were estimated by calculating the average number of days (M = 12.14) between Treatment Summaries and Treatment Plans in the rest of the sample. Data analyses were limited to codes that were available in both the SGRF and the Treatment Summary.
Because similarity among responses can result from both the repeated measures within clients and multiple responses referencing the same service element, the most appropriate modeling approach for these data is a two-level model with a two-way cross-classification of clients and service elements (i.e., targets and practices) at Level 2. LaPlace estimation was used to fit all cross-classified models, whereas maximum likelihood was used to fit all other multilevel models.
To test our hypotheses, we first examined the effect of a practice element in the Treatment Plan on the probability of its delivery during the treatment episode. Next, we examined how that recommendation effect changed over time. We then examined the influence of deviations from the Treatment Plan on the resulting practices. That is, if Treatment Summaries listed different targets compared to the Treatment Plan, were providers more apt to also use different practices? Similarly, we examined the effect of changing targets from month to month on the probability of changing practices from month to month as indicated in the Treatment Summaries. Finally, leveraging the PracticeWise Evidence-Based Service Database (PWEBS; Chorpita et al., 2005), which allows providers to map evidence-based practices to their client’s characteristics (e.g., age, treatment target), we were able to explore whether the “clinical fit” between targets and practices listed on the treatment plan impacted the probability of a provider delivering the practices in question for participants who had targets that existed within the PWEBS (n = 92).
RESULTS
Treatment Plans, which were coded using the SGRF, included an average of 8.41 (SD = 4.15) targets and 9.61 (SD = 4.84) practices. The top three most frequently endorsed targets were Academic Achievement (n = 58), Positive Family Functioning (n = 57), and Oppositional/Noncompliant Behavior (n = 52). The top three most frequently endorsed practices were Cognitive/Coping (n = 52), Family Therapy (n = 48), and Communication Skills (n = 43).
On average, each Treatment Summary included 14.71 (SD = 5.75) targets and 25.37 (SD = 9.57) practices. This suggests that the actual identification of targets and delivery of practices during care was more diffuse compared to what was identified during the treatment planning stage. The top three most frequently reported targets were Positive Family Functioning (n = 82), Oppositional/Noncompliant Behavior (n = 74), and Academic Achievement (n = 64). The top three most frequently endorsed practices were Supportive Listening/Client-Centered Therapy (n = 83), Communication Skills (n = 82), and Problem Solving (n = 80).
Cross-classified generalized linear mixed models were used to examine the relationship between a target or practice listed on the Treatment Plan and its probability of being implemented during the treatment episode. Results indicated that the odds of a target being addressed in a treatment episode were 3.10 times more likely when it was included in the treatment plan than when it was not, b = 1.1306, t(5672) = 10.01, p < .0001. Similarly, the endorsement of a practice in the Treatment Plan predicted greater odds for the practice to be delivered in treatment. The odds of a practice that was listed on the Treatment Plan occurring during the 1st year of a treatment episode were 2.09 times greater than those not listed in the plan, b = .7410, t(6602) = 6.50, p < .0001. Predicted probabilities for targets and practices listed and not listed in the Treatment Plan are displayed in Figure 1.
FIGURE 1.
Predicted probability of a practice or target occurring in treatment if it was or was not recommended in the treatment plan.
In analyzing the effect of including practices and targets in the Treatment Plan over time, a Plan × Time interaction emerged indicating that the effect of including a target in the plan on the probability that it is addressed in service changed over time, b = −.0476, t(39522) = −4.26, p < .0001. Follow-up analyses of simple effects showed that the probabilities of addressing a target that was listed in the Treatment Plan were reduced by 5% for every month treatment continued, b = −.021, t(5429) = −2.07, p = .0386. Targets not included in the Treatment Plan were 1.02 times (2%) more likely to be addressed in treatment for each month that passed, b = .023, t(33948) = 3.45, p = .0006.
Alternatively, the effect of the Treatment Plan on the implementation of practices over a treatment episode did not change over time, odds ratio (OR) = .985, b = −.015, t(45912) = −1.43, ns. Therefore, the Plan × Time interaction was removed from the model. A main effects model, however, showed that the odds that a practice was delivered increased by 61% if it was included in the Treatment Plan, b = .4778, t(45913) = 11.77, p < .0001. In addition, for every month that passed in the treatment episode, the odds that a practice was delivered increased by 1.04 times (4%), b = .0401, t(45913) = 8.91, p < .0001. Figure 2 shows the trends in the predicted probabilities of targets and practices over the duration of a treatment episode.
FIGURE 2.
Predicted probability of a practice or target occurring in treatment over time if it was or was not recommended in the treatment plan.
Tetrachoric correlations were used to evaluate the relationship between Treatment Plans and Treatment Summaries, as well as continuity of services on a month-to-month basis. Using Fisher’s R to Z transformation, these relationships were transformed to standardized units and modeled as two-level multilevel models. The average correlation between practices listed in the Treatment Plan and those provided in the Treatment Summary was .143 (SD = .157, Min. = −.20, Max. = 1.00), whereas the average correlation for targets was .290 (SD = .186, Min. = −.19, Max. = .88), indicating that Treatment Plans were variably adhered to. A service agency’s choice to focus on targets identified in the Treatment Plan was associated with that service agency’s use of practices in the Treatment Plan at a given month, b = .0135, t(617) = 4.32, p < .0001. For every .1 increase in the correlation between targets in the Treatment Plan and those delivered each month, the expected transformed (e.g. in Fisher’s Z units) relationship of practices in the Treatment Plan and those delivered each month increased by .0135 units. More specifically, service agencies that were more inclined to address targets that were in the Treatment Plan were also more inclined to deliver practices that were in the Treatment Plan. Predicted correlations between practices implemented and practices in the Treatment Plan as a function of the correlation between targets implemented and those in the Treatment Plan are displayed in Table 1.
TABLE 1.
Relationship Between Practice Deviation From the Plan and Target Deviation From the Plan
| Correlation Between Targets and Plan |
Predicted Fisher’s Z |
Predicted R Transformed |
|---|---|---|
| .00 | 0.11 | 0.11 |
| .10 | 0.12 | 0.12 |
| .20 | 0.13 | 0.13 |
| .30 | 0.15 | 0.15 |
| .40 | 0.16 | 0.16 |
| .50 | 0.17 | 0.17 |
| .60 | 0.19 | 0.19 |
| .70 | 0.20 | 0.20 |
| .80 | 0.22 | 0.21 |
| .90 | 0.23 | 0.22 |
Note. Fisher’s Z is the z-transformed correlation coefficient between practices in the plan and those provided in treatment. The third column uses the inverse of the Fisher transformation to return the predicted values back to the correlation metric.
Regardless of the Treatment Plan, we were also interested in determining whether some of the change in practices over time was simply due to a shift in targets being addressed. That is, as service providers focused on different concerns, they might employ different clinical procedures. Analyses were conducted to examine the relationship between targets changing over time and practices changing over time. Results indicated that in general, reported use of practices (M = .711, SD = .255, Min. = −.09, Max. = 1.00) and targets (M = .714, SD = .282, Min. = −.10, Max. = 1) were highly correlated with themselves in consecutive months. There was a direct relationship between the correlation of targets with the z-transformed correlation of practices, b = .097, t(366) = 11.66, p < .0001. Increases in the correlation of targets in adjacent months were related to increases in the expected correlation between practices over the same time frame. In other words, as targets endorsed on the Treatment Summary changed across months, practices tended to change as well. Predicted values of the correlation between practices at successive time points as a function of the correlation between targets can be found on Table 2.
TABLE 2.
Relationship Between Changes in Practices Over Time and Changes in Targets Over Time
| Correlation Between Targets |
Predicted Fisher Z |
Predicted R Transformed |
|---|---|---|
| .00 | .28 | .28 |
| .10 | .38 | .36 |
| .20 | .48 | .44 |
| .30 | .57 | .52 |
| .40 | .67 | .59 |
| .50 | .77 | .65 |
| .60 | .87 | .70 |
| .70 | .96 | .75 |
| .80 | 1.06 | .79 |
| .90 | 1.16 | .82 |
Note. Fisher’s Z is the z-transformed correlation coefficient between practices in the plan and those provided in treatment. The third column uses the inverse of the Fisher transformation to return the predicted values back to the correlation metric.
Fit of Target to Practice
Finally, we sought to examine if the “clinical fit” between a target and a practice as supported by research influenced the probability of a practice being implemented and how that fit effect differed depending on the practice’s inclusion in the Treatment Plan. Using the PWEBS database, we matched targets to practice elements. It was not possible to map all practices to all targets, as the Treatment Summary uses a broad array of targets, some of which do not yet have a corresponding evidence-based treatment (e.g., health management). Only targets that were indicated in the Treatment Plan were included in this analysis. Results indicated a significant interaction between the treatment target–practice match and the practice’s inclusion in the plan, OR = .68, b = −.3841, t (20879) = −2.97, p = .003. Simple effects indicated no effect of match on the probability of a practice occurring for practices not indicated in the plan. That is, practices that were not listed in the original treatment plan were no more or less likely to be implemented depending on whether it mapped onto the target, OR = 1.57, b = .45, t = 1.59, p = .11. Alternatively, practices listed in the Treatment Plan were more likely to be implemented in treatment if they fit the target, OR = 5.80, b = 1.76, t = 2.46, p = .02. Practices in the Treatment Plan that fit the indicated targets were 5.80 times more likely to be used in treatment than those that did not fit the target. The predicted probabilities for implementing practices that fit and did not fit the identified targets are displayed in Figure 3.
FIGURE 3.
Predicted probability of implementing practices with or without clinical fit for the target whether or not the practice was recommended in the treatment plan.
DISCUSSION
Building upon Young and colleagues’ (2007) work, which examined the stability of recommendations across treatment planning stages, the present study examined how recommendations from service planning documents (i.e., Treatment Plans) influenced the services provided to youths. The collective results indicated a general ability for service-planning documents to help organize and guide care. Unfortunately, they also suggested potential limitations to the current system.
Overall, the probability of targets being addressed or practices being implemented if they were listed in the Treatment Plan was surprisingly low. Yet, when one considers the low probability of any specific target and practice being utilized given the breadth of available service elements, as well as the dynamic nature of a clients’ status, this may not seem so surprising. When considering these results in terms of odds, we see that incorporating targets and practices in Treatment Plans is associated with a threefold increase in the likelihood of the listed targets being addressed and a twofold increase in the likelihood of listed practices being implemented. This indicates that Treatment Plans are indeed linked to service provisions among mental health providers. However, this also suggests that Treatment Plans may function more as a strategic recommendation rather than as a tactical work plan; providers may use a variety of resources, in addition to the Treatment Plan, to formulate their day-to-day treatment decisions. Furthermore, continuous monitoring of a youth’s clinical status may have also led to changes in the type of targets a provider addressed and subsequently the type of practices they implemented.
When examining the effect of the Treatment Plan over time, we found that, for targets, the effect of the Treatment Plan diminished over time, whereas the effect of the Treatment Plan on practices remained unchanged. Practices indicated in the Treatment Plan were more likely to be implemented than those not included in the Treatment Plan. Likewise, any practice, regardless of its inclusion in the Treatment Plan, was more likely to be implemented in treatment over time. As circumstances evolved and as information from the original assessments used to create the service plan becomes less relevant, delivery of practices may potentially become less focused. For instance, as treatment progresses, providers may find the inability for recommended practices to mitigate the client’s target problems and thus begin employing practices that were not in the Treatment Plan in the hopes of better addressing the client’s concerns. This is consistent with Orimoto and colleagues’ (2014) findings indicating that providers of youths with high comorbidity also administered more and more varied types of practices.
With findings indicating changes in treatment target over time, we examined whether the shift in selection of practices was associated with a shift in treatment targets given that some practices are theoretically better paired with certain targets (Chorpita & Daleiden, 2009). We found that the more closely a service agency followed the targets recommended from the Treatment Plan during treatment, the more closely a service agency followed the recommended practices. In addition, as service agencies changed their treatment targets across time, they also altered the practices that they implemented. This suggests that service agencies may not necessarily become less focused in implementation of practices over time, but instead they may be putting careful thought into the selection of targets and practices that had not been recommended at the start of care as treatment progresses.
Our last set of analyses explored how the clinical fit, as supported by scientific evidence, between targets and practices influenced the implementation of practices. Given CAMHD’s strong emphasis on the use of evidence-based practices (Daleiden, Chorpita, Donkervoet, Arensdorf, & Brogan, 2006), our findings indicating that recommended practices were more likely to have been implemented if they had demonstrated efficacy for the listed targets may not be surprising. With increasing movement toward incorporating available research into best practice standards (Anderson, 2006; Hogan, 2003), results are promising, as this indicates that providers are developing and implementing Treatment Plans that are supported by research.
Limitations
Although the development of Treatment Plans always preceded the delivery of services, a causal relationship could not be established given that the study did not employ a control group. Moreover, replication in larger studies on different populations in different SOCs using different service planning procedures can help determine whether these findings are generalizable beyond the confines of this study. Furthermore, continuity between the targets and practices listed in the Treatment Plans and what was reportedly addressed and delivered during care could also potentially be attributed to reporting tendencies, not actual behavioral consistency. In addition, a rather coarse criterion was set for identifying treatment components (i.e., present or absent, as opposed to how extensively or with what degree of quality). This methodological approach likely portrays the “bestcase” scenario for stability across treatment planning and practice. Further research using more refined and conservative characteristics of treatment is necessary before more definitive conclusions can be made.
Conclusion
Despite the aforementioned limitations, evidence from prior research scrutinizing the early stages of service planning (Young et al., 2007) coupled with our investigation examining the link between service planning and service delivery highlights a risk of loss of information across the complex processes that exist within a system of care. As such, examining the processes that exist in coordinating how providers move from the information gathering phase, to the plan development phase, and finally to the service delivery phase, offers several potential quality improvement targets. However, our knowledge regarding the capacity for increased level of organization to increase quality of care still remains extremely limited.
With that said, our ultimate goal is not to ensure that providers strictly follow service plans. Instead, our practical question concerns how we can help providers develop a focused approach to care that maximizes the use of available resources in a way that also provides them with the flexibility to formulate effective adaptations when provided with new information. Service plans serve as only one potential method for achieving this aim. There is a need to explore additional venues to service provision that balances comprehensive contextual information gathering with codified real-time data to optimize care.
Acknowledgments
FUNDING
This work was supported by Grant 12-103104-000-USP from the John D. and Catherine T. MacArthur Foundation. This research was also supported in part from grants awarded to Charles Mueller, Ph.D., author from the State of Hawai'i Department of Health, Child and Adolescent Mental Health Division.
Contributor Information
Katherine H. Tsai, Department of Psychology, University of California, Los Angeles
Andrew L. Moskowitz, Department of Psychology, University of California, Los Angeles
Roxanna E. Lynch, Department of Psychology, University of Hawai‘i at Hilo
Eric Daleiden, PracticeWise, LLC.
Charles Mueller, Department of Psychology, University of Hawai‘i at Mānoa.
Jennifer L. Krull, Department of Psychology, University of California, Los Angeles
Bruce F. Chorpita, Department of Psychology, University of California, Los Angeles
REFERENCES
- American Psychiatric Association. Diagnostic and statistical manual of mental disorders. 4th. Washington, DC: Author; 1994. [Google Scholar]
- Anderson NB. Evidence-based practice in psychology. American Psychologist. 2006;61(4):271–285. doi: 10.1037/0003-066X.61.4.271. [DOI] [PubMed] [Google Scholar]
- Bickman L, Noser K, Summerfelt WT. Long-term effects of a system of care on children and adolescents. The Journal of Behavioral Health Services & Research. 1999;26(2):185–202. doi: 10.1007/BF02287490. [DOI] [PubMed] [Google Scholar]
- Bickman L, Summerfelt WT, Noser K. Comparative outcomes of emotionally disturbed children and adolescents in a system of services and usual care. Psychiatric Services. 1997;48:1543–1548. doi: 10.1176/ps.48.12.1543. [DOI] [PubMed] [Google Scholar]
- Borntrager C, Chorpita BF, Higa-McMillan CK, Daleiden EL, Starace N. Usual care for trauma-exposed youth: Are clinician-reported therapy techniques evidence-based? Children and Youth Services Review. 2013;35(1):133–141. [Google Scholar]
- Burns BJ, Farmer EM, Angold A, Costello EJ, Behar L. A randomized trial of case management for youths with serious emotional disturbance. Journal of Clinical Child Psychology. 1996;25(4):476–486. [Google Scholar]
- Child and Adolescent Mental Health Division. Honolulu: Hawaii Department of Health Child and Adolescent Mental Health Division; 2003. Instructions and codebook for provider monthly treatment and progress summary. Retrieved from http://www.hawaii.gov/health/mental-health/camhd/resources/prov-agency/library/pdf/paf/paf-001.pdf. [Google Scholar]
- Chorpita BF, Daleiden EL. Mapping evidence-based treatments for children and adolescents: Application of the distillation and matching model to 615 treatments from 322 randomized trials. Journal of Consulting and Clinical Psychology. 2009;77(3):566–579. doi: 10.1037/a0014565. [DOI] [PubMed] [Google Scholar]
- Chorpita BF, Daleiden EL, Ebesutani C, Young J, Becker KD, Nakamura BJ, Trent L. Evidence-based treatments for children and adolescents: An updated review of indicators of efficacy and effectiveness. Clinical Psychology: Science and Practice. 2011;18(2):154–172. [Google Scholar]
- Chorpita BF, Daleiden EL, Weisz JR. Identifying and selecting the common elements of evidence based interventions: A distillation and matching model. Mental Health Services Research. 2005;7(1):5–20. doi: 10.1007/s11020-005-1962-6. [DOI] [PubMed] [Google Scholar]
- Daleiden EL, Chorpita BF, Donkervoet C, Arensdorf AM, Brogan M. Getting better at getting them better: Health outcomes and evidence-based practice within a system of care. Journal of the American Academy of Child and Adolescent Psychiatry. 2006;45(6):749–756. doi: 10.1097/01.chi.0000215154.07142.63. [DOI] [PubMed] [Google Scholar]
- Daleiden EL, Lee J, Tolman R. Annual evaluation report: Fiscal year 2004. 2004 Retrieved from http://hawaii.gov/health/mental-health/camhd/library/pdf/rpteval/ge/ge011.pdf.
- Hawai'i Department of Education and Department of Health. Honolulu, HI: Author; 2006. Interagency performance standards and practice guidelines. Available from http://health.hawaii.gov/camhd/files/2013/07/purplebook.pdf. [Google Scholar]
- Hogan MF. New Freedom Commission report: The president’s New Freedom Commission: Recommendations to transform mental health care in America. Psychiatric Services. 2003;54(11):1467–1474. doi: 10.1176/appi.ps.54.11.1467. [DOI] [PubMed] [Google Scholar]
- Nakamura BJ, Daleiden EL, Mueller CW. Validity of treatment target progress ratings as indicators of youth improvement. Journal of Child and Family Studies. 2007;16(5):729–741. [Google Scholar]
- Orimoto TE, Higa-McMillan CK, Mueller CW, Daleiden EL. Assessment of therapy practices in community treatment for children and adolescents. Psychiatric Services. 2014 doi: 10.1176/appi.ps.201100129. [DOI] [PubMed] [Google Scholar]
- Schwarzer R. Modeling health behavior change: How to predict and modify the adoption and maintenance of health behaviors. Applied Psychology. 2008;57(1):1–29. [Google Scholar]
- Stroul BA, Friedman RM. A system of care for children and youth with severe emotional disturbances. Washington, DC: Georgetown University Child Development Center, CASSP Technical Assistance Center; 1986. [Google Scholar]
- Verbiest ME, Presseau J, Chavannes NH, Scharloo M, Kaptein AA, Assendelft WJ, Crone MR. Use of action planning to increase provision of smoking cessation care by general practitioners: Role of plan specificity and enactment. Implementation Science. 2014;9(1):180. doi: 10.1186/s13012-014-0180-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Young J, Daleiden EL, Chorpita BF, Schiffman J, Mueller CW. Assessing stability between treatment planning documents in a system of care. Administration and Policy in Mental Health and Mental Health Services Research. 2007;34(6):530–539. doi: 10.1007/s10488-007-0137-8. [DOI] [PubMed] [Google Scholar]