Measurement-based care as a practice improvement tool: Clinical and organizational applications in youth mental health

Abstract

Measurement-based care (MBC) is systematic, ongoing assessment to monitor treatment progress and inform clinical decision-making. MBC is considered an evidence-based practice, with extensive research support in adult clinical populations and emerging evidence in youth populations. This paper describes both clinical (e.g., enhancing therapy alliance, informing case conceptualization) and organizational (e.g., informing quality improvement efforts) applications of MBC, and illustrates the clinical applications through two case examples from publicly-funded mental health agency settings. The paper concludes by detailing future research needed to enhance the clinical and organizational utility of MBC.

Measurement-based care (MBC), sometimes also referred to as routine outcome monitoring, has been operationalized as the systematic collection of patient-reported measures to monitor treatment progress and inform clinical decision-making (Scott & Lewis, 2015). Like any assessment tool, MBC measures should be reliable and valid. However, additional criteria that have been suggested for MBC measures include being brief (i.e., about 5–8 minutes per administration), sensitive to change, clinically relevant (i.e., producing results that are immediately actionable), and relevant to key stakeholders (i.e., addressing topics of interest to clients, caregivers, etc.)(Douglas Kelley & Bickman, 2009).

In addition to assessing treatment progress using symptom measures, the utility of MBC may be enhanced with non-symptom measures. For example, a recent study found that the addition of measures of skill confidence and use by adult patients contributed to early identification of potential non-responders, allowing the treatment team to better support patients in skill development (Hooke & Page, 2017). Measures consistent with a precision mental health approach (Bickman, Lyon, et al., 2016), whereby treatment is tailored to an individual patient’s needs, preferences, and prognostic possibilities, may encompass personal data, aims and risks, strengths, service preferences, intervention and progress data, process variables (e.g., therapeutic alliance), and contextual data. Measures may be standardized (i.e., rating scales with standard items formed into scales) or individualized (i.e., items measuring a specific assessment target for an individual); however, how the data are used may be more important than distinguishing the types of assessments; the same type of assessment tool might be used for standardized (e.g., comparing a client’s score on a rating scale to the clinical cutoff for the measure) or individualized (e.g., looking at a single client’s changes over time on that same rating scale) purposes (Lyon et al., 2017).

While more research is needed on MBC measure composition and frequency, standard practice is for measures to be administered before or during client encounters, with variations in frequency for more intensive treatment settings (e.g., day treatment, inpatient) and more severe mental illness. The resulting feedback is then presented in an interpreted format for the clinician (e.g., plotted over time, with feedback to help interpret the scores); this feedback is viewed by the clinician, alerts them to any unexpected developments or lack of progress, and is ideally addressed together with the client in session. As a form of client-centered communication (Levinson et al., 2010; Roter et al., 2014), MBC provides a valuable way for clients to share information with their clinicians, thus potentially increasing clinician responsiveness and patient engagement in treatment.

The purpose of this paper is to provide an overview of the research support for MBC, and then to discuss its utility for informing clinical practice and organizational decision-making. While MBC can be used across settings, here we will particularly focus on publicly-funded youth mental health agencies, such as community mental health agencies and federally qualified health centers. MBC might be especially useful in these settings, where clients and their families often enter treatment with complex clinical presentations and high levels of psychosocial stressors, which can present challenges for case conceptualization and treatment engagement. MBC is also well-suited for working with youth and families, as it supports collection of both child and parent symptom reports and facilitates identification of any discrepancies present.

Also, given differences between youth in community settings and those involved in the efficacy trials used to develop evidence-based treatments (EBTs)(e.g., Ehrenreich-May et al., 2011), MBC can also provide contextually-rich and real-time information to help clinicians make decisions about how to have “flexibility with fidelity” to EBTs (Kendall & Beidas, 2007). Indeed, excellent examples already exist demonstrating the complimentary nature of EBTs and MBC. For example, clinicians using the Unified Protocols for the Transdiagnostic Treatment of Emotional Disorders in Adolescents (Ehrenreich-May et al., 2017) establish youth- and caregiver-reported individualized top problems (Weisz et al., 2011) in the first session, use them to generate treatment goals, and monitor them throughout treatment to guide decision-making. Similarly, Parent-Child Interaction Therapy (PCIT; Eyberg & Funderburg, 2011) uses session-by-session caregiver reports on the Eyberg Child Behavior Inventory to identify targets for coaching, review progress with parents, and relate progress to skill practice outside of session.

Because they primarily serve low-income patients and Medicaid recipients, publicly-funded agencies face a number of challenges to treatment access, including high no-show and dropout rates (e.g., Ofonedu et al., 2017). While there are certainly institutional and societal barriers that must be addressed, below we discuss the growing evidence that MBC may also serve to improve treatment engagement and reduce dropout rates. In addition, publicly-funded agencies face substantial organizational challenges, including lack of funding, high turnover, burdensome reporting requirements, and an ever-changing policy landscape (e.g., Aarons et al., 2011; Dishop et al., 2019; Semansky et al., 2012). As discussed below, MBC can provide the foundation for data-driven organizational decision-making.

What do we know about how well MBC works?

Most of what is known about the effectiveness of MBC comes from studies of adults. Lewis and colleagues (2019) recently summarized the MBC literature, noting that nine systematic reviews and meta-analyses showed significant impacts on outcomes, particularly for patients identified as not-on-track (NOT; i.e., not responding to treatment). In contrast, a recent Cochrane review of MBC in adults (Kendrick et al., 2016) concluded that there was no differential effect between treatment as usual and treatment with MBC, while another Cochrane review of MBC in youth concluded that there were too few studies and insufficient evidence to draw any conclusions (Bergman et al., 2018). However, the lack of significant findings may have been at least partially attributed to the Cochrane methodology, which excluded studies where MBC was used to enhance care quality. This conflicts with the definition of MBC as proposed above, which explicitly calls for active use of MBC feedback to inform care decisions throughout treatment. The largest meta-analysis to date (De Jong, 2017), found a small but significant effect of MBC over and above usual care, although the bulk of this research was with adult clients. One randomized controlled trial has found positive benefits of MBC in a youth sample (Bickman et al., 2011) and a second found effects in a clinic with high fidelity to MBC, but no effects in a clinic where the efforts to get clinicians to use MBC routinely were not successful (Bickman, Douglas, et al., 2016)

While more research is needed on the mechanisms of action that contribute most to effectiveness, MBC is considered an evidence-based practice that is transtheoretical and transdiagnostic, with broad applicability across mental health settings, treatment types, and populations (American Psychological Association Presidential Task Force on Evidence-Based Practice, 2006; Lewis et al., 2019). MBC is supported by clinical practice guidelines (e.g., American Psychological Association Task Force on Evidence-Based Practice for Children and Adolescents, 2008) and is identified as a key evidence-based approach by a new consensus statement endorsed by more than two dozen professional organizations (Coalition for the Advancement and Application of Psychological Science, 2018). Since 2018, MBC is also required for accreditation by the Joint Commission, a United States quality public healthcare accreditation organization (The Joint Commission, 2018).

The Role of MBC in Clinical Care and Case Conceptualization

Although MBC seems somewhat simple on the surface, the heart of MBC is the clinician leveraging assessment and feedback to enhance their work with clients and their families. There are a variety of ways in which the collection of assessment data and utilization of feedback can impact and ideally enrich the course of treatment, which are detailed in Table 1.

Table 1.

Clinical and Organizational Applications of Measurement-Based Care

Clinical Applications
Enhances communication with clients	Facilitates session “check ins” If completed outside of session, can make check ins more efficient Facilitates multi-informant assessment
Enhances client engagement and alliance	Provides opportunities to identify and address alliance ruptures Supports agreement on tasks and goals Helps clients learn about their own symptoms and think about what to discuss in session
Identifies problems to target in treatment	Helps identify target problems at the beginning of treatment Helps detect new problems that might emerge so that clinicians may address them more quickly
Facilitates collection of perspectives from multiple informants	Facilitates multi-informant assessment Highlights areas of agreement and disagreement across informants Helps inform case conceptualization around issues such as communication problems or differing expectancies
Supports case conceptualization	Facilitates ongoing testing and revision of the case conceptualization hypotheses
Identifies when clients are not-on-track for successful treatment outcomes	Helps therapist overcome biases toward seeing treatment as going well Can compare clients to norms to flag when they have gone off course Signals to clinicians that a change in approach may be warranted
Helps determine why clients are not-on-track	Measures of therapy process (e.g., therapy alliance) or change processes (e.g., behavioral approach behavior) can be used to revise the treatment approach
Facilitates decisions about treatment termination	Clinical cutoffs can identify when clients are “well” Provides information about whether individual goals have been attained
Organizational Applications
Supports efficiency in internal and external business functions	Data facilitates ease of responding to requests for proposals and reporting to funders Key component of value-based payment models Mandated measures can be incorporated to meet compliance requirements MBC data can be used to assign new cases and inform workforce development opportunities
Basis for quality improvement in health care	Readily monitor adoption, installation, and ongoing maintenance of improvement initiatives Supports a culture of change and rapid improvement cycles Review of aggregated clinical data can lead to identification of opportunities for quality improvement
Contributes to organizational culture of transparency and accountability	Enhances intra-organizational transparency in communication between patients and clinicians, and throughout the organization with aggregated data Visibly sharing patient-level feedback on a screen during treatment team meetings may enhance peer support and team cohesion Supports an atmosphere of accountability through proactive monitoring and participation in safety and effectiveness efforts Expectations for MBC implementation may be incorporated in staff performance evaluations (similar to any documentation)

First, the collection of assessment data serves to enhance communication with clients in several ways. Administering measures each session allows for a quick check-in about an array of issues beyond what might come up the routine check-in that typically launches a therapy session. This facilitates monitoring of both the client’s main target problems and of new symptoms that could arise over the course of treatment. Clinicians working in settings where families can complete measures ahead of time or with the ability to send measures electronically to clients between sessions can collect this data before the session even starts without taking up valuable session time, and then use responses to guide that check in process. Child-serving clinicians can also use MBC to collect information efficiently from multiple individuals in the family (e.g., teen, mother, & father), even if all of those individuals are not directly involved in that week’s session. In a systematic review by Carlier and colleagues (2012), improved patient-provider communication was found to be associated with the use of progress feedback in physical and mental health care.

Importantly, this “check-in” should include active sharing of MBC feedback with clients and caregivers (Scott & Lewis, 2015; Lewis et al., 2019). While this may involve the clinician sharing a graph or other feedback view on their computer screen, clinicians should use their best judgment on how (but not whether) to share feedback. For example, a clinician may choose to reference feedback only through verbal means for clients where graphs and charts may be inappropriate. The clinician may also choose to only highlight the aspects of feedback that the think would be most helpful for that client at that particular point in treatment.

This enhanced communication is thought to also be one way in which MBC can enhance client engagement and alliance. Theoretical papers have argued that monitoring outcomes, particularly when combined with monitoring the therapy alliance, can enhance client engagement and alliance by providing opportunities for collaboration, addressing alliance ruptures as they happen, and modifying treatment targets if there is lack of agreement regarding the tasks and goals of treatment (e.g., Ackerman et al., 2001; Fortney et al., 2017). Having additional tools to support the alliance might be particularly use in youth mental health care, where the clinician is maintaining multiple alliances (e.g., child-clinician, caregiver-clinician).

Although empirical support for this theorized impact of MBC on alliance and engagement is relatively sparse, a few supportive studies in adult samples are beginning to emerge. One qualitative study of clients and clinicians’ views of MBC found that it supported collaboration, facilitated open conversation, and enhanced interpersonal processes (Moltu et al., 2018), and another found that clients reported that completing questionnaires helped them decide what to talk about in session and to learn about which aspects of therapy were most helpful to them (Delgadillo et al., 2017). Brattland and colleagues (2019) found that individuals receiving MBC had better treatment outcomes than individuals receiving treatment as usual (TAU) and that these differences were mediated by higher increases in therapy alliance in the MBC group. Miller and colleagues (2006) found that monitoring alliance in session was associated with less likelihood of treatment dropout, although other studies have not found a relationship between use of MBC and increased engagement (e.g., Janse et al., 2017).

Throughout treatment, items or scale scores from MBC measures can also help identify problems to target in treatment. Rating scales that include a broad range of symptoms can be particularly useful for identifying problems at the beginning of treatment and any new problems that emerge over the course of treatment. Indeed, research has shown that addressing problems raised on MBC measures may be a key mechanism of action for MBC, and that clinicians who receive feedback from MBC measures address client concerns more quickly than clinicians who do not receive this feedback (Douglas et al., 2015).

As noted above, MBC facilitates efficient collection of perspectives from multiple informants. Multi-informant assessment is considered critical in treatment for children and adolescents, given the well-documented fact that different individuals in a youth’s life often understand the youth’s problems in different ways, due to a host of factors including differing perspectives, differing standards for behavior, personal history of psychopathology, etc. (De Los Reyes et al., 2015). Having MBC data from multiple individuals (e.g., the client and a caregiver) can help clinicians identify when these differences are present, facilitating discussions about them. This also can enrich the clinician’s understanding of issues relevant to case conceptualization, such as communication problems in the family, differences in expectancies around typical child behavior, etc.

MBC is also consistent with many models of case conceptualization, which view the initial case conceptualization as a theory that should be developed using assessment data and then tested and revised as treatment proceeds. For example, the Science-Informed Case Conceptualization Model (McLeod et al., 2013) recommends that the case conceptualization process should generate a list of hypothesized mechanisms of action that should improve the client’s symptoms if addressed in treatment. For example, a therapist might hypothesize that a client’s behavioral avoidance is maintaining their anxiety. If so, then addressing that avoidance through exposure therapy should improve the client’s anxiety. An important step in the model is designing a MBC plan to monitor whether the hypotheses in the initial case conceptualization are correct. In our example, if a therapist monitors the client’s avoidance of feared objects, their level of compliance with exposures, and their anxiety each week, this assessment data should show that as the client completes exposures, their level of behavioral avoidance should decrease, and their anxiety should subsequently improve. If not, then the clinician should consider revising the case conceptualization and treatment plan. In this way, MBC can serve as a feedback loop to test and revise the case conceptualization throughout treatment.

In turn, the case conceptualization can facilitate the interpretation of MBC data. For example, in the case of Diana, below, there were several points in treatment where her self-reported symptoms increased. In some situations, an increase in symptoms might indicate a need for the therapist to make a change to the treatment plan. In this case, however, these symptom increases were consistent with the case conceptualization that symptom reports might increase as Diana became more comfortable disclosing to her therapist and also later as Diana began to discuss past traumatic events. The relationship between MBC and case conceptualization can therefore be thought as bi-directional, where MBC data can be used to revise the case conceptualization, but the case conceptualization can also support MBC.

Arguably one of the most important benefits of MBC is identifying when clients are not on track for successful treatment outcomes. Numerous authors have noted that therapists, like all other human beings, are subject to a number of cognitive biases that limit their ability to identify clients who are not responding well to treatment and to adjust their therapeutic techniques accordingly (e.g., Lambert, 2015; Riemer & Bickman, 2011). For example, in a seminal study on clinician judgment, Hannan et al. (2005) found that therapists grossly under-estimated the likelihood that their clients would deteriorate over the course of treatment, and Costantino and colleagues (2019) found that clinicians are no better than chance at identifying clients at risk for negative treatment outcomes.

MBC can combat these biases directly by alerting clinicians when clients are not doing well in treatment; many systems have data regarding typical trajectories of clients who ultimately have positive treatment outcomes, and have algorithms to flag clients who are not following these trajectories (e.g. Cannon et al., 2010; Delgadillo et al., 2017). Theoretical models have argued that one essential mechanism of action of MBC is that clinicians attend to this feedback, accept it as accurate, and adjust their behavior accordinly (Claiborn & Goodyear, 2005; Riemer et al., 2005; Riemer & Bickman, 2011). In support of this notion, a recent trial demonstrated that a MBC feedback condition that included flags that clients were NOT and guidance about how to discuss this feedback in supervision was associated with better treatment outcomes than a MBC conditions without these risk signals and supports (Delgadillo et al., 2018). Also consistent with these theories is that data often indicate that the positive effects of MBC on treatment outcomes may be most, or perhaps even solely, present for clients who are flagged as being NOT (Kendrick et al., 2016).

When clients are not on track for optimal outcomes, MBC can also help clinicians determine why clients are not on track. As noted above, MBC can be used to monitor both outcomes and proposed mechanisms of action; if a client is not improving, these data can be used to understand whether treatment is, indeed, targeting the correct mechanisms. As illustrated below in our case examples, it can also be very helpful for MBC to monitor therapy processes and client strengths in addition to therapy outcomes. For example, the Peabody Treatment Progress Battery (PTPB; Bickman et al., 2010) includes brief measures of a variety of treatment-relevant variables in addition to symptoms, including therapy alliance, treatment expectations, treatment motivation, and caregiver strain. These additional measures can help clinicians identify next steps to take and strengths that can be leveraged to turn treatment around when progress is not being made (e.g., address alliance ruptures, revisit treatment goals, refer caregivers for ancillary services). Although the additive benefit of these process measures has not yet been tested in youth populations, studies with have adults found that even a single administration of a more comprehensive assessment battery after a client was flagged as NOT significantly improved treatment outcomes over simply continuing to monitor symptoms (Dyer et al., 2016; Harmon et al., 2005; Whipple et al., 2003).

Finally, MBC data can be used to support decisions about when to terminate treatment. Outcome data can be reviewed collaboratively with clients and caregivers to decide whether adequate progress has been made. Standardized MBC measures can be used to determine whether the client has made clinically-significant change and whether they are in the “well” range on the measure (Jacobson & Truax, 1991), and individualized MBC measures can be used to determine whether the client’s specific treatment goals have been attained. Interestingly, some studies have found that MBC may make treatment more efficient, with greater treatment gains attained in fewer sessions (Delgadillo et al., 2017; e.g., Janse et al., 2017; Reese et al., 2009), so MBC helps make clinicians and clients aware of improvements that may be unexpected.

The Role of MBC in Organizational Functioning and Decision-Making

The foundation of any kind of quality improvement process is in the systematic collection of data that can be aggregated as meaningful metrics for routine monitoring and evaluation. MBC serves a core function for practice improvement by providing a “golden thread” of patient-centered and clinically-relevant data from the bottom up throughout the agency (Douglas et al., 2016). Just as clinicians can use MBC feedback at the individual level to inform clinical decision-making, organizations can use MBC feedback aggregated at different levels to inform organizational decision-making.

MBC supports efficiency in internal and external business functions.

When supported by technology and linked to other relevant data systems, such as electronic health records, MBC provides ready access to the kinds of data that facilitate ease of responding to requests for proposals and reporting out to funders. For example, MBC is a key component of value-based payment models where reimbursement is based on patient-report quality rather than quantity of care (Centers for Medicare & Medicaid Services, 2016; Fortney et al., 2017; Porter et al., 2016). Of note for publicly-funded treatment, ongoing monitoring is a staple of the behavioral health home model (Akinci & Patel, 2014; Bao et al., 2013). Health homes and patient-centered medical homes are increasingly recommended in community mental health settings as a foundation for value-based care (Unützer et al., 2012) and improved patient engagement (Schuster et al., 2018). Measures that are mandated for reimbursement or regulatory purposes can be incorporated into MBC technology and ideally used for both informing individual treatment progress and meeting compliance requirements. Existing mechanisms for mandating measures could be leveraged to extend the “golden thread” beyond the walls of the organization to the state and federal levels. This would serve to promote a data-driven population health approach to better learn about such performance-based care models in service of improving reimbursement and patient quality needs. Internal business functions facilitated by MBC may include accounting for initial severity when assigning new cases to clinicians, or aggregating data to inform workforce development opportunities.

Federal agencies encourage organizations to use MBC as a basis for quality improvement in health care (Agency for Healthcare Research and Quality, 2018; National Quality Forum, 2018). Depending on how it is structured, MBC can readily provide the data needed to monitor adoption, installation, and ongoing maintenance of improvement initiatives in real-time using existing resources, rather than having to overlay an external data collection system that adds cost and burden without the benefit of clinical relevance. Notably, real-world quality improvement does not need to be complicated, and the organization, analysis, and interpretation of data should emphasize a culture of change and rapid improvement cycles (King & Hodges, 2013).

MBC contributes to organizational transparency and accountability.

The organization development literature suggests that transparency and accountability are two key components of a strong organizational culture that promotes trust and contributes to workforce engagement and improved performance. Transparency in health care has been defined as the free flow of information in four domains, two of which are intra-organizational (between patients and clinicians, among clinicians) and two of which are inter-organizational (among organizations, between organizations and the public) (National Patient Safety Foundation’s Lucian Leape Institute, 2015). MBC, when implemented as intended, directly contributes to transparency in communication between patients and clinicians, and can facilitate communication within the larger organization through use of data to support business functions and quality improvement efforts as described above. Importantly, accountability must be balanced with continuous improvement and an atmosphere of fairness, where the workforce is empowered to proactively monitor and participate in safety and effectiveness efforts (Boysen, 2013; Kaplan, 2018). When incorporating MBC data into staff performance evaluations, organizations should consider focusing on expectations for implementation (i.e., measure completion, feedback viewing, addressing feedback with patients) consistent with other documentation requirements but proceed cautiously with expectations for clinical outcomes, especially at the individual level. Transparency is not a unidimensional construct, but rather has multiple dimensions beyond mere disclosure, such as clarity and accuracy of information (Fox, 2007; Schnackenberg & Tomlinson, 2016), that need to be understood further as they relate to MBC in real world practice settings. Anecdotally, in S.D.’s experience implementing MBC in community mental health settings, a team of clinicians in one site that began sharing patient-level feedback reports on a screen visible to all during case review meetings reported increased team cohesion and found peer comments and suggestions to be very helpful.

Case Illustrations

Below, we present two case examples, both of which are “hybrid cases” combining two different cases in order to protect client confidentiality. The first is based on two community mental health clinic clients participating in an effectiveness trial that included a MBC arm (Jensen-Doss et al., 2018). The second is based on two clients receiving behavioral health services in a federal health qualified center, where MBC has been standard operating practice for over two years. The two case examples are included here to illustrate some of the clinical applications summarized in Table 1. In the first case example, MBC information contributed to identifying problems to target in treatment, identifying when and why a client became not-on-track for successful treatment outcomes, and facilitating decisions about treatment termination. In the second case example, MBC information contributed to enhancing communication about multiple perspectives from different informants and informing client engagement and alliance. These cases also illustrate how MBC can be used in both an evidence-based treatment approach and in a more common factors approach (Douglas Kelley et al., 2010).

Case example 1: Combining MBC with Evidence-Based Treatment

Diana was a 13-year-old Hispanic female referred to treatment after her teachers contacted her parents about anger outbursts and irritability in the classroom. Treatment began in January of Diana’s 8th grade school year. Diana lived with her mother and two siblings – an older half-brother and a younger sister. Her father lost custody of her and her sister due to charges of neglect. Diana had previously been diagnosed with depression and oppositional defiant disorder by her pediatrician and exhibited high levels of anger, irritability, and sadness.

Presenting Problem

Diana and her mother came to treatment seeking help for Diana’s anger and temper outbursts at school, which Diana attributed to other children bothering her and teachers singling her out in the classroom. At both school and in the home, Diana exhibited irritability and defied requests from teachers, administrators, and her mother. She had a small friend group, was involved in extracurriculars, and received above average grades, but her attitude was often perceived negatively. With her friends and at home, Diana also exhibited depressed mood, which sometimes resulted in her isolating herself from others. Diana’s low moods had been occurring regularly for the past two years. Diana’s mother hoped that treatment would help Diana manage her temper and reduce her irritability and defiance, while Diana hoped that treatment would help boost her mood and get her accommodations at school. Diana, her mother, and her therapist agreed that treatment focused on anger management would be a good fit for these goals.

Progression of Treatment

The Youth Outcomes Questionnaire (YOQ) 30-TA (Burlingame et al., 2005), which includes 30 symptom items and 4 (caregiver report) or 5 (youth report) items to assess the therapy alliance, was administered every session. The YOQ can be administered electronically, with feedback reports automatically generated that include trajectories of change over time, feedback messages that flag when clients are NOT, and critical items that have been found to predict clinical risk variables such as suicidality or posttraumatic stress. Figure 1 presents Diana’s YOQ symptom data. Diana completed the measures every session, but her mother did not attend every session.

Figure 1.

Measurement-based care data with significant clinical events for the clinical case of Diana. Youth and caregiver reports from the Youth Outcomes Questionnaire 30-TA (Burlingame et al., 2005) were plotted for each treatment session.

Treatment began with psychoeducation and skills training. During this phase, Diana’s reports of therapy alliance on the YOQ were somewhat low, and inspection of her responses to items indicated that she was reporting minimal symptoms in the areas of irritability and conduct problems, particularly compared to her mother’s ratings of those same items. Her total score on the YOQ fell below the clinical cutoff at the beginning of treatment, suggesting that she either did not believe symptoms were present or did not feel comfortable disclosing them on the measure.

During the first seven sessions, Diana and her therapist used a CBT workbook and focused on identifying triggers for her anger, identifying and reframing distorted thoughts, and learning coping skills. As Diana got more comfortable with therapy and her alliance scores went up, she also began to disclose more of her feelings in session to her therapist, and her symptom ratings increased on the YOQ as well, suggesting that she was becoming more willing to report symptoms as the alliance improved. However, she continued to report fewer symptoms than her mother and neither of them were reporting any meaningful improvements in symptoms.

During the 8th session, the YOQ report flagged that Diana had endorsed a critical item that is associated with increased risk for post-traumatic stress disorder (PTSD). Although Diana and her mother had not previously endorsed any trauma history, the therapist decided to follow up by having Diana complete the Child and Adolescent Trauma Screen (Sachser et al., 2017); the clinician was particularly open to this further assessment since Diana’s lack of improvement in treatment suggested that the current treatment plan may not have been appropriate. The results of this assessment indicated that Diana met criteria for PTSD, and she revealed a previously undisclosed traumatic event of witnessing her father commit a violent act against her mother’s ex-husband. After discussing this new information with Diana and her mother, the therapist changed her treatment plan from anger management to trauma-focused cognitive behavioral therapy (TF-CBT) (Cohen et al., 2012).

After the initiation of TF-CBT, there was a relatively steady decrease in both youth- and caregiver-reported YOQ scores, and both Diana and her mother reported high therapy alliance. The skills training phase of TF-CBT lasted through session 14. At this point, both Diana and her mother reported symptoms that fell below the clinical cutoff, but Diana still did not want to discuss the trauma, providing support for continuing with the trauma narrative (i.e., exposure) portion of TF-CBT rather than terminating treatment. The therapist discussed the rationale for doing the trauma narrative with Diana and her mother, and also warned them that Diana might experience a short-term exacerbation in her symptoms as she began discussing her trauma in detail with her therapist and to prepare to also discuss it with her mother.

Diana wrote and shared her trauma narrative with her therapist over the course of the next five sessions. As anticipated, Diana’s symptoms increased in the first week of the trauma narrative process, and there was also a brief dip in her report of the therapy alliance, although this did not last past that first week. By session 20, the family was prepared to do the joint sessions where Diana shared the narrative with her mother. Although this was an emotional experience for both of them, they both reported being glad that they could finally discuss these events together, and Diana said she felt that her mother was very supportive of her in sessions. By the next session, both Diana and her mother reported decreases in her symptoms, which continued to decrease over the rest of treatment. Diana’s therapist spent the last few sessions working on reviewing coping skills, reinforcing strategies for healthy communication between Diana and her mother, and developing an aftercare and safety plan for post-treatment termination. Diana’s therapist terminated treatment with strong alliance ratings on the YOQ from both the parent and the child, and symptom reports much lower than the clinical cutoff for both reporters.

Reflections on the Utility of the YOQ in Treatment

Diana’s therapist reported that the YOQ helped her identify when things were off track in treatment and gave her a very clear picture of Diana’s progress and alliance throughout their time together. The critical item that was flagged on the YOQ led to a change in the treatment plan that she made sooner than she might have if she had not had access to that feedback. She also found it very helpful that the YOQ detected the anticipated symptom exacerbation and subsequent improvement following the trauma narrative, and she found it helpful to share this feedback with the family during this phase of treatment. On a practical note, the clinician reflected that she sometimes gave the YOQ at the end of the session, rather than at the beginning, and found this to provide less valid information because the reports would pick up on the client’s feelings about the session, rather than the symptoms she experienced during the week, so she resolved to do the YOQ at the beginning of sessions whenever possible. Ultimately, she felt that combining a tool like the YOQ with a flexible treatment like TF-CBT was very helpful in guiding decisions about when to proceed to the trauma narrative and when to end treatment.

Case example 2: Using MBC to Direct Attention to Common Factors of Treatment

Manuel was a 12-year-old Hispanic male who was referred to treatment by his school due to excessive absences and refusal to attend school. Manuel was born in the United States as the son of undocumented immigrants. Until recently, he lived as an only child with both parents, several aunts and uncles, and four small cousins. Six months ago, Manuel’s mother left the family and moved back to Mexico where she was living with a boyfriend and pregnant with his child. At the time of the initial assessment, Manuel was exhibiting high levels of worry and sadness about his mother and guilt about not being with her. Manuel was refusing to leave the home and played video games for up to ten hours a day. He had no previous history of behavioral health concerns.

Initial Session

As the session began, the therapist asked some initial questions about what brought Manuel and his father to treatment. Manuel’s father stated that his son had always been well-behaved, doing well at school and spending time with friends, and he described Manuel’s current difficulties as temporary and something that he would “get over soon.” He described an authoritarian style of parenting, with strict discipline and expectations for good behavior. He stated that he felt he had been lenient in allowing Manuel to spend so much time playing video games, and that he was going to start setting time limits. He also expressed anger with Manuel’s mother for leaving, and stated that he limited communication with her because he felt she was not a good influence on Manuel. Manuel’s father spent long days at work outside the home, seeing Manuel mostly in the evenings around mealtimes.

At first, Manuel was mostly quiet in the session, talking only when spoken to by the therapist or his father. Manuel described himself as sad that his mother had left and acknowledged that he played video games for several hours a day to keep his mind off of his concerns. At this point in the session, the therapist introduced the concept of MBC, stating that she asked parents and youths to complete measures prior to the session because it helped her learn more about they thought was important. She went on to say that she thought it could be useful to look at the results together, and asked for their permission to show the MBC feedback on a computer screen so they could all see it. Manuel and his father both agreed and turned to view the screen.

Manuel and his father had completed several measures on a tablet in the waiting room, including the Symptoms and Functioning Severity Scale (SFSS)(Athay et al., 2012), which is part of the PTPB (Bickman et al., 2010). The SFSS is a brief measure of youth internalizing and externalizing symptoms created specifically for frequent use in MBC that can be completed by multiple respondents including youths, caregivers, and teachers. When the therapist showed the initial results for the SFSS, they looked similar to the graph presented in Figure 2. Manuel had self-reported a high level of severity on the internalizing subscale, with a moderate level of severity on the externalizing subscale. His father had reported low levels of severity for Manuel’s symptoms on both subscales. Manuel’s father commented that he had no idea how much Manuel seemed to be suffering, and Manuel affirmed how sad and worried he was and began to cry. The therapist guided them through a conversation about Manuel’s symptoms by going through some of the items that were endorsed in the severe range. This enabled Manuel to begin to talk about his feelings in the session, including guilt about not being able to talk to his mother as much as he would like. The therapist also directed attention to their motivation for treatment, measured by youth and caregiver-report on the eight-item Motivation for Youth’s Treatment scale (MYTS)(Breda & Riemer, 2012), also part of the PTPB. As can be seen in Figure 2, the youth expressed a higher overall motivation for treatment than his father. By looking at the items, it became clear that the father’s lower score was consistent with a lower recognition that Manuel’s feelings were causing problems, and disagreement that mental health treatment was needed. The therapist used the discussion of the MYTS to talk with Manuel and his father about how mental health treatment could be helpful, and they agreed to continue in therapy.

Figure 2.

Measurement-based care data from the first session for the clinical case of Manuel. Youth and caregiver reports from the Symptoms and Functioning Scale (SFSS) (Athay et al., 2012) and the Motivation for Youth’s Treatment Scale (MYTS) (Breda & Riemer, 2012) reflect a lack of shared understanding of symptoms and differing levels of motivation for treatment.

Fourth Session

As she typically did right before a session, the therapist opened the most recent feedback report on her computer for a brief glance prior to going to the waiting room to get her clients. She saw an alert on the screen related to therapeutic alliance, which was self-reported by Manuel using the Therapy Alliance Questionnaire (TAQS)(Bickman et al., 2012), a five-item measure that is part of the PTPB (see Figure 3). Manuel had reported overall ratings of therapeutic alliance in the moderate range in the previous two treatment sessions, indicating a fairly high level of agreement on treatment tasks and goals as well as feeling a bond with his therapist. However, his responses this time were as low as possible. Notably, Manuel’s father had set time limits on video game playing in the previous session, supported by the therapist. When the therapist met with Manuel, she stated that she understood he was upset with her and invited him to talk to her about it. He was visibly angry and the therapist encouraged Manuel to express his feelings, which led to a conversation about how his video game playing was his only active coping strategy at the time. After introducing the idea of building more adaptive coping strategies, Manuel and the therapist agreed that Manuel would be asked to indicate when he was ready to reduce time on video games as he gained practice with and became more comfortable with other coping strategies. This discussion facilitated an agreement between Manuel and his therapist to focus future sessions on developing skills to manage strong emotions. In addition, the therapist intentionally used the conversation about the feedback as a starting point to continue building an environment of psychological safety in future sessions, where the youth could experience the interaction as a safe space for strong emotions to be communicated and accepted.

Figure 3.

Measurement-based care data tracking the therapy appliance with significant clinical events for the clinical case of Manuel. Youth reports from the Therapeutic Alliance Quality Scale (TAQS) (Bickman et al., 2012) were used to identify and repair an alliance rupture.

Reflection on the Utility of the PTPB in Treatment

Manuel’s therapist reported that the PTPB measures were helpful both in terms of facilitating communication and understanding between the youth and his father, as well as helping her to identify treatment process issues (e.g., motivation for treatment, therapeutic alliance) that were important to address as treatment progressed. In particular, she felt that the feedback on the drastic change in the youth’s rating of therapeutic alliance helped her to realize just how critical the role of the video games was to Manuel. The therapist found that by intentionally addressing the therapeutic rupture, she was better able to understand Manuel’s perspective and move away from thinking about the games as an adult might, which made it easy to minimize their importance.

Future Directions in MBC

We hope we have made clear the many benefits of using MBC. However, despite the promising evidence and theoretical benefits of this practice, there are several unanswered questions that need to be addressed in future research.

How well does MBC actually work and for whom does it work best?

As noted above, while many systematic reviews have concluded that MBC leads to improved treatment outcomes, two Cochrane reviews have concluded that there is insufficient high-quality research to support such a conclusion, particularly in youth samples (Bergman et al., 2018; Kendrick et al., 2016). For example, for the majority of MBC studies, the primary study outcome measure is also the measure used in MBC, which raises the possibility that effects of effects of MBC could be partly explained by social desirability bias (e.g., clients reporting improvements to make their therapists happy) or clients attending to a specific outcome measure that might not capture other changes that may or may not be taking place. While some trials have found effects of MBC on independent outcome measures (e.g., Delgadillo et al., 2017), there is a need for studies to include a wider range of independent measures, such as quality of life and social functioning (Bergman et al., 2018; Kendrick et al., 2016). There is also a need for studies with larger samples, as the effects of MBC are likely small and may differ across subgroups of participants. Although the literature has established that feedback is more effective in NOT cases, there may be other moderators of effectiveness, such as level of impairment, client developmental level, etc. Although MBC has been shown to be effective across a wide range of diagnostic groups and clinical settings, there is a need for studies that formally examine potential moderators of MBC effects. Perhaps most critically, there is also a need for more MBC trials focused on youth populations.

Another issue concerns researchers’ allegiances. Some authors study feedback systems that they have developed themselves and for which they may have a commercial or at least a vested interest, which may bias results. A related issue is that often only the developers of a certain feedback system conduct studies about its effects, and there is a lack of replicated studies from independent research groups. Moreover, as discussed next, feedback systems are frequently offered as packages with many different features, and we do not know much about which features are essential.

What are the best approaches to MBC?

Models of MBC vary in a number of ways, including the frequency and content of assessment, the format and target of feedback, etc. Efforts to draw conclusions about these components have been hampered by differences between studies, and there is a need for studies to directly compare different forms of MBC. The majority of MBC studies have compared MBC to treatment as usual conditions wherein MBC data are gathered, but no feedback is provided to clinicians (De Jong, 2017). Only a few studies have compared the effectiveness of different features of feedback systems (Harmon et al., 2007; Slade, Lambert, Harmon, Smart, & Bailey, 2008; Trudeau, 2000), which means there is limited information to guide recommendations about how clinicians should structure MBC.

One key question is what should be measured in MBC. Current approaches can be grouped into approaches that only track symptoms (e.g., De Jong et al., 2014; Lambert et al., 2001) and approaches that also track therapy processes such as therapy alliance (e.g., Bickman et al., 2010); here we will refer to the former as “unidimensional” MBC and the latter as “multidimensional” MBC. Because multidimensional MBC focuses on both treatment progress and processes, it likely leads to feedback that is more actionable than unidimensional approaches. However, multidimensional MBC may have higher burden for clients and clinicians than unidimensional MBC, which could have implications for fidelity and sustainability, particularly when factoring in multiple reporters in youth MBC.

Systematic reviews and meta-analyses of MBC have not discriminated between these approaches, but studies have supported both. Although no studies have directly compared the routine use of unidimensional and multidimensional MBC, three studies have examined the effects of adding a single administration of a multidimensional MBC battery after a unidimensional system flags an adult client as NOT, finding improved outcomes for the group that received the multidimensional feedback (Harmon et al., 2005; Slade et al., 2008; Whipple et al., 2003). Other key questions related to MBC in need of additional research include the optimal frequency of administering measures, when measures should ideally be administered (e.g., before or after session), how feedback should be formatted in order to best support clinical decision-making, how best to share feedback with clients, etc.

What are the mechanisms of action for MBC?

Another gap in the literature that impedes the field’s ability to identify the essential components of MBC is a lack of data regarding the specific mechanisms of action fir MBC. Above, we have articulated several ideas about how MBC might impact outcomes, including client-level mechanisms (e.g., enhanced therapy alliance), clinician-level mechanisms (e.g., decisions to act upon feedback), and organizational-level mechanisms (e.g., enhanced quality improvement initiatives), but also noted that have been very few empirical tests of most of those ideas. However, some theoretical models of MBC have been proposed that could set the stage for future research; here we will highlight a few of those theories.

At the client level, therapeutic assessment theory (Finn & Tonsager, 1997) posits that assessment and feedback enhances outcome by improving the client’s understanding of their problems, their treatment engagement, and the therapy alliance. While originally written with traditional diagnostic assessment in mind, this theory holds great potential for helping guide research in MBC. At the clinician level, the contextual feedback intervention theory (CFIT) proposes that clinicians recognize a discrepancy between treatment goals and client status, and experience cognitive dissonance (Riemer & Bickman, 2011). This cognitive dissonance then prompts the clinician to take action, such as addressing the feedback in an upcoming session, modifying a treatment plan, discussing it with a supervisor, or doing nothing at all.

At the clinician and organizational level, Brown and colleagues (2019) recently synthesized 30 pre-existing feedback theories into the Clinical Performance Feedback Intervention Theory (CP-FIT), a model designed to understand the effects of physician feedback in healthcare settings, including specific hypotheses that could be tested in future research. For example, the model proposes that feedback will be more effective if it includes benchmarks against which to compare a physician’s performance and if the system for feedback has support from organizational leadership. This model has several hypotheses in common with the CFIT model, although the CFIT model was not included in the development of the CP-FIT model, nor were any other models specific to feedback in mental health. Therefore, while the CP-FIT model is very promising to guide future research in MBC, some modifications to the model may be needed for it to fit the mental health context.

What are the best approaches to MBC implementation support?

MBC implementation is complex, involving changes at the client (measure completion), clinician (feedback viewing and utilization), and organizational (e.g., training, implementing new technology) levels (Lewis et al., 2019). This complexity likely contributes to MBC being implemented with high quality in fewer than 20% of clinical settings (Fortney et al., 2017). This is concerning given evidence that when implementation within a setting trying to use MBC is poor, the effects of MBC are diluted (Bickman, Douglas, et al., 2016). Lewis and colleagues (2019) published a recent review that applied an implementation science lens to the MBC literature, including a review of studies on barriers to implementation. They identified a number of barriers to using MBC at the client (e.g., finding time for clients to complete measures), practitioner (e.g., concerns about practical barriers to MBC, concerns about how the data will be used by agency administrators), organizational (e.g., lack of resources for training, staff turnover), and system (e.g., lack of funding for MBC) levels. They discussed a number of implementation strategies that could be used to address these barriers. However, they also noted that, while there is emerging evidence of some successful MBC implementation strategies, such as use technology to support MBC, identifying an agency “champion” to help shepherd MBC implementation efforts, and the use of ongoing consultation, there is a need for additional studies focused on identifying evidence-based MBC implementation approaches. They concluded their paper by presenting a research agenda to guide such efforts, including developing standards for MBC fidelity monitoring, developing addition brief and psychometrically-strong MBC measures, and identifying evidence-based implementation strategies to support MBC.

Conclusions

MBC in mental health settings provides value at multiple levels, from the individual patient to the organization. Research efforts to improve our understanding of MBC mechanisms of action and approaches to measurement will ideally improve both effectiveness and implementation as a tool for practice improvement. Regardless of the need for optimization, MBC appears well-established as an evidence-informed practice that will increasingly be required for funding and regulatory purposes. The good news for providers is that these requirements, although a practice shift for many, should result in more use of this effective, patient-centered practice.