Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2013 Dec 1.
Published in final edited form as: Couple Family Psychol. 2012 Dec;1(4):274–284. doi: 10.1037/a0031022

The Technology of Measurement Feedback Systemsi

Leonard Bickman 1, Susan Douglas Kelley 1, Michele Athay 1
PMCID: PMC3779359  NIHMSID: NIHMS510146  PMID: 24066274

Abstract

Usual care in the community is far from optimal. Sufficient evidence exists that dropout rates are significant, treatment is effective for only a small proportion of clients, and that the translation of evidence-based treatments to the real world is problematic. Technology has been shown to be helpful in health care in improving the effectiveness of treatment. A relatively new technology being used in mental health is measurement feedback systems (MFSs). MFSs are particularly applicable to couple and family psychology (CFP) because of its ability to provide information on the multiple perspectives involved in treatment. The Contextualized Feedback Systemstm (CFS®), developed at Vanderbilt University is used as an example of what can be accomplished with an MFS. The advantages and limitations of this technology are described as well as the anticipated reimbursement requirements that mental health services will need.

Why Clinicians Need Technology

Couples and family therapists face unique challenges compared to many other mental health professionals. Whereas many mental health professionals are primarily focused on one individual (the ‘patient’), couples and family therapists are focused on the entire family system. This means that the therapist must integrate and interpret multiple sources of information and different points of view from individuals with different idiographic experiences (e.g., symptom patterns, overt behaviors, and covert personality dynamics) within the context of their specific social, cultural, and environmental factors (Ridley, Tracy, Pruitt-Stephens, Wimsatt, & Beard, 2008). Only when this complete clinical picture is appreciated can the therapist know how best to treat the family. Yet this picture is not static, it is ever moving. Therefore, this picture must be continually reassessed so that the therapist can make informed decisions about therapeutic goals and appropriate treatments on an ongoing basis. Thus, the ability of the therapist to obtain this information and then accurately integrate and interpret it, directly influences the resulting clinical decisions and treatment outcomes. Indeed, as Ridley and Shaw-Ridley (2009) stated, “Clinical judgment is foundational because positive therapeutic outcomes hinge on establishing reasonable goals and selecting appropriate treatments, and appropriate treatment selection hinges on sound judgment and accurate decision making” (p. 401).

Despite the ongoing and complex task of generating an accurate clinical picture of a dynamic family system, the efficacy of couple and family psychology (CFP) is well established (Sexton, Datachi, Evans, LaFollette, & Wright, in press). However, CFP is not immune to the challenges faced by all areas of psychotherapy. These challenges include high rates of premature termination of treatment (i.e., dropouts) and the gap between research and practice. These have direct influences on the quality and outcomes of clinical care. For example, not only is premature dropout a costly waste of mental health resources (Garfield, 1994) but outcomes for clients who dropout early in therapy are similar to those who never begin therapy (Stark, 1992). Dropout rates in individual therapies range from 20 – 50% (Swift, Greenberg, Whipple, & Kominiak, 2012; Wierzbicki, & Pekarik, 1993), with comparable dropout rates in couples and family therapy (Masi, Miller, & Olson, 2003).

The gap between research and practice is also a well-known challenge for improving the effectiveness of mental health care (Kazdin, 2008). This can be demonstrated by the difficulty in translating efficacious interventions developed in laboratory settings into real world practice (i.e., “usual care” or UC). For example, using data collected from over 6000 adult clients receiving UC, treatment was successful with only 20% of the clients while typical lab-based studies showed about three times that level of effectiveness (Hansen, Lambert, & Foreman, 2002). There is less research investigating the translation of interventions from the lab to the field in CFP. This is often attributed to increasing trends of couple and family therapists toward being eclectic and integrative in their approaches (Pinsof & Wynne, 2000). Thus, multiple models or treatment approaches are utilized in an effort to help clients, rather than relying on a single structured approach as often used in lab studies. Additionally, large caseloads and the little time between clinical sessions to complete required paperwork make it difficult to integrate new interventions into practice.

One such intervention is the use of outcome monitoring and feedback for all practices as recommended by the 2006 American Psychological Association (APA) Presidential Task Force on Evidence-Based Practice (EBP). Although some research with couple therapy has shown the effectiveness of incorporating ongoing assessment into clinical work (Anker, Duncan, & Sparks, 2009; Reese, Toland, Slone, & Norsworthy, 2010), more research is needed on how to readily incorporate this type of measurement and feedback into clinical work (Teachman et al., 2012). The pressure to do this may increase rapidly given that the incorporation of treatment progress instruments is likely to become a requirement for reimbursement in the future (Stewart & Chambless, 2008).

Given the complex tasks that couples and family therapists face in treating dynamic family systems, how can we facilitate their delivery of effective treatment while also addressing the challenges inherent in the field of psychotherapy that limit the quality and effectiveness of mental health care? We believe that well designed technology has the potential to provide the tools necessary to do both, particularly with regard to outcome monitoring and feedback.

Technology in Mental Health Care

Although numerous definitions exist, technology often refers to tools and machines that are used to solve real-world problems in an easier and/or more efficient manner. While this definition may include simple tools such as a hammer or lever, technology is now increasingly associated with scientific knowledge and machines, in particular computers. Within mental health care, technologies include any protocol, system, treatment, or intervention that is used to prevent, diagnose, monitor, or treat mental health conditions. In other words, a technology is any innovation that improves the quality of mental healthcare delivery. Thus, this definition includes innovations such as manualized treatments as well as advances that employ electronic devices.

Despite being relatively new, the integration of electronic devices such as computers, cell phones or hand held devices (e.g., PDA's or tablets) into mental health services is increasing. For example, e-health or Telemental Health systems utilize the Internet to provide access to mental health providers from long distances. From the computer, clinicians and patients have clinical sessions where all assessment and treatment can be administered through the Internet (Christensen & Hickie, 2010; Yuen, Goetter, Herbert, & Forman, 2012; King, Bickman, Shochet, McDermott & Bor, 2010). While this delivery of services is still emerging in the mental health field, there is some evidence of its effectiveness for improving patient outcomes for several mental health conditions (for example, see Germain, Merchand, Bouchard, Guay, & Droui, 2010; Griffiths, Farrer, & Christensen, 2010; Nelson, Barnard, & Cain, 2006; Yuen et al., 2010). Cell phones and handheld devices are being used to enhance the treatment of psychiatric disorders (Ehrenreich, Righter, Rocke, Dixon, & Himelhoch, 2011; Granholm, Ben-Zeev, Link, Bradshaw, & Holden, 2011). For example, Granholm and colleagues (2011) found that sending regular text messages to cell phones provided to individuals with schizophrenia resulted in a significant increase in medication adherence. Similarly, several studies reviewed by Ehrenreich et al. (2011) found increased numbers of participants who remained abstinent from smoking by utilizing cell phones and computers to send text messages, voicemail messages, and emails. However, there is little research with these new technologies in CFP.

The integration of electronic technologies into mental health care is relatively recent but is increasing at a rapid rate. Some may even call this a new era for mental health services (Christensen & Hickie, 2010; Kazdin, 2012; Rotheram-Borus, Swendeman & Chorpita, 2012). Smartphone technology is being applied to mental health in many ways. In their search, Luxton, McCann, Bush, Mishkind & Reger (2011) found hundreds of unique mobile phone applications (i.e., “apps”) in the BlackBerry App World site (http://appworld.blackberry.com/webstore/) that were specifically associated with behavioral health. And this number was on the rise. These apps were designed for various purposes such as clinical assessment, symptom monitoring, psychoeducation, resource location, tracking treatment process, skills training and communication with healthcare providers.

There are a multitude of possibilities for integrating such technology into mental health care and researchers are working to keep up with these rapid developments to evaluate their effectiveness. In the next section, we describe measurement feedback systems (MFSs), a technology that has the ability to improve clinical care while also addressing the challenges faced by the field concerning the integration of ongoing outcome monitoring and feedback into practice as well as dropout rates.

Measurement Feedback Systems: The Technology of Delivering Information

As described by Bickman (2008b), a measurement feedback system (MFS) is composed of two components. The first is a measure or a battery of comprehensive measures that are administered regularly throughout treatment to collect ongoing information concerning the process and progress of treatment. In CFP it is critical that information is collected from all participants, including the therapist. The second component of an MFS is the presentation of this information to provide timely and clinically useful feedback to clinicians. This feedback provides information about what does and does not seem to be working so that clinicians can be more responsive to the needs of the client by continuing, discontinuing, or altering treatment plans. MFSs have been used successfully to improve outcomes in treatment with couples (Morten, Anker, Duncan, & Sparks, (2009) in treatment with families (Bickman, Kelley, Breda, De Andrade, & Riemer, 2011) and in general health care (Carlier, Meuldijk, Van Vilet, Van Fenema, Van der Wee, & Zitman, 2012).

However, the focus is often on the first aspect of the MFS technology, the measures. Questions arise on what to measure, when to measure it, who to include as reporters, etc. These are indeed important and necessary components of a MFS—the resulting feedback is obviously limited by the type and validity of the measures that are used to produce the feedback. In addition to having acceptable psychometric properties, measures used for feedback throughout treatment should also be very brief, sensitive to change, and clinically relevant (Kelley & Bickman, 2009). These are the basic measurement properties for any high quality implementation of outcome monitoring intended to inform mental health treatment that incorporates frequent administration of measures throughout treatment. While there have been important advances in measurement development and testing, measurement technology (psychometrics) has been with us for over 100 years.

Despite the importance of the measures in the MFS, it is the second component focused on the delivery of feedback that has the potential to address the challenges faced in the mental health care field in its quest to improve clinical services. A MFS utilizes electronic technology to transform measurement data into information that can influence treatment as it occurs. In other words, information electronically entered into a MFS generates automatic feedback that is available to the clinician for immediate use. This immediate information can then be incorporated into the treatment planning for the very next clinical session or even for use in the current session.

There are two key benefits of an electronic technology over a non-technological solution (e.g., paper and pencil) for promoting the integration of ongoing outcome monitoring and feedback into practice. First, technology provides ease of access to collect and utilize data in the treatment process. The use of computers and hand-held computing devices (e.g., tablets) allows for real-time feedback with the electronic administration of measures, automated calculation of scores, interpretation of results, and immediate presentation of feedback. Second, technology makes it possible to achieve integration of data-informed decision making throughout all aspects of clinical workflow. We call this the ‘golden thread’, a concept based on best practices in clinical documentation systems that facilitates linking individual client needs, treatment goals, therapeutic interventions, and monitoring of client responses (Lloyd, 2011). A MFS makes it possible to spread the ‘golden thread’ of clinical information through all levels of an organization with real time useable information immediately available, from aggregated data for executive planning at the agency leadership level to specific session data that informs what the clinician addresses in the next encounter with a client. Importantly, all the data is accessible in one location. However, this does not mean that all information is available to all users. Instead, a MFS that utilizes a permission-based login allows users to access actions and data tailored to their role in the clinical workflow, restricting or allowing access to functional or administrative aspects as well as levels of aggregation of data. For example, clinic directors could view data aggregated across treatment programs or service settings; supervisors could view data aggregated for clinicians that they supervise; clinicians could view data for only those clients they treat; and clients could have access only to functions related to administration of their own accounts and completion of measures.

The primary purpose of a MFS is that it provides feedback that is used to inform clinical practice. In other words, it is actionable meaning it can be used to influence clinician behavior. A general rule of thumb for any type of feedback to be actionable is that it must be considered by the recipient to be relevant to the issue at hand and delivered in a way that incorporates an understanding of how cognitive processes and learning styles influence responses to feedback (Cannon & Witherspoon, 2005). In addition, emerging guidelines for the design and evaluation of mental health technologies emphasize a user centered and evolving design process (Doherty, Coyle, & Matthews, 2010). User centered design involves not only engaging end users (e.g., clinicians, clients, agency directors) in the development of the functions and interface of the MFS but also incorporating theoretical principles relevant to the use of information to promote clinician behavior change (Riemer, Rosof-Williams, & Bickman, 2005; Sapyta, Riemer, & Bickman, 2005). An evolving design process entails the application of continuous quality improvement to incorporate end user experience into ongoing development of the electronic MFS itself and accompanying support for operation and implementation processes.

In sum, MFSs aim to improve clinical practice by utilizing valid and reliable measures to provide immediate feedback for clinicians concerning key process indicators and measures that reflect progress in treatment. An additional advantage of a MFS is the ability to aggregate information instantly for clinical supervisors and agency leadership for informed decision-making. However, for a MFS to be effective, feedback must be actionable and the system must be user-centered and adopt an evolving design process such that it is able to meet the different and changing needs of organizations and users. As is described in the next section, these principles were considered when the authors developed a MFS called Contextualized Feedback Systems® (CFS™) over the last decade.

Description of Contextualized Feedback Systems® (CFS™)

CFS™ is a quality improvement tool that assists clinicians and supervisors in assessing change as a client progresses. It was designed to enhance clinical effectiveness and improve organizational efficiency by providing feedback to inform clinical sessions, supervision, program planning, and professional development. CFS™ is a web–based, Software as a Service (SaaS) application that was specifically designed to be easy to operate and provide new information to those who treat individuals with behavioral health problems. Thus, CFS™ seamlessly integrates real-time clinical feedback about indicators of treatment progress (e.g., symptoms and functioning, life satisfaction) and process (e.g., motivation for treatment, therapeutic alliance) as reported by the clinician, client, and other respondents (e.g., caregivers). These indicators are acquired through the completion of measures included in the Peabody Treatment Progress Battery (PTPB). The PTPB is a comprehensive set of very brief clinical measures that have demonstrated strong psychometric properties in large samples of youth, and their respective caregivers and clinicians, receiving mental health treatment (Bickman et al., 2007, 2010; Bickman & Athay, 2012).

CFS™ has undergone over ten years of research and development and can be used to support the promotion of evidence-based practice (EBP) in community settings. A hallmark of CFS™ is its flexibility that allows ongoing tracking of any relevant data for individual clients and aggregation of information across units and time. Furthermore, measures and resulting feedback can be contextualized to fit the quality improvement needs of any organization with varied client populations and/or treatment models. In addition, CFS™ provides accountability via implementation feedback with performance benchmarks and a rapid-response data system focused on clinicians and programs.

Evidence-based effectiveness

CFS™ is the only MFS with demonstrated effectiveness in youth mental health settings (Bickman, et al., 2011). In a large randomized trial, results showed that youth whose clinicians were randomly assigned to receive feedback improved significantly faster according to data provided from the clinician, youth and caregiver respondents. Furthermore, there was a dose-response relationship between feedback viewing and clinical outcomes. The more reports viewed by the clinician the faster clinical improvement by the client based on data from the clinician, the youth and the caregiver.

Theory-based development

Research clearly indicates that progress in treatment is multidimensional, and that feedback information is needed not only on traditional outcomes such as a reduction in symptoms, but also on the treatment processes that mediate such outcomes. CFS™ is based on a theory of change and grounded in psychological and organizational research. The feedback approach applies several behavior change theories, including goal theory, cognitive dissonance theory, attribution theory, and strength-based self-efficacy theory (Riemer et al., 2005; Sapyta et al., 2005). In addition, the feedback itself is informed by research on common factors in psychotherapeutic treatment (Kelley, Bickman, & Norwood, 2010). Common factors are related to the impact of client and clinician characteristics, relationship factors (e.g., therapeutic alliance), and other ingredients of the therapeutic process that are common to any psychotherapy, regardless of treatment modality or model.

User centered and evolving design process

CFS™ was developed as a collaborative effort between mental health services researchers and practitioners (Riemer, Kelley, Casey, & Taylor-Haynes, 2012). The research-practice partnership was essential to the development of the measures, the ‘look and feel’ of the computer user interface, the determination of needed administrative functions and controls, and the design of the feedback reports. Software application development is a very complex and expensive ongoing process, with the most recent version (3.0) representing a substantial revision that integrates user feedback to improve incorporation in the therapist workflow. In addition, the content of support materials and processes has evolved over time to include pre-adoption consultation, pre- and post-implementation support, online operations support, and ongoing quality improvement support. This was done to ensure CFS™ could have a sustainable effect throughout the system. Live and web-based training and ongoing coaching support the utilization of CFS™ in clinical, supervisory, applied research, and management activities. Quarterly quality assurance reports and learning collaborative teleconferences are provided beyond the first year of intensive support to reinforce advanced integration of data-informed decision-making and share ongoing learning of promising implementation strategies. This is an undeveloped research area and we are just beginning to establish empirical principles that go beyond the limited effectiveness of traditional technical assistance and continuing education.

CFS™ technology is evolving and improving. For example, our earlier versions suffered from implementation problems associated with all evidence based treatments. Therapist adherence in completing questionnaires and viewing reports were low and thus effectiveness suffered. In the latest version of CFS™ we anticipate that we ameliorated some of these problems by displaying adherence rates that can be observed by the therapist, supervisor and management and thus each therapists' adherence is easily tracked. Second, we are able to integrate the outcome of the measures used in CFS™ into the typical reporting requirements (e.g., session-based documentation) each therapist must complete in order to be reimbursed. By making adherence more transparent and visible as well as building feedback into the required reporting we expect adherence to be greatly improved.

Learning organization support

CFS™ is designed to assist in organizational change by sustaining a continuous learning environment, and encouraging and facilitating the appropriate use of empirically based mental health interventions. The mental health care field, like much of the medical community, lacks the electronic infrastructure to efficiently generate and collect clinical data. In commentating on medical care, Lindor (2012) notes that the absence of a method to gather clinical data in usual care is a major barrier to the development of evidenced based treatments. This limits the ability of provider organizations to learn how to improve their services, minimizing their ability to contribute knowledge to the field as a whole. To develop and understand an intervention's real world effectiveness there must be an easy mechanism to efficiently collect and analyze clinical data from real world clients. CFS™ provides that mechanism. New measures can be introduced into the system at nominal cost and measures can be revised and improved as new data are gathered. Additionally, CFS™ allows the testing of new interventions in the real world with essentially no cost associated with data collection. Moreover, the system includes a random assignment generator that supports the organization in the design of relatively easy to implement randomized clinical trials. In this way, service providers can help fill the gaps in our knowledge by identifying effective treatments that take into account the context, therapist and client characteristics. A MFS such as this provides an easy and inexpensive technology for on-going data collection as part of usual clinical practice and forms the infrastructure for data collection from all participants.

MFSs in Couples and Family Therapy

MFSs appear to have a positive effect on client outcomes for both families and couples, however the number of research projects is very small. It is a new technology in need of great deal more research. Despite this, there is great potential for MFSs in couples and family therapy; particularly for addressing the general challenges facing the field of psychotherapy discussed previously, namely significant dropout rates and the gap between research and practice. In this section, we will briefly discuss how a MFS can address these challenges specifically within the unique setting of couple and family therapy. Given the authors extensive knowledge about the development, use, and potential of CFS™, we will utilize this specific MFS for demonstration purposes.

Challenge of dropout

As previously discussed, premature dropout from therapy is costly for the mental health system and client's clinical outcome. Although there are many factors that relate to premature dropout, one factor consistently found is the quality of the relationship between the client and the therapist (i.e., therapeutic alliance). Research has demonstrated that good therapeutic alliance is positively correlated with lower dropout rates (Kazdin, Holland, & Crowley, 1997; Shields, Sprenkle, & Constantine, 1991). Given the greater complexity couples and family therapists face in establishing good therapeutic alliance with each member in the dynamic family system, research suggests a potentially greater risk for premature dropout compared to individual therapy (Johnson & Wright, 2002). The challenge in establishing good therapeutic alliance is especially apparent when considering that each individual in a couple or family begins treatment with several presenting problems, different levels of motivation, degrees of interest, goals, and beliefs about how change should occur (Rait, 2000).

Although measures of symptom severity are often the only focus of clinical feedback, CFS™ was specifically designed to incorporate a wide variety of measures relevant to clinical care. These can include measures assessing therapeutic alliance, motivation for treatment, and expectations about therapy. By collecting such information from each participant in treatment, CFS™ creates feedback that alerts the clinician to potential issues or discrepancies that may need to be addressed with individual clients or couple or family. While we recognize that clinicians are continuously gathering similar information informally (i.e., through interactions with clients, observation of body language, etc.), CFS™ enhances the clinician's ability to make appropriate clinical judgments by highlighting relevant information and tracking it over time, reducing the cognitive burden of the clinician. Thus the clinician is better equipped to address issues surrounding therapeutic alliance and prevent premature dropout.

Challenge of the research and practice gap

The other challenge faced by the general field of mental health services is the gap between research and practice. Two ways this gap is manifested are the difficulties in translating interventions from laboratory settings into real world practice and incorporating outcome monitoring into clinical care. First, as discussed previously, it is common that treatments developed and evaluated in a laboratory setting (i.e., evidence based treatments) do not display similar results in real world settings. Using a MFS such as CFS™ can provide the information necessary to get a clearer picture about what treatment are effective and for who in UC (Bickman, 2008a). For example, CFS™ has been paired with Functional Family Therapy (FFT) to explore the effectiveness of FFT in UC settings and help to understand the conditions under which it works best. Because CFS™ was not developed to be treatment-specific, it can be utilized with any evidence based treatment or combination of treatments. This is especially useful in the field of CFP where therapists report utilizing multiple different approaches and techniques.

The other manifestation of the research-practice gap is the difficulty in incorporating outcome monitoring in UC. Despite the increasing evidence supporting the effectiveness of this for improving client outcomes (e.g. Anker et al., 2009) and the frequent calls for its use in all practices (e.g., APA Presidential Task Force on Evidence-Based Practice, 2006), work is needed to assist therapists in adopting and implementing outcome monitoring in their practice. CFS™ was developed as an automated, computerized, and contextualized user-friendly and user-centered system that can be integrated into a large variety of clinical settings. In consideration of the tasks already required by therapists and support staff, as well as the limitations of time, CFS™ measures were designed to collect maximum information in a limited amount of time (i.e., the last 5 minutes of a session) and are able to be completed electronically (by computer, iPad, etc.) to eliminate the need for manual data entry. Additionally, feedback was designed to be delivered electronically and presented in an easy-to-understand manner that highlights potentially critical clinical information (provides “alerts”) yet allows for the clinician to view any and all data. This is accomplished with a ‘dashboard’ system that displays critical and summary information at a glance with the ability of the clinician to select vivid icons to explore desired information in more detail.

Technology, Therapy, MFSs and the Future

MFSs such as CFS™ have demonstrated a positive effect on client outcomes for both individual therapy settings as well as couple and family therapy settings. Many of the features of MFSs are theoretically and conceptually compelling, such as the need for feedback for individual learning and the urgency in creating large-scale data infrastructures to aid in organizational learning. Simply put, a MFS provides the means to collect data from multiple perspectives relevant to CFP frequently throughout treatment, produce real-time feedback to inform treatment as it occurs, and provide aggregated data that can be used to build practice-based evidence within real world community settings. However, we are well aware of the key aspects of MFSs that are unknown at this time. Therefore, MFSs are not being presented as panaceas.

There are many limitations associated with MFSs because of the small amount of research being conducted with them. For one, we do not know how therapists actually use the feedback in practice. There is little research that ties the feedback alerts given by most MFSs to actions that the clinician can or should take. For example we know that the correlations among a youth, a caregiver and a therapist on ratings of client severity is very low but it is not clear what clinicians should do with this information. Similarly, from our own research on therapeutic alliance (Bickman, 2012) we know that the clinician's estimate of the youth's or caregiver's rating of therapeutic alliance has a low correlation with the clinician's estimate. When is it appropriate for the clinician to address such discrepancies? Does merely knowing the discordance help the clinician? A MFS provides new knowledge to clinicians but we need further research on effective clinical strategies needed to apply this specific and client-centered knowledge to clinical practice. Fortunately, most MFSs can be used to productively explore such questions.

Similar to any other intervention, the effectiveness of a MFS is dependent on the success of implementation. As noted above, in version 3 of CFS™ key indicators of the quality of implementation are visible on every dashboard to aid in monitoring. Additionally, separate and more detailed implementation reports are available to therapists, supervisors and management. But this information must be used by the provider organization for it to be effective. Organizations that will not or cannot support change and the use of technology are unlikely to be successful in implementing a MFS. Simply put, it won't work if it is not used. As an emerging technology, there is a lack of research on the most effective design that promotes both adherence and integration of feedback into clinic workflow. We need more research on the application of user centered design to promote evidence-based evaluation of MFSs.

While there are now several commercially available MFSs that have shown positive effects on clinical outcomes, the adoption of MFSs are limited by the lack of incentives for delivering effective services. Without such incentives MFSs will be limited to the providers who can find funding and who share a vision with the developers of MFSs about how to improve mental health services. Significant financial incentives are being offered in by the Centers for Medicare & Medicaid Services (CMS) for electronic health records that include “meaningful use” of these records. However, mental health services are only tangentially eligible for these financial incentives at this time although it should be noted that pressure is being applied to extend the incentives to mental health professionals.

Meaningful use is being implemented in stages. Stage 1 does not require the comprehensive feedback provided by many MFSs. However, the CMS Electronic Health Record Incentive Programs stage 2 requirement for meaningful use scheduled for implementation in 2014 will require Clinical Quality Measures (CQMs). CMS has recommended nine core CQMs for both adult and pediatric populations. The CQMs for both eligible professionals and hospitals must cover three of six National Quality Strategy (NQS) domains: Patient and family engagement; Patient safety; Care coordination; Population and public health; Efficient use of healthcare resource; and Clinical processes/effectiveness. We anticipate changes in policy and funding in mental health services that will follow CMS's approach, which demands more performance indicators from mental health providers than simply how many clients have been seen. The technology of MFSs will help us meet those anticipated requirements.

Footnotes

i

This paper was partially supported by a grant from the National Institute of Mental Health (5 R01 MH087814) to Leonard Bickman.

References

  1. APA Presidential Task Force on Evidence-Based Practice. Evidence-based practice in psychology. American Psychologist. 2006;61:271–285. doi: 10.1037/0003-066X.61.4.271. [DOI] [PubMed] [Google Scholar]
  2. Bickman L. Integrating Functional Family therapy and CFS into a community based mental health system “Contextualized Feedback Systems” paper presented at the American Psychological Association Convention 2012 [Google Scholar]
  3. Bickman L. Why don’t we have effective mental health services? Administration and Policy in Mental Health and Mental Health Services Research. 2008a;35:437–439. doi: 10.1007/s10488-008-0192-9. [DOI] [PubMed] [Google Scholar]
  4. Bickman L. A measurement feedback system (MFS) is necessary to improve mental health outcomes. Journal of the American Association of Child and Adolescent Psychiatry. 2008b;47:1114–1119. doi: 10.1097/CHI.0b013e3181825af8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Bickman L, Athay MM, editors. Youth Mental Health Measurement (special issue) Administration and Policy in Mental Health and Mental Health Services Research. 2012;39(1-2) [Google Scholar]
  6. Bickman L, Athay MM, Riemer M, Lambert EW, Kelley SD, Breda C, Tempesti T, Dew-Reeves SE, Brannan AM, Vides de Andrade AR, editors. Manual of the Peabody Treatment Progress Battery, 2nd ed [Electronic version] Nashville, TN: Vanderbilt University; 2010. http://peabody.vanderbilt.edu/ptpb/ [Google Scholar]
  7. Bickman L, Kelley SD, Breda C, de Andrade AR, Riemer M. Effects of routine feedback to clinicians on mental health outcomes of youths: Results of a randomized trial. Psychiatric Services. 2011;62(12):1423–1429. doi: 10.1176/appi.ps.002052011. [DOI] [PubMed] [Google Scholar]
  8. Cannon MD, Witherspoon R. Actionable feedback: Unlocking the power of learning and performance improvement. The Academy of Management Executive. 2005;19(2):120–134. doi: 10.5465/AME.2005.16965107. [DOI] [Google Scholar]
  9. Carlier IV, Meuldijk D, Van Vliet IM, Van Fenema E, Van der Wee N, Zitman FG. Routine outcome monitoring and feedback on physical or mental health status: evidence and theory. Journal of Evaluation in Clinical Practice. 2012;18(2012):104–110. doi: 10.1111/j.1365-2753.2010.01543.x. [DOI] [PubMed] [Google Scholar]
  10. Christensen H, Hickie IB. Using e-health applications to deliver new mental health services. Medical Journal of Australia. 2010;192:53–56. doi: 10.5694/j.1326-5377.2010.tb03695.x. https://www.mja.com.au/ [DOI] [PubMed] [Google Scholar]
  11. Dew S, Riemer M. Why inaccurate self-evaluation of performance justifies feedback interventions Paper presented at the 16thAnnual Research Conference, A System of Care for Children's Mental Health. Tampa: University of South Florida; 2003. [Google Scholar]
  12. Doherty G, Coyle D, Matthews M. Design and evaluation guidelines for mental health technologies. Interacting with Computers. 2010;22(4):243–252. doi: 10.1016/j.intcom.2010.02.006. [DOI] [Google Scholar]
  13. Ehrenreich B, Righter B, Rocke DA, Dixon L, Himelhoch S. Are mobile phones and handheld computers being used to enhance delivery of psychiatric treatment? A systematic review. The Journal of Nervous and Mental Disease. 2011;199:886–891. doi: 10.1097/NMD.0b013e3182349e90. [DOI] [PubMed] [Google Scholar]
  14. Garfield SL. Research on client variables in psychotherapy. In: Bergin AE, Garfield SL, editors. Handbook of psychotherapy and behavior change. 4th. New York: John Wiley & Sons; 1994. pp. 190–228. [Google Scholar]
  15. Germain V, Marchand A, Bouchard S, Guay S, Drouin M. Assessment of the therapeutic alliance in face-to-face or videoconferencing cognitive-behavioral therapy for PTSD delivered via videoconferencing technology. Behavior Modification. 2010;31:856–866. doi: 10.1177/0145445507302125. [DOI] [Google Scholar]
  16. Granholm E, Ben-Zeev D, Link PC, Bradshaw KR, Holden JL. Mobile assessment and treatment for schizophrenia (MATS): A pilot trial of an interactive text-messaging intervention for medication adherence, socialization, and audiotory hallucinations. Schizophrenia Bulletin. 2011 doi: 10.1093/schbul/sbr155. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Griffiths KM, Farrer L, Christensen H. The efficacy of internet interventions for depression and anxiety disorders: A review of randomized controlled trials. Medical Journal of Australia. 2010;192(11 Suppl):S4–S11. doi: 10.5694/j.1326-5377.2010.tb03685.x. https://www.mja.com.au/ [DOI] [PubMed] [Google Scholar]
  18. Hannan C, Lambert MJ, Harmon C, Nielsen SL, Smart DW, Shimokawa K, Sutton SW. A lab test and algorithms for identifying clients at risk for treatment failure. Journal of Clinical Psychology. 2005;61:155–163. doi: 10.1002/jclp.20108. [DOI] [PubMed] [Google Scholar]
  19. Hansen HB, Lambert MJ, Forman EM. The psychotherapy dose-response effect and its implications for treatment delivery services. Clinical Psychology: Science and Practice. 2002;9:329–343. doi: 10.1093/clipsy.9.3.32. [DOI] [Google Scholar]
  20. Johnson LN, Wright DW. Revisiting Bordin's theory on the therapeutic alliance: Implications for family therapy. Contemporary Family Therapy: An International Journal. 2002;24:257–269. doi: 10.1023/A:1015395223978. [DOI] [Google Scholar]
  21. Kazdin A. Advancing New Frontiers With Clinical Psychological Science. APS Observer. 2012;25(3):14–16. 34. [Google Scholar]
  22. Kazdin AE. Evidence-based treatment and practice: New opportunities to bridge clinical research and practice, enhance the knowledge base, and improve patient care. American Psychologist. 2008;63:146–159. doi: 10.1037/0003-066X.63.3.146. [DOI] [PubMed] [Google Scholar]
  23. Kazdin AE, Holland L, Crowley M. Family experience of barriers to treatment and premature termination from child therapy. Journal of Consulting and Clinical Psychology. 1997;65:453–463. doi: 10.1037/0022-006X.65.3.453. [DOI] [PubMed] [Google Scholar]
  24. Kelley SD, Bickman L. Beyond outcomes monitoring: Measurement feedback systems in child and adolescent clinical practice. Current Opinion in Psychiatry. 2009;22(4):363–368. doi: 10.1097/YCO.0b013e32832c9162. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Kelley SD, Bickman L, Norwood E. Evidence-based treatments and common factors in youth psychotherapy. In: Duncan BL, Miller SD, Wampold BE, Hubble MA, editors. The heart and soul of change: Delivering what works in therapy. 2nd. Washington, DC: American Psychological Association; 2010. pp. 325–355. [DOI] [Google Scholar]
  26. King R, Bickman L, Shochet I, McDermott B, Bor B. Use of the internet for provision of better counselling and psychotherapy services to young people, their families and carers. Psychotherapy in Australia. 2010;17(1):62–70. [Google Scholar]
  27. Lindor RA. Comment, Advancing Evidence-Based Medicine by Expanding Coverage with Evidence Development. Jurimetrics. 2012;52:209–237. [Google Scholar]
  28. Lloyd D. Reviewing and Applying State and Federal Audit Reports to Enhance Knowledge Base Regarding Audit Expectations, Trends and Results. 2011 Oct 27; Webinar for the National Council at www.nationalcouncil.org. Retrieved on September 26, 2012.
  29. Luxton DD, McCann RA, Bush NE, Mishkind MC, Reger GM. mHealth for mental health: Integrating smartphone technology in behavioral healthcare. Professional Psychology: Research and Practice. 2011;42:505–512. doi: 10.1037/a0024485. [DOI] [Google Scholar]
  30. Masi MV, Miller RB, Olson MM. Differences in dropout rates among individual, couple, and family therapy clients. Contemporary Family Therapy. 2003;25:63–75. doi: 10.1023/A:1022558021512. [DOI] [Google Scholar]
  31. Morten G, Anker MG, Duncan BL, Sparks JA. Using client feedback toimprove couple therapy outcomes: A randomized clinical trial in a naturalistic setting. Journal of Consulting and Clinical Psychology. 2009;77(4):693–704. doi: 10.1037/a0016062. [DOI] [PubMed] [Google Scholar]
  32. Nelson EL, Barnard M, Cain S. Feasibility of telemedicine intervention for childhood depression. Counseling and Psychotherapy Research. 2006;6:191–195. doi: 10.1080/14733140600862303. [DOI] [Google Scholar]
  33. Pinsof WM, Wynne LC. Toward progress research: Closing the gap between family therapy practice and research. Journal of Marital and Family Therapy. 2000;26:1–8. doi: 10.1111/j.1752-0606.2000.tb00270.x. [DOI] [PubMed] [Google Scholar]
  34. Rait DS. The therapeutic alliance in couples and family therapy. Psychotherapy in Practice. 2000;56(2):211–224. doi: 10.1002/(SICI)1097-4679(200002)56:2. [DOI] [PubMed] [Google Scholar]
  35. Reese RJ, Toland MD, Slone NC, Norsworthy LA. Effect of client feedback on couple psychotherapy outcomes. Psychotherapy Theory, Research, Practice, Training. 2010;47:616–630. doi: 10.1037/a0021182. [DOI] [PubMed] [Google Scholar]
  36. Riemer M, Kelley SD, Casey S, Taylor Haynes K. Developing effective research-practice partnerships for creating a culture of evidence-based decision making. Administration and Policy in Mental Health and Mental Health Services Research Special Issue: Research Practice Partnerships. 2012;39(4):248–257. doi: 10.1007/s10488-011-0368-6. [DOI] [PubMed] [Google Scholar]
  37. Riemer M, Rosof-Williams J, Bickman L. Theories related to changing clinician practice. Child and Adolescent Psychiatric Clinics of North America. 2005;14(2):241–254,viii. doi: 10.1016/j.chc.2004.05.002. [DOI] [PubMed] [Google Scholar]
  38. Rotheram-Borus MJ, Swendeman D, Chorpita BF. Disruptive Innovations for Designing and Diffusing Evidence-Based Interventions, American Psychologist. 2012;67(6):463–476. doi: 10.1037/a0028180. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Sapyta J, Riemer M, Bickman L. Feedback to clinicians: Theory, research & practice. Journal of Clinical Psychology. 2005;61(2):145–153. doi: 10.1002/jclp.20107. [DOI] [PubMed] [Google Scholar]
  40. Sexton TL, Datachi Phillips C, Evans L, LaFollette J, Wright L. The effectiveness of couple and family-based clinical interventions. In: Lambert M, editor. Handbook of Psychotherapy and Behavior Change. Wiley; New York: in press. [Google Scholar]
  41. Shields CG, Sprenkle DG, Constantine JA. Anatomy of an initial interview: The importance of joining the structuring skills. American Journal of Family Therapy. 1991;19:3–18. doi: 10.1080/01926189108250829. [DOI] [Google Scholar]
  42. Stark MM. Dropping out of substance abuse treatment: A clinically oriented review. Clinical Psychology Review. 1992;12:93–116. doi: 10.1016/0272-7358(92)90092-M. [DOI] [Google Scholar]
  43. Stewart RE, Chambless DL. Treatment failures in private practice: How do psychologists proceed? Professional Psychology: Research and Practice. 2008;39:176–181. doi: 10.1037/0735-7028.39.2.176. [DOI] [Google Scholar]
  44. Swift JK, Greenberg RP, Whipple JL, Kominiak N. Practice recommendations for reducing premature termination in therapy. Professional Psychology: Research and Practice. 2012;43:379–387. doi: 10.1037/a0028291. [DOI] [Google Scholar]
  45. Teachman BA, Drabick DAG, Hershenberg R, Vivian D, Wolfe B, et al. Bridging the gap between clinical research and clinical practice: Introduction to the special section. Psychotherapy. 2012;49:97–100. doi: 10.1037/a0027346. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Wierzbicki M, Pekarik G. A meta-analysis of psychotherapy dropout. Professional Psychology: Research and Practice. 1993;24:190–195. doi: 10.1037/0735-7028.24.2.190. [DOI] [Google Scholar]
  47. Yuen EK, Goetter EM, Herbert JD, Forman EM. Challenges and opportunities in internet-mediated telemental health. Professional Psychology: Research and Practice. 2012;43:1–8. doi: 10.1037/a0025524. [DOI] [Google Scholar]
  48. Yuen EK, Herbert JD, Forman EM, Goetter EM, Juarascio AS, Rabi SJ, et al. Bouchard S. Using Skype videoconferencing and Second Life virtual environments to deliver acceptance-based behavior therapy for social anxiety disorder; Paper presented at the 44th Annual Convention of the Association for Behavioral and Cognitive Therapies; San Francisco, CA. 2010. Aug, [Google Scholar]

RESOURCES