The Case For Measuring Quality In Mental Health And Substance Abuse Care

Abstract

Over the past decade, efforts to measure and improve quality have permeated health policy and health care generally but have barely penetrated mental health and substance abuse care. We review barriers and recent activities in these areas and propose a short list of quality measures to engage the policy and practice community in a discussion about how best to evaluate the care of people with these conditions. Quality measures could include, for example, screening, brief intervention, and referral for alcohol abuse. Because proposing a list is only a first step, we suggest other elements of a broader strategy to bring mental health and substance use care into the mainstream of health care quality improvement.

Ever since the Institute of Medicine in 2001 released its landmark report, Crossing the Quality Chasm: A New Health System for the 21st Century,¹ quality measurement activities have increasingly permeated the overall health policy landscape as well as the day-today operations of medical and surgical inpatient and outpatient care settings. The report proposed a framework of quality improvement built on six aims—safe, effective, timely, efficient, equitable, and patient-centered health care— and a series of rules and strategies. It furthered the development and expansion of a plethora of quality measurement–related policy initiatives, many of which are reflected in this issue of Health Affairs.

These activities include the reporting of quality indicators on public websites; pay-for-performance programs for hospitals and physicians; and organizations’ efforts to develop, test, and vet quality measures. Quality measurement achieved even greater importance with the passage of the Affordable Care Act of 2010, which incorporates additional quality initiatives.

Yet the concept of quality measurement does not appear to have penetrated very far into mental health and substance abuse care, despite a subsequent report by another Institute of Medicine committee five years later that introduced a comprehensive strategy for applying the Quality Chasm approach to the mental health field.² The committee emphasized several key issues: the high prevalence, costs, and individual and societal burdens of mental health and substance use disorders; the co-occurrence of diabetes, cardiovascular disease, and other general medical conditions among many people with mental health and substance use conditions; and evidence of serious quality problems in this field.

In fact, not long after that report was issued, the National Committee for Quality Assurance reported that despite important gains in quality in the general medical/surgical sector, “there are…disturbing exceptions to this pattern of [overall health care quality] improvement. The quality of care for Americans with mental health problems remains as poor today as it was several years ago.”³

The Institute of Medicine committee’s report on mental health care offered two overarching recommendations: First, mental health and substance abuse treatment should be more closely coordinated with general medical care. And second, the aims, rules, and redesign strategies set forth in the original Quality Chasm report should be applied throughout the day-to-day operations of treating mental health and substance use disorders and tailored to address the particular context of these illnesses.

The committee cited distinctive characteristics and barriers that impede quality improvement in these areas: fewer objective, standardized methods for diagnosing mental health and substance use disorders than for general health conditions; weakness of the evidence base supporting quality measures; inadequate leadership and resources for developing such measures; and lack of coherent strategies for adopting and implementing quality measures. The report described and recommended multiple strategies to enhance the “less well developed” infrastructure to measure, analyze, publicly report, and improve the quality of health care for such patients.

Remarkably, the response to these recommendations has been tepid, at best. No entity has stepped in to take responsibility for leadership in implementing these recommendations. There have been no announcements of major new initiatives or programs in this regard from federal agencies or major nongovernmental organizations. No coordinated efforts for research programs to develop better methods or measures have emerged from major federal research agencies or foundations.

What’s more, the National Committee for Quality Assurance’s Healthcare Effectiveness Data and Information Set measures have not greatly improved, and fewer than 5 percent of the National Quality Forum’s list of more than 650 vetted indicators specifically relate to care for people with mental health and substance use conditions.⁴

Mixed Views On The Behavioral Health Quality Movement

Some in the behavioral health field may accept or even prefer the scenario that has left mental health on the sidelines of the quality movement, believing that it has protected the field from bureaucratic intrusion and “cookbook medicine.” But the failure to develop a robust response to the Institute of Medicine report on mental health and substance abuse care, however, has serious consequences.

Ineffective or unsafe care and the inability to obtain effective care have consequences both for individuals and families affected by these conditions and for the nation as a whole. For example, people with severe mental illnesses are now dying twenty-five years earlier than the general population; mortality rates for this population are increasing.⁵

Furthermore, mental illness is nearly universal among the highest-cost, most frequently hospitalized Medicaid beneficiaries.⁶ In fact, the total economic burden of mental illness— excluding homelessness, incarceration, other health conditions, and early mortality—was estimated to be $317.6 billion in 2002, the equivalent of more than $1,000 for every man, woman, and child in the United States.⁷

It is not that the field simply lacks quality measures. In fact, Benjamin Herbstman and Harold Alan Pincus identified more than forty different behavioral health quality measurement initiatives in the United States.⁸ These initiatives have been established by various federal and state government agencies, professional organizations, and accreditation bodies, and they vary widely in their degree of development and scope. And recently there have been a number of important, although disparate, efforts to develop and apply quality measurement tools.

The Department of Veterans Affairs, for instance, commissioned a team from the RAND Corporation and the Altarum Institute to conduct a comprehensive evaluation of the quality of mental health and substance abuse care provided by the Veterans Health Administration.⁹ The evaluation focused on five high-volume, high-cost populations: veterans with schizophrenia and major depressive, bipolar, post-traumatic stress, and substance use disorders. This involved developing more than eighty quality indicators and populating them with data from multiple sources.¹⁰

At the international level, a group of clinical experts under the auspices of the International Initiative for Mental Health Leadership Clinical Leaders Group initiated a project to develop consensus on an overarching framework of core performance and outcome measures that could be used by all participating countries. The goal is to compare system performance across countries to help transform mental health services.

Thus far, the project has identified an inventory of more than 630 indicators currently applied or proposed across the twelve participating countries.¹¹ Phase II of the project is focusing on selecting a limited number of core indicators to be included in the framework based on their importance, feasibility, and validity.

More recently, the Affordable Care Act has stimulated quality measurement activities, including some that apply to mental health and substance use disorders. For example, late in 2010 the secretary of health and human services issued a notice in the Federal Register that recommended an initial core set of health quality measures for Medicaid-eligible adults, as required by section 2701 of the Affordable Care Act, for voluntary use by state Medicaid programs.¹² This core set of fifty-one measures includes eleven specifically focused on mental health and substance use disorders.

The problem is that there is little coordination among these efforts, and no entity has responsibility to provide leadership, create energy and focus, and marshal resources to respond to the challenges posed by the Institute of Medicine’s Quality Chasm report for mental health and substance use services.

Against this backdrop, what is the best way to push the field into the mainstream of the larger quality measurement “movement”? In a recent conversation, a leading policy official suggested one component of a broader strategy: “This field always makes things too complicated. Just come up with a simple list of eight to ten measures in a generally understandable framework to get things going.”

A Framework For Measuring Quality

Exhibit 1 contains ten measures (some with multiple parts) presented in the framework suggested by the Quality Chasm report and balanced across structure, process, and outcomes, as well as across mental health and substance use conditions. The measures could be applied to any health care setting.

Exhibit 1.

List Of Proposed Mental Health Quality Indicators

Domain and indicator description	Categorya	Source
SAFETY
1. Appropriate monitoring of metabolic/cardiovascular side effects for individuals receiving antipsychotic medication	Process	Based on VHA Mental Health Program evaluation and HHS Medicaid Federal Register notice
EFFECTIVENESS
2. Meaningful use of disease registries and evidence-based decision support for (at least two) behavioral health conditions	Structure	Adapted from VHA Mental Health Program evaluation and NCQA medical home criteria
3a. Depression screening and follow-up	Process	Depression screening based on recommendation of the
3b. Use of standardized assessment tools (for example, PHQ-9) for depression	Process	US Preventive Services Task Force and HHS Medicaid Federal Register notice (3a); depression
3c. Depression remission at 6 months	Outcome	measures (3b-d) are based on recommendations from
3d. Depression remission at 12 months	Outcome	NQF Mental Health Outcomes Steering Committee
4. Screening, brief intervention, and referral for alcohol abuse	Process	Based on recommendation of US Preventive Services Task Force and Physician Consortium for Performance Improvement
5. Appropriate number of visits after initiating ADHD treatment	Process	NQF-endorsed measure stewarded by NCQA
PATIENT-CENTEREDNESS
6. Experience of care/satisfaction with care/recovery consumer survey items	Process/outcome	Selected items drawn from Consumer Assessment of Healthcare Providers and Systems and recovery section of Client Survey used in national VHA Mental Health Program evaluation
TIMELINESS
7. Initiation and engagement in alcohol and drug dependence treatment within 14 days, 30 days	Process	Measure developed by Washington Circle Group, and endorsed by NQF and NCQA
EFFICIENCY
8. 30-day rehospitalization for individuals hospitalized for a mental health or substance use condition	Process/outcome	Based on similar measures for general medical conditions or all conditions as in HHS Medicaid Federal Register notice
EQUITY
9a. Items 1, 3–8 analyzed for disparities with regard to race/ethnicity, sex, and age (over age 65 and under age 18)	Process/outcome	Adaptation of existing measures segmented to assess performance for specific populations; also, important for
9b. General medical quality indicators for chronic conditions such as diabetes, cardiovascular disease, and preventive care analyzed for population denominators with mental illness comorbidity		establishing mutual accountability across mental health and substance use and general medical care
10. Availability and distribution materials for shared decision making, self-management, and recovery that are culturally relevant to populations in community being served	Structure	Key elements of Chronic Care Model and cultural competency

SOURCE Authors’ analysis, based on the following sources. (1) Notes 9 and 12 in text. (2) Note 9 in text. National Committee for Quality Assurance. New PCMH 2011 content and scoring summary [Internet]. Washington (DC): NCQA; 2011 [cited 2011 Mar 14]. Available from: http://www.ncqa.org. (3) US Preventive Services Tax Force. Screening for depression in adults [Internet]. Rockville (MD): The Task Force; 2009 Dec [cited 2011 Feb 20]. Available from: http://www.uspreventiveservicestaskforce.org/uspstf/uspsaddepr.htm. Notes 12 and 4 in text. (4) US Preventive Services Task Force. Screening and behavioral counseling interventions in primary care to reduce alcohol misuse [Internet]. Rockville (MD): The Task Force; 2004 April [cited 2011 Feb 20]. Available from: http://www.uspreventiveservicestaskforce.org/uspstf/uspsdrin.htm. American Medical Association. Physician Consortium for Performance Improvement (PCPI) performance measure status report [Internet]. Chicago (IL): AMA; 2010 Jan [cited 2011 Feb 22]. Available from: http://www.ama-assn.org/ama1/pub/upload/mm/370/measures.pdf. (5) Note 4 in text. (6) Note 9 in text. (7) National Committee for Quality Assurance. NQF-endorsed national voluntary consensus standards for physician-focused ambulatory care. Appendix A—NCQA measure technical specifications April, 2008, V.7 [Internet]. Washington (DC): NCQA; 2008 Apr [cited 2011 Mar 24]. Available from: http://www.ncqa.org/tabid/59/Default.aspx. (8) Note 12 in text. (9) Same as for items 3–8 in exhibit. (10) Note 2 in text. NOTES VHA is Veterans Health Administration. HHS is Department of Health and Human Services. NCQA is National Committee for Quality Assurance. NQF is National Quality Forum. PHQ is Patient Health Questionnaire. ADHD is attention deficit hyperactivity disorder.

^aCategory refers to the three-prong Donabedian classification system, which is used to assess the quality of health care.

We do not claim that this is the ideal or even an adequate list. However, the measures we selected focus attention across a set of specific issues and populations of high priority, such as co-occurring health conditions, the health of children and adolescents, and the mental health recovery movement.

Moreover, as noted above, this or any portfolio of measures is just one element of a broader and deeper strategy. Our hope is that by proposing a concrete set of indicators, we will engage the policy and practice community in a spirited discussion of how to best measure the quality of care for people with mental health and substance use disorders, ending the relative inattention to these issues. We hope that this discussion can lead to systematic approaches to understand and improve the quality of care and, by extension, health outcomes.

Clearly much more needs to be done. Possible action steps to take in response to the list of measures include the following.

PROVIDING RESOURCES AND STEWARDSHIP

Mark Chassin and colleagues¹³ point out that measures should have tight, evidence-based links between process performance and patient outcomes, making them useful for accountabil ity purposes such as accreditation, public reporting, and pay-for-performance. However, few fully validated and reliable performance measures now exist for mental health and substance use disorders.

There is also a variable evidence base supporting the linkage between process and outcomes for many of the measures identified in Exhibit 1. For example, there is good evidence that initiating and engaging in substance abuse treatment generally results in better outcomes. Yet a recent study has raised questions about the linkage of proposed indicators (for example, indicator 7 in Exhibit 1) with substance abuse outcomes.¹⁴

Also, formal empirical evidence for the particular measure specifications—such as frequency of testing—is often lacking even when there is a strong linkage between process and outcomes. For example, there is good evidence supporting the need for laboratory tests to monitor the metabolic side effects of antipsychotic medications prescribed for people with schizophrenia and bipolar illnesses. However, empirical evidence is lacking for how to specify the frequency of such monitoring.

The same is true for the frequency of monitoring children receiving medication for attention deficit hyperactivity disorder. It is also difficult to assess the quality of nonpharmacologic—that is, psychosocial or psychotherapeutic—interventions and to define minimal standards for delivering a particular psychotherapeutic intervention in the way it was delivered in research studies supporting its efficacy.

These complexities point to the need for careful stewardship to achieve a consensus on what quality domains are most important to measure, and to coordinate studies aimed at gathering evidence to build a more robust portfolio of measures. No entity is now providing leadership for the development of mental health and substance use measures. Moreover, there is no clear source of funding to support the testing and generation of the data suggested by Chassin and colleagues.¹³

IMPLEMENTING STANDARDIZED MEASUREMENT

Systematic diagnostic evaluations and longitudinal clinical assessments are cardinal features of medical care.¹⁵ These practices also represent scaffolding on which to build quality measurement and improvement practices. Although laboratory tests are not now available for mental health and substance use disorders, standardized and validated clinician- or patient-administered assessment tools do exist and should be routinely implemented in clinical settings.

For example, the PHQ-9 (Patient Health Questionnaire), a nine-item tool to assess depression, has been well documented as being reliable and practical and has become a key component of evidence-based quality improvement strategies. Similar measures should be developed for other mental health and substance use conditions; their use needs to be integrated into routine practice. Widespread implementation of “measurement-based care” would also allow clinically relevant process and outcomes data to be aggregated for the evaluation of quality at multiple levels.

INCORPORATING MENTAL HEALTH CARE INTO HEALTH INFORMATION TECHNOLOGY

Data elements related to the quality of care for mental health and substance use disorders need to be incorporated into health information technology (IT) tools and databases and used to measure and improve quality. Doing so would also standardize the way in which providers record key data elements and would allow patient care to be systematically tracked, coordinated, and evaluated.

Unfortunately, behavioral health has lagged other areas of health care in IT implementation. An important policy barrier is the failure of legislatively authorized incentives for the adoption of such technology by mental health and substance abuse treatment providers and facilities.

SETTING BENCHMARKS, COMPARISONS, AND ACCOUNTABILITY

Beyond developing a better set of measures, it is important to delineate appropriate benchmarks and methods for standardization and comparison across health care settings. Performance expectations often need to be adjusted across providers and settings that serve different types of patients with varying severity of illness. These are issues for quality measurement in health care more broadly, but there are specific issues facing behavioral health.

For example, although accounting for patients’ preferences cuts across all of health, applying performance measures to populations that may have impaired decision-making capacity or may be receiving treatment involuntarily is more complex. Risk-adjustment methodologies also need to be adapted specifically for measuring quality in this arena.

The care of people with these disorders often involves much interaction among multiple organizations responsible for particular domains of care. These include mental health, substance abuse, primary, and specialty general health care and social and vocational services, housing, and criminal justice. How should accountability across these entities be established, given the interdependence of, for example, housing and employment outcomes with symptoms, quality of life, and functional status?

INTEGRATING MENTAL HEALTH CARE INTO GENERAL HEALTH

Health care for people with mental health and substance use disorders is poorly integrated with primary and specialty general medical care, particularly for those whose care is based in the mental health and substance abuse specialty care sector. Integrated clinical information systems and co-location of services are examples of strategies to link mental health and general medical settings. A variety of delivery models exist for providing more integrated care, but they need to be augmented with a strategy for measuring quality and outcomes that emphasizes mutual accountability for mental health and substance use disorders and for general medical conditions.

INVESTING IN RESEARCH

As noted above, more research and stewardship are needed to develop better measures and methods to improve the quality of care for people with these disorders. But the need for more research goes well beyond the quality measurement domain. The fact is that, as in other areas of medicine, we simply do not have enough evidence of what works best for which groups of patients.

Such questions surrounding the care of these disorders were among the highest priorities identified in a 2009 Institute of Medicine report on comparative effectiveness research.¹⁶ Nonetheless, no clear, coordinated strategy for implementing this comparative effectiveness research agenda has yet emerged.

The Agency for Healthcare Research and Quality has taken the initiative by conducting stake-holder meetings to refine priorities and incorporate mental health into several of its comparative effectiveness initiatives. On the other hand, the National Institutes of Health appear to be shifting more strongly toward a “discovery” research agenda with relatively limited involvement in the application of research findings to quality improvement. And the Substance Abuse and Mental Health Services Administration is limited in the type of knowledge development it can support.

All public sources of research support are being affected by the current economic situation, and private foundations have been moving away from a focus on mental health and substance use disorders. The formation of a new public-private, nonprofit Patient Centered Outcomes Research Institute may hold the promise of creating resources and leadership for filling these evidence gaps. Advances in knowledge from comparative effectiveness research should have direct application for informing the development of clinical practice guidelines and quality measures.

Conclusion

The ability to measure quality stands at the center of improvement efforts and forms the basis for establishing accountability for providing high-quality care. A concerted effort by multiple public and private groups will be needed to bring care for mental health and substance use disorders into the mainstream of quality measurement and improvement. Our intent in proposing an initial framework for measuring the quality of mental health and substance abuse care is to establish a conversation among these groups and a serious commitment to achieving this goal. ▪

Biographies

Harold Alan Pincus is a professor of psychiatry at Columbia University.

Harold Pincus, Brigitta Spaeth-Rublee, and Katherine Watkins argue in this issue of Health Affairs that the fields of mental health and substance abuse services have yet to embrace the quality movement, even as other fields have done so. They propose a short list of quality measures that they hope will serve as a starting point for a broader discussion about improving care in these neglected fields.

“In part,” says Pincus, “it’s a story about the separation of mental health and general medicine,” dating back to Hippocrates and running through the Enlightenment and the birth of psychoanalysis around the turn of the twentieth century. Even today, this separation is perpetuated by managed care, which generally “carves out” mental health to be managed by a separate organization. But, in Pincus’s view, the problem is also rooted in the nature of behavioral disorders, which are more difficult than physical disorders to define and are often not readily diagnosed.

Pincus is a professor and vice chair of psychiatry and codirector of the Irving Institute for Clinical and Translational Research at Columbia University. He is also director of quality and outcomes research at New York Presbyterian Hospital and serves as a senior scientist at RAND. His research interests are the practice of evidence-based medicine, quality improvement, and relationships among mental health, general medicine, and substance abuse.

Pincus is the recipient of the William C. Menninger Memorial Award of the American College of Physicians for distinguished contributions to the science of mental health. He received his medical degree from the Albert Einstein College of Medicine.

Brigitta Spaeth-Rublee is a research associate and program coordinator in the Department of Psychiatry, Columbia University.

Spaeth-Rublee is a research associate and program coordinator in Columbia University’s psychiatry department as well as at the Research Foundation for Mental Hygiene, a private, not-for-profit corporation whose goal is to enhance research and training for the New York State Department of Mental Hygiene and its component agencies. Her research interests are in mental health policy and comparative health systems. Spaeth-Rublee received a master’s degree in political science and sociology from the University of Hagen, Germany.

Katherine E. Watkins is a senior natural scientist at the RAND Corporation.

Watkins is a senior natural scientist at RAND. Her research focuses on measuring the quality of mental and substance abuse care and on developing and evaluating treatments for people with cooccurring disorders in both realms. She is the co–principal investigator of a congressionally authorized evaluation of services for US veterans with mental health and substance abuse disorders. She received her medical degree from the University of Pennsylvania and her master’s degree in health services from the University of California, Los Angeles.