Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2006 Jun 5.
Published in final edited form as: J Healthc Qual. 2002;24(3):24–35. doi: 10.1111/j.1945-1474.2002.tb00430.x

A Framework for Selecting Performance Measures for Opioid Treatment Programs

Luc R Pelletier 1,✉, Jeffrey A Hoffman 2
PMCID: PMC1473224  NIHMSID: NIHMS3483  PMID: 14692185

Abstract

As a result of new federal regulations released in early 2001 that move the monitoring and evaluation of opioid treatment programs from a government regulation to an accreditation model, program staff members are now being challenged to develop performance measurement systems that improve care and service. Using measurement selection criteria is the first step in developing a performance measurement system as a component of an overall quality management (QM) strategy. Opioid treatment programs can “leapfrog” the development of such systems by using lessons learned from the healthcare quality industry. This article reviews performance measurement definitions, proposes performance measurement selection criteria, and makes a business case for Internet automation and accessibility. Performance measurement sets that are appropriate for opioid treatment programs are proposed, followed by a discussion on how performance measurement can be used within a comprehensive QM program. It is hoped that through development, adoption, and implementation of such a performance measurement program, treatment for clients and their families will continuously improve.

Keywords: accreditation, CARF, JCAHO, performance measurement, quality improvement

New federal regulations to improve the clinical outcomes in opioid treatment programs were released in January 2001 and implemented in May 2001. The regulations shift responsibility for the monitoring and evaluation of opioid treatment programs from a regulatory to an accreditation model. The need for this change, a review of methadone treatment oversight, a review of the development of accreditation requirements, and a discussion of the public’s response to the new federal regulations, as well as performance domains of importance to opioid treatment programs, have been addressed (Pelletier & Hoffman, 2001). Opioid treatment program staff members are now being challenged to develop performance measurement systems that improve care and service.

Performance Measurement

A system of performance measurement must be a component of an organization’s overall quality strategy. The strategy must conform to healthcare quality, defined as “the degree to which health services for individuals and populations increase the likelihood of desired healthcare outcomes and are consistent with current professional knowledge” (Committee to Design a Strategy for Quality Review and Assurance in Medicare, 1990, p.4). The critical elements of this definition and those that are inextricably linked to it form the basis of a comprehensive program to monitor and measure healthcare quality. These elements are as follows:

  • “health services” (includes processes of care)

  • “desired health outcomes” (includes the clinical consequences of care delivered)

  • “consistent with current professional knowledge” (includes new technologies in the art and science of healthcare as well as in quality management [QM] principles and practices [Palmer, Lawthers, Banks, Peterson, & Caputo, 1996, p. 100]).

In addition, the performance measurement system should conform to an accepted healthcare quality framework. The Institute of Medicine (IOM), in its work on developing a framework for a national report card (Committee on the National Quality Report on Health Care Delivery [CNQRHCD], 2001), developed four components of healthcare quality: safety, effectiveness, patient-centeredness, and timeliness (p.7). These are taken from the six specific aims for healthcare improvement proposed by the IOM Committee on Quality of Health Care in America (CQHCA, 2001). (See Figure 1.)

Figure 1.

Figure 1

Open in a new tab

Institute of Medicine Specific Aims for Healthcare Improvement

In response to increasing market demands, a comprehensive quality improvement (QI) program must include a performance measurement system. A performance measurement system is defined as

“an entity consisting of an automated database that facilitates performance improvement in healthcare organizations through the dissemination and collection of process and/or outcome measures of performance. Measurement systems must be able to generate internal comparisons of organization performance over time, and external comparisons of performance among participating organizations at a comparable time” (Joint Commission on Accreditation of Healthcare Organizations [JCAHO], 1998).

This definition is broader than the IOM definition of performance measurement, which relates only to “the process of care” (National Roundtable on Health Care Quality, 1999, pp. 7–8). On a community level, the Committee on Using Performance Monitoring to Improve Community Health (CUPMICH) (1997) defines performance monitoring as

a continuing community-based process of selecting indicators that can be used to measure the process and outcomes of an intervention strategy for health improvement, collecting and analyzing data on those indicators, and making the results available to the community to inform assessments of the effectiveness of an intervention and the contributions of accountable entities (p. 4).

Furthermore,

clinical performance measures are tools that assess the delivery of clinical services. Good clinical performance requires providing services that are appropriate for each client’s condition, providing them safely, competently, and in an appropriate time frame, and achieving desired outcomes in terms of those aspects of patient health and patient satisfaction that can be affected by clinical services (Palmer et al., 1996, p. 101).

Performance measurement systems have been developed and implemented in managed care organizations and clinical programs (large- and small group practices, mental health clinics, staff model health maintenance organizations) and by purchasers themselves. The American Managed Behavioral Healthcare Association (1999) recommended that measures be developed that are performance-based and provide some indication of access to care, consumer satisfaction, and quality of care. Treatment staff members can use performance results to monitor and improve outcomes, processes of care, quality of care, accountability, satisfaction and perception, and values determination.

For providers and program administrators, performance measurement systems will provide the opportunity to benchmark with local, state and national norms and best practices and to conduct comparative analyses within and among clinics. Providers will be able to assess the effects of treatment interventions on health-related quality of life. Data will be analyzed to inform clinical care; clinic-level data could inform the development and refinement of the overall treatment program. Furthermore, local, state, and national data could serve as a resource for clinics across the country and for health services researchers interested in methadone treatment.

Over the past 10 years, payers have increasingly demanded more accountability and evidence of positive clinical outcomes. Various stakeholders, including insurance companies, the federal government (Medicaid and Medicare), and businesses in the public and private sectors, all want to know that their employees and beneficiaries are getting better and that functional status has improved because of treatment and the outlay of benefit dollars. In addition, they want more information about the effects of treatment (e.g., improvements in general health status, improvements in work performance) on their employees, beyond utilization, costs, and satisfaction with services. Performance measurement plays a critical role in meeting the accountability demands of various stakeholders.

National Performance Measurement Selection Criteria

In developing a comprehensive performance measurement system, it is critical to incorporate national and international selection criteria standards. A number of selection criteria guideline statements have been developed over the past few years, including statements from the following:

The performance measurement attributes common to the above entities’ guideline statements have been reported in the set of criteria proposed to be used for a national healthcare quality report (CNQRHCD, 2001, p. 81). Common performance measurement selection criteria are listed in Figure 2. The adoption of these performance measurement selection criteria is the first step in developing a performance measurement system.

Figure 2.

Figure 2

Open in a new tab

Common Performance Measurement Selection Criteria*

A Business Case for Automation and Online Access

Forty percent of all “hits” on the Internet involve consumer searches for health-related information (Shalala, 2000); consumers are hungry for health information that can help them improve their lives. Whereas interpersonal communications traditionally have influenced health beliefs, attitudes, and behavior change (Backer, Rogers, & Sopory, 1992), the Internet is now evolving as a “hybrid channel with persuasive capabilities of interpersonal communication and the broad reach of the mass media.” Furthermore, “Internet-based resources…enable health professionals to [access] an expanding global audience at a relatively low cost” (Cassell, Jackson, & Cheuvront, 1998, p. 77).

Quality programs in many opioid treatment programs consist of paper-driven processes and reports. There are inherent disadvantages in monitoring and measuring quality by relying on paper-based systems. Disadvantages include the following:

  • lack of efficient and standardized data collection methods

  • lack of a database to compare information over time and to benchmarks

  • inability to query a data warehouse to perform ad hoc clinical studies

  • inability to report on variables contained in a data set.

Rosenstein (1999) has made the case for a computerized QM support system, citing it as an investment in preventing costs attributable to waste and errors. As applied to QM and performance measurement decision support systems, the benefits of electronic data capture and transmission are great and may include:

  • real-time results for viewing

  • alerts and reminders (educational and evidence-based prompts)

  • improved data access/availability/accuracy/coding/analysis

  • reduced duplication

  • improved lines of communication/reporting capabilities

  • improved productivity/reduced documentation time

  • standardization/reduced variation (Rosenstein, p. 265).

An Internet-based performance measurement solution would obviate several barriers as experienced, for example, by the developers of the Methadone Treatment Quality Assurance System (MTQAS). In phase II of the project (a feasibility study) it was found that treatment programs in various states did not have the hardware and software necessary to support MTQAS. The National Institute on Drug Abuse provided funds to purchase hardware (Ducharme, Luckey, Ahn, Fulmer, & Graham, 1999).

Online accessibility would allow forms, reports, and other analyses to be immediately available to treatment staff. For those programs not equipped with a computer, arrangements could be made to purchase services from a service bureau to input the data and generate reports.

Budgets

Opioid treatment programs operate on fixed budgets, funded primarily by public sources. Zarkin and Dunlap (1999) studied the potential impact of applying managed care on traditional methadone treatment operations. They reviewed 44 methadone clinics associated with five major organizations in New York. Annual treatment expenses amounted to $73,533,965 for 14,941 patient slots. The average annual cost per patient was $5,567; on average, Medicaid funded 62% of the costs. The authors, after interviewing the staff at various sites, concluded that

If integrating methadone treatment into a managed care system results in increased administrative burden [review processes such as prior treatment authorizations and utilization reviews, both of which require extensive record-keeping], then programs believe that patient care will be adversely affected. Most programs, realizing that they probably will not be given any additional funding to cover the additional administrative expense, predict that counselors would devote more time to record-keeping and less time to actual treatment (Zarkin & Dunlap, p. 32).

Staff Concerns

These findings, which may be characteristic of a growing national concern about new financing systems and their impact upon administrative burden and treatment, may affect whether treatment program staff embrace or resist a performance measurement system. Administrative burden is always an issue when quality or performance programs are introduced. Staff members commonly react by fearing more paperwork, rather than by entering into a process improvement mindset that could potentially free them to spend even more time with their clients. Such concerns might be quelled by the creation of work groups that would incorporate (embed) performance measures into current hard copy and electronic documentation systems. A cultural shift typically is required to allow staff the opportunity to embrace the idea of decision making based upon performance measurement data (evidence) versus usual documentation and anecdotal systems. Fear generally decreases as staff members recognize the benefits of analyzing objective data, helping them to care for their clients better and more efficiently.

Positive Outcomes

Implementation of monitoring and evaluation activities typically results in eventual acceptance of the measurement process. Organizations that embrace a continuous quality improvement (CQI) process eventually begin to take measurement seriously and to consider how they can improve their treatment services. Such was the case in Oregon after a statewide quality monitoring process had been implemented in conjunction with a managed care strategy (Petillo, 2000):

People would say, ‘If you’re gonna monitor somebody, isn’t that gonna create more resistance?’ I think not; it’s gonna cause less resistance. People know where they stand, and there’s a competitive spirit about wanting to improve their ranking (p. 60).

This monitoring system further proved that completion of substance abuse treatment (when compared with data on patients who had not completed treatment) was associated with the following positive societal outcomes:

  • substantially fewer incarcerations in the state prison system, with fewer days of incarceration (those who completed residential treatment were incarcerated at a rate 70% lower than that of the other group)

  • higher wages in the period subsequent to treatment (those who completed treatment received 65% higher wages due to improvement in earning power and greater number of weeks worked)

  • significantly fewer food stamps used (those who completed treatment used only one-third the number of food stamps used by the other group)

  • decreased child welfare cases (the number of child welfare cases was reduced by 50% subsequent to treatment completion)

  • substantially lower medical expenses (those who did not complete treatment showed a dramatic increase in the use of hospital emergency rooms during the period following treatment, compared with those who completed treatment) (Petillo, 2000, p.63–64).

A performance measurement system would provide program staff with easy-to-use tools for measuring and monitoring performance within their organizations. Developed by using Internet technology, such systems could maximize organizations’ ability to show evidence of the efficacy of their treatment programs and to expend minimal financial resources in the form of typical hardware and software costs.

Performance Measurement Sets for Opioid Treatment

Performance measurement has various uses within a comprehensive quality strategy. These include internal quality or performance improvement and responsiveness to accreditation organizations.

Internal Quality Improvement

A process of continuous appraisal and self-analysis, using objective tools of measurement, is a characteristic of a progressive, consumer-centric organization. Although many programs and organizations develop programs to meet external regulatory standards, organizations that wanted to distinguish themselves in the marketplace in the latter part of the 20th century and early 21st century developed continuous improvement programs based upon quality as a core value. Organizations use data gathered by their performance/CQI programs to develop improvement opportunity plans that enhance the care and services delivered to individuals and populations.

Compliance with Accreditation Standards

Accreditation programs publish specific standards related to performance measurement and the development and maintenance of formal quality strategies and initiatives. Typically, accreditation standards do not proscribe the specific measures or tools to be used, but do identify important measurement domains. Standards emphasize components (including structure, process, outcomes indicators, and an improvement mechanism) of a comprehensive quality strategy. Over the past few accreditation cycles, consumers’ input and utilization of valid, reliable measures and tools to monitor and improve healthcare quality have been emphasized. There is a national movement to describe and define “an initial core set of performance measures, for the nation’s general acute care hospitals, and an enduring framework and process for updating such measures” (National Quality Forum, 2001, p. 1). The Joint Commission has established standardized, core measure sets for hospitals to use with the following health conditions: acute myocardial infarction, heart failure, community-acquired pneumonia, and pregnancy and related conditions (JCAHO, 2001). It is likely that standard measure sets eventually will be defined for the opioid population.

Recommended Performance Measurement Set Descriptions

Using the selection criteria described in Figure 2, we proposed tools for opioid treatment programs as a “menu” of valid, reliable performance measures to use as part of their overall quality strategy. It is critical that treatment programs not create new measures, but instead draw upon existing measures that have been developed based upon sound evidence and that have been tested for reliability and validity. The recommended performance measurement sets and their respective performance domains are summarized in Table 1; Figure 3 outlines the performance measurement set descriptions.

Table 1.

Recommended Opioid Performance Measurement Sets by Performance Domain

Performance Measurement Domain MTQAS Addiction Severity Index Treatment Services Review Family Burden Interview/Short Form (SF) Schedule SF-12
Caregiver burden X
Clinical status X X X
Drug use X X X X
Employment status X X X
Functional status (mental/emotional) X X X X
Functional status (physical) X X
Healthcare utilization (including self-help and community services) X X X
Legal status X X X
Mental/emotional health status X X X X
Physical health status X X X X
Satisfaction X X
Social support X X X X
Open in a new tab

Figure 3.

Figure 3

Figure 3

Figure 3

Figure 3

Open in a new tab

Performance Measurement Set Descriptions

Performance Measurement for Quality Improvement

A formal, comprehensive quality program must first be in place to support the implementation and integration of a performance measurement system into an organization’s QM strategy. The overall quality strategy must include the following components:

  • an organizational culture that supports (through human capital and resources) and values a continuous improvement process

  • adequate resources to support the planned activities of the quality program

  • automation of documentation, data collection, and reporting and analysis of performance results

  • integration of quality processes into the daily work of staff

  • hardware and software that support the process

  • continuous training in current and emerging quality measurement technologies (National Community Mental Healthcare Council [NCMHC], 1995).

Organizational Culture

Quality must be valued by an organization for its performance improvement activities to succeed. NCMHC states that organizational culture is a critical component of an organization’s readiness to measure outcomes, and that it should include “evaluation as a strategy to outline priorities for improvement and to strengthen components that are working well” (NCMHC, 1995, p. 17). Furthermore, Bader (2000) proposes that quality should be elevated “to the status of finance and strategy on the board [of director’s] agenda” (p. 12).

Adequate Resources

Adequate human resources must be available to support the quality effort. There must be staff who are fully dedicated to quality, as well as other personnel who give time to support the program’s efforts (e.g., in project and committee work). It is preferred that quality professionals be certified in healthcare quality. Staff who are assigned accountability for QM or process improvement programs should possess knowledge, skills, and abilities in the following areas:

  • management and leadership (strategic, operational)

  • information management (design and data collection, analysis, interpretation)

  • education, training, and communication

  • performance measurement and improvement (planning, implementation, evaluation, integration) (Healthcare Quality Certification Board, 2001, insert).

Resources also include dedicated funding for training and in-service education for quality professionals and other staff who have a critical role in deploying the quality program and, more specifically, in implementing the performance measurement system.

Automation and Integration of Quality Processes

Clinical staff members in various treatment settings continue to be bombarded with requests for various data and information. Health systems in the past 20 years have attempted to help staff by developing and implementing automated systems to capture and report on various data. It comes as no surprise, therefore, that automation also should be applied to the performance measurement and improvement process. The purpose of the performance measurement system is to provide staff with easy, immediate access to the tools that they require in their quality and process improvement activities. It is possible that Web-based tools might take less time to complete than paper forms. They are less of a burden because electronic tools are programmed to ask only those questions pertinent to the particular client. Subcategories appear onscreen only if a client/staff member responds in the affirmative to a key question.

When confronted with a new process, clinical staff tend to have concerns about how this activity, which usually is administrative in nature, will affect the time they have to care for clients. As much as possible, performance measurement and quality improvement data elements should be embedded into current data collection activities. An integrated information system, in which financial, clinical, and quality information reside in the same place, makes the process of performance measurement, reporting, and improvement an easy task. Unfortunately, many organizations have been confronted with various legacy systems that do not have the built-in functionality to integrate the necessary data.

Hardware and Software Support

Hardware and software configurations must support the work of clinical staff and enhance the collection of important data to show evidence of the treatment process and its efficacy. As previously described, the developers of MTQAS found that the clinics they were working with did not have the hardware and software to support the collection of important performance measurement data. States had to help local programs to purchase equipment.

Continuous Training and Emerging Technologies

Training is a component of system implementation that typically receives inadequate resources. The performance measurement system will have to include collateral tutorials on such topics as quality management and improvement, performance measurement, quality improvement tools (e.g., Pareto charts, cause-and-effect diagrams), and data management (collection, analysis, and reporting). New employees and new process improvement team members will need to have access to just-in-time training, which is more effective for staff who have limited time away from direct care activities.

The information revolution continues to broaden our access to information and technology. As new systems of communication and interaction emerge, the performance measurement system will have to exploit these technologies (e.g., personal digital assistants, wireless technologies) to provide enhanced support for performance measurement and improvement activities.

Applying Performance Measurement to Improve Care and Services

The performance measurement system described here will help program staff meet or exceed accreditation standards. Table 2 is a summary of accreditation standards and recommended performance measurement sets to meet these standards.

Table 2.

Accreditation Standards and Their Opioid Performance Measurement System Components

Accrediting Bodies/Standards Performance Measurement System Components That Will Enable Program Staff to Meet or Exceed Accreditation Standards
Center for Substance Abuse Treatment
Measurement and monitoring of treatment processes and outcomes, including the following:
1. Reducing or eliminating associated criminal activities 1. MTQAS, ASI, and TSR measurement sets track legal status.
Using these measure sets, reports, tutorials, QI tools, and performance improvement techniques, program staff will be able to monitor outcomes related to legal status (incarcerations, illegal/criminal activities) and develop strategies to decrease illegal activities.
2. Reducing behaviors contributing to the spread of infectious diseases 2. MTQAS, ASI, TSR, and SF-12 measurement sets track physical health status (including injection drug use for high-risk patients). Using these measurement sets, reports, tutorials, QI tools, and performance improvement techniques, program staff will be able to monitor and decrease risk of infectious diseases among populations treated.
3. Improving quality of life by restoration of physical and mental health and functional status (CSAT, 1999) 3. MTQAS, ASI, FBIS, and SF-12 measurement sets track functional status (physical and mental) and physical and mental/emotional status. As above, using these measurement sets, reports, tutorials, QI tools, and performance improvement techniques, program staff will be able to monitor and improve quality of life for individuals and populations.
The Rehabilitation Accreditation Commission—CARF
1. Quality and appropriateness of services 1. The performance measurement system described herein can become a part of a formal, comprehensive quality strategy for an organization. Using the data gleaned from the measurement sets to improve care and service, the program will have evidence of a systematic monitoring and evaluation process.
The organization has a systematic monitoring and evaluation process related to the ongoing measurement of quality, appropriateness, and utilization of services.
2. Outcomes management 2. Measurement sets capture outcome information. With these data and the tools and techniques described in this article, programs will have evidence of an outcomes management system and examples of improving care and service for individuals and populations served.

  • A formal outcomes management system measures program effectiveness (quality of life, symptomatology, functional status, health status), efficiency (access, use appropriateness, cost), and customer satisfaction (CARF, 1999)
Joint Commission on Accreditation of Healthcare Organizations
Outcomes and processes such as the following should be measured and monitored: The measurement sets described herein track the outcomes as described in the Joint Commission’s opioid accreditation standards. Once these data elements are collected, reports will be generated to inform the identification of improvement opportunities. Program staff will then be able to use tutorials, QI tools, and performance improvement techniques to improve care and services to individuals and populations.

  • reducing and eliminating use of illicit opioids and illicit drugs and the problematic use of licit drugs

  • reducing or eliminating associated criminal activity

  • reducing behaviors contributing to the spread of infectious diseases; improving quality of life by restoration of physical and mental health and functional status (JCAHO, 1999a)
Open in a new tab

Summary

A comprehensive performance measurement system for opioid treatment programs should include a minimal set of integrated components. These (preferably automated) components would be available for use by program staff and would match the level of resources and capabilities available for CQI activities.

The steps that an opioid treatment program can take in developing a performance measurement system include the following:

  1. determine performance measurement selection criteria

  2. apply selection criteria to potential measurement sets

  3. automate the performance measurement tools (as much as possible, embed measures into current documentation systems) and consider Internet technologies

  4. incorporate performance measurement data and findings into a CQI cycle.

A comprehensive performance measurement system that is specific to the population being treated in an opioid treatment program potentially can make a difference in the care and services provided to clients and their families. It is critical that treatment programs not create new performance measures, but instead draw upon measures that have been developed based upon evidence and that have been scientifically tested in the field. Also critical is the application of selection criteria to potential measures and measurement sets. These processes conform to current national initiatives such as the National Quality Forum hospital performance measures project. Built upon scientific evidence, these programs have the potential for improving the overall quality of care and service to clients, families, and populations.

Acknowledgments

This article was part of background research for the Performance Measurement for Treatment Improvement project, which was funded under a Small Business Innovation Research contract, Phase I, from the National Institute on Drug Abuse (NIDA), Contract No. IR43 DA 13067-01. The authors thank Jerry Flanzer, DSW, NIDA Project Officer, for supporting this project. The opinions expressed by the authors do not necessarily reflect the official policies or positions of the NIDA or of any other part of the U.S. Department of Health and Human Services.

Contributor Information

Luc R. Pelletier, Luc R. Pelletier is a healthcare consultant in Washington, DC, and editor-in-chief of JHQ. He was project director for the Performance Measurement for Treatment Improvement Project at Danya International, Inc., where this work was originally produced..

Jeffrey A. Hoffman, Jeffrey A. Hoffman, PhD, is president and CEO of Danya International, Inc. Dr. Hoffman served as principal investigator for this project..

References

  1. American Managed Behavioral Healthcare Association and the National Association of State Alcohol and Drug Abuse Directors. Public substance abuse and managed behavioral healthcare. Behavioral Healthcare Tomorrow. 1999;8(1):25–30. [PubMed] [Google Scholar]
  2. Backer, T.E., Rogers, E.M., & Sopory, P. (1992). Designing health communication campaigns: What works? Newbury Park, CA: Sage.
  3. Bader, B. (2000). Quality begins in the boardroom. Modern Healthcare, 30(3), 26. Retrieved March 7, 2002, from ebscohost database. [PubMed]
  4. Cassell MM, Jackson C, Cheuvront B. Health communication on the Internet: An effective channel for health behavior change? Journal of Health Communication. 1998;3:71–79. doi: 10.1080/108107398127517. [DOI] [PubMed] [Google Scholar]
  5. Center for Substance Abuse Treatment (CSAT). (1999). CSAT guidelines for the accreditation of opioid treatment programs Retrieved April 16, 2002, from http://www.samhsa.gov/centers/CSAT/opat/guidelnsx.pdf
  6. Chassin MR. Assessing strategies for quality improvement. Health Affairs. 1997;16:151–161. doi: 10.1377/hlthaff.16.3.151. [DOI] [PubMed] [Google Scholar]
  7. Committee to Design a Strategy for Quality Review and Assurance in Medicare, Institute of Medicine. (1990). Medicare: A strategy for quality assurance, Vol. 1 (K. Lohr, Ed.). Washington, DC: National Academy Press.
  8. Committee on Federal Regulations of Methadone Treatment, Institute of Medicine. (1995). Federal regulation of methadone treatment (R.A. Rettig & A. Yarmolinsky, Eds.). Washington, DC: National Academy Press.
  9. Committee on Leading Health Indicators for Health People 2010, Institute of Medicine. (1999). Leading health indicators for Healthy People 2010: Final Report (C.A. Chrvala & R.J. Bulger, Eds.). Washington, DC: National Academy Press. [PubMed]
  10. Committee on the National Quality Report on Health Care Delivery (CNQRHCD), Board on Health Care Services. (2001). Envisioning the national health care quality report. (M.P. Hurtado, E.K. Swift, & J.M. Corrigan, Eds.). Washington, DC: National Academy Press.
  11. Committee on Quality of Health Care in America (CQHCA), Institute of Medicine. (2001). Crossing the quality chasm: A new health system for the 21st century Washington, DC: National Academy Press.
  12. Committee on Using Performance Monitoring to Improve Community Health, Institute of Medicine. (1997). Improving health in the community: A role for performance monitoring (J.S. Durch, L.A. Bailey, & M.A. Stoto, Eds.) Washington, DC: National Academy Press.
  13. Ducharme, L.J., Luckey, J.W., Ahn, R., Fulmer, E.B., & Graham, L.A. (1999). Methadone treatment quality assurance system: A feasibility study—final report Research Triangle Park, NC: Research Triangle Institute.
  14. Foundation for Accountability. (2002). Supporting quality-based decisions. Retrieved March 7, 2002, from www.facct.org/
  15. Healthcare Quality Certification Board. (2001). Examination information [Online]. Retrieved February 5, 2002, from http://www.cphq-hqcb.org/examin.htm
  16. Joint Commission on Accreditation of Healthcare Organizations. (1998). Glossary of terms for performance measurement systems [Online]. Retrieved February 5, 2002, from www.jcaho.org/perfmeas/glossry.html
  17. Joint Commission on Accreditation of Healthcare Organizations. (1999). Attributes of core performance measures and associated evaluation criteria [Online]. Retrieved February 5, 2002, from www.jcaho.org/perfmeas/coremeas/cm_att.html
  18. Joint Commission on Accreditation of Healthcare Organizations. (1999a). Revisions to the comprehensive accreditation manual for behavioral healthcare intent statement for SAMHSA. Retrieved April 16, 2002, from http://www.samhsa.gov/centers/CSAT/content/opat/methstandards.pdf
  19. Joint Commission on Accreditation of Healthcare Organizations. (2001). Abstract: Facts about core measures for ORYX. [Online]. Retrieved February 5, 2002, from www.jcaho.org/perfmeas/coremeas/cm_facts.html
  20. McLellan AT, Alterman AL, Woody GE, Metzger D. A quantitative measure of substance abuse treatments: The treatment services review. Journal of Nervous and Mental Disorders. 1992;181:100–109. doi: 10.1097/00005053-199202000-00007. [DOI] [PubMed] [Google Scholar]
  21. McLellan AT, Arndt IO, Woody GF, Metzger D. Psychosocial services in substance abuse treatment? A dose-ranging study of psychosocial services. Journal of the American Medical Association. 1993;269:1953–1959. [PubMed] [Google Scholar]
  22. McLellan, A.T., Cacciola, J.S., & Fureman, I. (1996). The Addiction Severity Index (ASI) and Treatment Services Review (TSR). In L.I. Sederer and B. Dickey (Eds.) Outcomes Assessment in Clinical Practice Baltimore: Williams & Wilkins.
  23. McLellan AT, Cacciola J, Kushner H, Peters R, Smith I, Pettinati H. The fifth edition of the Addiction Severity Index: Cautions, additions, and normative data. Journal of Substance Abuse Treatment. 1992;9:461–480. doi: 10.1016/0740-5472(92)90062-s. [DOI] [PubMed] [Google Scholar]
  24. McLellan AT, Luborsky L, Cacciola J, Griffith J. New data from the Addiction Severity Index: Reliability and validity in three centers. Journal of Nervous and Mental Disorders. 1985;173:412–423. doi: 10.1097/00005053-198507000-00005. [DOI] [PubMed] [Google Scholar]
  25. National Committee for Quality Assurance. (2000). HEDIS 2001, Vol. 1: Desirable attributes of HEDIS measures [Online]. Retrieved June 27, 2001, from www.ncqa.org/pages/programs/hedis/desirable%20attributes.html
  26. National Community Mental Healthcare Council (NCMHC). (1995). Behavioral healthcare outcomes: A reference guide to measurement tools Rockville, MD: Author.
  27. National Quality Forum. (2001). Standardizing measures of hospital quality. [Online]. Retrieved March 8, 2002, from www.qualityforum.org/txNQFprojectsummaryhospitals.pdf
  28. National Roundtable on Health Care Quality, Institute of Medicine. (1999). Measuring the quality of healthcare: A statement by the National Roundtable on Health Care Quality. (M. Donaldson, Ed.). Washington, DC: National Academy Press.
  29. Palmer, R.H., Lawthers, A.G., Banks, N.J., Peterson, L.E., & Caputo, G.L. (1996). Conquest 1.1: A computerized needs-oriented quality measurement evaluation system Boston: Center for Quality of Care Research and Education, Harvard School of Public Health.
  30. Palmer S, Torgerson DJ. Definition of efficiency. British Medical Journal. 1999;318:1136. doi: 10.1136/bmj.318.7191.1136. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Pelletier LR, Hoffman JA. From regulation to accreditation: New federal regulations for improving quality in opioid treatment programs. Journal for Healthcare Quality. 2001;23(6):29–34. [PubMed] [Google Scholar]
  32. Petillo, J. (2000). Management information systems for integrated public health and Medicaid substance abuse treatment programs: Case studies in three states. (DHHS Publication No. SMA 00-3365). Rockville, MD: Substance Abuse and Mental Health Services Administration, Center for Substance Abuse Treatment.
  33. President’s Advisory Commission on Consumer Protection and Quality in the Health Care Industry. (1998). Quality first: Better health care for all Americans Washington, DC: Government Printing Office.
  34. Quality Metric, Inc. (2000). The SF–12: An even shorter health survey [Online]. Retrieved February 6, 2002, from www.sf-36.com/tools/sf12.shtm
  35. The Rehabilitation Accreditation Commission—CARF. (1999). CARF opioid treatment program standards Retrieved April 16, 2002 from http://www.samhsa.gov/centers/CSAT/content/opat/guideline.pdf
  36. Rosenstein AH. Measuring the benefit of performance improvement and decision support. American Journal of Medical Quality. 1999;14:262–269. doi: 10.1177/106286069901400606. [DOI] [PubMed] [Google Scholar]
  37. Shalala, D.E. (2000, January 25). Press briefing at the Healthy People 2010: Partnerships for Health in a New Millennium conference, Washington, DC.
  38. Smith GR, Manderscheid RW, Flynn LM, Steinwachs DM. Principles of assessment for patient outcomes in mental health care. Psychiatric Services. 1997;48:1033–1036. doi: 10.1176/ps.48.8.1033. [DOI] [PubMed] [Google Scholar]
  39. Tessler, R.C., & Gamanche, G.M. (1995). Toolkit for evaluating family experiences with severe mental illness Cambridge, MA: Human Services Research Institute.
  40. Tessler, R.C., & Gamanche, G.M. (1996). The family burden interview schedule—short form (FBIS/SF). In L.I. Sederer & B. Dickey (Eds.), Outcomes assessment in clinical practice (pp. 110–112; 258–263). Baltimore: Williams & Wilkins.
  41. U.K. Department of Health. (2002). NHS performance indicators: February 2002. London: Author. Retrieved March 7, 2002, from www.doh.gov.uk/nhsperformanceindicators/2002/index.html
  42. Ware, J.E. (1996). The MOS 36-item short-form health survey (SF-36). In L.I. Sederer & B. Dickey, Eds., Outcomes assessment in clinical practice (pp. 61–64). Baltimore: Williams & Wilkins.
  43. Ware JE, Kosinski M, Keller SD. A 12-item short-form health survey: Construction of scales and preliminary tests of reliability and validity. Medical Care. 1996;34:220–233. doi: 10.1097/00005650-199603000-00003. [DOI] [PubMed] [Google Scholar]
  44. Zarkin GA, Dunlap LJ. Implications of managed care for methadone treatment: Findings from five case studies in New York state. Journal of Substance Abuse Treatment. 1999;17(1–2):25–35. doi: 10.1016/s0740-5472(98)00062-2. [DOI] [PubMed] [Google Scholar]

RESOURCES