Specifying and Pilot Testing Quality Measures for the American Society of Addiction Medicine's Standards of Care

Abstract

Objectives

In 2013, American Society of Addiction Medicine (ASAM) approved their Standards of Care for the Addiction Specialist Physician. Subsequently, an ASAM Performance Measures Panel identified and prioritized the standards to be operationalized into performance measures. The goal of this paper is to describe the process of operationalizing three of these standards into quality measures, and to present the initial measure specifications and results of pilot testing these measures in a large health care system. By presenting the process rather than just the end results, we hope to shed light on the measure development process in order to educate, as well as stimulate debate about the decisions that were made.

Methods

Each measure was decomposed into major concepts. Then each concept was operationalized using commonly available administrative data sources. Alternative specifications examined and sensitivity analyses were conducted to inform decisions that balanced accuracy, clinical nuance, and simplicity. Using data from the US Veterans Health Administration (VHA), overall performance and variation in performance across 119 VHA facilities were calculated.

Results

Three measures were operationalized and pilot tested: pharmacotherapy for alcohol use disorder, pharmacotherapy for opioid use disorder, and timely follow-up after medically managed withdrawal (aka detoxification). Each measure was calculable with available data, showed ample room for improvement (no ceiling effects) and wide facility-level variability.

Conclusions

Next steps include conducting feasibility and pilot testing in other health care systems and other contexts such as standalone addiction treatment programs, as well as to study the specification and predictive validity of these measures.

Introduction

To help health care systems, medical specialty certification boards, quality managers, and individual physicians monitor quality and performance, as well as to support quality improvement efforts, the Board of Directors of the American Society of Addiction Medicine (ASAM) convened a Practice Improvement and Performance Measurement Action Group (PIPMAG). PIPMAG was comprised of a Steering Committee, a Standards and Outcomes of Care Panel, and a Performance Measures Panel. Using a consensus process, the Standards Expert Panel developed, and the ASAM Board of Directors approved, the Standards of Care for the Addiction Specialist Physician (The Standards) in 2013. (American Society of Addiction Medicine, 2014) The Standards outline the minimum aspects of care that an addiction specialist physician should consider for, and when appropriate provide to, a person with an addictive disorder, especially substance use disorder (SUD). Then, the Performance Measures Panel, comprised of individuals conducting research and clinical work in diverse settings, was tasked with determining which of the standards could feasibly be operationalized into performance measures using administrative data, which should be prioritized for specification and pilot testing, and to identify areas for future research and development.

The report from the Performance Measures Panel describes its process as well the rationale for the measures selected for initial specification and pilot testing.(American Society of Addiction Medicine, 2015) The mission of the Performance Measures Panel was to propose and develop measures that would operationalize elements of The Standards, and to prioritize measures that would be most likely to improve patients’ health, and promote high-quality, cost-efficient health care for addictive disorders. Thus, initial measure selection was based on a number of factors, based on criteria used by the National Quality Forum, and used by other measure developers (National Quality Forum, 2013; Thomas et al., 2011): clinical importance, scientific evidence, reliability and validity, usability, risk of unintended consequences, and, as is always the case with performance measure development, feasibility given available data. Although the initial intention was to develop physician-level measures, the challenges of operationalizing such measures (e.g., small numbers of patients, assigning patients to individual physicians vs. care team or systems of care), and the importance and influence of organizational and payer structures led the Performance Measures Panel to scope the measures to settings (e.g., program, facility or system-level) that usually have more than one clinician or provider.

Of the nine standards that were recommended for measure development (see report for details), three were identified as highest priority for measure specification and pilot testing, primarily based on the existence of supporting scientific evidence, earlier models for measurement and specification, and the availability of relevant data elements.

Measure 1

Percent of patients receiving a medication for alcohol use disorder (AUD).

Measure 2

Percent of patients receiving a medication for opioid use disorder (OUD).

Measure 3

Percent of withdrawal management episodes with outpatient follow-up within 7 days

Although not often appreciated, specifying quality measures is a process that involves making scores of more or less satisfying decisions about how to operationalize the major concepts given available data. Each measurement concept needs to be decomposed into elemental concepts, which then need to be operationalized using data that often exist for billing rather than quality measurement purposes. The goal of this paper is to describe the decision making process the Performance Measures Panel undertook in operationalizing and pilot testing these three measures, as well as to present the initial measure specifications. By presenting the process rather than just the end result, we hope to shed light on the “black box” of measure development in order to educate, as well as stimulate debate about the decisions that were made. This study protocol was approved by the Institutional Review Board of Stanford University and the VA Palo Alto Healthcare System.

Methods and Results

Specifying Measure 1: Percent of patients prescribed a medication for alcohol use disorder (AUD)

The two major concepts for this measure are “AUD medications” and “Patients with AUD”. The Performance Measures Panel operationalized all major concepts with International Classification of Disease – Version 9 (ICD-9) and ICD-10 diagnostic codes, as well as Common Procedure Terminology (CPT), Healthcare Common Procedure Coding System (HCPCS), and ICD-9 CM and ICD-10 CM procedure codes. Other concepts and decisions include the timing and sequence of the diagnosis and medication, medication persistence, and lead-in and follow-up period durations. We describe each below.

Patients with AUD

The FDA has approved four medications for the treatment of alcohol dependence rather than all alcohol use disorders, including abuse. However the abuse/dependence distinction has been abandoned in the American Psychiatric Association Diagnostic and statistical manual of mental disorders (DSM-5)(American Psychiatric Association, 2013). Thus, the Performance Measures Panel included all ICD alcohol abuse and dependence codes, except in-remission diagnoses. The rationale for excluding in-remission codes is that those are intended to signify that a patient's symptoms are not in need of current treatment. (American Psychiatric Association, 2013) In addition to the abuse and dependence codes, the Performance Measures Panel also included alcohol-related medical condition codes, such as ICD-9 291.98 Alcohol Related Sleep Disorders and 425.5 Alcoholic Cardiomyopathy, if there was not an AUD in-remission code recorded on the same day. The rationale for including these codes is that they would rarely be used in the absence of a diagnosable, but perhaps undiagnosed, AUD. Future validation work should confirm this assumption.

Finally, it is not uncommon for patients to receive medications for AUD without having an AUD diagnosis documented in the medical record. The Performance Measures Panel deliberated whether to include undiagnosed patients in the denominator (and numerator) if they were receiving AUD medications, but ultimately decided to only include patients who had a qualifying AUD diagnosis documented in the medical record. The rationale for this decision was two-fold: First, high quality care includes both documenting and treating disorders, not just treating them. Second, naltrexone and topiramate in particular have other indications, thus including patients without a documented AUD might erroneously include many patients without disordered drinking. Tables 1 and 2 present the ICD-9 and ICD-10 codes that have been included and excluded based on these decisions.

Table 1.

Specifications of Major Concepts using ICD-9

Concept	ICD-9 Codes Included		ICD-9 Codes Excluded
Alcohol Use Disorder	303	Acute alcoholic intoxication in alcoholism, unspecified	303.03	Acute alcoholic intoxication in alcoholism, in remission
	303.01	Acute alcoholic intoxication in alcoholism, continuous
	303.02	Acute alcoholic intoxication in alcoholism, episodic	303.93	Other and unspecified alcohol dependence, in remission
	303.9	Other and unspecified alcohol dependence, unspecified	305.03	Alcohol abuse, in remission
	303.91	Other and unspecified alcohol dependence, continuous
	303.92	Other and unspecified alcohol dependence, episodic	V11.3	History of alcoholism
	305	Alcohol abuse, unspecified
	305.01	Alcohol abuse, continuous
	305.02	Alcohol abuse, episodic
	291	Alcohol withdrawal delirium
	291.1	Alcohol-induced persisting amnestic disorder
	291.2	Alcohol-induced persisting dementia
	291.3	Alcohol-induced psychotic disorder with hallucinations
	291.5	Alcohol-induced psychotic disorder with delusions
	291.81	Alcohol withdrawal
	291.82	Alcohol induced sleep disorders
	291.89	Other alcohol-induced mental disorders
	291.9	Unspecified alcohol-induced mental disorders
	357.5	Alcoholic polyneuropathy
	425.5	Alcoholic cardiomyopathy
	535.30	Alcoholic gastritis, without mention of hemorrhage
	535.31	Alcoholic gastritis, with hemorrhage
	571.0	Alcoholic fatty liver
	571.1	Acute alcoholic hepatitis
	571.2	Alcoholic cirrhosis of liver
	571.3	Alcoholic liver damage, unspecified
Medications for AUD	Naltrexone (oral and IM)
	Acamprosate
	Disulfiram
	Topiramate
Opioid Use Disorder	304.00	Opioid type dependence, unspecified
	304.01	Opioid type dependence, continuous
	304.02	Opioid type dependence, episodic
	305.50	Opioid abuse, unspecified
	305.51	Opioid abuse, continuous
	305.52	Opioid abuse, episodic
Medications for OUD	Suboxone
	Naltrexone (oral and IM)
	Buprenorphine excluding patches AND IV medications
	Subutex
Detoxification Procedures	94.62	Alcohol detoxification
	94.63	Alcohol rehabilitation and detoxification
	94.65	Drug detoxification
	94.66	Drug rehabilitation and detoxification
	94.68	Combined alcohol and drug detoxification
	94.69	Combined alcohol and drug rehabilitation and detoxification
	H0008	Alcohol and/or drug services; sub-acute detoxification (hospital inpatient)
	H0009	Alcohol and/or drug services; acute detoxification (hospital inpatient)
	H0010	Alcohol and/or drug services; sub-acute detoxification (residential addiction program inpatient)
	H0011	Alcohol and/or drug services; acute detoxification (residential addiction program inpatient)
	H0012	Alcohol and/or drug services; sub-acute detoxification (residential addiction program outpatient)
	H0013	Alcohol and/or drug services; acute detoxification (residential addiction program outpatient)
	H0014	Alcohol and/or drug services; ambulatory detoxification

Table 2.

Specifications of Major Concepts Using ICD-10

Concept	ICD-10 Codes Included		ICD-10 Codes Excluded
Alcohol Use Disorder	F10.229	Alcohol dependence with intoxication, unspecified	F10.21	Alcohol dependence, in remission
	F10.129	Alcohol abuse with intoxication, unspecified	F10.929	Alcohol use, unspecified with intoxication, unspecified
	F10.220	Alcohol dependence with intoxication, uncomplicated
	F10.221	Alcohol dependence with intoxication delirium	F10.920	Alcohol use, unspecified with intoxication, uncomplicated
	F10.20	Alcohol dependence, uncomplicated
	F10.10	Alcohol abuse, uncomplicated	F10.921	Alcohol use, unspecified with intoxication delirium
	F10.230	Alcohol dependence with withdrawal, uncomplicated
	F10.239	Alcohol dependence with withdrawal, unspecified	F10.981	Alcohol use, unspecified with alcohol-induced sexual dysfunction
	F10.231	Alcohol dependence with withdrawal delirium
	F10.232	Alcohol dependence with withdrawal with perceptual disturbance	R78.0	Finding of alcohol in blood
	F10.96	Alcohol use, unspecified with alcohol-induced persisting amnestic disorder	Y90	Evidence of alcohol involvement determined by blood alcohol level
	F10.97	Alcohol use, unspecified with alcohol-induced persisting dementia	T51.0X1A	Toxic effect of ethanol, accidental (unintentional), initial encounter
	F10.27	Alcohol dependence with alcohol-induced persisting dementia	T51.91X1A	Toxic effect of unspecified alcohol, accidental (unintentional), initial encounter
	F10.951	Alcohol use, unspecified with alcohol-induced psychotic disorder with hallucinations
			K099.310-315	Alcohol use complicating pregnancy, childbirth, and the puerperium
	F10.950	Alcohol use, unspecified with alcohol-induced psychotic disorder with delusions
	F10.239	Alcohol dependence with withdrawal, unspecified
	F10.232	Alcohol dependence with withdrawal with perceptual disturbance
	F10.182	Alcohol abuse with alcohol-induced sleep disorder
	F10.282	Alcohol dependence with alcohol-induced sleep disorder
	F10.982	Alcohol use, unspecified with alcohol-induced sleep disorder
	F10.159	Alcohol abuse with alcohol-induced psychotic disorder, unspecified
	F10.180	Alcohol abuse with alcohol-induced anxiety disorder
	F10.181	Alcohol abuse with alcohol-induced sexual dysfunction
	F10.188	Alcohol abuse with other alcohol-induced disorder
	F10.250	Alcohol dependence with alcohol-induced psychotic disorder, with delusions
	F10.251	Alcohol dependence with alcohol-induced psychotic disorder, hallucinations
	F10.259	Alcohol dependence with alcohol-induced psychotic disorder, unspecified
	F10.280	Alcohol dependence with alcohol-induced anxiety disorder
	F10.281	Alcohol dependence with alcohol-induced sexual dysfunction
	F10.288	Alcohol dependence with other alcohol-induced disorder
	F10.29	Alcohol dependence with unspecified alcohol-induced disorder
	F10.994	Alcohol use, unspecified with alcohol-induced mood disorder
	F10.959	Alcohol use, unspecified with alcohol-induced psychotic disorder, unspecified
	F10.980	Alcohol use, unspecified with alcohol-induced anxiety disorder
	F10.988	Alcohol use, unspecified with other alcohol-induced disorder
	F10.19	Alcohol abuse with unspecified alcohol-induced disorder
	F10.120	Alcohol abuse with intoxication, uncomplicated
	F10.121	Alcohol abuse with intoxication delirium
	F10.122	Alcohol abuse with intoxication, unspecified
	F10.14	Alcohol abuse with alcohol-induced mood disorder
	F10.150	Alcohol abuse with alcohol-induced psychotic disorder with delusions
	F10.151	Alcohol abuse with alcohol-induced psychotic disorder, unspecified
	F10.24	Alcohol dependence with alcohol-induced mood disorder
	F10.26	Alcohol dependence with alcohol-induced persisting amnestic disorder
	F10.99	Alcohol use, unspecified with unspecified alcohol-induced disorder
	G62.1	Alcoholic polyneuropathy
	I42.6	Alcoyholic cardiomyopathy
	K29.20	Alcoholic gastritis without bleeding
	K29.21	Alcoholic gastritis with bleeding
	K70.0	Alcoholic fatty liver
	K70.10	Alcoholic hepatitis without ascites
	K70.11	Alcoholic hepatitis with ascites
	K70.2	Alcoholic fibrosis and sclerosis of liver
	K70.30	Alcoholic cirrhosis of liver without ascites
	K70.31	Alcoholic cirrhosis of liver with ascites
	K70.40	Alcoholic hepatic failure without coma
	K70.41	Alcoholic hepatic failure with coma
	K70.9	Alcoholic liver disease, unspecified
	K85.2	Alcohol-induced acute pancreatitis
	K86.0	Alcohol-induced chronic pancreatitis
	E24.4	Alcohol-induced pseudo-Cushing's syndrome
	G31.2	Degeneration of the nervous system due to alcohol
	G72.1	Alcoholic myopathy
Medications for AUD	Naltrexone (oral and IM)
	Acamprosate
	Disulfiram
	Topiramate
Opioid Use Disorder	F11.20	Opioid dependence, uncomplicated	F11.21	Opioid dependence, in remission
	F11.10	Opioid abuse, uncomplicated
	F11.10		F11.922	Opioid use, unspecified with intoxication with perceptual disturbance
	F11.122	Opioid abuse with intoxication with perceptual disturbance
			F11.929	Opioid use, unspecified with intoxication, unspecified
	F11.222	Opioid dependence with intoxication with perceptual disturbance
	F11.120	Opioid abuse with intoxication, uncomplicated
	F11.129	Opioid abuse with intoxication, unspecified
	F11.229	Opioid dependence with intoxication, unspecified
	F11.23	Opioid dependence with withdrawal
	F11.99	Opioid use, unspecified with unspecified opioid-induced disorder
	F11.14	Opioid abuse with opioid-induced mood disorder
	F11.188	Opioid abuse with other opioid-induced disorder
	F11.159	Opioid abuse with opioid-induced psychotic disorder, unspecified
	F11.150	Opioid abuse with opioid-induced psychotic disorder with delusions
	F11.151	Opioid abuse with opioid-induced psychotic disorder with hallucinations
	F11.181	Opioid abuse with opioid-induced sexual dysfunction
	F11.182	Opioid abuse with opioid-induced sleep disorder
	F11.19	Opioid abuse with unspecified opioid-induced disorder
Medications for AUD	Suboxone
	Naltrexone (oral and IM)
	Buprenorphine excluding patches AND IV medications
	Subutex
Detoxification Procedures	HZ2ZZZZ	Detoxification Services for Substance Abuse Treatment

Medications for AUD

Many medications have been studied for the treatment of AUD, many with varied or equivocal evidence of effectiveness. (Jonas et al., 2014; Maisel, Blodgett, Wilbourne, et al, 2013) The Performance Measures Panel strove to develop criteria that could be used not only to select which medications to include in the measure specifications, but also to add or subtract medications as new evidence emerges. The panel decided that medications should be included if they meet at least one of the following criteria: 1) The FDA has approved the medication for AUD (or alcohol dependence); 2) Effectiveness of the medication for AUD is supported by high-quality meta-analytic studies. Using these criteria, the following medications are included in the measure specifications: Naltrexone (oral and injectable), acamprosate, disulfiram, and topiramate. (Blodgett, Del Re, Maisel, et al, 2014; Jonas et al., 2014; Maisel et al., 2013; Skinner, Lahmek, Pham, et al, 2014) The Performance Measures Panel decided that other commonly used or perhaps promising medications such as gabapentin and baclofen, sometimes being used in practice, did not meet these criteria.

Other Measure Design Decisions

In order to operationalize these concepts into a measure, the Performance Measures Panel needed to decide what time period to use for denominator qualification and then what time period to use for the numerator qualification. There are many possibilities with varying complexity, all with dissatisfying aspects. The simplest version includes all patients who had at least one documented AUD diagnosis in any setting (e.g., inpatient, outpatient, primary care, addiction treatment program) any time during a measurement year, and determine which of these qualified patients received at least one of the medications at any time during the measurement year. This version provides a simple, administratively feasible, snapshot of AUD pharmacotherapy access at a program, facility, or system level.

Potential problems with this version are that patients can get the medication before they get the documented diagnosis, and patients who get diagnosed late in the year have less time to receive medications that satisfy the measure criteria. A major test of whether differences in measure specification are consequential is whether the overall level and rank order of performance changes substantially when different specifications are used.(Fernandes-Taylor & Harris, 2012; Harris, Rubinsky, & Hoggatt, 2015) To test some of the alternatives in sensitivity analyses, Performance Measures Panel limited the qualification period to the first 9 months of the measurement year in order to give all patients at least 3 months to receive medication after diagnoses. Another version was tested that only counted medications received after the index diagnosis. Although these more complicated versions predictably shifted the level of measured performance slightly (~1%) lower, the change in relative performance of facilities was minimal. Therefore, the panel decided to adopt the simple version: Proportion of patients who get an AUD diagnoses during the measurement year who also fill a prescription for one of the medications at any time during the measurement year.

The Performance Measures Panel decided to rely on receipt of medication as determined through pharmacy records rather than the provision of a prescription, as data on the latter is not as commonly accessible. Finally, the panel decided to focus on initiation/access rather than persistence/adequate course of medications because lack of access is the most immediate problem. Further, there is no consensus on what constitutes an adequate course of these medications as well as some evidence of efficacy for as-needed (PRN) use, (Heinala et al., 2001) making persistence a more difficult concept to operationalize.

Pilot Testing Measure 1

In order to pilot test these specifications, the measures were calculated using Fiscal Year 2013 (FY13) inpatient and outpatient clinical and pharmacy data from the Veterans Health Administration (VHA). These data cover VHA's 119 major health care systems, each including large medical centers as well as smaller community clinics. The pilot testing was accomplished using the ICD-9 specifications. The target medications were identified using the “Drug Name” variable in VA pharmacy datasets.

In FY13, 356,116 patients had at least one clinical encounter with an AUD diagnosis which qualified them for the measure denominator, of which 21,093 (5.92%) filled at least one prescription for at least one of the medications. Facility-level descriptive statistics for the measure are presented in Table 3. Substantial facility-level variability existed, ranging from 1%-19%.

Table 3.

Facility-Level Descriptive Statistics of Quality Measures within VHA

Measure	Mean	Minimum	25th %tile	50th %tile	75th %tile	Maximum
Pharmacotherapy for AUD (N = 119)	6.05	1.16	4.16	5.48	7.12	19.25
Pharmacotherapy for OUD v1 (N = 119)	21.63	0.23	11.50	19.97	29.73	62.23
Pharmacotherapy for OUD v2 (N = 119)	27.55	0.23	15.38	28.28	38.39	62.96
Follow-up after Detox (N = 116)*	34.67	5.55	25.97	35.29	42.23	59.40

^*excluding facilities with less than 5 detoxification episodes in the measurement year

Other Details and Sensitivity Analyses

Of qualifying patients, 3.8% (13,683) qualified by having an alcohol-related medical condition and no concurrent in-remission diagnosis or other AUD diagnosis, of which 1.50% received medication. Among patients who did not meet the qualifying criteria, 55,912 patients had in-remission AUD codes only (i.e., no active AUD diagnoses) and 1,980 of these in-remission patients (3.5%) received a medication. Furthermore, there were patients who received the medications without a qualifying diagnoses: 118 with Acamprosate; 1352 with oral naltrexone; 521 with disulfiram; 34 with injectable naltrexone; and 49,904 with topiramate. Note that naltrexone and, to a greater extent, topiramate have other indications than AUD. It is more likely that those getting disulfiram and acamprosate are getting treated for AUD without a properly documented diagnosis.

Specifying Measure 2: Percent of patients prescribed a medication for opioid use disorder (OUD)

The two major concepts for this measure are “OUD medications” and “Patients with OUD”. Other concepts and decisions include the timing and sequence of the diagnosis and medication, medication persistence, and lead-in and follow-up period durations. We describe each below.

Patients with OUD

Using the same rationale as Measure 1, the Performance Measures Panel included all OUD's not in-remission. Unlike alcohol, there are no opioid-related mental health or medical conditions to consider. Tables 1 and 2 present the ICD-9 and ICD-10 codes that have been included and excluded based on these decisions.

Medications for OUD

Using the same criteria for inclusion as Measure 1, the performance measure panel determined that three medications are FDA-approved and have meta-analytic support for the treatment of OUD: Buprenorphine (Suboxone, Subutex; excluding buprenorphine patches and IV medications, which are primarily used for pain), naltrexone (oral and injectable), and methadone.

Because methadone for OUD must be dispensed by a licensed Opioid Treatment Program (OTP), and is usually not recorded in the pharmacy data, data elements to capture methadone, if they exist at all, vary from system to system. For example, VA operates many OTPs that generate clinical encounter codes when providing services, even though no specific data exist on the methadone dispensed. When a patient has a recorded encounter in an OTP (clinic stop 523), an OUD diagnose, but not currently receiving buprenorphine or naltrexone, it is assumed they are receiving methadone treatment. However, some VA facilities have no OTP but pay for these services through a contract provider. Patient data on these services are not available, so its looks like patients from these facilities do not have access to methadone. Due to this potential data problem and undercounting of services in many locations, it is even more important than usual to understand the context of the facilities and systems being assessed. In particular, although this measure will be useful for within-system quality assurance and improvement, using this measure for between-system comparisons, especially public reporting, could be very problematic.

Pilot Testing Measure 2

As mentioned, VA provides methadone in OTP's and buprenorphine and naltrexone for OUD in OTP's and other settings. Here we present results from two versions of the measure: Version 1 (v1) that includes naltrexone and buprenorphine but not methadone, and Version 2 (v2) that includes all three medications.

In FY13, 51,655 patients had at least one clinical encounter with an OUD documented, thereby qualifying them for the measure denominator, of which 11,065 (21.8%) filled at least one prescription for naltrexone or buprenorphine (v1), and 16,316 (32.2%) had at least one visit to an OTP with an OUD diagnosis or filled at least one prescription for naltrexone or buprenorphine (v2). Facility-level descriptive statistics for both versions of the measure are presented in Table 3. Substantial facility-level variability existed, ranging from 0.2%-62% for v1 and 0.2%-63% for v2.

Other Details and Sensitivity Analyses

Among patients who did not meet the qualifying criteria, 4,926 patients had in-remission OUD codes only (i.e., no active OUD diagnoses) of which 457 (9.2%) received either buprenorphine or naltrexone. The Performance Measures Panel excluded “Combinations of opioid type drug” diagnosis codes (304.70, 304.71, 304.72). Among the 1,264 patients with these codes and no other active OUD, 106 (8.4%) received medications. Furthermore, there were 799 patients who received buprenorphine without a qualifying diagnoses. There were 1573 patients with a 523 clinic stop without a OUD diagnosis, which we know from previous validation work were mostly patients getting counseling treatment for other addictive disorders, not methadone or other medications for OUD.

Specifying Measure 3: Percent of withdrawal management episodes with outpatient follow-up within 7 days

The major concepts for this measure are “Withdrawal Management Episode”, i.e. detoxification, and “Follow-up”. Other concepts and decisions include the timing and nature of the follow-up. We describe each below.

Withdrawal Management (i.e., Detoxification)

The denominator for this measure is composed of detoxification episodes, which were operationalized using the procedure codes presented in Tables 1 and 2. At first, an attempt was made to develop separate versions of this measure for inpatient and ambulatory detoxification episodes. However, at least in VA data, it was impossible to be certain if a detoxification episode occurred in an inpatient or outpatient setting. For example, the code H0009 (Alcohol and/or drug services; acute detoxification (hospital inpatient)) occurred frequently in the VA outpatient files. For this reason, and for overall simplicity, the Performance Measures Panel decided to construct one combined measure to capture information about follow-up from detoxification episodes regardless of the clinical context (inpatient or outpatient).

An attempt was made to capture information about detoxification episodes that were not coded with one of the procedure codes by looking for short-term courses of medications typically used for medically managed withdrawal. When paired with a diagnosis of AUD or OUD, this method identified about ten percent more patients than the method only using procedure codes. However, due to the added complexity and very cursory validation of this method, it was not included in the measure specifications.

Finally, some patients have what appeared to be, but may not be, multiple closely spaced outpatient detoxification episodes. The panel considered a denominator composed of unique patients rather than episodes to partially address this problem. Because making the distinction between possibly separate detoxification episodes was more difficult in the outpatient setting, the Performance Measures Panel decided to include only the last ambulatory detoxification episode for each patient in the measurement year. Additionally, each inpatient detoxification with a unique discharge day was included. Thus, each patient might contribute at most one outpatient detoxification but perhaps many inpatients detoxifications to the denominator.

Outpatient Follow-up

The main concept for the measure numerator is outpatient follow-up care for SUD within 7 days after detoxification. The Performance Measures Panel entertained various windows besides 7 days for the follow-up to occur, including 3 and 10 days. The general consensus was that 3 days would be clinically more conservative but was perhaps too stringent in some contexts. Pilot testing the different follow-up lengths revealed that although the overall level of performance increased with longer follow-up windows, the relative performance of facilities did not appreciably change. Therefore, the 7-day window was chosen as a compromise and because some evidence exists for the predictive validity of this time period. (Harris et al., 2013) For outpatients, the follow-up window is 7 days starting the day after the last documented detoxification code. For inpatients, the follow-up is 7 days starting the day after discharge.

Clearly, one visit is not adequate treatment for SUD after a detoxification episode, but it is a necessary and critical first step. Completion of the first visit is an indicator that proper coordination and communication may exist. Future work by the Performance Measures Panel will develop a measure of SUD treatment engagement after detoxification.

Even though detoxification is almost always for AUD or OUD, the Performance Measures Panel decided that follow-up would be defined consistent with the HEDIS Initiation and Engagement measures’ specifications for any outpatient SUD treatment.(National Committee for Quality Assurance, 2014) According to the HEDIS specifications, any health care encounter with combinations of specific diagnosis and procedure codes are consider SUD treatment, including encounters outside of SUD programs (e.g., primary care). (Harris et al., 2015; Harris, et al, 2011; National Committee for Quality Assurance, 2014)

Pilot Testing Measure 3

In FY13, 38,514 detoxification episodes were recorded per the measure specifications, of which 876 were outpatient and 37,638 were inpatient services. Overall, 35.3% (13,594) were followed within 7 days with an outpatient SUD treatment encounter per HEDIS specifications. For outpatient and inpatient detoxification episodes, 60.7% and 34.7% respectively were followed within 7 days with outpatient SUD treatment. Facility-level descriptive statistics for the measure are presented in Table 3. Substantial facility-level variability existed, ranging from 5.5%-59.4%.

Other Details and Sensitivity Analyses

For outpatient detoxification episodes, 43.3% had an outpatient follow-up visit within 3 days and 63.9% had an outpatient follow-up visit within 10 days. For inpatient detoxification episodes, 20.1% had an outpatient follow-up visit within 3 days and 39.4% had an outpatient follow-up visit within 10 days.

Discussion

The purpose of this paper is to describe the decision making process that the Performance Measures Panel undertook in operationalizing and pilot testing these three measures. By transparently presenting the process rather than just the end results, we hope to shed light inside of the measure development process in order to stimulate debate about the decisions that were made. The analyses presented in this paper demonstrate that the measure specifications can be used in a large health care system that has a robust electronic medical record system to produce quality data to describe the overall level and variation in the underlying standards of care. It is worth noting that systems or programs that lack comparable data systems will be challenged to implement these measures.

Next steps include conducting feasibility and pilot testing in other health care systems and other contexts such as standalone addiction treatment programs, as well as to study the specification and predictive validity of these measures. To ease future implementation, pilot testing in other systems should also include the development of a list of National Drug Codes relevant to these measures. With the switch to ICD-10 in 2016, these measures should also be pilot tested with the new codes and compared to the values obtained using ICD-9. Also, the Performance Measures Panel still needs to specify and pilot test the other measures proposed for later development.