Conversion of the Agency for Healthcare Research and Quality's Quality Indicators from ICD‐9‐CM to ICD‐10‐CM/PCS: The Process, Results, and Implications for Users

Abstract

Objective

To convert the Agency for Healthcare Research and Quality's (AHRQ) Quality Indicators (QIs) from International Classification of Diseases, 9th Revision, Clinical Modification (ICD‐9‐CM) specifications to ICD, 10th Revision, Clinical Modification and Procedure Classification System (ICD‐10‐CM/PCS) specifications.

Data Sources

ICD‐9‐CM and ICD‐10‐CM/PCS classifications, General Equivalence Maps (GEMs).

Study Design

We convened 77 clinicians and coders to evaluate ICD‐10‐CM/PCS codes mapped from ICD‐9‐CM using automated GEMs. We reviewed codes to develop “legacy” specifications resembling those in ICD‐9‐CM and “enhanced” specifications addressing enhanced capabilities of ICD‐10‐CM/PCS.

Data Collection/Extraction Methods

We tabulated the numbers of mapped codes, added nonmapped codes, and deleted mapped codes to achieve the specifications.

Principal Findings

Of 212 clinical concepts (sets of codes) that comprise the QI specifications, we either added nonmapped codes to or deleted mapped codes from 115 (54 percent). The legacy and enhanced specifications differed for 46 sets (22 percent), affecting 67 of the 101 QIs (66 percent). Occasionally, concepts that defied conversion required reformulation of indicators.

Conclusions

Converting the AHRQ QIs to ICD‐10‐CM/PCS required a detailed, thorough process beyond automated mapping of codes. Differences between the legacy and enhanced versions of the QIs are frequently minor but sometimes substantive.

On October 1, 2015, the U.S. Department of Health and Human Services began requiring hospitals to report diagnoses and procedures using the International Classification of Diseases, 10th Revision, Clinical Modification and Procedure Classification System (ICD‐10‐CM/PCS). Whereas the International Classification of Diseases, 9th Revision, Clinical Modification (ICD‐9‐CM) (version 32) includes 14,567 diagnosis codes and 3,882 procedure codes, ICD‐10‐CM/PCS (Fiscal Year 2016) uses new codes and classification approaches to list 69,823 diagnosis codes and 71,974 procedure codes (National Center for Health Statistics 2015). This shift to ICD‐10‐CM/PCS was necessary because major structural limitations of ICD‐9‐CM could no longer adequately accommodate important disease and procedure concepts (National Center for Health Statistics 2015).

Several authors have converted health care quality measures from ICD‐9‐CM to the World Health Organization (WHO) version of ICD‐10 (Quan et al. 2008a; Januel et al. 2011a) and have tested these ICD‐10 specifications using data from multiple countries (Drosler et al. 2009, 2012; Quan et al. 2013), but the U.S. Clinical Modification has about five times as many diagnosis codes as the WHO version. Relevant to quality measurement, ICD‐10‐CM/PCS captures new details about disease severity; procedure location, laterality, and approach; and the timing of perioperative complications (National Center for Health Statistics 2015). The specificity of ICD‐10‐CM/PCS creates important but poorly understood opportunities and challenges for those who develop or use quality measures based on coded data (Giannangelo and Hyde 2012).

The Agency for Healthcare Research and Quality's (AHRQ) Quality Indicators (QIs) encompass 101 measures that use diagnosis and procedure codes, along with other data elements, to assess the quality of health care. Initially released during 2001–2003, the AHRQ QIs were developed during, and until recently have been applied exclusively to data from, the period when ICD‐9‐CM codes were in effect. By 2015, the AHRQ QIs were widely used by hospitals and vendors, public health agencies, health data organizations, payers and purchasers of health care, and researchers for numerous applications, including quality improvement, research and evaluation, public reporting, and value‐based purchasing (U.S. Agency for Healthcare Research and Quality 2017). As such, the AHRQ QIs represent an important test case for conversion of quality measures to ICD‐10‐CM/PCS.

With the transition to ICD‐10‐CM/PCS, AHRQ sought to take advantage of the additional specificity of new code sets to improve the validity and usefulness of the QIs through “enhanced specifications,” but also to create a separate set of “legacy specifications” that resembled the most recent ICD‐9‐CM version of the QIs (5.0) as closely as possible. The latter approach is consistent with that taken by the Centers for Medicare & Medicaid Services (CMS) in converting Medicare Severity Diagnosis Related Groups (MS‐DRGs); some researchers have reported initially that this conversion has “minimal impact on MS‐DRG assignment because the ICD‐10 MS‐DRGs are a replication of the ICD‐9 MS‐DRGs and do not take advantage of the increased specificity of ICD‐10” (Mills et al. 2011). In some countries (Januel et al. 2011b), but not all (Li et al. 2008; Quan et al. 2008b; Walker et al. 2012), the introduction of ICD‐10‐based morbidity classification has improved the accuracy of administrative data, and the increased specificity of the Canadian ICD‐10‐CA classification might allow new QIs to be developed (Southern et al. 2016).

The Agency for Healthcare Research and Quality's two‐level conversion process was planned several years ago, began in earnest in 2012 (when version 4.5 of the QIs was most current), and culminated in the release of version 6.0 of the QIs in 2016. In this article, we describe the process of converting the QIs from ICD‐9‐CM‐ to ICD‐10‐CM/PCS‐based logic, including the implications of “legacy” versus “enhanced” specifications, and the resulting opportunities and challenges for measuring quality of care.

The AHRQ quality indicators’ conversion process

The AHRQ QIs are grouped into four modules: Prevention Quality Indicators (PQIs), Inpatient Quality Indicators (IQIs), Patient Safety Indicators (PSIs), and Pediatric Quality Indicators (PDIs). The detailed specifications of the QIs and software are publicly available (U.S. Agency for Healthcare Research and Quality 2018). The general format of the indicators involves a ratio or count, and the logic for each indicator uses sets of diagnosis and/or procedure codes to characterize clinical concepts as numerator, denominator, exclusion, stratification, or risk adjustment criteria. These sets of codes are comprised of either diagnosis codes or procedure codes (but not both) and represent the building blocks of the QIs (Appendix SA2, Figure S1). The sets serve as approximations of the clinical concepts embedded in each indicator; the clinical concepts—and thus the sets of codes that represent them—must be interpreted in the context of the surrounding indicator logic. For example, the numerator of PSI 09 “Postoperative Hemorrhage or Hematoma” requires both a diagnosis of a perioperative hemorrhage or hematoma and a procedure to treat the hemorrhage/hematoma; therefore, the set of procedure codes for treatment of the hemorrhage/hematoma can be broadly defined because the diagnosis lends specificity. Typically, clinical concepts that recur across indicators use the same or a similar set of codes. To convert the QIs to ICD‐10‐CM/PCS, we had to map each set to new diagnosis and procedure codes and review its relationship to the clinical concept used within the ICD‐9‐CM QI technical specifications, while considering concomitant contextual changes to the logic of each QI.

Based on guidance from the National Quality Forum (NQF) (National Quality Forum 2010), we sought to complete six steps with the conversion process:

1. Convene Clinical and Coding Experts—We desired a team approach to identify and resolve specific problems in the conversion process. Coding experts needed to be facile with both ICD‐9‐CM and ICD‐10‐CM/PCS. Clinical experts ideally had both extensive clinical knowledge of the topics in question and some appreciation of coding practices and guidelines.

2. Determine Intent—We planned to accurately translate the QI specifications from ICD‐9‐CM to ICD‐10‐CM/PCS, but we also sought to capitalize on the added granularity and specificity of ICD‐10‐CM/PCS to improve the function of the indicators when possible. This involved examining the original intent of each measure and selecting codes on a conceptual basis rather than by mapping alone. Rarely, the original intent of an indicator could not be retained using the current version of ICD‐10‐CM/PCS. (Conversely, though outside the scope of this project, ICD‐10‐CM/PCS might allow creation of new indicators not possible in ICD‐9‐CM.)

3. Use an Appropriate Conversion Tool—We used existing General Equivalence Mapping (GEMs) files, available from the National Center for Health Statistics and CMS, to narrow the choice of target codes and facilitate conversion of codes from ICD‐9‐CM to ICD‐10‐CM/PCS.

4. Assess for Material Change—We sought to evaluate how well the function of the ICD‐10‐CM/PCS version of the indicators compared with that of the ICD‐9‐CM version. Because of varying influences on numerator and denominator criteria, conversion of the indicators could result in complex and unpredictable changes, which might be attenuated or magnified by differences in coding practices prompted by ICD‐10‐CM/PCS (Gibson et al. 2016). Unfortunately, large dual‐coded datasets were never made available to researchers and measure developers, so these assessments were performed using small local or regional datasets.

5. Solicit Stakeholder Comments—AHRQ has continually encouraged QI users to submit comments and questions about the design and function of the indicators. We sought to incorporate feedback and concerns from a wide variety of stakeholders, a process that included beta‐testing of an early release of the ICD‐10‐based QI software (U.S. Agency for Healthcare Research and Quality 2013) and continues as users apply the new specifications.

6. Assign Versions to the Updated Measure—As with prior changes to the QIs, AHRQ identifies QI specifications with version numbers. The QIs will be available in both ICD‐9‐CM and ICD‐10‐CM/PCS specifications in version 6.0 but only ICD‐10‐CM/PCS beginning with version 7.0.

Mapping Methods

The GEM files are invaluable in converting a large number of codes. Although they cannot be relied on to fully automate conversion (Butler, Mills, and Averill 2011), they are helpful to narrow the pool of possible codes to those of greatest interest. However, this “reference mapping” approach requires careful consideration of the mapping output and different directions of cross‐walk between ICD‐9‐CM and ICD‐10‐CM/PCS, to avoid conversion problems (Mills et al. 2011; Drosler et al. 2012; Fenton and Benigni 2014). To facilitate “batch mapping” of many codes simultaneously, collaborators at Battelle Memorial Institute created an automated stand‐alone mapping tool called “MapIT” (U.S. Agency for Healthcare Research and Quality 2015).

Using GEMs, MapIT allows “forward,” “backward,” and “reverse” mapping. The forward and backward mapping methods refer to the intuitive process of determining which code(s) in the target classification correspond to a code in the baseline classification, with “forward” and “backward” signifying the ICD‐9‐to‐ICD‐10 and the ICD‐10‐to‐ICD‐9 directions, respectively. These two methods generally provide the best match in code descriptions. Reverse mapping reveals all codes that map to the original code, even if the original code does not map to them. It can involve either forward or backward applications: using ICD‐10 codes in the ICD‐9‐to‐ICD‐10 mapping to find all possible ICD‐9 equivalents (reverse forward) or using ICD‐9 codes in the ICD‐10‐to‐ICD‐9 mappings to find all possible ICD‐10 equivalents (reverse backward). For example, 556.9 “Ulcerative colitis” maps to K51.90 “Ulcerative colitis, without complications” in the forward direction and, conversely, K51.90 maps to 556.9 in the backward direction. However, in the reverse backward direction, additional codes besides K51.90 are revealed from reverse mapping of 556.9: K51.911 “Ulcerative colitis, with rectal bleeding”; K51.912 “Ulcerative colitis, with intestinal obstruction”; K51.913 “Ulcerative colitis, with fistula”; K51.914 “Ulcerative colitis, with abscess”; K51.918 “Ulcerative colitis, with other complications”; and K51.919 “Ulcerative colitis, with unspecified complication.” We incorporated all these methods to identify potentially suitable codes for the ICD‐10‐CM/PCS QI specifications (and possibly optimize the ICD‐9‐CM specifications).

First‐Stage Review

Because the GEM files are more complex than a simple translation from ICD‐9‐CM to ICD‐10‐CM/PCS, the conversion process requires review of all possible alternative codes and their descriptions. With automated mapping, we focused on reviewing the translation of meaning from ICD‐9‐CM to ICD‐10‐CM/PCS. Complicating factors included the following: (1) changes in diagnosis specificity, such as encounter information and laterality; (2) the complete revision of procedure codes with root objectives, approaches, and body parts; (3) the absence of diagnoses in procedure codes (e.g., “aneurysm repair”); (4) the use of multiple codes to represent what conceptually was one procedure in ICD‐9‐CM (e.g., “radical pancreaticoduodenectomy”); (5) the absence of eponyms (e.g., “Fontan procedure”); (6) changes in coding guidelines; and (7) the absence of an appropriate code for specific procedures (e.g., operations for certain congenital cardiac anomalies).

To review the automated mapping results for all 212 code sets, we solicited physicians, coding professionals, nursing quality improvement specialists, and data users familiar with the QIs and/or ICD‐10‐CM/PCS code sets. We convened 10 workgroups with a total of 77 experts (see Acknowledgments), based on clinical categories: Cancer, Cardiac, Critical Care/Pulmonary, Infection, Internal Medicine, Neonatal/Pediatric, Neurology, Obstetrics and Gynecology, Orthopedic, and General and Trauma Surgery. In August 2012, the original coding expert on our team, an American Health Information Management Association (AHIMA)–Approved ICD‐10‐CM/PCS Trainer, instructed panelists on ICD‐10‐CM/PCS, the mapping methods employed, and how to review the automated mapping results. Each work group had at least one AHIMA‐Approved ICD‐10‐CM/PCS Trainer expert (in addition to our team's coding expert), who served as a resource to discuss coding guidelines and practices, and to provide explanations as needed.

During September and October 2012, the work groups evaluated the mapping results and participated in follow‐up conference calls to discuss all disagreements and to provide specific recommendations. Our team explained the rationale and function of each code set. Panelists recommended deletion of mapped codes and suggested additional codes that were not generated from the automated GEMs, as well as respecification of some of the QIs. Although we attempted to remain faithful to the existing clinical intent of each indicator, clinical concepts in ICD‐10‐CM/PCS do not coincide completely with those in ICD‐9‐CM, such that “identical” specifications across the ICD‐9‐CM to ICD‐10‐CM/PCS transition are rarely achievable.

Second‐Stage Review

After merging comments and recommendations for each code set, members of our team (an internist/pediatrician, a surgeon, and a coding expert) reviewed all candidate codes and categorized the recommendations (both additions to and deletions from the mapped codes) into three levels: 1, 2, and 3. Level 1 recommendations included “inappropriate codes” involving clinical concepts that were never intended to be part of the indicator specifications, independent of the version of ICD (Table 1). The intent of Level 1 recommendations was to align the ICD‐10‐CM/PCS specifications as closely as possible to those in ICD‐9‐CM to create a “legacy” version of the specifications. These legacy specifications would primarily be of interest to users wanting to minimize the impact of the ICD‐9‐CM to ICD‐10‐CM/PCS transition in measuring QI performance trends over time.

Table 1.

Categories of Recommended Additions or Deletions to GEM‐Mapped Codes, with Example Comments

Level	Description	Examples of Reasons for Designation at This Level
1	For “legacy” specifications: Codes involving concepts that were never intended to be part of the QI specifications; addressing these recommendations would theoretically align the ICD‐10‐CM/PCS version of the QIs as closely as possible to the ICD‐9‐CM version	The mapped code is specific to the incorrect gender
		The mapped code is specific to an incorrect age group (e.g., adult condition for a pediatric concept)
		The mapped code is an unnecessary component of a cluster that is better captured by other code(s)
		The mapped code is included in a different code set (redundant)
		The mapped code involves a newly classified clinical concept that does not fit with the intent of this set
		The mapped code is specific to an incorrect anatomic site
		The mapped code pertains to a specific (and incorrect) time duration whereas the original code is expressly non‐specific
2	For “enhanced” specifications: Codes involving clinical concepts that can be specified more precisely in the ICD‐10‐CM/PCS version of the indicator specifications, either because of limitations of ICD‐9‐CM or enhanced capabilities of ICD‐10‐CM/PCS; addressing these recommendations would theoretically optimize the indicators for use with ICD‐10‐CM/PCS (as version 6.0 of the QIs)	The mapped code is not as specific to the intent of the code set as other code(s)
		The unmapped code represents a clinical concept that fits with the intent of this set
3	For “deferred” consideration: Codes involving more complex mapping problems or entailing a fundamental re‐examination of prior choices regarding the design of the indicator; addressing these recommendations would be outside the scope of the conversion process, though potentially warranted at a future time	The unmapped code represents a clinical concept that might fit with the intent of this set
		The mapped code arguably does not fit with the intent of this set

Level 2 recommendations involve clinical concepts that can be specified more precisely in the ICD‐10‐CM/PCS version of the indicator specifications, either because of limitations of ICD‐9‐CM or enhanced capabilities of ICD‐10‐CM/PCS (Table 1). These recommendations were incorporated into “enhanced” specifications of the QIs, suitable for widespread use, including evaluation by NQF. GEMs frequently offered many plausible translations for one ICD‐9‐CM code, and this situation required clinicians’ input. For example, with the novel structure of ICD‐10‐PCS, we needed surgeons’ input to determine whether some root operations involved the intended concept. Adding or deleting these codes from the results of the automated mapping may lead to some discontinuity in indicator rates before and after ICD‐10‐CM/PCS implementation, but they are expected to enhance the performance of the indicators while remaining faithful to their original clinical intent.

We categorized more complex mapping problems as Level 3. Such recommendations involved clinical concepts that were included (or not) in the ICD‐9‐CM version of the specifications, but that warrant consideration for removal (or addition) to refine the function of the indicators for future use. Because Level 3 changes would involve re‐examining choices that were made when the original ICD‐9‐CM QI specifications were developed, we did not consider these proposals central to the current conversion process.

Third‐Stage Review

Based on NQF guidance, our team (all authors) conducted a third level of clinical review to ensure that the ICD‐10‐CM/PCS codes resulting from the second level of review were consistent with the original intent for each QI. The primary aims of this review were to ensure consistency across the work of the 10 work groups and 77 experts who were involved in the first stage of the review process, to cross‐check mappings against targeted manual look‐up using the indices and tabular lists of ICD‐10‐CM/PCS, to address complex clinical issues that had been deferred for more in‐depth evaluation, and to identify critical issues that need to be addressed with the ICD‐10‐CM/PCS Coordination and Maintenance Committee and Coding Clinic for ICD‐10‐CM and ICD‐10‐PCS. This clinical review resulted in:

• Small adjustments based on annual updates to the GEM files;

• Identifying questionable mappings that need to be addressed further with the federal agencies involved in the annual updates of GEMs;

• Revising level assignment, specifically Level 2 and Level 3, to ensure consistent treatment across sets of codes;

• Incorporating changes from version 5.0 of the QI software (an update to the ICD‐9‐CM‐based specifications) to maintain the clinical equivalence of the version 6.0 ICD‐10‐CM/PCS specifications;

• Identifying additional ICD‐10‐CM/PCS codes that might need to be implemented, or existing codes that might need to be clarified, by the federal agencies involved with code set maintenance;

• Consulting with surgeons to determine whether some of the ICD‐10‐PCS procedures can actually be performed;

• Evaluating clusters of codes that must be used together to describe a condition or procedure (i.e., to determine whether the set requires multiple codes from the cluster, or whether one code is sufficient to capture the clinical concept);

• Merging clinically related sets of codes used in the QI specifications to minimize duplication of codes across sets (e.g., pressure ulcer location and stage are captured using separate sets of codes in ICD‐9‐CM but these concepts are combined in ICD‐10‐CM, requiring only one set);

• Deleting sets when there are no ICD‐10‐CM/PCS equivalences or where none of the ICD‐10‐CM/PCS codes apply to the clinical intent (e.g., “incidental appendectomy”) and identifying alternative strategies, when appropriate; and

• Developing diagnosis‐based logic to be added to indicators involving procedure codes when a particular diagnosis is conceptually embedded in the ICD‐9‐CM procedure code(s).

Results of the Conversion Process

Among 212 code sets that comprised the QIs, 148 involved diagnosis codes and 64 involved procedure codes. The number of codes per set in ICD‐9‐CM varied from 1 to 1759, and generally mapped to an increased number in ICD‐10‐CM/PCS, particularly for sets involving procedure codes (Figure 1). After review of the mapped codes, we deleted one or more of the mapped codes for 84 sets (range 1–1,641 codes/set) and added nonmapped codes to 11 sets (range 1–45 codes/set) for the legacy specifications (Appendix SA3).

Figure 1.

Association between the Number of ICD‐9‐CM Codes in Version 4.5 of the QI Specifications and the Number of ICD‐10‐CM/PCS Codes Resulting from Automated Use of GEMs, Depicted Separately for Each Clinical Concept (“Set of Codes”) Embedded in the QI Specifications (N = 212 Sets)Note. Data points are offset by a small random distance (“jittered”) to facilitate presentation.

For the enhanced specifications, we additionally deleted one or more of the mapped codes for 39 sets (range 1–283 codes/set) and added more nonmapped codes to 11 sets (range 1–11 codes/set); these additions involved condition(s) or procedure(s) newly classified in ICD‐10‐CM/PCS and/or problems discovered through reverse mapping. In total, the enhanced specifications differed from those suggested by the GEMs by at least one code for 118 code sets (56 percent) (Figures 2 and 3). The legacy and enhanced specifications differed from each other by at least one code for 46 sets (22 percent). For 67 of the 101 QIs (66 percent), the enhanced specifications differed to some degree from the legacy specifications. Most of these differences likely would have minimal impact on the indicator rate or count, but some potentially entail substantial effects (Table 2).

Figure 2.

Changes in the Numbers of ICD‐9‐CM or ICD‐10‐CM Diagnosis Codes for Each Clinical Concept (“Set of Codes”) in the QI Specifications as a Result of Converting Mapped Codes to the “Enhanced” QI Specifications Note. Arrows indicate the changes for each set. (Only 75 sets with a change in the number of codes in ICD‐9‐CM or ICD‐10‐CM are depicted in this figure. Due to the logarithmic scale, three sets that we could not convert to a single code in ICD‐10‐CM are depicted only as a mapped set.)

Figure 3.

Changes in the Numbers of ICD‐9‐CM or ICD‐10‐PCS Procedure Codes for Each Clinical Concept (“Set of Codes”) in the QI Specifications as a Result of Converting Mapped Codes to the “Enhanced” QI Specifications Note. Arrows indicate the changes for each set. (Only 38 sets with a change in the number of codes in ICD‐9‐CM or ICD‐10‐PCS are depicted in this figure. Due to the logarithmic scale, two sets that we could not convert to a single code in ICD‐10‐PCS are depicted only as a mapped set.)

Table 2.

Quality Indicators with Substantive Differencesa between Legacy and Enhanced Specifications

Indicator(s)b	Clinical Concept	Role in Indicator	Difference between Legacy and Enhanced Specifications and Anticipated Effect on Indicator Rates
IQI 19 “Hip Fracture Mortality Rate”	Hip fracture	Denominator	The enhanced set includes osteoporosis‐related pathologic fractures (which likely contributed little to the ICD‐9‐CM version of this indicator); the possible effect on the enhanced rate is uncertain, but it may increase if such hospitalizations are associated with greater mortality
IQI 22 “Vaginal Birth after Cesarean (VBAC) Rate, Uncomplicated” IQI 34 “Vaginal Birth After Cesarean (VBAC) Rate, All”	Previous Cesarean delivery	Denominator	The denominator of the enhanced specifications includes a code for failed attempted vaginal delivery after prior Cesarean (O66.41), so the enhanced rate may be substantially lower than the legacy rate
PSI 07 “Central Venous Catheter‐Related Blood Stream Infection Rate” PSI 23 “Central Venous Catheter‐Related Blood Stream Infection Rate” PDI 12 “Central Venous Catheter‐Related Blood Stream Infection Rate”	Central venous catheter‐related blood stream infection	Numerator	The numerator diagnosis criteria are slightly more restrictive in the enhanced specifications (catheter site infections no longer contribute), so the rate may be moderately lower
PSI 08 “Postoperative Hip Fracture Rate”	Hip fracture	Numerator	The enhanced set includes osteoporosis‐related pathologic fractures (which likely contributed little to the ICD‐9‐CM version of this indicator), so the enhanced rate may be higher than the legacy rate
PSI 11 “Postoperative Respiratory Failure Rate” PDI 09 “Postoperative Respiratory Failure Rate”	Mechanical ventilation <96 hours	Numerator	The numerator procedure criteria involving mechanical ventilation are slightly more restrictive in the enhanced specifications (ventilation <24 hours no longer contributes), so the rate may be moderately lower
PSI 17 “Birth Trauma Rate—Injury to Neonate”	Birth trauma	Numerator	The enhanced specifications will capture records with intracranial hemorrhage due to birth trauma. Because these events are rare, it is unlikely the rate will increase substantially; however, if coding errors result in spontaneous intracranial hemorrhage (from prematurity) being misclassified as birth trauma, the rate could increase more substantially
PQI 01 “Diabetes Short‐term Complications Admission Rate” PDI 15 “Diabetes Short‐term Complications Admission Rate”	Short‐term complications of diabetes mellitus	Numerator	The legacy specifications will capture records with complications from “other specified diabetes mellitus” (non‐type 1, non‐type 2) as the principal diagnosis but the enhanced specifications will not, so the rate will likely be slightly lower
PQI 03 “Diabetes Long‐term Complications Admission Rate”	Long‐term complications of diabetes mellitus	Numerator	The legacy specifications will capture records with complications from “other specified diabetes mellitus” (non‐type 1, non‐type 2) as the principal diagnosis but the enhanced specifications will not, so the rate will likely be slightly lower
PQI 05 “Chronic Obstructive Pulmonary Disease (COPD) or Asthma in Older Adults Admission Rate” PQI 15 “Asthma in Younger Adults Admission Rate” PDI 14 “Asthma Admission Rate”	Asthma requiring hospitalization	Numerator	The legacy specifications will capture records with mild cases of asthma as the principal diagnosis but the enhanced specifications will not, so the rate will likely be appreciably lower
PQI 09 “Low Birth Weight Rate”	Low birthweight	Numerator	The legacy specifications will capture records with newborns with weight unspecified but described as “light” or “small” for gestational age. The enhanced specification will not capture such records, so the rate will likely be slightly lower
NQI 03 “Neonatal Blood Stream Infection Rate”	Neonatal sepsis	Numerator	The legacy specifications will flag records with certain sepsis diagnoses with limited relevance to health care‐associated neonatal infections, so the enhanced specifications will likely yield lower rates, though more specific to the condition of interest

^aNumerous other indicators involved minor differences between the legacy and enhanced specifications that would not be expected to affect rates to an appreciable degree (e.g., a code set involving an infrequently applicable exclusion criterion in the indicator logic might differ for a small number of rarely used codes). Such instances are not included in this table.

^bComposite Quality Indicators (IQIs 90 and 91; PSI 90; PQIs 90, 91, and 92; PDIs 19, 90, 91, and 92) are not listed in this column, but each of these would be affected to the extent that the component indicators are affected.

IQI, Inpatient Quality Indicator; NQI, Neonatal Quality Indicator; PDI, Pediatric Quality Indicator; PSI, Patient Safety Indicator; PQI, Prevention Quality Indicator.

By virtue of the mapping process, we occasionally identified codes for addition to or deletion from the ICD‐9‐CM specifications (version 4.5). These represented clarifications or reinterpretations of the original clinical concept (informed by discussion with clinical panelists) in 13 cases, newly discovered software errors in three cases, and recent changes to ICD‐9‐CM in two cases.

Challenges Arising in Conversion

Adherence to the NQF guidance regarding the conversion process served us well, but we still encountered numerous challenges related to fundamental differences in the ICD‐9‐CM and ICD‐10‐CM/PCS classifications. The most obvious of these, the profusion of available ICD‐10‐CM codes, was straightforward—albeit tedious—to address for diagnoses. For example, regarding PSI 08 “Postoperative Hip Fracture,” 18 codes for hip fractures in ICD‐9‐CM mapped to 216 codes in ICD‐10‐CM. Many of these new codes involved permutations of fracture characteristics (nondisplaced vs. displaced; open vs. closed) and laterality (right, left, or unspecified), which were unimportant distinctions from our standpoint—and thus could be easily grouped by more important characteristics. However, distinctions involving new anatomic descriptors, such as “apophyseal,” “unspecified physeal,” and “other physeal” fractures, required nuanced input from orthopedic surgeons to determine their relevance to fall‐related hip fractures. Similar issues arose for many clinical topics, requiring detailed advice from clinician panelists.

The entirely new architecture of ICD‐10‐PCS posed greater challenges. Even expert clinicians struggled to comprehend what some of the procedure codes represented, particularly for approaches for regions such as the oral cavity and genitals, for which distinctions between “open,” “external,” and “via natural or artificial opening” seemed abstruse. Because ICD‐10‐PCS directs coders to select separate codes for each distinct portion of a procedure, it is necessary to focus on the defining portion(s) of the procedure. The overarching difficulty across all body regions was to determine which code(s) might correspond to a typical procedure readily identifiable in ICD‐9‐CM. Additionally, some ICD‐10‐PCS codes seem to border on the impossible (e.g., 0DT97ZZ “Resection of duodenum, via natural or artificial opening”).

The conversion process was not readily soluble for five code sets, affecting 22 indicators, requiring that we either fundamentally reformulate the indicators or accept them as suboptimal. For example, the numerator of PSI 14 “Postoperative Wound Dehiscence” relied solely on procedure code 54.61 “Reclosure of postoperative disruption of abdominal wall” in ICD‐9‐CM, but procedure codes in ICD‐10‐PCS do not include any information about the indication for the procedure. The mapped procedure codes for closure of the abdominal wall are not restricted to procedures involving treatment of wound dehiscence. Thus, we introduced a new set for the diagnosis of wound dehiscence [T81.32XA “Disruption of internal operation (surgical) wound, not elsewhere classified”], which must be coupled with a procedure for closing the abdominal wall (occurring after the index operation on the abdomen) in the PSI 14 logic.

Similarly, IQI 24 “Incidental Appendectomy” could not be directly converted because there is no distinction between incidental and nonincidental appendectomy in ICD‐10‐PCS. In this instance, we opted for indicator logic that focuses on appendectomy occurring without a diagnosis of appendicitis, and not in the context of another abdominal operation that would naturally include appendectomy (e.g., right hemicolectomy).

Some of these insoluble conversions do not have a readily identifiable work‐around. IQIs 21 “Cesarean Delivery Rate, Uncomplicated,” 22 “Vaginal Birth after Cesarean Delivery Rate, Uncomplicated,” and 33 “Primary Cesarean Delivery Rate, Uncomplicated” all exclude breech delivery in ICD‐9‐CM—a concept that is no longer identifiable in ICD‐10‐PCS—from the population of interest. Because the diagnosis codes corresponding to breech presentation do not necessarily involve breech delivery, there is currently no remedy to this mapping problem. Similarly, ICD‐9‐CM's V29 codes (which help identify neonates) lack direct mappings to ICD‐10‐CM, so other criteria—such as age in days, admission type “newborn,” or a diagnosis of in‐hospital live birth—must suffice to identify this population for several of the PDIs.

Five code sets included external cause of injury and poisoning codes (“E codes”) in ICD‐9‐CM, affecting 10 indicators and involving the key numerator criteria for three concepts: retained surgical item, accidental puncture or laceration, and transfusion reaction. However, many concepts related to iatrogenic injury are no longer classified in the corresponding V‐W‐X‐Y chapters in ICD‐10‐CM and are instead represented by organ‐specific diagnoses scattered through the other chapters. For this reason as well as the fact that E codes contribute only small increases in sensitivity, are frequently nonspecific, and are considered optional for reporting in some states, AHRQ discontinued using these codes (with one unavoidable exception involving a set for self‐inflicted injuries) beginning with version 5.0.

A final challenge in converting the QIs arose from CMS's categorization of ICD‐10‐PCS codes as “major” versus “minor” operating room (OR) procedures. Major OR procedure status typically determines that a hospitalization is categorized into a Surgical (rather than Medical) Diagnosis Related Group (DRG), and six of the PSIs concern only records with a Surgical DRG. Thus, inconsistencies in the classification of major OR procedure status between ICD‐9‐CM and ICD‐10‐PCS could substantially impact the function of these PSIs. We initially neglected this possibility, but now that we and others have recognized inconsistencies in CMS's definition of major OR procedures, CMS has implemented changes in its categorization (Centers for Medicare & Medicaid Services 2017, 2018), and AHRQ has decided not to rely entirely on CMS’ categorization.

Discussion: Advantages and Disadvantages of ICD‐10‐CM/PCS

In contrast to suggestions by some authors (Boyd et al. 2015), conversion of the AHRQ QIs to ICD‐10‐CM/PCS involved a detailed, thorough evaluation of the universe of codes to select the most appropriate specifications of the indicators, considering both legacy and enhanced functions. Over half of the 212 code sets required modification from what was suggested by the GEMs. As byproducts of this process, we identified at least three possible errors in the GEMs, three errors in the ICD‐9‐CM QI software, and 13 instances in which we adjusted the ICD‐9‐CM specifications (i.e., added or deleted at least one code) based on clarification or reinterpretation of the clinical concept.

Rather than relying rotely on the GEMs, we used the suggested codes to consider contextual factors such as documentation and coding practices, the prevalence of different diagnoses, unmapped alternative codes, and the surrounding indicator logic. For example, the original mapping of hip fractures did not include the M80.05 series of codes (“Age‐related osteoporosis with current pathological fracture, femur”) because the corresponding codes in ICD‐9‐CM (733.14 “Pathologic fracture of neck of femur” and 733.15 “Pathologic fracture of other specified part of femur”) were not part of the definition of PSI 08 “Postoperative Hip Fracture Rate.” However, ICD‐10‐CM coding guidance suggests that coders might frequently select one of the M80.05 codes for hospital‐acquired fractures because they combine the concepts of osteoporosis and pathologic fracture (Table 2), leading AHRQ to add these codes to the numerator specification for PSI 08.

In a few cases, we were able to exploit the specificity of ICD‐10‐CM/PCS by shedding suboptimal codes or clinical concepts that were embedded in ICD‐9‐CM codes. For example, procedure code 96.70 “Continuous mechanical ventilation of unspecified duration” had to be accommodated in the PSI 11 numerator specification, but since all mechanical ventilation codes in ICD‐10‐PCS (series 5A19) involve a specified duration, postoperative respiratory failure can now be defined more precisely. Similarly, the general diagnosis codes for neonatal sepsis in ICD‐9‐CM, which did not specify organisms consistent with health care‐acquired infections, are now supplanted with ICD‐10‐CM codes that do. Such changes may modestly improve the validity of some of the QIs. Additionally, the increased specificity of ICD‐10‐CM/PCS might allow development of new QIs, for example, amputation of the distal part of an extremity following placement of an arterial line into that extremity.

Some sets that were distinct in ICD‐9‐CM converged to the same group of codes in ICD‐10‐CM/PCS. For example, regarding PSI 09 “Perioperative Hemorrhage or Hematoma,” postprocedural hemorrhage and hematoma involve separate diagnosis codes in ICD‐9‐CM, but, in the Fiscal Year 2016 release of ICD‐10‐CM, they are subsumed under the same codes, permuted by organ system. The ICD‐10‐CM Coordination and Maintenance Committee separated the concepts of postprocedural hemorrhage, hematoma, and seroma (formerly indexed under hematoma) in the Fiscal Year 2017 release of ICD‐10‐CM in response to requests by clinicians and quality monitoring advocates.

The large number of codes available in ICD‐10‐CM/PCS sometimes allows for more refined indicator logic, but it also represents a significant challenge in establishing optimal definitions. The most frequent circumstance involved a multitude of new, highly specific codes in ICD‐10‐CM/PCS—both for diagnoses and procedures—that were not anticipated when the QIs were developed, are not inconsistent with the original intent (e.g., they frequently mapped to nonspecific ICD‐9‐CM codes included in the indicator specifications), but also do not represent concepts central to the indicators.

For some of the sets that mapped to several thousand codes in ICD‐10‐CM/PCS, it was helpful to review the candidate codes by means other than a simple tabular format. For example, for one set that encompassed a large number of injury diagnoses, it was equally helpful to consider which of the almost 12,000 injury diagnoses corresponding to an initial encounter were not mapped, in addition to the large proportion that were mapped. Similarly, for the sake of consistency and coherence, we found it helpful to review large numbers of ICD‐10‐PCS procedure codes by disaggregating the seven characters so that we could examine all codes in a set pertaining to a particular root operation (third character) or approach (fifth character).

We anticipate that ICD‐10‐CM/PCS coding guidelines and practices will continue to evolve as the classifications are put to use. For example, it was not initially clear which procedure root(s) best applied to abdominal aortic aneurysm repair or whether drainage of a perirectal abscess should be coded as an open or a percutaneous approach (Utter et al. 2013). The ICD‐10‐CM/PCS Coordination and Maintenance Committee and Coding Clinic on ICD‐10‐CM/PCS has begun to resolve such discrepancies and facilitate conformity in coding practices. Given the sheer volume of codes involved in the QIs, it will be necessary to adjust the initial ICD‐10‐CM/PCS‐based specifications over time.

Little is currently known about the validity and reliability of the ICD‐10‐CM/PCS‐based version of the QIs, largely due to their novelty and the current unavailability of broadly representative ICD‐10‐CM/PCS‐based hospitalization data. However, the developers of the code sets have argued that ICD‐10‐CM/PCS should improve quality measurement (National Center for Health Statistics 2015) and the validity of the QIs because “ICD‐10‐CM greatly expands the codes for medical complications and medical safety issues” (Bowman 2008). Performance trends spanning the transition to ICD‐10‐CM/PCS will likely be smooth for some indicators but discontinuous for others, including those that required significant reformulation or allowed substantial enhancement in ICD‐10‐CM/PCS.

In summary, our experience is consistent with that of other researchers (Drosler et al. 2012; Fenton and Benigni 2014), in that “naïve” mappings of quality measures from ICD‐9‐CM to ICD‐10‐CM/PCS are very problematic, and the recent code set conversion may have unpredictable but substantial effects on quality indicator rates. Although we attempted to anticipate and minimize these effects through careful three‐stage review, researchers and policy makers should now focus on evaluating the function of respecified AHRQ QIs in ICD‐10‐CM/PCS and their continual refinement. The accrual of broadly representative administrative data will be central to both of these processes. AHRQ has encouraged feedback from QI users as they apply the ICD‐10‐CM/PCS‐based QI software.

Supporting information

Appendix  SA1: Author Matrix.

Appendix  SA2: Figure Legend for Figure S1.

Figure  S1: Technical Specifications of One AHRQ Quality Indicator, Patient Safety Indicator 12.

Appendix  SA3: Irregularities in General Equivalence Mappings Identified through the AHRQ QI Conversion Process.