The Validity of International Classification of Diseases, Ninth Revision, Clinical Modification Diagnosis Codes for Identifying Patients Hospitalized for COPD Exacerbations

Abstract

Background:

Acute exacerbations of COPD (AE-COPD) are a leading cause of hospitalizations in the United States. To estimate the burden of disease (eg, prevalence and cost), identify opportunities to improve care quality (eg, performance measures), and conduct observational comparative effectiveness research studies, various algorithms based on the International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) codes have been used to identify patients with COPD. However, the validity of these algorithms remains unclear.

Methods:

We compared the test characteristics (sensitivity, specificity, positive predictive value, and negative predictive value) of four different coding algorithms for identifying patients hospitalized for an exacerbation of COPD with chart review (reference standard) using a stratified probability sample of 200 hospitalizations at two urban academic medical centers. Sampling weights were used when calculating prevalence and test characteristics.

Results:

The prevalence of COPD exacerbations (based on the reference standard) was 7.9% of all hospitalizations. The sensitivity of all ICD-9-CM algorithms was very low and varied by algorithm (12%-25%), but the negative predictive value was similarly high across algorithms (93%-94%). The specificity was > 99% for all algorithms, but the positive predictive value varied by algorithm (81%-97%).

Conclusions:

Algorithms based on ICD-9-CM codes will undercount hospitalizations for AE-COPD, and as many as one in five patients identified by these algorithms may be misidentified as having a COPD exacerbation. These findings suggest that relying on ICD-9-CM codes alone to identify patients hospitalized for AE-COPD may be problematic.

Acute exacerbations of COPD (AE-COPD) are a leading cause of hospitalizations, deaths, and health-care expenditures in the United States, accounting for nearly $30 billion in health-care expenditures and 120,000 deaths each year.^1,2 The number of deaths from COPD has risen dramatically over the past several decades, and hospitalization rates for AE-COPD in the United States appear to be increasing.^3,4 Studies we and others have conducted using administrative data suggest that the quality of care and clinical outcomes vary substantially among US hospitals for patients admitted for AE-COPD.^5,6 Accordingly, there is considerable interest in understanding and improving the care and outcomes of these patients.

Quality-of-care indicators already have been developed and implemented by the Centers for Medicare & Medicaid Services for patients hospitalized for acute myocardial infarction, heart failure, and pneumonia, and there has been recent attention devoted to developing performance measures for COPD.^7‐9 These performance measures typically rely on the primary International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) discharge diagnosis codes from hospital administrative data to identify appropriate patients for inclusion. However, the assignment of primary and secondary ICD-9-CM discharge diagnosis codes for hospitalized patients is done specifically for reimbursement and may be influenced by both the clinical evidence in the medical record and the relative rate of reimbursement anticipated for a diagnosis, raising concerns about the validity of these data for the identification of patients with a specific condition.^10,11 Additionally, there is no consensus about which set of ICD-9-CM diagnosis codes (ie, algorithm) is most appropriate for identifying patients hospitalized for AE-COPD. Various groups have used different algorithms, which range from a single discharge diagnosis code to more complex sets of codes based on primary and secondary discharge diagnoses.^12,13 Despite the widespread use of these codes in epidemiologic studies and, more recently, for performance measurement⁷ and observational comparative effectiveness research (CER) studies,^14,15 there is little information on the validity of ICD-9-CM coding algorithms to identify hospitalizations for AE-COPD. For this reason, there have been calls for validation studies in this area.¹⁶

To estimate the burden of COPD and to ensure that quality-of-care initiatives and observational CER in AE-COPD target the intended population, it is essential to understand whether ICD-9-CM coding algorithms for AE-COPD can accurately distinguish patients admitted for AE-COPD from those who are not. Therefore, we conducted a study to evaluate the validity of identifying patients hospitalized for AE-COPD using several previously published ICD-9-CM coding algorithms compared with chart review.

Materials and Methods

In the COPD-ARTIFACT (Administrative Data to Identify COPD or Heart Failure) study, hospital discharge databases at two academic health centers affiliated with different universities in Chicago were used to select a stratified probability sample of 200 adults admitted between November 2005 and October 2006. Inclusion criteria were age ≥ 25 years and hospital admission to a medical service (Fig 1); this lower boundary for age was used based on previous studies.^17,18 To avoid gaps in chart documentation, patients were excluded if they had been transferred from another hospital. For patients with multiple admissions, we included the first admission. To estimate the sensitivity and specificity of various ICD-9 algorithms for AE-COPD, we needed to include hospitalizations with and without AE-COPD. A simple random sample of hospitalizations may have resulted in too few hospitalizations for AE-COPD, so we developed strata based on ICD-9-CM discharge codes and then randomly selected hospitalizations within each stratum for chart abstraction (Fig 1). This stratified probability sampling strategy (using sampling weights, see Analysis section) allowed us to make inferences about the overall study population without having to conduct chart abstractions on all hospitalizations.

Figure 1.

Patient selection. Algorithms 1 (expanded COPD algorithm), 2 (primary COPD or respiratory failure codes), 3 (multiple primary COPD codes), and 4 (single primary AE-COPD code) are shown. **Excluded if 491.20 (obstructive chronic bronchitis without exacerbation) in the primary or secondary diagnosis positions. AE-COPD = acute exacerbation of COPD; ICD-9-CM = International Classification of Disease, Ninth Edition, Clinical Modification.

Medical charts were abstracted by one of two physician-reviewers (B. D. S., A. B.) masked to ICD-9-CM diagnosis code. Physician notes from the index hospitalization and outpatient and inpatient diagnostic tests within the prior 2 years also were reviewed. The 2004 American Thoracic Society/European Respiratory Society guidelines define an AE-COPD as a change in cough, dyspnea, or sputum sufficient to warrant a change in therapy.¹⁹ We operationalized this guideline-recommended definition for the purpose of chart abstraction using an approach similar to that used in a study conducted in EDs.^19,20 Patients were determined to have AE-COPD (reference standard) if all of the following three criteria were met: (1) physician diagnosis of COPD (documented diagnosis of chronic bronchitis, emphysema, chronic obstructive pulmonary disease, or COPD in the admission note, progress note, or discharge summary from the index hospitalization); (2) presence of cough, dyspnea, or sputum production on presentation; and (3) hospitalization for one of these respiratory symptoms. Prior to beginning the study, 20 charts were abstracted independently by each reviewer to establish abstraction procedures. Interrater reliability for AE-COPD using the reference standard was high (10% random sample of records, κ = 0.88; 95% CI, 0.64-1.00).^21,22

Patients with COPD may have other acute or chronic cardiopulmonary conditions (eg, pneumonia, heart failure) that lead to respiratory symptoms and hospitalization.^13,18 Thus, we also used a more restrictive AE-COPD reference standard that classified patients with other clinically significant cardiopulmonary disorders that commonly result in respiratory symptoms as “not AE-COPD.” Based on this reference standard, all patients with a physician-documented diagnosis of heart failure, cardiac arrhythmia, acute myocardial infarction, asthma, pneumonia, interstitial lung disease, pneumothorax, or pulmonary embolism were reclassified as not AE-COPD, even if they met all criteria of the original reference standard.

We tested the validity of four ICD-9-CM algorithms for AE-COPD of varying complexity (Fig 1, Table 1). Algorithm 1 (expanded COPD algorithm) is a multicode algorithm that represents the union of multiple published ICD-9-CM coding algorithms^1,6,7,17,23; it uses both primary codes for COPD as well as primary codes for respiratory failure coupled with secondary codes for COPD. Algorithm 2 (primary COPD or respiratory failure codes) uses both primary codes for COPD and primary codes for respiratory failure coupled with secondary codes for COPD.^5,23 Algorithm 3 (multiple primary COPD codes), perhaps the most commonly applied algorithm,⁷ is a multicode algorithm that uses several primary codes for COPD, whereas algorithm 4 (single primary AE-COPD code) consists of a single primary code specifically for AE-COPD.⁶

Table 1.

—ICD-9-CM Codes Used in Algorithms to Identify AE-COPD

ICD-9-CM Code	Description
490	Bronchitis, not specified as acute or chronic
491.0	Simple chronic bronchitis
491.1	Mucopurulent chronic bronchitis
491.20	Obstructive chronic bronchitis without exacerbation
491.21	Obstructive chronic bronchitis with acute exacerbation
491.22	Obstructive chronic bronchitis with acute bronchitis
491.8	Other chronic bronchitis
491.9	Unspecified chronic bronchitis
492.0	Emphysematous bleb
492.8	Other emphysema
493.22	Chronic obstructive asthma with acute exacerbation
496	Chronic airway obstruction, not elsewhere classified
518.81	Acute respiratory failure
518.82	Other pulmonary insufficiency not elsewhere classified
518.84	Acute and chronic respiratory failure
799.1	Respiratory arrest

All ICD-9-CM codes used to identify patients hospitalized for AE-COPD in the four coding algorithms examined are included. AE-COPD = acute exacerbation of COPD; ICD-9-CM = International Classification of Disease, Ninth Edition, Clinical Modification.

Patient demographics and outcomes (ie, length of stay [LOS], in-hospital mortality) were recorded. LOS was divided into quartiles based on the overall study population (25th quartile, 2 days; median, 4 days; 75th quartile, 6 days) and compared across algorithms. We reviewed physician documentation and test results in written and electronic medical records at the time of the index hospitalization to determine whether spirometry had been performed within a 2-year period prior to the index hospitalization; we selected a 2-year window because spirometry is recommended every 1 to 2 years.⁹ A waiver of informed consent and Institutional Review Board approval for the study was obtained at each institution (protocol numbers 2007-0237 and 15341B).

Analysis

Sampling weights based on the stratified probability design were used for descriptive statistics (ie, means, proportions), calculations of test performance (ie, sensitivity, specificity, positive predictive value [PPV], negative predictive value [NPV]), and comparisons across algorithms. The sensitivity, specificity, PPV, and NPV of ICD-9-CM algorithms for AE-COPD were based on comparisons with the reference standard for AE-COPD. A logistic regression model was used to determine whether the ability of ICD-9 algorithms to identify AE-COPD (as defined by the reference standard [yes vs no]) was modified by age (≥ 65 years vs < 65 years), LOS (≥ 4 days vs < 4 days), study site, and comorbid heart failure (based on abstracted physician diagnosis). For these analyses, interaction terms were used. Comparisons between the test characteristics of the four algorithms used bootstrapped SEs. All reported P values are two sided, and P ≤ .05 was considered statistically significant. Analyses were performed using STATA, release 10.0 (StataCorp Inc) software.

Results

Patient Characteristics

Of the 49,239 patients (hospital 1, 28,643; hospital 2, 20,596) admitted during the study period, 8,790 met eligibility criteria (hospital 1, 5,602; hospital 2, 3,188). Based on the reference standard, the prevalence (using sampling weights) of hospitalizations for AE-COPD was 7.9% (Table 2). As expected, the prevalence of AE-COPD in stratum 1 (81.2%) was higher than in stratum 2 (26.0%) or stratum 3 (4.0%). Only 20.8% of patients hospitalized for AE-COPD had a documented spirometry in the previous 2 years.

Table 2.

—Patient Characteristics

	All Patients	Not AE-COPD Based on Reference Standard	AE-COPD Based on Reference Standard
Weighted No. (%)	8,790	8,099 (92.1)	691 (7.9)
Patient characteristics
Age, y	56.1 (52.8, 59.4)	55.5 (52.0, 59.0)	62.9 (55.7, 70.1)
Female, %	49.3 (40.2, 58.4)	47.8 (38.2, 57.5)	66.4 (45.6, 87.3)
Length of Stay, d
≤ 2	33.7	33.8	33.4
3-4	23.0	23.8	14.5
5-6	20.8	20.8	21.4
> 6	22.4	21.7	30.7
Spirometrya	9.9 (6.0, 15.8)	9.0 (4.0, 14.0)	20.8 (3.7, 38.0)
In-hospital mortality	0.9 (0.1, 2.0)	0.1 (0, 1.7)	1.7 (0, 4.2)

Data are presented as mean (95% CI) or % unless otherwise indicated. Weighted estimates using the stratified probability sample. The AE-COPD reference standard was based on (1) physician documentation of a COPD diagnosis; (2) cough, dyspnea, or sputum production on presentation; and (3) hospitalization for one of these respiratory symptoms. See Table 1 legend for expansion of abbreviation.

^aSpirometry performed in past 2 y.

Sensitivity and NPV

Overall, the sensitivity of the ICD-9-CM coding algorithms was low, regardless of the algorithm used, ranging from ∼ 12% (algorithm 4 [single primary AE-COPD code]) to ∼ 25% (algorithm 1 [expanded COPD algorithm]) (Fig 2). The sensitivity was highest in algorithms that used a combination of primary and secondary discharge codes (algorithms 1 and 2). The NPV was > 90% for all algorithms. Although some differences in the NPV among algorithms were statistically significant, these differences were small (< 1%). Sex, age, LOS, study site, and presence of comorbid heart failure had no effect on the sensitivity or NPV (data not shown).

Figure 2.

Test characteristics of ICD-9-CM algorithms for identifying AE-COPD. Algorithms 1 (expanded COPD algorithm), 2 (primary COPD or respiratory failure codes), 3 (multiple primary COPD codes), and 4 (single primary AE-COPD code) are shown. Records were weighted by 1/probability of being sampled. The circle or square represents the estimated test characteristic, and whiskers represent the 95% CI. The 95% CI for specificity ranged between 99% and 100% for all algorithms. All pairwise comparisons of sensitivity between algorithms were statistically significant (P < .05), except for algorithms 1 vs 2 (P = .40). All differences in the NPV between algorithms were significant (P < .05) except for the difference between algorithms 1 and 2 (P = .51). Differences in specificity were significant for algorithms 1 vs 3, algorithm 1 vs 4, and algorithm 2 vs 4 (all P < .05); other pairwise comparisons were not significant. The PPV of algorithm 4 was significantly better than that of algorithm 1 (P = .01) and algorithm 2 (P = .04); other pairwise comparisons were not significant. NPV = negative predictive value; PPV = positive predictive value. See Figure 1 legend for expansion of other abbreviation.

Specificity and PPV

Specificity was high for all algorithms at > 99%. Although some differences in specificity among algorithms were statistically significant, these differences were small (< 0.5%) and, therefore, of doubtful importance. The PPV varied significantly (81.2%-97.2%) and was highest in algorithm 4 (single primary AE-COPD code). Sex, age, LOS, study site, and presence of comorbid heart failure had no effect on the specificity or PPV (data not shown).

Test Characteristics Using Restrictive Reference Standard

The prevalence of AE-COPD dropped from 7.9% to 0.4% after applying the more restrictive reference standard because only 37 of the 691 hospitalizations identified as AE-COPD using the original reference standard met the more narrowly defined criteria for AE-COPD. The sensitivity of all algorithms greatly increased (eg, 100% in algorithm 1), and the PPV of all algorithms dropped substantially (eg, 14.4 in algorithm 2) (Fig 3). The NPV and specificity of all algorithms remained high (all > 98%).

Figure 3.

Test characteristics of ICD-9-CM algorithms for identifying AE-COPD (restrictive reference standard). Algorithms 1 (expanded COPD algorithm), 2 (primary COPD or respiratory failure codes), 3 (multiple primary COPD codes), and 4 (single primary AE-COPD code) are shown. Records were weighted by 1/probability of being sampled. The circle or square represents the estimated test characteristic, and whiskers represent the 95% CI. The 95% CI ranged from 98% to 99% for specificity and 99% to 100% for NPV for all algorithms. See Figure 1 and 2 legends for expansion of abbreviations.

Discussion

About one in 13 hospitalized patients in this study (7.9%) had AE-COPD. We demonstrated that algorithms based on ICD-9-CM codes vary in their ability to distinguish patients admitted for AE-COPD from those who are not. The sensitivity of all four ICD-9-CM algorithms was poor, but they were generally better if a combination of primary and secondary diagnosis codes were used. The PPVs varied across algorithms and favored the approach that relies on a primary diagnosis of COPD exacerbation (algorithm 4). The specificity and NPV of all algorithms were high.

We are not aware of other studies that have evaluated the sensitivity and specificity of ICD-9-CM algorithms for hospitalizations for AE-COPD. Because the health and economic burden of COPD are based largely on ICD-9-CM codes in administrative data, the low sensitivity of all four algorithms we examined indicates that we may be substantially underestimating the burden of hospitalizations for AE-COPD.¹ Additional studies are needed to develop and test the feasibility and validity of other approaches (eg, extracting clinical information from narratives contained in medical records) for identifying patients hospitalized for AE-COPD.²⁴

A previous study in two EDs examined the PPV of ICD-9-CM codes for identifying AE-COPD and found that the overall PPV was extremely high at 97.5% but ranged from 60% to 75% when codes outside 491.2x (Table 1) were examined.²⁰ The present findings were consistent with these findings because the PPV for algorithm 4 (single primary AE-COPD code) was 97.2% but was lower for multicode algorithms. The relatively low PPV for algorithms based on other codes for COPD (only ∼ 80% or modestly better) have important implications for quality improvement initiatives and observational CER that rely on ICD-9-CM algorithms. The current findings suggest that roughly 15% of hospitalizations identified by algorithm 3 (primary code of 491.x, 492.x, or 496), an algorithm presently used to identify hospitalized patients for performance measurement, will be a false-positive result.⁷ Thus, performance measures based on this algorithm will lead to an underestimation of the quality of care for patients with AE-COPD. In the present study, for example, false-positive results based on algorithm 3 were due to hospitalizations for acute viral bronchitis in patients without a history of COPD and for patients with obliterative bronchiolitis. We found that algorithm 4 (single primary AE-COPD code 491.21) offers the highest PPV (97%) for identifying hospitalizations for AE-COPD. If confirmed in other studies with a broader set of hospitals, we would recommend using algorithm 4 in quality improvement studies and CER.

The present study has some important limitations. First, it was performed at two university-affiliated teaching hospitals in Chicago, Illinois, so the results may not be applicable to all inpatient settings (eg, community hospitals or hospitals in other regions). It is encouraging, however, that we did not find significant differences in the validity of the ICD-9-CM coding algorithms between study sites (eg, sensitivity of algorithm 1 [expanded COPD algorithm] in hospital 1 vs hospital 2, 25.9% vs 22.3%; specificity of algorithm 1 in hospital 1 vs hospital 2, 99.4% vs 99.7%). Nevertheless, we acknowledge that a large multicenter study based on chart abstraction would help to confirm our findings.

Second, we did not require evidence of airflow obstruction based on spirometry to establish a diagnosis of COPD. However, even with a 2-year window prior to the index hospitalization, we found that spirometry rarely had been performed (Table 2). Thus, requiring spirometry data to confirm the diagnosis of COPD would have led to a narrowly selected (and potentially biased) sample with which to evaluate the validity of ICD-9-CM algorithms. We also were aware that our reference standard may have included patients with COPD who were hospitalized for an alternative cause of respiratory symptoms. To limit the possibility of misclassification, we repeated the analysis using a restrictive reference standard and reclassified all patients with other cardiopulmonary conditions as not AE-COPD. As expected when using this restrictive reference standard, the prevalence of AE-COPD dropped, the sensitivity improved, and the PPV dropped. Another option to limit the risk of misclassification would have been to exclude patients with comorbid cardiopulmonary conditions all together. However, both approaches are problematic because they ignore the substantial number of patients with COPD and other cardiopulmonary disorders with AE-COPD.^17,25 Additional studies are needed to determine the optimal approach to identifying AE-COPD in patients with multiple cardiopulmonary disorders. Third, it is possible that these test characteristics operate differently in patients identified as having COPD in outpatient encounters who are subsequently hospitalized with a respiratory exacerbation. Finally, we tested only four coding algorithms, and findings may not generalize to other ICD-9 algorithms.

In summary, using ICD-9-CM coding algorithms to identify hospitalizations for AE-COPD based solely on hospitalization records appears to be imperfect; however, until another approach is available, the present findings provide an evidence-based approach for identifying hospitalizations for AE-COPD. We have demonstrated that use of ICD-9-CM codes will greatly underestimate the burden of hospitalizations for COPD. Further, there are substantial differences in the PPVs of different ICD-9-CM algorithms. Until larger-scale, multicenter studies are conducted, we advocate the use of algorithm 4 (single primary AE-COPD code) in quality improvement initiatives and observational CER studies targeting patients hospitalized for AE-COPD.

Abbreviations

AE-COPD

acute exacerbations of COPD

CER

comparative effectiveness research

ICD-9-CM

International Classification of Diseases, Ninth Revision, Clinical Modification

LOS

length of stay

NPV

negative predictive value

PPV

positive predictive value