Abstract
Background
Administrative claims data are increasingly being used in public reporting of provider performance and health services research. However, the concordance between administrative claims data and the clinical record in lower extremity total joint arthroplasty (TJA) is unknown.
Questions/purposes
We evaluated the concordance between administrative claims and the clinical record for 13 commonly reported comorbidities and complications in patients undergoing TJA.
Methods
We compared 13 administratively coded comorbidities and complications derived from hospital billing records with clinical documentation from a consecutive series of 1350 primary and revision TJAs performed at three high-volume institutions during 2009.
Results
Concordance between administrative claims and the clinical record varied across comorbidities and complications. Concordance between diabetes and postoperative myocardial infarction was reflected by a kappa value > 0.80; chronic lung disease, coronary artery disease, and postoperative venous thromboembolic events by kappa values between 0.60 and 0.79; and for congestive heart failure, obesity, prior myocardial infarction, peripheral arterial disease, bleeding complications, history of venous thromboembolism, prosthetic-related complications, and postoperative renal failure by kappa values between 0.40 and 0.59. All comorbidities and complications had a high degree of specificity (> 92%) but lower sensitivity (29%–100%).
Conclusions
The data suggest administratively coded comorbidities and complications correlate reasonably well with the clinical record. However, the specificity of administrative claims is much higher than the sensitivity, indicating that comorbidities and complications coded in the administrative record were accurate but often incomplete.
Level of Evidence
Level II, prognostic study. See the Guidelines for Authors for a complete description of levels of evidence.
Introduction
As the demand for total joint arthroplasty (TJA) continues to escalate, the aging TJA patient population in the United States presents with a complex array of comorbidities that can profoundly impact the success of TJAs [6]. Certain comorbidities such as diabetes, congestive heart failure, and ischemic heart disease are known to influence the risk of postoperative morbidity and mortality in patients undergoing TJA [1, 6, 7]. Because administrative claims databases are increasingly used in health services research, public reporting of provider performance, and value-based payment strategies, it is imperative to ensure the comorbid conditions and complications reported in administrative claims databases are both accurate and complete. Inaccurate claims related to postoperative complications (eg, postoperative bleeding and infection) could unfairly characterize the performance of certain providers by overstating their rate of complications. Similarly, undercoding of certain comorbid conditions (eg, diabetes and obesity) could lead to inadequate risk adjustment, which could adversely impact the performance ratings of providers who care for the most complex patients who are at the highest risk for complications.
Administrative claims are coded manually by nonclinical (administrative) personnel, and thus the clinical accuracy and legitimacy of administrative claims data have been called into question. Previous investigators have examined the correlation between administrative databases and clinical records in both orthopaedic and nonorthopaedic settings. There is some evidence to support the claim that administrative data can be used to accurately reflect clinical outcomes [3, 5, 11] for certain clinical conditions. However, despite increased reliance on administrative claims data for health services research, public reporting of provider performance, and value-based payment strategies, the concordance between administrative claims and the clinical record for TJAs remains unknown.
We therefore evaluated the concordance between administrative claims and the clinical record for 13 commonly reported comorbidities and complications in patients undergoing TJA.
Patients and Methods
We compared 13 commonly reported comorbidities and complications derived from hospital administrative claims data with clinical records from all 1350 primary and revision TJAs, including 548 primary THAs, 513 primary TKAs, 129 revision THAs, and 160 revision TKAs performed at three high-volume TJA centers (University of California, San Francisco, San Francisco, CA, USA; Massachusetts General Hospital, Boston, MA, USA; Newton-Wellesley, Newton, MA, USA). We used a series of codes from The International Classification of Diseases, 9th Revision, Clinical Modification comorbidity (Table 1) and complication codes (Table 2).
Table 1.
ICD-9-CM comorbidity codes included in this study
| ICD-9-CM codes | Code description |
|---|---|
| 250.00–250.93 | Diabetes mellitus |
| 278.00, 278.01, v85.3, v85.4 | Morbid obesity, obesity not otherwise specified |
| 412 | Prior myocardial infarction (MI) |
| 414.00, 414.01, 414.06 | Coronary artery disease (CAD) |
| 428.0 | Congestive heart failure |
| 440.1, 440.20–440.24, 440.29, 440.30–440.32, 440.4, 440.8, 440.9, 441.00–441.03, 441.3–441.7, 441.9, 442.3, 442.82–442.84, 442.89, 443.0, 443.1, 443.2–443.24, 443.29, 443.81, 443.82, 443.89, 443.9, 440.0, 444.1, 444.21, 444.22, 444.81, 444.89, 445.01, 445.02, 445.81 | Peripheral arterial disease |
| v12.51 | History of thromboembolism |
| 490, 491.0, 491.1, 491.20–491.9, 492.0–492.8, 493.00–493.92, 494.0, 494.1, 495.0–495.9, 496 | Chronic lung disease |
ICD-9-CM = International Classification of Diseases, 9th Revision, Clinical Modification.
Table 2.
ICD-9-CM complication codes included in this study
| ICD-9-CM code | Code description |
|---|---|
| 410.00–410.02, 410.10–410.12, 410.20–410.22, 410.30–410.32, 410.40–42.12, 410.50–410.52, 410.60–410.62, 410.70–410.72, 410.80–410.82, 410.90–410.92 | Postoperative myocardial infarction |
| 415.11, 415.12, 415.19 | Postoperative pulmonary embolism |
| 428.0, 428.1, 428.20–428.23, 428.30–428.33, 428.40–428.43, 428.9 | Postoperative heart failure |
| 453.41, 453.51 | Postoperative proximal deep vein thrombosis |
| 453.40–453.42, 453.50–453.52, 453.9 | Postoperative deep vein thrombosis or pulmonary embolism |
| 584.5–584.9, 585.1–585.9, 586 | Postoperative renal failure |
| 682.6, 686.9, 835.00–835.13, 996.41, 996.42–996.44, 996.60, 996.66, 996.67, 998.31, 998.32, 998.51, 998.59, 998.6, 998.7, 998.83, 998.89 | Prosthetic-related complications |
| 998.1.1, 998.12, 998.13, 719.1 | Bleeding complications |
| 957.8, 957.9, 956.0, 956.1, 956.8, 956.9 | Postoperative nerve injury |
ICD-9-CM = International Classification of Diseases, 9th Revision, Clinical Modification.
These procedures were consecutive and performed between January 1, 2009 and December 31, 2009 at the three institutions. The procedures were first identified from the administrative record and then clinical documentation was retrieved for each patient record. Data were abstracted from the clinical record based on operative notes and discharge summaries by fellowship-trained orthopaedic surgeons (RKB, SA, BS). Administratively coded comorbidity and complication codes were assigned by medical coders, who assign codes from their interpretation of the inpatient and outpatient medical records, including office notes, daily progress notes, operative notes, and discharge summaries. These administrative codes are used for hospital billing purposes to support the level of complexity of the services provided to the patient. Concordance between administrative codes and clinical documentation was examined using four different measures of concordance for each comorbidity and complication: (1) sensitivity (the proportion of comorbidities and complications from the clinical notes that have a corresponding administratively coded comorbidity and complication code); (2) specificity (the proportion of true-negative comorbidities or complications from the clinical notes that do not have a corresponding administratively coded comorbidity or complication codes); (3) positive predictive value (the proportion of administrative claims that correctly identified a particular comorbidity or complication); and (4) the kappa statistic (used to describe the degree of concordance between the clinical and administrative record with established grading criteria).
Results
Concordance between the administrative and clinical record varied across comorbidities and complications. For comorbidities, the kappa value for diabetes was 0.86; between 0.73 and 0.75 for chronic lung disease and coronary artery disease; 0.47 and 0.59 for congestive heart failure, obesity, prior myocardial infarction, and peripheral artery disease; and 0.46 for history of thromboembolism (Table 3). For complications, the concordance was 0.86 for postoperative myocardial infarction; 0.60 for postoperative any deep vein thrombosis/pulmonary embolism; 0.45 to 0.46 for bleeding complications and prosthetic-related complications; 0.37 for postoperative renal failure; and < 0.09 for postoperative pulmonary embolism, postoperative proximal deep vein thrombosis, postoperative nerve injury, and postoperative heart failure (Table 4).
Table 3.
Concordance of comorbidity codes
| Comorbidity | Kappa value | Sensitivity | Specificity | Positive predictive value | Percent of cases |
|---|---|---|---|---|---|
| Diabetes | 0.86 | 86.5% | 98.7% | 89.4% | 11.2 |
| Chronic lung disease | 0.75 | 76.9% | 96.7% | 80.8% | 14.7 |
| Coronary artery disease | 0.73 | 76.8% | 97.2% | 73.8% | 9.6 |
| Congestive heart failure | 0.59 | 94.4% | 98.4% | 43.6% | 2.9 |
| Obesity | 0.55 | 76.4% | 91.8% | 49.2% | 14.6 |
| Prior myocardial infarction | 0.51 | 54.8% | 98.3% | 51.1% | 3.3 |
| Peripheral arterial disease | 0.47 | 50.0% | 99.2% | 45.0% | 1.5 |
| History of thromboembolism | 0.46 | 86.4% | 96.9% | 31.1% | 4.5 |
Table 4.
Concordance of complication diagnosis codes
| Complication | Kappa value | Sensitivity | Specificity | Positive predictive value | Percent of cases |
|---|---|---|---|---|---|
| Postoperative myocardial infarction | 0.86 | 100.0% | 99.9% | 75.0% | 0.3 |
| Postoperative any deep vein thrombosis/pulmonary embolus | 0.60 | 85.7% | 99.5% | 46.2% | 1.0 |
| Bleeding complications | 0.46 | 37.5% | 99.7% | 60.0% | 0.7 |
| Prosthetic-related complications | 0.45 | 29.4% | 100.0% | 100.0% | 0.4 |
| Postoperative renal failure | 0.37 | 33.3% | 99.2% | 47.6% | 1.6 |
| Postoperative pulmonary embolism | 0.00 | 0.00% | 99.8% | 0.0% | 0.2 |
| Postoperative proximal deep vein thrombosis | 0.00 | 0.00% | 100.0% | 0.0% | 0.0 |
| Postoperative nerve injury | 0.00 | 0.00% | 0.00% | 0.0% | 0.1 |
| Postoperative heart failure | 0.09 | 5.9% | 99.8% | 25.0% | 0.3 |
All comorbidities and complications had a high degree of specificity (> 92%) but lower sensitivity (29%–100%), indicating the comorbidities and complications coded in the administrative record were accurate but often incomplete.
Discussion
Epidemiological data regarding comorbidities and surgical complications, which are used in health services research and public reporting of provider performance, are commonly derived from administrative claims data. Previous investigators have attempted to evaluate the concordance between clinical and administrative claims data. Daneshvar et al. [3] were able to show a high level of concordance between International Classification of Diseases codes and clinical data when attempting to simply identify the type of procedure (THA, TKA, revision THA, revision TKA) performed. Stickler et al. [11] reported an 85% sensitivity rate when examining International Classification of Diseases coding and clinical data for deaths resulting from 270 known amyotrophic lateral sclerosis cases. Recently, Furlan and Fehlings [5] calculated an accuracy of 87% when examining coding data for 92 patients with spine trauma who were admitted from a single institution. However, when the same database was queried for a more specific injury (complete spinal cord injury), accuracy dropped to 32.6%. Similarly, we found a high degree of concordance between the clinical and administrative records for certain comorbidities (diabetes, chronic lung disease, and coronary artery disease) and complications (postoperative myocardial infarction) in patients undergoing TJA but only moderate concordance for other comorbidities (congestive heart failure, obesity, prior myocardial infarction, peripheral artery disease, and a history of thromboembolism) and complications (deep vein thrombosis/pulmonary embolism, bleeding complications, and prosthetic-related complications).
Our findings are subject to certain limitations. First, data were only abstracted from high-volume, academic institutions, and therefore, our results may not be generalizable to lower-volume community hospitals. However, no attempt was made to train the administrative coders with respect to the purpose of our study, and therefore, we believe our findings reflect the current state of administrative coding in academic hospitals. Second, we chose to evaluate diagnosis codes that are known to influence clinical outcomes. Therefore, it is possible that the level of concordance may not be as great for less clinically relevant outcomes (eg, electrolyte imbalance). Finally, we only evaluated concordance between the administrative and clinical records during a single calendar year; further study would be necessary to determine if concordance improves over time after education of clinical and administrative coding personnel on the importance of accurate clinical documentation and coding.
In 2002, Mears et al. [8] studying a population of 100 consecutive primary THAs performed by two surgeons at a single institution reported that comorbidities for THA were being undercoded at a rate of 33.5%. Fox et al. [4] examined the accuracy of administrative claims in 343 hip fracture cases from eight hospitals with a high volume of hip fractures and found a sensitivity of 66.7% and a specificity of 78.9% when comparing identification of postoperative complications between the administratively coded data set and detailed chart review. Romano et al. [10] reported hospital-recorded International Classification of Diseases, 9th Revision, Clinical Modification data captured only 37% and 4% of mild and severe complications, respectively, in 991 randomly sampled surgical patients who underwent elective lumbar discectomies at 30 nonfederal acute care hospitals in California. Another study from the same investigators examined comorbid conditions in patients undergoing coronary artery bypass grafting and found that when relative risk was calculated from administrative coding data, there was a discrepancy ranging from 80% underestimation to 35% overestimation for variable comorbidities when compared with actual clinical data [9].
To make meaningful clinical and policy decisions using administrative claims data, it is imperative comorbidities in patients undergoing TJA and the complications associated with TJAs be accurately coded in administrative claims. We found administratively coded comorbidities and complications have reasonable concordance with the clinical record. However, the specificity of administrative claims is much higher than the sensitivity, which suggests administratively coded comorbidities and complications are highly accurate but often incomplete. These findings have important implications for using administrative claims data to assess provider performance and for value-based payment strategies such as pay-for-performance and bundled payments. These payment strategies modify payments to hospitals and physicians based on quality, rather than quantity, of care provided. Our findings suggest public databases that report hospital and surgeon outcomes (such as hospital www.compare.gov and www.physician.gov) likely underreport postoperative complications. However, they also suggest certain comorbid conditions, including peripheral arterial disease and prior myocardial infarction, are underreported, which could influence risk adjustment models and unfairly penalize hospitals and surgeons who treat more medically complex patients who are at increased risk for postoperative morbidity and mortality [1, 2].
Additional efforts should be focused on ensuring clinically relevant comorbidities and complications are routinely captured in the administrative record. These additional efforts could include more precisely defined diagnostic criteria for comorbidities and complications, incentives to motivate and educate clinicians to document the baseline comorbidities in their patients and the adverse consequences of their treatments, and additional clinical training for administrative coding personnel. These steps will be important because there is increased emphasis placed on public reporting of provider performance and payment systems that incentivize and reward quality over quantity of services provided.
Acknowledgments
We thank Mr Michael P. Fleming for his assistance with data collection.
Footnotes
Financial support was received from the Orthopaedic Research and Education Foundation. One author (HER) receives royalties from Zimmer, Inc. (Warsaw, IN, USA).
All ICMJE Conflict of Interest Forms for authors and Clinical Orthopaedics and Related Research editors and board members are on file with the publication and can be viewed on request.
Each author certifies that the institution where the work was performed approved the human protocol for this investigation and that all investigations were conducted in conformity with ethical principles of research.
This work was performed at University of California, San Francisco, San Francisco, CA, USA, and Massachusetts General Hospital, Boston, MA, USA.
References
- 1.Bozic KJ, Lau E, Kurtz S, Ong K, Berry DJ. Patient-related risk factors for postoperative mortality and periprosthetic joint infection in Medicare patients undergoing TKA. Clin Orthop Relat Res. 2012;470:130–137. doi: 10.1007/s11999-011-2043-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Bozic KJ, Lau E, Kurtz S, Ong K, Berry DJ. Patient-related risk factors for periprosthetic joint infection and postoperative mortality following total hip arthroplasty in Medicare patients. J Bone Joint Surg Am. 2012; doi:10.2106/JBJS.K.00072 [Epub ahead of print]. [DOI] [PubMed]
- 3.Daneshvar P, Forster AJ, Dervin GF. Accuracy of administrative coding in identifying hip and knee primary replacements and revisions. J Eval Clin Pract. 2011; doi:10.1111/j.1365-2753.2010.01622.x [Epub ahead of print]. [DOI] [PubMed]
- 4.Fox KM, Reuland M, Hawkes WG, Hebel JR, Hudson J, Zimmerman SI, Kenzora J, Magaziner J. Accuracy of medical records in hip fracture. J Am Geriatr Soc. 1998;46:745–750. doi: 10.1111/j.1532-5415.1998.tb03810.x. [DOI] [PubMed] [Google Scholar]
- 5.Furlan JC, Fehlings MG. The National Trauma Registry as a Canadian spine trauma database: a validation study using an institutional clinical database. Neuroepidemiology. 2011;37:96–101. doi: 10.1159/000330835. [DOI] [PubMed] [Google Scholar]
- 6.Kurtz SM, Ong KL, Schmier J, Mowat F, Saleh K, Dybvik E, Karrholm J, Garellick G, Havelin LI, Furnes O, Malchau H, Lau E. Future clinical and economic impact of revision total hip and knee arthroplasty. J Bone Joint Surg Am. 2007;89(Suppl 3):144–151. doi: 10.2106/JBJS.G.00587. [DOI] [PubMed] [Google Scholar]
- 7.Lai K, Bohm ER, Burnell C, Hedden DR. Presence of medical comorbidities in patients with infected primary hip or knee arthroplasties. J Arthroplasty. 2007;22:651–656. doi: 10.1016/j.arth.2006.09.002. [DOI] [PubMed] [Google Scholar]
- 8.Mears SC, Bawa M, Pietryak P, Jones LC, Rajadhyaksha AD, Hungerford DS, Mont MA. Coding of diagnoses, comorbidities, and complications of total hip arthroplasty. Clin Orthop Relat Res. 2002;402:164–170. doi: 10.1097/00003086-200209000-00014. [DOI] [PubMed] [Google Scholar]
- 9.Romano PS, Roos LL, Luft HS, Jollis JG, Doliszny K. A comparison of administrative versus clinical data: coronary artery bypass surgery as an example. Ischemic Heart Disease Patient Outcomes Research Team. J Clin Epidemiol. 1994;47:249–260. doi: 10.1016/0895-4356(94)90006-X. [DOI] [PubMed] [Google Scholar]
- 10.Romano PS, Schembri ME, Rainwater JA. Can administrative data be used to ascertain clinically significant postoperative complications? Am J Med Qual. 2002;17:145–154. doi: 10.1177/106286060201700404. [DOI] [PubMed] [Google Scholar]
- 11.Stickler DE, Royer JA, Hardin JW. Accuracy and usefulness of ICD-10 death certificate coding for the identification of patients with ALS: results from the South Carolina ALS Surveillance Pilot Project. Amyotroph Lateral Scler. 2012;13:69–73. doi: 10.3109/17482968.2011.614253. [DOI] [PubMed] [Google Scholar]