A recent analysis of the Nationwide Inpatient Sample (NIS) found that inpatient echocardiography (echo) was associated with improved survival but was used only infrequently in clinical situations where an echo is often indicated.1 One potential limitation of this study was highlighted by an internal validation which showed that echo utilization rates were much higher at the author’s hospital than reported in NIS. For example, among patients with myocardial infarction, validation rates were 75% vs 6.3% for the NIS. Given this discrepancy, we sought to determine the accuracy of claims data, such as the NIS, for quantifying echo use. We hypothesized that the International Classification Disease-9 Clinical Modification (ICD-9 CM) code for echo (88.72) would show poor sensitivity for echo utilization, and that actual echo utilization rates would be higher than reported in NIS.
We used the Premier Healthcare Informatics (Charlotte, NC) dataset that includes data on a geographically and structurally diverse group of hospitals from 2014 to assess inpatient echo across eight broad patient conditions for which an echo is commonly ordered. Unlike NIS which reports only standard demographic data and ICD-9 procedure and diagnostic codes, Premier draws from hospital internal cost-accounting systems to record date-stamped hospital service codes for medications, procedures, diagnostic tests, and therapeutic services, including echo. Premier can also distinguish between the types of echo procedures (transthoracic, stress, and transesophageal), thus providing more detail than ICD-9 coding alone. Premier has been previously utilized in studies about echo.2, 3
Using the Clinical Classification Software diagnostic groups, we included patients with sepsis, valvular disease, acute myocardial infarction, atherosclerosis, non-specific chest pain, non-hypertensive heart failure, arrhythmias, and acute cerebrovascular disease.1 We then determined sensitivity and specificity of the ICD-9 code using Premier service codes as the gold standard and calculated 95% asymptotic confidence intervals via Wald test. Jeffreys interval was computed for percentages. We used SAS Version 9.4, SAS Institute, Cary, NC. Baystate Medical Center Institutional Review Board approved this study and waived the requirement for informed consent.
At 518 hospitals, we included 825,908 admissions. (Table) Using Premier codes, we found that echo use ranged from 35% (sepsis) to 74% (valve disease). The rate of transthoracic echo use was highest among patients with acute cerebrovascular disease (70.4%) and acute myocardial infarction (70.2%). Stress echo use was low and mostly utilized among patients with non-specific chest pain (4.4%). Patients with valve disease were the most frequent group to have a transesophageal echo (48.5%) as well as a repeat transthoracic echo during the same admission (20.8%). Consistent with data from NIS, we found that the ICD-9 procedure code recorded low echo utilization across all 8 patient conditions. Overall sensitivity of the ICD-9 code was low at 16.2 (range 12.5% to 44.3%) while specificity was high at 97.5 (range 65.3% to 99%).
Table.
Echocardiography Use, Types, and ICD-9 Code Diagnostic Characteristics among Common Inpatient Clinical Conditions
| CCS Diagnostic Group | Hospitals (N) | Admissions (N) | Echo TTE %, (95% CI) | TEE %, (95% CI) | Stress Echo %, (95% CI) | Admissions with ≥1 TTE, %, (95% CI) | Total Echo %, (95% CI) | ICD-9 echo % (95% CI) | Sensitivity of ICD-9 Code (95% CI) | Specificity of ICD-9 Code (95% CI) |
|---|---|---|---|---|---|---|---|---|---|---|
| All Conditions | 518 | 825,908 | 52.3 | 7.0 | 0.8 | 4.8 | 54.3 | 9.6 | 16.2 | 97.5 |
| (52.18 – 52.40) | (6.96 – 7.08) | (0.79 – 0.73) | (4.72 – 4.81) | (54.19 – 54.40) | (9.57 – 9.70) | (16.0 – 16.3) | (97.47 – 97.57) | |||
| Acute Myocardial Infarction | 497 | 100,051 | 70.2 | 5.4 | 0.8 | 6.9 | 72.3 | 10.0 | 13.2 | 98.1 |
| (69.95 – 70.52) | (5.26 – 5.54) | (0.72 – 0.83) | (6.80 – 7.11) | (71.99 – 72.54) | (9.87 – 10.24) | (12.9 – 13.4) | (97.9 – 98.2) | |||
| Atherosclerosis | 490 | 69,990 | 39.4 | 10.7 | 1.7 | 4.0 | 44.7 | 11.6 | 19.4 | 94.7 |
| (39.03 – 39.76) | (10.51 – 10.96) | (1.65 – 1.85) | (3.90 – 4.19) | (44.33 – 45.06) | (11.37 – 11.85) | (19.0 – 19.9) | (94.5 – 94.9) | |||
| Chest Pain, non-specific | 496 | 50,478 | 45.6 | 0.8 | 4.4 | 1.3 | 47.9 | 6.2 | 12.5 | 99.5 |
| (45.12 – 45.99) | 0.73 – 0.89) | (4.25 – 4.60) | (1.26 – 1.46) | (47.41 – 48.29) | (6.01 – 6.44) | (12.1 – 12.9) | (99.4 – 99.6) | |||
| Heart Failure | 511 | 141,173 | 59.9 | 3.1 | 0.54 | 3.2 | 60.9 | 7.8 | 12.5 | 99.55 |
| (59.65 – 60.17) | (2.98 – 3.16) | (0.50 – 0.58) | (3.11 – 3.29) | (60.59 – 61.10) | (7.62 – 7.90) | (12.2 – 12.7) | (99.49 – 99.60) | |||
| Arrhythmias | 511 | 109,324 | 60.0 | 13.1 | 0.9 | 5.4 | 63 | 13.9 | 21.4 | 98.97 |
| (59.67 – 60.25) | (12.96 – 13.36) | (0.84 – 0.95) | (5.25 – 5.52) | (62.70 – 63.27) | (13.65 – 14.06) | (21.1 – 21.7) | (98.87 – 99.07) | |||
| Sepsis | 516 | 239,194 | 33.9 | 3.4 | 0.16 | 3.6 | 34.5 | 5.0 | 14.3 | 99.9 |
| (33.74 – 34.12) | (3.33 – 3.44) | (0.14 – 0.18) | (3.52 – 3.67) | (34.34 – 34.72) | (4.91 – 5.08) | (14.0 – 14.5) | (99.88 – 99.91) | |||
| Acute Cerebrovascular Disease | 505 | 96,703 | 70.4 | 9.0 | 0.2 | 6.1 | 71.7 | 12.7 | 17.5 | 99.5 |
| (70.13 – 70.71) | (8.79 – 9.15) | (0.18 – 0.23) | (5.97 – 6.28) | (71.38 – 71.95) | (12.48 – 12.90) | (17.2 – 17.8) | (99.45 – 99.61) | |||
| Valvular Disease | 422 | 18,995 | 61.3 | 48.5 | 0.8 | 20.8 | 73.8 | 41.8 | 44.3 | 65.3 |
| (60.57 – 61.95) | (47.81 – 49.23) | (0.70 – 0.95) | (20.2 – 21.36) | (73.21 – 74.46) | (41.07 – 42.48) | (43.5 – 45.1) | (64.0 – 66.6) |
CCS = Clinical Classification Software; Echo = echocardiography; TTE = Transthoracic echo; TEE = Transesophageal Echo; ICD = International Classification of Disease; CI = Confidence Interval
In this large national sample of hospital admissions, we found that echo is commonly used across a wide range of inpatient conditions, and was employed in > 2/3rds of patients with valve disease, myocardial infarction, or cerebrovascular conditions. Moreover, use of transthoracic, stress, transesophageal, and repeat echo matched our general expected patterns of use. We found that the ICD-9 procedure code for echo was highly insensitive to actual echo performance and underreported use by as much as a factor of 7, depending on patient condition. Thus, the inpatient ICD-9 code for echo does not reliably reflect actual echo performance and should not be used to determine the frequency of echo use or any outcomes associated with echo testing.
Although our results differ substantially from research using the NIS,1 our rates of echo use are quite similar to those reported in the single-center validation sub-study. 1 Specifically, echo rates were reported as 75%, 47%, and 35% for patients with myocardial infarction, nonspecific chest pain, and sepsis, respectively. These three results almost exactly match our results from Premier, although agreement was lower in other groups. Our results are also similar to a 78% echo use rate from a general Italian cardiology ward.4 We suspect the reason for the discrepancy between NIS and Premier is that echo procedures do not influence the Diagnosis Related Group and thus do not impact hospital reimbursement and therefore are inconsistently recorded in administrative databases.
Limitations of this study include that hospitals based in the southern US are overrepresented in Premier, although this should not influence reliability of the ICD-9 code. Additionally, neither NIS nor Premier record the appropriate use classifications of each echo, so it is not possible to assess if echo rates reported here are appropriate.
Rather than finding infrequent echo use, we found that echo is commonly used among inpatients with conditions where an echo is often clinically indicated. Additionally, we found that the ICD-9 code for echo is not reliable, which highlights the importance of understanding the limitations of administrative databases and validating diagnostic codes before using them in research studies.5
Data Sharing Statement.
The data, analytic methods, and study materials will not be made available to other researchers for purposes of reproducing the results or replicating the procedure. The authors do not determine access to the Premier database. However, researchers interested in reproducing our results may be able to obtain database access directly from Premier Inc.
Acknowledgments
Funding Sources
Dr. Pack, Dr. Lagu, and Dr. Lindenauer were supported by grants from the National Heart, Lung and Blood Institute of the National Institutes of Health, under award numbers 1K23HL135440, K01HL114745, and 1K24HL132008 respectively.
Footnotes
Conflicts of Interest Disclosures
None for all authors.
References
- 1.Papolos A, Narula J, Bavishi C, Chaudhry FA, Sengupta PP. U.S. Hospital use of echocardiography: Insights from the nationwide inpatient sample. J Am Coll Cardiol. 2016;67:502–511. doi: 10.1016/j.jacc.2015.10.090. [DOI] [PubMed] [Google Scholar]
- 2.Safavi KC, Li SX, Dharmarajan K, Venkatesh AK, Strait KM, Lin H, Lowe TJ, Fazel R, Nallamothu BK, Krumholz HM. Hospital variation in the use of noninvasive cardiac imaging and its association with downstream testing, interventions, and outcomes. JAMA Intern Med. 2014;174:546–553. doi: 10.1001/jamainternmed.2013.14407. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Main ML, Hibberd MG, Ryan A, Lowe TJ, Miller P, Bhat G. Acute mortality in critically ill patients undergoing echocardiography with or without an ultrasound contrast agent. JACC Cardiovasc imaging. 2014;7:40–48. doi: 10.1016/j.jcmg.2013.08.012. [DOI] [PubMed] [Google Scholar]
- 4.Ballo P, Bandini F, Capecchi I, Chiodi L, Ferro G, Fortini A, Giuliani G, Landini G, Laureano R, Milli M, Nenci G, Pizzarelli F, Santoro GM, Vannelli P, Cappelletti C, Zuppiroli A American College of Cardiology F, American Society of E. Application of 2011 American College of Cardiology Foundation/American Society of Echocardiography appropriateness use criteria in hospitalized patients referred for transthoracic echocardiography in a community setting. J Am Soc Echocardiogr. 2012;25:589–598. doi: 10.1016/j.echo.2012.03.006. [DOI] [PubMed] [Google Scholar]
- 5.Khera R, Krumholz HM. With great power comes great responsibility: Big data research from the national inpatient sample. Circ Cardiovasc Qual Outcomes. 2017;10:e003846. doi: 10.1161/CIRCOUTCOMES.117.003846. Epub ahead of print. [DOI] [PMC free article] [PubMed] [Google Scholar]
