Abstract
Objective
To develop algorithms to identify patients with advanced heart failure (HF) that can be applied to administrative data.
Patients and Methods
In a population-based cohort of all residents of Olmsted County, Minnesota, with greater than or equal to 1 HF billing code 2007-2017 (n=8657), we identified all patients with advanced HF (n=847) by applying the gold standard European Society of Cardiology advanced HF criteria via manual medical review by an HF cardiologist. The advanced HF index date was the date the patient first met all European Society of Cardiology criteria. We subsequently developed candidate algorithms to identify advanced HF using administrative data (billing codes and prescriptions relevant to HF or comorbidities that affect HF outcomes), applied them to the HF cohort, and assessed their ability to identify patients with advanced HF on or after their advanced HF index date.
Results
A single hospitalization for HF or ventricular arrhythmias identified all patients with advanced HF (sensitivity, 100%); however, the positive predictive value (PPV) was low (36.4%). More stringent definitions, including additional hospitalizations and/or other signs of advanced HF (hyponatremia, acute kidney injury, hypotension, or high-dose diuretic use), decreased the sensitivity but improved the specificity and PPV. For example, 2 hospitalizations plus 1 sign of advanced HF had a sensitivity of 72.7%, specificity of 89.8%, and PPV of 60.5%. Negative predictive values were high for all algorithms evaluated.
Conclusion
Algorithms using administrative data can identify patients with advanced HF with reasonable performance.
Abbreviations and Acronyms: ED, emergency department; EF, ejection fraction; ESC, European Society of Cardiology; HF, heart failure; NPV, negative predictive value; PPV, positive predictive value; REP, Rochester Epidemiology Project; VA, ventricular arrhythmia
Many patients with heart failure (HF) will progress to having advanced HF, which is characterized by refractory severe symptoms of HF.1,2 Advanced HF is important clinically because patients are at high risk of adverse outcomes and may be eligible for unique treatment options (eg, mechanical circulatory support, cardiac transplantation, palliative inotropes) that can improve the quality of life and survival. Despite the profound impact that advanced HF has on afflicted patients, our understanding of the population impact and characteristics of patients with advanced HF has been limited in large part because there is no one single criterion for advanced HF. Case definitions have historically been quite complex and challenging to be applied to populations.3 Existing data have largely reflected either referral cohorts or clinical trial populations,4, 5, 6 which experience referral and selection bias and comprise primarily younger individuals with very low ejection fraction (EF). There have been no prior studies of advanced HF using administrative data because, until now, there were no validated algorithms to identify patients with advanced HF.
To tackle this problem, we recently took steps to be able to identify patients with advanced HF more easily. First, we leveraged a population-based cohort of all residents of Olmsted County, Minnesota, with HF from 2007 to 2017 and used a multistep process (including electronic health record manual review) to apply the gold standard European Society of Cardiology (ESC) advanced HF criteria.7 This resulted in a comprehensive, population-based cohort of individuals with advanced HF, and we have described their characteristics and outcomes in detail.8 However, this process was not facile or easily reproducible. Our next step was to develop algorithms that could be more easily applied but still accurately identify patients with advanced HF so that we could begin to better understand and treat this population. In this article, we describe our efforts to develop advanced HF algorithms that can be applied to administrative data.
Patients and Methods
This was a population-based retrospective cohort study conducted in Olmsted County, Minnesota, using the resources of the Rochester Epidemiology Project (REP).9 The REP comprises all health care for individuals living in the community and enables comprehensive longitudinal assessment of health care utilization and outcomes for the local population. Patients without Minnesota Research Authorization were excluded from the analysis (1.2% of potential patients excluded). The study was approved by the Mayo Clinic and Olmsted Medical Center Institutional Review Boards.
Development of Candidate Algorithms to Identify Patients With Advanced HF
The goal of this project was to develop algorithms to identify patients with advanced HF that could be applied to administrative data (eg, data that comprise billing codes and pharmacy records). The gold standard definition for advanced HF is from the ESC.7 However, the ESC criteria (described under “Validation Cohort” section) cannot be used to develop algorithms that can be applied to administrative data, as the definition relies on data that are not available in administrative data (and, for which, there are no reasonable administrative surrogates), including echocardiographic data, exercise-related data, and symptoms. Therefore, we used the table “findings useful to identify patients with advanced HF” from the 2013 American College of Cardiology Foundation/American Heart Association HF guidelines to select factors that could be found in administrative data and may be predictive of advanced HF.2 This included hospitalizations or emergency department (ED) visits for HF or ventricular arrhythmia (VA) and 5 other signs of advanced HF such as hyponatremia, hypotension, acute kidney injury/dialysis, use of high-dose loop diuretics, and use of metolazone. All algorithms were defined using International Classification of Diseases, Ninth Revision, or International Statistical Classification of Diseases, Tenth Revision, billing codes and/or prescription medication data (Table 1).10,11 Using this framework, candidate algorithms were constructed, which comprised 1 or more hospitalizations/ED visits for HF/VA combined with additional sign(s) of advanced HF. The 4 base algorithms were 1 hospitalization for HF/VA, 1 hospitalization or ED visit for HF/VA, 2 hospitalizations for HF/VA in a year, and 2 hospitalizations or ED visits for HF/VA in a year. To each of these base models, the requirement for 1, 2, 3, or 4 of any of the additional signs of advanced HF in a year was added, resulting in 20 total algorithms.
Table 1.
Algorithm Definitions
| Criterion | Definition |
|---|---|
| Hospitalization for HF/VA10 or advanced therapies (left ventricular assist device, heart transplant) | ICD-9 or ICD-10 codes in the primary (first) position HF/VA: 428.X, 402.X1, 404.X1, 404.X3, 427.1, 427.4X, 427.5; I11.0, I13.0, I13.2, I50, I47.2, I49.01, I49.02, I46.2, I46.9 Left ventricular assist device: 37.66, PCS 02HA0QZ, 02HA3QZ, 02HA4QZ Heart transplant: 37.51, PCS 02YA0Z0 |
| ED visits for VA10 | ICD-9 or ICD-10 codes in the first position: 427.1, 427.4X, 472.5, I47.2, I49.01, I49.02, I46.2, I46.2, I46.9 |
| ED visits for HF11 | ICD-9 or ICD-10 codes in the first position: 428.X, 402.X1, 404.X1, 404.X3, I11.0, I13.0, I13.2, I50 Abovementioned codes in the second or third position if the first position code was: 276.6, 514, 518.4, 518.81, 518.83, 518.84, 786, 782.3, E8770, E8779, J182, J811, J810, J9600, J9690, J9610, J9620, R069, R0601, R0602, R0600, R0609, R600, R601, R609 |
| Hyponatremia | ICD-9 276.1 or ICD-10 E87.1 |
| Hypotension | ICD-9 458 or ICD-10 I95 |
| Acute kidney injury or dialysis | ICD-9 584.X, ICD-10 N17.X ICD-9 403.01, 403.11, 403.91, 404.02, 404.03, 404.12, 404.13, 404.92, 404.93, 585.5, 585.6, 792.5, V45.1, V45.11, V45.12, V56.X; ICD-10 I12.0, I13.11, I13.2, I95.2, N185, N186, R88.0, T81.502X, T81.512X, T81.522X, T81.532X, T81.592X, T85.611X, T85.621X, T85.631X, T85.651X, T85.71X, Y84.1, Z49, Z49.0, Z49.01, Z49.02, Z49.3, Z49.31, Z49.32, Z91.15, Z99.2; or procedure codes ICD-9 39.95, ICD-10 5A1D70Z, 5A1D80Z, 5A1D90Z |
| High-dose loop diuretic use | Prescription order of total daily dose of at least 80 mg of furosemide, 40 mg of torsemide, or 2 mg of bumetanide |
| Metolazone use | Prescription order of any dose of metolazone |
ED, emergency department; HF, heart failure; ICD-10, International Statistical Classification of Diseases, Tenth Revision; ICD-9, International Classification of Diseases, Ninth Revision; VA, ventricular arrhythmia.
Validation Cohort
Using the resources of the REP, we identified all adult residents of Olmsted County with at least 1 inpatient or outpatient code for HF (ICD-9 428.X, ICD-10 I50.X) from 2007 to 2017. The date of cohort entry was the date of the first code in the study period. We focused on developing and assessing the performance of our advanced HF algorithms in this enriched group with possible HF, rather than all adults in the community, so that the cohort size was manageable. These codes have been found to be sensitive for identifying individuals with HF12 and, therefore, would not hinder our evaluation of algorithms to identify the subset of patients in whom HF is advanced. We recognized, however, that narrowing our population to those with possible HF in this way does decrease the potential performance of the algorithms (in particular, lowering specificity), as all those without an HF code in the community are not included in the denominator.
In this cohort of patients with a diagnostic code for HF, the patients with advanced HF were known, as they had been previously identified by our group using a rigorous process that included manual review of electronic health record data. The process used to identify patients with advanced HF has been previously described8; in brief, we applied the ESC’s advanced HF case definition.7 The ESC case definition requires all of the following within a 12-month period: (1) hospitalizations or ED visits for HF/VA; (2) 1 or more echocardiographic signs of severe HF, including severely reduced EF, significantly reduced right ventricular function, severe inoperable valvular heart disease or congenital heart disease, or significant diastolic dysfunction; (3) severe exercise limitations; and (4) persistent New York Heart Association class III (advanced) or IV symptoms despite attempts to optimize medical, surgical, and device-based therapy. The first hospitalization/ED event in which the patient met all criteria was used as the advanced HF index (diagnosis) date.
For this cohort, data regarding all billing codes, provider-ordered and documented medications, and mortality status were obtained from the electronic health record. Mortality was captured in the REP using deaths noted in clinical care, local death notices, and the State of Minnesota death files that are interrogated quarterly.
Application of the Algorithms to the Cohort
Each of the 20 candidate algorithms was applied to the cohort. For algorithms that required 2 events (hospitalizations or hospitalizations/ED visits) within 12 months, we anchored off a hospitalization or ED event and then searched ±12 months to see whether another qualifying hospitalization or ED event occurred. A similar approach was taken to investigate signs and symptoms of advanced HF (±12 months of anchoring hospitalization or ED event date). In patients with multiple qualifying events, each event was considered separately in evaluating algorithm performance, such that these patients had multiple algorithm performance values; this approach was accounted for in the analysis as described below. Some patients had advanced HF (as defined by the abovementioned ESC criteria) only for a part of the study period. When evaluating algorithm performance, each of the hospitalization/ED visits was considered separately, and the patient was considered to have advanced HF only on or after their advanced HF index date. For example, if a patient had hospitalizations for HF in January 2010, February 2011, and March 2012 and the patient developed advanced HF in March 2012 (according to the ESC criteria), when evaluating the accuracy of the algorithm requiring a single hospitalization for HF/VA, each of those episodes (January 2010, February 2011, and March 2012) would be considered separately. The algorithm would identify them as advanced HF for all 3 episodes but would be accurate only for the March 2012 episode; the patient would not be considered to have advanced HF before that time.
Assessment of Algorithm Performance
We assessed the performance of each of the 20 candidate algorithms by calculating the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). The test outcome (positive or negative) was determined by whether the algorithm identified the patient as having advanced HF. The disease outcome (advanced HF or no advanced HF) was determined by the known advanced HF status defined by manual application of the ESC criteria as described above. Some patients had multiple qualifying hospitalizations/ED events during the study period and, therefore, were evaluated more than once by the algorithm. A naïve approach treating each qualifying hospitalization/ED visit as an independent observation may yield CIs for prediction metrics that are too narrow. We accounted for the correlation of outcomes among repeated measures using a cluster bootstrap.13, 14, 15 We performed 1000 bootstrap replicates with replacement to calculate the metrics reported. Ultimately, the large sample size in the current study yielded narrow CIs. We evaluated the performance of key algorithms stratified by age (<65 years vs ≥65 years), gender (men vs women), and race/ethnicity (non-Hispanic White vs Hispanic and/or non-White patients). All analyses were performed using SAS version 9.4 (SAS Institute Inc).
Results
A total of 8657 residents of Olmsted County had at least 1 HF code during the study period and were included in the cohort. Of those residents, 847 were determined to have advanced HF on the basis of the gold standard ESC criteria and manual chart review. At the time of cohort entry, patients who had or developed advanced HF in the study period were older (mean age, 75.4 vs 73.9 years) and more often men (n=466 [55.0%] vs n=3882 [49.7%], Table 2). Most of the total cohort was White (n=7887 [91.1%]) and non-Hispanic (n=8459 [97.7%]). There were 4941 hospitalizations and 860 ED visits that did not result in hospitalization for the cohort during the study period.
Table 2.
Patient Characteristics at Cohort Entry
| Characteristic | Total population (n=8657) | Had advanced heart failurea (n=847) | Did not have advanced heart failure (n=7810) |
|---|---|---|---|
| Age (y), mean ± SD | 74.1±15.0 | 75.4±14.2 | 73.9±15.0 |
| Female, n (%) | 4309 (49.8) | 381 (45.0) | 3928 (50.3) |
| White, n (%) | 7887 (91.1) | 739 (87.2) | 7148 (91.5) |
| Hispanic, n (%) | 198 (2.3) | 13 (1.5) | 185 (2.4) |
| Renal disease, n (%) | 1673 (19.3) | 250 (29.5) | 1423 (18.2) |
| Chronic obstructive pulmonary disease, n (%) | 2540 (29.3) | 324 (38.3) | 2216 (28.4) |
| Diabetes, n (%) | 2644 (30.5) | 330 (39.0) | 2314 (29.6) |
| Peripheral vascular disease, n (%) | 2250 (26.0) | 276 (32.6) | 1974 (25.3) |
| Cerebrovascular disease, n (%) | 1189 (13.7) | 147 (17.4) | 1042 (13.3) |
| Charlson comorbidity index, median (interquartile range) | 3 (2, 4) | 3 (2, 5) | 3 (1, 4) |
| Body mass index (kg/m2), median (interquartile range) | 28.6 (24.5, 33.8) | 29.0 (24.7, 34.2) | 28.5 (24.5, 33.8) |
Had advanced heart failure in the study period but not necessarily at cohort entry.
The sensitivity, specificity, PPV, and NPV of each of the advanced HF algorithms are shown in Table 3 and the Figure. Given that at least 1 hospitalization for HF/VA is included in the ESC criteria for advanced HF,7 the algorithm that required a single hospitalization for HF/VA identified all patients with advanced HF (sensitivity, 100%; 95% CI, 100%-100%); however, the specificity (66%; 95% CI, 65.9%-66.0%) and PPV (36.4%; 95% CI, 36.3%-36.5%) were low. Expanding the acute event to be either a hospitalization or an ED visit lowered the specificity (59.4%; 95% CI, 59.4%-59.5%) and PPV (33.1%; 95% CI, 33.1%-33.2%) with the same sensitivity. Requiring 2 hospitalizations in 1 year instead of one decreased the sensitivity (74.1%; 95% CI, 74.0%-74.1%) but improved the specificity (89.3%; 95% CI, 89.2%-89.3%) and PPV (59.7%; 95% CI, 59.6%-59.8%). Similarly, requiring patients to have 1 or more signs of advanced HF (hyponatremia, acute kidney injury, hypotension, metolazone use, or high-dose loop diuretic use) in addition to 1 or 2 hospitalizations for HF/VAs decreased the sensitivity but increased the specificity and PPV. The effect on sensitivity, specificity, and PPV increased as more signs of advanced HF were required (ie, 1, 2, 3, or 4). Because advanced HF was relatively uncommon, NPV was high (84.5% or greater) for all algorithms. Algorithm performance was overall similar when stratified by age (<65 years vs ≥65 years), gender (men vs women), and race/ethnicity (non-Hispanic White vs other racial/ethnic groups), with slightly higher sensitivity and lower specificity in those aged <65 years, men, and Hispanic and/or non-White individuals (Supplemental Table, available online at http://www.mcpiqojournal.org).
Table 3.
Algorithm Performance in Identifying Patients With Advanced Heart Failurea
| Algorithm | Sensitivity | Specificity | Positive predictive value | Negative predictive value |
|---|---|---|---|---|
| 1 hospitalization HF/VA | 100% | 66.0% | 36.4% | 100% |
| 1 hospitalization HF/VA + 1 signb | 97.3% | 69.0% | 37.9% | 99.2% |
| 1 hospitalization HF/VA + 2 signsb | 81.5% | 76.7% | 40.5% | 95.5% |
| 1 hospitalization HF/VA + 3 signsb | 55.3% | 87.9% | 47.1% | 91.0% |
| 1 hospitalization HF/VA + 4 signsb | 20.6% | 97.4% | 61.1% | 86.3% |
| 1 hospitalization or ED visit HF/VA | 100% | 59.4% | 33.1% | 100% |
| 1 hospitalization or ED visit HF/VA + 1 signb | 97.3% | 63.5% | 34.8% | 99.1% |
| 1 hospitalization or ED visit HF/VA + 2 signsb | 81.7% | 72.7% | 37.5% | 95.2% |
| 1 hospitalization or ED visit HF/VA + 3 signsb | 55.8% | 85.9% | 44.3% | 90.7% |
| 1 hospitalization or ED visit HF/VA + 4 signsb | 21.1% | 97.0% | 58.5% | 86.0% |
| 2 hospitalizations HF/VAc | 74.1% | 89.3% | 59.7% | 94.1% |
| 2 hospitalizations HF/VAc + 1 signb | 72.7% | 89.8% | 60.5% | 93.9% |
| 2 hospitalizations HF/VAc + 2 signsb | 63.0% | 91.7% | 62.0% | 92.0% |
| 2 hospitalizations HF/VAc + 3 signsb | 44.5% | 94.8% | 64.6% | 88.8% |
| 2 hospitalizations HF/VAc + 4 signsb | 17.8% | 98.7% | 75.1% | 84.9% |
| 2 hospitalizations or ED visit HF/VAc | 78.1% | 85.5% | 54.9% | 94.5% |
| 2 hospitalizations or ED visit HF/VAc + 1 signb | 76.8% | 86.5% | 56.2% | 94.2% |
| 2 hospitalizations or ED visit HF/VAc + 2 signsb | 66.8% | 89.1% | 58.2% | 92.2% |
| 2 hospitalizations or ED visit HF/VAc + 3 signsb | 47.3% | 93.4% | 61.8% | 88.7% |
| 2 hospitalizations or ED visit HF/VAc + 4 signsb | 18.9% | 98.4% | 72.7% | 84.3% |
ED, emergency department; HF, heart failure; VA, ventricular arrhythmia.
Signs of advanced heart failure included hyponatremia, acute kidney injury, hypotension, high-dose loop diuretic use, and metolazone use.
Both events had to occur within a 12-month period.
Figure.
Performance of the algorithms in identifying patients with advanced heart failure. The positive predictive value and sensitivity for selected algorithms are shown. Hosp, hospitalization for heart failure or ventricular arrhythmia; signs of advanced heart failure include hyponatremia, hypotension, acute kidney injury/dialysis, use of high-dose loop diuretics, and use of metolazone.
Discussion
In this study, we found that algorithms relying on administrative data, including billing codes and prescription medication order data, identified patients with advanced HF with reasonable performance. Although a single hospitalization for HF or VA was highly sensitive in identifying patients with advanced HF, it had low specificity and PPV; requiring multiple hospitalizations and/or additional signs of advanced HF within a year increased the specificity and PPV with some loss of sensitivity.
Algorithms are widely applied to administrative data to identify patients with chronic conditions such as HF. Algorithms that can be applied to administrative data are different from those that have been developed to predict mortality in patients with HF, such as the Seattle Heart Failure Model16 and Get with the Guidelines Heart Failure Risk Score,17 as these types of algorithms include variables such as vital signs, laboratories, and EF, which are not available in administrative data. Furthermore, they were not developed or have ever been validated to identify patients with advanced HF, although comparing the performance of these types of algorithms to identify patients with advanced HF may be of interest for a future analysis that leverages electronic health record data. Application of algorithms to administrative data enables us to study a larger, and sometimes more representative, population than would be possible relying on labor-intensive cohort study and registry data. Advanced HF has been quite challenging to study at the population level, given the complexity of its definition(s). The gold standard ESC definition that we applied required manual review of electronic health record data by a clinician to identify patients with advanced HF.7,8 Although we recently described advanced HF in a population-based community cohort, it is imperative to gain a more comprehensive understanding of patients with advanced HF and their outcomes across broader populations. Development of electronic algorithms can help facilitate that goal.
We evaluated the performance of the 20 algorithms that were developed to rely on administrative data. We found that the simplest algorithm, comprising a single hospitalization for HF or VA, was highly sensitive at identifying patients with advanced HF (100%); however, the specificity and PPV (36%) were fairly low. This algorithm had been used, for example, in the Atherosclerosis Risk in Communities cohort study to describe a population with more advanced HF, which they called “stage C2,” and examine their outcomes.18 Our findings suggest that this algorithm can be useful as an initial screening tool when one might not want to miss anybody with advanced HF and/or plans to apply secondary screening criteria such as manual record review to confirm the advanced HF status. This algorithm may be useful, for example, to identify potential participants for advanced HF studies and clinical trials in which secondary screening is implemented before patients are enrolled. Although ED visits for HF/VA have been recommended as a risk marker for advanced HF,7 we found that adding ED visits to the algorithms did not improve their performance.
Requiring 2 hospitalizations for HF or VAs resulted in a large improvement in specificity and PPV (60%), with some decrease in sensitivity (74%). Requiring additional signs of advanced HF such as high-dose diuretic use or acute kidney injury within a 12-month period in addition to hospitalizations resulted in further modest incremental improvements in specificity and PPV, although lowering sensitivity further. On the basis of these data, it would be reasonable to use a definition of either 2 hospitalizations for HF/VA in a year or 2 hospitalizations for HF/VA plus 1 additional sign of advanced HF in a year to identify patients with advanced HF in administrative data in most circumstances in which higher PPV is desirable. These 2 algorithms had high specificity (89%-90%) and reasonable PPV (60%-61%), while still maintaining adequate sensitivity (73%-74%). Because advanced HF is relatively rare in the population, all algorithms evaluated had very high NPV. These algorithms may be useful to identify patients with advanced HF for comparative effectiveness research and other types of secondary data analysis using administrative data in which secondary verification of advanced HF status by manual chart review may be more challenging. Although the average age of our community cohort was 74 years, the algorithms had slightly higher sensitivity and PPV in those aged less than 65 years than in those aged 65 years or older. These findings suggest that these algorithms could potentially be useful to identify younger individuals with advanced HF who are more likely to be eligible for advanced HF therapies.
There are some limitations to acknowledge to aid in the interpretation of these data. First, we developed algorithms guided by clinical findings that are known to be useful in identifying patients with advanced HF, and we restricted to those elements that are available in administrative data. It is possible that algorithms that incorporate a broader range of factors, such as those available in electronic health record data, incorporate natural language processing of text, and/or leverage supervised or unsupervised machine learning techniques may be more accurate at identifying patients with advanced HF; our investigative team would encourage developing such algorithms in the future. However, the simple algorithms validated here have the advantages of being available in numerous settings in the health care system (electronic health record data, insurance claims data), standardized, and easily applied. Second, we validated these algorithms in a population-based community cohort in which all patients with advanced HF were known; the population characteristics of this cohort are like those of living in Minnesota and the Upper Midwest United States.19 It is possible that the algorithms will perform differently in other communities. Although our findings suggest similar algorithm performance in Hispanic and non-White individuals compared with non-Hispanic White individuals, it would be of interest to evaluate the performance of these algorithms in communities of varying racial and ethnic diversity. Overall, we would encourage future studies to validate the performance of these algorithms in other populations.
Conclusion
In conclusion, we developed algorithms that can be used to identify patients with advanced HF in administrative data with reasonable performance. We believe that the application of these algorithms will enhance our knowledge of the population with advanced HF and facilitate the development of interventions to improve care and outcomes.
Footnotes
Grant Support: The work was supported by grant R01-HL144529 (Principal Investigator: S.M.D.) from the National Institutes of Health, and this study was made possible by the Rochester Epidemiology Project (grant number R01-AG034676; Principal Investigators: Walter A. Rocca, MD, MPH, and Jennifer L. St Sauver, PhD).
Potential Competing Interests: The authors report no competing interests.
Supplemental material can be found online at http://www.mcpiqojournal.org. Supplemental material attached to journal articles has not been edited, and the authors take responsibility for the accuracy of all data.
Supplemental Online Material
References
- 1.Fang J.C., Ewald G.A., Allen L.A., et al. Advanced (stage D) heart failure: a statement from the Heart Failure Society of America Guidelines Committee. J Card Fail. 2015;21(6):519–534. doi: 10.1016/j.cardfail.2015.04.013. [DOI] [PubMed] [Google Scholar]
- 2.Yancy C.W., Jessup M., Bozkurt B., et al. 2013 ACCF/AHA guideline for the management of heart failure: a report of the American College of Cardiology Foundation/American Heart Association Task Force on Practice Guidelines. J Am Coll Cardiol. 2013;62(16):e147–e239. doi: 10.1016/j.jacc.2013.05.019. [DOI] [PubMed] [Google Scholar]
- 3.Truby L.K., Rogers J.G. Advanced heart failure: epidemiology, diagnosis, and therapeutic approaches. JACC Heart Fail. 2020;8(7):523–536. doi: 10.1016/j.jchf.2020.01.014. [DOI] [PubMed] [Google Scholar]
- 4.Costanzo M.R., Mills R.M., Wynne J. Characteristics of “stage D” heart failure: insights from the Acute Decompensated Heart Failure National Registry Longitudinal Module (ADHERE LM) Am Heart J. 2008;155(2):339–347. doi: 10.1016/j.ahj.2007.10.020. [DOI] [PubMed] [Google Scholar]
- 5.Rose E.A., Gelijns A.C., Moskowitz A.J., et al. Long-term use of a left ventricular assist device for end-stage heart failure. N Engl J Med. 2001;345(20):1435–1443. doi: 10.1056/NEJMoa012175. [DOI] [PubMed] [Google Scholar]
- 6.Kalogeropoulos A.P., Samman-Tahhan A., Hedley J.S., et al. Progression to stage D heart failure among outpatients with stage C heart failure and reduced ejection fraction. JACC Heart Fail. 2017;5(7):528–537. doi: 10.1016/j.jchf.2017.02.020. [DOI] [PubMed] [Google Scholar]
- 7.Crespo-Leiro M.G., Metra M., Lund L.H., et al. Advanced heart failure: a position statement of the Heart Failure Association of the European Society of Cardiology. Eur J Heart Fail. 2018;20(11):1505–1535. doi: 10.1002/ejhf.1236. [DOI] [PubMed] [Google Scholar]
- 8.Dunlay S.M., Roger V.L., Killian J.M., et al. Advanced heart failure epidemiology and outcomes: a population-based study. JACC Heart Fail. 2021;9(10):722–732. doi: 10.1016/j.jchf.2021.05.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Melton L.J., III History of the Rochester epidemiology project. Mayo Clin Proc. 1996;71(3):266–274. doi: 10.4065/71.3.266. [DOI] [PubMed] [Google Scholar]
- 10.McDonald K.M., Hlatky M.A., Saynina O., Geppert J., Garber A.M., McClellan M.B. Trends in hospital treatment of ventricular arrhythmias among Medicare beneficiaries, 1985 to 1995. Am Heart J. 2002;144(3):413–421. doi: 10.1067/mhj.2002.125498. [DOI] [PubMed] [Google Scholar]
- 11.Blecker S., Ladapo J.A., Doran K.M., Goldfeld K.S., Katz S. Emergency department visits for heart failure and subsequent hospitalization or observation unit admission. Am Heart J. 2014;168(6):901–908. doi: 10.1016/j.ahj.2014.08.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Saczynski J.S., Andrade S.E., Harrold L.R., et al. A systematic review of validated methods for identifying heart failure using administrative data. Pharmacoepidemiol Drug Saf. 2012;21(suppl 1):129–140. doi: 10.1002/pds.2313. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Field C.A., Welsh A.H. Bootstrapping clustered data. J Royal Stat Soc. 2007;69(3):369–390. [Google Scholar]
- 14.Janes H., Longton G., Pepe M.S. Accommodating covariates in ROC analysis. Stata J. 2009;9(1):17–39. [PMC free article] [PubMed] [Google Scholar]
- 15.Ting D.S.W., Cheung C.Y., Lim G., et al. Development and validation of a deep learning system for diabetic retinopathy and related eye diseases using retinal images from multiethnic populations with diabetes. JAMA. 2017;318(22):2211–2223. doi: 10.1001/jama.2017.18152. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Pocock S.J., Ariti C.A., McMurray J.J., et al. Predicting survival in heart failure: a risk score based on 39 372 patients from 30 studies. Eur Heart J. 2013;34(19):1404–1413. doi: 10.1093/eurheartj/ehs337. [DOI] [PubMed] [Google Scholar]
- 17.Peterson P.N., Rumsfeld J.S., Liang L., et al. A validated risk score for in-hospital mortality in patients with heart failure from the American Heart Association get with the guidelines program. Circ Cardiovasc Qual Outcomes. 2010;3(1):25–32. doi: 10.1161/CIRCOUTCOMES.109.854877. [DOI] [PubMed] [Google Scholar]
- 18.Shah A.M., Claggett B., Loehr L.R., et al. Heart failure stages among older adults in the community: the atherosclerosis risk in communities study. Circulation. 2017;135(3):224–240. doi: 10.1161/CIRCULATIONAHA.116.023361. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.St Sauver J.L., Grossardt B.R., Leibson C.L., Yawn B.P., Melton L.J., III, Rocca W.A. Generalizability of epidemiological findings and public health decisions: an illustration from the Rochester epidemiology project. Mayo Clin Proc. 2012;87(2):151–160. doi: 10.1016/j.mayocp.2011.11.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.