Abstract
BACKGROUND
There is growing interest to disentangle worsening heart failure (WHF) from location of care and move away from hospitalization as a surrogate for acuity.
OBJECTIVES
The purpose of this study was to describe the incidence of WHF events across the care continuum from ambulatory encounters to hospitalizations.
METHODS
We studied calendar year cohorts of adults with diagnosed heart failure (HF) from 2010-2019 within a large, integrated health care delivery system. Electronic health record (EHR) data were accessed for outpatient encounters, emergency department (ED) visits/observation stays, and hospitalizations. WHF was defined as ≥1 symptom, ≥2 objective findings including ≥1 sign, and ≥1 change in HF-related therapy. Symptoms and signs were ascertained using natural language processing.
RESULTS
We identified 103,138 eligible individuals with mean age 73.6 ± 13.7 years, 47.5% women, and mean left ventricular ejection fraction of 51.4 ± 13.7%. There were 1,136,750 unique encounters including 743,039 (65.4%) outpatient encounters, 224,670 (19.8%) ED visits/observation stays, and 169, 041 (14.9%) hospitalizations. A total of 126,008 WHF episodes were identified, including 34,758 (27.6%) outpatient encounters, 28,301 (22.5%) ED visits/observation stays, and 62,949 (50.0%) hospitalizations. The annual incidence (events per 100 person-years) of WHF increased from 25 to 33 during the study period primarily caused by outpatient encounters (7 to 10) and ED visits/observation stays (4 to 7). The 30-day rate of hospitalizations for WHF ranged from 8.2% for outpatient encounters to 12.4% for hospitalizations.
CONCLUSIONS
ED visits/observation stays and outpatient encounters accounting for approximately half of WHF events, are driving the underlying growth in HF morbidity, and portend a poor short-term prognosis.
Keywords: clinic visit, emergency department, heart failure, outpatient, worsening
There are >1 million hospitalizations for worsening heart failure (WHF) annually in the United States, accounting for 6.5 million hospital days and the majority of the ~$40 billion spent each year on heart failure (HF)-related care.1,2 To incentivize health systems to reduce 30-day readmissions, the Affordable Care Act launched the Hospital Readmission Reduction Program in 2012, which penalizes hospitals financially if they have higher than predicted risk-adjusted 30-day readmission rates for major conditions including HF.3 Although there has been a modest reduction in readmission rates, an unintended consequence of this policy may have been the shift of some HF-related care to the outpatient setting.4,5 Many tertiary care centers have developed protocols for administering intravenous therapies in the emergency department (ED), short-stay observation units, and/or same-day access clinics.6,7 In addition, recently completed pivotal clinical trials have found that adjudicating episodes of WHF (ie, defined by deteriorating signs and symptoms of HF requiring new administration of intravenous therapies and/or an augmentation of oral therapies) in ambulatory patients would increase the overall event rate by ~25%-30%.8 Thus, there is a growing interest in the field to disentangle WHF from location of care and move away from using hospitalization as a surrogate for acute decompensated HF. However, little is known about the contemporary epidemiology of outpatient WHF.9–11
We previously reported that machine learning-based natural language processing (NLP) algorithms applied to state-of-the-art electronic health record (EHR) data can accurately identify hospitalizations for WHF and resulted in a more than 2-fold increase in the perceived population burden of hospitalizations for WHF compared with diagnostic coding alone.12 Thus, the main objective of the present analysis was to use this systematic approach to further evaluate the epidemiology and temporal trends in the rate of ED visits/observation stays and outpatient WHF in the context of hospitalizations for WHF.
METHODS
SETTING AND SOURCE POPULATION.
Kaiser Permanente Northern California (KPNC) is a large integrated health care delivery system with 21 hospitals and >260 freestanding clinics where >4.5 million members receive comprehensive care (ie, inpatient, ED, and ambulatory encounters). Membership is highly representative of the local and statewide population with respect to age, sex, race/ethnicity, and socioeconomic status.12–14 This study was approved by the KPNC Institutional Review Board, and a waiver of informed consent was obtained as this is a retrospective, data-only study.
STUDY OVERVIEW AND COHORT ASSEMBLY.
We created 10 calendar year cohorts from 2010 through 2019 including all active KPNC members age ≥18 years on January 1 of each year (ie, the index date for each calendar year cohort) with previously diagnosed (ie, prevalent) HF. A diagnosis of HF is based on either having been previously hospitalized with a principal discharge diagnosis of HF and/or having ≥3 ambulatory visits coded for HF based on International Classification of Diseases-9th Edition (398.91, 402.x1, 404.01, 404.03, 404.11, 404.13, 404.91, 404.93, and 428.x) and −10th Edition (I09.81, I11.0, I11.9, I13.0, I13.1, I13.10, I13.11, I13.2, I50, I50.1, I50.2, I50.20, I50.21, I50.22, I50.23, I50.3, I50.30, I50.31, I50.32, I50.33, I50.4, I50.40, I50.41, I50.42, I50.43, I50.9, and I97.13) codes. These codes have been validated in multiple health care delivery systems and have a positive predictive value ≥95%.15–17 We excluded patients with end-stage kidney disease (ie, defined as receipt of chronic dialysis or kidney transplant), patients with stage D HF (ie, defined as receipt of left-ventricular assistive device or heart transplant), and patients who had <6 months of health plan membership before the index date to ensure sufficient capture of baseline characteristics.
FOLLOW-UP AND CENSORING.
Patients in each calendar year cohort were followed until December 31 of each year and censored at death or health plan disenrollment. Death was ascertained using comprehensive information from health plan administrative and clinical databases, member proxy reporting, Social Security Administration vital status files, and state death certificate information.18 Patients were included in consecutive calendar year cohorts, if eligible, until death or health plan disenrollment.
DEFINITION OF WHF EVENTS.
For the purposes of this study, qualifying clinical encounters included all hospitalizations (defined as admissions lasting >24 hours), ED visits including observation stays, and outpatient encounters (defined as an urgent care visit or a clinical appointment with a primary care provider or a cardiologist) with a diagnosis code for HF. Episodes of WHF were identified using EHR data and defined as including ≥1 qualifying clinical encounter, ≥1 symptom (ie, dyspnea, orthopnea, paroxysmal nocturnal dyspnea, fatigue, weight gain, and/or tachypnea), ≥2 objective findings (ie, tachycardia [heart rate >100 beats/min], elevated B-type natriuretic peptide [≥100 ng/L], and/or chest x-ray findings [pulmonary edema, pleural effusion, and/or cardiomegaly]) including ≥1 sign (ie, lower extremity edema, pulmonary rales/wheezing, jugular venous distension, third heart sound [S3 gallop], hepatomegaly, and/or abdominal swelling), and new administration of intravenous loop diuretic agents (ie, ≥2 doses if hospitalized or ≥1 dose if nonhospitalized) and/or new hemodialysis/continuous renal replacement therapy. For outpatient encounters, we also defined a change in HF-related therapy as either new initiation and/or augmentation of oral diuretic agents leveraging both structured (ie, pharmacy dispensing data) and unstructured (ie, written provider documentation) data elements. These diagnostic criteria are based on a standardized definition of inpatient and outpatient WHF previously developed and validated by a consensus panel of trialists with expertise in clinical endpoint classification in collaboration with the U.S. Food and Drug Administration.18 This multidimensional definition is specific for WHF and has been shown to be accurate and reproducible in multiple pivotal trials of investigational drugs and devices seeking regulatory approval.19–23
APPLYING NLP ALGORITHMS.
NLP was used to parse relevant unstructured documentation in the EHR (provider notes, discharge summaries, imaging reports) occurring within 72 hours of each qualifying encounter. For outpatient encounters only, chest x-ray imaging reports and laboratory values were also assessed within the preceding 30 days or since the last hospital encounter and up to 1 week after the encounter. The specific NLP approach has been described previously for hospitalizations,12 and the same approach and queries were used for ED and outpatient encounters. In brief, we first used regular expressions to preprocess all notes and exclude sections for past medical history or problem lists. We then used Linguamatics I2E software version 6.2.0, a rule-based NLP tool, to ascertain the presence or absence of WHF symptoms, signs, and augmentation of oral diuretics, accounting for clinical negations.
NLP algorithms for WHF were derived and validated against a “gold standard” consisting of manual chart review and validation by 2 physicians (A.N., R.M., P.Q.L., K.K., A.I., A.B.H., and J.F.K.) with final adjudication by a board-certified cardiologist (A.P.A.), where discrepancies existed. Chart review was recorded through an electronic survey tool where reviewers noted the presence or absence of each diagnostic criterion, as well as an overall assessment of WHF based on our operational definition. We initially identified a random sample of 75 ED visits and outpatient encounters with a code for HF to derive the queries. We then identified a random validation set of 300 encounters including 50 ED visits and 250 outpatient visits and measured the query performance for overall WHF by encounter type. For ED visits, we observed sensitivity 96%, specificity 95%, positive predictive value 96%, negative predictive value 95%, and accuracy 96%; for outpatient encounters, we observed sensitivity 80%, specificity 95%, positive predictive value 84%, negative predictive value 94%, and accuracy 92%. In the validation set, cases where NLP results differed from reviewer consensus were rereviewed, as reviewers sometimes missed positive mentions of criteria in encounters with extensive written documentation.
DATA SOURCES AND COVARIATES.
The KPNC Epic-based EHR system was the primary source for hospitalization data, patient progress notes, and cardiac imaging reports (ie, echocardiograms and chest x-rays). In addition, the KPNC Virtual Data Warehouse was used to ascertain concurrent comorbidities, outpatient medications, and outpatient laboratory values, as previously described and validated.24–26 Demographic information, including age, self-reported gender, and self-reported race/ethnicity, was obtained from the EHR. Comorbid conditions were ascertained within 5 years of each index date. Baseline laboratory results were defined as the most recent outpatient, nonemergency value within 365 days before each index date. Baseline medication use is based on outpatient dispensed prescriptions within 120 days before each index date. Data on left ventricular ejection fraction (LVEF), if available, was assessed using the most recent value within 2 years of each index date from structured results of echocardiograms, radionuclide scintigraphy, other nuclear imaging modalities, and left ventriculography, or extracted from echocardiogram reports using rule-based NLP algorithms.27 We categorized patients as heart failure with reduced ejection fraction (HFrEF) (defined as <40%), heart failure with midrange ejection fraction (HFmrEF) (defined as 40%-49%), or heart failure with preserved ejection fraction (HFpEF) (defined as ≥50%) using criteria congruent with current U.S. and European guidelines for the management of HF.27,28 Data on inpatient medications, laboratory values, vital signs, and procedures were obtained from EHR databases.
STATISTICAL ANALYSES.
We first present descriptive characteristics among all unique patients from calendar year cohorts at their first eligible calendar year, overall and stratified by LVEF category. Separately, we describe the characteristics of eligible patients in each calendar year cohort, updated at each calendar year index date. We then calculate the proportion of all unique encounters with a diagnosis code for HF that were determined to be positive for WHF through NLP, stratified by encounter type.
Because clinical episodes of WHF may include several consecutive encounters in various locations (ie, outpatient visit leading to ED visit and hospitalization), we applied a set of collapsing rules to delineate discrete (ie, unique) episodes of WHF. We first prioritized hospitalizations for WHF and removed any ED visits/observation stays for WHF occurring within 3 days before hospital admission or after discharge. Next, we removed outpatient WHF encounters within 3 days before or after the remaining ED visits/observation stays. Finally, we collapsed outpatient WHF encounters occurring within 3 days of each other.
Using the resulting unique episodes of inpatient, ED/observation, and outpatient WHF episodes of care, we calculated incidence rates of WHF per 100 person-years (and 95% Poisson CIs) within each calendar year cohort by encounter type and baseline LVEF category. Then, using each unique WHF episode as a separate index encounter, we calculated the risk of subsequent encounters within 30 days, including all-cause hospitalizations, all-cause ED visits/observation stays, hospitalizations for WHF, ED visits/observation stays for WHF, and death, stratified by encounter type of the index WHF event.
All analyses were performed using SAS software version 9.4 at KPNC’s Division of Research. This report follows the guidelines for reporting observational studies as outlined in the Strengthening the Reporting of Observational Studies in Epidemiology Statement.29
RESULTS
COHORT ASSEMBLY AND CLINICAL CHARACTERISTICS.
Between January 1, 2010, and December 31, 2019, we identified 103,138 unique eligible patients with prevalent HF, with mean age of 73.6 ± 13.7 years at their first calendar year of inclusion, 47.5% women, and more than 30% from racial/ethnic minority groups (Table 1). The breakdown of HF patients by LVEF included 15.8% with HFrEF (<40%), 11.6% with HFmrEF (40%-49%), 48.5% with HFpEF (≥50%), and 24.1% with unknown LVEF. The prevalence of cardiac and noncardiac comorbidities was high, including 37.4% atrial fibrillation/flutter, 41.1% diabetes, and 44.7% chronic kidney disease. The use of angiotensin-converting enzyme inhibitors/angiotensin receptor blockers/ angiotensin receptor-neprilysin inhibitors was 68.6%, β-blockers was 73.8%, and mineralocorticoid receptor antagonists was 9.2%, with 69.4% of patients being prescribed a loop diuretic agent at baseline. The clinical characteristics of the cohort by calendar year are shown in Supplemental Tables 1 and 2. The proportion of patients with a preserved LVEF has increased over time.
TABLE 1.
Baseline Characteristics of Cohort Overall and Stratified by Baseline Left Ventricular EF Category
| Overall (N = 103,138) | HFrEF (<40%) (n = 16,293) | HFmrEF (40%-49%) (n = 11,931) | HFpEF (≥50%) (n = 50,059) | Unknown EF (n = 24,855) | |
|---|---|---|---|---|---|
| Age, y | 73.6 ± 13.7 | 69.7 ± 14.2 | 71.6 ± 13.5 | 75.1 ± 12.6 | 74.0 ± 14.9 |
|
| |||||
| Women | 48,968 (47.5) | 5,387 (33.1) | 4,201 (35.2) | 27,146 (54.2) | 12,234 (49.2) |
|
| |||||
| Self-reported race | |||||
| White | 65,504 (63.5) | 9,726 (59.7) | 7,652 (64.1) | 32,086 (64.1) | 16,040 (64.5) |
| Black | 10,100 (9.8) | 2,051 (12.6) | 1,164 (9.8) | 4,296 (8.6) | 2,589 (10.4) |
| Asian/Pacific Islander | 10,693 (10.4) | 1,836 (11.3) | 1,288 (10.8) | 5,447 (10.9) | 2,122 (8.5) |
| Multiracial | 8,053 (7.8) | 1,133 (7.0) | 807 (6.8) | 4,001 (8.0) | 2112 (8.5) |
| American Indian/Alaska Native | 524 (0.5) | 86 (0.5) | 52 (0.4) | 254 (0.5) | 132 (0.5) |
| Unknown or Hispanic only | 8,264 (8.0) | 1,461 (9.0) | 968 (8.1) | 3,975 (7.9) | 1,860 (7.5) |
|
| |||||
| Hispanic ethnicity | 12,791 (12.4) | 2,141 (13.1) | 1,454 (12.2) | 6,193 (12.4) | 3,003 (12.1) |
|
| |||||
| Left ventricular ejection fraction, % | 51.4 ± 13.7 | 29.4 ± 6.0 | 43.6 ± 2.5 | 60.4 ± 5.4 | Not applicable |
|
| |||||
| Medical history | |||||
| Atrial fibrillation or flutter | 38,524 (37.4) | 5,440 (33.4) | 4,738 (39.7) | 21,339 (42.6) | 7,007 (28.2) |
| Ventricular fibrillation or tachycardia | 1,725 (1.7) | 544 (3.3) | 323 (2.7) | 627 (1.3) | 231 (0.9) |
| Ischemic stroke or transient ischemic attack | 5,622 (5.5) | 795 (4.9) | 675 (5.7) | 3,142 (6.3) | 1,010 (4.1) |
| Acute myocardial infarction | 8,277 (8.0) | 1,810 (11.1) | 1,470 (12.3) | 3,582 (7.2) | 1,415 (5.7) |
| Mitral or aortic valvular disease | 20,705 (20.1) | 3,069 (18.8) | 2,558 (21.4) | 12,156 (24.3) | 2,922 (11.8) |
| Venous thromboembolism | 3,933 (3.8) | 632 (3.9) | 469 (3.9) | 2,370 (4.7) | 462 (1.9) |
| Hospitalized bleed | 4,039 (3.9) | 523 (3.2) | 464 (3.9) | 2,122 (4.2) | 930 (3.7) |
| Diabetes mellitus | 42,403 (41.1) | 6,160 (37.8) | 4,545 (38.1) | 21,291 (42.5) | 10,407 (41.9) |
| Hypertension | 84,024 (81.5) | 11,711 (71.9) | 9,258 (77.6) | 42,960 (85.8) | 20,095 (80.8) |
| Dyslipidemia | 86,767 (84.1) | 13,866 (85.1) | 10,317 (86.5) | 42,411 (84.7) | 20,173 (81.2) |
| Hyperthyroidism | 4,813 (4.7) | 621 (3.8) | 508 (4.3) | 2,506 (5.0) | 1,178 (4.7) |
| Hypothyroidism | 20,877 (20.2) | 2,535 (15.6) | 2,099 (17.6) | 11,125 (22.2) | 5,118 (20.6) |
| Chronic liver disease | 4,575 (4.4) | 639 (3.9) | 503 (4.2) | 2,729 (5.5) | 704 (2.8) |
| Chronic kidney disease | 46,088 (44.7) | 6,711 (41.2) | 4,876 (40.9) | 24,174 (48.3) | 10,327 (41.5) |
| Chronic lung disease | 42,560 (41.3) | 5,522 (33.9) | 4,289 (35.9) | 22,692 (45.3) | 10,057 (40.5) |
| Diagnosed depression | 18,468 (17.9) | 2,367 (14.5) | 1,883 (15.8) | 9,725 (19.4) | 4493 (18.1) |
| Diagnosed dementia | 5,944 (5.8) | 756 (4.6) | 520 (4.4) | 2,881 (5.8) | 1,787 (7.2) |
|
| |||||
| Cardiac procedures | |||||
| Coronary artery bypass graft | 4,790 (4.6) | 845 (5.2) | 813 (6.8) | 2,357 (4.7) | 775 (3.1) |
| Percutaneous coronary intervention | 5,668 (5.5) | 1,349 (8.3) | 1,114 (9.3) | 2,317 (4.6) | 888 (3.6) |
| Implantable cardioverter defibrillator | 3,109 (3.0) | 1,541 (9.5) | 361 (3.0) | 549 (1.1) | 658 (2.6) |
| Right heart catheterization | 8,819 (8.6) | 2,270 (13.9) | 1,231 (10.3) | 4,053 (8.1) | 1,265 (5.1) |
| Coronary angiography | 25,861 (25.1) | 6,187 (38.0) | 4,406 (36.9) | 11,523 (23.0) | 3,745 (15.1) |
| Catheter ablation | 268 (0.3) | 32 (0.2) | 34 (0.3) | 170 (0.3) | 32 (0.1) |
| Cardiac resynchronization therapy pacemaker | 159 (0.2) | 37 (0.2) | 29 (0.2) | 46 (0.1) | 47 (0.2) |
| Cardiac resynchronization therapy defibrillator | 1,108 (1.1) | 443 (2.7) | 129 (1.1) | 266 (0.5) | 270 (1.1) |
| Pacemaker | 7,499 (7.3) | 1,332 (8.2) | 1,012 (8.5) | 3,701 (7.4) | 1,454 (5.8) |
|
| |||||
| Vital signs | |||||
| Body mass index, kg/m2 | 29.7 ± 7.8 | 28.3 ± 6.7 | 28.9 ± 6.9 | 30.2 ± 8.2 | 30.2 ± 8.0 |
| Missing | 4,141 (4.0) | 354 (2.2) | 208 (1.7) | 1,040 (2.1) | 2,539 (10.2) |
| Systolic blood pressure, mm Hg | 125.6 ± 18.4 | 119.4 ± 18.3 | 123.4 ± 18.3 | 127.4 ± 18.2 | 127.0 ± 18.0 |
| Missing | 2,556 (2.5) | 253 (1.6) | 125 (1.0) | 600 (1.2) | 1,578 (6.3) |
|
| |||||
| Medications | |||||
| Angiotensin-converting enzyme inhibitor/angiotensin receptor blocker/angiotensin-neprilysin inhibitor | 68,676 (66.6) | 13,043 (80.1) | 8,872 (74.4) | 31,054 (62.0) | 15,707 (63.2) |
| Aldosterone receptor antagonist | 9,444 (9.2) | 3,372 (20.7) | 1,295 (10.9) | 2,991 (6.0) | 1,786 (7.2) |
| Diuretic agent | 71,618 (69.4) | 12,195 (74.8) | 7,934 (66.5) | 36,504 (72.9) | 14,985 (60.3) |
| Alpha blocker | 9,157 (8.9) | 1,028 (6.3) | 1,021 (8.6) | 4,797 (9.6) | 2,311 (9.3) |
| Central alpha-adrenergic receptor agonist | 3,702 (3.6) | 197 (1.2) | 222 (1.9) | 2,332 (4.7) | 951 (3.8) |
| Beta-blocker | 76,145 (73.8) | 14,279 (87.6) | 9,927 (83.2) | 36,018 (72.0) | 15,921 (64.1) |
| Calcium-channel blocker | 29,418 (28.5) | 2,278 (14.0) | 2,553 (21.4) | 17,714 (35.4) | 6,873 (27.7) |
| Antiarrhythmic drug | 8,788 (8.5) | 1,792 (11.0) | 1,190 (10.0) | 4,593 (9.2) | 1,213 (4.9) |
| Oral anticoagulant | 29,954 (29.0) | 4,887 (30.0) | 3,890 (32.6) | 16,213 (32.4) | 4,964 (20.0) |
| Antiplatelet drug | 12,512 (12.1) | 2,686 (16.5) | 2,210 (18.5) | 5,663 (11.3) | 1,953 (7.9) |
| Any antihypertensive drugs | 93,536 (90.7) | 15,448 (94.8) | 11,193 (93.8) | 45,569 (91.0) | 21,326 (85.8) |
| Statins | 69,327 (67.2) | 11,379 (69.8) | 8,528 (71.5) | 33,928 (67.8) | 15,492 (62.3) |
| Other lipid-lowering drugs | 4,789 (4.6) | 686 (4.2) | 472 (4.0) | 2,154 (4.3) | 1,477 (5.9) |
| Nitrates | 17,792 (17.3) | 3,680 (22.6) | 2,491 (20.9) | 8,034 (16.0) | 3,587 (14.4) |
| Vasodilators | 23,216 (22.5) | 4,186 (25.7) | 2,915 (24.4) | 11,520 (23.0) | 4,595 (18.5) |
| Nonsteroidal anti-inflammatory drugs | 7,341 (7.1) | 871 (5.3) | 730 (6.1) | 3,682 (7.4) | 2,058 (8.3) |
| Diabetic therapy | 30,535 (29.6) | 4,457 (27.4) | 3,334 (27.9) | 15,410 (30.8) | 7,334 (29.5) |
| Sodium-glucose cotransporter-2 inhibitors | 6 (0.0) | 1 (0.0) | 0 (0.0) | 3 (0.0) | 2 (0.0) |
|
| |||||
| Baseline laboratory values | |||||
| Hemoglobin, g/dL | 12.8 ± 1.8 | 13.0 ± 1.9 | 12.9 ± 1.9 | 12.5 ± 1.8 | 13.1 ± 1.7 |
| Missing | 21,830 (21.2) | 3,216 (19.7) | 2,240 (18.8) | 8,459 (16.9) | 7,915 (31.8) |
| Hemoglobin A1C, % | 6.7 ± 1.4 | 6.8 ± 1.5 | 6.7 ± 1.4 | 6.7 ± 1.3 | 6.8 ± 1.4 |
| Missing | 49,061 (47.6) | 7,954 (48.8) | 5,540 (46.4) | 22,237 (44.4) | 13,330 (53.6) |
| Serum creatinine, mg/dL | 1.2 ± 0.5 | 1.2 ± 0.5 | 1.2 ± 0.5 | 1.2 ± 0.6 | 1.2 ± 0.5 |
| Missing | 8,565 (8.3) | 1,027 (6.3) | 719 (6.0) | 2,921 (5.8) | 3,898 (15.7) |
| Estimated glomerular filtration rate, mL/min/1.73 m2 | 61.7 ± 22.8 | 64.3 ± 22.7 | 64.2 ± 22.5 | 60.2 ± 22.4 | 61.9 ± 23.6 |
| Missing | 8,565 (8.3) | 1,027 (6.3) | 719 (6.0) | 2,921 (5.8) | 3,898 (15.7) |
| Proteinuria categories | |||||
| Mild (<30 mg/g) | 47,209 (45.8) | 6,793 (41.7) | 5,228 (43.8) | 23,212 (46.4) | 11,976 (48.2) |
| Moderate (30-299 mg/g) | 18,406 (17.8) | 2,697 (16.6) | 1,951 (16.4) | 9,389 (18.8) | 4,369 (17.6) |
| Severe (≥300 mg/g) | 11,213 (10.9) | 1,653 (10.1) | 1,339 (11.2) | 6,616 (13.2) | 1,605 (6.5) |
| Unknown | 26,310 (25.5) | 5,150 (31.6) | 3,413 (28.6) | 10,842 (21.7) | 6,905 (27.8) |
Values are mean ± SD or n (%).
HFmrEF = heart failure with a midrange ejection fraction; HFpEF = heart failure with a preserved ejection fraction; HFrEF = heart failure with a reduced ejection fraction.
IDENTIFICATION OF EPISODES OF WHF.
In total, there were 1,136,750 unique clinical encounters with an associated diagnosis code for HF including 169, 041 (14.9%) hospitalizations, 224,670 (19.8%) ED visits/observation stays, and 743,039 (65.4%) outpatient encounters (Central Illustration A). After applying NLP-based algorithms for WHF to EHR data, the proportion of clinical encounters meeting the diagnostic criteria for WHF ranged from a low of 4.7% for outpatient encounters to a high of 37.2% for hospitalizations. In aggregate, there were 126,008 encounters for WHF including 62,949 (50.0%) hospitalizations, 28,301 (22.5%) ED visits/observation stays, and 34,758 (27.6%) outpatient encounters for WHF (Supplemental Figure 1). These findings were similar among the subgroup of patients with HFrEF, HFmrEF, HFpEF, and unknown LVEF (Supplemental Figure 2). Among the 34,758 outpatient WHF encounters, the criterion for changes in HF-related therapy was met through new administration of IV loop diuretic agents in 1.8% of encounters, new initiation of oral loop or combination diuretic therapy based on pharmacy dispensing data in 18.3% of encounters, through doubling (i.e., 100% increase) of oral loop diuretic agents based on pharmacy dispensing data in 4.6% of encounters, and through note-based provider documentation of new initiation and/or any augmentation of oral diuretic agents in 75.3% of encounters.
CENTRAL ILLUSTRATION. Overall Number and Temporal Trends in Worsening Heart Failure Events.
Qualifying clinical encounters included all hospitalizations, emergency department visits including observation stays, and outpatient encounters with a diagnosis code for heart failure (HF). Episodes of worsening HF were defined as including ≥1 qualifying clinical encounter, ≥1 symptom, ≥2 objecting findings including ≥1 sign, and ≥1 change in HF-related therapy. In total, there were 1,223,616 unique clinical encounters with an associated diagnosis code for HF. After applying natural language processing-based algorithms, the proportion of clinical encounters meeting the diagnostic criteria for worsening HF ranged from a low of 4.7% for outpatient encounters to a high of 37.2% for hospitalizations (A). There has been a substantial increase in the annual incidence of episodes of worsening HF from 25 to 33 events per 100 person-years driven by emergency department visits and outpatient encounters (B).
TEMPORAL TRENDS IN EPISODES OF WHF.
From 126,008 encounters for WHF, we created 116,318 unique episodes of care for WHF. Overall, there has been a substantial increase in the annual incidence of episodes of WHF from 25 to 33 events per 100 person-years (Central Illustration B). A subgroup analysis stratified by LVEF showed that the annual incidence of hospitalizations for WHF had declined among patients with HFmrEF and HFpEF (Figure 1). In contrast, the annual incidence of ED visits/observation for WHF increased in the subset of patients with HFrEF and HFpEF. Finally, there was a steady rise in the annual incidence of outpatient WHF that was consistently seen across all LVEF categories. Point estimates with CIs for all rates are shown in Supplemental Table 3.
FIGURE 1. Temporal Trends in the Rate of WHF Events.
Temporal trends in the rate of worsening heart failure (WHF) events by encounter type for heart failure with a (A) reduced, (B) midrange, (C) preserved, and (D) unknown ejection fraction (EF). The rate of WHF (per 100 person-years [PY]) was calculated by calendar year and stratified by encounter type and left ventricular EF. There has been a steady and consistent rise in the annual incidence of outpatient WHF that was consistently seen across all left ventricular EF categories. This underscores the growing importance of outpatient WHF events to appreciating the patient journey. HFmrEF = heart failure with a midrange ejection fraction; HFpEF = heart failure with a preserved ejection fraction; HFrEF = heart failure with a reduced ejection fraction.
30-DAY READMISSIONS AFTER HOSPITALIZATIONS FOR WHF.
All-cause and HF-specific health care utilization rates and death from any cause by the index WHF event are shown in Figure 2. All-cause hospitalizations were highest following an initial hospitalization for WHF (20.8%) and lowest following an outpatient encounter for WHF (13.7%). Notably, there was a relatively narrow range of rates (8.2%-12.4%) for subsequent hospitalizations for WHF following an index episode of WHF. In contrast, the rate of all-cause and HF-related ED visits was highest following an initial ED visit/observation stay for WHF and similar following a hospitalization or outpatient encounter for WHF. Death from any cause was substantially higher following an index hospitalization for WHF (14.1%) compared with ED visits/observation stays (5.0%) and outpatient encounters for WHF (3.0%). The distribution of time-to-first event for the outcomes of interest following an index clinical encounter for WHF are shown in Supplemental Table 4.
FIGURE 2. Health Care Utilization and Death Occurring Within 30 Days.
The rate (%) of all-cause and heart failure-specific emergency department (ED) visits and hospitalizations and death from any cause occurring within 30 days was determined. In general, all-cause and heart failure-related health care utilization and death were highest following an initial hospitalization for worsening heart failure (WHF) and lowest following an outpatient encounter for WHF. However, there was a relatively narrow range of rates for subsequent hospitalizations for WHF following an index WHF episode, suggesting that any clinical encounter type represents a sentinel event in the natural history of the disorder.
DISCUSSION
This large-scale, systematic description of WHF events by encounter type in a diverse and contemporary community-based HF cohort provides important and comprehensive insights into the population burden of HF. Although both the relative proportion and absolute number of WHF events were highest for hospitalizations, ED visits/observation stays and outpatient encounters in aggregate accounted for more than 85% of all clinical encounters and approximately 50% of WHF events. In addition, there has been a large increase in the annual incidence of WHF events over the past decade, and this phenomenon has been largely driven by increased rates of ED visits/observation stays and outpatient encounters for WHF. Finally, although the risk of HF-related morbidity was highest following an index hospitalization or ED visit/observation stay, more than 10% of initial outpatient encounters resulted in a subsequent WHF event necessitating a higher level of care.
This comprehensive analysis of WHF events by encounter type provides several novel insights into the patient journey.30 First, most clinical encounters and nearly half of WHF events occurred outside of a hospitalized setting. To ensure that WHF events were truly independent episodes of care, we collapsed all clinical encounters occurring within a 72-hour timeframe into a single unique episode of care and assigned the highest level of acuity. For example, if a patient had an outpatient encounter meeting diagnostic criteria for WHF and shortly thereafter presented to the ED and was admitted to the hospital, then this entire sequence of events would have been reported as a single episode of care and classified as a hospitalization for WHF. Second, the dramatic rise in the cumulative event rate for WHF imply that the growing prevalence of HF has been accompanied by an increasingly greater HF severity. This temporal trend would not be discernible if hospitalizations were selected as the sole measure of HF-related morbidity. Finally, with few exceptions, these findings were directionally consistent across all LVEF categories.
All-cause and HF-related ED visits are often reported in observational research leveraging real-world datasets, and ED encounters are now commonly collected for global cardiovascular outcomes trials.31,32 However, outside of the context of pivotal studies conducted for regulatory approval, these events are rarely formally adjudicated. Thus, it is notable that when we applied a standardized definition for WHF (ie, deteriorating signs and symptoms resulting in a change in HF-related therapy) to structured and unstructured EHR data that only two-thirds of patients with a primary discharge diagnosis code for HF satisfied the diagnostic criteria. It is also important that the subsequent 30-day rates for all-cause and HF-related ED visits was highest following an index ED visits/observation stay, whereas the rate of all-cause and cause-specific ED utilization was comparable following an index hospitalization or outpatient encounter. Collectively, these findings suggest that ED visits with an associated diagnosis code for HF are a better indicator for overall level of health care utilization and not necessarily a reliable surrogate for WHF. This has implications for formulating public policy and conducting prospective studies based on ED-related outcomes.
There has been a longstanding interest in the scientific community to disassociate WHF from the clinical setting where the care is delivered to better capture and characterize the magnitude of WHF events across the care continuum from ambulatory encounters to hospitalizations.8,10,33 This study found that approximately 5% of outpatient encounters with an associated diagnosis code for HF met the prespecified diagnostic criteria for an episode of WHF. Although this represents a relatively small proportion of outpatient encounters, given the sheer absolute number of ambulatory visits, this cumulatively accounted for more than one-quarter of WHF events. In addition, it is highly likely that this is an underestimate of the true burden of outpatient WHF given the relatively lower sensitivity observed with outpatient encounters. This is not entirely unexpected because the completeness of provider documentation and the more prolonged time course of medical evaluations in the ambulatory setting may have limited our ability to comprehensively capture episodes of outpatient WHF. In contrast, it should be noted that the specificity of the NLP-based algorithms was uniformly high across all encounter types. Finally, it should be highlighted that following an index outpatient encounter for WHF, the subsequent rate of ED visits/observation stays or hospitalizations for WHF was more than 10% and only marginally lower than the event rate following an index hospitalization for WHF. This strongly supports the validity of outpatient WHF as a clinical construct, and future patient-oriented research should incorporate this emerging endpoint.
STUDY LIMITATIONS.
First, although we incorporated a rigorous definition of WHF based on expert consensus and U.S. Food and Drug Administration guidance, it was not technically feasible to incorporate several potential therapeutic interventions for WHF including intravenous vasoactive medications, temporary mechanical circulatory support, and/or chronic renal replacement therapy, because these treatments were either inconsistently documented and/or lacked diagnostic specificity for WHF. Second, we allowed for augmentation of oral diuretic therapy in the operational definition of outpatient WHF, which may have limited the diagnostic specificity and accuracy. However, the rates of outpatient intravenous diuretic agent administration have remained low (ie, 0.5%-1.0%) over time despite interest in exploring this care pathway as a potential alternative to hospitalization.34,35 In addition, as previously noted, following a unique episode of WHF, the rate of short-term HF-related morbidity is only slightly lower than the event rate following an index hospitalization for WHF. Third, we collapsed and prioritized clinical encounters occurring within a 72-hour timeframe to identify truly unique WHF events recognizing that there is no universal agreement as to what constitutes a discrete episode of care. Fourth, changes in outpatient diuretic agents may be communicated directly to patients and not necessarily lead to a prescription change or always include specific dose change information. We used NLP to review all provider documentation for newly prescribed and/or augmented oral diuretic agents, but this challenge may have limited the sensitivity and specificity of algorithms for identifying outpatient WHF. A final potential concern is that a subset of clinical encounters may have occurred at non-KPNC facilities with limited documentation in the KPNC EHR. However, at KPNC there is an exclusive relationship between the insurer, members, and providers, and prior studies have shown that >95% of events are captured in our EHR.
CONCLUSIONS
Applying validated NLP-based algorithms to structured and unstructured EHR data is technically feasible and highly accurate for detecting WHF events across the entire spectrum of care settings. In fact, ED visits/observation stays and outpatient encounters made up approximately half the episodes of WHF, and the relative proportion of WHF events occurring outside a hospitalized setting is increasing and driving the underlying growth in HF-related morbidity. The significance of ED visits/observation stays and outpatient encounters for WHF is underscored by the high rate of subsequent WHF events. Future patient-oriented research should incorporate composite outcome measures including WHF events across the care continuum from ambulatory encounters to hospitalizations.
Supplementary Material
PERSPECTIVES.
COMPETENCY IN SYSTEMS-BASED PRACTICE:
Most encounters for WHF involve ED and other outpatient visits, though the short-term likelihood of hospitalization is high following these index encounters.
TRANSLATIONAL OUTLOOK:
Further efforts are needed to understand the factors that provoke WHF events and determine modes of presentation and clinical outcomes.
FUNDING SUPPORT AND AUTHOR DISCLOSURES
The UTILIZE-WHF study received funding from Novartis AG and the Kaiser Permanente Northern California Community Benefit Program. The funder approved the study in advance but had no formal role in design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication. Dr Ambrosy is supported by a Mentored Patient-Oriented Research Career Development Award (K23HL150159) through the National Heart, Lung, and Blood Institute; has received relevant research support through grants to his institution from Amarin Pharma, Abbott, and Novartis; and has modest reimbursement for travel from Novartis. Drs Shen, Sanghera, and Cristino are employees of Novartis AG. Dr Go has received relevant research support through grants to his institution from the National Heart, Lung, and Blood Institute; National Institute of Diabetes, Digestive and Kidney Diseases; National Institute on Aging; Amarin Pharma Inc; Novartis; Janssen Research and Development; and CSL Behring. All other authors have reported that they have no relationships relevant to the contents of this paper to disclose.
ABBREVIATIONS AND ACRONYMS
- ED
emergency department
- EHR
electronic health record
- HF
heart failure
- HFmrEF
heart failure with a midrange ejection fraction
- HFpEF
heart failure with a preserved ejection fraction
- HFrEF
heart failure with a reduced ejection fraction
- LVEF
left ventricular ejection fraction
- NLP
natural language processing
- WHF
worsening heart failure
Footnotes
The authors attest they are in compliance with human studies committees and animal welfare regulations of the authors’ institutions and Food and Drug Administration guidelines, including patient consent where appropriate. For more information, visit the Author Center.
APPENDIX For supplemental tables and figures, please see the online version of this paper.
REFERENCES
- 1.Mozaffarian D, Benjamin EJ, Go AS, et al. Heart disease and stroke statistics–2015 update: a report from the American Heart Association. Circulation. 2015;131(4):e29–e322. 10.1161/CIR.0000000000000152 [DOI] [PubMed] [Google Scholar]
- 2.Blecker S, Paul M, Taksler G, Ogedegbe G, Katz S. Heart failure-associated hospitalizations in the United States. J Am Coll Cardiol. 2013;61(12):1259–1267. 10.1016/j.jacc.2012.12.038 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.McIlvennan CK, Eapen ZJ, Allen LA. Hospital readmissions reduction program. Circulation. 2015;131(20):1796–1803. 10.1161/CIRCULATIONAHA.114.010270 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Bergethon KE, Ju C, DeVore AD, et al. Trends in 30-day readmission rates for patients hospitalized with heart failure: findings from the Get With The Guidelines-Heart Failure Registry. Circ Heart Fail. 2016;9(6):e002594. CIRCHEARTFAILURE.115.002594 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.DeVore AD, Hammill BG, Hardy NC, Eapen ZJ, Peterson ED, Hernandez AF. Has public reporting of hospital readmission rates affected patient outcomes? Analysis of medicare claims data. J Am Coll Cardiol. 2016;67(8):963–972. 10.1016/j.jacc.2015.12.037 [DOI] [PubMed] [Google Scholar]
- 6.Blood AJ, Fraiche AM, Eapen ZJ. Is an admission for decompensated heart failure inevitable? Prog Cardiovasc Dis. 2017;60(2):171–177. 10.1016/j.pcad.2017.07.003 [DOI] [PubMed] [Google Scholar]
- 7.Collins SP, Pang PS, Fonarow GC, Yancy CW, Bonow RO, Gheorghiade M. Is hospital admission for heart failure really necessary? The role of the emergency department and observation unit in preventing hospitalization and rehospitalization. J Am Coll Cardiol. 2013;61(2):121–126. 10.1016/j.jacc.2012.08.1022 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Okumura N, Jhund PS, Gong J, et al. Importance of clinical worsening of heart failure treated in the outpatient setting: evidence from the prospective comparison of ARNI with ACEI to determine impact on global mortality and morbidity in heart failure trial (PARADIGM-HF). Circulation. 2016;133(23):2254–2262. 10.1161/CIRCULATIONAHA.115.020729 [DOI] [PubMed] [Google Scholar]
- 9.Greene SJ, Felker GM, Butler J. Outpatient versus inpatient worsening heart failure: distinguishing biology and risk from location of care. Eur J Heart Fail. 2019;21(1):121–124. 10.1002/ejhf.1341 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Greene SJ, Mentz RJ, Felker GM. Outpatient worsening heart failure as a target for therapy: a review. JAMA Cardiol. 2018;3(3):252–259. 10.1001/jamacardio.2017.5250 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Greene SJ, Felker GM. Innovation in diuretic therapy: the missing ingredient for treating worsening heart failure outside the hospital? JACC Basic Transl Sci. 2018;3(1):35–37. 10.1016/j.jacbts.2018.01.002 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Ambrosy AP, Parikh RV, Sung SH, et al. A natural language processing-based approach for identifying hospitalizations for worsening heart failure within an integrated health care delivery system. JAMA Netw Open. 2021;4(11):e2135152. 10.1001/jamanetworkopen.2021.35152 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Krieger N Overcoming the absence of socioeconomic data in medical records: validation and application of a census-based methodology. Am J Public Health. 1992;82(5):703–710. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Gordon NP. Characteristics of Adult Health Plan Members in the Northern California Region Membership, as Estimated from the 2011 Member Health Survey. Division of Research, Kaiser Permanente Medical Care Program; 2013. [Google Scholar]
- 15.Koebnick C, Langer-Gould AM, Gould MK, et al. Sociodemographic characteristics of members of a large, integrated health care system: comparison with US Census Bureau data. Comparative Study Research Support, Non-U.S. Gov’t. Perm J. 2012;16(3):37–41. 10.7812/tpp/12-031 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Smith DH, Thorp ML, Gurwitz JH, et al. Chronic kidney disease and outcomes in heart failure with preserved versus reduced ejection fraction: the Cardiovascular Research Network PRESERVE Study. Circ Cardiovasc Qual Outcomes. 2013;6(3):333–342. 10.1161/CIRCOUTCOMES.113.000221 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Go AS, Lee WY, Yang J, Lo JC, Gurwitz JH. Statin therapy and risks for death and hospitalization in chronic heart failure. JAMA. 2006;296(17):2105–2111. 10.1001/jama.296.17.2105 [DOI] [PubMed] [Google Scholar]
- 18.Curb JD, Ford CE, Pressel S, Palmer M, Babcock C, Hawkins CM. Ascertainment of vital status through the National Death Index and the Social Security Administration. Am J Epidemiol. 1985;121(5):754–766. 10.1093/aje/121.5.754 [DOI] [PubMed] [Google Scholar]
- 19.Hicks KA, Mahaffey KW, Mehran R, et al. 2017 Cardiovascular and stroke endpoint definitions for clinical trials. J Am Coll Cardiol. 2018;71(9):1021–1034. 10.1016/j.jacc.2017.12.048 [DOI] [PubMed] [Google Scholar]
- 20.Gheorghiade M, Konstam MA, Burnett JC Jr, et al. Short-term clinical effects of tolvaptan, an oral vasopressin antagonist, in patients hospitalized for heart failure: the EVEREST Clinical Status Trials. JAMA. 2007;297(12):1332–1343. 10.1001/jama.297.12.1332 [DOI] [PubMed] [Google Scholar]
- 21.Konstam MA, Gheorghiade M, Burnett JC Jr, et al. Effects of oral tolvaptan in patients hospitalized for worsening heart failure: the EVEREST Outcome Trial. JAMA. 2007;297(12):1319–1331. 10.1001/jama.297.12.1319 [DOI] [PubMed] [Google Scholar]
- 22.O’Connor CM, Starling RC, Hernandez AF, et al. Effect of nesiritide in patients with acute decompensated heart failure. N Engl J Med. 2011;365(1):32–43. 10.1056/NEJMoa1100171 [DOI] [PubMed] [Google Scholar]
- 23.Teerlink JR, Cotter G, Davison BA, et al. Serelaxin, recombinant human relaxin-2, for treatment of acute heart failure (RELAX-AHF): a randomised, placebo-controlled trial. Lancet. 2013;381(9860):29–39. 10.1016/S0140-6736(12)61855-8 [DOI] [PubMed] [Google Scholar]
- 24.Massie BM, O’Connor CM, Metra M, et al. Rolofylline, an adenosine A1-receptor antagonist, in acute heart failure. N Engl J Med. 2010;363(15):1419–1428. 10.1056/NEJMoa0912613 [DOI] [PubMed] [Google Scholar]
- 25.Go AS, Magid DJ, Wells B, et al. The Cardiovascular Research Network: a new paradigm for cardiovascular quality and outcomes research. Circ Cardiovasc Qual Outcomes. 2008;1(2):138–147. 10.1161/CIRCOUTCOMES.108.801654. Nov. [DOI] [PubMed] [Google Scholar]
- 26.Magid DJ, Gurwitz JH, Rumsfeld JS, Go AS. Creating a research data network for cardiovascular disease: the CVRN. Expert Rev Cardiovasc Ther. 2008;6(8):1043–1045. 10.1586/14779072.6.8.1043 [DOI] [PubMed] [Google Scholar]
- 27.Solomon MDTG, Allen A, Sung SH, Go AS. Large-scale identification of aortic stenosis and its severity using natural language processing on electronic health records. Cardiovasc Digit Health J. 2021;2(3):156–163. 10.1016/j.cvdhj.2021.03.003 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Yancy CW, Jessup M, Bozkurt B, et al. 2013 ACCF/AHA guideline for the management of heart failure: a report of the American College of Cardiology Foundation/American Heart Association Task Force on Practice Guidelines. J Am Coll Cardiol. 2013;62(16):e147–e239. 10.1016/j.jacc.2013.05.019 [DOI] [PubMed] [Google Scholar]
- 29.EQUATOR Network. Enhancing the QUAlity and Transparency Of health Research. Accessed May 16, 2022. https://www.equator-network.org/reporting-guidelines/strobe/ [Google Scholar]
- 30.Ponikowski P, Voors AA, Anker SD, et al. 2016 ESC guidelines for the diagnosis and treatment of acute and chronic heart failure: The Task Force for the diagnosis and treatment of acute and chronic heart failure of the European Society of Cardiology (ESC)Developed with the special contribution of the Heart Failure Association (HFA) of the ESC. Eur Heart J. 2016;37(27):2129–2200. 10.1093/eurheartj/ehw128 [DOI] [PubMed] [Google Scholar]
- 31.Ferreira JP, Metra M, Mordi I, et al. Heart failure in the outpatient versus inpatient setting: findings from the BIOSTAT-CHF study. Eur J Heart Fail. 2019;21(1):112–120. 10.1002/ejhf.1323 [DOI] [PubMed] [Google Scholar]
- 32.Collins SP, Jenkins CA, Harrell FE Jr, et al. Identification of emergency department patients with acute heart failure at low risk for 30-day adverse events: the STRATIFY Decision Tool. J Am Coll Cardiol HF. 2015;3(10):737–747. 10.1016/j.jchf.2015.05.007 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Blecker S, Ladapo JA, Doran KM, Goldfeld KS, Katz S. Emergency department visits for heart failure and subsequent hospitalization or observation unit admission. Am Heart J. 2014;168(6):901–908. 10.1016/j.ahj.2014.08.002. e1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Khan MS, Butler J, Greene SJ. Recognizing the significance of outpatient worsening heart failure. J Am Heart Assoc. 2020;9(14):e017485. 10.1161/JAHA.120.017485 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Greene SJ, Wilson LE, Abbasi SA, Yusuf AA, Hammill BG. Outpatient intravenous diuretic therapy for heart failure in the United States. J Am Coll Cardiol. 2019;73(9):1101–1103. 10.1016/j.jacc.2018.12.034 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.