Assessing clinical heterogeneity in sepsis through treatment patterns and machine learning

Abstract

Objective

To use unsupervised topic modeling to evaluate heterogeneity in sepsis treatment patterns contained within granular data of electronic health records.

Materials and Methods

A multicenter, retrospective cohort study of 29 253 hospitalized adult sepsis patients between 2010 and 2013 in Northern California. We applied an unsupervised machine learning method, Latent Dirichlet Allocation, to the orders, medications, and procedures recorded in the electronic health record within the first 24 hours of each patient’s hospitalization to uncover empiric treatment topics across the cohort and to develop computable clinical signatures for each patient based on proportions of these topics. We evaluated how these topics correlated with common sepsis treatment and outcome metrics including inpatient mortality, time to first antibiotic, and fluids given within 24 hours.

Results

Mean age was 70 ± 17 years with hospital mortality of 9.6%. We empirically identified 42 clinically recognizable treatment topics (eg, pneumonia, cellulitis, wound care, shock). Only 43.1% of hospitalizations had a single dominant topic, and a small minority (7.3%) had a single topic comprising at least 80% of their overall clinical signature. Across the entire sepsis cohort, clinical signatures were highly variable.

Discussion

Heterogeneity in sepsis is a major barrier to improving targeted treatments, yet existing approaches to characterizing clinical heterogeneity are narrowly defined. A machine learning approach captured substantial patient- and population-level heterogeneity in treatment during early sepsis hospitalization.

Conclusion

Using topic modeling based on treatment patterns may enable more precise clinical characterization in sepsis and better understanding of variability in sepsis presentation and outcomes.

INTRODUCTION

Sepsis, the life-threatening organ dysfunction arising from a dysregulated host response to infection, is a condition with tremendous global impact.¹ Sepsis affects at least 30 million patients worldwide and results in 5 million deaths each year.² It is also a major contributor to hospital and postdischarge mortality, morbidity, and health care utilization.^3–8 Survival in sepsis has steadily improved over time, owing to standardized care focused on heightening early identification and delivery of antibiotics.^9–11 However, sepsis protocols are built using a “one-size-fits- all” approach and do not target specific treatments to patients with differences in underlying illness or acute presentation—except within the simplest groupings, like shock.^12,13 Underlying heterogeneity in sepsis is universally cited as the major barrier to future improvements in treatment and is an issue of particular salience for a condition in which no new effective pharmacologic treatment has been identified in the past 50 years.^12–15

While heterogeneity in sepsis is widely acknowledged both by researchers and clinicians, few studies have attempted to comprehensively quantify its characteristics. This gap is partly explained by the varied sources of heterogeneity in sepsis including clinical factors, genetic predisposition, host–pathogen interactions, acute disease mechanisms, immune system responses, treatment received, and temporal trajectories of disease progression.^13–23 However, even within just the clinical domain, existing approaches to characterize sepsis rely on relatively narrow criteria-based or laboratory groupings.^10,23–29 For example, the Systemic Inflammatory Response Syndrome criteria, which were used as a foundation for sepsis definitions in prior decades, include only 4 variables.²⁷ A more contemporary schema, the PIRO (Predisposition, Infection, Response, Organ dysfunction) model,³⁰ similarly uses a limited set of variables that are poorly representative of the true heterogeneity that clinicians witness in treating sepsis on a daily basis. Recent work evaluating clinical sepsis subgroups in observational and prospective clinical trial data relied on a circumscribed set of 29 vital sign, laboratory, and demographic parameters.²³

Heterogeneity also impacts how sepsis care quality is measured. The past several years have seen new guidelines and mandates emerge at the state, federal, and national levels that require protocolized care in all sepsis patients within highly constrained timelines (ie, within 6, 3, or even 1 hours).^1,24,31 However, these guidelines similarly fail to account for the variability in patient presentation and how these differences impact the timeliness of care. For example, antibiotic administration is measured against the same timeline whether a patient presents with obvious infectious symptoms of cough, fever, and purulent sputum or with more uncertain infectious symptoms, like diffuse abdominal pain and vomiting.¹³ Thus, characterizing clinical heterogeneity with greater depth is an essential first step toward understanding how to measure the adherence to and benefits of current treatment paradigms.

Clinical heterogeneity is a significant limitation to the development of new treatments and to accurately assessing sepsis quality of care and, yet, no current methods are available to quantify that heterogeneity with a computable, non–rules-based approach using comprehensive electronic health record (EHR) data. Machine learning methods have proven highly successful in empirically identifying groupings within large, complex data. In particular, a number of unsupervised learning approaches can successfully generate computable subgroups with high clinical relevance. While a diversity of methods currently exists (eg, clustering, neural network-based), prior work has shown that probabilistic topic modeling, for example, that based on the Latent Dirichlet Allocation (LDA) algorithm, can uncover relevant themes within complex EHR data.^32–37 Using a library of books as a conceptual example, LDA assesses the frequency and co-occurrence of words within individual books to identify the topics represented across the entire library. Based on the words they contain, individual books can also be represented as proportions of separate topics. Importantly, these statistical approaches allow the development of a computable phenotype or subgroup that captures greater complexity of patients, rather than assigning a single label based only on simple and limited rules.

OBJECTIVE

In this study, we used an unsupervised topic modeling approach to assess treatment heterogeneity during the first 24 hours of sepsis hospitalization and to develop computable clinical signatures based on these topics that describe overall treatment patterns for each patient. By applying LDA to a heterogeneous set of EHR data from the first 24 hours of sepsis treatment, we sought to empirically describe topics early in sepsis that would reflect underlying heterogeneity in sepsis presentation based on the diversity of treatment needs. We assessed how the resulting LDA-derived topics were distributed across the entire sepsis population as well as within individual patients. Finally, we assessed how these topics impacted common quality metrics of sepsis care to evaluate how their use could impact clinical practice and quality of care.

MATERIALS AND METHODS

Overall approach and cohort

This study was approved by the Kaiser Permanente Northern California (KPNC) Institutional Review Board.

Figure 1 provides an overview of our study’s approach to characterizing early clinical treatment heterogeneity among sepsis patients by applying topic modeling to granular EHR data. Table 1 defines the terminology used throughout this article. Our cohort was drawn from 35 000 adult sepsis hospitalizations occurring within the 21 hospitals of KPNC between 2010 and 2013.³⁸ Sepsis was defined based on the Sepsis-2 framework prevalent during that period and all patients were admitted through the emergency department and given antibiotics within 6 hours of triage. We included the first sepsis hospitalization for each patient (n = 29 253).

Figure 1.

Schematic overview of EHR data extraction and LDA implementation in hospitalized sepsis patients.

Abbreviations: EHR, electronic health record; LDA, Latent Dirichlet Allocation.

Table 1.

Terms used in describing the approach and results, and their meaning

Term	Description
Latent Dirichlet Allocation (LDA)	Unsupervised topic modeling approach that derives topics based on the frequency and co-occurrence of EHR items in hospitalizations, allowing treatment themes within individual hospitalizations to be represented by proportions of those topics.
EHR Items	Granular data objects drawn from the EHR within the first 24 hours of hospitalization. These items represent treatment decisions, including orders placed, medications given, and procedures ordered.
Topic	The 42 latent treatment patterns derived from LDA based on the frequency and co-occurrence of EHR items across all of the hospitalizations.
Topic Label	Summary clinical interpretation based on post hoc consensus interpretation of the highest weighted EHR items in each topic.
Computable Clinical Signature	The overall treatment profile for each patient based on proportions of each of the 42 topics.
Dominant Topic	The topic comprising the greatest proportion of each patient’s computable clinical signature.

Abbreviations: EHR, electronic health record; LDA, Latent Dirichlet Allocation.

EHR data items

We extracted EHR items indicating clinician actions within the first 24 hours of a sepsis hospitalization including electronic orders (n = 3 478 677), administered medications (n = 452 193), and procedures (n = 17 806; Table 2). We aggregated individual orders into order sets if they were part of the same established order set and had the same time stamp. We grouped medications by EHR subclasses (eg, glucocorticoids, glycopeptides), as classified in Epic Clarity EHR systems. We excluded any EHR item that appeared only once (n = 537), producing an EHR count matrix of 1 891 198 total and 2521 unique items. Supplementary Appendix Table 1 lists the most frequent EHR items identified.

Table 2.

Total volume of EHR data related to clinician action for patients in the sepsis cohort within the first 24 hours after ED triage. Total items represent each instance of any EHR item while unique items represent the different types of items (eg, hospital admission order set, serum potassium, glycopeptide antibiotic). The table shows the total number of items and unique items initially extracted from the EHR (left), those removed for appearing only a single time (middle italics), and those ultimately included (right) in the count matrix for LDA implementation. Of the 3 478 677 total orders drawn from the EHR, 2 305 061 were part of an established order set and were included together with other orders that were part of the same order set and had the same time stamp as 1 instance of that order set, resulting in 248 120 total order sets.

	Items initially extracted from EHR		Items removed (appearing only once)	Items included in LDA count matrix
EHR item type	Total	Unique	Unique	Total	Unique
Orders
Individual	1 173 616	1657	236	1 173 380	1421
Order sets	248 120	238	24	248 096	214
Medications given	452 193	699	89	452 104	610
Procedures	17 806	464	188	17 618	276
Total	1 891 735	3058	537	1 891 198	2521

Abbreviations: EHR, electronic health record; LDA, Latent Dirichlet Allocation.

EHR item count matrix and topics

To reduce the influence of frequently occurring EHR items found across many hospitalizations (eg, saline preparations), we applied a “term frequency-inverse document frequency” algorithm and scaled the resulting EHR item count matrix so that each value was an integer (Figure 1).³⁹ We then used LDA to surface latent treatment topics within each patient record.^32,33 The LDA implementation generates a topic matrix which represents a probability distribution of EHR items within each topic, which can be used to identify which EHR items are most associated with each treatment topic (Figure 1). It also generates a patient matrix which describes the composition of topics that describe each patient’s computable clinical signature.

Because LDA lacks prior specification about the latent topics being modeled, users must define k number of topics. To determine the optimal k, we defined k ranging from 25 to 75. We chose the optimal value of k based on the minimum median Bayesian Information Criterion (BIC) from multivariable logistic regression models with an outcome of hospital mortality based on 50 random iterations for each k.⁴⁰ The models demonstrated a BIC minimum and inflection point at k = 42 (Supplementary Appendix Figure 1).

While LDA empirically surfaces latent treatment topics based on EHR items, these topics require human interpretation. Therefore, our study team applied a post hoc clinical label to each topic (ie, applying labels including pneumonia, gastrointestinal bleeding, and mechanical ventilation) based on consensus interpretation of the highest weighted items represented in the topic matrix for each topic (Supplementary Appendix Table 2). Topics with at least 4 antibiotic or microbe culture items in the top 10 highest-weighted EHR items were considered treatment for infection. We also grouped these 42 topics within 11 broader organ- or treatment-based categories (eg, respiratory, gastrointestinal, and critical care).

Assessing clinical heterogeneity within and between sepsis patients

We used the LDA output to generate sepsis clinical signatures: computable and visualizable patient-level profiles showing the proportional composition of each topic within individual patients. Within each patient’s computable clinical signature, we identified their dominant topic—the single topic which comprised the largest proportion of their signature—as well as the second largest topic to assess how often sepsis hospitalizations could be defined by a small number of main treatment topics. To demonstrate how a computable clinical signature could help identify relevant subgroups within a highly heterogeneous population, we compared visual signatures of 9 randomly selected sepsis patients with 9 of those selected by specific treatment topic co-occurrence. To visualize treatment heterogeneity between patients, we used a chord plot to visualize dominant topic co-occurrence across the entire cohort. In each plot, individual patients are represented once with a line connecting their dominant and second largest topic within their clinical signature.

Evaluating the role of heterogeneity in sepsis measures

We assessed the 42 treatment topics across 8 common measures used to characterize sepsis patients, treatments, and outcomes including (1) the time from emergency department triage to the first antibiotic;³⁸ (2) the total volume of intravenous fluid administered within the first 24 hours^41,42; (3) hospital mortality^3,43; and (4) the maximum Sepsis-related Organ Failure Assessment Score (SOFA) during hospitalization^7,44; (5) age; (6) acute severity of illness (based on Laboratory Acute Physiology Score, LAPS2)^43,45–48; (7) chronic comorbid disease burden (Comorbidity Point Score, COPS2)^43,45–48; and (8) length of stay, based on established methods. In these comparisons, patients were included only once and grouped by their dominant topic. We used scatterplots to display antibiotic timing and fluid administration amounts as well as comorbid disease burden and hospital mortality to characterize how treatment topic heterogeneity modifies commonly used outcome and quality reporting metrics.

Data are reported as number (%), mean ± standard deviation, or median (interquartile range). We conducted analyses STATA/SE 14.2 and R version 3.4.2 including packages “dplyr,”⁴⁹ “lda,”⁵⁰ “doParallel,”⁵¹ and “circlize.”⁵² The R code used in this study is included in the Supplementary Appendix.

RESULTS

Our cohort included 29 253 patients with a mean (± SD) age of 70 ± 17 years (Supplementary Appendix Table 3); hospital mortality was 9.6%. Based on Sepsis-2 strata, 10 212 (34.9%) had sepsis, 15 059 (51.5%) had severe sepsis, and 3982 (13.6%) had septic shock. The median time to antibiotics was 2.1 hours (interquartile range: 1.4–3.1).

Labeling LDA-generated treatment topics

Table 3 and Supplementary Appendix Table 2 show the most highly weighted EHR items within each of the 42 treatment topics. In most cases, topics were clinically recognizable representing specific infections or treatment needs. For example, the top 5 items of latent topic 22 were: Clostridium difficile panel; contact plus isolation; stool culture; stool white blood cell count; and metronidazole. We labeled this topic “diarrhea.” Topic 3 (“congestive heart failure”) included congestive heart failure order set, troponin I, loop diuretic, B-type natriuretic peptide, and electrocardiogram. Topic 26 (labeled “anemia”) included iron and TIBC, ferritin, vitamin B12, folic acid serum, reticulocyte count, and transferrin.

Table 3.

Top 10 items for each of 42 statistically generated treatment topics from an unsupervised machine learning approach using EHR data drawn from the first 24 hours of sepsis hospitalization. Item order is determined by item weighting from the treatment topic matrix produced using LDA. In some cases, item names have been shortened for visual clarity. Additional detail available within Supplementary Appendix Table 2. Headers represent summary clinical labels assigned following empiric algorithm topic determination as well as 11 larger categories in parentheses (italics).

Acute coronary syndrome (Cardiovascular)	Atrial fibrillation (Cardiovascular)	Congestive heart failure (Cardiovascular)	Complex care (Complex)	End of life (Complex)	Failure to tdrive (Complex)
Troponin I	Prothrombin time	CHF OS	Inpatient admission OS	Palliative care CS 1	Tube feeding diet
ACS inpatient OS	Digoxin level	Troponin I	CBC with differential	Palliative care CS 2	CBC with differential
Echocardiography	Anticoagulants coumarin	Loop diuretic	Palliative care CS 1	Inpatient admission OS	Enteral alimentation OS
Electrocardiogram, 12	Inpatient admission OS	B-type natriuretic peptide	Palliative care CS 2	CBC with differential	Inpatient admission OS
Salicylate analgesics	Complete blood count	Electrocardiogram, 12	Urinalysis without micro	Comfort care order set	Phenytoin level
Creatine kinase-MB	Digitalis glycosides	Inpatient admission OS	Lactic acid	IV Sodium chloride	NPO, tube feeding
Cardiology CS 2	WBC differential, auto	Beta blockers	Admit to hospital	Narcotic agonists	Pneumonia OS
Cardiology CS 2	Beta blockers	CBC with differential	Hospitalist initial CS	Urinalysis without micro	Urinalysis without micro
B-type natriuretic peptide	Lactic acid	Class IV antiarrhythmic	Blood culture	Admit to hospital	Nursing order
Wound care (Complex)	Life support (Critical)	Mechanical ventilation (Critical)	Shock (Critical)	Critical illness (Critical)	Diabetes (Endocrine)
Heparins	Blood culture	Prothrombin time	Urinalysis, micro only	ESBL Penicillin antibiotic	ESBL Penicillin antibiotic
Inpatient admission OS	Chest X-ray	Chest X-ray	Septic shock – EGDT	Mg replacement OS	Insulin sliding scale OS
Wound nurse CS	Arterial blood gas	Arterial blood gas	ICU admission OS 1	K+ replacement OS	Inpatient admission OS
WBC differential, auto	CV sympathomimetics	Clinical restraints	Chest X-ray	Ionized calcium	CBC with differential
CBC with differential	Septic shock -- EGDT	Mechanical vent OS 2	CV sympathomimetics	Phos replacement OS	Diabetes mellitus OS
Nursing wound care	ICU admission OS 1	Ventilator therapy	Venous blood gas	Troponin I, POCT	Insulins - short acting
Glycopeptide antibiotics	Venous blood gas	Ventilation OS	Venous catheterization	Venous lactate, POCT	Urinalysis, without micro
Hospitalist CS 1	ABO-Rh	Endotracheal tube	IV Sodium chloride	ICU admission OS 1	Antipyretic non-narcotic
Urinalysis, micro	Clinical restraints	Anesthetic phenol	ICU admission OS 2	Lactate, POCT	Lactic acid
Hospitalist CS 2	Ventilator therapy	Benzodiazepines	CBC with differential	Critical care CS 2	Admit to hospital
Urinalysis, without micro	Ventilation OS	ICU admission OS 1	Manual differential	Critical care CS 1	Blood culture
Diabetic ketoacidosis (Endocrine)	Thyroid (Endocrine)	Hematologic panel (General)	Electrolyte panel (General)	Hepatic panel (General)	Severe illness (General)
Insulin regular human	Serum albumin	Draw and hold plasma 1	Serum magnesium	Serum AST	Troponin I, POCT
Ketone bodies	TSH level	Draw and hold serum	Phosphorus	Serum ALT	Lactate, POCT
DKA OS	Prealbumin	Draw and hold EDTA	Serum calcium	Alkaline phosphatase	Venous lactate, POCT
Insulins - short acting	Serum magnesium	Draw and hold serum	Chemistry panel 7	Total bilirubin	Inpatient admission OS
Blood glucose, POCT	Serum calcium	Draw and hold plasma 2	CBC with differential 2	Serum magnesium	Venous blood gas
Insulin sliding scale OS	Phosphorus	Draw and hold heparin	WBC differential, auto	Serum calcium	Electrocardiogram, 12
ICU admission OS 1	Chemistry panel 7	CBC with differential	Lactic acid	Phosphorus	WBC differential, auto
Intensive insulin OS	WBC differential, auto	Inpatient admission OS	CBC with differential	Chemistry panel 7	CBC with differential
Chemistry panel 7	Incentive spirometry	Manual differential	Serum albumin	Lipase	Admit to hospital
Ketone, serum	Inpatient dietary CS	Blood culture 2	Serum potassium	CBC with differential	Pneumonia OS
Abdominal pain (Gastrointestinal)	Biliary disease (Gastrointestinal)	Diarrhea (Gastrointestinal)	Liver disease (Gastrointestinal)	Gastrointestinal bleeding (Gastrointestinal)	Hepatitis (Gastrointestinal)
CT abdomen/pelvis	Serum ALT	Clostridium difficile panel	Ammonia	ABO-Rh	Hepatitis C antibody
Narcotic agonists	Serum AST	Contact plus isolation	Ultrasound abdomen	Transfusion OS	Hepatitis B surface Ag
Gastroenterology CS 1	Alkaline phosphatase	Stool culture	Acute renal failure OS	Antibody screen	Hepatitis B surface Ab
Gastroenterology CS 2	Total bilirubin	Stool WBC	WBC differential, auto	Hemoglobin/hematocrit	Hepatitis B core antibody
Inpatient admission OS	Lipase	Metronidazole	Laxatives	Type and crossmatch	Hepatitis A virus IgM
CBC with differential	Chemistry panel 7	Clostridium difficile toxin	Services after hours	Type and screen	HIV 1/2 antibody
General surgery CS 2	CBC with differential	Inpatient admission OS	CBC with differential	Crossmatch, immediate	Ultrasound abdomen
General surgery CS 1	Lactic acid	CBC with differential	Sepsis ED OS	Crossmatch, electronic	US abdomen, B-scan
Metronidazole	WBC differential, auto	Protozoa smear	Draw and hold pink top	Prothrombin time	Antinuclear antibody
Serotonin antagonist	Serum amylase	WBC differential, auto	CT, 3D rendering	GI bleed OS	Acetaminophen level
Surgery (Gastrointestinal)	Anemia (Hematologic)	Cytopenia (Hematologic)	Coagulopatdy (Hematologic)	Cellulitis (Musculoskeletal)	Indolent infection (Musculoskeletal)
General surgery CS 1	Iron and TIBC	Transfusion OS	Lactate dehydrogenase	US Venous Doppler	ESR
General surgery CS 2	Ferritin	ABO-Rh	Fibrinogen activity	Cellulitis OS	C-reactive protein
Surgery admission OS	Vitamin B12	Antibody screen	APTT	Inpatient admission OS	Infectious disease CS 1
Surgery/intra-op OS	Folic acid, serum	Neutropenic fever OS	Fibrinogen degradation	Vancomycin level, trough	Infectious disease CS 2
Narcotic agonists	Reticulocyte count	WBC differential, manual	Serum ALT	CBC with differential	Orthopedics CS 2
PACU/anesthesia OS	Transferrin	CBC with differential	D-dimer	Glycopeptide antibiotics	Orthopedics CS 1
Transfer level of care	Occult blood specimen	Antipyretic non-narcotic	Serum AST	WBC differential, auto	Narcotic agonists
CT abdomen/pelvis	Protein electrophoresis	Heme-Onc CS 1	Total bilirubin	Narcotic agonists	Vancomycin level, trough
Anesthetic – narcotic	TSH	Inpatient admission OS	Alkaline phosphatase	US duplex scan	Glycopeptide antibiotics
Anaerobic culture	Head and neck CS 1	Heme-Onc CS 2	Prothrombin time	Lactic acid	CBC with differential
Osteomyelitis (Musculoskeletal)	Weakness (Neurologic)	Confusion (Neurologic)	Acute kidney injury (Renal)	Dialysis (Renal)	Pyelonephritis (Renal)
Podiatry CS 1	CT head, no contrast	Drug screen, urine	Sodium, urine	Hemodialysis OS	Urology CS 1
Podiatry CS 2	CT head	CT head, no contrast	Creatinine, urine	Nephrology CS 2	Urology CS 2
Radiologic exam, foot	Inpatient admission OS	CT head	Osmolality, urine	Nephrology CS 1	Urine culture
Culture, miscellaneous	WBC differential, auto	Alcohol withdrawal OS	Osmolality, serum	Phosphate binders	CBC with differential
Gram stain	CBC with differential	Alcohol level	Sodium, serum	Vancomycin level	Gentamicin level
Insulin sliding scale OS	Creatine kinase	MRI brain	Ultrasound abdomen	Calcineurin inhibitors	CT abdomen/pelvis
Anaerobic culture	Physical therapy	Lumbar puncture OS	Eosinophils, urine	Insulin sliding scale OS	IV Gentamicin
Vancomycin level, trough	Urinalysis, without micro	Neurology CS 1	Chemistry panel 7	Diet, renal	Lactic acid
Glycopeptide antibiotics	Troponin I	Neurology CS 2	Acute renal failure OS	Mycophenolate	Narcotic agonists
Urinary tract infection (Renal)	Atypical pneumonia (Respiratory)	COPD/Astdma (Respiratory)	Non-invasive ventilation (Respiratory)	Pneumonia (Respiratory)	Viral pneumonia (Respiratory)
Inpatient admission OS	AFB culture and smear	Pneumonia OS	BIPAP ventilation	Pneumonia OS	Pneumonia OS
CBC with differential	Pneumonia OS	Glucocorticoids	Arterial blood gas	Inpatient admission OS	Droplet isolation
Narcotic agonists	CT chest	COPD/asthma OS	Pneumonia OS	CBC with differential	Inpatient admission OS
WBC differential, auto	L. pneumophila Ag	Inpatient admission OS	Non-invasive ventilation	WBC differential, auto	CBC with differential
Antipyretic non-narcotic	M. pneumoniae IgM	Respiratory culture	Glucocorticoids	Glucocorticoids	Flu A/B, RSV PCR
Lactic acid	Respiratory culture	Respiratory gram stain	B-type natriuretic peptide	COPD/asthma OS	WBC differential, auto
Urinalysis, without micro	Respiratory gram stain	CBC with differential	ICU admission OS	Admit to hospital	Antipyretic non-narcotic
Serotonin antagonists	Pulmonary initial CS	Lactic acid	Chest X-ray	Blood culture	3rd gen Cephalosporin
IV Sodium chloride	Head and neck CS 1	Heme-Onc CS 2	Prothrombin time	Blood culture 2	Neuraminidase inhibitors

ACS, acute coronary syndrome; CBC, complete blood count; CS, consult; CHF, congestive heart failure; CV, cardiovascular; CT, computed tomography; DKA, diabetic ketoacidosis; EGDT, early goal directed therapy; ESBL, extended-spectrum beta-lactamase; ICU, intensive care unit; IV, intravenous; K+, potassium; Mg, Magnesium; NPO, nil per os; Phos, phosphorus; POCT, point of care testing; US, ultrasound; WBC, white blood cell.

Figure 2 shows the overall occurrence of each treatment topic across the entire study cohort with the most prevalent topics attributable to “diabetes” (6.0%), “viral pneumonia” (5.4%), “pneumonia” (4.8%), and “urinary tract infection” (4.7%). Evaluating the composition of topics across the entire cohort, only 39.1% of treatments were directly for infections, while the majority of treatment was for noninfectious causes of hospitalization.

Figure 2.

Aggregate representation of each of 42 statistically generated treatment topics based on electronic health record data, with post hoc assigned clinical labels (top) and categories (bottom and color bars). The width of each individual colored bar represents the proportion of that treatment topic within the sepsis cohort. The highest aggregate proportions are attributable to “diabetes,” “viral pneumonia” (viral PNA), “pneumonia,” and “urinary tract infection” (UTI).

Abbreviations: Cardio, cardiovascular; COPD, chronic obstructive pulmonary disease; GI, gastrointestinal; Heme, hematologic; MSK, musculoskeletal; Neuro, neurologic; PNA, pneumonia; UTI, urinary tract infection.

Computable clinical signatures and heterogeneity within sepsis patients

Clinical signatures are the proportional representation of treatment topics within individual patients and facilitate computable approaches to describing heterogeneity within each patient. In our cohort, we found that 56.9% of hospitalizations did not have a single dominant topic which accounted for more than half of their overall clinical signature, demonstrating that most sepsis patients’ treatments could not be defined only by a single label (Supplementary Appendix Figure 2). Only a small minority (7.3%) of patients had a single dominant topic that comprised >80% of their clinical signature, quantifying the clinically familiar scenario in which most sepsis patients are treated concurrently for multiple co-existing conditions.

Figure 3 compares the visual representation of clinical signatures of 9 randomly assigned sepsis patients (left) with another 9 randomly chosen but based on 3 specific pairings of treatment topics (cellulitis and pneumonia; complex care and diarrhea; and heart failure and urinary tract infection) on the right. The left panel displays the heterogeneity present among randomly assigned patients with diverse combinations of treatment topics, yet all were defined as “sepsis” patients. The computable signature approach allows for sepsis patients to be defined by key dominant topics that can be used to identify similar subgroups within the overall sepsis population, as shown on the right, where signatures are similar across patients.

Figure 3.

Computable clinical signatures of individual patients based on the LDA topic modeling approach. Each bar color represents a different topic as displayed in Figure 2 and the width of the color bar represents the proportion of the clinical signature that topic composes. On the left are 9 randomly selected sepsis patients, including 3 each from sepsis (top), severe sepsis (middle), and septic shock (bottom) severity strata. On the right are 9 sepsis patients randomly chosen but based on 3 specific pairings of treatment topics (overall clinical signature comprised of at least 0.33 from both cellulitis and pneumonia; complex care and diarrhea; and heart failure and urinary tract infection).

Heterogeneity in treatment was present not only within individual patients, but also across the entire sepsis population. Figure 4 displays the aggregate frequency of topic co-occurrence with each link representing a single hospitalization and exhibits the tremendous diversity in topics across the cohort. Of a total of 29 253 co-occurrence topics, even the most common ones were relatively rare, including: abdominal pain and biliary disease (n = 254, 0.9%), chronic obstructive pulmonary disease and diabetes (n = 222, 0.8%), diabetes and cellulitis (n = 217, 0.8%), and viral pneumonia with acute coronary syndrome (n = 165, 0.6%). The circle plot confirms that sepsis patients are highly diverse in their clinical signatures in a way that would not be easily characterized by a simple set of criteria.

Figure 4.

“Dominant topic” chord plot representing the co-occurrence of EHR topics within individual computable clinical signatures. The 42 topics are arrayed on the periphery with the width of each band representing the number of patients with that topic being their dominant or second topic in the clinical signature. Each line represents a single hospitalization connecting a dominant topic (bands around the periphery and lines arising from the bands of the same color) to the next topic (endpoint of the line with different color than the adjacent band). The width of the lines represents the number of hospitalizations with that same co-occurrence as the dominant and second topics in their clinical signature.

Abbreviations: AbdPain, abdominal pain; ACS, acute coronary syndrome; AFib, atrial fibrillation; AKI, acute kidney injury; CHF, congestive heart failure; COPD, chronic obstructive pulmonary disease; DKA, diabetic ketoacidosis; FTT, failure to thrive; GIBleed, gastrointestinal bleeding; NIPPV, non-invasive ventilation; Osteo, osteomyelitis; PNA, pneumonia; UTI, urinary tract infection.

Evaluating treatment topics and sepsis measures

The heterogeneity revealed in the clinical signatures and circle plot also had significant impact on commonly used sepsis measures of care processes and outcomes. For example, Figure 5a displays the variation in commonly measured sepsis care processes (antibiotic timing and fluid resuscitation) across the population when patients were grouped by their dominant topics. Among an overall cohort that all received antibiotics within a very compressed emergency department timeline, the mean time to antibiotics was shorter (<2.2 hours) for conditions in which patient presentation was much more clear, including those requiring intensive care (ventilation, critical illness, shock) and with clinically obvious infections (ie, cellulitis, osteomyelitis, pneumonia). In contrast, among patients who had more uncertain presentations like weakness or abdominal pathology (abdominal pain, diarrhea, hepatitis, liver disease), the time to antibiotics was considerably longer on average.

Figure 5.

Clinical sepsis measures, stratified by dominant treatment topic of each patient’s computable clinical signatures during sepsis hospitalization. a) Mean time to first antibiotic from emergency department triage in relation to mean amount of intravenous fluid administered in the first 24 hours of hospitalization; b) Mean hospital mortality in relation to mean chronic comorbid disease burden (COPS2) score.

Similarly, large fluid volumes (>3 liters) were given to patients requiring intensive care and to other conditions that commonly require substantial fluid resuscitation (diabetic, coagulopathy). In contrast, small fluid volumes (<1.6 liters) were given to patients on dialysis or with heart failure, who are at increased risk of fluid overload, reflecting clinically familiar patterns. When antibiotic timing and fluid resuscitation were arrayed against one another, the LDA-based groupings revealed the challenge of using a “one-size-fits-all” approach to measuring adequacy in early sepsis treatment.

Figure 5b also shows considerable variability in comorbid disease burden and hospital mortality in sepsis when patients were grouped by their dominant treatment topics. Not surprisingly, patients with very high inpatient mortality (>20%) included not only those with critical illness, but also those with end-of-life care needs and coagulopathy. On the other hand, even patients with a very high presepsis burden of illness often exhibited low hospital mortality. For example, among those with substantial preexisting disease and with dominant topics of atrial fibrillation, end-stage kidney disease, or wound care, mortality was relatively low at <8%. Supplementary Appendix Table 4 and Supplementary Appendix Figure 3 similarly show wide variability in the characteristics and outcomes across the 42 topics.

DISCUSSION

In a multicenter cohort of sepsis patients treated with early antibiotics, we used machine learning to empirically identify EHR-based topics and develop computable clinical signatures to quantify the treatment heterogeneity present in early sepsis. Applying an unsupervised approach to nearly 2 million EHR items and 30 000 patients, we uncovered 42 treatment patterns or topics that were clinically recognizable and displayed the breadth and diversity of treatments used in the early part of hospitalization. Our findings highlighted the fact that, while all these patients were “septic,” their actual clinical signatures—the composition of treatment topics within a single patient—belied easy characterization by any single label. Only a minority of patients were even found to have had a single dominant topic that explained most of their hospitalization. Thus, our findings quantitatively demonstrate that singular or narrowly defined sepsis groupings fail to capture the true clinical and treatment diversity that comprises early sepsis. Similarly, when we assessed treatment topics across the entire cohort, we found tremendous heterogeneity. Further, because we were able to quantify the contribution of different topics throughout the population, we found that only 39.1% of overall treatments were definitively for infection. In sum, our study describes a computable and empiric approach to display and characterize the profound clinical heterogeneity of early sepsis treatment within individual sepsis hospitalizations and across the entire sepsis population.

While heterogeneity is universally cited as a key barrier to progress in sepsis research and treatment, to our knowledge, this is the first study that actually quantifies this treatment heterogeneity in the clinical domain and uses computable clinical signatures as a means for identifying diverse subgroups of patients.^12–15 Traditional approaches to characterizing the clinical dimensions of sepsis rely on rules- or criteria-based frameworks^{10,24–29,53,54} and have shown value for identifying high-risk patients,^19,25,44 standardizing treatment protocols,^24,55,56 and enabling outcomes comparisons.^10,27 However, they categorize patients across very narrow dimensions and, because of their significant limitations in capturing the diversity that is recognized clinically in sepsis, are rarely used. Rather than relying on a proscriptive approach that would require extensive clinical labeling and data curation, we sought to leverage machine learning approaches that would surface treatment subgroups without preexisting bias. We also chose to focus on clinician actions mediated through the EHR, because these digital artifacts would simultaneously capture underlying patient characteristics and clinician judgment in a way that common EHR data models might not. Finally, we chose to focus on the first 24 hours of hospitalization in order to describe sepsis heterogeneity during the most dynamic interval of inpatient care.

Our findings confirm the clinical reality that traditional approaches which rely on single labels to characterize a hospitalization—“this patient has pneumonia”—routinely fail to capture the diversity of coexisting clinical conditions present in early sepsis. Indeed, we found that for nearly half of patients with a main treatment topic of “pneumonia,” the majority of their overall clinical signature was explained by non-“pneumonia” topics. Our findings have important implications on future research in sepsis, which is currently at a crossroads when it comes to identifying clinically actionable subgroups that will be similarly responsive to treatment.^{12,13,22,23,57} This is of particular salience because sepsis has seen every novel therapy fail in randomized trials over the prior 5 decades. Heterogeneity is now universally identified as the major barrier to progress; however, no other methods are currently available to empirically quantify and characterize this clinical treatment diversity. Thus, even while clinicians recognize the conundrum of applying a “one-size-fits-all” treatment to highly variable patients—a commonly recounted scenario is that the same approach is taken for a young healthy patient with pneumonia as for a chronically ill elderly patient with immunosuppression and urosepsis—the lack of computable approaches means that this blunt approach to sepsis care continues to persist.¹²

Our findings also have important implications for current metrics that are used to assess and report quality of care in sepsis. Sepsis was recently recognized by the World Health Organization as a global health priority and is the subject of many public health awareness campaigns.¹ This highly recognized status has also spurred the development of national and international standards and guidelines that use compliance with timed bundles to grade hospitals on their sepsis performance.^24,31 However, as we show in this study, there is tremendous variation in the timing of antibiotics and the volume of fluid resuscitation that is attributable to the complement of coexisting clinical conditions within each patient. On average, patients with abdominal pathology received antibiotics the latest, reflecting the uncertainty of confirming infection as the reason for symptoms in these patients. Similarly, patients with conditions marked by a high risk for fluid overload—congestive heart failure and kidney disease with dialysis—received the lowest volume of resuscitation. Again, a “one-size-fits-all” approach for measuring sepsis care quality ignores the reality of underlying diversity that is revealed when computable clinical signatures can be used to quantitatively describe sepsis clinical heterogeneity.

There are several potential future applications and refinements to our approach that can facilitate improved scientific discovery and clinical treatment in sepsis. First, this method can be applied to existing randomized controlled trial or observational data to understand how patients’ clinical signatures modify their response to treatment. For example, recent landmark trials compared various protocolized treatment approaches in sepsis and found no differences in outcomes between patients.⁵⁸ Quantifying the clinical signatures of individual patients has begun to show promise for revealing subgroups within the overall study population who responded differentially to protocolized care.²³ We have provided our code so that our approach is easily reproducible in any EHR-based data set. Second, quantifying the clinical heterogeneity in sepsis patients can help ensure that public reporting sepsis metrics are applied to the right population. For example, the timing of antibiotic administration should account for differences in early treatment when infections are easily identifiable (eg, cellulitis, pneumonia) versus when they are more challenging (eg, abdominal symptoms, weakness). Finally, identifying clinical subgroups in real-time could help enhance medical recommender systems,³³ resource allocation, and targeted care.⁵⁹

It is essential to note that, in this study, we examined early sepsis heterogeneity by focusing on treatment patterns captured with clinical EHR data. However, sepsis heterogeneity arises from several sources including genetic factors, host-pathogen interactions, immune system responses, pathophysiologic disease mechanisms, and temporal trajectories of illness.^12–22,25 What remains unknown is the degree to which the treatment heterogeneity we observed correlates with these other dimensions. For example, it may be that sepsis endotypes^,12,23 (subgroups that capture similarity across disease mechanisms or host responses) can cluster patients together who exhibited highly disparate computable clinical signatures but would respond positively to the same treatment. What is also unknown is the extent to which the treatment heterogeneity we observed among sepsis patients is common to other inpatients. For example, is the hallmark of heterogeneity in sepsis treatment substantially greater than that present in other acute, high-impact conditions like heart failure?

The primary strength of our study was the careful use of an empiric data-driven approach to identify treatment topics and clinical signatures without specifying any preexisting categorization or criteria. We evaluated the statistically-generated topics against clinical documentation and further compared them across a set of common sepsis measures. These comparisons confirmed wide variability in treatment topics and outcomes belying the population means. Our study thus demonstrates the potential value of this approach for precisely quantifying and comparing clinical heterogeneity within and between populations using treatment topics within granular EHR data.

The main limitation of our study is that it was designed for hypothesis generation; thus, future studies are needed to confirm that these topics and computable clinical signatures reliably distinguish clinical subgroups that are responsive to differential treatments. Second, our study was conducted within a single health care system which may impact the generalizability of our findings. Third, we could not account for potential heterogeneity arising from individual clinical practice which could impact the reliability of topic generation. It is possible that some of the heterogeneity we captured actually arises from differences in practice rather than differences among sepsis patients. Fourth, while we assigned summary clinical labels to the treatment topics to improve recognition, the labels should be viewed as only approximations. Similarly, we used an empirical approach for identifying the optimal number of topics based on the findings of prior studies; however, it is possible that we captured only a local minima for BIC in our data. Finally, we limited ourselves to a single interval in hospitalization which does not fully capture the preceding and subsequent trajectory of illness. We also did not incorporate the longitudinal sequencing of EHR items.

In summary, in a multicenter cohort of sepsis patients, we applied machine learning to generate computable EHR-based clinical signatures that quantified treatment topics and, therefore, clinical heterogeneity in early sepsis care. Our findings confirmed that substantial treatment heterogeneity in sepsis manifests at both the patient- and population-level. Future research is needed to establish whether the profound heterogeneity we uncovered can drive improvements in the targeted and personalized care of sepsis patients.

FUNDING

This work was supported by The Permanente Medical Group, the Kaiser Permanente Division of Research Delivery Science Fellowship (AEF), and NIH/NIGMS R35GM128672 (VXL).

CONTRIBUTIONS

All authors have contributed substantially to the conception or design of the work or the acquisition, analysis, or interpretation of data for the work; and to drafting the work or revising it critically for important intellectual content. All authors approve of the version submitted and agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Specific author contributions to project conception, analysis, interpretation, and completion are as follows: Design (AEF, JC, PK, GJE, VXL); Acquisition/analysis (AEF, JDG, BLL, VXL); Interpretation (AEF, JC, PK, GJE, VXL) Drafting/revising/final approval/agreement (all authors).

SUPPLEMENTARY MATERIAL

Supplementary material is available at Journal of the American Medical Informatics Association online.

Conflict of interest statement

None declared.