Learning Bundled Care Opportunities from Electronic Medical Records

Abstract

Objective

The traditional fee-for-service approach to healthcare can lead to the management of a patient’s conditions in a siloed manner, inducing various negative consequences. It has been recognized that a bundled approach to healthcare - one that manages a collection of health conditions together - may enable greater efficacy and cost savings. However, it is not always evident which sets of conditions should be managed in a bundled manner. In this study, we investigate if a data-driven approach can automatically learn potential bundles.

Methods

We designed a framework to infer health condition collections (HCCs) based on the similarity of their clinical workflows, according to electronic medical record (EMR) utilization. We evaluated the framework with data from over 16,500 inpatient stays from Northwestern Memorial Hospital in Chicago, Illinois. The plausibility of the inferred HCCs for bundled care was assessed through an online survey of a panel of five experts, whose responses were analyzed via an analysis of variance (ANOVA) at a 95% confidence level. We further assessed the face validity of the HCCs using evidence in the published literature.

Results

The framework inferred four HCCs, indicative of 1) fetal abnormalities, 2) late pregnancies, 3) prostate problems, and 4) chronic diseases, with congestive heart failure featuring prominently. Each HCC was substantiated with evidence in the literature and was deemed plausible for bundled care by the experts at a statistically significant level.

Conclusions

The findings suggest that an automated EMR data-driven framework conducted can provide a basis for discovering bundled care opportunities. Still, translating such findings into actual care management will require further refinement, implementation, and evaluation.

Graphical abstract

Introduction

Under a fee-for-service healthcare model, each of a patient’s conditions is managed relatively independently [1–2]. This approach to care can lead to several problems, including delays in (or failure to deliver) service, testing and treatment redundancies, and increased costs for healthcare organizations (HCOs) and patients. In turn, these problems can lead to declines in quality, patient satisfaction, and cost effectiveness [3]. It is anticipated that a shift from fee-for-service to pay-for-value has the potential to resolve, or at least reduce the severity of, many of these problems [4–5]. To realize this alternative vision, HCOs are migrating towards a bundled care model, which is a middle ground between F4S and capitation reimbursement that aims to account for the interplay between various health conditions, rather than focus on each in isolation [6–7].

There are numerous challenges in realizing bundled care. Two of the more pressing are: 1) it is not always evident which health condition collections (HCCs) are appropriate for such a care model and 2) the cost of refining current healthcare systems to support bundled care should be minimized. While HCOs already manage certain complex health needs of patients (e.g., management of comorbidities when treating the primary health problem), such routines often arise in an ad hoc fashion and are not formalized. As such, there is an opportunity to design a data-driven approach to learn HCCs, which are, or could be, ripe for bundling. A data-driven approach may be further beneficial because, if models are based on the current activities of healthcare professionals, then HCOs could minimize implementation costs for newly established, or the formalization of existing, management routines.

Towards this goal, there is growing evidence that data derived from electronic medical records (EMRs) can be mined to discover associations between health problems [8–14], infer clinical phenomena [15–18], and model healthcare workflow patterns [19–23]. More recently, it has been shown that the relationship between health problems and workflows can be specialized for certain phenomena, such as congestive heart failure [24]. In this paper, we build on such observations and introduce an automated learning framework to discover more general HCCs, which share similar workflows according to the utilization records of an EMR system. We hypothesize that such HCCs could be bundled and managed together based on their shared workflows.

In this paper, we report on how we accomplished this goal by designing a data-driven framework that relies upon a generative topic modeling strategy to infer an association network between HCCs and workflows. We applied a community detection algorithm to infer HCC clusters via the association network. We evaluated this framework with four months of inpatient data (over 16,500 inpatient stays) from Northwestern Memorial Hospital (NMH) and confirmed the plausibility of inferred HCC clusters through an online survey with administrative and clinical experts. We further demonstrated the face validity of HCC clusters through evidence in the published literature.

Background

The past several years have witnessed a number of investigations both expert- and data-driven, into modeling and characterizing clinical phenotypes and workflows. However, there has been limited research into automatically establishing relationships between these phenomena. In this section, we review representative expert and data-driven methodologies and illustrate their relationship with our own approach.

Phenotyping Algorithms

Phenotyping aims to use EMR data to detect phenotypes of clinical interest. There has been a flurry of activity in various learning methods for high-throughput phenotyping over the past several years. There are two typical phenotyping algorithms: expert- and data-driven, the latter of which can be further partitioned into supervised and unsupervised techniques.

Expert-driven methods leverage rules developed by experts to identify phenotypes. These methods require substantial manual effort from domain experts. For instance, Kho et. al. developed rule-based phenotyping algorithms to identify subjects for five primary phenotypes (e.g., type 2 diabetes) to support further analysis in genomic association studies [61–62]. Their phenotyping algorithms were developed based on the analysis of EMR data and criteria managed by the Centers for Disease Control and Prevention.

Supervised data-driven phenotyping requires domain experts to exert a substantial of human manual effort to label cases (i.e., phenotypically positive individuals) and controls (i.e., phenotypically negative individuals) and then train and calibrate classifiers. For each new patient record, the classifiers can determine their class label (e.g., case versus control) according to their features. The NIH-sponsored Electronic Medical Records and Genomics (eMERGE) [51] consortium, a network of academic medical centers, has developed a variety of supervised phenotyping algorithms [15–16, 52–54]. For instance, Chen et. al. designed a supervised learning algorithm to detect rheumatoid arthritis, colorectal cancer, and venous thromboembolism [52]. Additionally, Zheng et. al. developed a machine learning algorithm to identify type 2 diabetes from EMR data [18].

Unsupervised data-driven phenotyping does not require manual chart review, but instead automatically clusters phenotypes or subtypes [17, 24, 26, 55–56]. The challenge for this type of learning is in the validation of the discovered phenotypes or subtypes. Specifically, there is no clear ground truth for the phenotypes or subtypes that are identified through such an approach. A traditional approach to evaluate the learned phenotypes is to involve clinical and administrative experts to let them review their plausibility [17, 64]. For instance, Ho et. al developed a tensor factorization model to automatically identify phenotypes for several major diseases (e.g., metabolic syndrome) from EMR data and, subsequently, recruited expert respondents to review their plausibility. Additionally, Chen et. al. developed a unsupervised learning algorithm to infer phenotypes from EMR data coming from two distinct healthcare systems and validated the similarity, stability and transferability of the learned phenotypes [26].

Workflow Modeling Algorithms

Workflow modeling algorithms can be grossly categorized into two types: i) observational and ii) data-driven.

Observational studies often rely on manual data collection approaches, such as observations and interviews. One such example was presented by Unertl and colleagues [63], which analyzed direct observations and interviews in hospitals to understand workflow and information flow in the care of chronic diseases. Data-driven algorithms, by contrast, have been proposed to infer clinical pathway patterns through the activity logs of healthcare systems [24–25, 57–59]. Almost all of these approaches followed a similar style: i) infer workflow patterns, then ii) evaluate the effectiveness of the methods in a clinical case study. For instance, Bouarfa et. al. derived a workflow consensus from clinical activity logs to detect outlying workflows without prior knowledge from experts [57]. They adopted a tree-guided multiple sequence alignment approach to model the consensus of workflows. This strategy was validated over the workflow processes associated with laparoscopic cholecystectomy, where the results indicated the derived consensus conforms to the main steps of the surgical procedure as described in best practice guidelines. In another example, Chen et. al. introduced an altered latent Dirichlet allocation (LDA) based framework to infer clinical workflows through the utilization of an EMR [25] and applied such framework to infer 8 different types of workflows for heart failure patients [24].

To date, phenotyping and workflow modeling algorithms have been developed independently. While there has been little investigation into relating these concepts together, this is vital to the establishment of effective care coordination strategy [60].

Research Design and Methods

The framework for relating phenotypes and workflows is generally composed of four parts: i) a workflow inference module, which is based on the electronically documented actions of EMR users, ii) a HCC inference module, based on patient-specific clinical phenomena indicated in an EMR (e.g., diagnosis codes), iii) an association module, which constructs the association network of HCCs and workflows, and then infer HCC clusters according to the similarity in their workflow patterns and iv) an evaluation module, which consists of online surveys from administrative and clinical experts to determine if the HCC clusters are worthy of consideration for bundling.

We begin with a high-level overview of the models and then proceed with a deeper dive into each component. The general relationships between the workflow module, HCC model and association modeling algorithm are depicted in Figure 1.

Figure 1.

A high-level architecture for discovering associations between clinical workflows and HCCs, which are further leveraged to infer HCC clusters. (Legend: e = action event, p = EMR patient, d = diagnosis, HCC = Health condition collection, s = action sequence, s′ = action subsequence and w = workflow)

Here, we take a moment to formalize the environment. Let P = {p₁, p₂, ⋯, p_n} be the set of patients, S = {s₁, s₂, ⋯, s_n} be the set of action sequences (issued by approved EMR users) and D = {d₁, d₂, ⋯, d_l} be the set of clinical phenomena (e.g., diagnosis codes). Each patient h_i in H is defined as a sequence s_i in S (as shown in Figure 1a) and a collection of clinical phenomena in D, as shown in Figure 1e. The set of workflows W = {w₁, w₂, ⋯, w_k} (Figure 1b-left) and HCCs HCCs = {h₁, h₂, ⋯, h_q} (Figure 1d-right) are learned from S and D, respectively. Specifically, a workflow w_i is defined as a probability distribution over a set of subsequences in S′ = {s′₁, s′₂, ⋯, s′_q} (Figure 1b-left). s′_i is defined as a subsequence that frequently occurs across the sequences in S. An HCC h_j is a probability distribution over a set of diagnoses (e.g., Figure 1d-right depicts three HCCs).

A patient is explained by their affinity to workflows and HCCs through φ_W (Figure 1b-right) and φ_H (Figure 1d-left), respectively. For instance, as shown in Figure 1b-right, workflow w₁ has a probability of 0.8 of explaining the affinity between the sequence for patient p₁ and w₁. The strength of association between a workflow and an HCC is measured via the set of patients in common they explain. The HCC clusters are inferred via the association network that was established between the HCCs and workflows (as shown in Figure 1c).

To focus on the knowledge learned from the EMR, we relied on existing inference algorithms to learn workflows and HCCs. For orientation, we provide two examples as shown in Figure 2 and 3 to illustrate how the algorithms work, but refer the reader to [25] and [26] for a more detailed description.

Figure 2.

The process by which workflows are generated from patient sequences. The input is the patient sequences (see a), and the outputs are the topics (workflows) (see d) and the explanations between the patient sequences and the topics (see e).

Figure 3.

The process by which HCCs topics are generated from patients’ diagnoses. The input is the patient diagnosis codes (see a), and the outputs are HCC topics (see b) and explanations between the topics and the patient diagnoses (see c).

Workflow Inference Algorithm

The workflow inference algorithm [25] learns topics, W = {w₁, w₂, ⋯, w_k}from the sequences in S via a modified LDA algorithm [27–28]. S is a set of patient sequences. Each patient sequence was revised to retain the strong event relations and filter out the weak event relations. The strength of an event relation is measured as the number of times the relation appearing in patient sequences S. As shown in Figure 2, if we remove event relations with scores less than a threshold of 2, then s₃ = e₂ → e₁ → e₅ → e₆ (e.g., both e₂ → e₁ and e₁ → e₅ have a score of 1) will be revised to s₃ = e₅ → e₆. Each revised sequence is made up of subsequences, each of which is a series of events with strong relations between neighbors. For instance, s₂ = e₁ → e₂ → e₅ → e₆ was revised to consist of two subsequences: s′₁ = e₁ → e₂ and s′₂ = e₅ → e₆. Briefly, the set of workflow topics is inferred from a matrix R_|P|×|S′|. Here, R_|P|×|S′|(i, j) corresponds to the number of times a subsequence s′_j was in a patient sequence s_i. Each workflow topic is represent by a set of subsequences with their corresponding probabilities. For instance, as shown in Figure 2, the algorithm inferred two workflow topics. The first topic t₁ consists of two subsequences e₁ → e₂ and e₅ → e₆, and the second topic t₂ consists of one subsequence e₄ → e₃. φ_W corresponds to a matrix of the likelihoods that the patients’ sequences in S are explained by the topics in W. As shown in Figure 2, the first two patient sequences are explained by topic t₁, while the last two patient sequences are explained by topic t₂.

It is often the case that the fitness of an LDA model, and thus the number of topics k, is determined through an information theoretic measure, such as perplexity [27–28]. However, in our situation, we aim to determine the value that maximizes the separation between the workflow topics, which are more semantically meaningful. As such, we calibrate k by setting it to the value that minimizes the average covariance between the workflow topics.

HCC Modeling Algorithm

The HCC modeling algorithm [26] also learns topics HCCs = {h₁, h₂, …, h_q} via a modified LDA method as well. Briefly, the set of topics is inferred from a matrix R_|P|×|D|. Here, R_|p|×|D|(i, j) corresponds to the number of times that diagnosis code d_j was assigned to patient p_i. Figure 3 depicts an example of three topics. Each topic is represented by a collection of diagnosis codes. For instance, as shown in Figure 3b, topic t₁ consists of d₁ and d₂, t₂ consists of d₃ and d₄, and t₃ consists of d₅ and d₆. φ_h is a matrix of the likelihoods that patients are explained by the topics. Figure 3c depicts an example of the probabilities that the patients’ conditions are explained by topics in the form of HCCs. We use the same strategy invoked for workflow topics to set the number of topics for HCCs, which we denote as q.

Measuring Associations

Each workflow and phenotypic topic is leveraged to explain the patients (Figure 2 and Figure 3). We use the patients they explain in common to measure their association. Specifically, the degree of association between a workflow topic w_i and an HCC topic h_j is measured as the cosine of their respective vectors:

$$\mathrm{Cosine}(w_i,h_j)=\frac{{\phi }_W(i)·{\phi }_H(j)}{|{\phi }_W(i)|\left|{\phi }_H(j)\right|},$$ (1)

where φ_W(i)is a vector indicating the distribution of probabilities that a workflow topic w_i explains each patient. For instance, as shown in Figure 2e, the first workflow explains four patients with the following vector of probabilities (〈p₁, 0.8〉, 〈p₂, 1〉, 〈p₃, 0.8〉, 〈p₄, 0.8〉). Similarly, φ_H(j) is a vector specifying the distribution of probabilities that a phenotypic topic h_j explains each patient. For instance, as shown in Figure 3c, the first HCC topic explains four patients with a vector of probabilities (〈p₁, 0.8〉, 〈p₂, 1.0〉, 〈p₃, 0〉, 〈p₄, 0〉). According to Equation (1), the cosine similarity between the first workflow and HCC topic Cosine(w₁, h₁) is 0.7494.

Our goal is to infer HCC clusters that share workflows because we anticipate that each cluster is a candidate for bundled care management. Thus, we use a community detection algorithm [29] to infer HCC clusters via the association network of HCCs and workflows. We guide the algorithm using a heuristic that is based on the optimization of the modularity measure [30], which is efficient (in running time) and effective (in quality of communities) for weighted and undirected graphs. Modularity is defined as:

$$\mathrm{Q}=\frac{1}{2m}{\displaystyle {\sum }_{vw}{\displaystyle {\sum }_r[A_{vw}-\frac{k_v{k}_w}{2m}]S_{vr}{S}_{wr},}}$$ (2)

where m is the number edges in the network, k_v, k_w is the degree of vertex v and w respectively, A_vw = 1 means there is an edge between the two vertices and S_vr is defined as 1 if vertex v belongs to group r and zero otherwise. Clusters with high modularity have dense connectivity within HCCs, as well as workflows within clusters, but sparse connectivity between clusters.

Plausibility Evaluation for Bundled Care

We investigated if the HCC clusters are potential candidates for bundled care management. To do so, we designed a survey that consisted of paired 〈inferred, random〉 HCC clusters that we provided to administrative and clinical experts for review. We did not indicate which cluster was inferred or which was randomly generated to the experts. Each inferred HCC topic was represented as the list of diagnoses that exhibited the largest probabilities for a specific topic. A random cluster was generated by selecting a number of HCC topics at random, the number of which was set equal to the number of HCC topics within the inferred cluster. Each random topic was also represented as a list of diagnoses. Each random cluster contained the same number of diagnoses as its inferred counterpart.

We recruited a set of experts to answer questions of the following form, “To what extent do you believe health conditions in the displayed group can be managed in a bundled way?” For each question, we provided five candidate answers (in the form of Not At All Likely, Slightly Likely, Moderately Likely, Very Likely and Completely Likely). To perform hypothesis testing, we converted these answers into values in the range 0 to 1 (e.g., Not = 0, Slightly = 0.25, Moderately = 0.5, Very = 0.75, and Completely = 1). Further details about the survey design, including the specific questions, are provided in online Appendix A.

Given the responses, we conducted a series of formal hypothesis tests, each of which can be summarized as: “For a given pair of 〈inferred, random〉 clusters of health conditions, experts can distinguish the inferred from the random in terms of bundled care”. We applied a linear regression model and analysis of variance (ANOVA) [31] to test the significance of difference at the 95% confidence level.

To achieve a power of 0.8 with a standard deviation of 0.4 in the difference in experts’ scores for inferred and random clusters, the required sample size was five respondents. As such, we invited five knowledgeable professionals with a diverse array of expertise, including HCO management, internal medicine, and emergency care. Each participant was emailed an introduction to the goals of the research and a link to access a REDCap survey [32]. The response rate was 100% because all respondents agreed to participate in the survey beforehand.

Experimental Design

Dataset

This study focused on four months of inpatient EMR data from Northwestern Memorial Hospital (NMH) during 2013. In this data, an event corresponds to an instance of an EMR access and is represented as a 〈role, patient〉 pair, which indicates an HCO employee affiliated with the role had an access event on the patient’s records. We rely on the employee’s role, as opposed to the employee themselves, to represent the access events because the role has been shown to be more stable to represent clinical workflow [64]. There were 1,138,317 access events distributed over 16,569 patient encounters. Additionally, each patient was associated with a set of ICD-9 codes assigned after discharge from the hospital. The EMRs contained 144 unique roles and 4,543 unique ICD-9 codes.

In recognition of the fact that multiple ICD-9 codes may be applied to describe the same clinical phenomena [33–34], various phenotyping investigations (e.g., [35–36]) have adopted alternative vocabularies for the secondary analysis of EMRs, such as the Phenome-Wide Association Study (PheWAS) vocabulary [15]. PheWAS codes correspond to groups of ICD-9 codes more closely match the clinical and biological basis of diseases and reduce variability in identifying diseases. Based on this expectation, we translated a patient’s ICD-9 codes to PheWAS codes, which compressed the space into 1,374 unique PheWAS codes.

Number of Topics

The number of workflow and HCC topics were determined by minimizing the similarity over the range of 15 to 35 possible topics. This occurred when k = q = 25. At this point, the workflows and HCCs exhibited a minimum similarity of 0.003 and 0.031, respectively.

Results

To provide context for the findings, we begin with a depiction of the learned workflow and HCC topics. Next, we report on the clusters of HCCs and the extent to which they were deemed plausible for bundled care and had face validity according to evidence in the published literature.

Learned Workflow and HCC Topics

Recall that each workflow and HCC topic is expressed as a probability distribution over terms (i.e., subsequences of actions and PheWAS codes, respectively). To illustrate each topic succinctly, we depict the 10 terms with the largest probabilities. This cutoff was selected because the terms beyond this point had a negligible contribution to the probability mass for the affiliated topic. Specifically, none of these terms contributed a probability that was larger than 0.01.

We use ProM [37], a software tool for process mining, to visualize workflow topics as a directed graph. The graphs for all 25 workflow topics and their corresponding top 10 subsequences are provided in Appendix B. To orient the reader to workflow topics, we list workflow topic 15, which consists of two loops (a pair of + symbols represents the beginning and ending of a loop), as an example in Figure 4.

Figure 4.

The directed graph of an echocardiography-based prenatal workflow. This visualization is based on the 10 subsequences with largest probabilities for the workflow topic. Note that, in this diagram, a pair of + symbols represents the beginning and ending of a loop.

The first loop resides between a Radiology Technologist (RAD) and an NMH Physician Hospitalist invoking Computerized Physician Order Entry (CPOE). This loop was associated with the process of an echocardiography, where a physician approves the quality of a radiological report or participates in the peer review process of a report. The second loop resides between an NMH Physician CPOE and a Patient Care Staff Nurse - Lactation. This loop is likely associated with a primary physician and staff nurse responsible for an inpatient’s care associated with obstetrics.

Each HCC topic is expressed as a probability distribution over the PheWAS codes. The top 10 PheWAS codes, along with their associated probabilities, for each HCC topic is provided in Appendix C. We summarized each HCC topic and provided a label to refer to them (as shown in Appendix C). To provide intuition, we report on an example of the topics associated with childbirth in Table 1. This topic shows that interventions are required for complicated pregnancies and delivery associated problems (e.g., short gestation, endocrine and metabolic disturbances of fetus or newborn).

Table 1.

The ten PheWAS codes in a HCC topic that are the most indicative of childbirth.

PheWAS Code	Description	Probability
1010	Tests associated with child birth	0.25
637	Short gestation; low birth weight; and fetal growth retardation	0.18
656	Other perinatal conditions	0.16
656.1	Perinatal jaundice; isoimmunization	0.10
651	Multiple gestation	0.05
656.3	Endocrine and metabolic disturbances of fetus and newborn	0.05
747.11	Cardiac shunt; heart septal defect	0.05
656.2	Other respiratory conditions of fetus and newborn	0.02
647	Infectious & parasitic conditions complicating pregnancy	0.02
747.13	Congenital anomalies of great vessels	0.01

Clusters of HCC and Workflow Topics

The modularity of the HCC and workflow topic cluster was 0.62. This indicates that the HCC topics and workflow topics within each cluster exhibited strong associations, while they exhibited weak associations between clusters. Figure 5 depicts the four inferred HCC topic cluster (shown in blue, green, purple and red) and their affiliated workflow topics.

Figure 5.

Four clusters of HCC topics inferred via their shared workflow topics. The edges represent the association strength between HCC and workflow topics. The wider the edge, the stronger the association. (Legend: h = HCC topic; w = workflow topic)

Cluster C₁ (in green) is associated with fetal abnormality; C₂ (in red) is associated with late pregnancy; C₃ is associated with prostate problems and its corresponding complications (in purple); while C₄ is complex, but is associated with various chronic problems, including cerebrovascular disease, coronary atherosclerosis, congestive heart failure (CHF), diabetes, and kidney failure (in blue).

To gain a deeper understanding of the inferred clusters and their associated workflow patterns, let us consider C₁ as an example. The health conditions affiliated with C₁ are the following HCC topics:

• h₁₂: Birth trauma,

• h₁₇: Fetal abnormality, and

• h₂₄: Mother complicating pregnancy,

which were associated with care patterns that incorporated the following workflow topics:

• w₃: Interactions between physicians and staff nurses,

• w₁₁: Interactions between physicians, anesthesiologists, advanced practice clinicians and pharmacists,

• w₁₃: Interactions between physicians and unit secretaries,

• w₁₄: Interactions between physicians, anesthesiologists and staff nurses, and

• w₂₂: Interactions between physicians, radiologists and unit secretaries.

This suggests that pregnancy complications (e.g., fetal abnormality and mother complicating pregnancy) are already managed together, requiring communication between various clinicians, obstetricians, anesthesiologists, radiologists, nurses, pharmacists, and administrative personnel.

Plausibility of Phenotypic Clusters for Bundled Care

The results of the plausibility survey are provided in Table 2. It can be seen that the experts always scored the inferred clusters as the more plausible for bundled care. All four clusters were statistically significantly higher than the randomized cluster in terms of the respondents’ scores (based on a 95% confidence interval). This suggests that the HCC clusters associated with fetal abnormality, late pregnancy, prostate problems and CHF are plausible candidates for bundled care.

Table 2.

Survey results of the knowledgeable experts (n = 5) regarding the plausibility of HCC clusters for bundled care. Each cluster is represented as a list of PheWAS codes and a brief summary. Each row reports the distance between the Likert score of the inferred HCC cluster and its random counterpart. Note that a positive distance indicates the inferred cluster received a higher Likert score.

Cluster	PheWAS Code and Description		Likert Score Difference	P-value
Informal Description: Fetal abnormality leads to complicated pregnancy and additional delivery problems (e.g., fetal distress), which requires interventions, such as those provided by the birth trauma service.
C1	649	Other conditions of the mother complicating pregnancy	0.95	6.09 × 10−8*
	652	Malposition and malpresentation of fetus or obstruction
	654	Abnormality pelvic soft tissues & organs complicating pregnancy
	658	Problems associated with amniotic cavity and membranes
	659	Indications for care or intervention related to labor and delivery NEC
	663	Umbilical cord complications during labor and delivery
	665	Obstetrical/birth trauma
Informal Description: Late pregnancy suggests a larger size infant requiring intervention (e.g. use of suction or forceps), which can cause temporary skull injuries.
C2	637	Short gestation; low birth weight; and fetal growth retardation	0.95	6.09 × 10−8*
	645	Late pregnancy and failed induction
	649	Other conditions of the mother complicating pregnancy
	656	Other perinatal conditions
	656.1	Perinatal jaundice/isoimmunization
	665	Obstetrical/birth trauma
	819	Skull fracture and other intracranial injury
	1010	Other tests
	1008	Internal injury to organs
Informal Description: Anemia and hypogonadism are often complications of prostate cancer and can lead to bone loss. When the thyroid does not produce a sufficient amount of hormones, it can cause lower esophageal sphincter dysfunction. This allows stomach contents and digestive juices to enter the esophagus, which may lead to gastroesophageal reflux disease.
C3	244	Hypothyroidism	0.65	2.80 × 10−4*
	272.1	Hyperlipidemia
	276.14	Hypopotassemia
	285.9	Anemia
	327.32	Obstructive sleep apnea
	401.1	Essential hypertension
	495	Asthma
	530.11	Gastroesophageal reflux disease
	600	Hyperplasia of prostate
	740.1	Osteoarthritis; localized
Informal Description: Cerebrovascular disease and coronary atherosclerosis are the most common cause of congestive heart failure (CHF); smoking and diabetes are associated with all three diseases. Depression is associated with coronary disease. Liver test abnormalities and renal failure may occur with CHF.
C4	250.2	Type 2 diabetes	0.70	7.04 × 10−5*
	272.1	Hyperlipidemia
	286.5	Hemorrhagic disorder due to intrinsic circulating anticoagulants
	296.2	Depression
	316	Substance addiction and disorders
	318	Tobacco use disorder
	401.1	Essential hypertension
	401.22	Hypertensive chronic kidney disease
	427.21	Atrial fibrillation
	428	Heart failure
	428.1	Systolic/diastolic heart failure
	433.31	Transient cerebral ischemia
	452	Venous embolism & thrombosis
	585.3	Chronic renal failure
	591	Urinary tract infection
	707.1	Decubitus ulcer

^*= statistical significance at the 0.05 confidence level)

Additionally, to orient the reader to each HCC cluster, we provide each, along with an informal summary, in Table 2.

Evidence in the Published Literature for HCC Clusters

While the HCC clusters were deemed plausible for bundled care from a management perspective, we further investigated if the health conditions within such clusters were clinically related. If there was support from both care process and clinical perspectives, we anticipate that the identified HCC clusters would be better received by HCO administrators.

Towards this goal, we reviewed evidence for the inferred HCC cluster in the peer-reviewed literature. Evidence was uncovered for each cluster, a summary of which is shown in Table 3. For instance, within cluster C₃, bone loss is known to be caused by hypogonadism following prostate cancer [38]. Furthermore, acid reflux is known to be affiliated with thyroid problems [39].

Table 3.

Evidence in the peer-reviewed literature to support the face validity of the HCCs within each inferred cluster.

Cluster	Evidence of Associations in the Literature
C1	Birth trauma associated with fetal big size and fetal distress [40] Trauma in pregnancy [41–42]
C2	Late pregnancy and child birth [43] Mode of delivery in nulliparous women has an effect on neonatal intracranial injuries [44] Most fetal injuries occur in late pregnancy [45]
C3	Bone loss following hypogonadism with prostate cancer [38] The acid reflux-thyroid connection [39] Anemia associated with advanced prostate cancer [46]
C4	Tobacco and alcohol usage had increased risk of mortality for cerebrovascular disease and liver disease [47] Thrombotic complications in heart failure [48–49] Associations among diabetes, kidney disease, and cardiovascular disease [50]

Discussion

Main Findings

This study proposed a data-driven framework to automatically learn clinical workflow patterns, phenotypic patterns and the relationships between them. We believe this approach, and the associated empirical analysis, provide evidence to assist in identifying, as well as refining, bundled care models. The study has several notable implications.

First, the associations we uncovered between patients with a bundle of healthcare conditions and clinical workflows can provide knowledge that HCOs can build upon to establish care teams that are in alignment with a patient’s needs. In particular, we learned workflow patterns at the level of HCO employees and their affiliated roles. We believe this provides evidence to depict who works with whom in a care team and what are the interactive relationships between the team members. Moreover, the relationship between workflow and phenotypic patterns suggests that a set of related health conditions could be managed by a set of HCO employees in the form of a workflow. This is important because it suggests such workflows could be leveraged to put the right HCO employees in place for the right patients. For instance, consider patients who are nearing childbirth. These patients’ conditions may be affiliated with three specific HCC topics: h₁₂: Birth trauma, h₁₇: Fetal abnormality, and h₂₄: Mother complicating pregnancy. And, these HCC topics are associated with care patterns that incorporate three specific workflow topics: w₃: associated with physicians and care staff nurses, w₁₄: associated with anesthesiologists, and w₂₂: associated with radiologists. Now, if a patient is associated with the three HCC topics, it may be better to assign the patient to a care team that includes all HCO employees from all three workflows. In doing so, the HCO could manage the patient’s conditions in a bundled manner, as opposed to attempting to manage each health condition independently.

Second, the associations between workflow and HCC topics should provide support for HCOs to manage patients and conduct resource allocation more efficiently. For instance, if the volume of patients associated with complicated pregnancies (e.g., birth trauma, fetal abnormality) grows, then HCOs could dedicate a larger amount of resources to workflow topics w₃, w₁₄ and w₂₂.

Third, we believe that the evidence derived through our framework can be leveraged to design testable hypotheses regarding workflow and patient outcomes. Specifically, the output of our framework could be relied upon to investigate the differences between learned care teams and existing patient management protocols. This could be accomplished in terms of their impact on patient outcomes, such as readmission rates or length of stay in the hospital.

Limitations and Next Steps

Despite the merits of our findings, there are several limitations that we wish to highlight for future investigations. First, this study focused on the development of a methodology to infer general collections of health conditions that share similar workflow patterns according to EMR system utilization. However, we did not validate the clinical meaning (e.g., semantic context) for each of the inferred HCCs nor their workflows. If such HCC and workflow are to be relied upon in care management applications, their semantics will require further interpretation by administrative experts.

Second, while all four HCC clusters were deemed plausible for bundled care, several associations within congestive heart failure cluster C₄ were not clear to the experts. Specifically, there are a number of reasons why renal failure and liver disease might co-occur in a patient, such that this cluster may be too general in nature. In this respect, our study indicates health conditions have the potential to be managed in a bundled manner, but what precisely should be managed is an open question and will require guidance by process management experts.

Third, we acknowledge that this is a pilot only, which focuses on a case study of four months of data from a single HCO. As such, we uncovered only four HCC clusters. It is unknown if the proposed strategy directly generalizes to other healthcare systems.

Conclusions

In this paper, we introduced a data-driven framework to mine EMRs for HCC clusters that might benefit from the establishment, or formalization, of bundled care routines. We evaluated our approach with four months of inpatient data from a large hospital system and uncovered four clusters of HCCs, which were deemed plausible for bundled care by knowledgeable experts and evidence in the literature. We anticipate working with process management and clinical experts to assess the workflow patterns affiliated with each inferred cluster to ascertain how they can support bundled care. Furthermore, we plan to test the performance and efficacy of such the framework with data from additional healthcare systems.

Highlights.

1. A data-driven framework to infer an association network of health condition collections (HCCs) and workflows.

2. A strategy to learn HCC clusters based on the associations network.

3. An online survey method to evaluate the plausibility of HCC clusters managed in a seemingly bundled way.

Appendices

Appendix A

Survey questions.

Appendix B

Workflow topics, each of which is represented by its top 10 subsequences and visualized as a process graph via Business Process Model and Notation (BPMN) in ProM.

Appendix C

HCC topics, each of which is represented by its top 10 PheWAS codes.