Home Health Care Clinical Notes Predict Patient Hospitalization and Emergency Department Visits

Abstract

Background:

About 30% of home health care patients are hospitalized or visit an emergency department (ED) during a home health care (HHC) episode. Novel data science methods are increasingly used to improve identification of patients at risk for negative outcomes.

Objectives:

To identify patients at heightened risk hospitalization or ED visits using HHC narrative data (clinical notes).

Methods:

This study used a large database of HHC visit notes (n = 727,676) documented for 112,237 HHC episodes (89,459 unique patients) by clinicians of the largest nonprofit home health care agency in the United States. Text mining and machine learning algorithms (Naïve Bayes, decision tree, random forest) were implemented to predict patient hospitalization or ED visits using the content of clinical notes. Risk factors associated with hospitalization or ED visits were identified using a feature selection technique (gain ratio attribute evaluation).

Results:

Best performing text mining method (random forest) achieved good predictive performance. Seven risk factors categories were identified, with clinical factors, coordination/communication, and service use being the most frequent categories.

Discussion:

This study was the first to explore the potential contribution of HHC clinical notes to identifying patients at risk for hospitalization or an ED visit. Our results suggest that HHC visit notes are highly informative and can contribute significantly to identification of patients at risk. Further studies are needed to explore ways to improve risk prediction by adding more data elements from additional data sources.

Every year, more than 11,000 home health care (HHC) agencies across the United States provide care to more than 5 million older adults (MedPac, 2014). Currently, about one in three HHC patients are hospitalized or visit an emergency department (ED) during the 30–60 day HHC episode (Centers for Medicare and Medicaid Services, 2019). These numbers have not improved over the last several years (Centers for Medicare and Medicaid Services, 2019; MedPac, 2014), despite several national and local efforts to improve quality of care and patient outcomes (Agency for Healthcare Research & Quality, 2016; Busby et al., 2015; Hodgson et al., 2019).

In settings other than HHC, novel, data science methods increasingly are used to improve identification of patients at risk. For example, in hospital settings, recent systematic reviews (Jayasundera et al., 2018; Linnen et al., 2019) found that early warning systems are effective in identifying patients who are likely to die or be transferred to an intensive care unit (ICU). Studies developing early warning systems often use advanced data analytics methods to identify specific patterns in electronic health record (HER) documentation that are a proxy of a clinician’s concern about a patient. Some researchers demonstrate the ability to predict patient deterioration up to 48 hours before adverse outcomes happen using these methods (Collins & Vawdrey, 2012; Dykes et al., 2017; Linnen et al., 2019).

Researchers have used previously unexplored data sources to improve further identification of patients at risk. Several recent studies showed that information from narrative clinical notes, extracted via natural language processing (NLP), is a key data source about patient risk (Korach et al., 2019; Woo et al., 2019). For instance, some researchers have found that a patient’s social risk factors (Navathe et al., 2018) or poor self-management status (Topaz et al., 2016) extracted automatically from inpatient clinical notes are significantly associated with rehospitalizations. Other researchers have shown that information extracted from inpatient clinical notes, including nursing notes, can be used to identify patients at risk for mortality (Lehman et al., 2012; Marafino et al., 2015; Waudby-Smith et al., 2018). These previous studies suggest that NLP can be used to develop early warning systems in the hospital setting by identifying key risk factors expressed in the narrative notes and using that information to trigger real-time interventions to mitigate risk and prevent poor outcomes.

Compared to the hospital setting, developing early warning systems in HHC is challenging due to several major differences between hospitals and HHC settings. First, care encounters in HHC (e.g., nursing visits) typically happen a few days apart, compared to minute or hourly encounters in hospitals. Second, most of the data in HHC is generated by nurses and physical or occupational therapists compared to a larger array of health professionals in the inpatient setting. Our study is the first to explore the potential for HHC narrative data to help identify patients at heightened risk for poor outcomes. Specifically, we used a large patient sample from the largest nonprofit HHC agency in the U.S. to examine whether information documented in HHC visit notes can help predict patient hospitalization or ED visits. Answering the study question is critical in order to determine the feasibility of implementing early warning systems in HHC using innovative data science methods, such as those developed for the hospital setting, to improve care for the highest-risk patients. Thus, we explored the potential for HHC narrative nursing notes to help identify patients at high risk for hospitalization or ED visits.

Methods

This was secondary data analysis using a retrospective cohort data of narrative HHC visit notes and patient hospitalization or ED visits outcome from the Outcomes Assessment and Information Set (OASIS). HHC notes and the OASIS were not collected specifically for this study. The study was approved by the institutional review board at Columbia University and the HHC agency.

Data Collection

We used HHC visit notes (n = 1,149,586) documented for 112,237 HHC episodes (89,459 unique patients) by clinicians of the largest nonprofit HHC agency in New York, NY between January 1, 2014 and December 31, 2014. The patient sample had a mean age of 70.8, was 60.8% female, and 37% lived alone. The sample was racially and ethnically diverse, with 43% White, 27% Black or African American, 24% Hispanic/Latino, and 6% Asian.

Visit notes were completed by HHC clinicians (e.g., nurses, physical or occupational therapists, social workers, etc.) using the agency’s EHR. We used the narrative part of the HHC visit description, which ranged from lengthy admission notes (often written by a registered nurse) to shorter progress notes (e.g., nurse follow-up notes, physical therapy progress notes, etc.). The average note length was 150 words. Since all visit notes were in the HHC agency’s EHR, the text was in an electronic and computer readable format (txt) rather than Portable Document Format (PDF). Each HHC episode in the sample had at least one visit note, thus there was no missing data.

Study endpoints consisted of patient hospitalization or ED visits and were extracted from OASIS. The OASIS assessments are federally mandated of all HHC agencies certified to accept the Centers for Medicare and Medicaid Services payments. OASIS assessments are conducted at the patient’s HHC admission and discharge. We used OASIS item “M2300: Emergent Care” from the discharge assessment to document study outcomes.

Clinical Notes Selection

Our goal was to generate a predictive model identifying patient’s risk for hospitalization or ED visit early during the HHC episode. Early identification of risk affords enough time for nursing or other care management interventions to potentially reduce hospitalizations and ED visits. To accomplish our goal, we generated a subset of notes documented early during the HHC episode and omitted notes documented relatively close to an event (i.e., hospitalization or ED visit or regular discharge from HHC). Specifically, we reduced the original data set to a subsample of 70% “initial” visit notes documented early in the HHC episode. For example, when a patient had 10 visit notes, we used only the first seven chronological visit notes documented during the episode (time 1→ time 7) and omitted the last three visit notes (time 8 → time10). There were 18.5 visit notes per episode on average (median = 7) and the final study sample included 727,676 visit notes.

Data Analysis

To identify patients at high risk of ED visit and hospitalization, we developed an analytical pipeline that included text mining and machine learning methods (see Figure 1). This pipeline can be applied to a large body of clinical notes to: (a) identify patients with high risk of ED visit and hospitalization; and (b) identify key risk factors associated with ED visit and hospitalization, and subsequently, to develop early warning to trigger real time interventions for high-risk patients. The analytical pipeline consists of five components:

1. text preprocessing;

2. feature selection;

3. training of machine learning algorithms;

4. evaluation of the predictive performance of machine learning algorithms; and

5. identification of the most important features.

Figure 1: Study methods overview.

The figure visualizes data analytics methods applied in the study. The analytical pipeline consists of five components: 1) text pre-processing, 2) feature selection, 3) training of machine learning algorithms, 4) evaluation of the predictive performance of machine learning algorithms, and 5) identification of the most important features.

In general, no special computing resources were needed. All study methods were implemented in KNIME open-source data analytics platform that can be downloaded for free at https://www.knime.com/downloads (KNIME, n.d.). In the following sections, we provide detailed information on each component.

Component 1: Text Preprocessing

Text preprocessing is an integral part of any text mining system. Appropriate text preprocessing is vital for the downstream analytical pipeline components. To improve identification of risk factors expressed in the text, we conducted the following text preprocessing tasks: omitting punctuation, omitting common stop words, omitting terms with high frequency, and converting all words to lowercase. See Appendix A (Supplemental Digital Content) for rational and examples of the text preprocessing steps.

Component 2: Feature Selection

Feature selection is the process of selecting a subset of words and expressions from clinical notes that are significantly associated with the outcome of the study. For example, a term “severe pain” might be highly associated with a patient’s risk for hospitalization or emergency department visit. Further, feature selection might improve the accuracy of text classification by excluding “unimportant” words from the clinical notes. We used n-gram as a feature. N-grams help identify common expressions that contain one word or more (e.g., “pain” is a 1-gram, “severe pain” is a 2-gram, and “very severe pain” is a 3-gram). Using n-grams can help identify expressions with relatively high predictive value for projecting hospitalizations and ED visits (Schonlau & Guenther, 2016).

Component 3: Training of Machine Learning Algorithms

Unlike other statistical models that are designed to infer causal relationships between study variables and outcomes (e.g., logistic or linear regression), many machine learning algorithms are designed to identify correlational patterns in the data that help make the most accurate prediction of the study outcome. We used three machine learning algorithms—Naïve Bayes, Decision tree, and Random Forest—to develop prediction models from clinical notes. Although other text mining methods exist (e.g., artificial neural networks), we selected these specific methods because they are: commonly used in biomedical informatics literature (Sun et al., 2018); can be easily replicated; and provide a good baseline assessment of text mining feasibility.

Naïve Bayes is a probabilistic classifier developed based on the Bayes Theorem (Witten et al., 2011) to evaluate the association between identified features with the outcome class (hospitalization and ED visit) assigned to the document (clinical notes; Witten et al., 2011). Decision Tree algorithm uses decision tree models to go from observations about features such as “severe pain” to assigned class to the document (Witten et al., 2011). Random Forest generates numerous independent trees operating as an ensemble to predict accurate class for a document (Witten et al., 2011). See Appendix B (Supplemental Digital Content) for more information about the machine learning algorithms.

These three algorithms use numeric data to develop prediction models. Therefore, as a first step, we converted all clinical notes into numerical values; a process referred to as vectorization. For this purpose, we used a “one-hot” encoding method, which computes the frequency of each word and expression in a set of clinical notes. For instance, “pain” could be represented as [1, 0, 4, …], indicating that it was repeated 1 time in the first clinical note, 0 times in the second clinic note, 4 times in the third clinical note, and so on.

To create the training data set for training the machine learning algorithms, we first stratified the sample by the study outcome. This created two subsamples of all HHC episodes—episodes with hospitalization or ED visit versus episodes without hospitalization or ED visit. Next, we randomly selected 70% of the data from each of the two subsamples (n = 78,566 episodes) to create the training data set. We saved the remaining 30% of data as a test data set (see Component 4). We used the KNIME platform to train the machine learning algorithms using the training data set. All machine learning algorithms were applied with default settings, including: (a) Algorithm Naïve Bayes was applied without debugging or Kernel estimators; (b) Algorithm Decision Tree (specific type: J48, recursive partitioning) was applied with gain ratio (tree quality measure), reduced error pruning, and minimum number records per node = 2; and (c) Algorithm Random Forest was applied with number of iterations = 100, minimum number of instances = 1, minimum variance for split = 1e−3, depth = unlimited. See Appendix B (Supplemental Digital Content) for more information about machine learning algorithms applied in this study.

Component 4: Evaluation of Predictive Performance Machine Learning Algorithms

Once machine learning models were developed on the training data set, we applied these models on the test data set (30% all HHC episodes, n = 33,671). Specifically, we generated a machine learning predicted class for each HHC episode (predicted hospitalization or ED visit vs. none) and compared it with the observed HHC class (actual hospitalization or ED visit vs. none). For each machine learning method, we computed standard weighted metrics (Beger, 2016) of: precision—the number of true positives out of the total number of predicted positives; recall—the number of true positives out of actual number of positives; F-score, the weighted harmonic mean of the precision, and recall; and Area Under the Precision-Recall Curve (AUC) —a single scalar value that measures the overall performance of a binary classifier. For all metrics, scores range between 0 and 1, with higher scores indicating better predictive performance. In general, an F-score > 0.8 would indicate that a machine learning algorithm achieves good predictive performance.

Component 5: Identification of the Most Important Features

To identify words and expressions strongly associated with risk for hospitalizations and ED visits, we used gain ratio attribute evaluation technique (Witten et al., 2011), which measure the predictive power of each word and expression for the outcome class (hospitalization and ED visit). We set the gain ratio threshold to > 0 to ensure that only the most informative features were retained by the method. Two PhD prepared nurses with experience in the HHC setting independently reviewed the list of final selected words and expressions and categorized them into clinically meaningful categories (e.g., clinical factors, social/environmental factors). We computed the Kappa interrater agreement (McHugh, 2012) for the category assignments, which was high (Kappa interrater agreement = .94), indicating high initial agreement between the reviewers. All disagreements were resolved via a consultation with an additional PhD-prepared nurse with expertise in clinical informatics.

Results

Overall, 19,301 HHC episodes (17.2% of all episodes) resulted in hospitalization or ED visits. Table 1 shows the performance of machine learning methods, with a best method (random forest) achieving an F-measure of .83 and AUC of .76, indicating good predictive performance.

Table 1.

Performance of machine learning methods

Machine learning method	Recall	Precision	F-score	Area under precision-recall curve (AUC)
Naïve Bayes	0.59	0.71	0.62	0.61
Decision Tree (J48)	0.73	0.74	0.73	0.68
Random Forest	0.81	0.83	0.82	0.76

Using gain ratio attribute evaluation, 388 words and expressions were identified as most associated with patient hospitalization or ED visits. Two HHC nurse reviewers used the six categories seen in Table 2 to classify each word or expression. The interrater agreement was very high (Kappa interrater agreement = 96%) and final consensus of 100% was achieved on all category assignments via reviewer discussion. Table 2 summarizes the distribution of categories associated with patient hospitalization or ED visits. The most prevalent categories were clinical factors (28%), and coordination/communication (23%), while almost one third of the words and expressions were marked as “other” (28%). Table 3 shows the top 10 words and expressions (sorted by gain ratio value) identified in each category.

Table 2.

Category definitions for words and expressions associated with hospitalization or emergency department visits among HHC patients

Category name	Category description
Clinical factors	Any words or expressions suggesting clinical problems, including symptoms, diseases, functional status limitations, clinical interventions, etc.
Coordination/communication	Any words or expressions mentioning coordination/communication activities related to patient care.
Service use	Any words or expressions mentioning care settings or professional services.
Social/environmental factors	Any words or expressions mentioning social or environmental factors like patient’s living situation, family support, etc.
Temporal	Any words or expressions mentioning time.
Device/equipment	Any words or expressions mentioning medical device/equipment
Other	Other ambiguous words or expressions.

Table 3.

Examples of top 10 words and expressions identified in each category

Clinical factors	Coordination/ communication	Service use	Social/ environmental factors	Temporal	Device/ equipment	Other
dehydrated	d/c	to er	shopping	in am	catheter	goals met
vomited	with ns	no skilled	lives alone	started on	diaper	negotiate
nausea	md called	to hospital	in apt	today vn	with cane	all goals
wound care	c[are] planning	skilled nursing	income	last night	Foley [catheter]	final
incision	plan discussed	outpatient	pt lives with	this am	rollator	no further
kidney	pt in agreement	no further skilled	private insurance	today for	in chair	experienced
cleansed	c[are] plan	hospitalized for	children	when he	with rollator	large
worsening	supplies ordered	surgery	spouse	at pm	rolling walker	render
unsteady gait	vn called	physical therapy	child	next week	device	yo female
hx falls	vn unable to	pharmacy	house	tomorrow	wheel chair	pt was

Note. Hx = history; d/c = discharge; ns = nursing service; pt = patient; vn = visiting nurse; er = emergency room; yo = years old

Discussion

This study is the first to explore the potential contribution of narrative clinical notes in HHC to identify patients at risk for hospitalization or ED visits. Using only HHC visit notes documented relatively early during the HHC episode, we were able to build machine learning models with good predictive performance. These results have several critical implications. First, ability to predict negative outcomes from the HHC EHR data highlight the value of HHC clinical documentation. Although several previous studies in HHC attempted to create models predicting a patient’s risk for hospitalization (Fortinsky et al., 2014; Lohman et al., 2018; Ma et al., 2018; Rosati et al., 2003; Rosati & Huang, 2007), their methodological approaches had several marked limitations. In a recent systematic review, Ma et al. (2018) concluded that major limitations of these studies were due to a focus on patients with a specific disease (often heart failure), reliance on a single administrative data source, and small sample size. The predictive performance of the existing models also remains low and often not reported (Fortinsky et al., 2014; Lohman et al., 2018; Ma et al., 2018). Our findings suggest that using routinely collected narrative data from the HHC EHR is a critical step in improving early identification of patients at risk for poor outcomes. Research from hospital settings supports these findings (Jayasundera et al., 2018; Linnen et al., 2019).

Our results support the feasibility of creating early warning systems in the HHC setting. We have purposefully used a subsample of 70% of notes documented first (chronologically) during the HHC episode and showed good predictive performance. In practice, this means that warning signs of patient deterioration can be identified from clinical documentation before an adverse outcome occurs. Our findings can be used to develop an early warning system that will notify HHC clinicians of an increasing patient risk, which in turn should trigger risk reduction interventions.

Our results suggest that it is feasible to implement such text predictive pipelines in the HHC clinical settings. Ideally, such risk prediction models would be developed jointly by HHC clinicians and informatics experts and implemented by either the HHC software vendors or other companies that specialize in predictive health analytics.

We also explored a subset of factors highly associated with patient risk predictions and found that clinical factors, such as dehydration or vomiting, were the most prevalent category. Other related factors included care coordination and communication activities (e.g., a conversation with a physician), health service use (e.g. outpatient clinic or physical therapy), or social factors (e.g. living alone or mentions of income. Importantly, many of these factors are found mostly in narrative notes rather than structured data elements within many EHR systems. Other examples of factors found only in notes include patient symptoms (Koleck et al., 2019) or social risk factors (Navathe et al., 2018). For instance, several researchers found that social isolation is documented in the clinical notes (Conway et al., 2019; Zhu et al., 2019) and is associated with rehospitalizations (Navathe et al., 2018).

Our findings also support and extend previous categorizations of risk factors in HHC. For example, in a systematic review focused on identifying factors affecting HHC patient risk for rehospitalization, the authors found that of all socioeconomic factors, only insurance type was predictive of patient rehospitalization (Ma et al., 2018). In another instance, functional limitations were identified as risk factors in HHC previously, and our results further enrich this category adding specific devices like a cane or wheelchair.

Our results also suggest that temporal text features (e.g., “this a.m.”) are associated with negative outcomes. One possible explanation is that these expressions are often used in conjunction with other clinical risk factors (e.g., “fall this a.m.”; Topaz et al., 2019) and thus highlight other risk factors. However, the importance of temporal expressions needs further examination. Finally, about one third of the predictors did not fit into one particular category, requiring further review and potential classification.

Text mining methods, similar to those we used, were applied in settings other than HHC to generate early warnings of a patient’s risk. For example, Mehrabi et al. (2015) developed a text mining system to identify patients with risk factors of pancreatic cancer in clinical notes. The system provided early warnings that helped clinicians identify patients at risk of pancreatic cancer for further screening and early interventions. In other work, text mining systems were used to identify adverse drug reactions by Zolnoori et al. (2019). Text mining systems were used to identify patients with likely peripheral artery disease based on findings described in radiology reports by Savova et al. (2010). In HHC, our text mining methods can be used to extract risk factors from text and present them to HHC nurses to improve timely risk identification and inform interventions.

Our results suggest that it is feasible to build early warning systems in HHC using routinely collected data from electronic health records. Further studies might explore improvement of risk prediction models by adding more data elements, such as specific symptoms, social risk factors, patterns of previous HHC visits, patient characteristics from standard assessment tools, etc. Researchers might also apply advanced statistical methods that will allow for better adjustment for time-dependence and other potential interdependence between the data element used as predictors, for example by applying time-series analysis and/or mixed effects modeling. Finally, more research is needed in order to understand whether and which early warning systems are effective in the HHC setting.

Limitations

This study has several limitations. First, our results are based on one data set of visit notes and generalizability of our findings to other HHC agencies is limited. Further studies should explore the methods’ generalizability by using data from several HHC agencies. Second, some of the outcome data might have been missing due to limited ability of HHC clinicians to follow-up with the discharged patients. In the future, service utilization data from the Centers of Medicare and Medicaid Services might be used to improve outcome completeness. In addition, more granular outcome data might be used to develop different risk models: One for hospitalizations and one for ED visits. Third, our methods only allow for partial understanding of important risk factors and we might have missed other risk categories documented less frequently in visit notes. Additional machine learning methods, such as latent Dirichlet allocation for topic modelling (Blei et al., 2003), can help identify additional categories of risk associated with negative outcomes. In addition, machine learning algorithms might be better adjusted to the data by fine-tuning the model parameters rather than using the default settings (e.g., random forest algorithm’s number of iterations might be increased). Finally, this study only used data from clinical narratives; however, integrating other data sources (e.g., structured data about patient medications and diseases) might have resulted in better risk prediction performance.

Conclusion

This study was the first to explore the potential contribution of HHC narrative notes to identifying patients at risk for hospitalization or ED visits. Our results suggest that HHC visit notes are highly informative and can contribute significantly to identification of patients at risk. In addition, predictive risk factors extracted from the text are clinically meaningful and should be used by other studies exploring HHC patient risk factors.