Exploring the Value of Clinical Data Standards to Predict Hospitalization of Home Care Patients

Abstract

Background

There is a critical need to reduce hospitalizations for Medicare patients and electronic health record (EHR) home care data provide new opportunities to evaluate risk of hospitalization for patients.

Objectives

The objectives of this study were to 1) develop a measure to predict risk of hospitalization among home care patients, the Hospitalization Risk Score (HRS), and 2) compare it with an existing severity of illness measure, the Charlson Index of Comorbidity (CIC).

Methods

A convenience sample of clinical data from 14 home care agencies’ EHRs, representing 1,643 home care patient episodes was used for the study. The development of the HRS was based on review of the literature, and expert panel evaluation to construct the HRS. Descriptive statistics and generalized linear models were used for comparative analysis; areas under curve (AUC) values were compared for receiver operating curves (ROC), and cut points predicting hospitalization were evaluated.

Results

The HRS for this sample ranged from 0 to 5.6, with a median of 1.25. The CIC for this sample ranged from 0 to 9 and with a median of 0. Nearly three fourths of the sample was hospitalized at an HRS of 2, and a CIC of 1. AUC values for ROC were 0.63 for HRS and 0.59 for the CIC. The ROC curves were significantly different (t = -7.59, p <0.003).

Conclusions

This preliminary study demonstrates the potential value of the HRS using Omaha System EHR data. There was a statistically significant difference for predicting hospitalization of home care patients with the HRS versus the CIC; however the AUC values for both were low. Continued research is needed to further refine the HRS, determine whether it is more sensitive for particular subgroups of patients, and combine it with additional risk factors in understanding rehospitalization.

1. Background

Home care is the provision of intermittent services in the patient’s home by nurses and other health providers [1]. Hospitalization of home care patients is high. Despite major efforts to address the problem, 29% of home care patients are hospitalized annually [2]. Medicare is the single largest payer for home care and hospital services [1]. Reducing hospitalization of home care patients by only three percent would result in $2.7 billion in Medicare savings annually, while preventing suffering and improving quality of life for patients [3]. Data from electronic health records (EHRs) may be used to identify home care patients most at risk for hospitalization; in order to proactively intervene to prevent hospitalization. The long term goals of our research program are to predict and prevent hospital admissions of home care patients, and to improve patient and caregiver quality of life. In this article, we describe the first step in development of a comprehensive measure to predict risk of hospitalization among home care patients: the evaluation of a Hospitalization Risk Score (HRS) computed from home care EHR data.

1.1 Hospitalization Risk Assessments

In 2007, a national campaign to address home care quality was jointly sponsored by the Centers for Medicare and Medicaid Services (CMS) and the Home Health Quality Improvement Organization Support Center. A major focus of this campaign was reducing avoidable hospitalization through implementing risk assessments [4]. Clinical assessments provide a broad overview of a patient’s status and may be used to identify a subset of factors that indicate risk for adverse events or poor outcomes, inform implementation of specialized interventions, and prevent costly hospital admissions [5].

A review of the literature revealed numerous hospitalization risk assessment studies [6]. Forty four articles were relevant to the purpose of the study, identifying home care patient risks for hospitalization. Of these, 28 articles described one or more patient problems [7–34]. The problems described were Skin, [7–12] Urinary function, [8,11, 12, 23, 30] Circulation, [9–12, 24–32] Respiration, [9–12, 20, 25–27] Mental health, [11, 12, 21–25] Medication regimen, [12–19] Neuro-musculo-skeletal function, [31–34], and Cognition, [34, 38, 56, 62] (►Table 1, Column 1).

Table 1.

Comparison of Hospitalization Risk Score (HRS) and Charlson Index of Comorbidity (CIC) Measures Derived from Home Care Electronic Health Record Data

Omaha System Problem (HRS Weight) Signs/symptom [46]	CIC Category (CIC Weight) ICD-9-CM [42]
Circulation (2)	Myocardial Infarction (1)
edema cramping/pain of extremities decreased pulses discoloration of skin/cyanosis temperature change in affected area varicosities syncopal episodes (fainting)/dizziness abnormal blood pressure reading pulse deficit irregular heart rate excessively rapid heart rate excessively slow heart rate anginal pain abnormal heart sounds/ murmurs abnormal clotting abnormal cardiac laboratory results	410.x Acute myocardial infarction 412.x Old myocardial infarction Congestive heart failure (1) 428.x Heart failure Peripheral vascular diseases (1) 443.9 Peripheral vascular disease, unspecified 441.x Aortic aneurysm and dissection 785.4, Gangrene V43.4 Blood vessel replaced by other means Cerebrovascular disease (1) 430.x–438.x Cerebrovascular Disease
Respiration (2)	Chronic Obstructive Pulmonary Disease (1)
abnormal breath patterns unable to breathe independently cough unable to cough/expectorate independently cyanosis abnormal sputum noisy respirations rhinorrhea/nasal congestion abnormal breath sounds abnormal respiratory laboratory results	490 Bronchitis, not specified as acute or chronic 491 Chronic bronchitis 492 Emphysema 493 Asthma 494 Bronchiectasis 495 Extrinsic allergic alveolitis 496 Chronic airway obstruction, not elsewhere classified 500.x–505.x Pneumoconioses And Other Lung Diseases Due To External Agents 506.4 Chronic respiratory conditions due to fumes and vapors
Neuro-musculo-skeletal function (1)	Hemiplegia (2)
limited range of motion decreased muscle strength decreased coordination decreased muscle tone increased muscle tone decreased sensation increased sensation decreased balance gait/ ambulation disturbance difficulty transferring fractures tremors/seizures difficulty with thermoregulation	344.1 Paraplegia 342.x Hemiplegia and hemiparesis
Urinary function (1)	Moderate-severe renal disease (2)
burning/ painful urination incontinent of urine urgency/ frequency difficulty initiating urination difficulty emptying bladder abnormal amount hematuria/ abnormal color nocturia abnormal urinary laboratory results	582.x Chronic glomerulonephritis 583–583.7 Nephritis and nephropathy not specified as acute or chronic 585.x Chronic kidney disease 586.x Renal failure, unspecified 588.x Disorders resulting from impaired renal function
Cognition (1)	Dementia (1)
diminished judgment disoriented to time/place/person limited recall of recent events limited recall of long past events limited calculating/sequencing skills limited concentration limited reasoning/abstract thinking ability impulsiveness repetitious language/behavior wanders	290.x Dementias
Medication regimen (2)	No comparison within CIC
does not follow recommended dosage/schedule evidence of side effects/adverse reactions inadequate system for taking medication improper storage of medication fails to obtain refills appropriately fails to obtain immunizations inadequate medication regimen unable to take medications without help
Skin (2)
lesion/pressure ulcer rash excessively dry excessively oily inflammation pruritus drainage bruising hypertrophy of nails delayed incisional healing
Mental health (2)
sadness/hopelessness/decreased self-esteem apprehension/undefined fear loss of interest/ involvement in activities/self-care narrowed to scattered attention/focus flat affect irritable/ agitated/ aggressive purposeless/ compulsive activity difficulty managing stress difficulty managing anger somatic complaints/ fatigue delusions
Mental health (2)	No comparison within CIC
hallucinations/illusions expresses suicidal/homicidal thoughts attempts suicide/homicide self-mutilation mood swings flash-backs
No comparison within HRS	Peptic ulcer disease (1)
	531 Gastric ulcer 532 Duodenal ulcer 533 Peptic ulcer site unspecified 534 Gastrojejunal ulcer
	Mild liver disease (1)
	571.2 Alcoholic cirrhosis of liver 571.4 Chronic hepatitis 571.5 Cirrhosis of liver without mention of alcohol 571.6 Biliary cirrhosis
	Diabetes without chronic complications (1)
	250.0 Diabetes mellitus without mention of complication – 250.3 Diabetes with other coma 250.8 Diabetes with other specified manifestations 250.9 Diabetes with unspecified complication
	Connective tissue disease (1)
	710.0 Systemic lupus erythematosus 710.1 Systemic sclerosis 710.4 Polymyositis 714.2 Other rheumatoid arthritis with visceral or systemic involvement 714.81 Rheumatoid lung 725.x Polymyalgia rheumatica
	Diabetes with chronic complications (2)
	250.4 Diabetes with renal manifestations 250.5 Diabetes with ophthalmic manifestations 250.6 Diabetes with neurological manifestations 250.7 Diabetes with peripheral circulatory disorders
No comparison within HRS	Any malignancy, including lymphoma and leukemia, except malignant neoplasm of skin (2)
	140.x–172.x, 174.x.–195.8, 200.x–208.x 140–149 Malignant Neoplasm Of Lip, Oral Cavity, And Pharynx 150–159 Malignant Neoplasm Of Digestive Organs And Peritoneum 160–165 Malignant Neoplasm Of Respiratory And Intrathoracic Organs 170–176 Malignant Neoplasm Of Bone, Connective Tissue, Skin, And Breast 179–189 Malignant Neoplasm Of Genitourinary Organs 190–199 Malignant Neoplasm Of Other And Unspecified Sites 200–209 Malignant Neoplasm Of Lymphatic And Hematopoietic Tissue
	Moderate-severe liver disease (3)
	456.0–456.21, 456 Varicose veins of other sites 572.2–572.8 Hepatic encephalopathy
	Metastatic solid tumor (6)
	196.x–199.1 190–199 Malignant Neoplasm Of Other And Unspecified Sites
	AIDS (6)
	042.x–044.x 042 Human immunodeficiency virus

1.2 Charlson Index of Comorbidity

Home care patients often have more than one medical diagnosis (comorbidities). The most recent statistics from the Centers for Disease Control for home care patients reported that 41.1% had essential hypertension, 31.3% had heart disease, 30.6% had diabetes mellitus, 13.5% had chronic obstructive pulmonary diseases and allied conditions, 10.0% had osteoarthritis (except spine), 8.6% had a malignant neoplasm, 7.1% had dementia, and 7.1% had cerebrovascular disease” [35]. The Charlson Index of Comorbidity (CIC) is an existing severity of illness measure based on medical diagnoses. It has been shown to have value in predicting mortality and hospitalization in general patients in previous studies [8, 36–41]. Three computed versions of the CIC are the Deyo, the Romano (Dartmouth-Manitoba adaptation), and the Elixhauser [8, 38–41]. The Deyo CIC and Romano CIC are similar. Each sums weighted scores for 14 diagnoses, and produces a score ranging from 0–37. The Elixhauser CIC is based on 30 disease diagnoses and diagnosis related groups (DRGs). Needham and colleagues compared Deyo and Romano and found similar results [41]. The Elixhauser out-performs the Deyo and Romano in predicting mortality [8]. However, the Elixhauser CIC is more complex to use and does not provide a summative score for comparison with other risk assessment measures [41]. We previously computed a CIC variable using the Deyo CIC (vs. Romano and Elixhayser) because it was most often cited in the literature, and an algorithm was available in the public domain [42–44] (►Table 1, Column 2).

1.3 Clinical Data from Electronic Health Records

Electronic health records in home care agencies provide large amounts of data useful for investigating potential causes for home care patients requiring hospitalization [5, 8, 9]. Two data sets are the Outcome and Assessment Information Set (OASIS) assessment tool [45] and the Omaha System [46, 47]. Clinical assessments recorded using OASIS and the Omaha System have potential to fill a gap in previous hospitalization risk measures by incorporating the perspective a health professional within the patient’s home context.

1.3.1 OASIS

OASIS is the CMS mandated tool for collecting and reporting home care patient assessments and outcomes. [45] OASIS is a reliable and valid tool developed with the support of multiple private and federal grants. It contains hospitalization data and ICD-9-CM codes. The ICD-9-CM codes describe patient medical conditions in response to several assessment items: primary diagnosis, secondary diagnoses, reasons for recent hospitalization if applicable, reasons for recent treatment change, and payment diagnosis for Medicare patients. In addition to describing the medical conditions of patients, ICD-9-CM codes are used for billing purposes. In contrast, the Omaha System is a multi-disciplinary terminology that is used internationally [46, 47]. OASIS and the Omaha System are nationally recognized data sets that are available in home care EHRs [45–47].

1.3.2 Omaha System

The Omaha System is a terminology developed through four federally funded research projects [46, 47]. The Omaha System consists of three components: Problem Classification Scheme, Intervention Scheme, and Problem Rating Scale for Outcomes. The Problem Classification Scheme functions as a patient problem list, and consists of 42 problems conceptually organized within four domains: environmental, psychosocial, physiological, and health related behaviors. The problems, such as “Skin” and “Cognition”, have codes, unique definitions, and signs/symptoms. The complete problem list is available in the Omaha System book and web site [46, 47]. Clinician documentation of any signs/symptoms designates the presence of the problem on the patient problem list. This logical conceptual organization of problems and signs/symptoms enhances communication within the health care team and enables meaningful aggregation of the granular signs/symptoms within the broad problem concepts. ►Table 1 provides complete lists of signs/symptoms for eight Omaha System problems.

Problems are addressed by interventions that are described using the multi-axial hierarchical Intervention Scheme. Four actions (category terms) and 75 descriptors (target terms) may be combined with any problem, for 12,600 possible problem-category-target combinations. An additional care description term may be customized to further describe the intervention. Evidence-based standardized care plans (EB-SCPs) have been developed using the Intervention Scheme [48, 49]. These are similar to computerized provider order entry (CPOE) using order sets. CPOE has resulted in improved patient care [50].

Each problem that is applicable for a patient is rated by the assessing nurse using three Likert-type scales: knowledge (what the patient knows; 1 = no knowledge, 5 = superior knowledge), behavior (what the patient does; 1 = not appropriate, 5 = consistently appropriate), and status (severity of signs/symptoms; 1 = extreme, 5 = none) [46, 47]. Typically, problems are rated upon admission, over time as indicated by the patient condition, and at discharge.

1.3.3 Omaha System Clinical Decision Support Measures

Omaha System assessment data have been used to develop clinical decision support measures, such as a maternal risk index (MRI) that forecasts public health nurse home visiting service needs for high risk mothers [51]. The MRI was the first measure based on a generalizable conceptual framework for arranging and adjusting Omaha System data to predict patient risk. In this framework, Omaha System problems are weighted and summed to calculate the magnitude of client problems. Baseline ratings corresponding to these problems are averaged to calculate a denominator that adjusts for problem severity [51].

2. Objective

Using clinical assessment data from home care EHRs, the objectives of this study were to

• 1.develop a measure to predict risk of hospitalization among home care patients (HRS), and
• 2.compare it with an existing severity of illness measure (CIC).

Our rationale for studying the HRS and CIC in isolation from demographic and administrative data was to compare the unique contributions of these two EHR data sources.

3. Methods

In this retrospective, comparative study, we employed a de-identified observational data set of patient home care records that we originally obtained from two vendors, CareFacts^TM and Champ Software. Both software programs incorporate the Omaha System and OASIS data. The software vendors contacted their home care agency customers, obtained a signed agreement to share a limited data set, extracted the data from the agency EHRs, and then securely shared the data with the UMN investigators. The convenience sample was obtained by the software vendors and provided to the investigators; therefore, the number of agencies declining to participate in studies is unknown. Participating agencies included 14 small-to-mid-sized Medicare-certified home care agencies located in the Midwest and one on the east coast. Agencies offered diverse home-based services including skilled nursing; home health aides; and physical, occupational, and speech therapy. The agencies represented various types of ownership: government/county (n = 9), hospital (n = 2), free-standing for profit (n = 1), and not for profit (n = 2). Eight of the 14 agencies reported the number of annual visits, ranging from 3,165 to 24,000 visits per year.

3.1 Sample

The unit of analysis for this study was a patient episode of care. Inclusion criteria were: the first episode of care for patients aged 65 and older served and discharged by a Medicare certified home care agency between January 1, 2004 and December 31, 2004. The hospitalization rate for this sample was 30.2%, slightly higher than published national rates [2].

3.2 Measures

The measures in this study include demographic variables, hospitalization, the CIC derived from OASIS ICD-9-CM data in a previous study [43–44], and the HRS derived from Omaha System data based on a measure developed in a previous study [51].

3.2.1 Hospitalization

Hospitalization is defined as a patient admitted for more than 24 hours to a hospital for an emergent or urgent issue and occurring while receiving services from a home care agency. In a previous study, we created a variable for “Hospitalization” (yes or no) using two methods [43–44]. First, we determined if a patient had an OASIS assessment that was a “Transfer to an Inpatient setting” and a subsequent Discharge Assessment. Second, we validated two fields in OASIS assessments that the transfer to an inpatient facility was to an acute care facility (versus nursing home or other facility) and the patient was hospitalized for an emergent or urgent reason, not a planned hospitalization. These fields were “M0855: To which Inpatient Facility has the patient been admitted?” and “M0900: Reason for hospitalization.” The inclusion of the “transfer assessment” increased the reliability of the hospitalization variable.

3.2.2 Charlson Index of Comorbidity (CIC)

As described in 1.2, the CIC is a weighted summative score based on ICD-9-CM codes obtained from the OASIS assessments. To compute the Deyo-adapted CIC in our previous study, we used SAS code that is available on-line through the University of Manitoba [42]. Applying the code to the data compiled a unique listing of ICD-9 codes from the following 18 OASIS fields: primary diagnosis (n = 1), secondary diagnoses (n = 5), reasons for inpatient treatment (n = 2), reasons for treatment change (n = 4), and payment diagnoses for Medicare patients (n = 2). It then applied weights to the 17 diagnoses as shown in ►Table 1, and computed the CIC value [42].

3.2.3 Hospitalization Risk Score

Development of the HRS was based on methodology used to create the MRI described in 1.3.2, in which the sum of selected patient problems (numerator) is adjusted by averaged baseline assessment scores (denominator) [48]. For example, upon admission if a patient had the problems of Cognition, Urinary function, Circulation, and Medication regimen, the sum of weights for these problems would be a score of 6. If the status of each problem was 3 upon admission, the average status score would be 3. Dividing the weighted sum for the problems by the average status would result in an HRS score of 2.

We developed the HRS theoretically based on the literature before viewing the sample data [6–34]. Team members were nurses (KM, BW, HS), with experience in home care (KM, BW). Three had experience with home care and informatics research (KM, CO, BW). First, we conducted a literature review described in 1.1 to identify factors that predicted hospitalization for home care patients. One author (HS) independently produced a list of candidate predictors summarized from the literature review. Second, the team reached consensus on the predictors retained in the HRS model. Third, team members with extensive experience in terminology mapping (KM, BW) mapped the predictors to eight Omaha System problems (►Table 1). Finally, all authors weighted the problems based on literature review findings.

3.2.4 Demographic and Administrative Variables

Additional variables to describe the population included: age, gender, race/ethnicity, living status, and payer.

3.3 Analytic Strategy

We used SAS (v.9.2) to conduct generalized linear models with a logit link to analyze the data with each individual encounter having two measures (HRS and CIC) related to hospitalization outcomes (yes/no). We established cut off points to illustrate differences between the HRS and CIC in describing varying levels of risk for the sample. We compared receiver operating characteristic (ROC) curves for both models, and computed area under each ROC curve (AUC). The AUC is a ranking-based measure of classification performance that enabled statistical and visual comparison of the HRS and CIC. The AUC can be interpreted in this study as the probability of correctly predicting hospitalization (yes/no) based on sensitivity and specificity. The closer the AUC is to one, the better the measure [49]. We computed the difference between the AUC values for CIC and HRS, and the likelihood of hospitalization from the predictive hospitalization values. We cross-validated results by randomly selecting four subsets for analysis without replacement (25%, 50%, and 75% of the data).

4. Results

There were 1,643 episodes of care that met the inclusion criteria. Subjects ranged in age from 65 to 106, with a sample median age of 80 (mean 79.9, standard deviation 7.7). The sample was predominantly female (63.8%) and Caucasian (98.2%); 35.7% of patients reported living alone. Insurance coverage was primarily Medicare (96.8%). Of the episodes of care, 30.2% were hospitalized. The mean number of Omaha System problems identified on a patient problem list was 4.4, with a range of 1–20 problems. Only 6.2% of episodes of care had one problem. A comparison of the CIC and HRS is shown in ►Table 1, demonstrating similarities and differences between the two measures. The most frequently documented Omaha System problems are reported in ►Table 2. The most frequently documented ICD-9-CM codes are reported in ►Table 3.

Table 2.

Frequency of Omaha System Problems in the Sample^§

Omaha System Problem (n=7307)	Frequency	Percent
Neuro-musculo-skeletal function*	1019	14.0
Skin*	820	11.2
Pain	791	10.8
Medication regimen*	667	9.1
Circulation*	600	8.2
Respiration*	501	6.9
Nutrition	379	5.2
Personal care	357	4.9
Other: Functional status	313	4.3
Technical procedure	290	4.0
Urinary function*	283	3.9
Health care supervision	275	3.8
Bowel function	199	2.7
Mental health*	111	1.5
Cognition*	104	1.4
Physical activity	102	1.4

§ Only problems with at least 100 patients were included

*Problems used in HRS (8 of 8 problems)

Table 3.

Frequency of ICD-9-CM Codes by Charlson Index of Comorbility Categories (n = 973)

Charlson Index of Comorbidity Category	Frequency	Percent
Congestive heart failure	234	24.0
Chronic Obstructive Pulmonary Disease	214	22.0
Any malignancy, including lymphoma and leukemia, except malignant neoplasm of skin	173	17.8
Cerebrovascular disease	131	13.5
Moderate-severe renal disease	43	4.4
Metastatic solid tumor	40	4.1
Dementia	31	3.2
Peripheral vascular diseases	29	3.0
Myocardial infarction	28	2.9
Connective tissue disease	19	2.0
Peptic ulcer disease	15	1.5
Paraplegia and Hemiplegia	11	1.1
Mild liver disease	5	0.5
Diabetes without chronic complications	0	0.0
Diabetes with chronic complications	0	0.0
Moderate-severe liver disease	0	0.0
AIDS	0	0.0

The HRS ranged from 0 to 5.6, with a median of 1.25. The CIC ranged from 0 to 9 and with a median of 0. Comparing of the likelihood of hospitalization for HRS vs. CIC showed that nearly three fourths of the sample was hospitalized at an HRS of 2 and a CIC of 1 (►Table 4). The AUC value for the HRS (0.63) was significantly higher than the AUC value for the CIC (0.59) (t = –7.59, p<0.0027) (►Fig. 1). Cross validation comparing the HRS and CIC resulted in the HRS performing consistently better in 13 of the 16 tests. In 2 of the 16 tests, the CIC performed better with a 25% random sampling.

Table 4.

Comparison of Likelihood of Hospitalization for Hospitalization Risk Score (HRS) and Charlson Index of Comorbidity (CIC)

Values		Hospitalized Patients Cumulative Total (N=496)
HRS	CIC	HRS	CIC
0.5		10.1%
1	1	32.3%	72.2%
1.5		51.2%
2	2	75.6%	88.9%
2.5		88.3%
3	3	96.2%	95.0%
3.5		99.2%
4	4	100.0%	96.2%

Fig. 1.

Comparison of Receiver Operating Characteristic (ROC) Curves for Hospitalization Risk Score (HRS) and Charlson Index of Comorbidity (CIC)

5. Discussion

Our study was a descriptive retrospective comparative design using a data set from previous research to test the performance of two EHR-data derived measures. Clinical OASIS data were used to derive the CIC and Omaha System data were used to derive the HRS. The advantage of using data from a previous study was that data were pre-processed, and CIC and hospitalization outcome variables had been calculated. This allowed the team to focus on the development and comparison of the HRS. The HRS predicted hospitalization better than the CIC in 13 of the 16 tests. Low values for CIC and HRS may be due to limitations of the data. For both the CIC and HRS, incomplete or inconsistent documentation, or limited fields for data collection may have resulted in incomplete scores, inherently biasing the findings of the study.

The first objective of this study was to develop a new Omaha System measure, the HRS, based on the general framework of the MRI, review of the literature, and clinical expertise of the team. This expert-based approach is a first step in developing a new measure for risk of hospitalization. The HRS reflected the most common problems in the sample with the exception of the Pain problem, and generated a score that predicted hospitalization to a limited degree. The range of 1–20 problems per patient episode of care shows that some home care patients have many health problems. Inconsistencies in documentation due to time constraints in home care practice settings, variations in agency documentation policies, and semantic equivalence in patient care plans may bias Omaha System data and HRS results. These threats should be addressed through uniform data collection protocols and policies across EHRs to enable data comparison and to ensure process interoperability, and the use of standardized care plans integrating evidence-based practice [48–50].

The second objective of the study was to compare the HRS with the CIC for predicting hospitalization of home care patients. There was a limited, but statistically significant advantage for predicting hospitalization with the HRS versus the CIC. From the cross validation results, it can be concluded that for a sufficiently large sample size (greater than 800), the AUC for HRS consistently outperformed the CIC. An AUC between 0.7 and 0.9 is considered highly accurate [52]; the AUC values for both the HRS (0.63) and CIC (0.59) were very low. Both HRS and CIC would be likely to show improved performance if variables for other known demographic and administrative predictors of hospitalization were incorporated within the algorithms.

The OASIS assessment form restricts secondary diagnoses to five fields in addition to diagnoses representing the reasons for recent hospitalization if applicable, reasons for recent treatment change, and payment diagnosis for Medicare patients [45]. However, patients often have many more co-morbid conditions. The limitation of diagnoses based on Medicare’s OASIS assessment may preclude complete and accurate data for calculation of the CIC.

Post-hoc analysis in this study showed that 4 of the 17 CIC disease categories had no patients (►Table 3). The four conditions were diabetes with or diabetes without chronic complications, mild to severe liver disease, and AIDS. Compared with recent statistics from the Centers for Disease Control for home care patients reported in 1.2, it was surprising that our sample did not include diabetic patients with chronic complications [35]. It may be that the specific complication was coded rather than diabetes (e.g. heart disease, stroke, or kidney disease). It also may be that the CIC is not sensitive to the conditions found for home care patients. The Omaha System data were more complete than the ICD-9-CM data relative to the operational definitions of the HRS and CIC, as all Omaha System problems that comprised the HRS were present in the data set.

The predictive ability of the CIC in this study is consistent with the findings of the value of the CIC in predictive models found in other studies [5]. For example, a study that examined the CIC with other variables to predict 30-day hospital readmission, the mean CIC score was 1 with a range of 0 – 2 [37]. In combination with other variables to predict readmission, the AUC was 0.65. This finding suggests that ICD-9-CM diagnoses may not be comprehensive in describing the needs of patients receiving home care [26].

This study is the first step in developing a new risk assessment measure for predicting hospitalization of home care patients; additional studies are needed. Only standardized EHR data (ICD-9-CM and the Omaha System) were included in our study to compare the value of the two EHR-based risk assessments as core measures for predicting hospitalization of home care patients. Because the same data set was used when comparing the CIC and HRS measures, no attempt was made to control for other factors in this initial phase of development of the HRS.

The addition of Omaha System problems may increase the predictive ability of the HRS. One method of selecting additional problems would be to evaluate the likelihood of hospitalization with all Omaha System problems. For example, the Pain problem occurred frequently in the sample but was not identified in the literature review. Another step to optimize the HRS is to obtain relative risk estimates for assigning weights to problems included in the HRS. Future studies should take into account that the performance of the model can be overestimated if using the entire sample for model construction. Internal validation methods, such as cross-validation, should be employed, by repeatedly partitioning the data into 10 parts, creating the model with a training set, and then validating it with a hold out data set [53].

After further substantiation of the HRS as a risk of hospitalization measure, it then should be tested by combining it with additional predictors and evaluating the performance in particular populations. The HRS should be combined with additional variables in predictive models for hospitalization of home care patients such as demographics and financial indicators [8], medication complexity [44], number and timing of visits [7], interventions planned or provided during an episode of care [54], and whether the HRS is more useful for specific populations such as frail vs. non-frail elderly [54].

Our study employed an observational convenience sample with data from two software companies from 2004 representing 14 home care agencies predominately form the Midwest. Large clinical data sets are observational data with bias inherent in data collection processes; no controls can be implemented to improve data collection or quality. As in many retrospective observational data analyses, home care nurses were unaware of the future re-use of ICD-9-CM and Omaha System data, and there were no a priori data collection protocols in place within the home care agencies to support a standard data collection process for the CIC or HRS. Such limitations must be considered when interpreting findings of any research involving EHR data. The findings of the study are preliminary and should not be generalized to other settings or populations. However, despite these limitations, the use of large EHR data sets may be a useful strategy in development of clinical decision support measures.

6. Conclusion

This study compared the value of using measures based on two standardized clinical data sets routinely documented in home care EHRs, the ICD-9-CM and the Omaha System, to identify, forecast, and describe varying levels of risk of hospitalization upon admission to home care. There was a statistically significant advantage for predicting hospitalization with the HRS versus the CIC; however the AUC values for both the HRS and CIC were less than desirable. Further research is needed to develop a robust risk measure based on patient data from home care settings. This preliminary study demonstrates the promise and challenge of use of EHR data for calculating hospitalization risk measures.

Clinical Relevance

Implementation of a refined HRS supported by a standardized assessment protocol within home care agencies may identify patients at risk for hospitalization, allowing for early intervention to prevent costly hospital admissions. Medicare costs may be reduced by preempting re-hospitalization with appropriate allocation of resources.

The Omaha System, used to develop the HRS, serves as a tool for describing evidence-based interventions, documenting services, and evaluating care, allowing for increased communication of patient needs across the continuum of patient care. This study has a number of limitations in terms of the sample size, representation by geographical location and completeness and consistency of the data. However, the initial development of the HRS in this study shows a potential way of using clinical EHR documentation in real time to predict hospital admissions.

Human Subjects

No human subjects were involved in the preparation of this manuscript. University of Minnesota Institutional Review Board approval was obtained for use of de-identified clinical data.

Conflicts of Interest

The authors are nursing faculty, informatics specialists, graduate students, and/or statisticians with expertise in use of the Omaha System in education and research. All authors declare no conflict of interest in the preparation of this manuscript. The content is solely the responsibility of the authors and does not necessarily represent the official views of the authors’ employers.