Supplemental Digital Content is available in the text
Abstract
Recognizing potentially avoidable hospital readmission and admissions are important health care quality issues. We develop prediction models for inpatient readmission and outpatient admission to hospitals for older adults
In the retrospective cohort study with 2 million sampling file of the National Health Insurance Research Database in Taiwan, older adults (aged ≥65 y/o) with a first admission in 2008 were enrolled in the inpatient cohort (N = 39,156). The outpatient cohort included subjects who had ≥1 outpatient visit in 2008 (N = 178,286). Each cohort was split into derivation (3/4) and validation (1/4) data set.
Primary outcome of the inpatient cohort: 30-day readmission from the date of discharge. The outpatient cohort included hospital admissions within the 1-year follow-up period. Candidate risk factors include demographics, comorbidities, and previous health care utilizations. Series of logistic regression models were applied with area under the receiver operating curves (AUCs) to identify the best model.
Roughly 1 of 7 (14.6%) of the inpatients was readmitted within 30 days, and 1 of 5 (19.1%) of the outpatient cohort was admitted within 1 year. Age, education, use of home health care, and selected comorbidities (e.g., cancer with metastasis) were included in the final model. The AUC of the inpatient readmission model was 0.655 (95% confidence interval [CI] 0.646–0.664) and outpatient admission model was 0.642 (95% CI 0.639–0.646). Predictive performance was maintained in both validation data sets. The goodness-to-fit model demonstrated good calibration in both groups.
We developed and validated practical clinical prediction models for inpatient readmission and outpatient admissions for general older adults with indicators easily obtained from an administrative data set.
INTRODUCTION
The optimal goals of inpatient admission are to cure, relieve, or comfort patients in dealing with their problems. However, the admission and hospitalization course can be major stresses to vulnerable older populations and result in a loss of independence in activities of daily living (ADLs).1,2 In previous reports, ∼18% of very frail elderly and 24% of heart failure cases in the community experienced admission during a 1-year follow-up.3,4 Furthermore, many older adults experience repeated admissions. In previous studies, early readmission (within 30 days after discharge) rates varied from 11% to 19% according to different settings and groups.5–7 A high readmission rate was associated with a poor patient outcome and incurred a considerable economic burden.8 The Centers for Medicare and Medicaid Services (Baltimore, MD) have used readmission rates as a pay-for-performance indicator, and hospitals would be penalized if their risk-adjusted readmission rates were high.9
A prediction model that estimates the possibility of admission or readmission among older adults may be helpful in clinical settings. Innovative managements may be used to target high-risk fragile patients with the goal of decreasing admission and readmission rates. It is also important for policy makers to reallocate resources to provide additional services such as transitional care to mitigate these problems.
Risk factors for admission or readmission could be multifactorial. A systematic review showed that sociodemographics, comorbidities, functional status, and previous health care utilizations were thought to have a significant effect on readmission.10 However, the risk of initial admission among community-dwelling elderly is still uncertain. A limited number of studies have targeted some diseases but without systemic model construction.11,12 Furthermore, even risk factors for readmission are inconsistent in different study settings and mainly focus on a specific disease such as heart failure, acute myocardial infarction, or chronic obstructive pulmonary disease13,14 rather than older adults in general, and some studies were smaller or designed for other purposes.6,15–17
For better prevention, risk factors for both admission and readmission should be considered equally. The objective of this study was to develop prediction models for admission and readmission using the National Health Insurance (NIH) Research Database in Taiwan.
METHODS
Study Design
The proposed study is a secondary data analysis from existing public database with retrospective cohort design.
Databases
The NHI program is a mandatory insurance plan implemented in 1995. NHI provides health care coverage to 99.6% of the population and has contracted with >90% of the health care institutions in Taiwan.18,19 For this analysis, we used a longitudinal health care database containing claims data for a cohort of 2 million subjects randomly selected from the total Taiwan population in 2000. The database was established and is held securely by the Collaboration Center of Health Information Applications, Ministry of Health and Welfare (Taiwan). Files of inpatient and outpatient expenditures, orders, and registry for beneficiaries were used to form the analytical database and can be linked through a scrambled personal identification number. Data linkage and analysis were performed at the center. The institutional review board of National Taiwan University Hospital approved this study.
Cohort Selection
Inpatient Cohort to Predict 30-Day Readmission
Using the inpatient claims, we identified 41,586 older adults (≥65 years of age) who had a first admission (the index admission) in 2008 (Figure 1). To reduce the likelihood of including scheduled readmissions, we excluded hospital admissions with a principal discharge diagnosis of cancer because these patients were likely to have a planned hospital readmission for cancer treatments. Patients also were excluded if they died during the index admission or if they had missing information on sociodemographic variables. The final cohort included 39,156 subjects. The 30-day readmission was observed starting from the discharge date (flowchart can be found in Figure 1).
FIGURE 1.

Flowchart for selecting the outpatient and inpatient cohorts. NHI = National Health Insurance.
Outpatient Cohort to Predict 1-Year Hospital Admission
We identified 200,165 older adults. The first outpatient visit date in 2008 was defined as the index date. Because prior admission was a strong determinant of next admission,17 we excluded patients hospitalized within 6 months before the index date. Subjects also were excluded if they had missing information on sociodemographic variables. The remaining 178,286 subjects were followed up for 1 year, and hospital admissions were ascertained.
Candidate Variables
The candidate predictors including demographic variables, clinical variables, and health care utilizations, and were selected based on the literature review and the knowledge of clinical experts.10 Clinical variables and health care utilizations were assessed by searching claims records within 1 year before the index date. Comorbid conditions, assessed using the diagnosis codes of the International Classification of Diseases, Ninth Revision, Clinical Modification (supplementary Table A1) were considered to be present if the diagnosis codes were recorded on ≥1 inpatient claim or ≥3 outpatient claims (variables are listed in Table 1).20 Medication prescribed to patients, such as antihypertensive drug, antidiabetic drugs, nonsteroidal anti-inflammatory drugs, antidepressant, anxiolytics, hypnotics, and antipsychotics, were analyzed, but there were no significant effect (data not shown). They are not included in prediction model at final.
TABLE 1.
Baseline Characteristics of Inpatient and Outpatient Cohorts, 2008

Statistical Analysis
We split the data set into a development data set (75% of the data set; n = 29,351 for inpatients; n = 133,726 for outpatients) and a validation data set (25% of the data set; n = 9805 for inpatients; n = 44,560 for outpatients). Summary statistics, including mean and standard deviation, were provided for continuous variables, such as age, numbers of admission, and so on. Frequencies and proportions were used to summarize discrete variables such as education level, disease status, and so on. Missing information on sociodemographic variables were excluded from analyses. Candidate variables not associated with 1-year hospital admission (P > 0.05) in the univariate logistic regression models were excluded from further analysis. We constructed a series of logistic regression models with the candidate variables added sequentially and compared the prediction performance among the models.
The model performance was assessed by using several metrics.21 Discrimination was measured using the area under the receiver operating curve (AUC).21 Akaike's information criterion (AIC)21 was computed to compare the overall model fit among models. We also calculated the calibration slope to assess the agreement between predicted probabilities and the observed outcomes.21 All of the analyses were completed using the SAS 9.3 package (SAS Institute, Inc, Cary, NC).
In the selection of comorbidities retained in the final model, the Charlson Comorbidity Index (CCI) and simple counts of several common comorbid diseases were both used. The final 6-disease model was finally chosen instead of CCI (19 items) because 2 models had similar performance in AUC, but the former one had fewer items and probably less administration time.
RESULTS
Description of the Inpatient and Outpatient Cohorts
In terms of baseline characteristics, men accounted for 52.1% of the inpatient cohort and 47.9% of the outpatient cohort. Subjects in the inpatient cohort were older (mean age, 76.4 ± 7.0 years vs 74.2 ± 6.5 years; P ≤ 0.001). In both cohorts, the majority of patients was married and did not continue their education after high school. Diabetes and heart disease were among the most prevalent of all comorbidities. Nearly 1 of 7 (14.6%) of the inpatient cohort was readmitted within 30 days, and 1 of 5 (19.1%) of the outpatient cohort was admitted within 1 year. In both cohorts, all variables of the subjects in the development data set and those in the full cohort were similar (Table 1).
Models Predicting 30-Day Hospital Readmission for the Inpatient Cohort
In the development data set of the inpatient cohort (Table 2), there was a steady increase in discriminative ability from model 1 (age only), to model 2 (adding marital status and education), model 3 (model 2 + health care utilization variables), model 4 (model 2 + 10 selected comorbidities), and model 5 (model 3 + selected comorbidities) (AUC from 0.547, 95% confidence interval [CI] 0.538–0.555 to 0.655, 95% CI 0.646–0.664). Model 5 was chosen as the final model because it had the highest AUC and the lowest AIC values among the 5 models. In the validation data set (Table 3), the discrimination of model 5 was similar to that in the development data set. The calibration measures indicated a satisfactory fit for model 5 in the validation data set.
TABLE 2.
Odds Ratios and Performance of Various Models in the Inpatient Development Data Set

TABLE 3.
Performance of Models Predicting 1-Year Admission and 30-Day Readmission

Models Predicting 1-Year Hospital Admissions for the Outpatient Cohort
In the development data set of the outpatient cohort (Table 4), using approaches similar to the inpatient cohort, there were significant improvements in discriminative ability from model 1 (AUC 0.580, 95% CI 0.577–0.584) to model 5 (0.642, 95% CI 0.639–0.646). Models 4 and 5 performed similarly in terms of AUC. However, the AIC was smaller (or better) for model 5; therefore, it was selected as the final model. The variables selected in model 5 for the outpatient cohort were quite similar to those used with the inpatient cohort. The differences were that anemia was selected in the inpatient cohort, but chronic kidney disease (CKD) was used in the outpatient cohort. Also, the number of admissions in the previous year was significant only in the outpatient cohort. In the validation data set (Table 3), the performance of model 5 was satisfactory (AUC 0.638, with a slope close to 1, indicating a good fit).
TABLE 4.
Odds Ratios and Performance Measures of Various Models in the Development Data Set of the Outpatient Cohort

DISCUSSION
Our study presented 2 practical models to predict the 30-day readmission rate of inpatients after discharge and the 1-year admission rate of outpatients, using a large population-based administrative data set. Our model included easily accessible sociodemographic factors, medical conditions, and prior health care utilizations with good fit in the validation data set.
Analysis of readmission in the general population is difficult because of the complexity of patient conditions and multifactorial determinants. This is the reason why most previous studies focused only on specific diseases with high readmission rates and attempted to use homogenous populations.22 However, the disease-orientated analysis had more limited generalizability. Also, recent literature on “posthospitalization syndrome” indicated that common risk factors for readmission may exist regardless of causes of index admissions.23 Therefore, the risk factors identified from our models for older adults in general can be targets for designing comprehensive strategies to mitigate “posthospitalization syndrome.”
Compared with previous studies for disease specific severity, the hospital lethality rate in our inpatient cohort during admission was 5.6%, which was similar with intermediate risk group of acutely decompensated heart failure24 and patients with acute ischemic stroke.25 It seemed lower than community-acquired pneumonia26 (acute infection) and myocardial infarction27 (critical illness) condition.
Our readmission rate within 30 days was 14.6%, which was lower than in patients with several specific conditions such as congestive heart failure13,28 or acute myocardial infarction29 but modestly higher than in a multicenter study in the United States with a general population.6
Studies on the prediction of admission of community-dwelling elderly are limited. Our study provides one of the first population-based estimates of the 1-year hospitalization rate (19%) in nationally representative samples. Other studies have reported estimates of specific diseases with relatively small sample sizes only, such as heart failure patients11 or very frail elderly.3
Our study confirms the findings that determinants of both admission and readmission are often multifactorial.10 However, we found strikingly similar predictors for both the inpatient and the outpatient cohorts. A plausible explanation was that determinants of admission and readmission share a common unmeasured geriatric condition such as frailty that increases the vulnerability of the elderly to admission and readmission.30
In our final model, age was a significant factor predicting admission and readmission. The postulated mechanism is that aging is related to several disease-susceptible conditions, such as vasomotor instability, reduced total body water content, and reduced ventilation.2 Comorbidity carried the greatest risk in admission or readmission analysis.31 Chronic obstructive pulmonary disease, heart disease, diabetes mellitus, and cancer status were included in both final prediction models. Cancer, especially metastatic cancer, carried the highest risk among all comorbidities. Consistent with another inpatient study from Taiwan, anemia, but not CKD, was predictive of 30-day readmission.31 However, in an outpatient setting, we found that CKD, but not anemia, was predictive of admission within 1 year. In outpatient settings, complications of CKD may prompt admission, whereas most chronic anemia can be managed as an outpatient.
Functional impairment such as needing ADLs assistance may have an association with readmission.32 Although there was a lack of functional variables in the administrative database, we used home health care service (HHS) as a surrogate of functional impairment in our model. In Taiwan, patients receiving HHS were often highly dependent on others for their ADLs (need assistance >50% of the time when the patient is awake or chair-bound/bed-bound) or for tube care needs (nasogastric tube, Foley catheter, and tracheostomy tube).33 Differences in health care utilization may indicate the different health profiles of elderly people.34 Our study showed prior admission history and emergency room visits had a significant effect in our final model, and the results were consistent with previous studies.15,35
We used the NHI program with its excellent coverage of the population in Taiwan, and selection and participation bias could be avoided with the setting.36 Our prediction model for both the community-dwelling and inpatient groups provides a wide application and good extrapolation. This is the first study to analyze outpatient and inpatient information in 1 setting model with systematic construction at the same time. Although not directly from our analysis, it seems that there were some common factors that might contribute initial hospital admission and subsequent 30-day readmission. A larger longitudinal data set that follows patients from outpatient status to first inpatient admission, and then extends to 30 days after discharge may help to identify the common risk factors.
This study presents some limitations. We used retrospective administrative data, and potential disease misclassification may exist. However, the strict selection in our study, using ≥1 inpatient claim or ≥3 outpatient claims may avoid the bias.20 Second, the data was derived from year 2008, which might not be applicable to current times. Before 2016, ICD-9CM is the only one disease coding system in Taiwan. Although health care technology may improve the disease diagnosis and treatment in the last 8 years, the ranking of major causes of death in the elderly seemed similar according the statistics data from the Ministry of Health and Welfare in Taiwan. Under the same disease coding system and similar major causes of death, we thought the periodical changes may not have significant effect to the final results.
Third, our retrospective administrative data lacked information on personal history (e.g., smoking) or global health status assessments that may be associated with the risk of readmission. The impacts of such variables on admission and readmission would be further clarified in prospective studies or other database collecting such information. Finally, the interference of readmission of community-dwelling subjects should be considered. To avoid the confounder, we excluded outpatient elderly with a previous admission within 6 months. In addition, our model has limited generalizability to the Taiwanese population. To increase generalizability, further international studies may be needed to calibrate the final model according to the different population in the world.
CONCLUSION
Overall, from a clinical and quality improvement perspective, our study presents a well-structured, large population-based prediction model of admission and readmission. Both models had fair-to-good discrimination with reliable validation.
Supplementary Material
Footnotes
Abbreviations: ADLs = activities of daily living, AIC = Akaike's information criterion, AUC = area under the receiver operating curve, CCI = Charlson Comorbidity Index, CKD = chronic kidney disease, HHS = home health care service, NHI = National Health Insurance.
The authors have no conflicts of interest to disclose.
REFERENCES
- 1.Covinsky KE, Palmer RM, Fortinsky RH, et al. Loss of independence in activities of daily living in older adults hospitalized with medical illnesses: increased vulnerability with age. J Am Geriatr Soc 2003; 51:451–458. [DOI] [PubMed] [Google Scholar]
- 2.Creditor MC. Hazards of hospitalization of the elderly. Ann Intern Med 1993; 118:219–223. [DOI] [PubMed] [Google Scholar]
- 3.de Stampa M, Vedel I, Buyck JF, et al. Impact on hospital admissions of an integrated primary care model for very frail elderly patients. Arch Gerontol Geriatr 2014; 58:350–355. [DOI] [PubMed] [Google Scholar]
- 4.Cowie MR, Fox KF, Wood DA, et al. Hospitalization of patients with heart failure: a population-based study. Eur Heart J 2002; 23:877–885. [DOI] [PubMed] [Google Scholar]
- 5.Jencks SF, Williams MV, Coleman EA. Rehospitalizations among patients in the Medicare fee-for-service program. N Engl J Med 2009; 360:1418–1428. [DOI] [PubMed] [Google Scholar]
- 6.Silverstein MD, Qin H, Mercer SQ, et al. Risk factors for 30-day hospital readmission in patients ≥65 years of age. Proc (Bayl Univ Med Cent) 2008; 21:363–372. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Comette P, D’Hoore W, Malhomme B, et al. Differential risk factors for early and later hospital readmission of older patients. Aging Clin Exp Res 2005; 17:322–328. [DOI] [PubMed] [Google Scholar]
- 8.Dignan FL, Potter MN, Ethell ME, et al. High readmission rates are associated with a significant economic burden and poor outcome in patients with grade III/IV acute GvHD. Clin Transplant 2013; 27:E56–E63. [DOI] [PubMed] [Google Scholar]
- 9.Averill RF, McCullough EC, Hughes JS, et al. Redesigning the Medicare inpatient PPS to reduce payments to hospitals with high readmission rates. Health Care Financ Rev 2009; 30:1–15. [PMC free article] [PubMed] [Google Scholar]
- 10.Kansagara D, Englander H, Salanitro A, et al. Risk prediction models for hospital readmission: a systematic review. JAMA 2011; 306:1688–1698. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Chaudhry SI, McAvay G, Chen S, et al. Risk factors for hospital admission among older persons with newly diagnosed heart failure: findings from the Cardiovascular Health Study. J Am Coll Cardiol 2013; 61:635–642. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Desai NR, Giugliano RP. Can we predict outcomes in atrial fibrillation? Clin Cardiol 2012; 35 Suppl 1:10–14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Amarasingham R, Moore BJ, Tabak YP, et al. An automated model to identify heart failure patients at risk for 30-day readmission or death using electronic medical record data. Med Care 2010; 48:981–988. [DOI] [PubMed] [Google Scholar]
- 14.Prieto-Centurion V, Gussin HA, Rolle AJ, et al. Chronic obstructive pulmonary disease readmissions at minority-serving institutions. Ann Am Thorac Soc 2013; 10:680–684. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Howell S, Coory M, Martin J, et al. Using routine inpatient data to identify patients at risk of hospital readmission. BMC Health Serv Res 2009; 9:96. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Halfon P, Eggli Y, Pretre-Rohrbach I, et al. Validation of the potentially avoidable hospital readmission rate as a routine indicator of the quality of hospital care. Med Care 2006; 44:972–981. [DOI] [PubMed] [Google Scholar]
- 17.Donze J, Aujesky D, Williams D, et al. Potentially avoidable 30-day hospital readmissions in medical patients: derivation and validation of a prediction model. JAMA Intern Med 2013; 173:632–638. [DOI] [PubMed] [Google Scholar]
- 18.National Health Insurance in Taiwan 2015–2016 Annual Report (bilingual). Taipei: National Health Insurance Administration, Ministry of Health and Welfare. http://www.nhi.gov.tw/English/webdata/webdata.aspx?menu=11&menu_id=290&webdata_id=2974&WD_ID=; 2015. Access date: April 12, 2016. [Google Scholar]
- 19.Cheng TM. Taiwan's new national health insurance program: genesis and experience so far. Health Aff (Millwood) 2003; 22:61–76. [DOI] [PubMed] [Google Scholar]
- 20.Lin CC, Lai MS, Syu CY, et al. Accuracy of diabetes diagnosis in health insurance claims data in Taiwan. J Formos Med Assoc 2005; 104:157–163. [PubMed] [Google Scholar]
- 21.Steyerberg EW, Vickers AJ, Cook NR, et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology 2010; 21:128–138. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Dharmarajan K, Hsieh AF, Lin Z, et al. Diagnoses and timing of 30-day readmissions after hospitalization for heart failure, acute myocardial infarction, or pneumonia. JAMA 2013; 309:355–363. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Krumholz HM. Post-hospital syndrome—an acquired, transient condition of generalized risk. N Engl J Med 2013; 368:100–102. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Royston P, Altman DG. Risk stratification for in-hospital mortality in acutely decompensated heart failure. JAMA 2005; 293:2467–2468. [DOI] [PubMed] [Google Scholar]
- 25.Heuschmann PU, Kolominsky-Rabas PL, Misselwitz B, et al. Predictors of in-hospital mortality and attributable risks of death after ischemic stroke: the German Stroke Registers Study Group. Arch Intern Med 2004; 164:1761–1768. [DOI] [PubMed] [Google Scholar]
- 26.Marrie TJ, Wu L. Factors influencing in-hospital mortality in community-acquired pneumonia: a prospective study of patients not initially admitted to the ICU. Chest 2005; 127:1260–1270. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Canto JG, Rogers WJ, Goldberg RJ, et al. Association of age and sex with myocardial infarction symptom presentation and in-hospital mortality. JAMA 2012; 307:813–822. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Hammill BG, Curtis LH, Fonarow GC, et al. Incremental value of clinical data beyond claims data in predicting 30-day outcomes after heart failure hospitalization. Circ Cardiovasc Qual Outcomes 2011; 4:60–67. [DOI] [PubMed] [Google Scholar]
- 29.Krumholz HM, Lin Z, Drye EE, et al. An administrative claims measure suitable for profiling hospital performance based on 30-day all-cause readmission rates among patients with acute myocardial infarction. Circ Cardiovasc Qual Outcomes 2011; 4:243–252. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Rockwood K, Song X, MacKnight C, et al. A global clinical measure of fitness and frailty in elderly people. CMAJ 2005; 173:489–495. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Shu CC, Lin YF, Hsu NC, et al. Risk factors for 30-day readmission in general medical patients admitted from the emergency department: a single centre study. Intern Med J 2012; 42:677–682. [DOI] [PubMed] [Google Scholar]
- 32.Coleman EA, Min SJ, Chomiak A, et al. Posthospital care transitions: patterns, complications, and risk identification. Health Serv Res 2004; 39:1449–1465. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Chang CB, Lai HY, Yang SY, et al. Patient- and clinic visit-related factors associated with potentially inappropriate medication use among older home healthcare service recipients. PloS One 2014; 9:e94350. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Liu LF, Tian WH, Yao HP. Utilization of health care services by elderly people with National Health Insurance in Taiwan: the heterogeneous health profile approach. Health Policy 2012; 108:246–255. [DOI] [PubMed] [Google Scholar]
- 35.Baker CL, Zou KH, Su J. Risk assessment of readmissions following an initial COPD-related hospitalization. Int J Chron Obstruct Pulmon Dis 2013; 8:551–559. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Hsing AW, Ioannidis JP. Nationwide Population Science: Lessons From the Taiwan National Health Insurance Research Database. JAMA Intern Med 2015; 175:1527–1529. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
