Predicting the Risk of Readmission in Pneumonia. A Systematic Review of Model Performance

Abstract

Rationale: Predicting which patients are at highest risk for readmission after hospitalization for pneumonia could enable hospitals to proactively reallocate scarce resources to reduce 30-day readmissions.

Objectives: To synthesize the available literature on readmission risk prediction models for adults who are hospitalized because of pneumonia and describe their performance.

Methods: We systematically searched Ovid MEDLINE, Embase, The Cochrane Library, and Cumulative Index to Nursing and Allied Health Literature databases from inception through July 2015. We included studies of adults discharged with pneumonia that developed or validated a model that predicted hospital readmission. Two independent reviewers abstracted data and assessed the risk of bias.

Measurements and Main Results: Of 992 citations reviewed, 7 studies met inclusion criteria, which included 11 unique risk prediction models. All-cause 30-day readmission rates ranged from 11.8 to 20.8% (median, 17.3%). Model discrimination (C statistic) ranged from 0.59 to 0.77 (median, 0.63) with the highest-quality, best-validated model, the Centers for Medicare and Medicaid Services Pneumonia Administrative Model performing modestly (C Statistic of 0.63 in 4 separate multicenter cohorts). The best performing model (C statistic of 0.77) was a single-site study that lacked internal validation. The models had adequate calibration, with patients predicted as high risk for readmission having a higher average observed readmission rate than those predicted to be low risk. None of the studies included pneumonia illness severity scores, and only one included measures of in-hospital clinical trajectory and stability on discharge, robust predictors of readmission.

Conclusions: We found a limited number of validated pneumonia-specific readmission models, and their predictive ability was modest. To improve predictive accuracy, future models should include measures of pneumonia illness severity, hospital complications, and stability on discharge.

Hospital readmissions among patients with pneumonia are frequent, costly, and potentially avoidable (1–5). Despite efforts to optimize inpatient care delivery, 30-day readmissions are estimated to occur in 17 to 25% of patients hospitalized for pneumonia, at a cost of $10 billion (2–5). Since the Centers for Medicare and Medicaid Services (CMS) implemented the Hospital Readmission Reduction Program in 2012, there has been an increased focus on pneumonia readmissions, because hospitals with higher-than-expected risk-adjusted 30-day readmission rates face major financial penalties (6).

Predicting which patients hospitalized for pneumonia are at highest risk for readmission could enable hospitals to proactively identify patients and deploy interventions in real time to reduce 30-day readmissions. A systematic review by Kansagara and colleagues has shown that most readmission risk prediction models have modest performance (7). However, this review did not identify a single pneumonia-specific readmission model in the peer-reviewed literature and focused primarily on multi-condition and cardiovascular disease models.

Therefore, the objective of this study was to conduct a high-quality synthesis of the available literature on readmission risk prediction models for patients hospitalized with pneumonia to assess model performance and methodologic quality.

Methods

Data Sources and Searches

We searched Ovid MEDLINE and Ovid MEDLINE InProcess, the Cumulative Index to Nursing and Allied Health Literature (CINAHL), the Cochrane Library (Cochrane Central Register of Controlled Trials, Cochrane Database of Systematic Reviews, and the Database of Abstracts of Reviews of Effect), and Embase from database inception through July 2015 for studies of readmission risk prediction models in adults hospitalized with pneumonia.

All citations were imported into an electronic database (EndNote X7; Thomson-Reuters Corp, New York, NY). We used subject headings and text words to identify articles that contained the following three concepts: (1) readmission (readmi*, re-admi*, rehosp*, re-hosp*, patient readmission/, readmission/), (2) risk (model*, predict*, risk*, util*, use*, usage, risk/, risk assessment/ risk factors/), and (3) pneumonia (pneumonia, pneumonia/). The search strategies are provided in detail in the online supplement.

Study Selection

Two authors (M.W. and D.W.) reviewed the abstracts and full-text articles of potentially relevant references identified from the literature search for eligibility. References of included articles were also searched to identify additional eligible studies. Criteria for inclusion were: (1) full text in English; (2) study population included adult patients 18 years or older discharged from the hospital with pneumonia; (3) article is a primary study that derives and/or validates a risk prediction model for hospital readmission after an index admission for pneumonia; (4) the model predicts the risk for the first 30-day hospital readmission, not a series or sequence of hospital readmissions; and (5) at least one measure of model performance (discrimination or calibration) was reported in the article or made available by contacting the corresponding author.

Data Extraction and Methodological Quality Assessment

Using a standardized form, two reviewers (M.W. and D.W.) extracted data on the population characteristics, setting, number of patients and hospitals in the derivation and validation cohorts, definition of pneumonia, method and time interval of readmission outcome ascertainment, method of derivation and validation, domains of predictors tested, predictors included in the final model, accuracy of risk prediction, and quality assessment.

To facilitate a comparison of the models, we classified predictors into one of nine categories on the basis of prior conceptual frameworks of readmission risk (demographics, socioeconomic status, comorbidities, utilization, laboratory results, vital signs, imaging, procedures, and medications) (4, 7). Disagreements between reviewers were resolved through discussion. If consensus could not be achieved, a third author (A.N.M) resolved discrepancies. Corresponding authors were contacted if data were missing.

The accuracy of risk prediction was assessed by evaluating the model’s discrimination and calibration. We assessed discrimination on the basis of the C statistic, which is the probability that, given two individuals hospitalized with pneumonia (one who was readmitted and the other who was not), the model will predict a higher risk for the readmitted patient than for the non-readmitted patient (8). A C statistic of 0.5 indicates a model performs no better than chance, 0.6 to 0.7 is considered modest discrimination, 0.71 to 0.8 indicates very good discrimination, and greater than 0.8 is considered strong (9). Model calibration is the degree to which predicted rates are similar to those observed in the population (7). To examine calibration, we reported the observed risk for readmission for the predicted lowest- and highest-risk groups.

We assessed the quality of included studies using elements from the standards of evidence for evaluating clinical prediction rules and the study quality assessment criteria used by Kansagara and colleagues (7, 10). Studies were considered to be high quality if they included an adequate description and generalizability of the population, had nonbiased selection of patients, ascertained readmissions within 30 days beyond the index hospital, and broadly validated the model in external cohorts (vs. narrow validation in a single cohort or no validation altogether).

Data Synthesis

A metaanalysis was not able to be performed due to the heterogeneity of the included studies. Results were qualitatively synthesized with a focus on the predictors included in each model, model performance, and methodological quality.

Results

Of 992 titles identified by our search algorithm, 91 qualified for abstract review, 12 for full-text review, and 7 met our inclusion criteria (Figure 1) (11–17). Of the seven included studies, 11 unique risk-prediction models were tested. The CMS Pneumonia Administrative Model was the most commonly studied—validated in five separate cohorts (12, 14, 15). The objective of eight of the models was to identify patients hospitalized for pneumonia at high risk for readmission for potential intervention (11, 13, 16, 17), whereas for three of the models the objective was to estimate hospital-level risk-adjusted 30-day readmission rates for the purpose of hospital profiling (12, 14, 15).

Figure 1.

Systematic review study selection flowchart.

Study characteristics are shown in Table 1. All studies were based in the United States except for Capelastegui and colleagues (11), which was conducted at a single academic medical center in Spain. All studies defined pneumonia as the primary discharge diagnosis using International Classification of Diseases, Ninth Revision codes for any type, except for one prospective study that defined community-acquired pneumonia using a clinical diagnosis of symptoms and imaging (11). The study populations ranged from single academic medical centers to national data (Medicare and Veterans Affairs). Five studies predicted all-cause 30-day readmissions, and two studies developed separate models to predict 30-day pneumonia-related and pneumonia-unrelated readmissions (11, 13). Across all the studies, the all-cause 30-day readmission rates ranged from 11.8 to 20.8% (median, 17.3%). Only two studies reported the rates of pneumonia-related readmissions, which were much lower (2.6 and 7.2%).

Table 1.

Details of pneumonia readmission prediction model studies

Study	Model	Purpose of Model	Setting	Population Age; Study Dates	Type of Pneumonia; Definition	Derivation Cohort, N	Validation Cohort, N (Method of Validation)	Observed 30-Day Readmit Rates, N (%)*
Capelastegui et al., 2009 (11)	Pneumonia related	Identify high-risk patients	1 AMC in Spain	≥ 18 yr; July 2003 to June 2007	CAP; prospective	1,117	None	81 (7.2)
	Pneumonia unrelated					1,117	None	51 (4.5)
Hebert et al., 2014 (17)	EMR model	Identify high-risk patients	1 AMC in Ohio	≥ 18 yr; August 2009 to July 2011	All types; ICD-9 codes	1,171	Cohort 1: 258 (split sample)	48 (18.6)
							Cohort 2: 552 (historical)	98 (17.8)
Lindenauer et al., 2011 (12)	CMS Pneumonia Administrative Model	Risk adjustment to profile hospitals	National Medicare data	≥ 65 yr; 2005–2006	All types; ICD-9 codes	226,545	Cohort 1: 226,706 (split-sample)	39,673 (17.5)†
							Cohort 2: 536,015 (historical)	92,730 (17.3)†
							Cohort 3: 47,429 (separate cohort)	8,063 (17.0)
	CMS Pneumonia Medical Record Model					47,429	None	8,063 (17.0)
Mather et al., 2014 (13)	All cause	Identify high-risk patients	1 AMC in Connecticut	≥ 65 yr; January 2009 to March 2012	All types; ICD-9 codes	956	956 (bootstrapping)	148 (15.5)
	Pneumonia related					956	956 (bootstrapping)	25 (2.6)
	Pneumonia unrelated					956	956 (bootstrapping)	123 (12.8)
	Modified CMS Pneumonia Medical Record Model‡					956	956 (bootstrapping)	148 (15.5)
Nagasako et al., 2014 (14)	SES-enriched CMS Pneumonia Administrative Model	Risk adjustment to profile hospitals	Medicare data from Missouri	≥ 65 yr; June 2009 to May 2012	All types; ICD-9 codes	25,729	29,855 (bootstrapping) †	3,877 (15.0)†
	CMS Pneumonia Administrative Model					N/A	29,855 (bootstrapping) †	3,877 (15.0)†
O’Brien et al., 2015 (15)	CMS Pneumonia Administrative Model	Risk adjustment to profile hospitals	National VA data	≥ 65 yr; October 2005 to September 2010	All types; ICD-9 codes	N/A	Cohort 1: 31,068 (VA data only)	5,499 (17.7)
							Cohort 2: 30,758 (VA & Medicare)	6,398 (20.8)
Tang et al., 2014 (16)	VA predictor model	Identify high-risk patients	National VA data	≥ 65 yr; October 2001 to September 2007	All types; ICD-9 codes	22,567	22,567 (split sample)	3,024 (13.4)†

Definition of abbreviations: AMC = academic medical center; CAP = community-acquired pneumonia; CMS = Centers for Medicare and Medicaid Services; EMR = electronic medical record; ICD-9 = International Classification of Diseases, Ninth Revision; N/A = not applicable; SES = socioeconomic status; VA = Veterans Affairs.

^*Reported for the respective validation cohort. If no validation cohort was used, then observed readmission rates were reported among the derivation cohort. All models predicted all-cause 30-day readmission, unless otherwise specified.

^†Data obtained from contacting study author.

^‡Included variables (n = 11) in the CMS Medical Record Model that were available to the study authors.

Predictors of Readmission

The predictors included in each model varied (Table 2). The number of predictors included per model ranged from 2 to 45. All models included medical comorbidities. Demographics were included in 9 models, socioeconomic status in 10, prior healthcare use in 6, laboratory values in 9, vital signs in 4, medications in 6, imaging in 4, and procedures in 2. No studies included pneumonia severity-of-illness scores, such as the Pneumonia Severity Index or CURB-65 (confusion of new onset, blood urea nitrogen, respiratory rate, blood pressure, age 65 yr or older), and only one study included predictors on the in-hospital evolution of clinical severity (treatment failure, decompensation of comorbidities, and number of instability factors on discharge) (11). The remaining studies included predictors available within the first day of admission. The complete list of included predictors and their associated effect sizes are shown in Table E1 in the online supplement.

Table 2.

Domains of predictors evaluated and included in pneumonia readmission risk prediction models

Study	Model	Domains of Predictors Evaluated	Domains of Predictors Included in Final Model
Capelastegui et al., 2009 (11)	Pneumonia related	D, SES, C, U, L, V, I, P, M	L, V, I, P
	Pneumonia unrelated	D, SES, C, U, L, V, I, P, M	D, C
Hebert et al., 2014 (17)	EMR model	D, SES, C, U, L, V, I, P, M	C, U, L, P, M
Lindenauer et al., 2011 (12)	CMS Pneumonia Administrative Model	D, C	D, C
	CMS Pneumonia Medical Record Model	D, C, U, L, V, I, M	D, C, L, V, I, M
Mather et al., 2014 (13)	All cause	D, SES, C, U, L, V, I, M	D, SES, C, U, L, M
	Pneumonia related	D, SES, C, U, L, V, I, M	D, SES, C, U, L, M
	Pneumonia unrelated	D, SES, C, U, L, V, I, M	D, SES, C, U, L, V, I
	Modified CMS Pneumonia Medical Record Model	D, SES, C, L, V, I, M	D, C, L, M
Nagasako et al., 2014 (14)	SES-enriched CMS Pneumonia Administrative Model	D, SES, C, U	D, SES, C, U
	CMS Pneumonia Administrative Model	D, C	D, C
O’Brien et al., 2015 (15)	CMS Pneumonia Administrative Model	D, C	D, C
Tang et al., 2014 (16)	VA predictor model	D, SES, C, U, P, M	D, SES, C, U, M

Definition of abbreviations: C = comorbidities; CMS = Centers for Medicare and Medicaid Services; D = demographics; EMR = electronic medical record; I = imaging; L = laboratory results; M = medications; P = procedures; SES = socioeconomic status; U = utilization; V = vital signs; VA = Veterans Affairs.

Model Performance

For predicting all-cause readmission, model discrimination (C statistic) ranged from 0.59 to 0.77 (median, 0.63). The CMS Pneumonia Administrative Model, which was the most commonly tested risk prediction model, consistently had a C statistic of 0.63 in four separate cohorts (Table 3) (12, 15). However, for unclear reasons, when validated using state-level Medicare data from Missouri, the C statistic for the CMS Model was 0.72 (E. Nagasako, M.D., Ph.D.; e-mail communication, September 2, 2015) (14). Notably, the addition of census tract–level socioeconomic data to the CMS Pneumonia Administrative Model did not improve model discrimination (C statistic of 0.72 for both) (14). Risk prediction models derived using more clinically granular data (i.e., laboratory results and vital signs) than administrative claims data did not necessarily have better discrimination, with C statistics ranging from 0.59 to 0.67 for all-cause readmission (13, 16, 17).

Table 3.

Performance of pneumonia-specific risk prediction models for 30-day readmission

Study	Model*	Discrimination, C Statistic†	Pseudo R2	Calibration: Observed Readmission Rate‡
				Lowest Predicted Risk Group (%)	Highest Predicted Risk Group (%)
Capelastegui et al., 2009 (11)	Pneumonia related	0.65 (derivation cohort)	Not reported	1.2 (tertile)§	6.3 (tertile)§
	Pneumonia unrelated	0.77 (derivation cohort)	Not reported	0.6 (sextile)§	16.2 (sextile)§
Hebert et al., 2014 (17)	EMR model	Cohort 1: 0.73	0.09§	6.0	36.0
		Cohort 2: 0.66	0.02§
Lindenauer et al., 2011 (12)	CMS Pneumonia Administrative Model	Cohort 1: 0.63	0.05	9	31
		Cohort 2: 0.63	0.05	8	31
		Cohort 3: 0.63	0.05	8	30
	CMS Pneumonia Medical Record Model	0.59 (derivation cohort)	0.02	10	26
Mather et al., 2014 (13)	All cause	0.67	0.13	7.5	43.0
	Pneumonia related	0.56§	0.16	3.3	36.6
	Pneumonia unrelated	0.39§	0.11	9.1	34.0
	Modified CMS Pneumonia Medical Record Model	0.48§	0.08	4.2	35.1
Nagasako et al., 2014 (14)	SES-enriched CMS Pneumonia Administrative Model	0.72§	0.14§	3.6§	38.7§
	CMS Pneumonia Administrative Model	0.72§	0.14§	3.6§	38.6§
O’Brien et al., 2015 (15)	CMS Pneumonia Administrative Model	Cohort 1: 0.63	Not reported	10.0	31.8
		Cohort 2: 0.63	Not reported	12.0	36.7
Tang et al., 2014 (16)	VA predictor model	0.61	0.03§	7.5 (quintile)§	21.1 (quintile)§

Definition of abbreviations: EMR = electronic medical record; CMS = Centers for Medicare and Medicaid Services; SES = socioeconomic; VA = Veterans Affairs.

^*All models predicted all-cause 30-day readmission, unless otherwise specified.

^†Discrimination reported is for predicting 30-day readmission in the validation cohort, unless otherwise specified.

^‡Range of mean observed risk for 30-day readmission is reported by lowest-risk to highest-risk decile, unless otherwise specified.

^§Data obtained from contacting study author.

The two studies that derived models separately predicting pneumonia-related and pneumonia-unrelated readmissions were conducted in single academic medical centers. One study was not internally validated (see quality assessment below) (11), thus limiting the interpretability of the study’s findings, and the other study derived and internally validated models with extremely poor discrimination (C statistic, 0.39 to 0.56) (13).

Calibration for included models is shown in Table 3. The models were able to adequately risk stratify patients, with observed readmission rates ranging from approximately a 3- to 10-fold difference between the lowest and highest predicted risk groups.

Aside from the study by Nagasako and colleagues, the CMS Pneumonia Administrative Model had a somewhat narrower spread of predicted risk than models derived from electronic health record (EHR) data (14).

Quality Assessment of Study Methods

Model quality was variable across studies (Table 4). All studies included an adequate description of the population and had nonbiased selection of patients; however, three studies developed models from a single academic medical center without external validation, which greatly limits external generalizability. Furthermore, these three studies only partially ascertained readmissions, because they only captured readmission outcomes at the index hospital. The level of evidence for model validation also varied across studies. The models derived in the study by Capelastegui and colleagues (11) were neither internally nor externally validated. The CMS Pneumonia Administrative Model had the highest level of evidence for model validation, as it was broadly validated in five distinct cohorts spanning different populations and time periods.

Table 4.

Assessment of study quality

Study	Model	Generalizability of Population	Nonbiased Selection	Readmission Adequately Ascertained	Level of Evidence for Model Validation
Capelastegui et al., 2009 (11)	Pneumonia related	No (single center)	Yes	Partly, only index hospital	No validation performed
	Pneumonia unrelated	No (single center)	Yes	Partly, only index hospital	No validation performed
Hebert et al., 2014 (17)	EMR model	No (single center)	Yes	Partly, only index hospital	Narrow validation (split cohort, historical cohort)
Lindenauer et al., 2011 (12)	CMS Pneumonia Administrative Model	Yes (national Medicare data)	Yes	Yes	Broad validation (split cohort, historical cohort, and separate cohort)
	CMS Pneumonia Medical Record Model	Yes (national Medicare data)	Yes	Yes	No validation performed
Mather et al., 2014 (13)	All cause	No (single center)	Yes	Partly, only index hospital	Narrow validation (bootstrapping)
	Pneumonia related	No (single center)	Yes	Partly, only index hospital	Narrow validation (bootstrapping)
	Pneumonia unrelated	No (single center)	Yes	Partly, only index hospital	Narrow validation (bootstrapping)
	Modified CMS Pneumonia Medical Record Model	No	Yes	Partly, only index hospital	Narrow validation (bootstrapping)
Nagasako et al., 2014 (14)	SES-enriched CMS Pneumonia Administrative Model	Partial (Medicare data in Missouri)	Yes	Yes	Narrow validation (bootstrapping using same cohort)
	CMS Pneumonia Administrative Model	Partial (Medicare data in Missouri)	Yes	Yes	Narrow validation (bootstrapping)
O’Brien et al., 2015 (15)	CMS Pneumonia Administrative Model	Yes (national VA data)	Yes	Yes	Broad validation (separate cohort)
Tang et al., 2014 (16)	VA predictor model	Yes (national VA data)	Yes	Yes	Narrow validation (split sample)

Definition of abbreviations: EMR = electronic medical record; CMS = Centers for Medicare and Medicaid Services; SES = socioeconomic; VA = Veterans Affairs.

Discussion

In this systematic review, we identified 11 unique pneumonia readmission risk prediction models. The median all-cause 30-day readmission rate was 17.3%, meaning that one in six patients hospitalized for pneumonia was readmitted within 30 days of discharge. The majority of models were developed in U.S. populations of patients 65 years of age or older. Three models were developed from administrative claims data with the intent of estimating risk-standardized readmission rates for hospital profiling and benchmarking purposes, including the CMS Pneumonia Administrative Model. Eight models were derived from EHR data or Veterans Affairs administrative data (which included more clinical detail than traditional claims data), with the goal of identifying patients at high risk for 30-day readmission for whom real-time identification and enrollment in a transitional care intervention may improve outcomes (18). Most models had poor to modest predictive ability (median C statistic of 0.63). The one model with very good discrimination was a single hospital site study of low methodological rigor (not internally or externally validated).

We found that models derived from more clinically granular data, which incorporated laboratory results and vital sign values, did not necessarily have better predictive ability than models derived from administrative claims data. Unlike for other disease-specific readmissions risk prediction models (e.g., congestive heart failure), the inclusion of more domains of predictors and increased granularity of data did not necessarily improve model performance for predicting 30-day readmissions among patients hospitalized with pneumonia (7, 19).

One potential explanation for this phenomenon is that models derived from more clinically enriched data did not adequately incorporate measures of pneumonia illness severity. None of the studies included in this review incorporated the Pneumonia Severity Index or the CURB-65 score, which are strong predictors of mortality and have also been shown to be associated with hospital readmissions (11, 20–24). Furthermore, the only study that included measures of in-hospital clinical trajectory and stability on discharge, robust predictors of postdischarge adverse outcomes, had very good discrimination (C statistic of 0.77); however, because of the study’s low quality due to limited generalizability, incomplete ascertainment of readmissions, and lack of validation, it is unclear whether inclusion of these measures improved readmission risk prediction (11, 23, 25–27). Nonetheless, this approach is promising and warrants further investigation.

As opposed to mortality, which is easier to predict based on illness severity and comorbidity burden (19), hospital readmissions are a more complex phenomenon stemming from the interplay among patient-, hospital-, community-, and environmental-level factors. Despite the fact that current readmission risk prediction models have not taken full advantage of incorporating measures of pneumonia illness severity, clinical trajectory, and stability on discharge that are currently available in the EHR, readmissions after pneumonia hospitalization may have less to do with traditional medical factors than once believed (28, 29).

Although several studies in this review included marital status, mental illness diagnoses, and census-tract income levels as predictors, these metrics may not fully capture an individual’s social support, self-sufficiency, health literacy, and socioeconomic disadvantage. Thus, further improvement in pneumonia readmission models may require accounting for psychosocial and behavioral factors not routinely captured in health information systems.

Another reason it may be difficult to predict readmissions among this population is because readmissions specifically related to the pneumonia itself are uncommon, ranging from 2.6 to 7.2%. Thus, most readmissions are not due to a recurrence or inadequate treatment of the pneumonia itself but rather from the impact of the acute illness on their other comorbidities and general health. To identify patients at high risk for readmission, it is essential to not only include predictors of pneumonia illness severity and compliance with guideline-concordant therapies but also include predictors of frailty and medical complexity that may put patients at greater risk for posthospital syndrome or decompensation of their other chronic conditions (30).

Although predictive accuracy at the individual level is modest, risk scores estimated from pneumonia-specific readmissions models corresponded to a clinically meaningful gradient of observed readmission risk, such that the group of patients predicted as being at high risk for readmission had an approximately twofold higher observed readmission rate than the median readmission rate and a 3- to 10-fold higher readmission rate than the group of patients identified by the risk prediction models as being at low risk. Therefore, hospitals and health systems can currently use available pneumonia risk prediction models to help identify a subset of patients at highest risk and enroll these patients in resource-intensive transitional care interventions to potentially prevent readmissions (18). This is essential to the sustainability and durability of interventions aimed at lowering readmission rates, because most hospitals do not have the resources to enroll every patient hospitalized for pneumonia for a transitional care intervention, nor would such an approach be cost effective.

An important caveat to this approach is that there is insufficient evidence that transitional care interventions specifically targeted to patients hospitalized for pneumonia decrease readmissions. However, given that the national pneumonia readmission rate decreased by 6% from 2009 to 2013 after the implementation of hospital financial penalties (31), a proportion of these readmissions are likely preventable. Further research is needed to assess the type and intensity of transitional care intervention best suited to this population. Because only a small proportion of readmissions are directly related to pneumonia itself, transitional care interventions for patients with pneumonia will need to be multifaceted and address factors unique to pneumonia, including appropriate antibiotic selection and readiness for discharge using Halm’s criteria for clinical stability, as well as the most common factors associated with potentially avoidable readmissions in general, including improved communication between inpatient and outpatient providers, patient education, and timely access to care after discharge (32).

An alternative strategy to identify patients with pneumonia at highest risk for readmission is to use existing multicondition risk prediction models, such as the LACE index (length of stay, acuity of admission, comorbidities, and emergency department visits), HOSPITAL score (hemoglobin level at discharge, discharge from oncology service, sodium level at discharge, procedure during hospital stay, index admission type as urgent or emergent, number of admissions in previous year, and length of stay), or EHR-based models (33–35). Implementing readmission risk prediction models for every condition may be time-consuming and costly. Although we hypothesize that the inclusion of predictors specific to pneumonia would result in superior risk prediction, future research is needed to perform head-to-head comparisons between these two strategies to test whether using a pneumonia-specific risk prediction model is worth the added effort and complexity. Furthermore, because pneumonia-specific readmissions risk prediction models would be most clinically useful if implemented earlier in the hospitalization when in-hospital components of transitional care interventions can be more effectively implemented (36), further research is needed to assess whether models that include predictors available on admission perform as well as models that include predictors available on the day of discharge.

Limitations

Our review has certain limitations. First, despite a comprehensive literature search strategy, we may have overlooked studies published in non-English languages or nonindexed studies in the gray literature. Second, few studies directly compared models within the same population, so caution should be used when directly comparing model performance across different populations.

Third, the included studies did not report positive predictive value and likelihood ratios for the highest risk group of patients for readmission. However, we assessed the ability of risk prediction models to identify patients hospitalized with pneumonia at highest risk for readmission using calibration, which is an appropriate method for stratifying individuals into different risk categories for prognosis of a future event (37, 38). Fourth, because all studies except for one defined pneumonia using discharge International Classification of Diseases, Ninth Revision codes, it is unclear whether defining pneumonia prospectively on admission meaningfully influences risk prediction.

Conclusions

There are currently a limited number of validated pneumonia-specific readmission risk prediction models. The predictive accuracy (discrimination) of published models is modest at best. However, model calibration of existing pneumonia readmissions models is adequate to enable a risk-stratified approach to identifying and enrolling high-risk patients for resource-intensive transitional care interventions. Future research in this area should include measures of pneumonia illness severity, hospital complications, and stability on discharge available in the EHR from the entire hospital course and not just from the first day of admission. At present, many factors influencing readmissions in this population likely remain unmeasured and/or unaccounted for.