Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2017 Jun 5.
Published in final edited form as: J Hosp Med. 2015 Jul 7;10(10):670–677. doi: 10.1002/jhm.2416

Development and validation of an algorithm to identify planned readmissions from claims data

Leora I Horwitz 1,2,3, Jacqueline N Grady 4, Dorothy Cohen 4, Zhenqiu Lin 4, Mark Volpe 4, Chi Ngo 4, Andrew L Masica 5, Theodore Long 6, Jessica Wang 7, Megan Keenan 4, Julia Montague 4, Lisa G Suter 4,8, Joseph S Ross 4,9,10, Elizabeth E Drye 4,11, Harlan M Krumholz 4,6,10,12, Susannah M Bernheim 4,9
PMCID: PMC5459369  NIHMSID: NIHMS855518  PMID: 26149225

Abstract

Background

It is desirable not to include planned readmissions in readmission measures because they represent deliberate, scheduled care.

Objectives

To develop an algorithm to identify planned readmissions, describe its performance characteristics and identify improvements.

Design

Consensus-driven algorithm development and chart review validation study at 7 acute care hospitals in 2 health systems.

Patients

For development, all discharges qualifying for the publicly-reported hospital-wide readmission measure. For validation, all qualifying same-hospital readmissions that were characterized by the algorithm as planned, and a random sampling of same-hospital readmissions that were characterized as unplanned.

Measurements

We calculated weighted sensitivity and specificity, and positive and negative predictive values of the algorithm (version 2.1), compared to gold standard chart review.

Results

In consultation with 27 experts, we developed an algorithm that characterizes 7.8% of readmissions as planned. For validation we reviewed 634 readmissions. The weighted sensitivity of the algorithm was 45.1% overall; 50.9% in large teaching centers and 40.2% in smaller community hospitals. The weighted specificity was 95.9%, positive predictive value was 51.6% and negative predictive value was 94.7%. We identified 4 minor changes to improve algorithm performance. The revised algorithm had a weighted sensitivity 49.8% (57.1% at large hospitals), weighted specificity 96.5%, positive predictive value 58.7%, and negative predictive value 94.5%. Positive predictive value was poor for the two most common potentially planned procedures: diagnostic cardiac catheterization (25%) and procedures involving cardiac devices (33%).

Conclusions

An administrative claims-based algorithm to identify planned readmissions is feasible and can facilitate public reporting of primarily unplanned readmissions.

Keywords: readmissions, preventable hospitalizations, administrative data, hospital quality

Introduction

The Centers for Medicare & Medicaid Services (CMS) publicly reports all-cause risk-standardized readmission rates after acute care hospitalization for acute myocardial infarction, pneumonia, heart failure, total hip and knee arthroplasty, COPD, stroke, and for patients hospital-wide.15 Ideally, these measures should capture unplanned readmissions that arise from acute clinical events requiring urgent re-hospitalization. Planned readmissions, which are scheduled admissions usually involving non-urgent procedures, may not be a signal of quality of care. Including planned readmissions in readmission quality measures could create a disincentive to provide appropriate care to patients who are scheduled for elective or necessary procedures unrelated to the quality of the prior admission. Accordingly, under contract to CMS, we were asked to develop an algorithm to identify planned readmissions. A version of this algorithm is now incorporated into all publicly-reported readmission measures.

Given the widespread use of the planned readmission algorithm in public reporting and its implications for hospital quality measurement and evaluation, the objective of this study was to describe the development process, and to validate and refine the algorithm by reviewing charts of readmitted patients.

Methods

Algorithm development

To create a planned readmission algorithm, we first defined “planned.” We determined that readmissions for obstetrical delivery, maintenance chemotherapy, major organ transplant and rehabilitation should always be considered “planned” in the sense that they are desired and/or inevitable even if not specifically planned on a certain date. Apart from these specific types of readmissions, we defined planned readmissions as non-acute readmissions for scheduled procedures, because the vast majority of planned admissions are related to procedures. We also defined readmissions for acute illness or for complications of care as unplanned for the purposes of a quality measure. Even if such readmissions included a potentially planned procedure, because complications of care represent an important dimension of quality that should not be excluded from outcome measurement, these admissions should not be removed from the measure outcome. This definition of planned readmissions does not imply that all unplanned readmissions are unexpected or avoidable. However, it has proven very difficult to reliably define “avoidable” readmissions even by expert review of charts, and we did not attempt to do so here.6,7

In the second stage, we operationalized this definition into an algorithm. We used the Agency for Healthcare Research and Quality’s (AHRQ’s) Clinical Classification Software (CCS) codes to group thousands of individual procedure and diagnosis International Statistical Classification of Disease, Ninth Revision, Clinical Modification (ICD-9-CM) codes into clinically coherent, mutually exclusive procedure CCS categories and mutually exclusive diagnosis CCS categories, respectively. Clinicians on the investigative team reviewed the procedure categories to identify those that are commonly planned and that would require inpatient admission. We also reviewed the diagnosis categories to identify acute diagnoses unlikely to accompany elective procedures. We then created a flow diagram through which every readmission could be run to determine whether it was planned or unplanned based on our categorizations of procedures and diagnoses (Figure 1, Appendix A). This version of the algorithm (v1.0) was submitted to the National Quality Forum (NQF) as part of the hospital-wide readmission measure. The measure (NQR #1789) received endorsement in April, 2012.

Figure 1.

Figure 1

Flow diagram for planned readmissions (referenced tables are in Appendix A)

In the third stage of development, we posted the algorithm for two public comment periods and recruited 27 outside experts to review and refine the algorithm following a standardized, structured process (Appendix B). Since the measures publicly report and hold hospitals accountable for unplanned readmission rates, we felt it most important that the algorithm include as few planned readmissions in the reported, “unplanned” outcome as possible (i.e. have high negative predictive value). Therefore, in equivocal situations in which experts felt procedure categories were equally often planned or unplanned, we added those procedures to the potentially planned list. We also solicited feedback from hospitals on algorithm performance during a confidential test-run of the hospital-wide readmission measure in the fall of 2012. Based on all this feedback, we made a number of changes to the algorithm, which was then identified as v2.1. Version 2.1 of the algorithm was submitted to the NQF as part of the endorsement process for the acute myocardial infarction and heart failure readmission measures and was endorsed by NQF in January, 2013. The algorithm (v2.1) is now applied, adapted if necessary, to all publicly-reported readmission measures.8

Algorithm validation: Study cohort

We recruited two hospital systems to participate in a chart validation study of the accuracy of the planned readmission algorithm (v2.1). Within these two health systems, we selected seven hospitals with varying bed size, teaching status, and safety net status. Each included one large, academic teaching hospital that serves as a regional referral center. For each hospital’s index admissions, we applied the inclusion and exclusion criteria from the hospital-wide readmission measure. Index admissions were included for patients: age 65 or older, enrolled in Medicare fee-for-service (FFS), discharged from a non-federal short-stay acute-care hospital or critical access hospital, without an in-hospital death, not transferred to another acute care facility, and enrolled in Part A Medicare for one year prior to discharge. We excluded index admissions for patients without at least 30 days post-discharge enrollment in FFS Medicare, discharged against medical advice, admitted for medical treatment of cancer or primary psychiatric disease, admitted to a Prospective Payment System (PPS)-exempt cancer hospital, or who died during the index hospitalization. In addition, for this study, we included only index admissions that were followed by a readmission to a hospital within the participating health system between July 1, 2011 and June 30, 2012. Institutional review board approval was obtained from each of the participating health systems, which granted waivers of signed informed consent and Health Insurance Portability and Accountability Act waivers.

Algorithm validation: Sample size calculation

We determined a priori that the minimum acceptable positive predictive value, or proportion of all readmissions the algorithm labels planned that are truly planned, would be 60% and the minimum acceptable negative predictive value, or proportion of all readmissions the algorithm labels as unplanned that are truly unplanned, would be 80%. We calculated the sample size required to be confident of these values +/− 10% and determined we would need a total of 291 planned charts and 162 unplanned charts. We inflated these numbers by 20% to account for missing or unobtainable charts for a total of 550 charts. In order to achieve this sample size, we included all eligible readmissions from all participating hospitals that were categorized as planned. At the five smaller hospitals, we randomly selected an equal number of unplanned readmissions occurring at any hospital in its healthcare system. At the two largest hospitals, we randomly selected 50 unplanned readmissions occurring at any hospital in its healthcare system.

Algorithm validation: Data abstraction

We developed an abstraction tool, tested and refined it using sample charts, and built the final the tool into a secure, password-protected Microsoft Access 2007 database (Appendix C). Experienced chart abstractors with RN or MD degrees from each hospital site participated in a one-hour training session to become familiar with reviewing medical charts, defining planned/unplanned readmissions, and the data abstraction process. For each readmission, we asked abstractors to review as needed: emergency department triage and physician notes, admission history and physical, operative report, discharge summary, and/or discharge summary from a prior admission. The abstractors verified the accuracy of the administrative billing data, including procedures and principal diagnosis. In addition, they abstracted the source of admission and dates of all major procedures. Then abstractors provided their opinion and supporting rationale as to whether a readmission was planned or unplanned. They were not asked to determine whether the readmission was preventable. To determine the inter-rater reliability of data abstraction, an independent abstractor at each health system recoded a random sample of ten percent of the charts.

Statistical analysis

To ensure that we had obtained a representative sample of charts, we identified the ten most commonly planned procedures among cases identified as planned by the algorithm in the validation cohort and then compared this with planned cases nationally. To confirm the reliability of the abstraction process, we used the kappa statistic to determine the inter-rater reliability of the determination of planned or unplanned status. Additionally, the full study team, including five practicing clinicians, reviewed the details of every chart abstraction in which the algorithm was found to have misclassified the readmission as planned or unplanned. In 11 cases we determined that the abstractor had misunderstood the definition of planned readmission (i.e. not all direct admissions are necessarily planned) and we reclassified the chart review assignment accordingly.

We calculated sensitivity, specificity, positive predictive value and negative predictive value of the algorithm for the validation cohort as a whole, weighted to account for the prevalence of planned readmissions as defined by the algorithm in the national data (7.8%). Weighting is necessary because we did not obtain a pure random sample, but rather selected a stratified sample that oversampled algorithm-identified planned readmissions.9 We also calculated these rates separately for large hospitals (>600 beds) and for small hospitals (≤600 beds).

Finally, we examined performance of the algorithm for individual procedures and diagnoses in order to determine whether any procedures or diagnoses should be added or removed from the algorithm. First, we reviewed the diagnoses, procedures, and brief narratives provided by the abstractors for all cases in which the algorithm misclassified the readmission as either planned or unplanned. Second, we calculated the positive predictive value for each procedure that had been flagged as planned by the algorithm, and reviewed all readmissions (correctly and incorrectly classified) in which procedures with low positive predictive value took place. We also calculated the frequency with which the procedure was the only qualifying procedure resulting in an accurate or inaccurate classification. Third, in order to identify changes that should be made to the lists of acute and non-acute diagnoses, we reviewed the principal diagnosis for all readmissions misclassified by the algorithm as either planned or unplanned, and examined the specific ICD-9 codes within each CCS group that were most commonly associated with misclassifications.

After determining the changes that should be made to the algorithm based on these analyses, we recalculated the sensitivity, specificity, positive predictive value, and negative predictive value of the proposed revised algorithm (v3.0). All analyses used SAS v9.3 (SAS Institute, Cary, NC).

Results

Study cohort

Characteristics of participating hospitals are shown in Table 1. Hospitals represented in this sample ranged in size, teaching status, and safety net status, although all were non-profit. We selected 663 readmissions for review, 363 planned and 300 unplanned. Overall we were able to select 80% of hospitals’ planned cases for review; the remainder occurred at hospitals outside the participating hospital system. Abstractors were able to locate and review 634 (96%) of the eligible charts (range 86%-100% per hospital). The kappa statistic for inter-rater reliability was 0.83.

Table 1.

Hospital characteristics

Description Hospitals, N Readmissions selected for review, N* Readmissions reviewed, N (% of eligible) Unplanned readmissions reviewed, N Planned readmissions reviewed, N Percent of hospital’s planned readmissions reviewed*
All hospitals 7 663 634 (95.6) 283 351 77.3
Number of beds >600 2 346 339 (98.0) 116 223 84.5
>300 <=600 2 190 173 (91.1) 85 88 87.1
<300 3 127 122 (96.0) 82 40 44.9
Ownership Government 0 -- -- -- -- --
For profit 0 -- -- -- -- --
Not-for-profit 7 663 634 (95.6) 283 351 77.3
Teaching status Teaching 2 346 339 (98.0) 116 223 84.5
Non-teaching 5 317 295 (93.1) 167 128 67.4
Safety net status Safety-net 2 346 339 (98.0) 116 223 84.5
Non-safety net 5 317 295 (93.1) 167 128 67.4
Region New England 3 409 392 (95.8) 155 237 85.9
South Central 4 254 242 (95.3) 128 114 64.0
*

non-selected cases were readmitted to hospitals outside the system and could not be reviewed

The study sample included 57/67 (85%) of the procedure or condition categories on the potentially planned list. The most common procedure CCS categories among planned readmissions (v2.1) in the validation cohort were very similar to those in the national dataset (Appendix D). Of the top 20 most common planned procedure CCS categories in the validation set, all but two, therapeutic radiology for cancer treatment (CCS 211) and peripheral vascular bypass (CCS 55), were among the top 20 most common planned procedure CCS categories in national data.

Test characteristics of algorithm

The weighted test characteristics of the current algorithm (v2.1) are shown in Table 2. Overall, the algorithm correctly identified 266 readmissions as unplanned and 181 readmissions as planned, and misidentified 170 readmissions as planned and 15 as unplanned. Once weighted to account for the stratified sampling design, the overall prevalence of true planned readmissions was 8.9% of readmissions. The weighted sensitivity was 45.1% overall and was higher in large teaching centers than in smaller community hospitals. The weighted specificity was 95.9%. The positive predictive value was 51.6% and the negative predictive value was 94.7%.

Table 2.

Test characteristics of algorithm

Cohort Sensitivity Specificity Positive predictive value Negative predictive value
Algorithm v2.1
Full cohort 45.1% 95.9% 51.6% 94.7%
Large hospitals 50.9% 96.1% 53.8% 95.6%
Small hospitals 40.2% 95.5% 47.7% 94.0%
Revised algorithm v3.0
Full cohort 49.8% 96.5% 58.7% 94.5%
Large hospitals 57.1% 96.8% 63.0% 95.9%
Small hospitals 42.6% 95.9% 52.6% 93.9%

Accuracy of individual diagnoses and procedures

The positive predictive value of the algorithm for individual procedure categories varied widely, from 0% to 100% among procedures with at least 10 cases (Table 3). The procedure for which the algorithm was least accurate was CCS 211, therapeutic radiology for cancer treatment (0% positive predictive value). By contrast, maintenance chemotherapy (90%) and other therapeutic procedures, hemic and lymphatic system (100%) were most accurate. Common procedures with less than 50% positive predictive value (i.e., that the algorithm commonly misclassified as planned) were diagnostic cardiac catheterization (25%); debridement of wound, infection or burn (25%); amputation of lower extremity (29%); insertion, revision, replacement, removal of cardiac pacemaker or cardioverter/defibrillator (33%); and other hernia repair (43%). Of these, diagnostic cardiac catheterization and cardiac devices are the first and second most common procedures nationally, respectively.

Table 3.

Positive predictive value of algorithm by procedure category (among procedures with at least 10 readmissions in validation cohort)

Readmission procedure CCS code Total categorized as planned by algorithm, N Verified as planned by chart review, N Positive predictive value
47 Diagnostic cardiac catheterization; coronary arteriography 44 11 25%
224 Cancer chemotherapy 40 22 55%
157 Amputation of lower extremity 31 9 29%
49 Other operating room heart procedures 27 16 59%
48 Insertion, revision, replacement, removal of cardiac pacemaker or cardioverter/defibrillator 24 8 33%
43 Heart valve procedures 20 16 80%
Maintenance chemotherapy (Diagnosis CCS 45) 20 18 90%
78 Colorectal resection 18 9 50%
169 Debridement of wound, infection or burn 16 4 25%
84 Cholecystectomy and common duct exploration 16 5 31%
99 Other OR gastrointestinal therapeutic procedures 16 8 50%
158 Spinal fusion 15 11 73%
142 Partial excision bone 14 10 71%
86 Other hernia repair 14 6 42%
44 Coronary artery bypass graft 13 10 77%
67 Other therapeutic procedures, hemic and lymphatic system 13 13 100%
211 Therapeutic radiology for cancer treatment 12 0 0%
45 Percutaneous transluminal coronary angioplasty 11 7 64%
Total 497 272 54.7%

The readmissions with least abstractor agreement were those involving CCS 157 (amputation of lower extremity) and CCS 169 (debridement of wound; infection or burn). Readmissions for these procedures were nearly always performed as a consequence of acute worsening of chronic conditions such as osteomyelitis or ulceration. Abstractors were divided over whether these readmissions were appropriate to call “planned.”

Changes to the algorithm

We determined that the accuracy of the algorithm would be improved by removing two procedure categories from the planned procedure list (therapeutic radiation [CCS 211] and cancer chemotherapy [CCS 224]), adding one diagnosis category to the acute diagnosis list (hypertension with complications [CCS 99]), and splitting two diagnosis condition categories into acute and non-acute ICD-9 codes (pancreatic disorders [CCS 149] and biliary tract disease [CCS 152]). Detailed rationales for each modification to the planned readmission algorithm are described in Table 4. We felt further examination of diagnostic cardiac catheterization and cardiac devices was warranted given their high frequency, despite low positive predictive value. We also elected not to alter the categorization of amputation or debridement because it was not easy to determine whether these admissions were planned or unplanned even with chart review. We plan further analyses of these procedure categories.

Table 4.

Suggested changes to planned readmission algorithm v2.1, with rationale

Action Diagnosis or procedure category Algorithm Chart N Rationale for change

Remove from planned procedure list Therapeutic radiation (CCS 211) Accurate The algorithm was inaccurate in every case. All therapeutic radiology during readmissions was performed because of acute illness (pain crisis, neurologic crisis), or because scheduled treatment occurred during an unplanned readmission. In national data, this ranks as the 25th most common planned procedure identified by the algorithm v2.1.
 Planned Planned 0
 Unplanned Unplanned 0
Inaccurate
 Unplanned Planned 0
 Planned Unplanned 12

Cancer chemotherapy (CCS 224) Accurate Of the 22 correctly identified as planned, 18 (82%) would already have been categorized as planned because of a principal diagnosis of Maintenance Chemotherapy. Therefore, removing CCS 224 from the planned procedure list would only miss a small fraction of planned readmissions but would avoid a large number of misclassifications. In national data, this ranks as the 8th most common planned procedure identified by the algorithm v2.1.
 Planned Planned 22
 Unplanned Unplanned 0
Inaccurate
 Unplanned Planned 0
 Planned Unplanned 18

Add to planned procedure list None The abstractors felt a planned readmission was missed by the algorithm in 15 cases. A handful of these cases were missed because the planned procedure was not on the current planned procedure list; however, those procedures (e.g. abdominal paracentesis, colonoscopy, endoscopy) were nearly always unplanned overall and should therefore not be added as procedures that potentially qualify an admission as planned.

Remove from acute diagnosis list None The abstractors felt a planned readmission was missed by the algorithm in 15 cases. The relevant disqualifying acute diagnoses were much more often associated with unplanned readmissions in our dataset.

Add to acute diagnosis list Hypertension with complications (CCS 99) Accurate This CCS was associated with only one planned readmission (for elective nephrectomy, a very rare procedure). Every other time this CCS appeared in the dataset, it was associated with an unplanned readmission (12/13, 92%); 10 of those, however, were misclassified by the algorithm as planned because they were not excluded by diagnosis (91% error rate). Consequently, adding this CCS to the acute diagnosis list is likely to miss only a very small fraction of planned readmissions, while making the overall algorithm much more accurate.
 Planned Planned 1
 Unplanned Unplanned 2
Inaccurate
 Unplanned Planned 0
 Planned Unplanned 10

Split diagnosis condition category into component ICD-9 codes Pancreatic disorders (CCS 152) Accurate ICD-9 code 577.0 (acute pancreatitis) is the only acute code in this CCS. Acute pancreatitis was present in two cases that were misclassified as planned. Clinically there is no situation in which a planned procedure would reasonably be performed in the setting of acute pancreatitis. Moving ICD-9 code 577.0 to the acute list and leaving the rest of the ICD-9 codes in CCS 152 on the non-acute list will enable the algorithm to continue to identify planned procedures for chronic pancreatitis.
 Planned Planned 0
 Unplanned Unplanned 1
Inaccurate
 Unplanned Planned 0
 Planned Unplanned 2

Biliary tract disease (CCS 149) Accurate This CCS is a mix of acute and chronic diagnoses. Of 14 charts classified as planned with CCS 149 in the principal diagnosis field, 12 were misclassified (of which 10 were associated with cholecystectomy). Separating out the acute and non-acute diagnoses will increase the accuracy of the algorithm while still ensuring that planned cholecystectomies and other procedures can be identified. Of ICD-9 codes in CCS 149, the following will be added to acute diagnosis list: 574.0, 574.3, 574.6, 574.8, 575.0, 575.12, 576.1.
 Planned Planned 2
 Unplanned Unplanned 3
Inaccurate
 Unplanned Planned 0
 Planned Unplanned 12

Consider for change after additional study Diagnostic cardiac catheterization (CCS 47) Accurate The algorithm misclassified as planned 25/38 (66%) unplanned readmissions in which diagnostic catheterizations were the only qualifying planned procedure. It also correctly identified 3/3 (100%) planned readmissions in which diagnostic cardiac catheterizations were the only qualifying planned procedure. This is the highest volume procedure in national data.
 Planned Planned 3*
 Unplanned Unplanned 13*
Inaccurate
 Unplanned Planned 0*
 Planned Unplanned 25*
* N of cases in which CCS 47 was the only qualifying procedure

Insertion, revision, replacement, removal of cardiac pacemaker or cardioverter/defibrillator (CCS 48) Accurate The algorithm misclassified as planned 4/5 (80%) unplanned readmissions in which cardiac devices were the only qualifying procedure. However, it also correctly identified 7/8 (87.5%) planned readmissions in which cardiac devices were the only qualifying planned procedure. CCS 48 is the 2nd most common planned procedure category nationally.
 Planned Planned 7*
 Unplanned Unplanned 1*
Inaccurate
 Unplanned Planned 1*
 Planned Unplanned 4*
*N of cases in which CCS 48 was the only qualifying procedure

The revised algorithm (v3.0) had a weighted sensitivity of 49.8%, weighted specificity of 96.5%, positive predictive value of 58.7%, and negative predictive value of 94.5% (Table 2). In aggregate, these changes would increase the reported unplanned readmission rate from 16.0% to 16.1% in the hospital-wide readmission measure, using 2011–2012 data, and would decrease the fraction of all readmissions considered planned, from 7.8% to 7.2%.

Discussion

We developed an algorithm based on administrative data that in its currently implemented form is very accurate at identifying unplanned readmissions, ensuring that readmissions included in publicly-reported readmission measures are likely to be truly unplanned. However, nearly half of readmissions the algorithm classifies as planned are actually unplanned. That is, the algorithm is over-cautious in excluding unplanned readmissions that could have counted as outcomes, particularly among admissions that include diagnostic cardiac catheterization or placement of cardiac devices (pacemakers, defibrillators). However, these errors only occur within the 7.8% of readmissions that are classified as planned and therefore do not affect overall readmission rates dramatically. A perfect algorithm would reclassify approximately half of these “planned” readmissions as unplanned, increasing the overall readmission rate by 0.6 percentage points.

On the other hand, the algorithm also only identifies approximately half of true planned readmissions as planned. Since the true prevalence of planned readmissions is low (approximately 9% of readmissions based on weighted chart review prevalence, or an absolute rate of 1.4%), this low sensitivity has a small effect on algorithm performance. Removing all true planned readmissions from the measure outcome would decrease the overall readmission rate by 0.8 percentage points, similar to the expected 0.6 percentage point increase that would result from better identifying unplanned readmissions; thus a perfect algorithm would likely decrease the reported unplanned readmission rate by a net 0.2%. Overall, the existing algorithm appears to come close to the true prevalence of planned readmissions, despite inaccuracy on an individual case basis. The algorithm performed best at large hospitals, which are at greatest risk of being statistical outliers and of accruing penalties under the Hospital Readmissions Reduction Program.10

We identified several changes that marginally improved the performance of the algorithm by reducing the number of unplanned readmissions that are incorrectly removed from the measure, while avoiding the inappropriate inclusion of planned readmissions in the outcome. This revised algorithm, v3.0, was applied to public reporting of readmission rates at the end of 2014. Overall, implementing these changes increases the reported readmission rate very slightly. We also identified other procedures associated with high inaccuracy rates, removal of which would have larger impact on reporting rates, and which therefore merit further evaluation.

There are other potential methods of identifying planned readmissions. For instance, as of October 1, 2013, new administrative billing codes were created to allow hospitals to indicate that a patient was discharged with a planned acute care hospital inpatient readmission, without limitation as to when it will take place.11 This code must be used at the time of the index admission to indicate that a future planned admission is expected, and was specified only to be used for neonates and patients with acute myocardial infarction. This approach, however, would omit planned readmissions that are not known to the initial discharging team, potentially missing planned readmissions. Conversely, some patients discharged with a plan for readmission may be unexpectedly readmitted for an unplanned reason. Given that the new codes were not available at the time we conducted the validation study, we were not able to determine how often the billing codes accurately identified planned readmissions. This would be an important area to consider for future study.

An alternative approach would be to create indicator codes to be applied at the time of readmission that would indicate whether that admission was planned or unplanned. Such a code would have the advantage of allowing each planned readmission to be flagged by the admitting clinicians at the time of admission rather than by an algorithm that inherently cannot be perfect. However, identifying planned readmissions at the time of readmission would also create opportunity for gaming and inconsistent application of definitions between hospitals; additional checks would need to be put in place to guard against these possibilities.

Our study has some limitations. We relied on the opinion of chart abstractors to determine whether a readmission was planned or unplanned; in a few cases, such as smoldering wounds that ultimately require surgical intervention, that determination is debatable. Abstractions were done at local institutions to minimize risks to patient privacy and therefore we could not centrally verify determinations of planned status except by reviewing source of admission, dates of procedures and narrative comments reported by the abstractors. Finally, we did not have sufficient volume of planned procedures to determine accuracy of the algorithm for less common procedure categories or individual procedures within categories.

In summary, we developed an algorithm to identify planned readmissions from administrative data that had high specificity and moderate sensitivity, and refined it based on chart validation. This algorithm is in use in public reporting of readmission measures to maximize the probability that the reported readmission rates represent truly unplanned readmissions.12

Acknowledgments

Financial support: This work was performed under contract HHSM-500-2008-0025I/HHSM-500-T0001, Modification No. 000008, entitled “Measure Instrument Development and Support,” funded by CMS, an agency of the US Department of Health and Human Services. Drs. Horwitz and Ross are supported by the National Institute on Aging (K08 AG038336 and K08 AG032886, respectively) and by the American Federation for Aging Research through the Paul B. Beeson Career Development Award Program. Dr. Krumholz is supported by grant U01 HL105270-03 (Center for Cardiovascular Outcomes Research at Yale University) from the National Heart, Lung, and Blood Institute. No funding source had any role in the study design; in the collection, analysis, and interpretation of data; or in the writing of the report. The CMS reviewed and approved the use of its data for this work and approved submission of the manuscript.

Footnotes

Conflicts of Interest: All authors have completed the Unified Competing Interest form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare that all authors have support from the Centers for Medicare & Medicaid Services for the submitted work. In addition, Dr. Ross is a member of a scientific advisory board for FAIR Health Inc. Dr. Krumholz chairs a cardiac scientific advisory board for UnitedHealth and is the recipient of research grants from Medtronic and Johnson&Johnson through Yale University. All other authors report no conflicts of interest.

References

  • 1.Lindenauer PK, Normand SL, Drye EE, et al. Development, validation, and results of a measure of 30-day readmission following hospitalization for pneumonia. J Hosp Med. 2011;6(3):142–150. doi: 10.1002/jhm.890. [DOI] [PubMed] [Google Scholar]
  • 2.Krumholz HM, Lin Z, Drye EE, et al. An administrative claims measure suitable for profiling hospital performance based on 30-day all-cause readmission rates among patients with acute myocardial infarction. Circ Cardiovasc Qual Outcomes. 2011;4(2):243–252. doi: 10.1161/CIRCOUTCOMES.110.957498. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Keenan PS, Normand SL, Lin Z, et al. An administrative claims measure suitable for profiling hospital performance on the basis of 30-day all-cause readmission rates among patients with heart failure. Circ Cardiovasc Qual Outcomes. 2008;1:29–37. doi: 10.1161/CIRCOUTCOMES.108.802686. [DOI] [PubMed] [Google Scholar]
  • 4.Grosso LM, Curtis JP, ZL, et al. [Accessed 25 March, 2014];Hospital-level 30-day all-cause risk-standardized readmission rate following elective primary total hip arthroplasty (THA) and/or total knee arthroplasty (TKA) 2012 http://www.qualitynet.org/dcs/ContentServer?c=Page&pagename=QnetPublic%2FPage%2FQnetTier3&cid=1219069855841.
  • 5.Horwitz LI, Partovian C, Lin Z, et al. Development and use of an administrative claims measure for profiling hospital-wide performance on 30-day unplanned readmission. Ann Intern Med. 2014;161(10 Suppl):S66–75. doi: 10.7326/M13-3000. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.van Walraven C, Jennings A, Forster AJ. A meta-analysis of hospital 30-day avoidable readmission rates. J Eval Clin Pract. 2011;18(6):1211–1218. doi: 10.1111/j.1365-2753.2011.01773.x. [DOI] [PubMed] [Google Scholar]
  • 7.van Walraven C, Bennett C, Jennings A, Austin PC, Forster AJ. Proportion of hospital readmissions deemed avoidable: a systematic review. CMAJ Canadian Medical Association Journal. 2011;183(7):E391–402. doi: 10.1503/cmaj.101860. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Horwitz LI, Partovian C, Lin Z, et al. [Accessed 3 June, 2015];Centers for Medicare & Medicaid Services Planned Readmission Algorithm – Version 2.1. 2013 http://hscrc.maryland.gov/documents/HSCRC_Initiatives/readmissions/Version-2-1-Readmission-Planned-CMS-Readmission-Algorithm-Report-03-14-2013.pdf.
  • 9.Obuchowski NA, Zhou XH. Prospective studies of diagnostic test accuracy when disease prevalence is low. Biostatistics. 2002;3(4):477–492. doi: 10.1093/biostatistics/3.4.477. [DOI] [PubMed] [Google Scholar]
  • 10.Joynt KE, Jha AK. Characteristics of hospitals receiving penalties under the Hospital Readmissions Reduction Program. JAMA. 2013;309(4):342–343. doi: 10.1001/jama.2012.94856. [DOI] [PubMed] [Google Scholar]
  • 11.Centers for Medicare and Medicaid Services. Inpatient Prospective Payment System/Long-Term Care Hospital (IPPS/LTCH) Final Rule. Federal Register. 2013;78:50533–50534. [Google Scholar]
  • 12.Long SK, Stockley K, Dahlen H. Massachusetts health reforms: uninsurance remains low, self-reported health status improves as state prepares to tackle costs. Health Aff (Millwood) 2012;31(2):444–451. doi: 10.1377/hlthaff.2011.0653. [DOI] [PubMed] [Google Scholar]

RESOURCES