Measuring Hospital Quality Using Pediatric Readmission and Revisit Rates

Abstract

OBJECTIVE:

To assess variation among hospitals on pediatric readmission and revisit rates and to determine the number of high- and low-performing hospitals.

METHODS:

In a retrospective analysis using the State Inpatient and Emergency Department Databases from the Healthcare Cost and Utilization Project with revisit linkages available, we identified pediatric (ages 1–20 years) visits with 1 of 7 common inpatient pediatric conditions (asthma, dehydration, pneumonia, appendicitis, skin infections, mood disorders, and epilepsy). For each condition, we calculated rates of all-cause readmissions and rates of revisits (readmission or presentation to the emergency department) within 30 and 60 days of discharge. We used mixed logistic models to estimate hospital-level risk-standardized 30-day revisit rates and to identify hospitals that had performance statistically different from the group mean.

RESULTS:

Thirty-day readmission rates were low (<10.0%) for all conditions. Thirty-day rates of revisit to the inpatient or emergency department setting ranged from 6.2% (appendicitis) to 11.0% (mood disorders). Study hospitals (n = 958) had low condition-specific visit volumes (37.0%–82.8% of hospitals had <25 visits). The only condition with >1% of hospitals labeled as different from the mean on 30-day risk-standardized revisit rates was mood disorders (4.2% of hospitals [n = 15], range of hospital performance 6.3%–15.9%).

CONCLUSIONS:

We found that when comparing hospitals’ performances to the average, few hospitals that care for children are identified as high- or low-performers for revisits, even for common pediatric diagnoses, likely due to low hospital volumes. This limits the usefulness of condition-specific readmission or revisit measures in pediatric quality measurement.

What’s Known on This Subject:

Readmissions have been identified as a priority area for pediatric inpatient quality measurement nationally. However, it is unknown whether readmission rates vary meaningfully across hospitals and how many hospitals would be identified as high- or low-performers.

What This Study Adds:

Only a few hospitals that care for children are high- or low-performers when their condition-specific revisit rates are compared with average rates across hospitals. This limits the usefulness of condition-specific readmission or revisit measures in pediatric quality measurement.

Preventable hospital readmissions are a topic of national focus as potential indicators of clinical failure and unnecessary expenditures.¹ In pediatrics, readmissions within a year of an index admission are common and cost more than $1 billion annually.^2,3 As a result, development of a pediatric readmissions measure has been identified as a priority for national quality reporting programs.^4,5 Adult condition-specific readmission rates for Medicare patients are already publicly reported online in Medicare’s national quality report. Outlier hospitals are identified by whether they have statistically different readmission rates compared with national benchmarks (18.5% for pneumonia, 19.7% for heart attack, and 24.7% for congestive heart failure).⁶ Starting fiscal year 2013 Medicare is deducting reimbursements to hospitals for excess readmissions.⁷

However, it is not known whether there is sufficient variation in readmission rates among hospitals admitting children to ensure that using those rates would allow meaningful comparative performance measurement. In particular, studies of other pediatric performance measures have shown that low event rates and low patient volume make identification of performance outliers challenging.^8–10 If we can identify outlier hospitals, studying those hospitals can better assess whether readmissions are preventable and help delineate best practices in pediatric inpatient care.¹¹

The existing literature on hospital variations in pediatric readmissions is limited, likely due to the difficulty of tracking readmissions over time and across hospitals. The Agency for Healthcare Research and Quality reports readmission statistics from a nationally representative sample of hospitals¹²; however, it does not describe hospital-level performance variation. Several other studies report pediatric readmission rates for multiple centers, but most were limited to freestanding children’s hospitals.^2,3,13–15 Other large studies did not describe hospital-level performance variation.^16–18 Berry et al described hospital-level variation on 30-day readmissions for multiple conditions, but data were again limited to freestanding children’s hospitals, which the authors estimated receive no more than 25% of all pediatric admissions nationally.¹⁹ Thus, it is not known how much variation in performance on readmission rates exists for the majority of pediatric providers.

We address this gap in the literature using a multistate database of pediatric hospitalizations to calculate condition-specific, risk-standardized readmission rates for common pediatric inpatient diagnoses and to assess the numbers of outlier hospitals with readmission rates that are higher or lower compared with the overall sample. Additionally, because previous research suggests that event rates for pediatric measures might be too low to distinguish among hospitals^9,10 and because unplanned visits to the emergency department (ED) may represent potentially preventable failures in care, we decided a priori also to evaluate a composite measure of “revisit” rates, that is, either a readmission or a return to the ED.

Methods

Data and Setting

We used statewide administrative databases because these are the only large data sets that identify readmissions or revisits to hospitals other than the index hospital. States participating in the State Inpatient Databases and State Emergency Department Databases send discharge abstracts from all non-federal hospitals to the Healthcare Cost and Utilization Project annually.²⁰ Several states also send unique patient identifiers that allow identification of patient readmissions or returns to the ED, whether at the index institution or any other in the state, after an index hospitalization. For this study, we used all State Inpatient Databases and State Emergency Department Databases states and years with revisit linkages available (California, Florida, North Carolina, and Nebraska for 2008–2009; Arizona and Utah for 2006–2007),²¹ combining data sets across years to broaden the study’s generalizability.

We compared characteristics of study hospitals to other hospitals nationally (number of beds, teaching status, profit-status and control, urban versus rural, and percent Medicaid patients) by using the American Hospital Association (AHA) database. Because previous readmissions work has focused on freestanding children’s hospitals, we also looked at proportion of freestanding children’s hospitals in our study sample and nationally, identifying them with the Children’s Hospital Association membership list.²²

Patients

We excluded visits for which records did not include a unique identifier because revisits for these records could not be recognized. Because unique identifiers in state administrative databases are often social security numbers, we hypothesized that very young patients would be disproportionately likely to be missing a unique identifier. We confirmed this hypothesis (Supplemental Table 6) and excluded patients <1 year old (n = 1 905 936) and then the remaining patients without a unique identifier (n = 419 792).

We chose to examine patients up to age 21 because we anticipated that some patients who were at greater risk of being readmitted were children with complex chronic conditions (CCCs),² who may be cared for in the pediatric setting up until age 21.²³ We excluded patients who were transferred from 1 facility to another because there is no consensus approach to attribution of the readmission in such cases. We also excluded patients who died during the index hospitalization.

Diagnoses of Interest

We focused on the most common diagnoses for pediatric admissions because these have higher sample size than other diagnoses, and the objective of the study was to assess hospital performance variation on condition-specific readmission rates. The diagnoses were identified by querying the top 10 principal admission diagnoses using the Agency for Healthcare Research and Quality’s Clinical Classification Software (CCS)²⁴ groupings in the nationally representative pediatric inpatient KID database.²⁵ We did not analyze pregnancy and childbirth visits (CCS 224) because obstetrics is an independently measured area of quality.²⁶ We also did not analyze bronchiolitis visits because 75% of patients with bronchiolitis in the KID 2009 database are <1 year old, and so it was not a top diagnosis for our study population.

Outcomes

We calculated 30-day and 60-day all-cause readmission and revisit rates for individual conditions. A revisit was defined as either a readmission or a return to the ED after discharge, even if that ED visit did not lead to readmission. For all numerators, each admission or ED visit to any hospital (the index hospital or another hospital) for any diagnosis was counted as a readmission or revisit. The denominators included any hospital discharges with a principal diagnosis for the disease.

If a patient had ≥1 additional admissions within 30 (or 60) days of discharge, we did not consider the additional admissions as index admissions. Thus, any admission was either an index admission or a readmission, but not both.^27,28

Analyses

We used Student’s t and χ² tests to compare hospital characteristics.

For each diagnostic group, we developed a separate risk-adjusted model for 30-day revisit rates. The base model for each group included age, gender, and the presence or absence of a CCC,²⁹ using a recently published list of CCC categories³⁰ because having a CCC is associated with an increased risk of readmission.^2,3 We used a binary “any CCC” indicator because cell numbers were too small to use individual CCCs. Because the epilepsy case definition shared some International Classification of Diseases, Ninth Revision (ICD-9) codes with the CCC ICD-9 codes, the model for epilepsy used a modified CCC variable that did not count the epilepsy diagnoses as a CCC. No other condition shared ICD-9 codes with the CCC definitions.

From our base models, we tested for all interactions. We included state as a fixed effect because pediatric readmission rates have been shown to vary across states and because many public reports are done at the state level, using only within-state comparisons.³¹ State was not included in the epilepsy model because of collinearity between individual hospitals and the state variable. This collinearity was due to a concentration of epilepsy patients in 1 or 2 hospitals in some states. For instance, ∼70% of epilepsy patients in Utah were seen in 1 hospital. Supplemental Table 7 shows the variables in each risk adjustment model, as well as the discrimination and calibration metrics for each model.

Because the Centers for Medicare and Medicaid Services (CMS) is currently adopting pediatric quality measures for meaningful use reporting,³² we adapted the CMS hospital readmissions methodology to calculate risk-adjusted rates and identify outliers.²⁸ CMS uses a hierarchical model to stabilize performance estimates for low-volume hospitals and avoid penalizing these hospitals for high readmission rates that may be due to chance.²⁸ This is particularly important in pediatrics, given the low pediatric volumes for many hospitals admitting children.^9,19

Like CMS, we fit a random effects logistic model to obtain best linear unbiased predictions (BLUPs) of risk-adjusted hospital random effects. The model was implemented by using the xtmelogit procedure in Stata (Stata Corp, College Station, TX). Odds ratios (ORs), calculated as the antilogit transformation of the BLUP for each hospital, capture the deviation of the predicted rate for each hospital from its expected rate, based on the risk-adjustment variables included in the model. The model provides fairer estimates of performance for low-volume hospitals by differentially “shrinking” their BLUPs closer to the group mean. We based inferences about outlier status on 95% confidence intervals (CIs) for the ORs, calculated using model-based SEs. Standardized readmission rates R_s for each hospital were calculated using the formula R_s = R_e * OR/[1 + R_e * (OR – 1)], where R_e is the expected rate, and OR is the hospital-specific OR based on the BLUP; 95% confidence limits for R_s were obtained by using the same equation, replacing OR with either the upper or lower confidence limits. Outliers were defined as those hospitals for which the entire 95% CI for OR was >1 (“worse” performer) or <1 (“better” performer).

To assess for hospital variation overall, we used, for each condition, a likelihood-ratio test comparing the risk-adjustment model to a simpler model omitting the hospital-level random effects. This tests for between-hospital variation not accounted for by the risk adjustment variables, with P values <.05 indicating variation in risk-adjusted readmissions across the group of hospitals.

We performed several sensitivity analyses. First, we excluded hospitals with <25 discharges for the condition of interest over the measurement period, following CMS practice.²⁸ Because of the possibility that hospitals with extensive missing data artificially create more variation in the performance estimates, increasing noise in the overall data set that would make it harder to identify outliers, we also performed an analysis that excluded hospitals with >50% of visits missing unique identifiers. In addition, we treated hospital as a fixed rather than random effect, providing nonshrunken estimates of the ratio of observed to expected readmission rates. We based inference on 95% exact binomial CIs for the observed readmission rate, treating the expected rate as known. This analysis was restricted to hospitals with ≥25 discharges because fixed effects estimates can unfairly penalize low-volume hospitals. Finally, because there is no gold standard for risk-adjusting pediatric readmission rates, we reran the hierarchical analysis using the All-Patient Refined Diagnosis Related Group (APR-DRG) code for severity of illness as a case-mix adjuster, in place of the binary CCC indicator. We used data from Utah because no other state provided APR-DRG data from 2006 through 2009. APR-DRG severity classification is a 4 level categorical variable of degree of loss of function that uses proprietary methodology from 3M.³³

All analyses used Stata 12. The University of California at San Francisco Committee on Human Research considered this study exempt.

Results

Compared with other hospitals nationally, hospitals in our study (n = 958) were more frequently large- and medium-sized, urban, and had higher proportions of Medicaid patients (Table 1).

TABLE 1.

Hospital Characteristics of Study Hospitals Compared With Nonstudy Hospitals Nationally

Variables	Study Hospitalsa	Nonstudy Hospitalsa	Pb
Beds, n (%)
1–99 beds	258 (31.5)	3166 (56.2)
100–299 beds	354 (43.3)	1690 (30.0)	<.001
≥300 beds	206 (25.2)	778 (13.8)
Teaching hospital, n (%)	53 (6.5)	329 (5.8)	.47
Ownership, n (%)
Government (nonfederal)	162 (19.8)	1275 (23.5)
Not-for-profit (private)	445 (54.4)	2753 (50.8)	.048
Investor-owned	211 (25.8)	1390 (25.7)
Urban hospital, n (%)	636 (77.8)	3568 (63.3)	<.001
Freestanding children’s hospital, n (%)	14 (2.2)	90 (2.1)	.88
% Medicaid, mean	19.2	14.5	<.001

^aHospital characteristics measured using AHA 2008 data. This includes hospitals to which patients aged 1 to 21 were admitted with 1 of the disease categories we analyzed, for which AHA data were available (n = 877 hospitals, 92% of study hospitals; n = 5636 other hospitals in the AHA).

^bχ² used for categorical variables; t test used for continuous variables.

The numbers of eligible visits for each disease category are shown in Table 2. The median and interquartile ranges for hospitals admitting ≥1 patient for each condition were low, and the percent of hospitals admitting at least 25 patients within 2 years was <40% except for appendicitis (49.3%) and mood disorders (63.0%).

TABLE 2.

Volume of Patients With Unique Identifier Admitted for Top Pediatric Inpatient Principal Diagnoses

Diagnoses	No. of Eligible Visits	No. of Patients	Range of Hospital Volumes, Mediana (interquartile range)	Hospitals With ≥25 Inpatient Visits, n (%)
	Visit Level	Patient Level	Visit Level	Hospital Level
Mood disordersb (n = 376 hospitals)	72 229	55 564	59 (2–153)	237 (63.0%)
Dehydration (n = 649 hospitals)	14 189	13 441	7 (2–23)	158 (24.3)
Pneumonia (n = 768 hospitals)	33 300	20 071	10 (3–44)	265 (34.5)
Asthma (n = 741 hospitals)	44 925	38 554	11 (3–55)	278 (37.5)
Appendicitis (n = 743 hospitals)	37 123	36 595	24 (11–54)	366 (49.3)
Epilepsy (n = 638 hospitals)	19 506	15 570	5 (2–15)	110 (17.2)
Skin and soft tissue infections (n = 769 hospitals)	20 473	19 734	9 (3–24)	189 (24.6)

The CCS grouper was used to categorize the principal diagnoses for each visit.

^aBecause the distribution of hospital volumes is skewed by regionalization of admissions, the median is well below the mean for all conditions.

^bThe CCS category of “mood disorders” includes primarily depression and bipolar disorder ICD-9 diagnoses.

Thirty-day readmission rates were <5%, except for epilepsy (6.1%), dehydration (6.0%), and mood disorders (7.6%). Including ED visits in the outcome resulted in 1.0- to 2.3-fold increase in rates. Sixty-day revisit rates were somewhat higher (range: 8.5%–19.1%), but rates were still ≤15% for all conditions, except for mood disorders (19.1%, Table 3).

TABLE 3.

All-Cause Readmission and Revisit Rates Within 30 and 60 Days of Discharge

Diagnoses	30-d Readmission Rates	30-d Revisit Rates	60-d Readmission Rates	60-d Revisit Rates
Mood disordersa	7.6 (7.4–7.8)	11.0 (10.8–11.3)	13.5 (13.2–13.8)	19.1 (18.7–19.4)
Dehydration	6.0 (5.6–6.4)	9.5 (9.0–10.0)	9.4 (8.9–10.0)	15.4 (14.8–16.0)
Pneumonia	4.6 (4.4–4.9)	7.9 (7.6–8.2)	8.9 (8.6–9.2)	15.1 (14.7–15.5)
Asthma	3.0 (2.8–3.1)	6.3 (6.1–6.6)	6.3 (6.0–6.5)	13.6 (13.3–13.9)
Appendicitis	3.7 (3.5–3.9)	6.2 (5.9–6.4)	4.8 (4.6–5.1)	8.5 (8.2–8.8)
Epilepsy	6.1 (5.7–6.5)	6.2 (5.8–6.6)	11.7 (11.2–12.2)	11.8 (11.3–12.3)
Skin and soft tissue infections	2.8 (2.6–3.1)	6.4 (6.1–6.8)	4.5 (4.2–4.8)	10.8 (10.3–11.2)

A revisit is defined as a readmission or a return to the ED after discharge; 95% CIs are presented after each mean rate. The CCS grouper was used to categorize the principal diagnoses for each visit.

^aThe CCS category of “mood disorders” includes primarily depression and bipolar disorder ICD-9 diagnoses.

There was a wide range of raw hospital-level 30-day revisit rates (Table 4), with smaller ranges after risk-standardization. There were few outliers identified for any diagnostic group: for hospitals admitting any children with mood disorders, 15 hospitals (4.2%) were outliers, including 5 “worse” performers; for all other conditions there were ≤5 outliers (Table 4). The variation in hospital random intercepts was significantly (P < .05) greater than zero for all diagnostic categories except for pneumonia (P = .06). In sensitivity analyses including only hospitals with ≥25 admissions for the diagnostic category (Table 5, Fig 1), including only hospitals with ≥50% of visits with a unique identifier (Supplemental Table 8) and using APR-DRGs for risk adjustment instead of CCCs (data not shown), there was almost no change in the number of outlier hospitals identified. A fixed effects model identified more outliers; nonetheless, except for mood disorders (40 outliers [17%]) and epilepsy (14 outliers [12.8%], Supplemental Table 9) the proportion remained under 5% of hospitals.

TABLE 4.

Variation in Hospital Performance on 30-Day Raw and Risk-Standardized All-Cause Revisit Rates

Diagnoses	Range of Raw Rates	Range of Risk-Adjusted Rates	“Better” Hospitals, n (%)b	No Different From Average, n (%)b	“Worse” Hospitals, n (%)b
Mood disorders (n = 357 hospitals)a	0–100	6.3–15.9	10 (2.8)	342 (95.8)	5 (1.4)
Dehydration (n = 628 hospitals)	0–100	8.1–12.2	0 (0)	627 (100.0)	0 (0)
Pneumonia (n = 725 hospitals)	0–100	7.0–9.1	0 (0)	725 (100.0)	0 (0)
Asthma (n = 717 hospitals)	0–100	4.7–9.9	1 (0.14)	712 (99.3)	4 (0.56)
Appendicitis (n = 727 hospitals)	0–100	3.9–9.4	2 (0.28)	723 (99.5)	2 (0.28)
Epilepsy (n = 619 hospitals)	0–100	5.4–8.0	0 (0)	618 (99.8)	1 (0.2)
Skin and soft tissue infections (n = 722 hospitals)	0–100	4.7–9.9	0 (0)	721 (99.86)	1 (0.14)

The CCS grouper was used to categorize the principal diagnoses for each visit. The variation in hospital random intercepts was significantly (P < .05) greater than zero for all diagnostic categories except for pneumonia (P = .06).

^aThe CCS category of “mood disorders” includes primarily depression and bipolar disorder ICD-9 diagnoses.

^b“Better” and “worse” hospitals were hospitals for which the entire 95% CI around the OR of risk-standardized performance from the random effects model was either <1 or >1, respectively.

TABLE 5.

Identifying Hospital Performance Outliers on 30-Day Risk-Standardized All-Cause Revisit Rates Excluding Low-Volume Hospitals

Diagnoses (n = hospitals)	Excluded Hospitals (<25 visits), n (%)b	Range of Raw Rates	Range of Risk-Adjusted Rates	“Better” Hospitals, n (%)b	No Different From Average, n (%)b	“Worse” Hospitals, n (%)b
Mood disorders (n = 237)a	139 (37.0)	0–36	6.4–15.6	10 (4.2)	222 (93.7)	5 (2.1)
Dehydration (n = 158)	491 (75.7)	0–26.0	8.6–11.3	0 (0)	158 (100)	0 (0)
Pneumonia (n = 265)	503 (65.5)	0–31.3	7.2–9.1	0 (0)	265 (100)	0 (0)
Asthma (n = 278)	463 (62.5)	0–28.0	4.6–9.8	1 (0.4)	273 (98.2)	4 (1.4)
Appendicitis (n = 366)	377 (50.7)	0–21.3	4.0–9.6	2 (0.55)	362 (98.9)	2 (0.55)
Epilepsy (n = 110)	528 (82.8)	0–20.0	5.4–8.3	0 (0)	109 (99.1)	1 (0.9)
Skin and soft tissue infections (n = 189)	580 (75.4)	0–24	5.0–8.8	0 (0)	188 (99.5)	1 (0.53)

The CCS grouper was used to categorize the principal diagnoses for each visit. Low-volume hospitals were defined as hospitals with <25 discharges for the condition of interest over the measurement period.

^aThe CCS category of “mood disorders” includes primarily depression and bipolar disorder ICD9 diagnoses.

^bPercent of all hospitals admitting children for the condition.

FIGURE 1.

Risk-standardized hospital-level 30-day pediatric asthma readmission rates. Each point represents the risk-standardized readmission rate of a hospital with at least 25 pediatric asthma readmissions over 2 years (n = 278); the error bars represent the 95% CI. Outliers are hospitals with CIs above (poor performers, n = 4) or below (superior performers, n = 1) the group mean rate of 6.3% (horizontal line). Risk-adjustment variables were age, gender, and presence or absence of a CCC.^29,30

Discussion

In this first multistate, all-hospital study assessing variation in 30-day pediatric readmission and revisit rates for common pediatric conditions, admission volumes as well as readmission and revisit rates were low. For condition-specific revisit measures, there were few hospital performance outliers, a finding that persisted in sensitivity analyses.

The mean readmission rates we observed are consistent with previous studies^8,13,14,19 and with rates available from the HCUPnet tool.¹² Furthermore, the range of rates we observed is similar to previous studies. For instance, Brogan et al report risk-adjusted 14-day revisit rates for pneumonia from 1.5% to 4.4%.¹⁴ Thus, it is unlikely that the overall low number of hospital outliers we found reflects substantial underrecognition of readmissions or anomalies of the performance distribution in our study sample.

Although the hospitals in our sample had more beds than hospitals nationally, we detected no outliers for some diagnostic categories with low median hospital volume, such as dehydration and skin and soft tissue infections. Our proportions of outliers are, for all conditions except mood disorders, smaller than the 3% to 6% of outlier hospitals that CMS reports for adult readmissions. In addition, the clinical relevance of adult readmission measures is greater because the absolute numbers of hospitals and patients are higher.³⁴

A recent analysis by Berry et al of all-condition (hospital-wide) 30-day readmission rates at freestanding children’s hospitals found higher proportions of outliers, likely because of higher admission volumes created by pooling all conditions (median hospital volume = 6943).^9,19 Our study and Berry et al’s detected statistically significant overall variation across hospital random effects for condition-specific readmissions, implying that there may be variations in quality of care across all hospitals. However, we demonstrate that it is difficult to identify individual hospitals as performing statistically significantly different from average. This is likely due to low patient volumes for specific conditions and due to mean readmission rates mostly hovering around 5%, leaving little space below the mean (eg, 0% to <5%) for hospitals to be identified as having statistically significantly better performance. Hence, providers and policymakers engaged in quality improvement cannot tell, with 95% confidence, which hospitals with a 0% readmission rate have a rate that is significantly lower than 5%, to describe the care they deliver that may be preventing readmissions. Given these limitations, from both a policy and quality improvement perspective, our findings suggest that pediatric condition-specific readmission rates may not be useful performance measures, in particular, for general hospitals admitting children.

The methods we used, adapted from CMS national public reporting methods, are not the CMS methods for decreasing reimbursement rates to hospitals for adult readmissions, so our findings do not reflect the effect of a similar reimbursement policy in pediatrics. The reimbursement rate calculations reward or penalize all hospitals with a predicted/expected ratio of readmissions different from 1,⁷ so many adult hospitals are affected, even if their readmission rates are not statistically significantly different from expected using the CMS public reporting methods. However, CMS adult readmissions reporting is long-standing, with evidence that some adult readmissions are potentially preventable, giving greater credence to the decision to link payment and performance, whereas the study of pediatric readmissions is still a developing field.

One potential alternative approach that might improve pediatric readmissions measurement is to pool patients with similar conditions, which might lead to identification of more outliers by increasing the sample size at each hospital. If a combined readmissions measure for common diseases was adopted, potential interventions could focus on delivering guideline-recommended care across similar diseases. Previous studies implementing care pathways and improving compliance with guidelines have reduced pediatric readmissions for bronchiolitis³⁵ and asthma,³⁶ although other studies have not shown improvements,^8,37 and thus additional work remains to be done. A pooled measure of readmissions for hospitals that admit a large number of complex chronically ill children,³⁸ who are known to be at risk for frequent readmissions,^2,3 could focus improvements specifically on these children during their transitions of care. Focusing on improving transitions has been shown to be effective at reducing readmissions for adults with chronic illnesses.^39,40 If any of these pooled measures identify more statistically significant outliers than we found with condition-specific measures, and if studies show that focusing quality improvement on these pooled populations could reduce readmission rates, then CMS would have stronger evidence to justify basing payment on such measures.

There are several limitations to our analysis. First, our 6-state database is not nationally representative. There may be greater hospital performance variation in other states, and thus more condition-specific outliers. However, our sample does include hospitals serving 1 in 4 children in the United States.⁴¹

Second, we used administrative data, the only currently available source that can capture readmissions or revisits outside the index institution on a statewide basis. However, because patients <1 year of age usually lack a unique identifier, our approach may not be feasible for conditions affecting the youngest pediatric patients, such as bronchiolitis. It is unclear, however, whether a bronchiolitis readmission measure calculated at individual hospitals using medical record numbers (and hence excluding returns outside the index institution) would perform better in identifying outliers. A previous study at a large pediatric hospital had an annual average of 154 bronchiolitis admissions and only 7 readmissions.⁴² We also excluded visits lacking unique identifiers, which affected volume at some hospitals, although excluding hospitals with >50% of visits missing the unique identifier did not change our results. Administrative data also provide limited information for risk adjustment. However, this probably would not adversely affect our power to detect outliers, because more between-hospital variation is left unexplained.

In addition, the inclusion of nonpreventable readmissions in this analysis may have introduced noise that reduced our ability to detect hospitals with excess preventable readmissions. However, excluding any readmissions would also have reduced event rates even further, so it is not clear that an approach using readmission rates for only preventable readmissions would find more outliers. Nonetheless, a measure excluding nonpreventable readmissions may be easier to interpret and more reliably lead to quality improvements.

Conclusions

Using currently available data and nationally accepted public reporting methods, these analyses demonstrate that although there is statistically significant variation overall across hospitals on condition-specific pediatric 30-day revisit rates, few performance outliers can be identified, likely because of low patient volumes at most hospitals. Pooling across similar conditions, collecting better data for patient tracking, and focusing on children with CCCs in high-volume centers may have some potential to improve the utility of readmission rates as a performance measure.

Glossary

AHA

American Hospital Association

APR-DRG

All-Patient Refined Diagnosis Related Group

BLUP

best linear unbiased prediction

CCC

complex chronic condition

CCS

Clinical Classification Software

CI

confidence interval

CMS

Centers for Medicare and Medicaid Services

ED

emergency department

ICD-9

International Classification of Diseases, Ninth Revision

OR

odds ratio