Abstract
OBJECTIVE:
To understand factors associated with pediatric inpatient safety events, we test 2 hypotheses: (1) scarce resources (as measured by Medicaid burden) in safety-net hospitals relative to non–safety-net hospitals result in higher rates of safety events; and (2) higher levels of severity and more chronic conditions in patient populations lead to higher rates of safety events within hospital category and in children’s hospitals in comparison with non-children’s hospitals.
METHODS:
All nonnewborn pediatric hospital discharge records, which met criteria for potentially experiencing at least 1 pediatric quality indicator (PDI) event (using Agency for Healthcare Research and Quality’s 2009 Nationwide Inpatient Sample and PDI) and weighted to represent national level estimates, were analyzed for patterns of PDI events within and across hospital categories by using bivariate comparisons and multivariable logit models with robust SEs. The outcome measure “ANY PDI” captures the number of pediatric discharges at the hospital level with 1 or more PDI event.
RESULTS:
High Medicaid burden does not seem to be a factor in the likelihood of ANY PDI. Severity of illness (adjusted odds ratio high relative to low, 15.12) and presence of chronic conditions (adjusted odds ratio 1 relative to 0, 1.78; relative to 2 or more, 3.38) are the strongest predictors of ANY PDI events.
CONCLUSIONS:
Our findings suggest that the patient population served, rather than hospital category, best predicts measured quality, underscoring the need for robust risk adjustment when incentivizing quality or comparing hospitals. Thus, problems of quality may not be systemic across hospital categories.
KEY WORDS: quality indicator, AHRQ, complications
What’s Known on This Subject:
Previous studies suggest that hospitals under the greatest financial strain may be more prone to adverse events because they have limited resources to invest in quality and safety.
What This Study Adds:
The patient population served, rather than hospital category, best predicts measured quality, underscoring the need for robust risk adjustment when incentivizing quality or comparing hospitals. Thus, problems of quality may not be systemic across hospital categories.
At the turn of the 21st century, the Institute of Medicine focused national attention on preventable medical errors and the quality of medical care with the publication of To Err Is Human1 and Crossing the Quality Chasm.2 These reports resulted in a cascade of studies, discussions, and efforts to improve health care quality, generally, and to prevent avoidable medical errors, specifically.3–6 Central to these policy discussions was how to measure quality and how to transform Medicare, Medicaid, and private insurers from passive payers to active purchasers of high-quality health care.
In pursuit of measuring quality, the Agency for Healthcare Research and Quality (AHRQ) released a set of patient safety indicators (PSIs) in 2003. The PSIs can be constructed by using administrative data and allow hospitals to track and assess their quality improvement efforts. The PSIs were specifically developed to target events that likely represent preventable medical errors.7
Using the 2003 AHRQ PSI release, Miller and Zahn6 applied AHRQ’s PSI event-flagging software to the 2000 Healthcare Cost and Utilization Project KID Inpatient Database. In so doing, they found significant “incidence” of PSI events among the young and those on Medicaid. Despite these findings, Sedman and colleagues8 made the case for developing pediatric-specific measures. In 2006, AHRQ released a set of pediatric quality indicators (PDIs) that (1) reflect quality of pediatric inpatient care and (2) identify potentially avoidable hospitalizations among children.9 The PDIs are “intended to screen for problems that pediatric patients experience as a result of exposure to the health care system and that may be amenable to prevention by changes at the system or provider level.”
Scanlon et al10 assessed the validity of the PDI by applying the AHRQ PDI software to 76 children’s hospitals, calculating PDI rates. Then, identifying 1703 flagged discharge records for chart review by pediatric clinicians, they found that a subset of the PDIs were “reasonable screening tools” for institutional use, but less so for cross-hospital comparison in their “present form,” because true preventability of the complications is low, and because they do not identify complications present on admission (POA) or adjust for patient risk.
Although the PSIs and PDIs were available, when the Centers for Medicare and Medicaid Services began to use its purchasing power to enhance quality on October 1, 2008, it chose a set of “never events” as indicative of poor quality care for which Medicare would no longer provide reimbursement.11 Never events differ from the PSIs in that they may be more certainly attributed to medical error, although measured occurrence is less frequent. The Centers for Medicare and Medicaid Services now requires all state Medicaid programs to have similar nonpayment policies for never events.12
However, children’s complications often differ from those of adults. The number of pediatric inpatients at risk for many never events, which are defined for adult populations, is unlikely to be substantial. Savings from never events are estimated to be modest ($35 million over 5 years12), which suggests that never events as a quality improvement lever may be limited in the pediatric context. Consequently, we focus our attention on a subset of AHRQ PDI indicators in pursuit of pediatric inpatient quality metrics that can support quality improvement more broadly.
Because demonstration of quality performance is increasingly necessary, we seek to better understand the implications of factors associated with higher rates of PDI events. We empirically test (descriptively and in multivariable analyses) whether there are differences across hospital types in the occurrence of PDI events that could disproportionately, and perhaps ineffectively, penalize some types of hospitals under a nonpayment policy scenario, or a public reporting requirement. In particular, because previous studies suggest that hospitals under the greatest financial strain may be more prone to adverse events because they have limited resources to invest in quality and safety,13–16 we categorized hospitals by share of patients on Medicaid as a proxy for financial strain (safety net and non-safety net) and status as children’s hospitals (which also serve Medicaid-dependent populations, but may have additional sources of financial support). Moreover, we test for these differences while controlling for patient-level severity and the presence of chronic conditions, factors that may be correlated with these hospital characteristics.
Specifically, we hypothesize:
Scarce resources (as measured by Medicaid burden) in safety-net hospitals relative to non–safety-net hospitals result in higher rates of PDI events; and
Higher levels of severity and more chronic conditions in the patient population lead to higher rates of PDI events within each hospital category and in children’s hospitals compared with nonchildren’s hospitals.
Methods
Data Source
These analyses relied on AHRQ’s 2009 Nationwide Inpatient Sample (NIS) from the Healthcare Cost and Utilization Project. The 2009 NIS contains complete inpatient discharge information for 1050 hospitals in 44 states, ∼20% of US community hospitals. Discharge- and hospital-level weights allowed for national estimates of event frequency. Hospitals with <30 discharges were excluded from the sample, because these estimates were likely to be unstable.
Study Sample and Variables
The study sample consisted of the discharge records associated with all nonnewborn (ages 0 to <18) pediatric hospital stays which met AHRQ’s criteria17 for potentially experiencing at least 1 PDI event (the risk pool) as identified by a subset of AHRQ’s PDIs: accidental lacerations and puncture, decubitus ulcers, iatrogenic pneumothorax, selected infections due to medical care, and several postoperative events: hemorrhage or hematoma, respiratory failure, sepsis, and wound dehiscence. Several PDI events were excluded because of rarity (pediatric heart surgery mortality, transfusion reactions), or because they did not represent patient care (pediatric heart surgery volume). Additionally, illnesses or injuries identified as POA were not counted as events.
Assignment of PDI events was performed by using the September 2010 release (version 4.2) of AHRQ’s PDI program (SAS Institute, Cary, NC).18 The PDI algorithm identified all pediatric discharge records that met criteria for potentially experiencing a particular PDI event. These discharge records make up the set of inpatient stays at risk (the risk pool) for experiencing a particular PDI. Within the risk pool, an indicator variable was created that is equal to 1 if an event occurred, 0 otherwise.
“ANY PDI” is a constructed variable representing a nonweighted composite of experiencing at least 1 PDI event.17 Smith and colleagues16 noted that the composite measure captures the number of pediatric discharges at the hospital level with ≥1 PDI event. Thus, patients with multiple events within a hospital stay are counted as a single event that is intuited to be attributable to the same underlying cause(s) (patient complexity, hospital system failure, and/or provider failure), providing a sense of the extent of the patient population adversely affected by such events.
At the patient-discharge level, we controlled for age, gender, race/ethnicity, payer, severity, and number of chronic conditions. Illness severity is measured by an AHRQ-constructed variable that assigns a rating on an ordinal scale between 1 (minor loss of function) and 4 (extreme loss of function). The NIS includes an indicator variable identifying the number of chronic conditions associated with the patient at discharge. The NIS19 defines a chronic condition as one “that lasts 12 months or longer and meets one or both of the following tests: (a) it places limitations on self-care, independent living, and social interactions; (b) it results in the need for ongoing intervention with medical products, services, and special equipment.” Each diagnosis is classified as chronic or not by the Clinical Classification Software. If the discharge has multiple diagnoses that map to the same chronic condition, only 1 chronic condition is recognized. The variable we created categorizes the number of chronic conditions identified as 0, 1, or >1.
Not all hospitals and states record race/ethnicity in the NIS. AHRQ’s SAS program that creates indices for PDI events recodes missing race/ethnicity values as “Other.” We recategorized Asian or Pacific Islander and Native American children experiencing ANY PDI as other because of their small numbers.
Identification of safety-net versus non–safety-net hospitals followed the classification of Werner et al20 who relied on the Institute of Medicine definition “safety net hospitals deliver a significant level of health care …to uninsured, Medicaid, and other vulnerable patients.” They used the percentage of patients insured by Medicaid because it was available for all hospitals, and Medicaid is the main payer for low-income populations. We too created quartiles using the share of the entire (not just pediatric) discharge profile of the hospital that was Medicaid-dependent to assess the financial security of the hospital. Although Werner used the top quartile, Smith et al17 found a positive relationship between financial strain and adverse events in third-quartile Medicaid hospitals (suggesting that Disproportionate Share Program payments may mitigate the impact of financial strain on PDI events in the fourth quartile). Accordingly, we categorized hospitals as non–safety-net hospitals if the percentage of Medicaid-dependent discharges was less than or equal to the median level for all hospitals. Safety-net hospitals are the complement. Although the NIS does not uniquely identify children’s hospitals, we categorized hospitals where at least 75% of all patient discharges are associated with patients <19 years of age as children’s hospitals. One large hospital with 53% nonnewborn pediatric discharges was also included as a children’s hospital, because it was likely to have large pediatric specialties. (A hospital with 28% child discharges is the highest nonchildren’s hospital. Our main results described below do not change when this hospital is included as a children’s hospital, nor do they change when we identify hospitals with 4000 or more pediatric discharges.)
Statistical Analyses
To test our first hypothesis, we conducted bivariate comparisons of the unadjusted average rate per 1000 at risk for each event, and ANY PDI, across safety-net hospitals, children’s hospitals, and non–safety-net hospitals (the referent category), by using t tests. A descriptive analysis is presented for patient demographic, payer, and medical complexity characteristics, for all hospitals and across hospital categories. Differences across hospital categories are identified by P values associated with discharge-weighted Rao-Scott χ2 statistics.
To test our second hypothesis, several multivariable logistic regressions were conducted. First, we used the dichotomous constructed composite variable ANY PDI as the dependent variable and the full sample to estimate differences in the probability of ANY PDI event between hospital categories with the use of non–safety-net hospitals as the referent case. We adjusted these analyses for demographic, payer, and medical complexity characteristics. Next, we stratified logistic regression models by hospital category. The goal of these analyses was to identify specific demographic, payer, and medical complexity characteristics that may be differentially associated with ANY PDI event within each hospital category.
For each model, discharge-level weights and robust SEs were included to account for clustering of discharges within hospital. A sensitivity test using Werner et al’s21 alternative safety-net classification was conducted.
Results
PDI Event Rates by Hospital Category
Table 1 presents the unadjusted frequency rate per 1000 inpatient discharges at risk for each of the individual components of ANY PDI, overall and stratified by hospital category. Looking at “overall” discharge records for each PDI event, sepsis occurred most frequently, followed, in descending order, by respiratory failure, hematoma or hemorrhage, infections, ulcer, wound dehiscence, puncture or laceration, and iatrogenic pneumothorax. There are significant differences in unadjusted PDI event rates across hospital categories.
TABLE 1.
Unadjusted Rate of Any PDI Event per 1000 At-risk Pediatric Discharges by Hospital Category
| Overall | Hospital Category | |||
|---|---|---|---|---|
| Non–Safety-Net (Reference) | Safety-Net | Children’s Hospital | ||
| Share of Medicaid discharges, % | 0.00–0.74 | 0.00–0.15 | 0.15–0.69 | 0.27–0.74 |
| Average severity rating | 1.67 | 1.56 | 1.64a | 1.86a |
| Average number of chronic conditions | 1.17 | 1.05 | 1.07 | 1.60a |
| Puncture and laceration | ||||
| Per 1000 discharges | 0.81 | 0.72 | 0.80 | 0.91a |
| Risk pool (denominator) | 1 873 960 | 368 614 | 1 122 836 | 382 509 |
| Decubitus ulcer | ||||
| Per 1000 discharges | 0.96 | 0.56 | 0.83 | 1.45a |
| Risk pool (denominator) | 290 904 | 46 433 | 167 776 | 76 695 |
| Iatrogenic pneumothorax | ||||
| Per 1000 discharges | 0.27 | 0.11 | 0.31a | 0.31a |
| Risk pool (denominator) | 1 806 746 | 360 725 | 1 082 863 | 363 158 |
| Hemorrhage/hematoma | ||||
| Per 1000 discharges | 2.29 | 1.50 | 3.61a | 0.99 |
| Risk pool (denominator) | 102 883 | 17 353 | 47 977 | 37 553 |
| Respiratory failure | ||||
| Per 1000 discharges | 10.29 | 7.40 | 12.64a | 8.87 |
| Risk pool (denominator) | 86 124 | 16 078 | 38 594 | 31 451 |
| Sepsis | ||||
| Per 1000 discharges | 20.34 | 26.60 | 22.17 | 16.63a |
| Risk pool (denominator) | 82 097 | 10 341 | 41 991 | 29 765 |
| Wound dehiscence | ||||
| Per 1000 discharges | 0.83 | 1.08 | 0.65 | 1.14 |
| Risk pool (denominator) | 59 202 | 9228 | 36 834 | 13 140 |
| Infections | ||||
| Per 1000 discharges | 1.22 | 0.26 | 1.15a | 2.30a |
| Risk pool (denominator) | 1 387 966 | 261 405 | 836 983 | 289 578 |
| Any PDI event | ||||
| Per 1000 discharges | 3.41 | 2.03 | 3.30a | 5.06a |
| Risk pool (denominator) | 1 887 452 | 371 102 | 1 127 878 | 387 563 |
Denominators represent weighted discharges. Source: Healthcare Cost and Utilization Project 2009 Nationwide Inpatient Sample.
Significantly different (5%, 2-tail) from reference group non–safety-net hospitals, and each group has >25 weighted PDI events.
Compared with pediatric discharges from non–safety-net hospitals, those discharged from safety-net hospitals are associated with higher rates of ANY PDI event, iatrogenic pneumothorax, hemorrhage or hematoma, respiratory failure, and infections. Compared with pediatric discharges from non–safety-net hospitals, those from children’s hospitals are associated with higher rates of ANY PDI event, punctures or lacerations, decubitus ulcers, iatrogenic pneumothorax, and infections, and lower rates of sepsis.
Although safety-net hospitals have higher rates of ANY PDI and individual PDI events as we hypothesized, they, along with children’s hospitals, treat sicker children. Discharges from safety-net hospitals have higher average severity and more chronic conditions, and those from children’s hospitals even higher average severity and more chronic conditions, than non–safety-net hospitals.
ANY PDI and Patient Characteristics by Hospital Category
Based on 1281 unweighted pediatric discharges with ANY PDI event in our 2009 sample, we estimated that 6436 discharges were associated with 1 or more PDI event nationally.
Table 2 provides a categorical distribution summary of patient discharge characteristics, overall and by hospital categories. For example, in safety-net and children’s hospitals, ANY PDI is more prevalent in children under age 2 in comparison with non–safety-net hospitals. The difference in insurance payer is inherent in the definition of hospital category, but note that the unadjusted rate of ANY PDI is higher for Medicaid and privately insured children in safety-net and children’s hospitals than in non–safety-net hospitals. Among patient-discharge characteristics, the largest differences in ANY PDI rates occur between patients with low and high severity, and between patients with none and 2 or more chronic conditions. Patients with higher levels of severity and more chronic conditions were associated with higher percentages of ANY PDI events across all hospital categories.
TABLE 2.
Hospital Summary and Any PDI by Pediatric Patient Discharge Characteristics and Hospital Category (Category Percentages of All Discharges, Any PDI, and Rate per 1000)
| Overall | Hospital Category | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Non–Safety-Net | Safety-Net | Children’s | |||||||||||||
| No. of hospitals | 962 | 450 | 501 | 11 | |||||||||||
| No. of discharges (unweighted) | 374 946 | 72 092 | 228 083 | 74 771 | |||||||||||
| No. of discharges (weighted) | 1 887 452 | 371 102 | 1 128 787 | 387 563 | |||||||||||
| Total weighted discharges with ANY PDI event | 6436 | 755 | 3720 | 1961 | |||||||||||
| Rate per 1000 of weighted discharges with ANY PDI event | 3.4 | 2.0 | 3.3 | 5.1 | |||||||||||
| Patient Discharge Characteristic | Category | Overall | Non–Safety-Net | Safety-Net | Children’s | P value* | |||||||||
| Category Percent | Rate per 1000 | Category Percent | Rate per 1000 | Category Percent | Rate per 1000 | Category Percent | Rate per 1000 | ||||||||
| All Discharges | Any PDI | All Discharges | Any PDI | All Discharges | Any PDI | All Discharges | Any PDI | ||||||||
| Age, y | <1 | 27.5 | 28.2 | 3.5 | 24.5 | 21.7 | 1.8 | 28.3 | 27.9 | 3.2 | 27.8 | 31.4 | 5.7 | .005 | |
| 1–2 | 18.0 | 15.4 | 2.9 | 16.4 | 8.5 | 1.0 | 18.5 | 15.5 | 2.8 | 18.2 | 17.9 | 5.0 | |||
| 3–5 | 10.8 | 10.7 | 3.4 | 10.8 | 17.2 | 3.2 | 11.0 | 9.8 | 2.9 | 10.4 | 10.1 | 4.9 | |||
| 6–12 | 21.7 | 21.2 | 3.3 | 21.0 | 20.9 | 2.0 | 20.6 | 21.3 | 3.4 | 25.6 | 21.0 | 4.1 | |||
| 13–17 | 22.0 | 24.5 | 3.8 | 27.2 | 31.7 | 2.4 | 21.6 | 25.6 | 3.9 | 18.0 | 19.5 | 5.5 | |||
| Gender | Male | 54.6 | 55.4 | 3.5 | 53.8 | 57.2 | 2.2 | 54.9 | 56.6 | 3.4 | 54.4 | 52.6 | 4.9 | .438 | |
| Female | 45.4 | 44.6 | 3.3 | 46.1 | 42.8 | 1.9 | 45.1 | 43.4 | 3.2 | 45.6 | 47.4 | 5.3 | |||
| Race/ethnicity | White | 43.7 | 46.0 | 3.6 | 55.3 | 49.3 | 1.8 | 43.0 | 51.5 | 3.9 | 34.9 | 34.3 | 5.0 | .003 | |
| Black | 12.0 | 11.1 | 3.2 | 9.2 | 4.8 | 1.1 | 13.3 | 14.7 | 3.6 | 10.9 | 6.8 | 3.1 | |||
| Hispanic | 20.3 | 20.3 | 3.4 | 8.7 | 3.8 | 0.9 | 21.3 | 17.5 | 2.7 | 28.8 | 32.2 | 5.6 | |||
| Othera | 23.9 | 22.6 | 3.2 | 26.7 | 42.1 | 3.2 | 22.5 | 16.4 | 2.4 | 25.4 | 26.8 | 5.3 | |||
| Insurance payer | Medicaid | 50.1 | 51.5 | 3.5 | 27.5 | 26.4 | 2.0 | 54.2 | 51.4 | 3.1 | 59.7 | 61.4 | 5.2 | <.001 | |
| Private | 41.9 | 40.2 | 3.3 | 64.7 | 59.8 | 1.9 | 36.4 | 39.2 | 3.6 | 36.0 | 34.4 | 4.8 | |||
| Uninsured | 4.0 | 2.3 | 2.0 | 2.9 | 4.7 | 3.3 | 5.1 | 2.6 | 1.7 | 1.8 | 0.8 | 2.2 | |||
| Other | 4.1 | 6.0 | 5.1 | 4.9 | 9.0 | 3.8 | 4.3 | 6.8 | 5.2 | 2.6 | 3.4 | 6.7 | |||
| Severityb | Low | 86.0 | 20.5 | 0.8 | 90.5 | 33.6 | 0.8 | 87.3 | 22.0 | 0.8 | 77.7 | 12.7 | 0.8 | .001 | |
| High | 14.0 | 79.5 | 11.94 | 9.5 | 66.4 | 11.43 | 12.7 | 78.0 | 12.03 | 22.3 | 87.3 | 11.98 | |||
| No. of chronic conditionsc | 0 | 43.1 | 9.6 | 0.8 | 46.6 | 15.2 | 0.7 | 45.9 | 9.8 | 0.7 | 31.7 | 6.9 | 1.1 | .028 | |
| 1 | 28.5 | 15.1 | 1.8 | 27.8 | 21.4 | 1.6 | 28.4 | 16.5 | 1.9 | 29.3 | 10.1 | 1.7 | |||
| 2 or more | 28.4 | 75.3 | 9.0 | 25.6 | 63.4 | 5.0 | 25.8 | 73.7 | 9.4 | 39.1 | 83.0 | 11.07 | |||
Any PDI categorical cell numbers within a patient discharge characteristic may sum to total weighted discharges with any PDI owing to missing values or rounding of estimated weighted discharges. The association between patient discharge characteristics and hospital categories are each significant at the <.001 level. Source: Healthcare Cost and Utilization Project 2009 Nationwide Inpatient Sample.
Missing race/ethnicity values are also contained in this category.
Low, APRDRG_Severity rating of 1 (minor loss of function) or 2 (moderate loss of function); High, APRDRG_Severity rating of 3 (major loss of function) or 4 (extreme loss of function).
The weighted (to national level) means for the number of chronic conditions by hospital category are: Non–Safety-Net = 1.06; Safety-Net = 1.08; and Children’s = 1.58.
A P value <.05 indicates a significant difference in the incidence of ANY PDI between patient discharge characteristic category and hospital categories.
Risk-Adjusting the Probability of ANY PDI Event by Discharge Characteristics and Hospital Category
Results from a logistic regression model of the probability that a discharged patient experienced ANY PDI event demonstrated that, in comparison with patient discharges at non–safety-net hospitals, those from safety-net or children’s hospitals are not more likely to be associated with ANY PDI event as the unadjusted results previously suggested (Table 3). The only clear characteristics associated with the probability of ANY PDI event are severity and the number of chronic conditions. Indeed, when these 2 variables were removed from the regression, the c-statistic value dropped to 0.54 (not shown). Chronic conditions and severity alone account for approximately one-third of the ability to discriminate between the prediction of ANY PDI event or not. Unlike other authors,6,14,16 who were unable to account for POA, we did not find the Medicaid status of the discharge significant.
TABLE 3.
All Hospitals Probability of Any PDI Event by Patient-Discharge Characteristics (aOR, 95% Confidence Interval)
| Covariates | Reference | Category | aOR | 95% CI | |
|---|---|---|---|---|---|
| Hospital category | Non–safety-net | Safety-net | 1.34 | 0.93 | 1.93 |
| Children’s Hospital | 1.22 | 0.79 | 1.90 | ||
| Age, y | 13–17 | <1 | 1.01 | 0.84 | 1.22 |
| 1–2 | 1.05 | 0.87 | 1.32 | ||
| 3–5 | 1.04 | 0.81 | 1.32 | ||
| 6–12 | 0.98 | 0.83 | 1.16 | ||
| Gender | Female | Male | 1.02 | 0.90 | 1.15 |
| Race/ethnicity | White | Black | 0.88 | 0.61 | 1.27 |
| Hispanic | 0.93 | 0.75 | 1.16 | ||
| Other/missinga | 0.91 | 0.72 | 1.15 | ||
| Insurance payer | Private | Medicaid | 0.95 | 0.82 | 1.10 |
| Uninsured | 1.00 | 0.71 | 1.41 | ||
| Other | 1.12 | 1.00 | 1.25 | ||
| Severityb | Low | High | 15.12 | 12.67 | 18.04 |
| No. of chronic conditionsc | 0 | 1 | 1.78 | 1.39 | 2.28 |
| 2 or more | 3.38 | 2.65 | 4.30 | ||
| c = 0.84 | |||||
CI, confidence interval. Source: Healthcare Cost and Utilization Project 2009 Nationwide Inpatient Sample.
Missing race/ethnicity values are also contained in this category.
Low, APRDRG_Severity rating of 1 (minor loss of function, includes cases with no comorbidity or complications) or 2 (moderate loss of function); High, APRDRG_Severity rating of 3 (major loss of function) or 4 (extreme loss of function).
The weighted (to national level) means for the number of chronic conditions by hospital category are: Non–Safety-Net = 1.06, Safety Net = 1.08, and Children’s = 1.58.
Hypothesis 2 (hospitals that serve more severely ill patient populations with more chronic conditions on average will have higher PDI rates) is supported by 2 of our findings: (1) the unadjusted PDI rates and (2) differences in ANY PDI rates across hospital categories become insignificant once severity and chronicity are taken into account.
To determine if patient characteristics were associated with ANY PDI events in different hospital categories, we present results stratified by hospital category in Table 4. In non–safety-net hospitals, patient characteristics mattered. Children ages 3 to 5 years were almost twice as likely (adjusted odds ratio [aOR] = 1.71) to have ANY PDI event compared with children ages 13 to 17; compared with white children, Hispanic children were half as likely (aOR = 0.46) to have ANY PDI events; compared with those with private insurance, the uninsured were more than twice as likely (aOR = 2.21) to have ANY PDI event. The only remaining characteristics associated with ANY PDI events for discharges in safety-net and children’s hospitals were severity and number of chronic conditions.
TABLE 4.
Stratified Hospital Category Probability of Any PDI Event by Patient-Discharge Characteristics (aOR, 95% Confidence Interval)
| Covariates | Reference | Category | Stratified Hospital Category Model | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Non–Safety-Net | Safety-Net | Children’s Hospital | |||||||||
| aOR | 95% CI | aOR | 95% CI | aOR | 95% CI | ||||||
| Age, y | 13–17 | <1 | 0.94 | 0.63 | 1.41 | 1.01 | 0.79 | 1.30 | 1.04 | 0.70 | 1.54 |
| 1–2 | 0.72 | 0.43 | 1.21 | 1.07 | 0.82 | 1.41 | 1.12 | 0.81 | 1.53 | ||
| 3–5 | 1.71 | 1.08 | 2.72 | 0.97 | 0.66 | 1.41 | 0.92 | 0.71 | 1.18 | ||
| 6–12 | 0.98 | 0.55 | 1.76 | 1.04 | 0.82 | 1.31 | 0.89 | 0.73 | 1.09 | ||
| Gender | Female | Male | 1.15 | 0.79 | 1.66 | 1.06 | 0.89 | 1.27 | 0.91 | 0.79 | 1.06 |
| Race/ethnicity | White | Black | 0.60 | 0.22 | 1.63 | 0.99 | 0.67 | 1.47 | 0.65 | 0.25 | 1.74 |
| Hispanic | 0.46 | 0.22 | 0.93 | 0.90 | 0.73 | 1.13 | 1.04 | 0.62 | 1.75 | ||
| Other/missinga | 1.17 | 0.64 | 2.13 | 0.74 | 0.57 | 0.96 | 1.10 | 0.66 | 1.83 | ||
| Insurance payer | Private | Medicaid | 0.95 | 0.74 | 1.22 | 0.91 | 0.73 | 1.13 | 1.02 | 0.82 | 1.29 |
| Uninsured | 2.21 | 1.15 | 4.24 | 0.86 | 0.55 | 1.35 | 0.86 | 0.37 | 1.99 | ||
| Other | 1.73 | 0.81 | 3.67 | 1.02 | 0.73 | 1.44 | 0.99 | 0.59 | 1.66 | ||
| Severityb | Low | High | 13.43 | 9.83 | 18.34 | 15.11 | 11.96 | 19.08 | 16.08 | 10.27 | 25.16 |
| No. of chronic conditionsc | 0 | 1 | 1.84 | 0.95 | 3.58 | 2.02 | 1.56 | 2.64 | 1.17 | 0.60 | 2.26 |
| 2 or more | 2.48 | 1.26 | 4.87 | 3.84 | 2.86 | 5.15 | 2.64 | 1.54 | 4.50 | ||
| c = 0.81 | c = 0.84 | c = 0.85 | |||||||||
CI, confidence interval. Source: Healthcare Cost and Utilization Project 2009 National Inpatient Sample.
Missing race/ethnicity values are also contained in this category.
Low, APRDRG_Severity rating of 1 (minor loss of function, includes cases with no comorbidity or complications) or 2 (moderate loss of function); High, APRDRG_Severity rating of 3 (major loss of function) or 4 (extreme loss of function).
The weighted (to national level) means for the number of chronic conditions by hospital category are as follows: Non–Safety-Net = 1.06; Safety Net = 1.08; and Children’s =1.58.
Under Werner’s20 safety-net classification, the results (not presented) were maintained; severity and chronic condition, not hospital category, were the only significant predictors of the probability of ANY PDI. Support for Smith et al’s16 finding that third-quartile hospitals based on Medicaid burden performed worse than fourth-quartile hospitals was found. Shifting the third-quartile hospitals from safety-net to non–safety-net status reduced the difference in probability of ANY PDI between safety-net and non–safety-net hospitals (from aOR = 1.34 to 1.13), and between non–safety-net and children’s hospitals (from aOR = 1.22 to 1.01).
Discussion
Both within and between hospital categories, discharges with higher severity and/or 2 or more chronic conditions (>1 in safety-net hospitals) are more likely to experience ANY PDI event. Safety-net hospitals, without adjusting for severity and number of chronic conditions, have higher rates of PDI events than non–safety-net hospitals. Children’s hospitals have even higher rates. However, once adjusted for severity and number of chronic conditions, significant differences across hospital categories disappear. Within hospital categories, variables most predictive of ANY PDI event are severity and number of chronic conditions. Overall our findings suggest:
High Medicaid burden does not seem to be a factor in the rate of ANY PDI.
Severity and chronic conditions are the strongest predictors of ANY PDI events both within and across hospital categories, rendering payer status (Medicaid), and other patient characteristics statistically insignificant. Because safety-net hospitals serve patients with higher severity and more chronic conditions, on average, than non–safety-net hospitals, as do children’s hospitals, our results indicate a need for robust risk adjustment when incentivizing or comparing quality.
PDI events do not appear to be a systemic problem within a particular set of hospitals based on financial strain or medical complexity and do not differ across categories once adjusted for severity and number of chronic conditions.
Findings with respect to the quality performance of safety-net hospitals have been mixed.14,20–22 Our findings suggest that it is not the hospital category but the patient population served that best predicts measured quality. Thus, problems of quality may not be systemic across hospital categories. A next step is to assess variation within hospital categories to identify which characteristics of safety-net and children’s hospitals are associated with poor outcomes, and which are, in fact, associated with achieving high-quality care (positive deviance) despite serving less healthy patient populations. Identifying hospital-level procedures and practices associated with the best performance within each hospital category will enable policymakers to incorporate such findings into efforts to achieve the twin goals of reducing error and providing high-quality care to all patients.
Our study has limitations. As Scanlon et al10 noted, not all PDI events are in fact preventable, nor are all coded PDI events true events. Rosen et al23 study the positive predictive value of software-flagged PSIs in the VA against chart review. POA and coding issues, in particular, miscoding, lack of coding specificity, differences in hospital coding practices, and poor documentation, were the main reasons for cases to be misflagged or missed. Adjusting for POA and identifying the number of chronic conditions and severity improve these metrics substantially, but coding limitations remain a concern. In addition, our sample of children’s hospitals is suggestive rather than representative, because it does not represent all children’s hospitals in the nation.
Glossary
- AHRQ
Agency for Healthcare Research and Quality
- aOR
adjusted odds ratio
- NIS
Nationwide Inpatient Sample
- PDI
pediatric quality indicator
- POA
present on admission
- PSI
patient safety indicator
Footnotes
FINANCIAL DISCLOSURE: Drs Smith, Fairbrother, and Simpson received salary support associated with this project; and Drs Dynan and Goudie have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: Supported in part by grant R40 MC 11279 from the Maternal and Child Health Bureau (Title V, Social Security Act), Health Resources and Services Administration, Department of Health and Human Services. The grant provided salary support to Drs Simpson, Fairbrother, and Smith.
References
- 1.Institute of Medicine To Err Is Human: Building a Safer Health System. Washington, DC: National Academies Press; 1999 [PubMed] [Google Scholar]
- 2.Institute of Medicine Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academies Press; 2001 [PubMed] [Google Scholar]
- 3.Mangione-Smith R, DeCristofaro AH, Setodji CM, et al. The quality of ambulatory care delivered to children in the United States. N Engl J Med. 2007;357(15):1515–1523 [DOI] [PubMed] [Google Scholar]
- 4.McGlynn EA, Asch SM, Adams J, et al. The quality of health care delivered to adults in the United States. N Engl J Med. 2003;348(26):2635–2645 [DOI] [PubMed] [Google Scholar]
- 5.Schuster MA, McGlynn EA, Brook RH. How good is the quality of health care in the United States? 1998. Milbank Q. 2005;83(4):843–895 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Miller MR, Zhan C. Pediatric patient safety in hospitals: a national picture in 2000. Pediatrics. 2004;113(6):1741–1746 [DOI] [PubMed] [Google Scholar]
- 7.Kronman MP, Hall M, Slonim A, Shah SS. Charges and lengths of stay attributable to adverse patient-care events using pediatric-specific quality indicators: a multicenter study of freestanding children’s hospitals. Pediatrics 2008;121(6). Available at: www.pediatrics.org/cgi/content/full/121/6/e1653 [DOI] [PubMed] [Google Scholar]
- 8.Sedman A, Harris JM, II, Schulz K, et al. Relevance of the Agency for Healthcare Research and Quality Patient Safety Indicators for children’s hospitals. Pediatrics. 2005;115(1):135–145 [DOI] [PubMed] [Google Scholar]
- 9.Pediatric Quality Indicators Overview. Available at: www.qualityindicators.ahrq.gov/modules/pdi_overview.aspx. Accessed August 18, 2012
- 10.Scanlon MC, Harris II M, Levy, Sedman A. Evaluation of the Agency for Healthcare Research and Quality Pediatric Quality. Pediatrics 2008;121(6). Available at: www.pediatrics.org/cgi/content/full/121/6/e1723 [DOI] [PubMed] [Google Scholar]
- 11.Centers for Medicare and Medicaid Services (CMS), HHS . Medicare program; changes to the hospital inpatient prospective payment systems and fiscal year 2008 rates. Fed Regist. 2007;72(162):47129–48175 [PubMed] [Google Scholar]
- 12.Galewitz, P. Medicaid to Stop Paying for Hospital Mistakes. Kaiser Health News. Available at: www.kaiserhealthnews.org/stories/2011/june/01/medicaid-hospital-medical-error-payment-short-take.aspx?referrer=search. Accessed June 1, 2011 [Google Scholar]
- 13.Shen YC. The effect of financial pressure on the quality of care in hospitals. J Health Econ. 2003;22(2):243–269 [DOI] [PubMed] [Google Scholar]
- 14.Encinosa WE, Bernard DM. Hospital finances and patient safety outcomes. Inquiry. 2005;42(1):60–72 [DOI] [PubMed] [Google Scholar]
- 15.Bazzoli GJ, Clement JP, Lindrooth RC, et al. Hospital financial condition and operational decisions related to the quality of hospital care. Med Care Res Rev. 2007;64(2):148–168 [DOI] [PubMed] [Google Scholar]
- 16.Smith RB, Dynan L, Fairbrother G, Chabi G, Simpson L. Medicaid, hospital financial stress, and the incidence of adverse medical events for children. Health Serv Res. 2012;47(4):1621–1641 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Pediatric Quality Indicators Technical Specifications – Version 4.4 March 2012. Available at: www.qualityindicators.ahrq.gov/modules/PDI_TechSpec.aspx. Accessed April 2012
- 18.AHRQ. AHRQ Quality Indicators Software Download. Rockville, MD: Agency for Healthcare Research and Quality; July 2010. Available at: www.qualityindicators.ahrq.gov/software.htm. Accessed March 2, 2011
- 19.Healthcare Cost and Utilization Project. NIS description of data elements. Rockville, MD: Agency for Healthcare Research and Quality; 2008. Available at: www.hcup-us.ahrq.gov/db/vars/nchronic/nisnote.jsp. Accessed April 2012
- 20.Werner RM, Goldman LE, Dudley RA. Comparison of change in quality of care between safety-net and non-safety-net hospitals. JAMA. 2008;299(18):2180–2187 [DOI] [PubMed] [Google Scholar]
- 21.Marshall L, Harbin V, Hooker J, Oswald J, Cummings L. Safety net hospital performance on National Quality of Care process measures. J Healthc Qual. 2012;34(2):21–31 [DOI] [PubMed] [Google Scholar]
- 22.McHugh M, Kang R, Hasnain-Wynia, R. Understanding the safety net: inpatient quality of care varies based on how one defines safety-net hospitals. Med Care Res Rev. 2009;66(5):590–605 [DOI] [PubMed] [Google Scholar]
- 23.Rosen AK, Kamal MF, Itani MF, et al. Validating the patient safety indicators in the Veteran’s Health Administration: do they accurately identify true safety events? Med Care 2012;50(1):74–85 [DOI] [PubMed] [Google Scholar]