Abstract
background:
Transitions between sites of care are inherent to all hospitalizations, yet we lack pediatric-specific transitions-of-care quality measures. We describe the development and validation of new transitions-of-care quality measures obtained from medical record data.
methods:
After an evidence review, a multistakeholder panel prioritized quality measures by using the RAND/University of California, Los Angeles modified Delphi method. Three measures were endorsed, operationalized, and field-tested at 3 children’s hospitals and 2 community hospitals: quality of hospital-to-home transition record content, timeliness of discharge communication between inpatient and outpatient providers, and ICU-to-floor transition note quality. Summary scores were calculated on a scale from 0 to 100; higher scores indicated better quality. We examined between-hospital variation in scores, associations of hospital-to-home transition quality scores with readmission and emergency department return visit rates, and associations of ICU-to-floor transition quality scores with ICU readmission and length of stay.
results:
A total of 927 charts from 5 hospitals were reviewed. Mean quality scores were 65.5 (SD 18.1) for the hospital-to-home transition record measure, 33.3 (SD 47.1) for the discharge communication measure, and 64.9 (SD 47.1) for the ICU-to-floor transition measure. The mean adjusted hospital-to-home transition summary score was 61.2 (SD 17.1), with significant variation in scores between hospitals (P < .001). Hospital-to-home transition quality scores were not associated with readmissions or emergency department return visits. ICU-to-floor transition note quality scores were not associated with ICU readmissions or hospital length of stay.
conclusions:
These quality measures were feasible to implement in diverse settings and varied across hospitals. The development of these measures is an important step toward standardized evaluation of the quality of pediatric transitional care.
Hospital care in the inpatient setting accounts for >40% of all pediatric health care expenditures in the United States, representing ~2 million pediatric hospitalizations annually.1 Inherent to each of these hospitalizations are transitions between sites of care, which may include transitions within the hospital, such as between the ICU and inpatient unit, and from the hospital to home or another health care facility at the time of discharge. Several studies have documented risks associated with these transitions, including medication errors and communication failures, all of which can adversely affect patients and their families.2–7 As a result, the Centers for Medicare & Medicaid Services and the Agency for Healthcare Research and Quality have selected transitions between sites of care as a priority area for the development of pediatric quality measures.
The Children’s Health Insurance Program Reauthorization Act of 2009 Pediatric Quality Measures Program provided funding to 7 Centers of Excellence to develop and validate pediatric-specific quality measures.8 The Center of Excellence on Quality of Care Measures for Children With Complex Needs (COE4CCN) was tasked with development and validation of quality measures for pediatric transitions of care, applicable to all children regardless of medical complexity. The objective of this article is to describe the development and validation of these new pediatric transitions of care quality measures, obtained from medical record data.
METHODS
Development and validation of these measures involved review of the pediatric and adult transitions of care literature, with a focus on studies examining relationships between transitional care processes and improved health and health care outcomes5; development of draft quality measures based on this evidence review, with evaluation of these measures by a multistakeholder Delphi panel; operationalization of measures into detailed measure specifications that could be applied to medical record data; medical record abstractions at 5 hospitals; calculation and interpretation of measure scores; and validation of medical record measures via outcomes available in hospital administrative data. These steps are detailed below. All study procedures were approved by the participating institutions’ institutional review boards.
Measure Development
Detailed methods regarding our evidence review process and multistakeholder Delphi panel are provided in “Methods 1” of the Supplemental Information. In brief, after targeted literature review, COE4CCN research staff drafted a set of 5 potential medical record-based quality measures to assess pediatric transitions of care (Table 1). These measures were evaluated by a panel of 9 experts nominated by professional bodies that conduct activities related to pediatric transitions of care, with participants representing emergency medicine, rehabilitation medicine, hospital medicine, general pediatrics, case management, complex care, state Medicaid agencies, and the family’s perspective. By using the RAND/University of California, Los Angeles (UCLA) modified Delphi method, panelists independently scored each potential measure on a 9-point Likert scale, rating their face validity and feasibility.9 After the initial independent scoring, panelists received a synopsis of responses, including the score distribution and their own score for each measure. Panelists met as a group to discuss controversial measures, after which they independently rescored the face validity and feasibility of each measure.9 Four of the 5 measures were endorsed by the panel (Table 1).
TABLE 1.
Brief Description | Full Measure | Recommended for Use (Rationale for Exclusion) |
---|---|---|
Documentation of postdischarge appointments | Children discharged from the hospital should have the date and time of all follow-up appointments documented in their hospital medical record before discharge. | No (not endorsed by Delphi panel) |
Transition needs assessment | Children admitted to the hospital for >24 h should have an assessment of their transition needs documented in their hospital medical record before discharge. |
No (measure could not be operationalized reliably from the medical record and was not included in the medical record abstraction) |
Hospital-to-home transition record quality | Children admitted to the hospital should have documentation in their hospital medical record of a transition record that contained all of the following elements: a. Admission and discharge diagnoses b. Medication list at discharge c. List of follow-up appointments d. 24/7 telephone contact number if problems arise e. Number to call for assistance getting needed appointments f. Admit and discharge dates g. Pending test results h. Follow-up tests that need to be completed i. Immunizations given |
Yes |
Timely discharge communication between providers | Children discharged from the hospital should have documentation in their hospital medical record that the receiving outpatient follow-up provider was contacted (by phone, fax, or e-mail) by a hospital provider between 24 h before and 48 h after the patient’s discharge. | Yes |
ICU-to-floor transition note quality | Hospitalized children transferred between the ICU and general inpatient floor should have the following elements documented in a transition note: a. Current problem list b. Treatment plan for each problem |
Yes |
Measure Operationalization and Field Testing
These measures were transformed into a detailed data abstraction tool to ensure efficient and reliable data collection, and the feasibility of data collection was assessed with a random sample of medical records from the 3 children’s hospitals. At this stage, 1 of the 4 measures endorsed by the panel, documentation of a transition needs assessment, could not be reliably operationalized from medical records because of the very broad scope of the measure content (Table 1). The specifications for determining measure eligibility and scoring are detailed in the Supplemental Information; the data abstraction tool can be accessed online.10
Eligible patients were 2 months to 18 years of age and hospitalized during January to December 2013 at the children’s hospitals and January 2012 to December 2013 at the community hospitals. The 2-year time period was used for the community hospitals because of the lower pediatric patient volumes at these sites. The children’s hospitals were in different geographic regions of the country; the 2 community hospitals were located in the same state but were operationally independent. Both community hospitals had pediatric inpatient units with care provided by pediatric hospitalists but did not have PICUs. Thus the ICU-to-floor transition measure was not assessed at these sites. Patients who did not experience either an ICU-to-floor transition or a hospital-to-home transition, and patients who died in the hospital, were excluded. Patients were randomly selected with a target of 200 cases per children’s hospital and 150 cases per community hospital to ensure adequate precision in measure assessment within each hospital. For patients with >1 admission during the study period, only the first hospitalization was included. At the children’s hospitals, we used a stratified random sampling approach to include 25% who had an ICU stay to allow testing of the ICU-to-floor transition quality measure. Only patients who had been discharged to home were eligible for the hospital-to-home transition record quality measure. Eligibility for the timely discharge communication between providers quality measure required patients to have been discharged to home and have a primary care provider (PCP) who was not the hospital provider. Patients who experienced a transfer from the ICU to the inpatient floor were eligible for the ICU-to-floor transition note quality measure; for patients with >1 transfer, only the first transfer note was examined.
Variables manually extracted from the electronic medical records at all hospitals included content for scoring of the transition measures (Table 1; Supplemental Information), date of birth, dates and times of hospital admission and discharge, admission and transfer locations of care (ICU or floor), and discharge destination (home, transfer to another health care facility, left against medical advice, deceased). These data were merged with children’s hospital administrative data extracted from the Pediatric Health Information System, a comparative database used by 45 children’s hospitals nationally to document clinical and resource utilization,11 and from administrative records at community hospitals. These data included patients’ demographic characteristics, length of stay (LOS) in the hospital, 7- and 30-day all-cause readmission, and 7- and 30-day return visits to the emergency department (ED). At the children’s hospitals patient medical complexity was also assessed with the previously validated Pediatric Medical Complexity Algorithm (PMCA) applied to Pediatric Health Information System data.12 PMCA uses International Classification of Diseases, Ninth Revision, Clinical Modification codes to classify children into 1 of 3 disease groups: complex chronic disease, noncomplex chronic disease, and no chronic disease. These data were not available from the community hospitals’ administrative records.
After a 2-day training session, registered nurses from the research staff implemented the data abstraction tool at the 5 participating hospitals. Two nurses at each hospital each abstracted half of that hospital’s medical record sample, with each chart abstraction taking ~15 minutes. At 2 of the children’s hospitals, a randomly selected subsample of each nurse abstractor’s medical records was reabstracted by the other nurse to assess interrater reliability. Prevalence and rater bias–adjusted κ statistics were calculated to examine reliability in assessing patient eligibility for each measure and measure scoring.13
Analytic Approach
We examined differences in demographic and hospitalization characteristics by hospital type by using 2-sample t tests and Fisher’s exact tests, given that the populations of children cared for at freestanding children’s hospitals may differ from those receiving care at community hospitals.
Detailed measure specifications were used to calculate quality measure scores (Supplemental Information). For the individual-level binary measures (eg, timely discharge communication between providers), scores were 0 if absent (poor quality) and 100 if present (good quality). Binary subcomponents for multiple-component measures (eg, presence of a discharge medication list) were also scored in this manner and then summarized to produce a mean composite score for the measure on a 0 to 100 scale (eg, overall score for the hospital-to-home transition record quality measure). Measure subcomponents that were not applicable to a particular patient, such as inclusion of immunizations given during the hospitalization in the transition record for patients who received none, were not scored or included in the composite scores for such patients. Hospital-level scores, summarizing both binary and multicomponent measures, ranged from 0 to 100, with higher scores indicating better quality.
Because it is more challenging to achieve high scores on some measure components than others, and because not all patients were eligible for all measure components, we adjusted the hospital-to-home transition record quality measure scores and the hospital-to-home transition measure summary scores to account for the observed level of difficulty associated with achieving a high score on a given subcomponent. For patients who were eligible for only a subset of the components, this observed difficulty of delivery (ODD) adjustment accounted for the difference between the average overall pass rates for the patient’s subset of measures and the full set, adjusting upward if they were not eligible for an easy-to-pass item and vice versa.14,15 We performed this adjustment by subtracting each measure component for a given patient from its mean and averaging the “centered” measures applicable to a given patient; this patient-level average deviation from the mean was then added to the grand mean across all measure components to obtain a score on the original 0 to 100 scale.
We used 1-way analysis of variance to test the statistical significance of hospital-level variation in the hospital-to-home transition summary measure and the ICU-to-floor transition quality measure against the null hypothesis that all hospitals have the same mean scores.
Validation Measures
To evaluate the validity of these quality measures, we assessed the associations of the summary score and the 2 hospital-to-home transition quality measures with having ≥1 readmissions to the hospital within 7 and 30 days of discharge and with having ≥1 ED return visits within 7 and 30 days of discharge. For the ICU-to-floor transition note quality measure, we assessed associations between measure scores and having ≥1 ICU readmissions, and LOS, in days, truncated at the 99th percentile. Adjusted associations were calculated via logistic or linear regression according to the outcome, adjusted for sex, race or ethnicity, insurance type, and hospital. Medical complexity was also included in the validation analysis for the ICU-to-floor transition note quality measure; this variable was not included in the validation analyses for the hospital-to-home transition quality measures because it was not available from the community hospitals’ administrative data. However, recognizing that medical complexity is a potential confounder of the relationship between these quality measure scores and the validation outcomes, we conducted a sensitivity analysis in which we added PMCA classification to the models by using eligible observations from the children’s hospitals (n = 621, or 67% of the total field test sample).
RESULTS
A total of 927 charts were reviewed: 624 at 3 children’s hospitals and 303 at 2 community hospitals. Children receiving care at the children’s hospitals and community hospitals had similar age and sex distributions (Table 2). A greater proportion of children receiving care at the community hospitals were white and privately insured compared with those from the children’s hospitals. More than half of the patients in the children’s hospital sample had complex chronic medical conditions. Mean LOS was greater at the children’s hospitals than community hospitals, with higher rates of both 7-day and 30-day readmissions relative to the community hospitals.
TABLE 2.
Patient Characteristics | Children’s Hospitals (n = 624) | Community Hospitals (n = 303) | P | ||
---|---|---|---|---|---|
|
|||||
n | Median [Interquartile Range] or % | n | Median [Interquartile Range] or % | ||
Age, y | 624 | 5 [1, 12] | 303 | 5 [1, 11] | .43 |
Sex, malea | 349 | 56 | 160 | 54 | .63 |
Primary payera | |||||
Private | 205 | 33 | 201 | 68 | <.001 |
Public | 379 | 61 | 82 | 28 | <.001 |
Uninsured | 40 | 6 | 12 | 4 | .15 |
Race or ethnicityb | |||||
White | 285 | 46 | 102 | 61 | <.001 |
Hispanic | 159 | 26 | 11 | 7 | <.001 |
Black | 85 | 14 | 28 | 17 | .32 |
Asian or Pacific Islander | 32 | 5 | 12 | 7 | .31 |
Other | 56 | 9 | 13 | 8 | .62 |
Medical complexityc | |||||
No chronic disease | 134 | 21 | — | — | — |
Noncomplex chronic diseases | 140 | 22 | — | — | — |
Complex chronic disease | 350 | 56 | — | — | — |
Characteristics of hospitalization | |||||
Intensive care during Hospitalizationd | 121 | 19 | — | — | — |
LOS in dayse | 624 | 3 [2, 6] | 299 | 2 [1, 2] | <.001 |
7-d readmissione | 37 | 6 | 6 | 2 | .01 |
30-d readmissione | 89 | 14 | 10 | 3 | <.001 |
7-d return visit to EDf | 20 | 5 | 12 | 4 | .65 |
30-d return visit to EDf | 20 | 5 | 24 | 8 | .07 |
Sex and insurance status missing for 8 community hospital cases.
Race and ethnicity missing for 7 children’s hospital cases and 137 community hospital cases.
Pediatric medical complexity data missing for community hospitals.
Data missing for community hospitals because they did not have PICUs.
LOS and readmission data missing for 4 community hospital cases.
ED return visits missing for 1 children’s hospital (200 cases) and 4 community hospital cases.
Field testing of the medical record abstraction tool demonstrated almost-perfect interrater reliability on 2 levels: determination of eligibility for the quality measures (κ = 0.83–0.94) and scores on the quality measures (κ = 0.89–1.0; κ for the ICU-to-floor transition note quality measure could not be calculated because of the small sample size).13,16 A total of 924 patients from our sample were discharged to home and eligible for the hospital-to-home transition record quality measure. For this measure, scores for each component ranged from a low mean score of 3.4 (SD 18.0) for inclusion of pending test results to a high mean score of 96.3 (SD 18.8) for inclusion of discharge medications (Table 3). The mean ODD-adjusted composite score for the hospital-to-home transition record quality measure was 65.5 (SD 18.1). The mean score for the timely discharge communication between providers quality measure was 33.3 (SD 47.1), with 4 hospitals having mean scores <30 and 1 having a mean score of 98.0. The ODD-adjusted mean hospital-to-home transition summary score (combined score for hospital-to-home transition record quality measure and timely discharge communication between providers quality measures) was 61.2 (SD 17.1). A total of 126 patients in our sample experienced ICU-to-floor transitions; the mean score for the ICU-to-floor transition note quality measure was 64.9 (SD 47.1). Among the 119 patients who were eligible for all 3 quality measures, only 3 patients (2.5%) had a score of 100 on all 3 measures.
TABLE 3.
Number Eligible | Mean Score (SD) | Range of Hospital Mean Scores | |
---|---|---|---|
Measure 1: Hospital-to-home transition record qualitya | 924 | 65.5 (18.1) | 46.9–80.5 |
Diagnosis | 924 | 81.6 (38.8) | 67.3–98.5 |
Medication list | 924 | 96.3 (18.8) | 87.4–100.0 |
Follow-up appointments | 924 | 92.9 (25.8) | 88.4–98.0 |
24/7 hospital contact | 923 | 43.1 (49.5) | 2.0–98.2 |
Appointment assistance contact | 923 | 17.0 (37.6) | 0.0–58.9 |
Dates of service | 924 | 75.2 (43.2) | 0.0–99.5 |
Pending test results | 209 | 3.4 (18.0) | 0.0–14.3 |
Follow-up tests | 121 | 71.9 (45.1) | 45.5–90.0 |
Immunizations given | 69 | 42.0 (49.7) | 0.0–100.0 |
Measure 2: Timely discharge communication between providers | 896 | 33.3 (47.1) | 8.0–98.0 |
Hospital-to-home transition summary score (Measures 1 and 2 combined)a | 924 | 61.2 (17.1) | 42.4–71.3 |
Measure 3: ICU-to-floor transition note quality | 126 | 64.9 (47.1) | 48.6–71.5 |
Problem list | 126 | 65.9 (47.6) | 48.6–73.3 |
Treatment plan for each problem | 126 | 63.9 (47.0) | 48.6–69.9 |
ODD-adjusted quality scores.
The mean ODD-adjusted hospital-to-home transition quality summary scores, combining the 2 hospital-to-home transition measures, varied significantly across the 5 hospitals in our sample (Table 4). For the ICU-to-floor transition note quality measure, scores at 1 hospital differed significantly from scores at the other 2 children’s hospitals. We also observed variation in scores on these quality measures within hospitals. For example, Hospital B had the highest mean hospital-to-home transition summary score and the lowest mean ICU-to-floor transition note quality score.
TABLE 4.
Hospital Site | Eligible Patients, n | Hospital Mean Score (95% Confidence Interval) | P for Omnibus Test | Difference From Mean of All Other Hospitals | P for Difference From Overall Mean | |
---|---|---|---|---|---|---|
Hospital-to-home transition summary scorea | ||||||
All hospitals | 924 | 61.2 (60.1 to 62.3) | <.001 | — | — | |
Children’s hospitals | Hospital A | 198 | 70.7 (67.8 to 73.5) | — | 12.0 | <.001 |
Hospital B | 198 | 71.3 (69.7 to 73.0) | — | 13.3 | <.001 | |
Hospital C | 199 | 59.6 (58.3 to 60.9) | — | −2.1 | .13 | |
Community hospitals | Hospital D | 224 | 54.8 (52.8 to 56.9) | — | −7.7 | <.001 |
Hospital E | 150 | 42.4 (41.2 to 43.7) | — | −22.5 | <.001 | |
ICU-to-floor transition note quality measure scores | ||||||
All children’s hospitals | 126 | 64.9 (56.6 to 73.2) | .053 | — | — | |
Children’s hospitals | Hospital A | 46 | 70.8 (57.4 to 84.2) | — | 9.4 | .28 |
Hospital B | 35 | 48.6 (31.2 to 66.0) | — | −22.6 | .015 | |
Hospital C | 45 | 71.5 (58.3 to 84.7) | — | 10.3 | .24 |
ODD-adjusted summary scores, incorporating hospital-to-home transition record quality scores and timely discharge communication quality scores.
Results of the validation analyses are shown in Table 5. There were no significant associations of any of the hospital-to-home transition quality measures with hospital readmissions or return ED visits. Results were unchanged in the sensitivity analyses that added medical complexity to the models. Similarly, there were no significant associations of ICU-to-floor transition note quality measure scores with ICU readmissions or LOS.
TABLE 5.
Outcome | Hospital-to-Home Transition Quality Measure Scores |
Measure 3: ICU-to-Floor Transition Note Quality | ||
---|---|---|---|---|
Measure 1: Hospital-to-Home Transition Record Qualitya | Measure 2: Timely Discharge Communication Between Providers | Hospital-to-Home Transition Summary Score (Measures 1 and 2)a | ||
| ||||
Odds Ratio (95% Confidence Interval)b | Odds Ratio (95% Confidence Interval)b | Odds Ratio (95% Confidence Interval)b | Odds Ratio (95% Confidence Interval)b | |
7-d readmissionc | 0.91 (0.11 to 7.42) | 0.78 (0.24 to 2.47) | 0.74 (0.08 to 6.92) | — |
30-d readmissionc | 0.98 (0.25 to 3.85) | 0.78 (0.35 to 1.78) | 0.89 (0.20 to 3.91) | — |
7-d return ED visit c,d | 1.84 (0.15 to 22.46) | 0.55 (0.14 to 2.15) | 1.22 (0.09 to 16.42) | — |
30-d return ED visit c,d | 1.34 (0.15 to 12.30) | 0.63 (0.22 to 1.79) | 0.94 (0.09 to 9.32) | — |
ICU readmissione | — | — | — | 2.07 (0.35 to 12.27) |
— | — | Regression coefficient (95% confidence interval) | ||
LOSf | — | — | — | 0.96 (−3.99 to 5.90) |
These quality measures are adjusted for ODD.
Odds ratios are for a change of 0 to 100 on a 0 to 100 scale.
Models adjusted for hospital site and child sex, race or ethnicity, and insurance type.
Data missing for 1 children’s hospital.
Adjusted for hospital site and child sex, race or ethnicity, and insurance type; medical complexity not included, because all ICU readmissions were among children with complex chronic conditions.
Adjusted for hospital and child sex, race or ethnicity, insurance type, and medical complexity; LOS truncated at the 99th percentile.
DISCUSSION
Transitions in sites of care are inherent to all hospitalizations, and the risks associated with poor transitions are well documented among adult populations.3–5,17,18 The development of these new pediatric-specific transition of care quality measures, designed for evaluation with medical record data available at both children’s and community hospitals, is an important step toward a standardized assessment of care transition quality for hospitalized children.
Two of the quality measures endorsed by our multistakeholder panel and operationalized in this research assess hospital-to-home transitions, and the third evaluates the quality of transitions between the PICU and floor. Several of the measure subcomponents included in the hospital-to-home transition record quality measure align with the priorities for pediatric discharge summary content identified by hospitalists and PCPs in a previous national survey, including discharge diagnoses, discharge medications, dates of admission and discharge, immunizations given during hospitalization, follow-up appointment, and pending laboratory and test results. Consistent with results published by Coghlin et al,7 we observed low scores on transition record subcomponents related to immunizations given during the hospitalization, pending test results, and a hospital contact number for postdischarge problems. We also observed particularly low scores on the quality measure evaluating discharge communication between providers. Although it is possible that this measure underestimated the true frequency of communication because of incomplete documentation, our findings align with several previous studies in both adult and pediatric populations illustrating suboptimal rates of communication between inpatient and outpatient health care providers at the time of hospital discharge.2,4,7,19,20
Several studies among adults have shown improved patient outcomes and reduced health care utilization when individualized hospital-to-home transition records and improved provider communication were included in bundled hospital-to-home transition interventions.21–24 In our validation analyses, we observed no associations between hospital-to-home transition quality measure scores and hospital readmissions or return ED visit rates, perhaps because these outcomes are uncommon in pediatric populations or because our sample size was insufficient to detect associations between these quality measures and rare outcomes. Subsequent research examining associations between these measures and other established quality indicators such as postdischarge physical functioning, patients’ and families’ experiences of care, survey-based pediatric transitions of care measures, and frequency of medical errors and preventable adverse events may provide valuable data to inform transitional care priorities for pediatric populations.25–29
Although the importance of high-quality hospital-to-home transitions is supported by a growing body of research and national health care policy, the quality of transitions between hospital units has received less attention. We observed substantial variation across the children’s hospitals in ICU-to-floor transition note quality measure scores, with scores ranging from a low of 48.6 to a high of 71.5. Comprehensive transition documentation for transfers between clinical units may be challenged by differing clinical priorities and expectations, yet poor-quality handoffs can result in transfer delays, negative patient and family experiences, and increased risk of adverse events.28,30 Transitions between the ICU and general inpatient floor may put children at particular risk given the complexity of problems necessitating ICU care, although we did not observe associations between scores on this quality measure and ICU readmissions or hospital LOS. Building on the growing body of literature illustrating associations between standardized handoffs and decreased risk of medical errors, more studies are needed to understand how transitional care interventions affect pediatric outcomes for transfers between clinical units including the ED, ICU, operating room, and general pediatric floor.29,31–33
This quality measure development process had some limitations. First, pediatric evidence to guide measure development was sparse; the majority of studies illustrating associations between transition care processes and health care outcomes were conducted in adults.5 Therefore, these measures should be reviewed and revised as pediatric evidence continues to emerge. Second, although medical record–based measures are advantageous given the comprehensiveness of clinical data available relative to administrative data, it is possible that these measures may underestimate transition care quality if documentation is incomplete. Conversely, failure of documentation may itself indicate poor quality. Third, although our field test was conducted across 5 hospitals, including both children’s hospitals and community hospitals, performance of these measures may not be generalizable to all settings where children receive hospital-based care.
CONCLUSIONS
These newly developed quality measures were feasible to implement in both children’s and community hospitals and demonstrated variation in care between settings. The development of these measures is an important first step toward standardized evaluations and improvement of transition care quality for hospitalized children.
Supplementary Material
WHAT’S KNOWN ON THIS SUBJECT:
Transitions between sites of care, both within the hospital and from the hospital to home, are at-risk times for hospitalized children. However, we lack standardized methods to evaluate these transitions of care in pediatric populations.
WHAT THIS STUDY ADDS:
Three new pediatric quality measures were developed and field tested at children’s hospitals and community hospitals, where scores varied significantly across sites. These quality measures may be used to standardize evaluations of pediatric transitions of care for hospitalized children.
ACKNOWLEDGMENTS
We acknowledge the following members of our multistakeholder Delphi panel for their insight and constructive feedback: Julie Beckett (Family Voices); Gregory Conners, MD, MPH, MBA (American Academy of Pediatrics [AAP] Committee on Pediatric Emergency Medicine); David Cooperberg, MD (AAP Section on Hospital Medicine); John Gordon, MD (Children’s Hospital Association); Mary Keen, MD (AAP Home Care Committee); Nancy Skinner, RN-BC, CCM (Case Management Society of America); and Christopher Stille, MD, MPH (Academic Pediatric Association).
FUNDING:
Performed under funding from a cooperative agreement with the Agency for Healthcare Research and Quality and Centers for Medicare & Medicaid Services, grant U18HS020506, part of the Children’s Health Insurance Program Reauthorization Act Pediatric Quality Measures Program. Dr Leyenaar was supported by grant K08HS024133 from the Agency for Healthcare Research and Quality.
ABBREVIATIONS
- AAP
American Academy of Pediatrics
- COE4CCN
The Center of Excellence on Quality of Care Measures for Children With Complex Needs
- ED
emergency department
- LOS
length of stay
- ODD
observed difficulty of delivery
- PCP
primary care provider
- PMCA
Pediatric Medical Complexity Algorithm
- UCLA
University of California, Los Angeles
Footnotes
FINANCIAL DISCLOSURE:
Mr Burkhart, Dr Parast, Ms Roth, and Drs Gidengil and Elliott are employed by RAND; the other authors have indicated they have no financial relationships relevant to this article to disclose.
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
COMPANION PAPER: A companion to this article can be found online at www.pediatrics.org/cgi/doi/10.1542/peds.2016-1546.
REFERENCES
- 1.Lassman D, Hartman M, Washington B, Andrews K, Catlin A. US health spending trends by age and gender: selected years 2002–10. Health Aff (Millwood). 2014;33(5):815–822 [DOI] [PubMed] [Google Scholar]
- 2.Leyenaar JK, Bergert L, Mallory LA, et al. Pediatric primary care providers’ perspectives regarding hospital discharge communication: a mixed methods analysis. Acad Pediatr. 2015;15(1):61–68 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Snow V, Beck D, Budnitz T, et al. ; American College of Physicians. Society of General Internal Medicine; Society of Hospital Medicine; American Geriatrics Society; American College of Emergency Physicians; Society of Academic Emergency Medicine. Transitions of Care Consensus Policy Statement American College of Physicians–Society of General Internal Medicine–Society of Hospital Medicine–American Geriatrics Society–American College of Emergency Physicians–Society of Academic Emergency Medicine. J Gen Intern Med. 2009;24(8):971–976 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Kripalani S, LeFevre F, Phillips CO, Williams MV, Basaviah P, Baker DW. Deficits in communication and information transfer between hospital-based and primary care physicians: implications for patient safety and continuity of care. JAMA. 2007;297(8):831–841 [DOI] [PubMed] [Google Scholar]
- 5.Desai AD, Popalisky J, Simon TD, Mangione-Smith RM. The effectiveness of family-centered transition processes from hospital settings to home: a review of the literature. Hosp Pediatr. 2015;5(4):219–231 [DOI] [PubMed] [Google Scholar]
- 6.Roy CL, Poon EG, Karson AS, et al. Patient safety concerns arising from test results that return after hospital discharge. Ann Intern Med. 2005;143(2):121–128 [DOI] [PubMed] [Google Scholar]
- 7.Coghlin DT, Leyenaar JK, Shen M, et al. Pediatric discharge content: a multisite assessment of physician preferences and experiences. Hosp Pediatr. 2014;4(1):9–15 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Byron SC, Gardner W, Kleinman LC, et al. Developing measures for pediatric quality: methods and experiences of the CHIPRA pediatric quality measures program grantees. Acad Pediatr. 2014;14(5 suppl):S27–S32 [DOI] [PubMed] [Google Scholar]
- 9.Fitch K, Bernstein SJ, Mcdonnell J, Kahan JP. The RAND/UCLA Appropriateness Method User’s Manual. Santa Monica, CA: RAND Corporation; 2000 [Google Scholar]
- 10.Seattle Children’s Research Institute. Mangione-Smith lab measurement tools. Available at: www.seattlechildrens.org/research/child-health-behavior-and-development/mangione-smith-lab/measurement-tools/. Accessed February 23, 2016
- 11.Children’s Hospital Association Pediatric Health Information System. Available at: https://www.childrenshospitals.org/Programs-and-Services/Data-Analytics-and-Research/Pediatric-Health-Information-System. Accessed April 5, 2016
- 12.Simon TD, Cawthon ML, Stanford S, et al. ; Center of Excellence on Quality of Care Measures for Children With Complex Needs (COE4CCN) Medical Complexity Working Group. Pediatric Medical Complexity Algorithm: a new method to stratify children by medical complexity. Pediatrics. 2014;133(6). Available at: www.pediatrics.org/cgi/content/full/133/6/e1647 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Byrt T, Bishop J, Carlin JB. Bias, prevalence and kappa. J Clin Epidemiol. 1993;46(5):423–429 [DOI] [PubMed] [Google Scholar]
- 14.Min LC, Wenger NS, Fung C, et al. Multimorbidity is associated with better quality of care among vulnerable elders. Med Care. 2007;45(6):480–488 [DOI] [PubMed] [Google Scholar]
- 15.Reid RO, Friedberg MW, Adams JL, McGlynn EA, Mehrotra A. Associations between physician characteristics and quality of care. Arch Intern Med. 2010;170(16):1442–1449 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159–174 [PubMed] [Google Scholar]
- 17.Coleman EA, Smith JD, Frank JC, Eilertsen TB, Thiare JN, Kramer AM. Development and testing of a measure designed to assess the quality of care transitions. Int J Integr Care. 2002;2(June):e02. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Solan LG, Ranji SR, Shah SS. The successes and challenges of hospital to home transitions. J Hosp Med. 2014;9(4):271–273 [DOI] [PubMed] [Google Scholar]
- 19.Harlan GA, Nkoy FL, Srivastava R, et al. Improving transitions of care at hospital discharge: implications for pediatric hospitalists and primary care providers. J Healthc Qual. 2010;32(5):51–60 [DOI] [PubMed] [Google Scholar]
- 20.Harlan G, Srivastava R, Harrison L, McBride G, Maloney C. Pediatric hospitalists and primary care providers: a communication needs assessment. J Hosp Med. 2009;4(3):187–193 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Jack BW, Chetty VK, Anthony D, et al. A reengineered hospital discharge program to decrease rehospitalization: a randomized trial. Ann Intern Med. 2009;150(3):178–187 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Coleman EA, Parry C, Chalmers S, Min S. The care transitions intervention. Arch Intern Med. 2006;166(17):1822–1828 [DOI] [PubMed] [Google Scholar]
- 23.Koehler BE, Richter KM, Youngblood L, et al. Reduction of 30-day postdischarge hospital readmission or emergency department (ED) visit rates in high-risk elderly medical patients through delivery of a targeted care bundle. J Hosp Med. 2009;4(4):211–218 [DOI] [PubMed] [Google Scholar]
- 24.Balaban RB, Weissman JS, Samuel PA, Woolhandler S. Redefining and redesigning hospital discharge to enhance patient care: a randomized controlled study. J Gen Intern Med. 2008;23(8):1228–1233 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Desai AD, Zhou C, Stanford S, Haaland W, Varni JW, Mangione-Smith RM. Validity and responsiveness of the pediatric quality of life inventory (PedsQL) 4.0 generic core scales in the pediatric inpatient setting. JAMA Pediatr. 2014;168(12):1114–1121 [DOI] [PubMed] [Google Scholar]
- 26.Toomey SL, Zaslavsky AM, Elliott MN, et al. The development of a pediatric inpatient experience of care measure: Child HCAHPS. Pediatrics. 2015;136(2):360–369 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Berry JG, Ziniel SI, Freeman L, et al. Hospital readmission and parent perceptions of their child’s hospital discharge. Int J Qual Health Care. 2013;25(5):573–581 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Sheth S, McCarthy E, Kipps AK, et al. Changes in efficiency and safety culture after integration of an I-PASS-supported handoff process. Pediatrics. 2016;137(2):e20160166. [DOI] [PubMed] [Google Scholar]
- 29.Starmer AJ, Spector ND, Srivastava R, et al. I-PASS Study Group. Changes in medical errors after implementation of a handoff program. N Engl J Med. 2014;371 (19):1803–1812 [DOI] [PubMed] [Google Scholar]
- 30.van Sluisveld N, Hesselink G, van der Hoeven JG, Westert G, Wollersheim H, Zegers M. Improving clinical handover between intensive care unit and general ward professionals at intensive care unit discharge. Intensive Care Med. 2015;41 (4):589–604 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Starmer AJ, O’Toole JK, Rosenbluth G, et al. I-PASS Study Education Executive Committee. Development, implementation, and dissemination of the I-PASS handoff curriculum: a multisite educational intervention to improve patient handoffs. Acad Med. 2014;89(6):876–884 [DOI] [PubMed] [Google Scholar]
- 32.Bigham MT, Logsdon TR, Manicone PE, et al. Decreasing handoff-related care failures in children’s hospitals. Pediatrics. 2014;134(2). Available at: www.pediatrics.org/cgi/content/full/134/2/e572 [DOI] [PubMed] [Google Scholar]
- 33.Horwitz LI, Meredith T, Schuur JD, Shah NR, Kulkarni RG, Jenq GY. Dropping the baton: a qualitative analysis of failures during the transition from emergency department to inpatient care. Ann Emerg Med. 2009;53(6):701–710 [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.