Overall Health following pediatric critical illness: a scoping review of instruments and methodology

Abstract

Objective:

Families identify overall health as a key outcome after pediatric critical illness. We conducted a planned secondary analysis of a scoping review to determine the methods, populations, and instruments used to evaluate overall health outcomes for both children and their families after critical illness.

Design:

Planned Secondary Analysis of a Scoping Review

Setting:

We searched PubMed, EMBASE, PsycINFO, Cumulative Index of Nursing and Allied Health Literature, and the Cochrane Controlled Trials Registry databases from 1970 through 2017 to identify studies which measured post-discharge overall health of children who survived critical illness and their families.

Subjects:

Manuscripts reporting overall health outcomes after pediatric critical illness

Interventions:

None

Measurements and Main Results:

Among the 407 articles which measured outcomes following pediatric critical illness, 161 (40%) measured overall health. The overall health domain was most commonly measured in traumatic brain injury (44%) and the general PICU (16%) populations. In total, there were 39 unique measures utilized to evaluate overall health. Across all subjects, 7 measures accounted for 89% of instruments, with the Glasgow Outcome Scale (47%) and the Pediatric Overall Performance Category (17%) being most commonly used. Excluding studies targeting survivors of traumatic brain injury, Pediatric Overall Performance Category, Glasgow Outcome Scale, and the General Health Questionnaire were the most commonly used instruments. Patients were followed for a median 10.5 months (IQR 4.5–21).

Conclusions:

Overall health was commonly assessed post-PICU discharge, especially in the traumatic brain injury population, using a heterogenous array of measures. Evaluation and consensus are imperative to identify the most appropriate method to measure overall health with the goal of improving care efficacy and facilitating recovery across populations of critically ill children.

Introduction

Mortality rates in the pediatric intensive care unit (PICU) have declined, leading to increased attention to morbidities among survivors(1–3). In recognition, Post-Intensive Care Syndrome-pediatrics (PICS-p) provides a conceptual framework to evaluate the physical, cognitive, emotional, and social health outcomes after critical illness(4, 5). Family health and health-related quality of life were identified as additional important outcomes in a scoping review of studies evaluating outcomes after pediatric critical illness conducted by the POST-PICU PICU (Pediatric Outcomes STudies after PICU) Investigators of the Pediatric Acute Lung Injury and Sepsis Investigators (PALISI) network(6). The POST-PICU Investigators also identified overall health, defined as the general health and well-being including but not specific to physical, cognitive, emotional, social, health-related quality of life, or family domains of health, as a key outcome to evaluate after pediatric critical illness(6). This domain identifies measures that provide a general, but comprehensive evaluation which encompasses multiple domains, allowing for the clinician to detect patients or domains that require a more in-depth evaluation. Recently, the POST-PICU Investigators of the PALISI Network (Supplemental Table 1, Supplemental Digital Content) and Eunice Kennedy Shriver National Institute of Child Health and Human Development’s Collaborative Pediatric Critical Care Research Network (CPCCRN) developed a PICU Core Outcome Set (COS) for use in pediatric critical care research and clinical programs(7). In this, overall health was identified as a critically important outcome domain by more than 96% of respondents, including 100% of family participants(7). As 39 unique measures were used to evaluate overall health, this heterogeneous use of instruments limits our ability to compare outcomes across populations and over time (6).

A more systematic approach to evaluation of overall health after pediatric critical illness will result in a more complete understanding of this important outcome domain. Thus, our objective was to perform an analysis of the articles generated from the Pediatric Critical Care Scoping Review to specifically evaluate the methods, populations, and instruments used to evaluate overall health outcomes for both children and their families after critical illness as a resource for future PICU researchers evaluating the impact of critical illness on overall health(6).

Methods

As part of a scoping review to identify studies which measured outcomes of children who survived critical illness or families, we identified overall health measures as those that provided a general measure of health status. Analysis of the specific domains, including overall health, were planned a priori to explore the specific domain topic in greater detail. Institutional Review Board approval was not required given the scope of this study.

The POST-PICU Scoping Review Investigators (Supplemental Table 1, Supplemental Digital Content) conducted the scoping review by searching PubMed, EMBASE, PsycINFO, Cumulative Index of Nursing and Allied Health Literature, and the Cochrane Controlled Trials Registry databases from 1970 through 2017. We included articles that 1) assess post-discharge outcomes; 2) include more than 1 subject; and 3) include an instrument which measured the overall health domain. Articles were excluded if 1) survival was the only outcome assessed; 2) the study evaluated only the psychometric properties of an instrument; 3) the article did not report the relationship of the critical illness, technical procedure, or ICU care to the measured outcome; 4) the majority of the study population was >18 years old or preterm infants/neonates; 5) the study population had not been definitively admitted to an ICU; or 6) the study was not published in English. Further detail of study and search methods have been previously published(6). In brief, abstracts, and full text manuscripts were independently screened and evaluated in a two-stage process by two reviewers and discrepancies were resolved by a third reviewer. Two steering committee members (AM and NP) conducted an independent review of the instruments to identify domain(s) evaluated. Discrepancies were resolved through discussion and further review of the instruments.

Prespecified data were retrieved from each study including overall health domain instruments, method of assessment, source of data, participant retention rate, patient hospitalization characteristics, patient and family demographics, study location, and year of publication. Next, within the overall health domain, we explored, in detail, the seven most commonly used measures and the measures initially used between 2007–2017. Given the predominance of studies evaluating traumatic brain injury (TBI) patients, we examined measures used among 1) all overall health studies; 2) studies targeting the enrollment of TBI patients; and 3) studies which did not specifically target enrollment of TBI patients.

We describe measure characteristics including age range, validation, ease of administration, availability of normative data, feasibility and longitudinal assessment capabilities. Summary statistics are provided as counts for categorical data and median and interquartile range [IQR] for continuous data. Study data were collected and managed in the Research Electronic Data Capture (REDCap) database hosted at the University of Utah(8, 9). Instrument-specific study data were collected and managed in the REDCap database hosted by the University of Washington. Statistical analyses were performed by the University of Utah Data Coordinating Center using SAS version 9.4 (Cary, NC).

Results

Of 60,349 publications screened, 407 measured post-discharge outcomes following pediatric critical illness. Among these, 161 (40%) measured overall health and were included in this review. The list of studies included is available in the Online Supplement (Supplemental Table 2, Supplemental Digital Content). Overall, the number of studies that evaluated overall health published each year increased over time, with the majority (n=139; 86%) published after 2000 (Figure 1). Studies that measured overall health also measured physical (21.1%), cognitive (35.4%), emotional (15.5%), social (73.3%), health-related quality of life (22.4%), and family (11.2%) domains of health (Figure 2). The overall health domain also included general measures of healthcare use in an additional 21 articles. This included readmission, emergency department visits, medications, cost, tracheostomy decannulation, new technology, overall resource utilization, and placement of a feeding tube (Table 1).

Figure 1:

Number of manuscripts evaluating overall health outcomes after pediatric critical illness. The number of studies evaluating survivorship of pediatric critically ill children and, specifically, evaluation of the overall health domain have increased over the last two decades. This increase is notable in manuscripts targeting children who survived a traumatic brain injury as well as other pediatric critically ill populations.

Figure 2:

Overall health is often measured with other outcome domains following pediatric critical illness, most often Social Health and Cognitive Health.

Table 1.

Health Resource Use Measures

	Readmission* (N=14)	Medications (N=2)	Cost (N=2)	Decannulation (N=1)	New Technology (N=1)	Resource Utilization (N=1)	Feeding Tube (N=1)
Method of Assessment
Chart Review	8	1	1	1	0	0	1
In-person	0	0	0	0	0	1	0
Phone Interview	2	0	0	0	0	0	0
Other	4	0	1	0	0	0	0
Not specified	1	1	0	0	1	0	0
Source of data
Medical record	8	0	1	0	0	0	0
Parent/guardian	2	0	0	0	0	1	0
Patient/clinician	2	0	0	0	0	0	0
Other	2	0	1	1	0	0	0
Not specified	2	2	0	0	1	0	1
Number of time points evaluated, Median (IQR)	1.0 (1.0–1.0)	1.0 (1.0–1.0)	1.5 (1.0–2.0)	---	1.0 (1.0–1.0)	1.0 (1.0–1.0)	---
Anchor point of outcome
Hospital discharge	6	0	0	0	1	0	0
ICU discharge	1	0	0	0	0	0	0
Hospital or ICU admission	3	1	1	1	0	1	1
Other	0	1	0	0	0	0	0
Not specified	4	0	1	0	0	0	0
Final post-discharge time point
3 months	0	0	0	0	1	0	0
>9 to 12 months	3	0	1	0	0	1	0
>12 to 18 months	0	1	0	0	0	0	0
>18 to 24 months	1	0	0	0	0	0	0
>36 months	3	0	1	1	0	0	0
Not specified	7	1	0	0	0	0	1

The number of methods of assessment and source of data may not equal the total number of studies utilizing that instrument as each measure could have used multiple methods or sources. IQR: interquartile range; ICU: intensive care unit.

^*Readmission includes Emergency Department visits and hospital readmissions

Patient Population and Study Characteristics

Overall health measures were employed most commonly in studies targeting traumatic brain injury (70/161; 43.5%) and general PICU (26/161, 16.2%) patients (Table 2). Across all studies, the median percent of males enrolled was 62% [IQR 54.7–67.7] (Table 2). Mortality was evaluated during follow-up in 89 (55.3%) studies. The median post-discharge mortality rate during follow-up was 2.4% (IQR 0.0%–11.8%). Of the 161 studies which evaluated the overall health domain, 142 (88.2%) were observational, 13 (8.1%) interventional, and 6 (3.7%) mixed methods. No studies employed qualitative methods. In total, 31.1% of studies were performed in the United States, 11.8% in Australia, and 11.2% in the United Kingdom. The median enrolled study sample size was 58.5 (IQR 34–120) children, with 50 (IQR 27–118) children eligible for follow-up at hospital discharge. Among these patients, 90.5% were assessed at the final follow-up timepoint (Table 2).

Table 2.

Patient Study Characteristics

	All Studies n=161	Studies targeting enrollment of traumatic brain injury patients n=70	Studies which did not target traumatic brain injury patients n=91
Population Studied, n (%)
Traumatic Brain Injury	70 (43.5%)	70 (100.0%)	0 (0%)
General PICU	26 (16.2%)	0 (0%)	26 (28.6%)
Cardiac Arrest	10 (6.2%)	0 (0%)	10 (11.0%)
Sepsis	10 (6.2%)	0 (0%)	10 (11.0%)
Congenital heart disease	8 (5.0%)	0 (0%)	8 (8.8%)
Trauma	7 (4.4%)	1 (1.4%)	6 (6.6%)
Acute respiratory failure	2 (1.2%)	0 (0%)	2 (2.2%)
Solid organ transplant	2 (1.2%)	0 (0%)	2 (2.2%)
Oncology/Bone marrow transplant	2 (1.2%)	0 (0%)	2 (1.2%)
Other	37 (23.0%)	0 (0%)	34 (37.4%)
Percent Males Enrolled, Median (IQR)	62.0 (54.7–67.7)	66.7 (61.6–74.0)	58.3 (50.5–62.5)
Family Members Evaluated, n (%)
Anya	18 (11.2%)	2(2.9%)	16 (17.6%)
Parent/Grandparent	18 (11.2%)	2 (2.9%)	16 (17.6%)
Mortality measured during post-discharge follow-up, n (%)	89 (55.3%)	35 (50.0%)	54 (59.3%)
Died during post-discharge follow-up, median % (IQR)	2.4 (0.0–11.8)	0.0 (0.0–4.4)	4.1 (0.0–17)
Study Locationb, n (%)
United States	50 (31.1%)	26 (37.1%)	24 (26.4%)
Australia	19 (11.8%)	6 (8.6%)	13 (14.3%)
United Kingdom	18 (11.2%)	5 (7.1%)	13 (14.3%)
Canada	16 (9.9%)	2 (2.9%)	14 (15.4%)
Other Europe	31 (19.2%)	8 (11.4%)	23 (25.3%)
Other	47 (29.2%)	28 (40.0%)	19 (20.9%)
Enrolled sample size, median (IQR)	58.5 (34–120)	48 (29.0–85.5)	71.5 (37.0–150.0)
Enrollment rate, median % (IQR)	94.2 (66.1–100.0)	100.0 (76.9–100.0)	84.3 (58.3–100.0)
Enrollment rate not specified, n (%)	31 (19.3%)	18 (25.7%)	13 (14.3%)
Patients eligible for follow-up at hospital discharge, median (IQR)	50 (27–118)	45 (28.0–81.0)	66.5 (27.0–170.0)
Percent assessed at final point, median (IQR)	90.5 (71.5–100.0)	95.8 (81.1–100.0)	82.3 (66.5–98.7)
Number of participants assessed at final follow-up, median (IQR)	45 (21–83)	40.0 (22.0–69.0)	50.0 (21.0–114.0)
Number of overall health specific instruments per article, median (IQR)	2 (1–4)	2 (1–3)	2 (1–4)
Number of instruments per article, n (%)
1	59 (36.6%)	34 (48.6%)	25 (27.5%)
2–4	76 (47.2%)	24 (34.3%)	52 (57.1%)
5–7	11 (6.8%)	3 (4.3%)	8 (8.8%)
8–10	13 (8.1%)	7 (10.0%)	6 (6.6%)
>10	2 (1.2%)	2 (2.9%)	0 (0.0%)

n (%) is presented for categorical variables and median (interquartile range) is presented for continuous characteristics.

^aNo siblings were specifically evaluated in studies measuring overall health.

^bGroups are not mutually exclusive due to inclusion of studies conducted internationally.

In total, 70 (43.5%) studies were conducted in the TBI population with a total sample size of 5,401 patients (Table 2). Total sample size in the non-TBI population was 32,145 patients. Among the 91 studies which did not target enrollment of TBI patients, the general PICU population was most often studied (26 of 91 studies), followed by cardiac arrest (10 of 91 studies) and sepsis (10 of 91 studies). The overall mortality rate during post-discharge follow-up was 0.0% (IQR 0.0–4.4) among TBI patients and 4.1% (IQR 0%–17%) in the non-TBI population. The median enrolled sample size was 48.0 (IQR 29.0–85.5) patients in the TBI studies and 71.5 (IQR 37.0–150.0) patients among non-TBI studies (Table 2). A median of 2 (IQR 1–3) overall health specific instruments were used in TBI articles and 2 (IQR 1–4) in non-TBI articles.

Specific Measures

We identified 39 unique measures used to evaluate overall health following pediatric critical illness. Included studies used a median of 2 instruments (IQR 1–4) to measure overall health (Table 2). The most commonly used measures were the Glasgow Outcome Scales (46.6%), Pediatric Overall Performance Category (POPC)(17.4%), General Health Questionnaire (10.6%), general measures of school performance(7.5%), Functional Status Scale (FSS)(2.5%), King’s Outcome Scale for Childhood Head Injury (2.5%), and Royal Alexandria Hospital for Children Measure of Function (2.5%) (Supplemental Figure 1, Supplemental Digital Content). These accounted for 89.4% of measures used to evaluate overall health (Table 3). The Functional Status Scale score and King’s Outcome Scale are used in more recent manuscripts, with a median year of publication 2012.5 (IQR 2008–2017) and 2013.0 (IQR 2009 – 2015.5), respectively. Conversely, the General Health Questionnaire was used less frequently in the last decade, with a median publication date 2006.0 (IQR 2005–2010). After excluding studies targeting the TBI population, the top 5 instruments used were POPC (n=25), Glasgow Outcomes Scales (n=17), General Health Questionnaire (n=16), general school performance (n=7), and the Royal Alexandra Hospital for Children Measure of Function (n=4).

Table 3.

Measure Characteristics of 7 Most Commonly Used Instruments

	Glasgow Outcome Scale (N=75)	Pediatric Overall Performance Category (N=28)	General Health Questionnaire (N=17)	School Performance (N=12)	Functional Status Scale (N=4)	King’s Outcome Scale (N=4)	Royal Alexandra Hospital for Children Measure of Function (N=4)
Method of Assessment, N
In-person	20	4	7	3	1	2	1
Phone Interview	19	7	2	7	1	0	4
Chart Review	14	8	1	2	0	0	0
Standard mail	3	0	4	1	0	1	0
Electronic	0	0	0	1	0	0	0
Other	2	2	1	2	0	0	0
Not specified	31	11	5	4	0	1	0
Source of data, N
Patient/clinician	21	8	4	3	1	3	1
Parent/guardian	20	12	14	7	2	0	3
Medical record	12	8	2	1	0	0	0
Other	0	0	0	1	0	0	0
Not specified	39	8	1	4	0	1	1
Number of time points evaluated, Median (IQR)	1.0 (1.0–1.0)	1.0 (1.0–1.0)	1.0 (1.0–1.0)	1.0 (1.0–1.0)	1.5 (1.0–2.0)	1.0 (1.0–2.5)	1.0 (1.0–1.5)
Year of manuscript, Median (IQR)	2009.5 (2005.0–2013)	2010.0 (2006.0–2014.5)	2006.0 (2005.0–2010.0)	2008.5 (1995.0–2016.5)	2012.5 (2008.0–2017.0)	2013.0 (2009.0–2015.5)	2010.0 (2004.5–2013.0)
Anchor point of outcome, N
Hospital or ICU admission	29	10	4	2	1	2	0
Hospital discharge	9	8	5	3	1	0	0
ICU discharge	6	5	2	1	0	1	4
Discharge, NOS	11	0	2	3	0	0	0
Other	17	2	1	0	0	0	0
Not specified	0	0	0	0	2	0	0
Final post-discharge time point, N
<1 month	0	2	0	0	0	0	0
1 to 3 months	7	3	2	0	0	0	0
3 to 6 months	20	3	0	0	0	0	0
6 to 9 months	6	1	2	0	0	0	0
9 to 12 months	13	9	5	0	0	1	1
12 to 24 months	4	2	0	2	0	1	2
>24 months	17	2	5	4	1	0	1
Not specified	8	6	3	6	3	2	0

The number of methods of assessment and source of data may not equal the total number of studies utilizing that instrument as each measure could have used multiple methods or sources. ICU: Intensive Care Unit; NOS: Not otherwise specified

Among the top instruments, 2 (POPC and FSS) were validated in a PICU population (10–13), and 4 (Glasgow Outcome Scale, General Health Questionnaire-28, FSS, King’s Outcome Scale) were validated in a general pediatric or adolescent population (12–19). One instrument (General Health Questionnaire-28) required the patient as the sole source of information, while all others permitted a proxy or clinician to provide information (Table 4). Three of 6 instruments did not describe the specific age range in which the instrument was validated. While most instruments were available for public use and free of charge, 2 (General Health Questionnaire-28 and Royal Alexandria Hospital for Children Measure of Function) were proprietary (Supplemental Table 3, Supplemental Digital Content). Further information pertaining to each instrument including age range, validation population, and general description is available in the Online Supplement (Supplemental Table 3, Supplemental Digital Content).

Post-Discharge Assessment

Patients were followed for a median 10.5 months (IQR 4.5–21). The assessment time points were most commonly anchored from admission to the hospital or PICU (32.9%) and hospital (16.8%) or PICU (16.2%) discharge. The most common mode of assessment was in-person (26.9%), followed by phone interview (25.8%) and chart review (16.5%). However, 63 (34.6%) studies did not specify how data were collected. Assessments using the top 7 instruments were most often performed in-person or via phone interview (Table 3). The most common data sources were parent/guardian or patient/clinician. However, 52 (36.1%) and 54 (37.5%) did not specify method of assessment or source of data, respectively. The longest duration of follow-up was less than 1 year in 75 (52.1%) studies. Twenty-three (16.0%) final assessments performed at longest timepoint of 36 months or greater.

Discussion

In a scoping review of outcomes following pediatric critical illness, measurements of overall health were included in 40% of studies, representing the broad importance of this health domain. Among the 39 measures to assess overall health, seven tools were used in the majority of studies (~90%). These 7 measures are heterogeneous with regard to validation, applicability to a PICU population, and ability to encompass the broad age ranges and developmental status of PICU patients. The heterogeneity of instruments used to evaluate the overall health domain challenges our ability to compare post-discharge overall health across PICU populations.

Overall health is important to evaluate after pediatric critical illness given the wide reach of this domain in various aspects of children’s lives. Measures of overall health are often a composite of multiple aspects of daily life including functional status, communication, social interactions, feeding, and school participation and achievement. Thus, overall health is affected by age, developmental status, behavioral and emotional health, surrounding environment, and social support. This is especially important when considering post-intensive care outcomes such as PICS-p. A patient’s overall health is deeply entwined with each aspect of PICS-p including physical, cognitive, emotional, and social health and therefore highly relevant when evaluating PICS-p(4, 5). The relevance is clearly reflected by the preponderance of studies (~40%) that employed instruments that directly evaluate overall health in the Pediatric Critical Illness Scoping Review(6). Overall Health was also identified as a core outcome domain in the recently published multistakeholder informed Pediatric Critical Care Core Outcome Set(7). Additionally, a child’s functional status was deemed a key patient-centered outcome during recovery by both parents of children recovering from critical illness(20) and healthcare providers(21). Indeed, functional impairment was found to be common among survivors of critical illness in a recent systematic review, occurring in up to 1/3 of patients at hospital discharge and persisting in 10% of children after two years(22). Tools to systematically evaluate overall health are thus necessary to assess how a patient progresses after critical illness.

In order to comprehensively evaluate the overall health of survivors of pediatric critical illness, measures should be valid and reliable. A few scales or surveys used to measure overall health were created specifically for a pediatric population including the Functional Status Scale(12), Pediatric Overall Performance Category (11), King’s Outcome Scale for Childhood Head Injury(23), and Royal Alexandria Hospital for Children Measure of Function(24). Additionally, the Glasgow Outcome Scale, has both pediatric and adult versions, improving its applicability to younger children(16, 25). However, other measures commonly used in the PICU population were not developed for children. For example, the General Health Questionnaire was not designed for a pediatric population but has been validated in adolescent cohorts(19, 26). It is imperative to use pediatric-specific measures to incorporate the relevant age and developmental status factors when assessing overall health outcomes.

Additionally, baseline (pre-critical illness) overall health is a key determinant in post-discharge overall health. As such, it is vital to obtain baseline data to which discharge data can be compared. Not all instruments allow for the assessment of baseline health—for example, the Glasgow Outcome Scale is only applied following head injury. In most critically ill patients, this requires proxy report of baseline overall health due the patient’s clinical state and resultant inability to participate at the time of admission. For some instruments, proxy report is not a valid data source(e.g. General Health Questionnaire) and, thus, these instruments are less amenable to baseline data collection. Likewise, post-discharge data that relies on a clinician’s evaluations (e.g. King’s Outcome Scale for Childhood Head Injury) may not be readily obtained for patients who are unable to return to a clinical setting for evaluation. Greater flexibility (e.g. patient versus clinician administered or phone versus online) with the appropriate validation data will allow for a broader user base across populations and studies.

We recognize that our study has important limitations. First, the Pediatric Critical Illness Scoping Review included studies published through 2017; thus, we may have missed relevant publications or instruments published since then. Second, the category of overall health is broad and includes outcomes such as school performance, functional status, and new diseases or health conditions, potentially making it difficult to ultimately capture this outcome with a single measure. Additionally, the domain of overall health may have varying definitions depending on the population being studied and, thus, some instruments may have been missed. Finally, studies of overall health outcomes largely represented primarily English-speaking countries as we only included studies that were available in English. Of the top outcome measures, only the General Health Questionnaire has been translated into other languages by its authors. Translation of other instruments into additional languages may reveal other useful instruments with broader applicability to geographically and ethnically diverse cohorts and an international community.

The measurement of overall health uses instruments which incorporate multiple domains within PICS-p. For example, the FSS score assesses physical health such as motor function and respiratory impairments, as well as communication skills, while the Glasgow Outcome Scale-extended Pediatrics measures activities of daily living, social relationships, and the ability to function in school. Although impairments are often measured within a given domain, those which occur in one domain clearly affect ongoing developmental trajectory across other domains. For example, optimal social and emotional development are dependent on both physical and cognitive development both of which highly impact a child’s ability to interact socially. As such, evaluation of the overall health domain helps to address the interconnected nature of the specific PICS-p domains.

In conclusion, measures of overall health were commonly included in studies of long-term outcomes following pediatric critical illness. While overall health encompasses heterogenous outcomes, 7 measures were used nearly 90% of the time the domain was evaluated and most frequently targeted critically ill children who had survived a traumatic brain injury. These measures were not universally validated or developed for use in pediatric populations and do not consistently account for patient age, developmental stage, or baseline health status. Thus, further evaluation and consensus are necessary to identify the most appropriate methods and tools to measure overall heath and better characterize overall health outcomes among children who experience critical illness.