Understanding the Global Epidemiology of Pediatric Critical Illness: The Power, Pitfalls, and Practicalities of Point Prevalence Studies

Abstract

Objective

The point prevalence methodology is a valuable epidemiological study design that can optimize patient enrollment, prospectively gather individual-level data, and measure practice variability across a large number of geographic regions and health care settings. The objective of this manuscript is to review the design, implementation, and analysis of recent point prevalence studies investigating the global epidemiology of pediatric critical illness.

Data Sources

Literature review and primary datasets.

Study Selection

Multicenter, international point prevalence studies performed in pediatric intensive care units since 2007.

Data Extraction

Study topic, number of sites, number of study days, patients screened, prevalence of disease, use of specified therapies, and outcomes.

Data Synthesis

Since 2007, five point prevalence studies have been performed on acute lung injury, neurological disease, thromboprophylaxis, fluid resuscitation, and sepsis in pediatric intensive care units. These studies were performed in 59 to 120 sites in seven to 28 countries. All studies accounted for seasonal variation in pediatric disease by collecting data over multiple study days. Studies screened up to 6,317 patients and reported data on prevalence and therapeutic variability. Three studies also reported short-term outcomes, a valuable but atypical data element in point prevalence studies. Using these five studies as examples, the advantages and disadvantages and approach to designing, implementing, and analyzing point prevalence studies are reviewed.

Conclusions

Point prevalence studies in pediatric critical care can efficiently provide valuable insight on the global epidemiology of disease and practice patterns for critically ill children.

INTRODUCTION

Epidemiological data about disease burden, practice variability, and outcomesare essential to identify research priorities, design clinical trials, track disease-specific metrics, develop guidelines for diagnosis and therapy, and allocate health care resources (1). In pediatric critical illness, single institutions cannot provide sufficient patient volume to generate broadly representative epidemiological data (2). For example, in the United States, half of all pediatric intensive care units (PICUs) have ≤ 12 beds (3, 4). However, multi-center studies require substantial resources not available at many institutions. Administrative datasets offer a rich source of data, but are subject to important limitations including imprecise case ascertainment, incomplete patient-level data, lack of severity of illness adjustment, and difficulty in drawing comparisons across patient subgroups, health care systems, and geographic regions due to inconsistencies between databases (5). A practical alternative to optimize patient selection, prospectively collect desired variables, and study subgroups across diverse settings is the point prevalence study design.

Although point prevalence studies have been used widely in other disciplines (6, 7), this approach has only recently been applied to pediatric critical care. Five large multicenter point prevalence studies have recently been conducted through the Pediatric Acute Lung Injury and Sepsis Investigators (PALISI) and Australia and New Zealand Intensive Care Society (ANZICS) networks. These five studies will have screened almost 18,000 critically ill children worldwide, demonstrating the scope and wide-spread importance of these efforts. These international collaborations are providing new insight into the epidemiology of disease and practice patterns in pediatric critical illness, such as acute lung injury (8), neurological disease (9), thromboprophylaxis (10), fluid resuscitation (11), and sepsis (12). For the first time, prospective data are available that cross systems of care and permit direct comparisons of healthcare delivery strategies for critically ill children. While use of the point prevalence methodology to study epidemiological trends in pediatric critical care is increasing, there is little guidance available for clinicians, scientists, and policy makers to interpret and apply this type of data in the pediatric critical care setting.

Our objective is to review the design, implementation, and analysis of point prevalence studies in general and to describe the power, pitfalls, and practicalities of using this methodology to study the global epidemiology of pediatric critical illness.

Definition and Utility of Point Prevalence Studies

Point prevalence is defined as the proportion of a population with an existing condition at a defined moment in time (13). The point prevalence design is a prospective observational study used to obtain a “snap shot” of a condition of interest, often across diverse settings. These studies are best suited to estimate the prevalence of disease within a population, the characteristics and treatment of affected patients, and epidemiological trends over time (14). To account for seasonal variability in disease prevalence, several time points may be included as long as there is a sufficient “washout” period between observations to minimize the risk of including a patient more than once. In pediatric critical care, where the prevalence of any one disease is low but duration of illness tends to be short, this multiple time point approach has proven especially useful.

The epidemiological data generated through point prevalence studies can be used by clinicians to inform decision-making; by scientists to identify research priorities; by health care administrators to benchmark disease-specific metrics; and by policy makers to allocate resources (1). For example, the Pediatric Acute Lung Injury VEntilation (PALIVE) study reported a 10.8% prevalence of pediatric acute lung (ALI) by clinician diagnosis but only a 4.3% prevalence by published criteria (15, 16). In addition, substantial heterogeneity was noted in mechanical ventilation strategies and use of adjunctive therapies. These observations helped drive the establishment of the Pediatric Acute Lung Injury Consensus Conference to develop consensus definitions and recommendations for treatment and research priorities (17). Point prevalence studies also provide data to evaluate the feasibility of clinical trials and guide sample size estimations. The PALIVE study estimated that at least 60 PICUs would need to enroll patients over four years to have sufficient power in a clinical ALI trial if mortality was selected as the primary endpoint. Data from PALIVE have been instrumental in the development of studies to improve ventilation strategies in pediatric ALI (18, 19). In neonatology and pediatric critical care, small point prevalence studies have also been able to focus clinical, scientific, and public health attention on the problem of health care-associated infections (20, 21).

Advantages and Disadvantages of Point Prevalence Studies

The point prevalence design has several advantages for epidemiological studies. For example, both subject identification and data collection can be done prospectively using a priori defined criteria in point prevalence studies. In contrast, retrospective case ascertainment from administrative datasets often relies on billing codes, such as the International Classification of Disease-9, Clinical Modification (ICD-9-CM), that were not assigned using the rigorous standards expected for research. Several studies have highlighted the poor sensitivity of ICD-9-CM codes for case identification in research (22-25). Existing datasets also have fixed data elements and limited patient-level variables. Prospective data collection in point prevalence studies circumvents these limitations by ensuring that case ascertainment and data elements are defined as a starting point in the study design.

Since point prevalence studies aim primarily to identify a disease/condition of interest, the screening process itself suffices for study recruitment without the need for additional procedures, thereby minimizing the burdens of subject enrollment. Point prevalence studies will also typically satisfy the criteria for a waiver of informed consent and assent (6, 7). While individual informed consent is a core requirement for ethical human subjects’ research, waiver of consent is generally accepted for epidemiological studies in order to reduce selection bias (26-30).

Point prevalence studies in pediatric critical care have enhanced collaboration of a global network of investigators dedicated to improve outcomes for critically ill children through participation in research (31). Thus far, it has been feasible for sites with limited resources to participate and contribute meaningful data. Such broad global collaboration will help to understand the role of local factors in disease prevalence and resource utilization for critically ill patients. For example, a recent study from the Surviving Sepsis Campaign highlighted differences in ICU utilization for treatment of adult sepsis between the United States and Europe that could impact mortality (32). Comparisons across health care systems can also help shed light on the best approach to diagnosis and treatment.

A disadvantage (pitfall) of point prevalence studies is the limited ability to infer conclusions about “cause and effect”. As with all cross-sectional data, the co-existence of risk factors and disease at a single time point make it difficult to know which occurred first. Thus, point prevalence studies remain primarily descriptive as opposed to more analytical designs, such as case-control and cohort studies, that follow patients over time.

A second disadvantage is that obtaining a “snapshot” in time may underestimate the burden of rapidly fatal diseases and infrequently used therapies. Moreover, data collection limited to prevalent cases may inhibit a complete understanding of risk factors for death, as prevalent cases are more likely to be survivors. One strategy to minimize such potential biases is to limit enrollment and data collection only to those with active disease (i.e. the criteria for the disease are met on the study day), rather than all those alive with a history of the disease. In this way, the point prevalence estimate will reflect only active disease rather than all prevalent cases, the latter of which will overestimate survivors compared to non-survivors. The PALIVE and Sepsis PRevalence OUtcomes and Therapy (SPROUT) studies adopted this latter approach.

Another potential pitfall is that differences in epidemiology, treatments, and outcomes across geographic regions and health care systems, while key to understanding how local factors influence disease, can make it challenging to apply study results to a specific institution (33). Additionally, many point prevalence studies use PICU hospitalization to define the population of interest (denominator in prevalence estimate) but criteria for PICU admission can vary considerably. While this may skew prevalence estimates, it nonetheless represents the true burden of disease within the study PICUs—useful data when considering sample size estimations or resource allocation.

Design and Implementation of a Point Prevalence Study

Table 1 summarizes key elements for design and implementation of point prevalence studies. In addition, several critical points are summarized in this section.

Table 1.

Considerations for Designing and Implementing a Point Prevalence Study

Design Element	Comments
Study objective	The primary objective should be to define the point prevalence of a condition (e.g., disease, risk factor, therapy). Secondary objectives may describe characteristics in affected persons.
Study population	Clear, objective, and obtainable criteria are needed to define both the numerator (persons affected) and the denominator (population of interest). Any patient included in the denominator should have the potential to be included in the numerator, i.e. be at risk for the condition of interest. Inclusion and exclusion criteria must be well-delineated in order to minimize variability across sites.
Data variables	Data elements to be collected should be clearly defined in a written protocol. Value of information needs to be balanced against time and effort needed to accurately obtain data.
Data collection tools	Sites will need to record data on screening and affected patients. A web-based Case Report Form (CRF) can improve efficiency but confidential transmission of data needs to be ensured.
Pilot phase	Performing screening and data collection at a small number of sites can identify errors, ambiguities, sample size estimates, and expected time and effort for site participation. Practice scenarios can be used to improve familiarity with screening, data collection, and data entry.
Site recruitment	Networking through social media and clinical research networks, and advertising in critical care society newsletters can facilitate access to sites and improve participation.
Site communication	Efficient communication can be achieved with use of a dedicated study email account and website.
Ethics oversight & approval	Seek approval of study documents early at the primary site and allow at least six to nine months for local site approval. A “centralized” oversight committee can minimize burdens of redundant protocol review at multiple sites and hasten approval of study amendments. If ethically acceptable, a waiver of consent can reduce selection bias and improve accuracy of epidemiological data.
Statistical consultation	Decisions about data elements, sample size, and analytical plan should be established during the planning stages of the study.
Sample size	Sample size is typically estimated by determining the acceptable variability around an expected prevalence. This estimate will determine the number of patients, sites, and study days needed.
Data quality	Data entry should be monitored for quality and completeness. Tools built within electronic databases can help to query missing data and outliers.
Manuscript development	Guidelines for manuscript authorship should be made transparent at the outset. A committee to oversee access to the dataset for additional research questions should be established.

The primary objective of a point prevalence study is to define the proportion of affected persons in a population at a specific time. It is reasonable to also describe the characteristics of the affected persons at that point in time. For example, the Prevalence of Acute critical Neurologic disease in children: a Global Epidemiological Assessment (PANGEA) study obtained data regarding etiology of neurological disease, strategies to monitor and treat intracranial pressure, and use of rehabilitative therapies to determine practice variability for children with acute neurologic injury (9).

Well-defined criteria are needed to identify patients to be included in both the numerator (persons affected by condition of interest) and the denominator (population of interest). In addition, all variables to be collected should be defined in a data dictionary with explicit definitions. A web-based Case Report Form (CRF) can improve efficiency in data entry and provide safeguards for patient confidentiality, such as restricting investigators to only view data from their own site (34).

A pilot phase at a small number of sites can identify errors and ambiguities in study materials, as well as inform sample size calculations by supplying preliminary prevalence estimates. A pilot phase can also provide estimates of the time expected for screening and data collection (Table 2). Over the course of the study, data should be monitored for quality and completeness. While on-site monitoring is typical not feasible in large multicenter point prevalence studies, other approaches are useful such as double-data entry with interrater reliability testing, embedded electronic alerts within on-line CRFs that notify sites of missing data, and verification of data that fall outside of pre-defined ranges. In the SPROUT study, proficiency in data entry was determined by having sites enter data from two standardized practice scenarios that could be compared to known values. Pre-specified data quality checks were then performed after each study day to identify missing and erroneous values, for which sites were queried for correction.

Table 2.

Data from the pilot phase of SPROUT: Sepsis PRevalence, OUtcomes, and Therapies^a

Number of Sites	4
Screening
Total patients screened	82
Mean patients screened per site	21
Mean time for screening per patient	8.5 minutes (range 4-16)
Enrollment and Data Collection
Total patients enrolled (prevalence of severe sepsis or septic shock)	15 (18%)
Mean patients enrolled per site	4 (range 1-7)
Time needed for data collection per patient, mean	91 minutes (range 41-141)
Inter-rater Agreement
Percent agreement in double-data entryb	92%

^aInstitutional Review Board approval was either waived or expedited at each site

^bDuplicate data entry was performed for 1 enrolled patient at each site.

We estimate six to nine months for ethics approval and data sharing agreements to be confirmed at each site. In the United States, there is increasing interest in establishing a centralized process for Institutional Review Board (IRB) approval, whereby a single site would provide ethical oversight for multiple institutions (35, 36). A centralized process is likely to reduce administrative costs without infringing on the rights of human subjects (37). The PALIVE, PANGEA, Saline and Albumin Fluid Evaluation trial Extrapolation to Pediatric Intensive Care (SAFE-EPIC), and SPROUT studies all, in part, utilized a centralized IRB process.

Finally, guidelines for manuscript development and authorship should be made transparent. Although it is impractical to offer authorship to the dozens or even hundreds of site investigators, it is important to provide credit commensurate with participation, such as by listing participants in the manuscript appendix. In addition, a process for site investigators to access the dataset for related study questions should be delineated.

Measuring Patient Outcomes

The primary objective of a point prevalence study is to define the proportion of affected persons present in a population at a specific time. PALIVE, for example, determined the proportion of pediatric patients with ALI; the Prophylaxis against Thrombosis Practice (PROTRACT) study determined the proportion of critically ill children receiving thromboprophylaxis and compliance with recommended guidelines. However, neither study measured patient outcomes, which has hindered the ability to perform power calculations for interventional trials. With some methodological modifications, it is possible to collect data on short-term outcomes in point prevalence studies. The PANGEA, SAFE-EPIC, and SPROUT studies (Table 3) include data on a number of outcomes including hospital mortality, organ dysfunction, length of stay, and neurocognitive function. Although not part of classic point prevalence studies, efforts to estimate mortality and morbidity outcomes will greatly benefit future trial design in pediatric critical care and we highly recommend these be considered (38, 39). Indeed, such data has recently been requested by the National Institutes of Health prior to funding a large pediatric clinical trial (40).

Table 3.

Recent Point Prevalence Studies in Pediatric Critical Care

Study Name	PALIVE	PANGEA	PROTRACT	SAFE-EPIC	SPROUT
Topic	Acute lung injury	Acute neurologic disease	Thromboprophylaxis	Fluid resuscitation	Severe sepsis/septic shock
Year	2007	2011-2012	2012	2012-2013	2013-2014
Number of Sites	59	107	59	75	128
Number of Countries	12	23	7	28	28
Number of Study Days	6	4	4	2	5
Patients Screened	3823	3113	2909	1815a	6317a
Prevalence Measure	10.8% ALI by clinician diagnosis; 4.3% ALI by published guidelines	18.7% prevalence of acute neurologic disease; 12.4% hospital mortality	12.4% prevalence of pharmacologic thromboprophylaxis	Prevalence of ≥ 5 ml/kg of intravenous fluid in one hour	Prevalence of severe sepsis and septic shock defined by clinical diagnosis or consensus criteria
Treatment Data	Mechanical ventilation strategies, adjunctive therapies	Medical and surgical therapies for brain injury, infection, and intracranial hypertension	Type of pharmacologic and mechanical thrombo-prophylaxis	Volume, rate and type of fluid, indication, and rationale for administration	Antimicrobials, vasoactive infusions, mechanical ventilation, corticosteroids, ECMO, CRRT, plasma exchange
Outcome Data	None	LOS, neurocognitive function, mortality	None	LOS, mortality	NPMODS, LOS, neurocognitive function, mortality
Duration of Follow-up	NA	Hospital discharge or 90 days	NA	Hospital discharge or 28 days	Hospital discharge or 90 days
Centralized Ethics Committee	Available in France	Available in France	Not available	Available in France and UK	Available in USA, UK, and Brazil

PALIVE: Pediatric Acute Lung Injury Ventilation; PANGEA: Prevalence of Acute critical Neurologic disease in children: a Global Epidemiological Assessment; PROTRACT: Prophylaxis against Thrombosis Practice; SAFE-EPIC: Saline and Albumin Fluid Evaluation trial Extrapolation to Pediatric Intensive Care; SPROUT: Sepsis Prevalence, Outcomes, and Therapies; ECMO: extracorporeal membrane oxygenation; CRRT: continuous renal replacement therapy; LOS: length of stay; NPMODS: new or progressive multiple organ dysfunction syndrome; USA: United States of America; UK: United Kingdom

^aEstimated

Statistical Analysis

Point estimates of the prevalence should be reported with a measure of variability, such as 95% confidence interval. It is important to report the criteria used to define both the numerator and the denominator to allow future comparisons. The denominator may include all patients in the PICU during the study or only patients with a risk factor of interest. All data from each site and study day should be combined to calculate an overall prevalence. When studying the point prevalence of a disease, adjusting for study day is necessary given the seasonal variation of diseases in critically ill children. Alternatively, study days may be distributed evenly across the calendar year to minimize this confounder. When studying the point prevalence of a therapy, controlling for site may be more important because of clustering of treatment practices. Common statistical methods of adjustment include stratification by site (if sample size per site is large enough) or more commonly by geographic region, or the use of mixed effects regression models to account for clustering by site or region.

Site Recruitment

Establishing a network of committed sites is paramount. The aim of the network should be to expand the study population in number, geographic diversity, and range of health care systems in order to enhance the generalizability of the study results and facilitate direct comparisons of healthcare delivery strategies for critically ill children. The point prevalence studies in pediatric critical care have included 59 to 120 sites in seven to 28 countries. It is best to engage sites early in the study design process to ensure that the research is relevant across regions and to help local investigators become active stakeholders in the success of the study. We have found it beneficial to contact the leaders of established research networks, especially since critical care research networks are now active on all continents.

Ideally, a stable network of international sites that are committed to point prevalence studies would be established and maintained rather than rebuilding a new network with each protocol. An efficient network could simply move from one disease or condition of interest to another in a cyclical process, with a new study beginning as the last one ends. Consideration should also be given to ethics approval of a comprehensive or “umbrella” point prevalence protocol that would avoid delays and administrative burdens imposed by seeking ethics approval and data sharing agreements for each new topic. In this way, the global epidemiology of pediatric critical illness could be continuously re-assessed while reducing the time and effort needed to start each study. An example of a coordinated point prevalence program which achieves these goals is the ANZICS Point Prevalence Program which was established in 2009 (41).

Point Prevalence Studies in Pediatric Critical Care

Table 3 summarizes the five largest international point prevalence studies in pediatric critical care since 2007. Further details of PALIVE and PROTRACT have been previously published (8, 10), and interim results of PANGEA and SPROUT have recently been presented at scientific meetings (9, 12). PANGEA, SAFE-EPIC, and SPROUT will be submitted for peer-review once data analysis has been completed.

Future Directions

Point prevalence studies in pediatric critical care are providing new insight on the epidemiology of disease and practice patterns for children with critical illness. These studies demonstrate that rapid, prospective, and high-quality data collection can be obtained from patients at a large number of sites across geographic regions and health care systems. Familiarity with the design, implementation, and analysis of these epidemiological studies are important for clinicians, scientists, and other stakeholders to effectively use these data to identify research priorities, design clinical trials, track disease-specific metrics, develop guidelines for diagnosis and therapy, and allocate health care resources in pediatric critical illness.