Design and Rationale for Common Data Elements for Clinical Research in Pediatric Critical Care Medicine

Abstract

Objective:

Common Data Elements (CDEs), are a combination of a precisely defined question paired with a specified set of responses. CDEs contribute to the NIH-supported principle of FAIRness (Findable, Accessible, Interoperable, and Reusable) of research data. Routine use of CDEs and standardized definitions within pediatric critical care research is likely to promote collaboration, improve quality and consistency of data collection, improve overall efficiency of study or trial setup, and facilitate cross-study comparisons, meta-analysis and merging of study cohorts. The purpose of this PCCM Perspective is to establish a road map for the development of multinational, multidisciplinary consensus based CDEs that could be adapted for use within any pediatric critical care subject area.

Methods:

We describe a multi-step process for the creation of “core domains” of research (e.g. patient outcomes, health related conditions or aspects of health), and the development of CDEs within each core domain. We define a tiered approach to data collection based on relevance of each CDE to future studies and clinical practice within the field of interest. Additionally, we describe the use of the Delphi methods to achieve consensus of these CDE documents using an international, multi-disciplinary panel of experts.

Uniform Data Collection is Essential for The Advancement of Pediatric Critical Care

Pediatric critical illness can be devastating for patients and families, and it is imperative that the scientific community produce high quality evidence to guide clinical practice. The relatively small number of patients in each Intensive Care Unit renders the need for multi-center collaborations and combining of data to generate the highest level of evidence. Standardization of data definitions and collection has become a priority for The National Institute of Health (NIH) and other funders and research networks(1).

Common Data Elements (CDEs) are a combination of a precisely defined question (variable) paired with a specified set of responses. CDEs contribute to the NIH-supported principle of FAIRness (Findable, Accessible, Interoperable, and Reusable) of research data(2-4). Routine use of CDEs within a field of clinical research is likely to promote collaboration, improve quality and consistency of data collection, improve overall efficiency of study or trial setup, and facilitate cross-study comparisons, meta-analysis, and merging of study cohorts to create larger, more generalizable patient datasets. In this perspective, we describe our methods for the development and utilization of internationally supported CDEs within pediatric critical care medicine. At present, these methods are being applied to two pediatric critical care CDE projects, pediatric acute respiratory distress syndrome (PARDS) and nutritional assessment and needs during critical illness.

A Delphi Method Based International Consensus Approach for CDE Development

The methods we describe expand on those used by the National Institute of Neurological Diseases and Stroke (NINDS)(5, 6) by providing a structured hierarchical approach to CDE development and very clear, step-by-step instructions. Our methodology incorporates the expertise of physicians, scientists, research coordinators, data scientists, and statisticians; and is structured to be successfully utilized in any field of interest and be completed in the absence of funding or sponsorship. Figure 1 provides a summary diagram of the methods described below.

Figure 1.

Steps for common data element development within a given field of interest using the Delphi method for consensus development

Primary Panel

A primary panel of multinational, multi-disciplinary researchers with expertise in the area of interest should be formed. Responsibilities of this panel include data review of previous clinical studies, identification of core domains of data essential for all clinical research studies within the area of interest, development of a tiered, well-defined approach to determining importance of each variable within the core domains, and recruitment of a diverse group of experts for consensus purposes.

Development of core domains and data collection forms

Data collection forms from past clinical studies and trials within the field of interest are collected and reviewed by the primary panel. Determining what variables are commonly collected in pertinent studies will explicate the core domains of data essential for study within the field of interest. A "core domain” is defined as a patient outcome, health-related condition, or aspect of health that is essential to evaluate within a clinical field(7, 8). Example domains include demographics, prior comorbidities, admission data, severity of illness, disease or condition specific data (e.g. diagnostic criteria), and patient outcomes and hospitalization summary. For each identified core domain, a data collection form (DCF) should be created. The CDEs included in these DCFs are those determined from the data review. The primary panel is tasked with including a structured set of responses and explicit instructions and definitions for each CDE to ensure consistent data collection or timing of data collection.

Before these data collection forms can be presented to a larger panel for expert consensus, all members of the primary panel should review and vote on each data collection form. The Delphi method uses expert opinion to address questions for which empirical data are unavailable or inadequate(9). This method is also frequently used for consensus of core outcome sets for a specific field. Based on this method, the new CDEs and data collection forms for each core domain are voted on by the members of the primary panel. Their anonymous responses are aggregated and shared with all members of the primary panel and the results are collectively discussed. In subsequent voting rounds, members may adjust their votes based on how they interpret the group response. The voting and discussions can be done by in-person meetings, web-based video or teleconference or by web-based survey and discussion board applications. With each voting round, the primary panel members should vote on the clarity of provided instructions and definitions, the format of each CDE, and the relevance of each CDE for future studies and trials.

A three-tiered approach is used to determine relevance of each CDE for future studies within the field of interest (Figure 2). Tier 1 elements are those essential for all studies within the field, forming the ultimate "minimal data set". Tier 2 elements are supplemental CDEs, and tier 3 elements are more exploratory. The tier of importance of each CDE is noted in the DCF next to each CDE. For our CDE projects, we defined tier of importance agreement to be ≥80% agreement among the panel members. When the primary panel does not reach 80% agreement for a given CDE, that information, as well as any comments provided by individual panel members, are provided to the expert panel to inform their voting decision. Figure 3 provides a sample of three CDEs from a hospitalization summary domain as well as the survey questions for each CDE; a similar infrastructure will be utilized for expert panel consensus voting.

Figure 2.

Definitions of common data element (CDE) tiers of importance used for development of consensus-based CDE documents

Figure 3.

Sample common data elements and the associated survey questions to be presented to the primary and expert panels for consensus voting

Recruitment of an international multidisciplinary expert panel

A diverse, multinational, multi-disciplinary expert panel is essential for determination of consensus for the use and structure of CDEs. Identification of such experts can occur via several mechanisms: (i) review of relevant published articles from the past 5-10 years; (ii) invitation of national and international research network members and collaborators; and (iii) invitation of research coordinators, biostatisticians, and other members of the research/clinical community with experience in data collection and analysis. The target number of participants can vary depending on the scope of the project but should balance adequate representation of opinions with a manageable number of participants to achieve consensus. For our CDE projects, we have found 6-8 primary panel members and 25-35 expert panel members to be appropriate.

Achieving Expert Panel Consensus

Using the Delphi method, two rounds of voting within the expert panel occur. In round 1, the expert panel members receive the core domain CDE documents and the notations of agreement and disagreement that arose during primary panel voting. Similar to the primary panel, the expert panel must anonymously vote on clarity of provided instructions and definitions and relevance of each data element. CDEs which do not reach agreement (≥80%) regarding tier, format, or definition from the first round of voting are reviewed by the primary panel for rewording/clarification. Round 2 expert panel voting focuses only on those CDEs that do not reach agreement in Round 1. All expert panel members will receive the voting results from Round 1, comments pertinent to areas of disagreement, and the re-worded CDEs. Each member is provided the opportunity to revise their vote. CDEs that do not reach a level of agreement after 2 rounds of expert panel voting will be recorded as areas with insufficient evidence for consensus.

Dissemination of CDE Documents:

The finalized documents should be published in journals relevant to the field of interest. The core domain data collection forms with all tier 1, 2 and 3 CDEs must be made freely available on network websites so that they are easily accessible to all researchers within the field. Additionally, all the CDEs can be converted to HIPAA-protected electronic data capture formats, such as Research Electronic Data Capture (REDCap)(10). Because advances in medical management and study design occur at a rapid pace, members of the primary and expert panel should continue to review and adjust these documents approximately every five years. Vetting of any new or revised CDEs should use similar methods to provide quality assurance of the system.

Conclusion

Similar to Core Outcome Sets, which aim to improve the consistency and standardization of important outcomes for a given field, CDEs improve the consistency and standardization of minimum data to be collected. The use of CDEs, especially when they conform to accepted standards, can facilitate cross-study comparisons, improve overall efficiency; promote collaboration; and improve the quality of data collection.