Common data elements for preclinical epilepsy research: standards for data collection and reporting. A TASK3 report of the AES/ILAE Translational Task Force of the ILAE

Summary

The major objective of preclinical translational epilepsy research is to advance laboratory findings towards clinical application by testing potential treatments in animal models of seizures and epilepsy. Recently there has been a focus on the failure of preclinical discoveries to translate reliably, or even to be reproduced in different laboratories. One potential cause is a lack of standardization in preclinical data collection. The resulting difficulties in comparing data across studies have led to a high cost and missed opportunity which in turn impedes clinical trials and advances in medical care. Preclinical epilepsy research has successfully brought numerous anti-seizure treatments into the clinical practice, yet the unmet clinical needs prompted the reconsideration of research strategies to optimize epilepsy therapy development.

In the field of clinical epilepsy there have been successful steps to improve such problems, such as generation of common data elements (CDEs), case report forms (CRFs and standards of data collection and reporting) by a team of leaders in the field. Therefore the Translational Task Force was appointed by the International League Against Epilepsy (ILAE) and the American Epilepsy Society (AES), in partnership with the National Institute of Neurological Disorders and Stroke (NINDS) and the National Institutes of Health (NIH) to define CDEs for animal epilepsy research studies and prepare guidelines for data collection and experimental procedures. If adopted, the preclinical CDEs could facilitate collaborative epilepsy research, comparisons of data across different laboratories, and promote rigor, transparency and impact, particularly in therapy development.

Introduction

Epilepsy research has been among the most successful research areas, since there have been numerous anti-seizure drugs and treatments that have successfully entered clinical practice. However, there are still unmet clinical needs such as the lack of anti-epileptogenic or disease modifying treatments and the need to find therapies for seizures that are either drug resistant or do not respond to available treatments.¹ While much work has been done within the field of epilepsy to translate preclinical experimental paradigms, practices and procedures into targeted treatments for epilepsy, challenges remain due to a persistent lack of standard experimental designs and data collection practices.² This lack of standardization in research practices can make it difficult to interpret data and compare results between studies, laboratories and research institutions. The resulting inability to synthesize data often has led to difficulty in interpreting data, delaying advancement of research from a preclinical to clinical stage, increasing the length and cost of clinical trials, and prolonging the development of epilepsy treatments.²

Recent efforts to address the lack of standardization in epilepsy research have focused on promoting the usage of common data elements (CDEs). CDEs are standardized terms used to collect and record data as a way to systematically collect information essential to research. CDEs are increasingly used in clinical research,³ and the National Institute of Neurological Disorders and Stroke (NINDS) has developed a set of CDEs for clinical studies in epilepsy. Work must still be done to promote the widespread use of CDEs in a preclinical research setting. To achieve this, the AES/ILAE Translational Research Task Force of the ILAE is working towards a publically available database of CDEs to be used in preclinical epilepsy research, with the overall goal of standardizing data collection, facilitating data analysis and speeding the transition from epilepsy research to the development of epilepsy treatments. We are using the term translational here to refer mainly to animal studies that aim to identify therapies, biomarkers or other discoveries for which there is an interest in bringing them into clinical testing. In this review, we discuss the background and principles behind CDEs, the prior experience with the clinical CDEs, the community feedback received during an open forum organized by the TASK3 group at the last AES meeting in Houston and an overview of the preclinical epilepsy CDEs that are being generated by TASK3 of the AES/ILAE Translational Research Task Force of the ILAE, in partnership with NINDS.

CDEs: Standards for Experimental Practices

CDEs exist as a set of scientifically-vetted standards for conducting experiments and collecting information that is common to data sets across different studies.^4,5 They provide specifically-named and defined variables to be collected (i.e., data).^6,7 By providing a template for the collection of data, CDEs improve accuracy and ease of data collection and recording, as well as sharing data across labs.

Importantly, CDEs can be made accessible to the wider research community, providing a method of standardizing experimental practices across laboratories and institutions, and increasing transparency of studies conducted in different settings. By creating standards for data collection and ensuring that the same essential information is collected in all clinical or preclinical studies, CDEs facilitate experimental design and transparency, allowing for cross-study and cross-laboratory comparison and analysis of data sets.⁶ Adoption of a widespread system of CDEs also has the potential to reduce inter-experimenter variability by providing a common template for experimental procedures.⁴ Furthermore, by creating opportunities for data aggregation and analyses of multiple datasets, CDEs can speed the progression of research from preclinical to clinical, accelerating the creation of epilepsy treatments and decreasing the cost of therapy development.⁶ Moreover, in the current era of multicenter grants, which foster collaborations, CDEs are essential for consistent data collection and recording between groups.

Terminology

There are several important terms, defined below, that are often used when discussing the concept of CDEs.

CDEs

CDEs are the variables that should be collected for any given study.

There are three categories of CDEs (adapted from ⁷):

1. Core CDEs: essential information that should be collected across studies of all types (e.g., species, age).

2. Supplemental CDEs: information that is relevant to certain types of studies, but not others (e.g., type of EEG electrode or behavioral task).

3. Exploratory CDEs: CDEs that are not yet validated (under review).

Case Report Forms (CRFs)

CRFs are specialized documents specifically designed for the reporting of CDEs. CRFs systematically organize CDEs by topic into different modules (e.g., core animal characteristics or anti-epileptogenesis treatment).⁸ By providing a standard template for the collection of CDEs, CRFs aid in the systematic collection and organization of data.⁸

Data Dictionary

Tables of terms to be used when collecting data that is entered on a CRF, containing definitions of all terms.¹⁰ The data dictionary also indicates acceptable formats for the input of a CDE (e.g., if the CDE was “date”, the data dictionary would indicate that it should be entered as day-month-year).

The Use of CDEs in Clinical Epilepsy Research

The National Institutes of Health (NIH) has recognized the importance of collecting CDEs in scientific research for several years,⁶ and has issued a call for the widespread use of CDEs in “clinical research, patient registries, and other subject research” in an effort to improve data quality.¹¹ To date, much work has been done to promote the use of CDEs in research conducted in a clinical setting in various fields.⁶ For example, CDEs are available on the NINDS website for numerous disorders, including cerebral palsy, multiple sclerosis, Parkinson’s disease and traumatic brain injury (TBI).³ In particular, the CDEs available for TBI, first published in 2010, are quite expansive; documents available include CDEs, CRF modules, guidelines to provide information regarding the CDEs and a list of core elements to include when beginning TBI studies.¹²

NINDS has also been at the forefront of efforts to create and utilize CDEs in clinical epilepsy research. The NINDS began its CDE initiative in 2005 as a project to address inconsistencies in reporting and serve as a guide to the best research and data collection practices, as well as to maximize the results of NINDS-funded research.^4,13 The aims of the project are to disseminate standards for data collection and to create accessible tools to simplify data collection and improve data quality,¹³ with encouragement for the incorporation of CDEs into grant proposals and Phase-III clinical trials.⁴ A current list of CDEs for clinical epilepsy research as well as a list of CDEs under review and in development is available on the NINDS website (https://commondataelements.ninds.nih.gov).³ As with TBI CDEs, documents are organized into CDEs, CRF modules and guidelines, and sub-domains include demographic information, imaging diagnostic information, information on drugs and devices as well as outcomes and end points.¹⁴ All researchers who receive NINDS funding are asked to use CDEs in their collection of data, and CDEs are currently in use in several NINDS-funded clinical epilepsy studies.³ The use of CDEs is not limited to the NINDS; there have been collaborative efforts between NIH institutes, non-profit foundations and pharmaceutical companies to increase standardization in clinical epilepsy research via the use of CDEs.¹⁵ This has already led to improvement in comparability between studies. For example, use of the “Etiologies CRF” in randomized controlled trials of new antiepileptic drugs will allow better description of populations enrolled in different trials, to determine if there are substantial differences from trial to trial. Moreover, it may be possible to pool data from different trials, to determine if drugs are more efficacious for patients with certain etiologies (e.g., traumatic brain injury or cortical dysplasia) as compared to others. Before the use of CDEs, this would not have been possible.

Although researchers conducting clinical trials were initially concerned about the added burden that such forms would produce, the CDE forms have thus far been adopted with little evidence of concern from the community.

The Use of CDEs in Preclinical Epilepsy Research

While there has been much focus on the adoption of CDEs in a clinical research setting, less has been done to make the collection of CDEs standard practice in preclinical epilepsy research. Much of the preliminary work on creating publically available CDEs for preclinical epilepsy has been conducted by the FP7 funded EPITARGET Research Consortium, which has made available the first set of CRFs and CDEs for preclinical epilepsy studies as a protocol for the harmonization of the European Preclinical Biomarker Bank.⁸ To date, EPITARGET has developed a database of CDEs with the goal of using them as a tool in the identification of novel biomarkers for epileptogenesis and the creation of new treatments for epilepsy.^8,16 EPITARGET now mandates the use of these CDES for every animal that is included in the EPITARGET biomarker analysis study.¹⁶

In addition to work done by EPITARGET to promote the use of CDEs in EPITARGET epilepsy biomarker studies, CDEs are often used by individual researchers in their laboratories in the form of information that is commonly collected in studies. However, there remains the need for a broader system of CDEs to address studies conducted in the wider preclinical epilepsy field.

NINDS has also been at the forefront of efforts to increase rigor and transparency in preclinical research,¹⁷ and CDEs are a critical tool to be used to accomplish these goals. To this end, in addition to its role in the development of clinical CDEs, NINDS has been supportive of efforts to create preclinical CDEs in fields such as TBI where preclinical CDEs were first published in 2015.¹⁸ CDEs for preclinical TBI, organized into 10 different modules (i.e., Core CDE Module, Head/Brain Impact Models) and four different domains (i.e., Animal Characteristics and Animal History) are available on the NINDS website.^{19, 20} TASK3 of the Translational Research Task Force aims to expand upon these efforts by providing CRFs for more than 60 different preclinical epilepsy modules in multiple domains (see below).

Development of Preclinical CDEs by the AES/ILAE Translational Research Task Force of the ILAE

The third initiative of the AES/ILAE Translational Research Task Force (TASK3) is to create, vet and promote the adoption of CDEs and standardized procedures and protocols for preclinical epilepsy,²¹ as well as to create standardized forms for data acquisition, with the following goals:

1. Standardization of experimental practices and procedures

The adoption of CDEs in preclinical epilepsy research will create a standardized system of data collection and reporting, optimizing the quality and utility of data collected. CDEs will serve as a valuable guideline for the standardization of experimental design, information that should be collected and parameters to be analyzed for every epilepsy study. It is important to note that certain aspects of preclinical epilepsy research do benefit from the variability that may be similar to what is found in a heterogeneous human population; for example, the use of both male and female animals, different genetic strains and diverse environments all add to our understanding of outcomes presented by novel approaches. What is important, however, is that the experimental methods are consistent and the resulting data is collected and reported in a standardized fashion. In this way, standardization will improve cross-laboratory comparison and collaboration.

2. Harmonization of procedures where possible

The use of CDEs will increase transparency of procedures within preclinical epilepsy research by providing recommended experimental paradigms and methodological guidelines. Currently, disparate experimental paradigms and techniques are often used to address the same question. While a multi-faceted approach to preclinical epilepsy research is often ideal, it is also true that the harmonization of experimental procedures, where possible, would aid in experimental reproducibility and data interpretation. The difficulty of harmonization is apparent when one considers the multitude of animal models for seizures or epilepsy that are used in the development of anti-seizure, anti-epileptogenic and anti-comorbidity treatments.^22,23 For example, there are several different animal models of drug-resistant temporal lobe epilepsy in adult animals, including lamotrigine-pretreated kindled rats, and post-status epilepticus epilepsy models.²² In addition, amygdala kindling is an animal model that has utility in screening for drugs that may be useful in drug-resistant epilepsy. In the search for epilepsy treatments, variable data arises as a result of differences in the animal model that is chosen and/or experimental practices, making it difficult to demonstrate that a particular treatment is effective in preventing epileptogenesis.²⁴ Standardization and transparency of research practices through the use of CDEs will allow for greater synthesis of research conducted across laboratories, increasing the ability to draw critical conclusions from data and promoting the development of effective treatments for epilepsy.

3. Availability of procedures for examination when results are published

CDEs, collected via CRFs, can be stored in a database available for later examination once results are published. These CRFs will contain essential, detailed methodological information, and will allow anyone to review how a specific experiment was performed, which can illuminate the difference in results between two labs.

Currently, important methodological details are often omitted from scientific publications. Despite the availability of basic guidelines for study design and methodology outlined in the ARRIVE guidelines, a twenty-point checklist of general essentials for animal experimental procedures,²⁵ the lack of specific details available in many preclinical epilepsy reports can make it difficult for researchers to interpret results and to reproduce and build upon previous research. The negative impact that this lack of accessible procedures can have is evident when one examines research addressing sex differences in epilepsy, for example. Despite evidence that sex can affect various aspects of epilepsy such as the prevalence of epilepsy and seizure susceptibility (see ²⁶), sex differences in epilepsy are not well-elucidated. This may be because of inconsistencies in data collection and reporting – for example, there may be procedural differences in determining estrous cycle stage in female animals or the pooling of different types of epilepsy during analysis,²⁶ and the criteria for both may not be readily apparent when reading published works. The usage of CDEs specific to preclinical epilepsy research will help solve this problem by ensuring the proper recording and reporting of data, making interpretation and data analysis easier.

CDEs in Preparation by the AES/ILAE Translational Research Task Force of the ILAE

CDEs for preclinical epilepsy are currently being created. There is one core CDE and 5 supplemental CDEs: (pharmacology, physiology, behavior and electroencephalography (EEG), with work groups comprised of experts devoted to the creation of each. CDEs in each category are organized into CRF modules (e.g., respiration CRF modules for physiology CDEs).

CRFs in each category follow a similar format, containing the name of the CDE to be collected as well as a space in which to input the collected information. Figure 1 provides an example of the Core CRF module, which consists of data that is essential to collect for all types of preclinical epilepsy studies in order to optimize synthesis and standardization of results at the most basic level. This information includes animal ID, date of birth, animal source, animal sex, species, and housing (Figure 1). The intent is that CRF forms are clear and concise, yet contain the most pertinent information for any particular experimental procedure, in order to promote maximal use amongst researchers. CDEs contained within these CRFs can be filled out in the form of a checklist or using drop-down tabs; options to be selected will be binary, wherever possible, in an effort to make the forms as easy-to-use and intuitive as possible.

Figure 1.

An example of a Case Report Form (CRF) for Core CDEs is shown. The form gives researchers a template so that they can easily enter information about their animal subjects.

Additionally, guidelines will be included in the public database detailing suggested methods for conducting specific types of tests (e.g., a guideline for testing respiration). Guidelines will include information about how to implement methods, pros and cons of methods, details about performing methods, and references.

CDEs, CRFs and guidelines for preclinical epilepsy are subject to internal vetting by members of TASK3 as well as by other members of the Translational Research Task Force. CDE vetting has been an ongoing process since the start of the CDE project. Following internal vetting, CDEs, CRFs and guidelines will be open to vetting by the wider scientific community. CRFs will then be publically available for use in all preclinical epilepsy studies via open-access online forms. Guidelines will also be available online.

Summary of the TASK3 Open Forum to Discuss Preclinical CDEs

As an initial round of public vetting, TASK3 hosted an Open Forum to discuss preclinical CDEs on December 6, 2016 in Houston, Texas. The goal of the Open Forum was to present the objectives and work-to-date of TASK3 as well as to obtain feedback from a wider group of individuals interested in the creation of CDEs for preclinical epilepsy. Following an initial presentation outlining the goals of TASK3 and an overview of the definition and purpose of preclinical CDEs, CRFs and guidelines, audience demographic information and initial feedback was obtained. Feedback was obtained by real-time polling through the Audience Response System (ARS; polleverywhere.com) as well as through general discussion during the Open Forum. Members of the Task Force (TASK3) directly responsible for creating preclinical epilepsy CDEs, CRFs and associated documents (including Open Forum speakers) did not participate in the ARS poll. However, other individuals affiliated with the other TASKS of the Translational Task Force may have participated in polling; due to the anonymous nature of polling, specific percentages of Task Force-affiliated attendees participating in the poll are not available.

Results of the ARS poll indicate that the Open Forum attendees who responded to the ARS question about demographics consisted of a cross-section of basic scientists (29%), clinical researchers (38%), individuals falling into both categories (10%) and those identifying as ‘other’ (24%) (n=21; Table 1A1). 24% of respondents had been involved in research for less than 10 years, 33% for 11–20 years, and 43% for more than 20 years (n=21; Table 1A2). While a majority of respondents had never used CDEs in their research (52%, n=21; Table 1A3), 75% of respondents indicated they were very likely to consider using CDEs in the future, with an additional 21% reporting they might use them if provided more information; only 4% were not considering the use of CDEs (n=24; Table 1A4). In order to determine the prevalence of big database usage, attendees were asked if they had been depositing data into ‘big databases’ for sharing purposes; 41% of respondents indicated that they were, while 48% were not (n=27; Table 1A5).

Table 1.

Responses of the Audience to Questions During the TASK3 Open Forum to Discuss Preclinical CDEs.

A. Pre-Discussion Questions
Question		Positive responses (% of total)	Total responses (n)
1	What best describes your occupation?		21
	a. I do basic science research	29
	b. I do clinical research	38
	c. I do both basic and clinical science research	10
	d. None is applicable	24
2	For how long have you been involved in research?		21
	a. Less than 10 years	24
	b. 11–20 years	33
	c. More than 20 years	43
3	Have you used CDEs / CRFs in your research?		21
	a. Yes in clinical research	43
	b. Yes in basic research	5
	c. Yes in both clinical and basic research	0
	d. No	52
4	Are you considering using CDEs/CRFs in your research?		24
	a. Very likely	75
	b. Maybe, but I need more information	21
	c. No	4
5	Are you depositing data in “big databases” from your research for data sharing		27
	a. Yes	41
	b. No	48
	c. I will in the future	11
	d. I will not	0
B. Post-Discussion Questions
1	Are you likely to use CDEs / CRFs in your research?		8
	a. Very likely	38
	b. Maybe, but I need more information	25
	c. Maybe, but implementation is a barrier	25
	d. No	13
2	What would you consider barriers towards the use of CDEs / CRFs?		20
	a. It is time consuming	50
	b. I don’t have access to a database
	c. Cost	25
	d. Format is too complicated	5
	e. I have my own system of data collection	10
	f. These CDEs/CRFs are not applicable to me	5
3	What would encourage you to use CDEs / CRFs?		27
	a. If funding organizations recommended their use	7
	b. If funding organizations required their use	30
	c. If journals recommended their use	7
	d. If journal required their use	11
	e. If I were involved in a collaborative project that required their use	15
	f. If easy-to-use apps for CDEs/CRFs were available	30
	g. If I could get more familiar with their use	0
4	Does your lab have a systems analyst or access to one?		5
	a. Yes	0
	b. No	100
5	Would you consider using big databases in your research?		9
	a. Yes	56
	b. No	11
	c. Maybe	33

During the second segment of the Open Forum, TASK3 co-chairs from each WG (physiology in vivo, pharmacology, behavior and EEG) presented examples of their CDEs, CRFs and guidelines, and obtained critical feedback regarding the content of these forms. Audience members were then asked a second set of questions through the ARS, intended to reflect thoughts on preclinical CDEs following the afternoon of TASK3 presentations. Feedback on the use of CDEs for preclinical epilepsy was generally very positive, with 35% of respondents indicating they were very likely to use CDEs and 50% willing to consider using CDEs in their preclinical research provided more information and a lack of barriers to their implementation (n=8; Table 1B1). Perceived barriers included time (50%) and cost (25%) (n=21; Table 1B2). It is clear that some level of incentive would increase the use of CDEs, with 30% of respondents indicating they would use CDEs if funding organizations required their use and 30% reporting that easy-to-use apps for CDEs/CRFs would encourage their use (n=27; Table 1B3). Respondents also indicated they would use CDEs if they were involved in a collaborative project that used them (15%) or if journals required their use (11%) (n=27; Table 1B3). While there were fewer respondents for the final two questions, limiting the ability to interpret results, none of the individuals who did respond had a systems analyst available to them (n=5; Table 1B4), a factor that might make the implementation and maintenance of online CDE repositories and databases more difficult. Finally, 56% of respondents indicated they would use big databases in their research, with an additional 33% indicating they might use big databases, and 11% saying they would not (n=9; Table 1B).

Following the second set of ARS questions, a general discussion ensued that covered the following points:

1. The need to market the use of CDEs as a tool that will increase productivity by standardizing data collection, making lab work more efficient.

2. The hurdles to the use of CDEs, including the possibility that researchers will view the forms as too onerous to fill out and the general lack of a readily available systems analyst.

3. The possibility of making easy-to-use apps containing CDEs, CRFs and guidelines.

4. How forms would stay up-to-date because it will be necessary to update them regularly as new methods, definitions and interpretations evolve. The regular monitoring and updating could be viewed as too time-consuming and costly.

5. How errors in data entry can be avoided and corrected when they occur.

6. How to optimize the widespread implementation of CDEs in the field of preclinical epilepsy, including requiring they be used in order to obtain funding or to publish results.

Finally, attendees were asked to contact the Task Force if they had any further feedback regarding the creation of CDEs by TASK3, or were interested in participating in further vetting of the CDES.

Applications

The adoption of CDEs for preclinical epilepsy has several very important implications for preclinical epilepsy research, including facilitation of big data analysis, the publication of research, and grant funding. The increased standardization of experimental practices will facilitate data sharing, the pooling of data, and meta-analysis. CDEs contained in an online database will make the aggregation of multiple sets of data simpler and more feasible. The resultant large data sets can be examined via searches of key words or variables. Data may also be linked, integrated, and subject to exploratory analysis.⁵ This increased ability to combine and analyze large sets of data will increase statistical power and improve the predictive value of studies and collaboration between laboratories.

Finally, the adoption of CDEs may also have a positive impact on the quality and impact of publications, since they will facilitate the comparisons of data. Greater expediency of research due to the use of standardized forms and suggested guidelines would potentially decrease the amount of time it takes for a study to move from conception to completion. That in turn would potentially lead to the potential for more publications. At the same time, the rate of acceptance for published works would potentially increase and there might be improved quality and validity of studies.

We realize that adoption of preclinical CDEs will be a significant change in the way epilepsy researchers conduct and report their studies but will also require certain steps towards improving the infrastructure towards using them. TASK3 investigators plan to submit the currently prepared first set of preclinical epilepsy CDEs as open access publications at Epilepsia Open and solicit community feedback before finalizing. The ongoing dialogue and feedback with the research community will be critical in also addressing some of the obstacles in their implementation, e.g., creation of software or applications to accelerate data collection or workshops to disseminate and clarify their use.

Conclusions

The benefits of the widespread adoption of CDEs at least in preclinical translational epilepsy research are apparent: increased standardization, reproducibility and effectiveness. The use of CDEs represents an important change in the way that preclinical epilepsy research is conducted, with long-lasting benefits to individual research laboratories and the field as a whole. CDEs, CRFs and guidelines created by TASK3 of the Translational Research Task Force will provide detailed information on data to be collected, logically organized CRFs to be placed in an online database accessible to the public, and suggested guidelines for experimental procedures. The widespread adoption of these CDEs will truly standardize experimental research, increase transparency and allow for rapid advancement from preclinical epilepsy research to the creation of novel treatments for epilepsy. However, we also consider it important to engage the community in this process of optimizing the CDEs and the infrastructure towards implementing them (e.g., training courses, routes for community feedback, software and apps for using them) and minimize the burden upon the individual investigators who consider using them for the first time.

Key Points.

1. CDEs are scientifically-vetted standards for data collection and reporting and are essential to standardization and transparency in research.

2. TASK3 of the AES/ILAE Translational Research Task Force of the ILAE has agreed upon CDEs for preclinical epilepsy research, as well as case report forms (CRFs) and guidelines.

3. The use of CDEs, CRFs and adopting standardized methods will improve many aspects of preclinical epilepsy research.

4. Tools to simplify the use and accelerate the time to complete CDEs/CRFs, e.g., apps, would significantly encourage their implementation in preclinical research.