Design and Implementation of Electronic Health Record Common Data Elements for Pediatric Epilepsy: Foundations for a Learning Healthcare System

Abstract

Objective.

Common data elements (CDEs) are standardized questions and answer choices that allow aggregation, analysis, and comparison of observations from multiple sources. Clinical CDEs are foundational for learning healthcare systems, a data-driven approach to healthcare focused on continuous improvement of outcomes. We aimed to create clinical CDEs for pediatric epilepsy.

Methods.

A multiple stakeholder group (clinicians, researchers, parents, caregivers, advocates, and electronic health record (EHR) vendors) developed clinical CDEs for routine care of children with epilepsy. Initial drafts drew from clinical epilepsy note templates, CDEs created for clinical research, items in existing registries, consensus documents and guidelines, quality metrics, and outcomes needed for demonstration projects. The CDEs were refined through discussion and field testing. We describe the development process, rationale for CDE selection, findings from piloting, and the CDEs themselves. We also describe early implementation, including experience with EHR systems and compatibility with the International League Against Epilepsy (ILAE) classification of seizure types.

Results.

CDEs were drafted in August 2017 and finalized in January 2020. Prioritized outcomes included seizure control and seizure freedom, American Academy of Neurology quality measures, presence of common comorbidities, and quality of life. The CDEs were piloted at 224 visits at ten centers. The final CDEs included 36 questions in 9 sections: diagnosis (1 question), seizure frequency (9), quality of life (2), epilepsy history (6), etiology (8), comorbidities (2), treatment (2), process measures (5), and longitudinal history notes (1). Seizures are categorized as generalized tonic-clonic (regardless of onset), motor, non-motor, and epileptic spasms. Focality is collected as epilepsy type, rather than seizure type. Seizure frequency is measured in nine levels (all used during piloting). The CDEs were implemented in 3 vendor systems. Early clinical adoption included 1294 encounters at one center.

Conclusions.

We created, piloted, refined, finalized, and implemented a novel set of clinical CDEs for pediatric epilepsy.

INTRODUCTION

A learning healthcare system is an approach to care delivery that encourages continuous improvement of clinical outcomes through cycles of data aggregation, analysis, reporting, and change in clinical practice, in partnership with patients and caregivers.¹ For a learning healthcare system to succeed, standardized clinical data must be collected through routine care processes. A foundational step is to specify common data elements (CDEs). Common data elements (CDEs) are standardized questions and answer choices that allow aggregation, analysis, and comparison of observations from multiple sources.

As an example of epilepsy CDEs, the National Institute of Neurological Diseases and Stroke (NINDS) convened experts to create a repository of standardized clinical case report forms (CRFs) and validated scales for epilepsy clinical investigation.² The NINDS CDEs, however, were designed for a clinical research context, where dedicated research staff complete comprehensive assessments outside of standard clinical workflows. In contrast, clinical CDEs must facilitate rapid, efficient data collection in a potentially busy clinical practice, while also supporting billing requirements, quality improvement initiatives, and easy access to important data at subsequent clinical encounters. Furthermore, for CDEs to fully support learning healthcare systems, patients and caregivers must participate in development.

As part of our effort to build a pediatric epilepsy learning healthcare system, we developed CDEs that were relevant for patient care, usable to measure pre-identified quality improvement priorities, and compatible with the workflow demands of routine clinical encounters. People with epilepsy and caregivers were integral members of the research team throughout.

METHODS

We conducted an iterative series of mixed methods exercises to develop CDEs for pediatric epilepsy. Some activities included human subjects, which were described in a central IRB protocol (BRANY - Biomedical Research Alliance of New York) and approved by participating sites. Participating clinical sites were members of the Pediatric Epilepsy Learning Healthcare System (PELHS), a project affiliated with the Pediatric Epilepsy Research Consortium (PERC).³

We began with five parallel activities conducted with a multidisciplinary group including pediatric epilepsy clinicians, clinical researchers, epidemiologists, parents of children with epilepsy, people with epilepsy, and leaders of epilepsy advocacy organizations. (Appendix 1, Figure 1)

Figure 1.

Consensus Process to create and disseminate Pediatric Epilepsy Learning Health System registry questions for use at the point of care. NINDS, National Institute of Neurological Disorders and Stroke; CDE, Common Data Elements; NISC, National Infantile Spasms Consortium; ELES, Early Life Epilepsy Study; NSR, Neonatal Seizure Registry; pSERG, Pediatric Status Epilepticus Group; EHR, Electronic Health Record

We collected and reviewed existing documents with standardized clinical data, including (1) clinical notes and templates from multiple pediatric epilepsy centers, (2) existing sources of CDEs for clinical research (NINDS Epilepsy CDEs),² and (3) data dictionaries of three active US-based multicenter registries: the Early Life Epilepsy Study database⁴ (which included the National Infantile Spasms Consortium database⁵), the Neonatal Seizure Registry,⁶ and the Pediatric Status Epilepticus Research Group.⁷ These were reviewed by one reviewer (ZG), and used as preparatory material for group discussions.

Six group discussions were convened to identify important outcomes for children with epilepsy (each group with multiple stakeholders). In the first three group discussions, participants generated a list of important outcomes, then ranked their top three choices. In the second three group discussions, participants named published manuscripts that changed clinical practice. The outcomes reported in these publications were manually extracted by one investigator (ZG) and added to the list of outcomes generated in the first three groups. (Table 1).

Table 1.

Pediatric epilepsy outcomes and key references

Category	Outcome	Example References	Measurable by clinical CDEs developed in this study?
Seizure Control, Seizure Freedom	Seizure free for 1 mo, 6mo, 1yr, 2 yr	Berg AT 2008 Epilepsia	Yes.
	No generalized tonic clonic seizurs	Wirrell EC Ped Neuro 2017	Yes.
	No status epilepticus	Wirrell EC Ped Neuro 2017	Yes.
	Reduction in frequency by 50%	Devinsky O NEJM 2017 & 2018	Approximate. Categorical seizure frequency question.
	Ordinal outcomes	Berg AT Epilepsia 2014	Approximate. Would require development of computable phenotype.
	Freedom from treatment failure	Glauser TA NEJM 2010	Yes.
	5-year remission	Berg AT Brain 2014	If multiple visits (longest interval in questions is 2 yr).
Infantile Spasms	Resolution of clinical spasms by 2 weeks	Knupp KG Ann Neurol 2016	Yes.
	Sustained resolution of spasms at 3 months	Knupp KG Ann Neurol 2016	Yes.
	No infantile spasms from Day 14 – 42	O’Callaghan FJ Lancet Neuro 2017	Approximate. Answer choices for spasms allow detection of 2 week & 1 month freedom from spasms.
	Resolution of clinical spasms and hypsarrhythmia	Hussain SA Epilepsia 2014	No. EEG standard forms in development.
Quality Measures	Met AAN epilepsy / child neurology quality measures	Patel AD Neurology 2018a 2018b, Fountain NB Neurology 2015	Yes.
	Discussed SUDEP?	Ramachandran NR Epi Behav 2016	Yes.
Other Key Outcomes	Presence of common comorbidities	Ho NT J Pedtr 2018	Yes.
	Obtain genetic testing	Shellhaas RA Neurology 2017	Yes.
	Quality of life	Goodwin SW Epilepsia 2015, 2018	Yes. (Novel instrument, undergoing validation)
	Development	O’Callaghan FJK Lancet CAH 2018	Approximate. Clinical impression not standard measures.
	Sleep	Jain SV Semin Ped Neur 2015	Presence / absence of sleep disorder.
	Use of Health Services	Grinspan ZM Neurology 2015	No.
	Costs	Keros S JCN 2017	No.
	Medication Side Effects	Morita DA Neurology 2012	Indirectly. Quality of life instrument measures how side effects affect daily routines
	Patient Reported Outcomes	Moura LM Neurology 2016	No.
	Adherence	Modi AC JAMA 2011	No.
	Full Scale IQ	Berg AT Neurology 2012	No.
	Social (Education, Stigma, Driving)	Wirrell EC Arch Ped Adol Med 1997	No.
	Burden on Family	Spindler UP Eur J Ped 2017	No.
	Patient / Family Engagement	Begley C Epil Behav 2018	No.
	Successful Transition	Nabbout R Epil Behav 2017	No.
	Access to care	Bisgaier J NEJM 2011	No.

We also identified outcomes for five PELHS demonstration projects (planned prior to the CDE development process; ongoing). These five include three field tests of American Academy of Neurology Child Neurology Society (AAN/CNS) quality measures^8,9 and two projects to compare the effectiveness of treatments for neonatal encephalopathy and new onset focal epilepsy.

Next, one investigator (ZG) drafted question and answer choices (CDEs) designed to adhere closely to existing data elements, measure outcomes prioritized in the group discussions, and support the five demonstration projects. This draft was disseminated via email to PELHS participants for comment and iteratively refined, based on feedback from email, discussions at four PELHS in-person meetings (December 2017 to May 2018) and weekly PELHS operational calls December 2017 to November 2018. Concurrently, we consulted with EHR vendors¹⁰ and institutional operational health information technology teams (Appendix 1) for guidance on implementation.

The refined draft was piloted at 10 centers (on paper). Centers were selected from PELHS participants based on clinician volunteers. Participating clinicians completed the pilot CDEs during or after outpatient clinical encounters. The pilot CDEs did not include demographic information (which is collected by administrative staff, not clinicians). Responses were de-identified for analysis. Qualitative observations from clinicians were gathered via email, verbal feedback on teleconferences, and in-person discussions. Several clinicians provided feedback from patients and caregivers, and some reported the length of time required to fill out the form. The CDEs were refined based on this feedback.

The consensus CDEs were submitted to three EHR vendors for evaluation prior to implementation (Epic Systems Corporation, Verona, WI; Cerner Corporation, North Kansas City, MO; athenahealth Inc., Watertown, MA). The CDEs were again modified.

We report themes that emerged in discussion groups, in-person meetings, phone calls, and email exchanges, using a modified thematic analysis approach.¹¹ We did not assess thematic saturation.

We present the final set of CDEs, including a description of the survey logic, and the rationale for each question. We indicate which CDEs “copy forward”, i.e., persist from one clinical encounter to the next without requiring data re-entry. We were attentive to the associated risks (i.e., failure to update important information) and benefits (i.e., avoiding errors and saving time for entry of complex information that does not change frequently) associated with copy and paste functions in EHR notes.¹² We provide screen shots of implemented CDEs.

We provide additional detail about seizure documentation. First, the CDEs include seizure frequency response categories based on review of previous seizure frequency scales,¹³ subsequently revised by multi-stakeholder input and piloting. We describe these categories, and report preliminary frequencies for levels of the scale from the pilot (10 centers) and from clinical use at one center (Children’s Hospital of Philadelphia). Second, the CDEs track seizures in categories that are less specific than the seizure categories endorsed by the ILAE -- we demonstrate the relationship between these systems with a conceptual map.¹⁴

Statistical analyses were performed with R (R Foundation; Vienna, Austria).

RESULTS

We examined clinical notes and templates from 11 academic pediatric epilepsy programs (of 18 centers participating in phone calls). Clinicians at three of these 11 centers also dictate notes as free text.

Each discussion group included 3–8 individuals (Appendix 1). The prioritization process (first three groups) indicated improved seizure control was the single most important data element. Other highly prioritized outcomes included: presence of comorbidities (psychiatric / behavioral / developmental), medication side effects, health services outcomes (i.e., emergency department visits), and quality of life / burden on family. Table 1 describes all discussed outcomes.

Seizure frequency items ask about the time since the last seizure, then the frequency of that seizure type. Frequency is measured in nine levels: too many to count, multiple per day, daily, weekly but not daily, monthly but not weekly, at least once per year but not every month, less than once per year, uncertain, frequency not well defined. “Too many to count” was requested by parents because “multiple per day” did not adequately capture their experience. “Uncertain” and “frequency not well defined” were requested by clinicians, to distinguish between “insufficient information to characterize seizure frequency” and “frequency of seizures not well captured by other choices (e.g., seizure clusters)”.

Piloting occurred at 224 visits at 10 academic pediatric epilepsy centers. Most patients had established epilepsy (95%). Common characteristics included treatment resistance (42%), developmental delay (72%), and unknown etiology (61%). 30% had abnormal brain imaging related to epilepsy. 40% had undergone genetic testing, some with an identified genetic cause (17% of responses, 40% of those with testing). Epilepsy type was largely known (93%), with good representation of focal (38%), generalized (34%), and mixed (22%). Age of onset ranged from neonatal to adult. Some had a history of epilepsy surgery (12%), and a few had metabolic (2%), infectious (2%), and autoimmune (1.4%) etiologies. (Table S1) For most questions, there were 1–5 missing replies (i.e., 0.4% – 2.2%). There were more missing replies for epilepsy type (11% missing) and genetic testing (17%).

During piloting, 322 responses used the seizure frequency scale at 215 of the 224 visits (9 visits with no seizure categories, 104 with multiple seizure categories). This included 122 children with generalized tonic-clonic seizures (GTCs), 123 motor seizures, and 77 non-motor seizures. All nine choices were selected at least 12 times (range 12 for Too many to count to 81 for Less than once per year). (Figure 2) The seizure frequency scale was implemented and used at one center for 887 children (47% female) at 1294 visits (median age at visit 8.0 years [IQR 4–12 years; range 39 days to 37 years]), including 375 visits with children with GTCs, 621 motor seizures, 397 nonmotor seizures, and 58 epileptic spasms. The option “Frequency not well defined” was not included in the implementation. All eight remaining choices were used, (range 18 (1.4% of visits) for Uncertain to 340 (26% of visits) for At least once per year).

Figure 2.

Responses to questions during the pilot (blue; 10 centers) and implementation (orange; one center) about the most recent seizure (top row) and the overall frequency of seizures (bottom row) for four seizure types (columns). For infantile spasms, treatment response is all-or-none, thus frequency of epileptic spasms is not collected. Of note, three labels at the implementation site were slightly different than in the pilot: (1) “Too many to count” was labelled “Many per day”; (2)“Multiple per day” was labelled “Several per day”; and (3) “Frequency not well defined” was not included as a response.

Qualitative feedback from in-person meetings, phone calls, and emails indicated poor consensus about the number of questions and answer choices. Arguments favoring more were to (1) improve granularity of data collected, particularly about seizure types; (2) provide exhaustive lists of abnormalities; (3) adhere closely to the ILAEseizure and epilepsy classification;^14,15 and (4) increase the number of measurable outcomes. Arguments favoring fewer were to (1) emphasize outcomes over historical details; (2) consolidate ILAE concepts into fewer answer choices; (3) limit the cognitive burden of multiple mouse clicks;¹⁶ and (4) allow rapid completion of the questions (clinicians often said “less than 2 minutes” was acceptable).

Feedback from piloting lead to multiple changes. For example, because epilepsy type was often missing due to lack of clinical certainty, we added clarifying text: “best guess, taking into consideration history and diagnostic findings”. The final CDEs are summarized in Table 2 (See Appendix 2 for full text).

Table 2.

Common Data Elements for Pediatric Epilepsy, for Use at the Point of Care

Section	Question Italics if conditional	Notes
Diagnosis	Has epilepsy? (CF)	ILAE operational definition provided as reference.
Seizure Frequency	Last Generalized Tonic Clonic Seizure
	Frequency of GTCs	If any GTCs
	Any GTCs at night?	If any GTCs
	Last Motor Seizure
	Frequency of Motor Seizures	If any motor seizures
	Last Nonmotor Seizure
	Frequency of Nonmotor Seizures	If any nonmotor seizures
	Last† Epileptic Spasm	If age under 3
	Status Epilepticus in the past year?
Quality of Life	Seizure effect on routines AED effect on routines	These two novel questions are undergoing validation; manuscript in preparation
Epilepsy History	Epilepsy Type (CF)	Focal, generalized, both, unknown
	Intractable? (CF)
	New onset? (CF)
	Age at onset? (CF)
	Current Epilepsy Syndrome (CF)
	Specific Conditions (CF)	(i.e. Aicardi, Angelman, etc)
Epilepsy Etiology	Etiology known? (CF)
	Abnormal brain imaging? (CF)
	Detail on brain imaging (CF)	If abnormality
	Known genetic etiology? (CF)
	Detail on genetics (CF)	If known genetics
	Metabolic etiology? (CF)
	Infectious etiology? (CF)
	Autoimmune etiology? (CF)
Comorbidities	Developmental delay? (CF)	Based on clinical impression, not formal testing
	Selected comorbidities present? (CF)	Psychiatric, behavioral, cognitive, neurologic, head size, and technology dependence
Treatment	Ketogenic diet?
	Refer to epilepsy surgery conference?
Process Measures	Discussed SUDEP?
	Discussed transitions?	Age 13 or older
	Psych comorbidity screened for?	Age 7 or older
	Pregnancy / contraception counseling?	Age 12 or older & female
	Infantile spasms - 1st line therapy?	If infantile spasms
Longitudinal History Notes	Free text (CF)	Optional -- for other aspects of the history that are likely to persist from visit to visit.

CF = Copy Forward - i.e., the data persists and auto-populates subsequent notes

Diagnosis

(1 question; copies forward) asks if the child has epilepsy. The 2014 ILAE operational definition of epilepsy is provided as guidance.¹⁷ This question excludes children with single seizures and low risk of recurrence, febrile only seizures, or events that are uncertain to be seizures.

Seizure Frequency

(9 questions; does not copy forward) asks about five seizure types: GTCs (regardless of focal or generalized onset), motor seizures that are not GTCs, non-motor seizures, epileptic spasms, and status epilepticus. For GTCs, motor seizures, and non-motor seizures, the clinician first indicates when the most recent seizure occurred. If within 6 months, the clinician then selects the overall frequency. For GTCs, clinicians indicate if seizures occur at night or from sleep. The question on status epilepticus asks if any episodes occurred in the past 12 months. The question on epileptic spasms only asks when was the last spasm (i.e., and not the frequency), and only appears for children younger than 3 years old. In selecting these nine questions, the following considerations were discussed.

First, the form simplifies the 25+ ILAE¹⁴ seizure types into four (Figure 3). GTCs are a well-established risk factor for sudden unexpected death¹⁸ and families prioritize reducing this seizure type.¹⁹ Motor seizures are counted more reliably than non-motor seizures.^20,21 Tracking epileptic spasms in infants supports quality improvement initiatives and comparative effectiveness research.^9,22 Epileptic spasms in older children classified as motor seizures. The question text includes guidance classifying seizures (Appendix 2).

Figure 3.

Relationship between Pediatric Epilepsy Learning Healthcare System (PELHS) seizure outcomes and the International League Against Epilepsy (ILAE) seizure classification system (Fisher et al, Epilepsia 2017). Seizure types are in general mapped to three concepts: generalized tonic clonic seizures, regardless of onset (black lines, grey box), motor seizures (blue), and non-motor seizures (orange). For children under 3 years old, epileptic spasms are tracked separately (green). Aware vs impaired awareness is not explicitly captured. Unclassified seizures are categorized based on presence / absence of movement.

Second, occurrence of status epilepticus is separately assessed. Prior episodes predict future episodes,²³ most recurrences happen within 1 year,²³ and there are interventions that can reduce risk (i.e., home rescue benzodiazepines.²⁴)

Third, localization is captured with epilepsy type rather than seizure type.^14,15 Epilepsy type (focal, generalized, both, or unknown) supports medication selection and identification of surgical candidacy. Seizure type (focal onset, generalized onset, or unknown) helps determine epilepsy type, and can be described in free text.

Fourth, some ILAE concepts are not captured. The effect on awareness, for example, can be described in free text. “Unclassified” seizures are categorized based on the presence of movement.

Quality of Life

(2 questions; does not copy forward) asks two novel quality of life questions about the effect of seizures and ASM side effects on routines. These questions have been validated²⁵ and correlate with known scales, such as QOLCE-55²⁶ and PESQ.²⁷

Epilepsy History

(6 questions; copies forward) asks about epilepsy type (focal, generalized, both, or unknown), treatment resistance, recency of onset, age of first unprovoked seizure, current epilepsy syndrome, and specific conditions associated with epilepsy (e.g., Aicardi or Angelman Syndromes). Clinicians requested questions on prior epilepsy surgery, but they were removed by EHR vendors to consolidate questions on surgical history elsewhere.

Epilepsy Etiology

(8 questions; copies forward) asks if the etiology is known, about findings from brain imaging and genetic testing, and if metabolic, infectious, or autoimmune etiologies are present. Detailed sub-questions about imaging and genetics appear if the abnormalities explain the reason for epilepsy. The list of imaging findings was modified from work examining findings in children with infantile spasms.²⁸ Genetic and chromosomal etiologies were selected to capture abnormalities responsible for at least 1% of epilepsies with a known genetic etiology.²⁹

Comorbidities

(2 questions; copy forward) asks about development and common comorbidities. Clinicians are encouraged to assess development based on a standard neurological history and exam without further formal testing. Comorbidities include common psychiatric, behavioral, cognitive, and neurologic conditions (including sleep disorders), as well as abnormal head size and technology dependence.

Treatments

(2 questions; does not copy forward) cover dietary therapy and referral for epilepsy surgery evaluation. Clinicians had requested an extensive list of medical and surgical treatments as part of the CDEs themselves; however, EHR vendors noted that medications are documented elsewhere in the chart, and adding CDEs for medications would (a) introduce redundancies, (b) become outdated as new medications became available, and (c) potentially introduce medical errors by bypassing systems for review of allergies and drug-drug interactions.

Process Measures

(5 questions; does not copy forward) includes questions related to four AAN/CNS quality measures (transition to adult care, screening for mental health comorbidities, counseling about contraception and/or pregnancy, and 1^st line therapy for infantile spasms)^8,9 and one measure requested by families and advocacy groups (SUDEP counseling, an AAN Guideline).³⁰ Each question appears only for target populations.

Epilepsy Longitudinal History Note

(1 question; copies forward) provides free text for information that might persist from visit to visit, such as additional relevant history, a running list of previously tried ASMs, or detailed electroencephalography (EEG) or MRI findings. EHR vendors and information technology experts advised to avoid the term “continuity of care” for this item, to avoid confusion with text fields elsewhere in the EHR.

This set of questions, when linked with EHR data like visit history, demographics, medication administration, and prescription data, tracks many but not all of the outcomes listed by the discussion groups (Table 1). The questions have been implemented in three EHR vendor systems (Figure 4). The specifications (Table 2 and Appendix 2) were largely followed with minor changes.

Figure 4.

Examples of the PELHS question about the last generalized tonic clonic seizure, as implemented in three electronic health record systems at four centers: (A, B) Epic Systems Corporation (Verona, WI), (C) Cerner Corporation (North Kansas City, MO), (D) athenahealth Inc (Watertown, MA). The question and answers are conceptually identical; however, there are variations in interface (buttons vs radio boxes), wording, capitalization, and the order of answer choices. In one implementation (B), the question appears only if the patient is known to have tonic-clonic seizures, and so there is no option “Never/Does not have this seizure type”.

DISCUSSION

Summary of findings.

We designed, iterated, piloted, finalized, and implemented clinical CDEs for routine care of children with epilepsy, with multi-stakeholder input including clinicians, researchers, parents and caregivers, advocates, and EHR vendors. The CDEs include two novel quality of life questions²⁵ and a categorical seizure frequency scale with nine levels. Each seizure frequency level was selected during piloting and clinical use, without requests by users for additional answer choices, suggesting the scale is sufficiently granularity for clinical use. The CDEs have been implemented by 3 EHR vendors.

Learning healthcare systems.

The CDEs were designed to support a pediatric epilepsy learning healthcare system, in which clinical data are collected, aggregated across multiple sites, analyzed, and returned to centers to support quality improvement, comparative effectiveness research, surveillance and epidemiology, health services research, and direct improvements in clinical care and patient outcomes.¹ This approach to health care delivery has dramatically improved health outcomes for children with other pediatric diseases, notably cancer,³¹ inflammatory bowel disease,³² and hypoplastic left heart syndrome.³³ The success of the CDEs will depend on adoption and use, which in turn will depend on how well they support efficient clinical workflow.

Involvement of patients and caregivers.

We included, from the beginning, people with epilepsy, caregivers, and advocacy group leaders to align with the principles of patient-centered care and patient-centered research (i.e., the ethos of “nothing about me without me”^34,35). Multiple suggestions from non-clinician participants were adopted, such as inclusion of quality of life questions and the addition of a “too many to count” option for seizure frequency. We anticipate multi-stakeholder involvement will improve adoption.

Epilepsy and EHR Tools.

Several ongoing efforts have customized EHR systems to support care of people with epilepsy.³⁶ For example, a practice in Ireland described the socio-technical environment of clinical epilepsy documentation at their center,³⁷ followed by design and implementation of a custom-built system,³⁸ and subsequent upgrades to include clinical genetics evaluations.³⁹ An epilepsy practice in the United States created and implemented a tool for detailed data collection for use in a well-staffed outpatient practice.⁴⁰ One vendor (Epic Systems Corporation; Verona, WI) hosts neuroscience specialty steering boards (pediatric and adult), which successfully created multiple tools for neurologists’ use in clinical care, including for epilepsy.¹⁰ Our work builds upon and expands these efforts.

Expectation for use.

The CDEs are not comprehensive. Clinicians will need to separately document the history of present illness, past medical and surgical history, physical exam, results from laboratory and diagnostic studies, medical decision-making, and orders. This is by design, to preserve rich narrative detail not easily captured in standardized fields. We found that some clinicians dictate these narrative sections – they may continue to do so. The CDEs were designed for in-person office visits, but may also be used for phone calls, telehealth visits, or inpatient encounters.

Benefits and risks of data standardization.

Adoption and use of well-designed clinical CDEs has the potential to allow a clinician to document an epilepsy evaluation without prolonging the office visit. In addition, clinical CDEs reduce variation within EHRs – these variations increase the time required for clinicians to find key clinical data which may lead to patient harm.⁴¹ Standardized data can also power decision support, i.e. the automated delivery of targeted information to help clinical decision making at the point of care. CDEs can also support reporting for management and quality improvement initiatives, for example to identify patients with treatment resistant epilepsy for referral to surgical management. Finally, CDEs can support clinical research, in that they capture important clinical characteristics and outcomes through routine care processes, avoiding double entry into research databases.⁴²

There are risks, however. Clicking through standardized answer choices may limit free text narrative, which often contains rich, humanizing, and clinically important details.⁴³ Increasing standardization of data entry can increase time spent on documentation, at the expense of time for direct patient care.⁴⁴ Furthermore, poorly designed health information technology can contribute to physician burnout.⁴⁵ For example, cognitive burden of data entry into an EHR form can be measured in computer mouse clicks,⁴⁶ and our clinical CDEs will add clicks to each patient encounter. Successful implementation of the CDEs must include careful attention to clinician workflow, to minimize these and other potential negative consequences of standardization.

Clinical CDEs vs. other CDEs.

The NINDS Epilepsy CDEs were designed for research settings, whereas clinical CDEs are designed for clinical workflow, and thus prioritize ease-of-use over granularity to encourage adoption. For example, the clinical CDEs do not capture details of individual seizures, whereas the NINDS CDEs support documentation of multiple specific seizure types. In the clinical CDEs, epilepsy etiology is determined using a more-likely-than-not standard, whereas in the NINDS CDEs confidence is graded along a spectrum possible-probable-definite. The NINDS CDEs include etiologies important in adult epilepsy (e.g., dementia), which are not included in the pediatric-focused clinical CDEs. The clinical CDEs ask for a broad assessment of development (normal, mild delay, definite delay in one domain, definite delay in 2–3 domains) whereas the NINDS CDEs include multiple detailed validated instruments of development across several domains. Clinical CDEs should also be distinguished from CDEs for devices,⁴⁷ which provide detailed specifications for software developers.

Potential pitfalls.

Impactful health information technology must support interactions among clinicians, healthcare organizations, and the technology itself.⁴⁸ Recognition of four potential pitfalls will help ongoing implementation and dissemination of the CDEs.

First, the current answer choices will need revision over time, as conceptualization and knowledge of epilepsy evolves. For example, the list of important genetic etiologies and understanding of the pathogenicity of specific mutations will likely grow. As an interim solution, the current CDEs allow users to specify any mutation as free text. However, future versions may need to revise the choices. Of importance, to interpret genetic findings, clinicians should refer to vendor reports or centralized repositories such as the ClinGene Epilepsy Gene Curation Panel.⁴⁹

Second, variations in implementation (i.e., figure 4) demonstrate that user experience will differ from center to center. Dissemination of the CDEs centrally may help reduce these variations.¹⁰

Third, poor consensus about the number of questions indicates that some will find the CDEs too long, and others insufficiently detailed. Efforts to improve adoption might emphasize a core set of CDEs (i.e., the seizure frequency questions), leave other CDEs as optional, and allow sites to develop supplementary CDEs.

Fourth, frequent missing replies about epilepsy type (11% missing) and genetic testing (17% missing) and qualitative feedback suggest these questions require more thought by clinicians than the other CDEs. Determining epilepsy type requires synthesis of the neurologic history, seizure types, and diagnostic data. Understanding genetic testing history often requires searching through the chart for testing results. Quality improvement initiatives (such as encouraging use of the “longitudinal history” element) may help clinicians with these questions.

Limitations.

Several limitations merit discussion.

First, the clinical CDEs are closely aligned with the ILAE framework for seizures, epilepsies,^14,15 and epilepsy syndromes,⁵⁰ but the overlap is not exact. For example, some epilepsy syndromes have been merged into a single category (e.g., Benign Occipital Epilepsy combines Panayiotopoulos Syndrome and Idiopathic Childhood Occipital Epilepsy of Gastaut) and others are not explicitly listed (e.g., absence epilepsy with eyelid myoclonia). We note, however, that the ILAE framework is a working consensus that generated robust debate, and will continue to evolve.⁵¹

Second, input was almost exclusively from participants within the United States. Our choices may not capture variations across international practice.

Third, several important outcomes are not well captured, including EEG outcomes, burden on family, adherence, transition of care from pediatric to adult clinicians, social outcomes, and access to care. Linking these outcomes to data collected though the CDEs will require additional standardization work (for EEG), care delivery innovations (such as devices or integration of patient reported outcomes), and/or linkage with other datasets (health insurance claims data or social determinants of health databases). In addition, medication side effects are measured only broadly – sufficient for a global estimate without detailed tracking of specific side effects.⁵²

Fourth, our discussion and consensus methodology prioritized iterative development over traditional qualitative methods, such as focus groups, Delphi processes, or interviews. This process may have introduced biases, in that we did not explicitly guard against outsized influence from vocal participants.

Fifth, differences in workflow across different EHRs to the next may lead to differences in cognitive burden of data entry. This may affect adoption across institutions, biasing data collected from the CDEs towards centers with EHRs that more efficiently support clinical workflow.

Sixth, although most of the CDEs have fixed answer choices, there are several free text fields. Gathering usable data from these fields will require manual review or natural language processing techniques.

Finally, the CDEs were designed for general use for children with epilepsy, and may not include details tracked in specialized programs, such as for specific causes of epilepsy (i.e., renal angiomyolipomas for tuberous sclerosis) or for surgical evaluation. Development of CDEs in these contexts would require additional input from relevant stakeholders (i.e., advocacy groups, neurosurgeons, etc.).

Conclusions & Next Steps.

The CDEs have the potential to improve clinical care and support projects in quality improvement, surveillance and epidemiology, comparative effectiveness research, and health services research. Future work is needed to understand adoption, use, potential negative side effects, and sustainability of the CDEs.

KEY POINTS.

• Clinical common data elements are standardized questions and answers to allow observations from multiple sources to be compared.

• CDEs are foundational for learning healthcare systems, a data-driven approach to healthcare focused on continuous improvement of outcomes.

• A multiple stakeholder group created, piloted, finalized, and implemented pediatric epilepsy clinical CDEs.

• Seizures are categorized in four groups, focality is collected via epilepsy type, and seizure frequency is measured in nine levels.

• Successful piloting at ten centers (224 visits) and implementation at one center (1294 visits) are favorable for broad adoption and use.

APPENDIX

Appendix

APPENDIX 1. Participants and Contributors

DISCUSSION GROUP PARTICIPANTS

Amy Bennett, JD. American Academy of Neurology. Minneapolis, MN

Jeffrey Buchhalter, MD PhD. Phoenix, AZ.

Jason Coryell, MD MS. Oregon Health Sciences University. Portland, OR

Brandy Fureman, PhD. Epilepsy Foundation. Landover, MD

William Gaillard, MD. Children’s National Medical Center. Washington, DC

Howard Goodkin, MD PhD. University of Virginia. Charlottesville, VA

Zachary Grinspan, MD MS. Weill Cornell Medicine. New York, NY

Shaun Hussain, MD. University of California, Los Angeles. Los Angeles, CA.

Dennis J Dlugos, MD. Children’s Hospital of Philadelphia. Philadelphia, PA

Anna Jeong, MD. Lurie Children’s Hospital. Chicago, IL

Sookyong Koh, MD PhD. Children’s Hospital of Atlanta. Atlanta, GA

Alison Kukla, MPH. Person with epilepsy. Norfolk, VA

Matthew Lallas, MD. Children’s Hospital of Los Angeles. Los Angeles, CA

Tobias Loddenkemper, MD. Boston Children’s Hospital. Boston, MA

Juma Mbwana, MD. Children’s National Medical Center. Washington, DC

Tiffani McDonough, MD. Lurie Children’s Hospital. Chicago, IL

Patricia McGoldrick, NP MPA MSN. Mount Sinai School of Medicine. New York, NY

Diego Morita, MD. Cincinnati Children’s Hospital. Cincinnati, OH

Lindsey Morgan, MD. Seattle Children’s Hospital. Seattle, WA

Anup Patel, MD. Nationwide Children’s Hospital. Columbus, OH

Renée Shellhaas, MD MS. Michigan Medicine. Ann Arbor, MI

Nilika Singhal, MD. University of California, San Francisco. San Francisco, CA

Margaret Storey, PhD. Parent of a child with epilepsy. Chicago, IL

Anne T Berg, PhD. Lurie Children’s Hospital. Chicago, IL

Steven Wolf, MD. Mount Sinai School of Medicine. New York, NY

Elissa Yozawitz, MD. Montefiore Medical Center. Bronx, NY

ADDITIONAL CONTRIBUTORS & ACKNOWLEDGEMENTS

American Academy of Neurology Clinical Practice and Quality Staff

Gardiner Lapham RN MPH, BAND Foundation

Sarah Huenke, American Academy of Pediatrics

ELECTRONIC HEALTH RECORD ADVISORS

J Travis Gossey, MD. Weill Cornell Medicine

Lisa Jones RN, Cerner Corporation

Payton DePalma, Epic Corporation

Greg Strodtman, Epic Corporation

Stephanie Idell, Dell Children’s Hospital

Stephanie Ingle, Epic Corporation

Khair Lambaz, Epic Corporation

USER GROUP, CHILDREN’S HOSPITAL OF PHILADELPHIA

Pamela P McDonnell, MD

Ethan M Goldberg, MD PhD

Colin Ellis, MD

Xilma Ortiz-Gonazlez, MD PhD

Sudha K Kessler, MD

France W Fung, MD

Lawrence E Fried, MD

Ryan E Gill, MD

Robert Stowe, MD

Marissa P DiGiovine, MD

Naomi Lewin, MD PhD

Lila Worden, MD

Stacy D Elkhatib Smidt, MD

Alyssa R Rosen, MD

Amisha B Patel, MD MPH

Eric D Marsh, MD PhD

APPENDIX 2. FULL COMMON DATA ELEMENT SPECIFICATIONS

Section 1. Diagnosis.

[guidance text] ILAE 2014: Epilepsy means (1) at least two unprovoked (or reflex) seizures occurring greater than 24 hours apart OR (2) unprovoked (or reflex) seizure and probability of further seizures similar to the general recurrence risk (at least 60%) after two unprovoked seizures, occurring over the next 10 years or (3) diagnosis of an epilepsy syndrome

[question] Has epilepsy

[answer choices (select one) – copies forward]

Yes

No / Uncertain

Section 2. Seizure Frequency

[question] Last GENERALIZED TONIC CLONIC SEIZURE (including clonic-tonic-clonic, myoclonic-tonic-clonic, and focal onset seizures with secondary generalization)

[answer choices (select one)]

Never/does not have this seizure type

Today

1–6 days ago (up to 1 week)

1–4 weeks ago (up to 1 month)

5–12 weeks ago (1 to 3 months)

13–26 weeks ago (3 to 6 months)

6–12 months ago

13–24 months ago

More than 2 years ago

Uncertain

[logic – The following two questions only appear if the last generalized tonic clonic seizure occurred within the past 6 months]

[question] Recently, what is the frequency of GENERALIZED TONIC CLONIC Seizures?

[answer choices (select one)]

Too many to count

Multiple per day

Daily

Weekly but not daily

Monthly but not weekly

At least once per year but not every month

Less than once per year

Uncertain

Frequency not well defined

[question] In the past 12 months, have any generalized tonic clonic seizures occurred AT NIGHT or FROM SLEEP?

[answer choices]

Some at night or from sleep

None at night or from sleep

Unsure if ever at night or from sleep

[question] Last MOTOR Seizure (Not including GTCs; e.g., myoclonic jerks, drop attacks, tonic seizures, focal motor seizures that do not generalize, or epileptic spasms older than 2)

[answer choices (select one)]

Never/does not have this seizure type

Today

1–6 days ago (up to 1 week)

1–4 weeks ago (up to 1 month)

5–12 weeks ago (1 to 3 months)

13–26 weeks ago (3 to 6 months)

6–12 months ago

13–24 months ago

More than 2 years ago

Uncertain

[logic – The following question only appears if the last motor seizure occurred within the past 6 months]

[question] Recently, what is the frequency of MOTOR Seizures (not including GTCs)?

[answer choices (select one)]

Too many to count

Multiple per day

Daily

Weekly but not daily

Monthly but not weekly

At least once per year but not every month

Less than once per year

Uncertain

Frequency not well defined

[question] Last NON-MOTOR Seizure (for example, absence seizure, or seizures with impaired awareness only)

[answer choices (select one)]

Never/does not have this seizure type

Today

1–6 days ago (up to 1 week)

1–4 weeks ago (up to 1 month)

5–12 weeks ago (1 to 3 months)

13–26 weeks ago (3 to 6 months)

6–12 months ago

13–24 months ago

More than 2 years ago

Uncertain

[logic – The following question only appears if the last non-motor seizure occurred within the past 6 months]

[question] Recently, what is the frequency of NON-MOTOR Seizures?

[answer choices (select one)]

Too many to count

Multiple per day

Daily

Weekly but not daily

Monthly but not weekly

At least once per year but not every month

Less than once per year

Uncertain

Frequency not well defined

[logic – The following question only appears if the child’s age is younger than 3 years]

[question] Last EPILEPTIC SPASM (single spasm or cluster)

[answer choices (select one)]

Never/Does not have this seizure type

Today

1–6 days ago (up to 1 week)

7–14 days ago (up to 2 weeks)

At least 2 weeks free of spasms

At least 1 month free of spasms

At least 3 months free of spasms

At least 6 months free of spasms

At least 1 year free of spasms

Uncertain

[question] STATUS EPILEPTICUS requiring an emergency department visit in the past 12 months?

[answer choices (select one)]

Yes

No

Uncertain

Section 3. Quality of Life

[question] Think about the child’s usual routines. How often in the past 2 weeks have SEIZURES significantly changed those routines?

[answer choices (select one)]

Every day

Most days (more than half)

Some days (less than half)

Never

Uncertain/ Didn’t ask

[question] Think about the child’s usual routines. How often in the past 2 weeks have SEIZURE MEDICATION SIDE EFFECTS significantly changed those routines?

[answer choices (select one)]

Every day

Most days (more than half)

Some days (less than half)

Never

Uncertain/ Didn’t ask

Section 4. Epilepsy History

[question] Epilepsy type (best guess, taking into consideration history and diagnostic findings)

[answer choices (select one) - copies forward]

Focal

Generalized

Both Focal and Generalized

Unknown

[question] TREATMENT RESISTANT? (i.e. seizures continue despite adequate trials of two or more anti-epileptic seizure drugs)

[answer choices (select one) - copies forward]

Yes

No

Uncertain

[question] Epilepsy is NEW ONSET? (i.e., the first unprovoked seizure was less than 6 months ago)

[answer choices (select one)]

Yes

No

Uncertain

[question] Age at first unprovoked seizure

[answer choices (select one) - copies forward]

Neonatal (0 to 29 days)

Infant (1m up to 1y)

1y

2y

3y

4y

5y

6y

7y

8y

9y

10y

11y

12y

13y

14y

15y

16y

17y

18y

Roughly toddler (1–3 y)

Roughly preschool (4–6y)

Roughly school age (7–12 y)

Roughly adolescent (13–18y)

Adult (19+)

Unknown/Unavailable

[question] Current epilepsy syndrome

[answer choices (select one) - copies forward]

Not a syndrome (nonsyndromic, uncertain, too early to tell)

Neonatal. Ohtahara / Early infantile epileptic encephalopathy (EIEE)

Neonatal. Early myoclonic encephalopathy (EME)

Neonatal Infantile. Benign neonatal or infantile seizures (familial or nonfamilial)

Infantile. Epilepsy of Infancy with Migrating Focal Seizures (EIMFS)

Infantile. Infantile Spams (IS) / West syndrome (WS)

Infantile. Febrile seizures plus (GEFS+)

Infantile. Dravet syndrome (DS)

Childhood. Lennox Gastaut Syndrome (LGS)

Childhood. Doose Syndrome (EMAS- Epilepsy with Myoclonic-Atonic Seizures)

Childhood. Benign Occipital Epilepsy (Panayiotopolous or Gastaut)

Childhood. Benign Epilepsy with Centrotemporal Spikes (BECTS)

Childhood. Childhood Absence Epilepsy (CAE)

Adolescence. Juvenile Myoclonic Epilepsy (JME)

Adolescence. Juvenile Absence Epilepsy (JAE)

Other. Electrographic Status Epilepticus of Sleep / Continuous Spike and Waves in Sleep / Landau Kleffner Syndrome (ESES / CSWS / LKS)

Other. Progressive Myoclonic Epilepsy (PME)

[question] Other epilepsy syndrome

[answer choices - copies forward] (free text box)

[question] Specific conditions associated with epilepsy

[answer choices (select one) - copies forward]

None of these conditions apply/ uncertain

Aicardi Syndrome

Angelman Syndrome

Down Syndrome

Mowatt-Wilson Syndrome

Rasmussen Syndrome

Rett Syndrome

Sturge-Weber Syndrome

Tuberous Sclerosis

Williams Syndrome

Wolf-Hirschhorn Syndrome

Section 5. Epilepsy Etiology

[question] Epilepsy etiology is known?

[answer choices (select one) - copies forward]

Yes

No

[question] Most recent brain imaging study(ies)?

[answer choices (select one) - copies forward]

Not done or unavailable

Normal

Abnormal – unlikely related to epilepsy (including incidental findings)

Abnormal – likely related to epilepsy

[logic – The following questions only appear if the most recent brain imaging study(ies) was abnormal, likely related to epilepsy]

[question] What does the imaging show (select all that apply)?

[subheadings and answer choices – copies forward; may select more than one]

Malformation of Cortical Development

Focal Cortical Dysplasia

Hemimegalencephaly

Hypothalamic Hamartoma

Lissencephaly

Band Heterotopia

Polymicrogyria

Tuberous Sclerosis Related Dysplasia

Schizencephaly

Other Grey Matter Heterotopia

Hippocampal Abnormalities

Hippocampal Malrotation

Hippocampal Sclerosis

Other Hippocampal Abnormalities

Other Patterns

Corpus Callosum Hypoplasia or Absence

Holoprosencephaly Spectrum

Hydrocephalus (any Etiology)

Septo-Optic Dysplasia Spectrum

Vascular Malformation

Cerebral Angioma

Leptomeningeal Angiomatosis (Sturge-Weber)

AV Malformation

Acquired Injury and/or Sequelae

Hypoxic Ischemic Encephalopathy

Intracranial Infection

Periventricular Leukomalacia

Intraventricular Hemorrhage

Trauma

Solid Tumor

Stroke

Post-Operative Cavity

Other imaging finding (free text)

(Free text box)

[question] Genetic testing

[answer choices (select one) - copies forward]

No genetic testing has been done

Some testing done, but no identified genetic cause (i.e., unrelated genetic abnormality, testing pending, VUS, or negative)

There is genetic abnormality that explains the epilepsy (more likely than not)

[logic – The following questions only appear if there is a genetic abnormality that explains the epilepsy]

[question] What is the genetic or chromosomal abnormality?

[subheadings and answer choices – copies forward; may select more than one]

Chromosomal

Trisomy 21

Ring Chromosome 14

Ring Chromosome 20

CNV

dup15q syndrome

Phelan-McDermid syndrome

1p36 deletion syndrome

MECP2 duplication syndrome

Angelman

Uniparental Disomy

Imprinting Defect

15q11 microdeletion

Specific Genes (In Alphabetical Order)

ALDH7A1

ALG13

ARX

CACNA1A

CACNA1E

CDH2

CDKL5

CHRNA4

CHRNA7

DEPDC5

DNM1

FOXG1

GABRA1

GABRB3

GABRD

GABRG2

GNAO1

GRIN2A

GRIN2B

KCNQ2

KCNQ3

KCNT1

MECP2

PCDH19

POLG

PRRT2

SCN1A

SCN1B

SCN2A

SCN8A

SLC13A5

SLC2A1 (GLUT1)

STX1B

STXBP1

SYNGAP1

TSC1

TSC2

UBE3A

Other genetic finding (free text)

[question] Is there a metabolic etiology?

[answer choices (select one) - copies forward]

Yes

No (low clinical suspicion or testing is negative)

Uncertain

[question] Is there infectious etiology?

[answer choices (select one) - copies forward]

Yes

No (low clinical suspicion or testing is negative)

Uncertain

[question] Is there autoimmune etiology?

[answer choices (select one) - copies forward]

Yes

No (low clinical suspicion or testing is negative)

Uncertain

Section 6. Comorbidities

[question] Is there any DEVELOPMENTAL DELAY? (Based on clinical impression/history)

[answer choices (select one) - copies forward]

Normal development (no delay)

Mild developmental delay (or uncertain)

Definite delay in one domain (Gross Motor, Language, Social)

Definite delay in two or three domains (Gross Motor, Language, Social)

(Too old for developmental delay diagnosis)

[question] Any of the following selected COMORBIDITIES present? (Based on clinical impression/history)

[answer choices - copies forward; may select more than one]

None of these apply

Psychiatric/Behavioral/Cognitive

ADD/ADHD

Anxiety

Autism/PDD

Depression

Learning Disability

Intellectual Disability

Neurologic

Cerebral Palsy

Hearing Impairment

Migraines

Movement Disorder

Sleep Disorder

Visual impairment (other than refractive errors)

Head Shape

Macrocephaly (>95^th percentile)

Microcephaly (<5^th percentile)

Other abnormal head shape

Technology Dependent

Gastrotomy

Tracheostomy

V P Shunt

Wheelchair

Section 7. Treatment

[question] Other than medications, will any of the following be continued and/or recommended?

[subheadings and answer choices; may select more than one]

Diet

No Dietary Therapy for Epilepsy

Ketogenic Diet

Modified Atkins Diet

Low Glycemic Diet

Other Dietary Therapy

Surgery

Scheduled for neurosurgery conference

Not scheduled for neurosurgery conference

Section 8. Process Measures

[logic – The following question only appears for children 12 years old or older]

[question] TRANSITION to adult care was discussed at this visit?

[answer choices (select one)]

Discussed transition to adult care.

Did not discuss transition to adult care at this visit.

[question] Discussed SUDEP at this visit?

[answer choices (select one)]

Discussed SUDEP.

Did not discuss SUDEP at this visit.

[logic – The following question only appears for children 7 years old or older]

[question] Screened for MENTAL HEALTH COMORBIDITIES at this visit?

[answer choices (select one)]

Screened for mental health comorbidities.

Did not screen at this visit.

[logic – The following question only appears for female children who are 12 years old or older]

[question] Counseled patient or caregiver about how epilepsy and its treatment may affect CONTRACEPTION and/or PREGNANCY?

[answer choices (select one)]

Counseled about contraception and/or pregnancy at this visit.

Did not counsel about contraception and/or pregnancy at this visit.

Premenstrual, surgically sterile, reproductive organs absent.

[logic – The following question only appears for children who have infantile spasms]

[question] For INFANTILE SPASMS- what treatment was given as FIRST LINE?

[answer choices (select one)]

First line therapy for infantile spasms included ACTH, oral steroids, or vigabatrin (alone or in combination).

First line therapy for infantile spasms was not ACTH, oral steroids, or vigabatrin.

First line therapy for infantile spasms unknown.

Section 9. Pediatric Epilepsy Longitudinal History Note

[large free text box – copies forward]