Creation and implementation of an electronic health record note for quality improvement in pediatric epilepsy: Practical considerations and lessons learned

Abstract

Objective

To describe the development of the Pediatric Epilepsy Outcome‐Informatics Project (PEOIP) at Alberta Children's Hospital (ACH), which was created to provide standardized, point‐of‐care data entry; near‐time data analysis; and availability of outcome dashboards as a baseline on which to pursue quality improvement.

Methods

Stakeholders involved in the PEOIP met weekly to determine the most important outcomes for patients diagnosed with epilepsy, create a standardized electronic note with defined fields (patient demographics, seizure and syndrome type and frequency and specific outcomes‐ seizure type and frequency, adverse effects, emergency department visits, hospitalization, and care pathways for clinical decision support. These were embedded in the electronic health record from which the fields were extracted into a data display platform that provided patient‐ and population‐level dashboards updated every 36 hours. Provider satisfaction and family experience surveys were performed to assess the impact of the standardized electronic note.

Results

In the last 5 years, 3,245 unique patients involving 13, 831 encounters had prospective, longitudinal, standardized epilepsy data accrued via point‐of‐care data entry into an electronic note as part of routine clinical care. A provider satisfaction survey of the small number of users involved indicated that the vast majority believed that the note makes documentation more efficient. A family experience survey indicated that being provided with the note was considered “valuable” or “really valuable” by 86% of respondents and facilitated communication with family members, school, and advocacy organizations.

Significance

The PEOIP serves as a proof of principle that information obtained as part of routine clinical care can be collected in a prospective, standardized, efficient manner and be used to construct filterable process/outcome dashboards, updated in near time (36 hours). This information will provide the necessary baseline data on which multiple of QI projects to improve meaningful outcomes for children with epilepsy will be based.

Key Point Box.

• There is a need for standardized data fields to facilitate outcome‐oriented quality improvement.

• A pediatric epilepsy electronic note has been created by a multidisciplinary team that facilitates data entry within a clinical visit that can be contributed to by physician and nurse care providers.

• The technology exists to transfer that information directly into an analysis and display platform that produces outcome dashboards every 36 hours.

• The dashboards can be filtered within seconds to display outcomes stratified by age, gender, seizure and syndrome types, seizure frequencies, and treatment‐related adverse effects.

• Physician and family surveys indicate that the standardized electronic pediatric epilepsy note is a valuable contribution to patient care.

1. INTRODUCTION

In 2012, the Institute of Medicine published a report, Epilepsy Across the Spectrum, which recognized a significant gap in the quality of data collection and surveillance for people living with epilepsy. ¹ The report emphasized the need for standardized quality measures and common data elements (CDEs) to describe epilepsies as a basis for quality improvement (QI). However, the challenges to implementing QI in practice include the added efforts and resources required for data acquisition, analysis, and feedback in a format and time frame that is actionable. For example, the standard methodology for QI involves data entry on paper followed by Plan‐Do‐Study‐Act cycles, but this becomes limiting as the number of patients and data elements increases. Furthermore, data transfer to additional platforms is required for analysis and visualization of outcomes even if the primary entry is into an electronic note. The approach to overcome these limitations has included secondary data entry into electronic platforms such as Excel, REDCap, or Access followed by data analysis and display, but secondary data entry requires additional time and resources. It is now generally agreed that the optimal approach would be standardized data entry at the point of care directly into the patient electronic health record (EHR) followed by data analysis and the availability of outcome dashboards in near time. This has the advantage of avoiding the time and resources required for double data entry (and potential re‐entry errors) as the data elements are entered as the clinical note and subsequently used for QI. ² , ³ , ⁴ In addition, the use of CDEs, which are essential to evaluate the impact of treatments on outcomes, will also allow comparison of results with other institutions.

The purpose of this report was to describe the development of the Pediatric Epilepsy Outcome‐Informatics Project (PEOIP) at Alberta Children's Hospital (ACH), which was created to provide standardized, point‐of‐care data entry; near‐time data analysis; and availability of outcome dashboards as a baseline on which to pursue QI. We also present the results of parent experience and care provider satisfaction surveys with regard to the use of the standardized pediatric epilepsy electronic note. Hereafter, this will be referred to simply as the “note.” A detailed description of the seizure types, epilepsy syndromes, treatments (medications, epilepsy surgery, ketogenic diet, vagus nerve stimulator, immunomodulatory agents), and outcomes will be presented in subsequent manuscripts.

2. METHODS

2.1. Pediatric epilepsy note development and implementation

The ACH Comprehensive Children's Epilepsy Center (CCEC) is the only pediatric comprehensive epilepsy center in this region and is the primary referral center for seizures for the southern half of Alberta and eastern British Columbia. The mission statement of the CCEC is “To create a system of excellence to provide comprehensive clinical care, research that leads to improved outcomes & education for all stakeholders in a manner that optimizes societal resources.” PEOIP was created within this context with a goal of creating a data structure (electronic note) by which standardized common data elements could be entered, and individual and population‐based data were analyzed and reported to users in the form of dashboards. The intent was to use QI methodology to achieve incremental gains in clinical outcomes. An overview of the program processes is provided in Figure 1.

FIGURE 1.

PEIOP process map. The outcomes that were identified on which to collect baseline data included seizure frequency, adverse effects, cognition/quality of life, and resource utilization (hospitalizations and ED visits). Despite identification of potential cognition and quality‐of‐life measures, implementation proved not feasible during the office visit due to resource limitations. The figure demonstrates how data are accrued via a standardized note embedded in the EHR that is subsequently analyzed and made available for clinicians as outcome dashboards. Note that in the lower right of the figure, the opportunity for combining this data set with other provincial data sources is indicated. The possibility of rapidly altering the care pathways based upon translational research is acknowledged.

To create the note, stakeholders involved in the care of children with epilepsy at the ACH were invited to attend weekly, 1‐hour, multidisciplinary CCEC meetings beginning in October 2012. Participation in these meetings and work done for PEOIP was voluntary without allotted time away from other duties. Regular attendees included pediatric epileptologists and nurses involved in epilepsy care in addition to general pediatric neurologists, the pharmacist, psychologist, and neuropsychologists. Attendance was requested of others (eg, administrative staff, EEG technologists, dieticians) when the topic to be discussed required their input.

The note was divided into several sections (Figure 2 and Appendix S1 for the full note), each of which is composed of predefined fields and free text, the latter to be used to supplement the defined fields for clarity and/or context with the understanding that the text would not be “computable” for the quantitative outcome dashboards. The text boxes provide the opportunity to reflect the complexity of the epilepsies such as the sequencing of events, subjective description of the seizures by the patient family, and the ability to express the diagnostic formulation that is not permitted by checkboxes or pull‐down menus. The first page of the note is the visit details that contains fields for site of the patient encounter, as well as the visit type in addition to other intake information (data not shown). Event/seizure type description (Figure 2A) was structured to acknowledge that at the time of presentation a diagnosis of an epileptic seizure may not be certain and nonepileptic events occur with and without epileptic seizures. A diagnosis is not required until the Impression section of the note (Figure 2B) and in this manner, the frequency of the epileptic or nonepileptic event can be tracked through time independent of the ultimate diagnosis (eg, an event that appeared on presentation to be a focal seizure that was found to be a nonepileptic event following video‐EEG). A maximum of 5 event types are allowed per patient, and these are linked to an impression for each event type. In order to assess event/seizure frequency, a modification was made of a previously validated seizure frequency score (SFS) ⁵ based upon the input of the stakeholders of the PEOIP.

FIGURE 2.

Sample screenshots of the electronic note for a patient diagnosed with absence epilepsy. A, Event/seizure type description. Each event/seizure type contains a text field for description of semiology (preferably in the patient/caregiver's words) and defined fields for onset date and age, event/seizure frequency score duration, recovery to normal, diurnal variation, clusters, provoking factors, and warnings. B, Shows the fields available for the Impression. The use of the 2010 recommendations for seizure and epilepsies classifications is a reflection of when the project took place. Note that the following fields were made mandatory in order for the note to be considered completed (signed): Site of visit, visit date and type, events (minimum of 1 required), weight, seizure frequency score*, seizure duration*, recovery time*, diurnal variation*, provoking factors*, diagnosis*, seizure type*, syndrome*, subsyndrome*, etiology* (* indicates that completion was mandatory only for the first described event (event type 1)

Medications are recorded in the EHR (Sunrise Clinical Manager (SCM), Allscripts Healthcare Solutions, Inc) using the Prescription Writer feature. A previously validated instrument ⁶ was modified to record treatment‐related adverse effects in a way that the caregiver could indicate “absent, present but tolerable or present and intolerable.” However, due to the complexity of programming required, adverse events (AEs) were not linked to a specific medication. This could be remedied with future programming modifications and would require that the users indicate which of the AEs was linked to it for those patients taking multiple medications.

The Impression section includes predefined fields that were organized by the suggestions made in the revision of terminology and concepts in 2010 by the ILAE Classification and Terminology Task Force. ⁷ However, the fields are directly comparable to those relevant to the 2017 seizure ⁸ and syndrome ⁹ classifications. The highest level is “Diagnosis” that gives several options (generalized, focal, and unknown), “nonepileptic” and “not yet diagnosed” to reflect these possibilities. Logic is built into the note such that a diagnosis of “Generalized seizure” results in the appearance of the option to select the generalized seizure types, followed by syndrome and subsyndrome (Figure 2B). The fields that were mandatory to sign the note are indicated in Figure 2 legend. These are followed by general categories of etiologies.

Care pathways for some commonly encountered epilepsy syndromes (eg, “benign” epilepsy with centrotemporal spikes, childhood absence epilepsy (CAE)) and medications (eg, levetiracetam, ethosuximide) are programmed into the note (using standard decision (yes/no) nodes) to aid in decision making and to provide clinical decision support. An example for CAE is illustrated in Figure 3A,B. These pathays are created based on evidence when available and by consensus when not. The pathways are important to our future efforts to understand the variation in outcomes between providers and the effect of interventions (using QI methodology) on seizure frequency, for example.

FIGURE 3.

Childhood Absence Epilepsy Decision Support Tool available in the electronic note. A, Depicts the diagnostic pathway. B, Depicts the treatment pathway

Our model of care is one in which physicians and nurses work in dyads with regard to patient and family interactions during and after the clinic visit. The nurses focused on several aspects of the interaction including medications taken, associated adverse effects, social issues in the family, and education. This requires that both providers have access to a common note, sections of which are completed asynchronously at the point of care. The partnership worked by direct discussions at the time of the clinic visit (with and without the patient and family in attendance) and by electronic communications between visits. We restricted the total amount of information required to that considered essential for patient care and could be accomplished within the time allotted to a clinic visit (60 minutes for new patients, 30 minutes for return patients). However, additional information could be easily added as text in any of the note sections.

The note was refined based upon data collection on paper, and then, after multiple iterations it was coded into the EHR so that it could be completed by the provider at the time of the visit without dictation. The final version of the note became available on January 1, 2016. The initial implementation of the note was in the ambulatory epilepsy clinic at ACH, and then, it was made available in the emergency department, pediatric intensive care unit, neonatal intensive care unit, and a satellite neurology ambulatory clinic. The intent was to make the note available at every location at which a patient with epilepsy could be seen. Emergency department and hospitalization data can be collected at the point of care when seen in these locations. This information can be combined with the corresponding administrative data and incorporated into the dashboards. In addition, the note provides the opportunity for the clinician to indicate that the patient was seen in either of these locations prior to the visit (Supplement 1). The use of note was voluntary, but strongly encouraged within the pediatric epilepsy group. Mandatory fields are indicated in Figure 2.

2.2. Dashboard implementation

EHR data are transferred once daily post‐midnight to a provincial data warehouse, where the pediatric epilepsy note is extracted, normalized, and summarized before updating the analysis tool, Tableau (Seattle, Washington, USA). Tableau dashboard updates were automated with the intent of making information available to users within 24 hours of the most recent clinical encounter, but the reality of synchronizing multiple complex institutional data environments often took 36 hours. The “Population” dashboard was constructed to provide up‐to‐date information on the entire patient population as shown in Figures 4 and 5. The filters can be adjusted online with the data reconfigured according to filter settings within seconds. This allows for a rapid analysis of population outcomes stratified by age, gender, diagnosis, seizure and syndrome type, event/seizure frequency, and etiology, in addition to any of the other data elements entered in defined fields. The manner in which the filters are used to provide a snapshot of a specific epilepsy syndrome (CAE) is illustrated in Figure 5.

FIGURE 4.

Tableau Dashboard displaying select data for the entire population. A, As of June 1, 2020, there were 3245 unique patients in the database having 13,823 encounters (notes) and 5306 events. Master filters for the dashboard consist of site (ACH outpatient, ACH inpatient, ACH emergency room, ACH NICU, ACH PICU, SHC outpatient, other). These may include all notes or most recent data and start‐end dates. The notes, however, can be filtered with any combination of fields once the master filters have been set. The top panels show the diagnostic categories presented in a least to most specific fashion based on the 2010 ILAE classification revision. Bottom panels show the distribution of seizure frequency scores and seizure duration. The seizure frequency score scale is included in the dashboard to facilitate interpretation of the data. C, Shows the medication panels. Filters are available for type of medication, status active, historical or total (active + historical), number of medications, and medication combinations. D, Illustrates the etiologies for entire population

FIGURE 5.

Shows select panels from the population Tableau dashboard filtered for patients with childhood absence epilepsy and most recent data. A, There is s a slight female predominance (42.3.2% male vs 57.7% female). B, ~70% are seizure‐free. C, The most commonly prescribed medications are ethosuximide followed by valproate and lamotrigine

The patient‐specific dashboard was constructed to provide up‐to‐date information of the patient. Any patient‐specific data (categorical or free text) documented in the EHR can be displayed in the dashboard. An example of patient with CAE is provided in Figure 6.

FIGURE 6.

Patient‐specific dashboard for patient with childhood absence epilepsy. Provided in this dashboard are the age, event type, diagnoses, etiology, current medications, side effects, EEG, and neuroimaging (NI) results, as well as the last note free‐text impression and follow‐up plan. ER visits since the last outpatient appointment, if any would also be displayed. The dashboard also includes a treatment timeline, which shows variations in the seizure frequency score with changes in treatments. The treatment timeline draws information from both the phone note unusually generated by nurses (diamonds) and physician epilepsy consult note (circles). Scrolling over the event type, EEG, NI (neuroimaging), medications in the current medication list, or data points on the treatment timeline would provide detailed information. This is shown for the event type. A heat map for the seizure frequency score is also included in the dashboard to make possible a quick assessment of the current seizure frequency and changes in score overtime

2.3. Provider satisfaction survey

A brief (10 question) user satisfaction survey was distributed to physicians (5 pediatric epileptologists, 2 general pediatric neurologists) who were most involved in the development and use of the note. It was believed that these individuals would be best to judge its utility. The survey was created in Google Forms, and the respondents were anonymous. The questions were focused on assessing how providers viewed the efficiency provided by the standardized note compared to dictation with regard to time required for clinical documentation, need to complete the note after clinic, and utility in answering patient inquiries when on‐call. In addition, it was of interest to know whether the clinicians believed whether it was feasible to complete the note while in clinic met the goal of entering standardized information and whether the note significantly altered the doctor‐patient/family interaction (Appendix S2).

2.4. Family experience survey

Family experience with regard to receiving the standardized note immediately after the office visit was assessed with an iPad‐based survey developed by the Alberta Health Services Clinic Quality Metrics team at ACH. The survey was developed in SelectSurvey.NET software (Kansas City, MO, USA). It was made available to families (caregivers) whose children had a diagnosis of epilepsy based upon the MD and RN perception of who could complete the questionnaire (ie, adequate English skills). The families were asked to participate after the office visit between July 6 and October 7, 2016, inclusive. As providing the note was at the discretion of the provider, families who did not receive the note were asked whether they wished one was provided. The questions were designed to assess the families’ use of the note for description of their child's condition to others, whether it made the family feel more confident about the care received and participation as a partner in that care. The questions also assessed views on the value of receiving the note, and family knowledge about the diagnosis, seizure frequency reporting, use of a rescue plan, assessment of medication adverse effects, and when to contact the epilepsy clinic nurse. The questions were modified based upon consultation with the patient‐family group at ACH. The chi‐square analysis was used to compare proportions; P <.05 was considered statistically significant (Appendix S3). The surveys were anonymous.

The University of Calgary Conjoint Health Research Ethics Board referred review of the PEOIP proposal to a Project Ethics Community Consensus Initiative charged with review of QI projects and which subsequently provided the necessary permission to perform the activities described in this manuscript.

3. RESULTS

To date, the CCEC has collected standardized data on 3,245 unique patients comprising 13,831 electronic notes (as of June 1, 2020) in the space of five years. Description of the total population data is provided in the legends of Figures 4 and 5 and will not be elaborated in this manuscript as stratifying data by age, etiology, seizure type, and syndrome are required to meaningfully interpret the data. With regard to entering data into the defined fields, Table 1 illustrates that “Seizure Diagnosis” was the only field that had complete entry of data, likely due to it being a mandatory field. This table illustrates that the fields of seizure (eg, atonic), syndrome (eg, electroclinical adolescence), and subsyndrome (eg, Dravet syndrome) were frequently not completed (ie, nulls). It is unknown whether this reflects lack of a meaningful entry into that field (unlikely for a seizure type) or simply due to the provider not checking the appropriate box. “Etiology” is a mandatory field for the first seizure/event type. However, the 27% no completion rate was due to the occurrence of more than 1 seizure/event type for some patients in which “etiology” was not mandatory. An example of how documentation has changed over time can be found in the documentation of SFS as 969/2296 (42%) had no SFS documented on their first note, whereas only 121/2148 (6.5%) did not have a score documented on their last note. Adverse events had the largest number of nulls, ~98.5% of patients (data not shown). This was in large part due to an error in coding such that the field that could be selected when no adverse event was present was not functional. In this data set drawn from the general population, it is not surprising that the etiology of the seizures was unknown in the majority of children (Figure 4D).

TABLE 1.

Number of nulls for select fields for each event. Start date January 1, 2016, and end date June 1, 2020

Field	Nulls (n (%))
Total # of events	4599
Onset Age	868 (18.7)
Seizure Diagnosis	0
Seizure Type	1297 (28.2%)
Seizure Syndrome	1636 (35.6%)
Seizure Subsyndrome	2562 (55.7%)
Seizure Score	300 (6.5%)
Seizure Duration	791 (17.2%)
Seizure Recovery Duration	863 (18.8%)
Seizure Recovery Delay	979 (21.2%)
Etiology	1249 (27.2%)

3.1. Provider satisfaction survey

The 7 staff clinicians who are the most frequent users of the note each responded to the survey. All respondents had a significant clinical full‐time equivalent (FTE) (during the study period (57.1% 0.75 FTE and 42.9% 0.5 FTE). In summary, 6/7 responded in the affirmative to the question asking does the use of the standardized electronic note makes clinical documentation more efficient when compared to note dictation; 6/7 would not return to use of dictation if possible. With regard to feasibility for use during clinic, 5/7 responded in the affirmative. The complete results are presented in Appendix S2.

3.2. Family experience survey

Eighty‐four families completed the survey, 50 of whom received the standardized electronic note at the end of the clinic visit and 34 did not. Of those not receiving the electronic note after the office visit, 70.6% indicated that they would like to have received it. Of those that did receive the note, 47/50 (94%) answered that they read the note. With regard to value of the note, 43/50 (86%) answered that it was “valuable” or “really valuable.” The vast majority indicated that the note facilitated communication of the child's condition with family members (n = 35), school (n = 30), funding or support agencies (n = 12), daycare (n = 3), and other individuals (n = 5). Of those who received the note, 82% (41/50) strongly agreed or agreed with the statement that the note made them more confident about the care received. Also related to the parents’ confidence in care was that receiving the note made 78% (39/50) of respondents feel like they are partners with the care team.

Parents who received the consult note reported knowing the name of the child's epilepsy or seizure disorder more often than parents who did not receive a note (58.0% vs 35.3%). The difference was statistically significant (X = 4.176, P =.041; Appendix S3).

4. DISCUSSION

4.1. Program/Note

In this work in progress we describe methods to consistently collect standardized, prospective data on patients with epilepsy in real time using an EMR that can be displayed and updated frequently using commercially available analysis and visualization tools. Currently, the PEOIP has collected standardized, computable information on over 3,200 pediatric patients with epilepsy as part of routine clinical care, which to our knowledge is the largest single‐center pediatric epilepsy database of its kind in North America. Provider and patient surveys indicate overall satisfaction with the note. This is a proof of principle demonstration that available technologies can be leveraged to enter clinically important data at the point of care and have that data flow into an analysis platform in an automated fashion without need for “double entry” by staff, thereby leading to cost savings.

The program was made possible by the administrative support that recognized the importance of “secondary use” of data to improve patient outcomes and the willingness of clinicians to agree upon entry of common data elements at each visit with the goal of providing a baseline for QI. This required a “cultural change” substituting the idiosyncratic clinical practice of individuals for a common practice based upon data when possible so as to make possible knowledge acquisition in which the outcomes of every patient contribute. This is similar in concept to that used in other pediatric domains such as oncology. ¹⁰ This was accomplished with a commonly used EHR and the analytics reporting platform used in the province of Alberta. It should be noted that the intent was focused on patient care/QI, not research projects that would require a larger data set beyond that which could be accomplished in the office without additional personnel and funding. That said, the “face validity” of the data collected in this manner is reassuring as in the case for CAE the response to medication, use of specific medications, seizure duration, and recovery are consistent with our knowledge of what is to be expected.

This platform would facilitate projects such as comparative effectiveness research and evaluation of interventions stratified by seizure and syndrome types. However, these would require patient/caregiver consent and institutional review board approval. The creation and use of this type of note for QI and research have recently been described. ² , ⁴ We recognize that this approach may be limited in settings where nursing, and administrative and IT support are limited.

Standardized fields for patient care in an EHR have been implemented in very sophisticated systems but requiring significantly greater resources than what we used here ³ . The use of a physician‐nurse dyad in our experience led to increased consistency of documentation and potentially improving outcome.

The initial results presented in this manuscript demonstrate a broad range of compliance with entering the data fields with noted improvements overtime. However, the results provide a baseline and illustrate the opportunity to improve completeness of data ascertainment via QI methodology. The error in informatic field design that affected adverse events is one of the lessons learned that highlights the requirement for frequent testing of the note and data extraction processes in order to capture coding errors early, thereby minimizing their impact.

4.2. Dashboards

The implementation of dashboards to provide a visual display of patient information has been previously described with the source of the data being generated by the care provider ³ , ¹¹ as part of the EHR. In our data set, it seems likely that the requirement of entering data in a field in order to be able to sign the note is a powerful stimulus to adherence. This is evidenced by the large number of nulls (no data entered) in the syndrome and subsyndrome groups and for adverse events. However, the clinicians may have chosen not to enter anything if each was unknown, rather than specifically check the box to indicate that neither was present (unknown syndrome and not applicable, respectively). These dashboards raise the possibility that there can be multiple reasons for absent data, each of which would require a different remedy if it was deemed that the data were important to enter. Thus, the dashboards provide patient‐level information to evaluate care over time and population‐level data to determine outcomes across stratified age, seizure, and syndrome types. In addition, the dashboards currently provide the baseline data for outcomes to be improved by standard QI methodology that involves interventions, measuring effects with statistical process control charts, and then introducing additional interventions in an iterative manner. However, due to the manner in which software manages multiple filtering steps there is sometimes a loss of a small number (6%) of cases, which is being addressed.

4.3. Care pathways

Care pathways as embedded into our clinical note at the outset are one form of clinical decision system (CDS) that involves a variety of functions, one of which is clinical workflow; see ¹² for a recent review. CDSs such as ours are knowledge‐based, essentially using an if‐then approach to decision making, whereas others can take the form of computerized order entry, templates, and reminders to name a few. The application of CDSs to assist with guideline implementation ¹³ , ¹⁴ , ¹⁵ is particularly important given the literature that indicates that guideline adherence could be significantly improved. ¹⁶ Another potential use of CDS programming could be to aggregate those aspects of the clinical encounter to support reimbursement based on medical decision making that is required in the United States.

The care pathways provided clinical decision support to the user based upon the diagnosis made at the time of the office visit. Incorporation into the clinical note obviates the need to look online or on an institutional website for guidance. This is the type of added efficiency that technology should provide to facilitate clinical interactions. In addition, it “levels the playing field” so that each patient receives the highest standard of care as determined by consensus in the practice, independent of the provider's level of training. The standardization of care in an electronic environment allows clinical outcomes to be meaningfully evaluated as the interventions are used in a consistent manner and adherence to the pathway can be monitored. The use of a care pathway is recommended but is strictly voluntary. The provider is encouraged not to use a pathway if it did not best serve a patient's needs. However, there is the recommendation to document why the pathway was not used so as to improve it in an iterative manner. In addition, the care pathway served an educational role for users, including trainees, by providing an evidence‐based approach or learning scaffold for clinical epilepsy issues.

4.4. Provider satisfaction and family experience surveys

Overall, the provider results indicated that the standardized note was in general a useful adjunct to clinical care and not burdensome. Physician satisfaction with regard to use of an EHR and its prominent role in physician burnout has been extensively discussed in the literature. ¹⁷ One of the reasons commonly identified is the time required to interact with the EHR ¹⁸ to meet regulatory and reimbursement documentation requirements. In addition, this has been recognized as a potential cause of decreased quality of patient care by an expert panel assembled by the American Medical Informatics Association. ¹⁹ The first recommendation of this group was to “simplify and speed documentation” in addition to using data collected at the point of care to improve outcomes. Our findings from a limited but most involved group of physician users indicate the preference of direct entry into a structured note, rather than dictation. However, the input of users who were most involved in the creation of the note introduces a bias that should be evaluated with a future survey. Provider satisfaction as evidenced in our program is an important intervention against burnout. However, the documentation requirements in this Canadian province are significantly less than mandated for reimbursement in the United States. The use of clinical support staff such as nurses (as in our program) or scribes facilitates successful note completion during the patient encounter, which is a significant strategy that can be used to prevent physician burnout. We recognize that the physician‐nurse dyad relationship is not available in many institutions due to resource limitations. However, it would be informative to understand whether/how this team approach increases physician efficiency, reduces calls between visits, and adds to patient and family satisfaction with the clinical encounter.

The family surveys indicate a strong desire for them to receive the note at the end of visit as it results in increased confidence in the care provided, improved communication with others, and education with regard to epilepsy type. Patient satisfaction with the ability to read their notes has been assessed by evaluating responses in the OpenNotes project. ²⁰ Studies have revealed a variety of indicators of increased satisfaction in primary care, ²¹ Veterans administration, ²² and specialty clinics with now more than 100,000 patients having had the opportunity to read their electronic medical record. ²³ Our findings are concordant with these studies and demonstrate the advantages of sharing patient records.

4.5. Limitations

The ability to create the type epilepsy outcome‐oriented data structure as described is limited by several types of resources. It was essential that the stakeholders meet frequently to design and implement the electronic note. This was done on a voluntary basis, usually during the lunch hour. The completion of the note in clinic was greatly facilitated by the nurse interaction with the patient/family and ability to add to sections of the note. Finally, resources were necessary for the programming of the note and analysis by expert personnel. Thus, although feasible with existing technology, this project would have been impossible without the human and other resources noted.

4.6. Clinical relevance and future directions

In conclusion, this data set provides the basis on which to test the effects on seizure outcome of a variety of interventions in a clinical setting via QI methodology. The ability to quickly filter data allows for the rapid identification of candidates for research studies, epilepsy surgery, or to notify patients of drug shortages and/or life‐threatening adverse events. The implementation of our platform provides the first steps to achieving the “quadruple aim” of improving patient outcomes, patient‐family experience, provider satisfaction, and cost reduction. The incorporation of evidence‐based care and treatment pathways into the note is also an educational tool for trainees and physicians involved in the care of patients with epilepsy.

We confirm that we have read the Journal's position on issues involved in ethical publication and affirm that this report is consistent with those guidelines.

CONFLICT OF INTEREST

JB has received support from and/or has served as a paid consultant for the Epilepsy Foundation, the Epilepsy Study Consortium, Epilog, UCB. BLB receives royalties for the sales of the Pediatric Forensic Neuropsychology textbook (2012, Oxford University Press) and three pediatric neuropsychological tests [Child and Adolescent Memory Profile (ChAMP, Sherman and Brooks, 2015, PAR Inc), Memory Validity Profile (MVP, Sherman and Brooks, 2015, PAR Inc), and Multidimensional Everyday Memory Ratings for Youth (MEMRY, Sherman and Brooks, 2017, PAR Inc)]. He has previously received in‐kind support (free test credits) from the publisher of the computerized cognitive test (CNS Vital Signs, Chapel Hill, North Carolina). He has received funding from national organizations (eg, CIHR) and has received honoraria in exchange for scientific presentations. JMR has received support from and/or has served as a paid consultant for Accera, Inc, Ajinomoto USA, Aquestive Pharmaceuticals, Cerecin, Danone Nutricia, Eisai Pharma, Mallinckrodt Pharmaceuticals, and UC Pharma. The remaining authors have no conflicts of interest.

Supporting information

Appendix S1

Appendix S2

Appendix S3

Buchhalter JR, Scantlebury MH, D’Alfonso S, et al. Creation and implementation of an electronic health record note for quality improvement in pediatric epilepsy: Practical considerations and lessons learned. Epilepsia Open. 2021;6:345–358. 10.1002/epi4.12480