Simulated Electronic Health Record (Sim-EHR) Curriculum: Teaching EHR Skills and Use of the EHR for Disease Management and Prevention

Abstract

Electronic health records (EHRs) can improve many aspects of patient care, yet few formal EHR curricula exist to teach optimal use to students and other trainees. The Simulated EHR (Sim-EHR) curriculum was introduced in January 2011 at Oregon Health & Science University (OHSU) to provide learners with a safe hands-on environment in which to apply evidence-based guidelines while learning EHR skills. Using an EHR training platform identical to the OHSU EHR system, learners review and correct a simulated medical chart for a complex virtual patient with chronic diseases and years of fragmented care. They write orders and prescriptions, create an evidence-based plan of care for indicated disease prevention and management, and review their work in a small-group setting. Third-year students complete the Sim-EHR curriculum as part of the required family medicine clerkship; their chart work is assessed using a rubric tied to the curriculum’s general and specific objectives. As of January 2014, 406 third-year OHSU medical students, on campus or at remote clerkship sites, and 21 OHSU internal medicine interns had completed simulated charts.

In this article, the authors describe the development and implementation of the Sim-EHR curriculum, with a focus on use of the curriculum in the family medicine clerkship. They also share preliminary findings and lessons learned. They suggest that the Sim-EHR curriculum is an effective, interactive method for providing learners with EHR skills education while demonstrating how a well-organized chart helps ensure safe, efficient, and quality patient care.

Studies have demonstrated that electronic health record (EHR) use improves the quality, efficiency, and cost of patient care.^1,2 The Medicare and Medicaid EHR Incentive Programs, therefore, provide financial incentives to providers that “demonstrate meaningful use of certified EHR technology.”³ Although academic health centers are at the forefront of EHR implementation,⁴ most medical trainees receive little directed, uniform education in optimal use of EHRs.⁵ Further, because of concerns about liability, reimbursement/ billing, and patient safety,^6,7 medical students’ access to EHRs is often restricted,⁵ making them unable to document encounters, enter history, or create orders. This lack of education and access prevents students from learning to use EHRs in a “meaningful” way. Rather than being given opportunities to experience in simulated settings the mistakes and frustrations inherent in using EHRs, students often embark on a haphazard and variable learning curve in patient care settings, where their EHR education is directed by faculty with differing skill levels. Given this model, few students are likely to graduate feeling fully proficient or comfortable with their ability to use EHRs.

To address this training gap, Oregon Health & Science University (OHSU) implemented the Simulated EHR (Sim-EHR) curriculum in 2011. The aims of this innovative curriculum are to provide learners with a safe, supervised setting in which they place orders for a virtual patient in a simulated chart while also learning the fundamentals of chronic disease management and health care maintenance in the context of an EHR. In this article, we describe the Sim-EHR curriculum, including its development and implementation, and share participant feedback and lessons learned.

The OHSU Sim-EHR Curriculum

Setting

OHSU is located in a city and a state— Portland, Oregon—that both have high EHR adoption rates.^8,9 OHSU implemented its ambulatory and inpatient EHR platforms in 2005 and 2008, respectively. All first-year OHSU medical students receive a brief EHR training session and are then granted access to write notes, enter history, and pend orders. However, faculty comfort with use of the EHR varies within OHSU, and training sites outside OHSU have different policies on student EHR access. In academic year 2011–2012, third-year OHSU medical students participating in focused interviews during their family medicine clerkships reported using three to six different EHRs during their rotations. Some students reported regular use of EHRs during their clerkships, yet others reported they had never written an order in an EHR.

Curriculum development and objectives

We developed the Sim-EHR curriculum in 2010 to transform the haphazard, exposure-based EHR education described above into a uniform EHR experience with defined objectives and clear assessment measures.

We identified EHR-related general and specific objectives for the curriculum using existing guidelines¹⁰ as well as input from OHSU faculty members responsible for EHR implementation (see Table 1). We designed the objectives to impart general concepts common to all EHRs rather than related to a specific EHR platform. In the Sim-EHR curriculum, trainees learn to maintain an accurate EHR (correct/update social, surgical, and family histories, allergy alerts, and problem and medication lists), apply prevention and chronic disease guidelines in the EHR context, and order appropriate referrals/consultations, laboratory tests, imaging studies, and medications. With these objectives in mind, we created a virtual patient and a simulated chart in which trainees can practice these skills.

Table 1.

Simulated Electronic Health Record (Sim-EHR) Curriculum Objectives and Assessment Measures, Oregon Health & Science University

General objectives (ACGME competencies*)	Specific objectives	Assessment measures
Demonstrates skills in chart maintenance (P, PC, SBP)	Create and populate a problem list outlining a patient’s medical issues	All active medical problems are listed.
	Differentiate and correctly document elements of patient history in the appropriate subheadings of the electronic health record (EHR)	Family, social, obstetrical, past medical, and surgical history are listed in the appropriate sections of the EHR.
	Correctly document allergies	Documented allergies are listed in the allergies section.
	Identify and correct errors in the EHR	Inaccurate information is removed/corrected, and an addendum is added to notes with errors.
	Correctly reconcile a medication list	Medication list correctly reflects current doses and medicines. Old medicines/doses/duplicates are discontinued.
Demonstrates knowledge of evidence-based chronic disease management and disease prevention (MK, PC, PBLI, SBP)	Interpret EHR data accurately in making diagnoses	New diagnoses based on interpretation of EHR data (including diagnostics such as laboratory results, pulmonary function tests, and electrocardiograms) are correctly identified and listed.
	Locate and order evidence-based interventions for chronic disease management, surveillance, and comorbidity prevention for major chronic disease diagnoses	Chronic diseases (e.g., diabetes, chronic obstructive pulmonary disease, hypertension, hyperlipidemia) and other active medical problems have an appropriate evidence-based plan of care succinctly documented in the EHR comments section.
	Use a health maintenance tool for tracking routine prevention and chronic disease surveillance activities	Based on the patient’s age, gender, and personal risks, and using the U.S. Preventive Services Task Force recommendations, the health maintenance tab in the EHR accurately reflects all indicated routine screening activities for the prevention and surveillance of disease, timed at the correct intervals.
Demonstrates the ability to order prescriptions, laboratory tests, vaccines, imaging studies, and consultations (MK, PC)	Place orders for indicated medications based on the patient’s medical issues	Orders for new medications are placed in the EHR based on evidence based-guidelines for each diagnosis.
	Place orders for indicated diagnostic, treatment, and preventive measures including consultations, laboratory tests, and imaging studies	Orders are correctly placed for evidence-based chronic disease surveillance, prevention, and management, including screening tests, vaccines, laboratory tests, diagnostic imaging, referrals/consultations, and needed counseling.

^*Related Accreditation Council for Graduate Medical Education (ACGME) competencies: P, professionalism; PC, patient care; SBP, systems-based practice; MK, medical knowledge; PBLI, practice-based learning and improvement.

Simulated chart and virtual patient

One Sim-EHR chart was created for use in an EHR training environment (EPIC Train) that is identical to the OHSU EHR system (EPIC Production). The simulated chart includes a virtual patient’s outpatient, emergency, operative, and consultation notes; medication list; laboratory, imaging, and pathology results; electrocardiogram (ECG) tracings; and pulmonary function test results. The chronic disease states we chose for the virtual patient include those that have a large body of evidence-based guidelines, including diabetes, hyperlipidemia, hypertension, and chronic obstructive pulmonary disease. We provided details of the medical and social histories to encourage learners to alter their recommendations on the basis of a patient’s individual risks.

For example, to teach trainees to apply osteoporosis screening guidelines, we created a 60-year-old female virtual patient who has multiple osteoporosis risk factors in her family and medical histories. To teach clinical reasoning and data interpretation skills in the EHR context, we varied the presentation of medical issues: some are represented with results only (e.g., elevated fasting glucose levels), some are “buried” in the scanned documents (e.g., an ECG), and some appear as previously diagnosed in the notes. We included mistakes (e.g., incorrect entries) and omissions (e.g., no allergies listed) to provide training in identifying and correcting errors and to emphasize safety and professionalism in maintaining accurate patient records.

We tailored the simulated chart components to delineate varying trainee skill levels. For example, an early learner may be able to correct the family history section using a prior clinic note; a higher-level trainee should be able to read an ECG and add a diagnosis of left ventricular hypertrophy to the problem list; and a more advanced learner should be able to identify the diagnosis of diabetes on the basis of elevated hemoglobin A1c and glucose levels and also place multiple orders for appropriate evidence-based care.

Implementation in family medicine clerkship

The Sim-EHR curriculum was implemented for OHSU medical students in January 2011 as part of the required third-year family medicine clerkship. At the beginning of the academic year, an IT specialist duplicates the simulated chart to create one copy per learner; each copy bears a unique patient name. During the first week of the five-week clerkship, students receive their virtual patient’s name and attend a half-hour orientation session where a faculty facilitator provides an overview of the activity. The facilitator instructs learners to imagine they are interns taking care of a patient who has been recently discharged from the emergency department and has received fragmented care over the last five years because of a lack of health insurance. Learners must review and correct the chart in preparation for meeting this patient, create an evidence-based plan of care for prevention/ surveillance of the patient’s chronic diseases, and place indicated orders. At this session, the facilitator also distributes the learner guide, which outlines the objectives and grading guidelines and includes screenshots illustrating the step-by-step mechanics of manipulating a patient chart and placing orders.

Students are given two weeks to work in the chart. Small groups of six to eight students meet with a faculty facilitator during the second week of the clerkship for a two-hour session to discuss the first week’s work and the content of the “correct” chart (patient problem list, allergies, and history sections). They come to agreement on this content before beginning the next week of work in the chart, which involves editing the medication list, updating the health maintenance tool, and placing orders for future studies, referrals, and treatments (based on evidence-based guidelines). While the students are instructed to work individually on their charts during the first week, they are actively encouraged to work in groups for completion of the second week’s assignment. During the third week of the clerkship, the small groups assemble again for one hour to share the remainder of their work. Other topics touched upon in these sessions include the art of triaging care for complex patients, population-based EHR applications, and the professionalism, patient safety, and systems-based care reasons for excellent EHR chart hygiene habits.

To aid in these discussions, hard-copy summaries of each student’s chart work are available. In addition, the facilitator projects the open EHR onto a screen so that any student’s work can be displayed and reviewed by the group. To ensure consistent facilitation across groups, facilitators are asked to spend at least one hour prior to the first small-group session reviewing the simulated chart and the accompanying faculty guide, which outlines the session agenda and evidence-based recommendations for the virtual patient’s medical problems.

Student assessment

Students’ work in the virtual chart is assessed for this exercise, which counts for 5% of the family medicine clerkship grade. Faculty facilitators independently grade students’ work in the charts using a standardized rubric that addresses the three general objectives (see Table 1): chart maintenance skills (accuracy of history, allergies, and medication sections), knowledge of chronic disease management and disease prevention, and EHR skills (ability to place orders). For example, one point is given for each correct diagnosis included in the problem list. Students receive their simulated chart grades at clerkship completion. Those with questions about their grades are invited to review their work in the chart with a faculty member.

The EHR includes an audit trail with time and date stamps indicating when a student first starts work on the chart and documentation of the total amount of time spent on the exercise. Students who access the record within 24 hours of the start of the assignment receive a small increment of “extra credit” for their timely initiation of the work; they are made aware of this during the orientation session.

Resources required

The main resource the Sim-EHR curriculum requires is faculty time to create and update the virtual patient/ simulated chart, the grading rubric, and the learner and faculty guides. The curriculum also requires a core group of faculty small-group facilitators, one of whom will lead the orientation and be readily available to respond to learners’ questions during the week they are working on their charts. An IT specialist skilled in the EHR training environment is also needed to build and duplicate the simulated chart and address any IT issues that may arise. Importantly, the institution’s EHR system must include a segregated training environment that is not wiped clean of its data every day.

National Institutes of Health grant funding (NIH 1R25CA158571-01A1) supported faculty time for project development and startup (0.1 full-time equivalent [FTE]) and IT personnel time (0.2 FTE). This same grant funds ongoing IT support (0.1 FTE) and faculty evaluation and dissemination efforts (0.1 FTE). Getting support from OHSU and its hospital system was also critical. Early endorsement by the medical school dean and our academic medical center’s chief information officer helped ensure that students have remote access to the EHR training environment.

Chart completion to date

The curriculum remains part of the required family medicine clerkship. As of January 2014, three years after implementation, 406 students had completed simulated charts, both on campus and at remote clerkship sites in Oregon. The curriculum was also implemented over four hours of the OHSU internal medicine clinic orientations in July 2012 and July 2013; as of January 2014, 21 interns had completed simulated charts.

Faculty feedback

Small-group faculty facilitators met regularly from January to December 2011 to compare content discussed during and overall experiences of the small groups. The facilitators described students as actively engaged in the group discussions and identified the following common discussion points across groups: differing opinions regarding which diagnoses are chronic diseases as well as how and what to prioritize given the complexity of the patient’s issues; frustration over inadequacies of certain EHR functions; and uncertainty around how to respond to electronic queries when completing orders. They also reported that students were surprised as they discovered from one another the elements of the patient’s chart they had missed in their own reviews. On the basis of this feedback, we made some revisions to the faculty and learner guides.

Learner feedback

In August 2012, we e-mailed survey invitations to the 129 students and 12 IM interns who had by then completed the exercise to explore their perceptions of the Sim-EHR curriculum’s educational effectiveness. All of the students had learned their final grade in the clerkship. The survey, which was completed anonymously and hosted at SurveyMonkey, contained two questions regarding effectiveness of the activity and the facilitation.

All 12 interns and 67 (52%) of the students responded. The low response rate among students may be attributable to the long period of time (up to 18 months) between completion of the activity and receipt of the survey. Among respondents, about half (51%) of the students and almost all (92%) of the interns rated the activity as “effective” or “very effective”; the remaining 49% of students were evenly split between ratings of “neutral” and “ineffective.”

On the basis of student comments on the survey, it appears that many students felt more comfortable with finding information, inputting orders, and updating a health maintenance tool after completing the Sim-EHR curriculum. Some students recognized the value of the activity (“Vastly improved my chart navigating and documentation skills”; “I think EHR navigation is a huge part of the ‘job’ of being a doctor, and it should be explicitly taught like this”; “Helpful in preparing us for residency”). Others, however, expressed frustration with its timing (“This should be part of transition to clerkship, not the [family medicine] rotation”; “Probably most instructive for those who are in the first few blocks of the year”) and its opportunity costs (“Too time consuming for pay off ”; “Please just let us have time to study”).

Lessons Learned

Identifying strengths and gaps in the status quo

In September 2012, we conducted a preliminary analysis of student score data. When comparing data for students who completed the Sim-EHR exercise early versus later in their third year, we identified modest improvement in the more experienced students’ ability to place orders and update the chart, including the allergy section and medication list. However, we found no difference between the earlier and later students’ ability to use a health maintenance tool to create routine disease screening, prevention, and management alerts. We also found no difference in performance between all of the students and incoming interns, but we had data for only a small number of interns.

Such an analysis of score data may serve as a proxy for prioritizing which EHR skills need to be explicitly taught prior to graduation. We found universally low performance among both students and interns on specific objectives related to the knowledge of disease prevention, which suggests that there may be gaps in the predoctoral curriculum relating to the use of clinical decision support tools and the identification of evidence-based prevention guidelines. During our small-group sessions, students practice manipulating commonly used online tools and guidelines such as the World Health Organization’s Fracture Risk Assessment Tool (FRAX; http://www.shef.ac.uk/FRAX) and the University of Edinburgh’s cardiovascular risk calculator (http://cvrisk.mvm.ed.ac.uk/calculator/calc.asp). Potential enhancements to the Sim-EHR curriculum include placing an increased emphasis on training students to use these and other similar decision support tools in tandem with their EHR work.

Activity duration

Initially, students had three weeks to complete the Sim-EHR chart work, which was due in week four of the five-week clerkship. Audit trail data revealed that a majority of students completed the work one to seven days before the due date (i.e., they did not use the full period). Students participating in a June 2011 focus group reported that the timing of the Sim-EHR chart work competed with their final exam study time. Faculty facilitators reported that variable preparation by students, some of whom did minimal chart work, decreased the effectiveness of the small-group discussions. As a result of the audit trail analysis and comments, we changed the duration and timing of the activity. As noted above, Sim-EHR assignments are now distributed in week one of the clerkship, and the small-group discussions are held in weeks two and three. There is likely a point of diminishing returns with regard to time spent charting and resultant improved performance and educational benefit.

Elevating significance

The Sim-EHR exercise was initially graded pass/fail. Six months after implementation, because of variable student completion and lack of small-group preparation as described above, the significance of this activity was increased. As indicated above, it now represents 5% of the family medicine clerkship grade. This change resulted in overall improvement in performance, effort, and group session participation.

Looking Forward and Conclusions

Our experience with the Sim-EHR curriculum thus far is limited to third-year OHSU medical students and a small group of OHSU internal medicine interns. As of January 2014, two other academic health centers in the United States are planning to implement the Sim-EHR curriculum. At OHSU, we plan to expand the curriculum to family medicine interns and physician assistant students. We are considering methods for identifying control groups so that we may better evaluate the effectiveness of the intervention. In addition, we recently experimented with larger student groups (17–20 students) during the chart review sessions. We found that this resulted in richer discussions and opportunities for “team teaching”; it also reduced the required number of faculty facilitators. We expect to continue experimenting with group size and venue moving forward.

The Sim-EHR curriculum can be adapted to teach many aspects of medical care and for use with learners at many different levels. For example, a simulated chart could be created to teach about liver or renal disease, or a longitudinal aspect of care could be added. Elements of the exercise could be assigned by year of training. For example, first-year medical students could identify chart omissions and errors, second-year students could create problem lists and diagnoses based on test results, and third-year students could return to the chart to apply evidence to “treat” the patient.

In conclusion, we believe the Sim-EHR curriculum has a wide range of potential applications in clinical environments. While EHR use can decrease error and provide systems for delivering quality care, EHR adoption brings new challenges that can be addressed with sufficient hands-on training. The Sim-EHR curriculum is a way to reinforce, in a safe learning environment, important behaviors required for maintaining a well-organized chart that reflects current standards for chronic disease and routine prevention.