ABSTRACT
BACKGROUND
Increasing the use of electronic medical records (EMR) has been suggested as an important strategy for improving healthcare safety.
OBJECTIVE
To sequentially measure, evaluate, and respond to safety culture and practice safety concerns following EMR implementation.
DESIGN
Safety culture was assessed using a validated tool (Safety Attitudes Questionnaire; SAQ), immediately following EMR implementation (T1) and at 1.5 (T2) and 2.5 (T3) years post-implementation. The SAQ was supplemented with a practice-specific assessment tool to identify safety needs and barriers.
PARTICIPANTS
A large medical group practice with a primary care core of 17–18 practices, staffed by clinicians in family medicine, pediatrics, internal medicine.
INTERVENTIONS
Survey results were used to define and respond to areas of need between assessments with system changes and educational programs.
MAIN MEASURES
Change in safety culture over time; perceived impact of EMR on practice.
KEY RESULTS
Responses were received from 103 of 123 primary care providers in T1 (83.7 % response rate), 122 of 143 in T2 (85.3 %) and 142 of 181 in T3 (78.5 %). Safety culture improved over this period, with statistically significant improvement in all domains except for stress recognition. Time constraints, communications and patient adherence were perceived to be the most important safety issues. The majority of respondents in both T2 (77.9 %) and T3 (85.4 %) surveys agreed that the EMR improved their ability to provide care more safely.
CONCLUSIONS
Implementation of an EMR in a large primary care practice required redesign of many organizational processes, and was associated with improvements in safety culture. Most primary care providers agreed that the EMR allowed them to provide care more safely.
KEY WORDS: electronic medical records, EMR, safety culture, primary care, safety attitudes questionnaire
Despite efforts to improve patient safety, patients may still suffer harm during healthcare episodes. Because most healthcare services are delivered in primary care settings, understanding barriers to safe primary care is particularly important.1–3 Primary care physicians perceive time limitations and disorganized work systems as factors that influence safety,4,5 with communication, medication management, and test results tracking believed to be common sources of avoidable error.2,3,6,7 Work is underway to develop consensus about key processes and metrics to promote systematic safety monitoring and reduce risk in primary care.8
While working to develop these metrics, culture assessments have been used as indirect safety measures. Safety culture refers to a shared system of attitudes, values and practices that reflect an organization’s commitment to safety.9 An organization with a strong safety culture empowers staff to identify and report concerns, acknowledges errors, and allocates resources to promote these efforts.10 As safety culture improvement has been correlated with improved outcomes, assessing the safety culture and using results to guide education and program development can be used to enhance safety in healthcare systems.11–17
Increasing electronic medical records (EMR) use has been identified as a strategy to improve patient safety.18 EMR systems can help healthcare professionals and institutions organize data and improve workflows; surveys of clinicians who have adopted EMRs in ambulatory care settings support a perception of improved safety.19 However, many clinicians using EMRs often fail to utilize safety improvement functionalities, and EMR implementations can have unintended consequences, adversely impacting quality and safety.20–23 To promote use of such functionalities, the United States government has implemented standards for “meaningful use.”24
In 2007, our organization implemented an integrated EMR as part of ongoing safety and quality improvement efforts. In our post-implementation process, we completed a series of knowledge, quality and safety assessments to identify problems in our new EMR environment, using results to refine training and work processes in our clinical practices. We paid particular attention in our initial survey to primary care providers, due to their roles in patient care and team coordination. In this study, we focus on changes in perceptions of safety among our primary care provider group during the three years following EMR implementation.
METHODS
Setting
Our medical group practice is affiliated with an academic medical center and has central management functions coordinating quality, safety, operations, and information technology. Prior to EMR implementation, our group had electronic prescription management and business systems in place. In 2007, 17 practice sites delivered primary care services to approximately 150,000 patients. Between 2007 and 2009, one practice site split into two offices, for a total of 18 legacy primary care practice sites. Between 2009 and 2010, our organization expanded significantly and added additional practice sites. To compare historically similar units, we included only the 18 legacy practices in this evaluation.
EMR Implementation
We employed an extensive pre-implementation planning process similar to that described by others.25–28 This process included evaluation of several commercial EMR systems, hardware selection, and secure network installation to connect our practice sites. A medical director for informatics was named, and change champions were identified from multiple levels in our organization. Several new committees were developed, including an executive-level steering committee and a standards workgroup tasked with translating existing quality procedures into EMR workflows. A process for transferring data from the legacy prescribing and business systems was developed and tested.
Implementation of a new EMR (GE Healthcare Centricity) was completed in four phases, with “go-live” occurring between March 2007 and June 2007. The key phase features are shown in Table 1. EMR functionalities believed to improve quality and safety were progressively integrated into workflows, and their use was encouraged with incentives and regular feedback. Since 2007, we have completed two system upgrades; the current system (GE Centricity v9.5) is certified for meaningful use by the Certification Commission for Health Information Technology.
Table 1.
Implementation of Safety-Enhancing EMR Features
| EMR and Systems Processes | January 2006-February 2007 | March 2007 - June 2007 (T1) | July 2007 – June 2009 (T2) | July 2009 – Present (T3) |
|---|---|---|---|---|
| Pre-Implementation* | Implementation† | Stabilization‡ | Meaningful Use§ | |
| Organizing Information | ||||
| Electronic Medication Lists | Data conversiona | √ | √ | √ |
| Electronic Allergy Lists | Partial | √ | √ | √ |
| Electronic Problem Lists | Data conversionb | √ | √ | √ |
| Decision Support | ||||
| Drug Interactions & Allergy Checking | Partial | √ | √ | √ |
| Medication-Problem Correlation | No | Partial | √ | √ |
| Alerts for Abnormal Labs | No | Partial | √ | √ |
| Integrated Decision Support | No | Partial | √ | √ |
| Electronic provider Order Entry | No | Partial | √ | √ |
| Assessment Linking | No | Partial | √ | √ |
| Interface with External Data | ||||
| Lab Results Interface with Structured Data Entry | Partial | Partial | √ | √ |
| Scanning and Cataloging External Documents | No | Partial | √ | √ |
| Documentation | ||||
| Telephone Note Documentation | √ | √ | √ | √ |
| Office Visit Documentation | Partial | √ | √ | √ |
| Letters with Integrated Results | Partial | √ | √ | √ |
| Patient Specific Instructions | No | Partial | Partial | √ |
| Reminders and Tracking | ||||
| Completion Report for High Risk Tests | Partial | Partial | √ | √ |
| Patient Specific Alerts | No | Partial | √ | √ |
| Medication Reconciliation | Partial | Partial | √ | √ |
| Communication | ||||
| Feedback to Providers (Dashboards) | No | No | Partial | √ |
| Integrated Data Registry | No | No | No | √ |
| Data Quality Audits | Partial | Partial | √ | √ |
| Patient Portal | No | No | Partial | √ |
| Integrated Safety Audits | No | Partial | Partial | √ |
* Pre-implementation centered on data conversion from a previous electronic prescribing system and allowed users to familiarize themselves with new EMR as they used it for prescribing and limited clinical documentation. Super users and trainers provided some support during this time; most training was provided via written job aids. An assessment of EMR competencies was done. A direct lab interface was implemented
† Implementation occurred as a staged roll-out among 17 practices between March and June 2007; 3–4 practices went live every three weeks, with one trainer or super-user supporting 3–6 staff during the initial week; less support was provided in subsequent weeks. Productivity was reduced to 50 % of baseline during week 1 and 75 % during week 2. The first PSA-SAQ was completed at the end of this phase, in addition to a second competency assessment
‡ Stabilization allowed additional refinement of workflows and standardization of practice; the existing safety reporting system was changed. Reports were created to allow management to monitor workflow, workload and specific quality indicators; periodic provider feedback on these indicators was provided. Additional lab interfaces went live. A third assessment of EMR competency was completed and the second PSA-SAQ was completed in January 2009, near the end of this phase
§ The meaningful use phase was marked by software updates to provide additional functionality and decision support, acquisition of a data registry system, improved report generation and implementation of monthly feedback to providers. A third PSA-SAQ was completed
aFrom legacy electronic prescribing system
bFrom business system diagnosis codes
Assessment & Training
EMR training included a combination of self-study, work with super-users, and on-site training. Immediately prior to go-live, staff attended one four-hour classroom training session. Schedules were reduced to 50 % and then 75 % of baseline during the first two weeks, and on-site “super-user” support was provided. During and after implementation, input from a variety of indicators, including the safety assessments reported below, was used to inform and refine our EMR workflows, training, and support processes.
Assessment of Safety Needs and Culture
We assessed safety needs in our practices for the first time at the end of EMR implementation, in June 2007 (T1). We repeated the assessment in January 2009, 1.5 years after implementation (T2); and again in December 2010, approximately 2.5 years (T3) after the initial assessment. Each assessment bundled safety culture surveys and practice-specific needs into a web-based response collector, and links to the collector were distributed by email to employees. The T1 survey was completed using a general survey tool (SurveyMonkey.com, LLC; Palo Alto, CA), and T2 and T3 surveys were completed using a commercial surveyor (Pascal Metrics HealthBench™ Version 1.7; Pascal Metrics Inc., Washington, DC). Our institutional review board approved this study.
Safety culture was assessed with the Safety Attitudes Questionnaire (SAQ), a well-studied and psychometrically validated tool that categorizes organizational safety culture along seven domains: teamwork, safety climate, job satisfaction, stress recognition, working conditions, and perceptions of local and executive management.29 The SAQ rates agreement with a series of statements (e.g., “Medical errors are handled appropriately in this office”) on a five-point Likert scale, with both “agree” and “strongly agree” counting as a positive response. Positive responses are reported as a percentage of all responses. The percentage of positive responses in each domain suggests actionable items, with <60 % indicating “high risk” areas, and 60 % to ≤80 % suggesting areas that need attention.
Minor changes in SAQ question wording, as described by others, were made to tailor the survey to our ambulatory practices.30 Twenty-nine questions were common to all SAQ surveys. (Table 2) Responses were compared directly to evaluate our safety culture over time.
Table 2.
Characteristics of Survey Respondents
| T1 | T2 | T3 | |
|---|---|---|---|
| 2007 | 2009 | 2010 | |
| Respondent Counts | 103 | 122 | 142 |
| Respondents Surveyed | 123 | 143 | 181 |
| Percent Responding | 83.7 % | 85.3 % | 78.5 % |
| Respondent Characteristics | |||
| % Female | n.a. | 61.4 % | 63.2 % |
| Care for Adult Patients | 41.7 % | 45.5 % | 45.1 % |
| Care for Pediatrics | 20.4 % | 17.2 % | 17.6 % |
| Care for Both Adult and Pediatrics | 37.9 % | 37.3 % | 37.3 % |
Each survey was supplemented by a practice-specific needs assessment, developed by quality and safety leadership to assess actionable issues specific to our practices at a more detailed level than permitted by the SAQ. Issues presented through a variety of channels, including safety reports, committee input, and informal communications. Because most leaders were super-users, we directly observed how clinicians used and misused the EMR. Some practice-specific needs assessment questions were common to all surveys. For example, each survey requested respondents to indicate their greatest safety concern. In T1, the question was phrased “Of the issues listed below, which one do you think has the most important impact on safety.” In T2 and T3, that question was phrased as “Of the (safety) issues listed, which one do you think needs management attention the most.” In T2 and T3, the practice-specific needs assessment asked respondents to rate the EMR impact on safety.
Other practice-specific needs assessment questions were designed to collect detailed data on specific issues and were not repeated. For example, T1 included the open response question, “What can management do to improve safety issues reporting?”, which directly relates to the ‘safety climate’ SAQ domain. We addressed these issues, and monitored the impact of our efforts with the SAQ ‘safety climate’ domain in subsequent assessments.
Statistical Analysis
The percentage of positive responses was calculated using a denominator that excluded non-respondents and respondents answering “not applicable”. To calculate “domain” averages shown in Table 3, the percentage of positive responses in questions comprising that domain were averaged, after correcting for negatively worded questions by subtracting percent agreement from 100 %. We used chi-square test to calculate P-values assuming independent samples from all three years. Since survey data were collected anonymously, we were not able to adjust for the possible within-person correlations across three years.
Table 3.
Comparison of Safety Attitudes Questionnaire Results
| T1 | T2 | T3 | P values* | |||
|---|---|---|---|---|---|---|
| 2007 | 2009 | 2010 | T1 vs T2 | T2 vs T3 | T1 vs T3 | |
| Job Satisfaction | 74.1 % | 78.2 % | 86.2 % | 0.020 | ||
| I like my job | 85.2 % | 85.8 % | 89.4 % | |||
| Working here is like being part of a large family | 61.6 % | 71.1 % | 80.0 % | .007 | ||
| This work setting is a good place to work | 82.8 % | 83.4 % | 90.8 % | |||
| I am proud to work in this work setting | 87.4 % | 84.4 % | 91.5 % | |||
| Morale in this work setting is high | 53.3 % | 66.5 % | 79.4 % | 0.017 | <0.0001 | |
| Perceptions Of Executive Management | 59.1 % | 66.7 % | 72.6 % | 0.038 | ||
| I get adequate, timely info about events that might affect my work | 71.1 % | 79.4 % | 76.5 % | |||
| The staffing levels in this work setting are sufficient to handle the number of patients | 47.1 % | 53.9 % | 68.7 % | 0.016 | 0.001 | |
| Perceptions Of Local Management | 76.2 % | 84.6 % | 86.0 % | 0.043 | ||
| Practice management doesn't knowingly compromise the safety of patients | 86.4 % | 90.3 % | 94.3 % | 0.044 | ||
| Practice management supports my daily efforts | 77.5 % | 87.8 % | 85.9 % | |||
| I get adequate, timely info about events that might affect my work | 64.8 % | 75.7 % | 77.9 % | 0.028 | ||
| Safety Climate | 76.4 % | 84.2 % | 87.8 % | 0.037 | ||
| I would feel safe being treated here as a patient | 90.1 % | 89.4 % | 93.6 % | |||
| Medical errors are handled appropriately in this work setting | 79.2 % | 94.7 % | 90.5 % | <0.0001 | 0.024 | |
| I know the proper channels to direct questions regarding patient safety in this work setting | 87.8 % | 91.3 % | 92.2 % | |||
| I receive appropriate feedback about my performance | 71.5 % | 75.0 % | 83.5 % | 0.040 | ||
| In this work setting, it is difficult to discuss errors | 12.8 % | 14.0 % | 14.2 % | |||
| I am encouraged by others to report any patient safety concerns I may have | 65.7 % | 81.6 % | 92.3 % | 0.009 | 0.024 | <0.0001 |
| The culture in this work setting makes it easy to learn from the errors of others | 52.9 % | 71.4 % | 76.9 % | 0.008 | <0.0001 | |
| Stress Recognition | 68.4 % | 75.6 % | 74.8 % | |||
| When my workload becomes excessive my performance is impaired | 80.6 % | 84.1 % | 82.9 % | |||
| I am less effective at work when fatigued | 89.0 % | 90.5 % | 86.6 % | |||
| I am more likely to make errors in tense or hostile situations | 74.0 % | 75.1 % | 73.9 % | |||
| Fatigue impairs my performance during emergency situations | 29.9 % | 52.6 % | 55.6 % | 0.001 | <0.0001 | |
| Teamwork Climate | 77.4 % | 85.5 % | 88.9 % | 0.020 | ||
| Nurse input is well received in this work setting | 82.3 % | 92.8 % | 92.7 % | 0.024 | 0.024 | |
| In this work setting, it is difficult to speak up if I perceive a problem with patient care | 17.1 % | 16.8 % | 12.1 % | |||
| Disagreements in this work setting are resolved appropriately | 69.3 % | 78.5 % | 82.3 % | 0.014 | ||
| I have the support I need from others in this work setting to care for patients | 76.0 % | 83.9 % | 87.3 % | 0.017 | ||
| It is easy for personnel here to ask questions | 76.4 % | 88.9 % | 94.3 % | 0.021 | <0.0001 | |
| Working Conditions | 74.3 % | 74.2 % | 84.9 % | 0.044 | ||
| All the necessary information for diagnostic and therapeutic decisions is routinely available | 80.9 % | 79.7 % | 87.9 % | |||
| Trainees in my discipline are adequately supervised | 73.9 % | 77.7 % | 88.1 % | 0.029 | 0.004 | |
| This facility does a good job of training new personnel | 68.2 % | 65.2 % | 78.6 % | 0.026 | ||
* P-values reflect chi squared analysis comparing T1, T2 and T3 assuming all are independent. P-values are not reported if they are >0.05. Table presents the percent of respondents who “agreed” or “strongly agreed” with the indicated statement. Overall domain represent averaged responses to survey questions within the domain, and are highlighted
RESULTS
Response Rates and Respondent Demographics
The response rates and respondent characteristics for our safety assessments are shown in Table 2. Of primary care providers surveyed, 83.7 %, 85.3 %, and 78.5 % completed surveys at T1, T2, and T3, respectively. Despite growth of our primary care provider group during the study period, the distribution of characteristics and specialties was similar in all three surveys. More than 80 % of primary care providers included in the T1 survey were present at T3.
Comparison of Safety Attitudes Questionnaire Results
A comparison of positive responses to questions common to the three SAQ surveys is shown in Table 3. There were non-significant incremental improvements for most domains between T1 and T2, and again between T2 and T3. When assessing improvements in the 2.5 years between the surveys, these combined incremental gains resulted in significant improvements in five of seven domains assessed. Between T1 and T3, there were improvements in job satisfaction (74.1 % vs. 86.8 %, P < 0.05), perceptions of executive (59.1 % vs. 72.6 %, P < 0.05) and local management (76.2 % vs. 86.2 %, P < 0.05), safety climate (76.4 % vs. 87.8 %, P < 0.05), and teamwork climate (77.4 % vs. 88.9 %, P < 0.05). A significant improvement in the working conditions domain occurred between T2 and T3 (74.3 vs. 84.9, P < 0.05). There was a non-significant trend towards improvement in the stress domain. Table 3 also shows significant improvements in several sub-domains over the 1.5 years between T1 and T2, particularly within the safety domain, with significantly improved agreement that medical errors were handled appropriately (79.2 % vs. 93.6 %, P < 0.05), that reporting was encouraged (65.7 % vs. 92.3 %, P < 0.0001), and that it was easy to learn from errors of others (52.9 % vs. 76.9 %, P < 0.0001). A graphic representation of safety culture domain changes is shown in Figure 1.
Figure 1.
Comparison of safety culture by domain for primary care providers after EMR implementation. Safety culture domains were assessed by SAQ response among primary care providers immediately following EMR implementation (T1), 1.5 years after implementation (T2) and approximately 2.5 years after implementation (T3). Chi squared analysis is shown for T1 vs. T3, (*T2 vs. T3) and shows significant improvements (P < 0.05) in domains other than stress recognition.
Practice-Specific Needs Assessments
All practice-specific needs assessment surveys asked providers to identify what they perceived as the single most important safety issue affecting their practices. Results are shown in Table 4. Time constraints were the most significant concern (34.0 % in T1, 35.3 % in T2, and 30.8 % in T3), followed by communication problems. Test result tracking (11.0 %, 7.6 %, and 9.0 %, for T1, T2, and T3, respectively) and medication management (2.0 %, 0.0 %, and 3.0 %, for T1, T2, and T3, respectively) were seen as less significant. EMR issues were ranked as “most important” by only 2 % of providers in T1 and 5.3 % in T3, but 16.8 % of providers in T2 identified EMR-related issues as the most important safety concern. Conversely, only 5 % of respondents rated patient adherence as “most important” in T2, with over 20 % ranking it as “most important” in T1 and T3.
Table 5.
Safety Issues, Assessments and Resulting Safety Enhancements
| Safety Process | Issues Identified | Assessments | Response | Monitoring & Impact |
|---|---|---|---|---|
| Safety Issues Reporting | Significant volumes of safety concerns during implementation: Many made by email or verbal report rather than established process. Some were reported to information systems technical staff, who were perceived as “responsible” for some EMR functionalities. Technicians inconsistently recognized clinical significance, delaying response to some quality issues. | Assessment of barriers to reporting: The T1 assessment included questions to determine barriers to reporting safety issues: 36 % felt the system was hard to use; 19 % requested more training; 18 % were concerned about blame; 10 % felt there was lack of follow-up. Providers indicated a preference for reporting through an existing non-secure communication system within the new EMR. | New reporting system: We implemented a proxy system within the EMR that allowed clinicians to notify safety management about concerns after the T1 assessment. Safety management staff contacted the reporting clinician and completed a formal report. This made reporting easier, and allowed safety staff to respond directly to the reporter. | Improved Safety Culture, particularly in Safety Climate domain: We monitored impact of the new reporting system in the “Safety Climate” portions of T2 and T3 surveys. |
| Pre-EMR: | There was a 37 % increase in safety issues that were formally logged between 2007 and 2008 after implementing the new process. | |||
| To report a safety concern, staff members were required to open a secure external database and enter data themselves. | ||||
| Test Result Tracking | High risk test results were not consistently tracked: Standards for test result tracking and notification were not used consistently. Some clinicians notified patients only about abnormal test results, despite a standard that patients were to be notified of all results. Practice managers charged with implementing workflows were not consistently trained in EMR use. | Most important safety issue: Clinician concerns about tracking test results was queried in all surveys; it was 4th most important in T1. An EMR competency and knowledge assessment of clinicians and practice managers charged with tracking reports was done. An annual patient satisfaction survey included a question on test results notification. | Communication and education: Training materials were modified. An annual requirement for 8 hours of EMR training was implemented. Team-based training was done in practices. Patient satisfaction surveys were tied to performance evaluation. An ad campaign, asking patients to call if not notified of test results was developed. | Improvements in Safety Culture: We monitored impact particularly in the Working Conditions domain that focuses on training, and Teamwork Climate. |
| Pre-EMR | Test result notification metrics in the annual patient satisfaction survey showed a trend towards improvement. | |||
| Significant variation in implementation of test result tracking and patient notification standards existed among the primary care practice sites. | ||||
| Monitoring Workload and Workflow | Clinician workload and task completion varied: After EMR implementation, some providers maintained extensive desktop document queues. This presented safety concerns. Compliance reviews indicated that many of the same providers struggled to complete visit notes, review incoming results and complete prescription refills. | Monitoring workload and workflow: After defining expected turnaround times for standard office tasks regarded as markers for safety (document completion, prescription refills, laboratory test review) we developed reports to monitor adherence to time lines. Safety concerns regarding workload were monitored. (Table 4) | Enhanced clinician and support staff training: Clinicians with excessive work queues were paired with super-users to coach effective EMR use. Penalties for late documentation and bonuses for achieving targets were initiated. Standing protocols to reduce clinician workload and improve service delivery were implemented. | Improvements in Safety Culture: We monitored impact particularly in the Management domain “supports daily efforts” and “staffing levels adequate”. |
| Pre-EMR | Reports indicated adherence to time targets: prescriptions completed within 2 days of request; office notes completed within 2 days of visit. | |||
| Significant variation in workflow efficiency among clinicians was suspected, but systems for monitoring this were unavailable. |
Table 4.
Safety Concern Ranked Most Significant
| Most Important Safety Issue | T1 | T2 | T3 |
|---|---|---|---|
| 2007 | 2009 | 2010 | |
| Responses, N (% Total) | 101 (98.1 %) | 119 (97.5) | 133 (93.7) |
| Time constraints | 34.0 % | 35.3 % | 30.8 % |
| Patient adherence | 23.0 % | 5.0 % | 26.3 % |
| Communication problems | 20.0 % | 21.0 % | 21.8 % |
| Test result tracking | 11.0 % | 7.6 % | 9.0 % |
| EMR issues | 2.0 % | 16.8 % | 5.3 % |
| Medication lists | 2.0 % | 0.0 % | 3.0 % |
| Other/None identified | 8.0 % | 14.3 % | 3.8 % |
| Total | 100.0 % | 100.0 % | 100.0 % |
Perceived Impact of EMRs
The practice-specific needs assessment portion of T2 and T3, but not T1, asked respondents to indicate the extent to which they agreed with the question “Our electronic medical record has improved our ability to provide safe patient care”. Agreement was high in T2 and remained high in T3 (77.9 % and 85.4 %, respectively; P = 0.12).
Use of Assessments to Inform EMR and Organizational Processes
Safety assessments were conducted in conjunction with other analysis to inform decision making and facilitate meaningful use of the EMR; several applications are outlined in Table 5. For example, the practice-specific needs assessment in T1 asked respondents to indicate barriers to safety issues reporting in our new EMR environment. Responses were grouped into actionable themes, such as making it easier to report, providing more training and assuring blamelessness. Because of this, we implemented a system that allowed clinicians to notify management about safety concerns through a new EMR communication pathway immediately after the T1 assessment. Processes to improve test results tracking and workflow monitoring were also informed and guided by these assessments.
DISCUSSION
Primary care providers are challenged to provide safe and compassionate care to patients while managing large amounts of data in work systems that are often complex and poorly organized.4,5 Broadening EMR use has been suggested as a strategy to improve safety in primary care settings,31,32 but without careful monitoring implementations may have a negative impact on safety.21–23 Safety culture assessments have been used to evaluate and monitor the impact of safety improvement programs in complex settings.12–14,33,34 We used a validated safety culture survey in combination with practice-specific needs assessments to monitor and respond to safety issues after implementing an electronic medical record system. To our knowledge, assessment of safety culture proximate to an EMR implementation in a large primary care group has not previously been reported.
We found sustained improvement in safety culture over the three years that followed EMR implementation (Table 3, Fig. 1). Scores were lowest in the survey that occurred immediately after implementing paperless documentation in our primary care practices (T1). Comparison of responses in our three safety culture surveys showed improvement within its domains, which were significant between in all areas except stress recognition. In addition, the majority of our primary care providers agreed that the EMR itself enhanced their ability to provide care more safely.
Prior to EMR implementation, our organization did not regularly conduct safety culture assessments. As others have shown, adoption of a periodic safety culture assessment process can raise safety awareness in healthcare settings, and thereby contribute to ongoing improvements.13,14 The goal of improving safety was broadly communicated as one of our medical group practice‘s reasons for adopting the new EMR. The communications, training, and management processes that were integral to our EMR implementation plan, and assured that our workforce used the EMR consistently, would have directly impacted many of the domains measured by the safety culture assessment tool.
We believe that early adoption of many safety-enhancing EMR features also promoted safety culture improvements and the providers’ positive response to the EMR. Prior to implementation, most prescribing had been done in our legacy system, and this continued after transitioning to the new EMR. Electronic laboratory reports were available within our new EMR prior to T1. Further, at the time of the T1 survey, more than 90 % of active patients had medication and problem lists in the EMR, thanks to the data transfer from our legacy systems; virtually all active patients had these in place by T2. Regular feedback of quality and safety data to providers began as early as T1 and expanded during T2 and T3 intervals.
The practice-specific needs assessments were developed by management to characterize emerging safety issues, and complemented the SAQ in helping us define and respond to specific problems. For example, after providers indicated barriers to safety issues reporting in T1, we developed a new reporting system and addressed training between T1 and T2. We were able to monitor the impact of this change by following elements of the ‘safety climate’ SAQ domain, noting significant and sustained improvements. Emerging concerns about workflow management and test results tracking were addressed similarly. We continue to monitor perceptions of “the most important safety issue” in our practice-specific needs assessment and believe that this helps focus management efforts.
As shown in Table 3, time constraints and communications issues were consistently perceived as important safety issues by our providers. Relatively few staff identified medication management as ”the most important safety issue,” which contrasts with observations that medication management is a frequent source of medical error.3,5,35 There are several possible explanations for this difference. Our survey language may have mitigated the intention of assessing “medication errors” (the option was “medication reconciliation” in T1 and “medication lists” in T2 and T3), or our providers may have under-estimated the significance of this issue. However, we believe these responses reflect the impact of electronic prescription management systems in the EMR. We observed in T2 that concerns with “EMR issues” exceeded concerns with patient adherence. We are uncertain about the significance of this finding, but anecdotally noted around that time a tendency of providers to attribute safety issues broadly to “the EMR.”
Our study is subject to several limitations. This was an observational study, and there was no control group. We cannot determine what part of the culture change was related to the EMR, and what part related to the management, communications and educational interventions that accompanied implementation. No data specific to our organization was available prior to T1, which precludes determination of the pre-EMR culture. The T1 assessment may have been negatively biased, due to its delivery immediately following implementation. Additionally, secular attention to enhancing patient safety at national and local levels, or changes outside issues we specifically evaluated, could have caused the culture changes noted during the study period. A statistical limitation is that the response of each individual respondent may have been dependent on his or her first response, but due to de-identification of responses, we were unable to analyze the second administration as a repeat test at the individual level. Further, while a unit level of analysis is typically used for culture analysis, we compared our culture surveys at an organizational level, due to potential provider overlap between practices.36,37 These limitations are mitigated by comparing the same primary care practices over time. Finally, we do not have information on non-respondents since the surveys were conducted anonymously; however, similar respondent characteristics and response rates mitigates this limitation.
We believe that use of the safety culture and practice-specific assessments described above were valuable aids in steering our group’s EMR implementation. While we have not demonstrated patient safety outcomes, other data suggests that improved safety culture is positively correlated with patient safety.12,13,15,16 Implementing regular culture surveys as we adopted our EMR emphasized that safety was an organizational priority. We found that the surveys were useful in “taking the pulse” of our organization during this period of change and subsequent stabilization, and sequential results suggested we were on track with our overall goals of improving safety. Our experience with this safety assessment process may help others who are working through transitions into electronic medical record systems.
Acknowledgements
We would like to acknowledge our implementation and management teams for major contributions to the strategies listed in Table 1. Medical editor Michael Linde assisted in the preparation of this manuscript.
Conflict of Interest
The authors declare that they do not have a conflict of interest.
REFERENCES
- 1.Rosser W, Dovey S, Bordman R, White D, Crighton E, Drummond N. Medical errors in primary care: results of an international study of family practice. Can Fam Physician. 2005;51:386–7. [PMC free article] [PubMed] [Google Scholar]
- 2.Fernald DH, Pace WD, Harris DM, West DR, Main DS, Westfall JM. Event reporting to a primary care patient safety reporting system: a report from the ASIPS collaborative. Ann Fam Med. 2004;2:327–32. doi: 10.1370/afm.221. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Elder NC, Dovey SM. Classification of medical errors and preventable adverse events in primary care: a synthesis of the literature. J Fam Pract. 2002;51:927–32. [PubMed] [Google Scholar]
- 4.Manwell LB, Williams ES, Babbott S, Rabatin JS, Linzer M. Physician perspectives on quality and error in the outpatient setting. WMJ. 2009;108:139–44. [PubMed] [Google Scholar]
- 5.Linzer M, Manwell LB, Williams ES, et al. Working conditions in primary care: physician reactions and care quality. Ann Intern Med. 2009;151(28–36):W6–9. doi: 10.7326/0003-4819-151-1-200907070-00006. [DOI] [PubMed] [Google Scholar]
- 6.Tsang C, Palmer W, Bottle A, Majeed A, Aylin P. A review of patient safety measures based on routinely collected hospital data. Am J Med Qual. 2011. [DOI] [PubMed]
- 7.Sandars J, Esmail A. The frequency and nature of medical error in primary care: understanding the diversity across studies. Fam Pract. 2003;20:231–6. doi: 10.1093/fampra/cmg301. [DOI] [PubMed] [Google Scholar]
- 8.Kingston-Reichers J, Ospina M, Jonsson E, Childs P, McLeod L, Maxted D. Patient Safety in Primary Care. Edmonton AB: Canadian Patient Safety Institute and BC Patient Safety and Quality Council; 2010. [Google Scholar]
- 9.Wiegmann DA, Zhang H, von Thaden TL, Sharma G, Mitchell AA. A Synthesis of Safety Culture and Safety Climate Research. University of Illinois Aviation Research Lab Technical. 2002.
- 10.Shojania KG, Duncan BW, McDonald KM, Wachter RM, Markowitz AJ. Making health care safer: a critical analysis of patient safety practices. In: Quality AfHRa, ed. Rockville, MD: Agency for Healthcare Research and Quality; 2001. [PMC free article] [PubMed]
- 11.Colla JB, Bracken AC, Kinney LM, Weeks WB. Measuring patient safety climate: a review of surveys. Qual Saf Health Care. 2005;14:364–6. doi: 10.1136/qshc.2005.014217. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Singer S, Lin S, Falwell A, Gaba D, Baker L. Relationship of safety climate and safety performance in hospitals. Health Serv Res. 2009;44:399–421. doi: 10.1111/j.1475-6773.2008.00918.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Pronovost P, Sexton B. Assessing safety culture: guidelines and recommendations. Qual Saf Health Care. 2005;14:231–3. doi: 10.1136/qshc.2005.015180. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Sexton JB, Paine LA, Manfuso J, et al. A check-up for safety culture in "my patient care area". Jt Comm J Qual Patient Saf. 2007;33:699–703. doi: 10.1016/s1553-7250(07)33081-x. [DOI] [PubMed] [Google Scholar]
- 15.Weaver MD, Wang HE, Fairbanks RJ, Patterson D. The association between EMS workplace safety culture and safety outcomes. Prehosp Emerg Care. 2011;16(1):43–52. [DOI] [PMC free article] [PubMed]
- 16.Mardon RE, Khanna K, Sorra J, Dyer N, Famolaro T. Exploring relationships between hospital patient safety culture and adverse events. J Patient Saf. 2010;6:226–32. doi: 10.1097/PTS.0b013e3181fd1a00. [DOI] [PubMed] [Google Scholar]
- 17.Pronovost PJ, Berenholtz SM, Goeschel C, et al. Improving patient safety in intensive care units in Michigan. J Crit Care. 2008;23:207–21. doi: 10.1016/j.jcrc.2007.09.002. [DOI] [PubMed] [Google Scholar]
- 18.Bates DW, Ebell M, Gotlieb E, Zapp J, Mullins HC. A proposal for electronic medical records in U.S. primary care. J Am Med Inform Assoc. 2003;10:1–10. doi: 10.1197/jamia.M1097. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.DesRoches CM, Campbell EG, Rao SR, et al. Electronic health records in ambulatory care—a national survey of physicians. N Engl J Med. 2008;359:50–60. doi: 10.1056/NEJMsa0802005. [DOI] [PubMed] [Google Scholar]
- 20.Simon SR, McCarthy ML, Kaushal R, et al. Electronic health records: which practices have them, and how are clinicians using them? J Eval Clin Pract. 2008;14:43–7. doi: 10.1111/j.1365-2753.2007.00787.x. [DOI] [PubMed] [Google Scholar]
- 21.Han YY, Carcillo JA, Venkataraman ST, et al. Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system. Pediatrics. 2005;116:1506–12. doi: 10.1542/peds.2005-1287. [DOI] [PubMed] [Google Scholar]
- 22.Ash JS, Berg M, Coiera E. Some unintended consequences of information technology in health care: the nature of patient care information system-related errors. J Am Med Inform Assoc. 2004;11:104–12. doi: 10.1197/jamia.M1471. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Harrison MI, Koppel R, Bar-Lev S. Unintended consequences of information technologies in health care—an interactive sociotechnical analysis. J Am Med Inform Assoc. 2007;14:542–9. doi: 10.1197/jamia.M2384. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Blumenthal D, Tavenner M. The "meaningful use" regulation for electronic health records. N Engl J Med. 2010;363:501–4. doi: 10.1056/NEJMp1006114. [DOI] [PubMed] [Google Scholar]
- 25.Brokel JM, Harrison MI. Redesigning care processes using an electronic health record: a system's experience. Jt Comm J Qual Patient Saf. 2009;35:82–92. doi: 10.1016/s1553-7250(09)35011-4. [DOI] [PubMed] [Google Scholar]
- 26.Yoon-Flannery K, Zandieh SO, Kuperman GJ, Langsam DJ, Hyman D, Kaushal R. A qualitative analysis of an electronic health record (EHR) implementation in an academic ambulatory setting. Inform Prim Care. 2008;16:277–84. doi: 10.14236/jhi.v16i4.703. [DOI] [PubMed] [Google Scholar]
- 27.Lorenzi NM, Kouroubali A, Detmer DE, Bloomrosen M. How to successfully select and implement electronic health records (EHR) in small ambulatory practice settings. BMC Med Inform Decis Mak. 2009;9:15. doi: 10.1186/1472-6947-9-15. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Zandieh SO, Yoon-Flannery K, Kuperman GJ, Langsam DJ, Hyman D, Kaushal R. Challenges to EHR implementation in electronic- versus paper-based office practices. J Gen Intern Med. 2008;23:755–61. doi: 10.1007/s11606-008-0573-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Sexton JB, Helmreich RL, Neilands TB, et al. The safety attitudes questionnaire: psychometric properties, benchmarking data, and emerging research. BMC Health Serv Res. 2006;6:44. doi: 10.1186/1472-6963-6-44. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Modak I, Sexton JB, Lux TR, Helmreich RL, Thomas EJ. Measuring safety culture in the ambulatory setting: the safety attitudes questionnaire–ambulatory version. J Gen Intern Med. 2007;22:1–5. doi: 10.1007/s11606-007-0114-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.El-Kareh R, Gandhi TK, Poon EG, et al. Trends in primary care clinician perceptions of a new electronic health record. J Gen Intern Med. 2009;24:464–8. doi: 10.1007/s11606-009-0906-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Singh R, Servoss T, Kalsman M, Fox C, Singh G. Estimating impacts on safety caused by the introduction of electronic medical records in primary care. Inform Prim Care. 2004;12:235–42. doi: 10.14236/jhi.v12i4.131. [DOI] [PubMed] [Google Scholar]
- 33.Paine LA, Rosenstein BJ, Sexton JB, Kent P, Holzmueller CG, Pronovost PJ. Assessing and improving safety culture throughout an academic medical centre: a prospective cohort study. Qual Saf Health Care. 2010;19:547–54. doi: 10.1136/qshc.2009.039347. [DOI] [PubMed] [Google Scholar]
- 34.Nieva VF, Sorra J. Safety culture assessment: a tool for improving patient safety in healthcare organizations. Qual Saf Health Care. 2003;12(Suppl 2):ii17–23. doi: 10.1136/qhc.12.suppl_2.ii17. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Poon EG, Gandhi TK, Sequist TD, Murff HJ, Karson AS, Bates DW. "I wish I had seen this test result earlier!": dissatisfaction with test result management systems in primary care. Arch Intern Med. 2004;164:2223–8. doi: 10.1001/archinte.164.20.2223. [DOI] [PubMed] [Google Scholar]
- 36.Klein KJ, Bliese PD, Kozolowski SWJ, et al. Multilevel analytical techniques: commonalities, differences, and continuing questions. In: Klein KJ, Kozlowski SWJ, et al., editors. Multilevel Theory, Research, and Methods in Organizations: Foundations, Extensions, and New Directions. San Francisco, CA: Jossey-Bass, Inc.; 2000. pp. 512–53. [Google Scholar]
- 37.Zohar D. Safety climate in industrial organizations: theoretical and applied implications. J Appl Psychol. 1980;65:96–102. doi: 10.1037/0021-9010.65.1.96. [DOI] [PubMed] [Google Scholar]