Electronic Medical Record in Pediatric Intensive Care: Implementation Process Assessment

Abstract

The implementation of an electronic medical record (EMR) is a high-priority project in a majority of industrialized countries. The Healthcare Information and Management Systems Society (HIMSS) Analytics established an eight-stage EMR Adoption Model (EMRAM) to track progress against health care organizations across a country. In Canada, 36.5% of the hospitals are at the stage 3 or higher, whereas 0.2% have reached the seventh stage. To assess the impact on the safety and caregivers' satisfaction of a stage 7 EMR in a Quebec Pediatric Hospital initially at the EMRAM stage 3, a pilot customized implementation of paperless pediatric intensive care EMR was performed and evaluated. Six months after implementation, there was a nonsignificant decrease in severe medical incidents in comparison to the same period of time, the previous year. Most pediatric intensive care unit (PICU) staff were very or completely comfortable with the EMR, but the EMR satisfied 33.9% of all staff (everyday users [internal staff] and occasional user [external staff]) and 41.9% of internal staff only. The information gathered with this pilot EMR implementation using a 20-month preparation period and a continuous monitoring including change management (“living lab approach”) after the “go live” helped in the success of the implementation but did not improve significantly caregivers' satisfaction, in the first 6 months of this dramatic change in practice.

Introduction

The implementation of an electronic medical record (EMR) is a high-priority project in a majority of industrialized countries. The Healthcare Information and Management Systems Society (HIMSS) Analytics established an EMR Adoption Model (EMRAM) that classified hospitals in eight stages with the penultimate stage of a fully paperless environment.¹ In 2015, 3.7% of US hospitals had reached the seventh stage and 91.2% had reached stage three or higher, which included partial electronic nursing and clinical documentation, some clinical decision support system (CDSS) with error checking, and picture archiving systems available excluding radiology.² Canada has 36.5% of its hospitals that correspond to the stage 3 criteria or higher, whereas 0.2% have reached the seventh stage.

In the province of Quebec, the implementation of EMR in hospitals is closely controlled by the Quebec government and no hospital has reached the six or seven EMRAM stage yet. To assess the implementation impact on the delivery of care of a stage 7 EMR in a Quebec Pediatric Hospital, Sainte-Justine Hospital (CHUSJ), an EMRAM stage 3 hospital, it was decided to initially implement a complete paperless EMR in a single unit. The pediatric intensive care unit (PICU) was chosen because they were in favor of such an implementation, a specific complete paperless EMR was commercialized (IntelliSpace Critical Care and Anesthesia [ICCA], Philips Healthcare, France), and this software was already used in several French-speaking PICU in France (lack of language barrier). This EMR included a clinical information system, computerized physician order entry system, and an automated collection of data from medical devices. The objective was to study the model used in implementing this EMR and while assessing patient safety and caregivers' comfort during the first 6 months of the implementation.

Materials and Methods

Electronic Medical Record Description

The study took place in the PICU of CHUSJ a tertiary academic center in a free-standing pediatric hospital and evaluated satisfaction after implementation of an EMR named ICCA (Philips Healthcare, Suresnes, France). CHUSJ PICU is a medical and surgical center, including cardiac surgery PICU with 24 ICU beds and 6 beds of intermediate care. About 1,000 patients a year of all ages who come from all areas of Quebec to receive tertiary and quaternary care are admitted in this PICU. PICU caregivers include 10 physicians, 10 fellows/residents, approximately 100 nurses, and 40 respiratory therapists. The selection of the EMR software was made following a questionnaire in 2009 to the caregivers that documented that the second priority of the PICU should be the development of an EMR to avoid transcription of vital signs and laboratory examinations and to improve access to the medical record (the first priority was continuous nursing education).

Software

A description of the software is essential to understand the consequences of its implementation. The ICCA software is connected to a server that enables access to the medical records from multiple computers. Physiologic monitors, administrative data concerning admission-discharge-transfer (ADT), laboratory tests, mechanical ventilators, intravenous pumps and syringes, and the pharmacy are connected to ICCA, to centralize and display all patient information. Access to the software is permitted by a caregiver specific code to ensure security, confidentiality, and the display of adequate functionalities for each member of the interdisciplinary team. The software lists the patients treated in the PICU and enables access to their medical record. Each medical record is organized with specific tabs for different systems and allows various “interventions” checkboxes to allow caregivers to document their observations and actions. The organized display of information facilitates workflow and reduces the need to look through different tabs to locate or document observations.³ Physiologic monitors connected to the server send data continuously into the EMR for each patient, avoiding manual entry and allowing caregivers to track patients' progress with flowcharts and graphs. Other devices such as syringes and pumps also send data through the server to ICCA. In the prescription tab, physicians can prescribe using drop-down menus. Many preprogrammed orders, weight-dose functionality, and drop-down menus are available to facilitate the process. A treatment plan and reminders are established from the prescriptions and ICCA if and when doses are administered. ICCA offers improvements, as the prescriber's name and the date are written clearly, instead of scrawled in the paper record. Moreover, nurses can prescribe nurse specific techniques and change the times of medication administration, with subsequent reminders that pop up at appropriate time. ICCA is separated into sections for various caregivers in order for document their observations, which can be copied, pasted, and changed partially. There is a laboratory tab where results and text laboratory reports are sent electronically. Consultants have their own section to document their observations and recommendations, which trigger flags in the medical record to allow the treating team to act on the recommendations.

Hardware

The PICU is equipped with two mobile trolleys for physicians, 20 mobile trolleys with removable electronic tablets at patients' bedside for nurses, four tablets for respiratory therapists, and at least one desktop computer in each patient's room for other health care professionals and consultants. All the devices are amenable to disinfection procedures and are waterproof. Tablets use Wi-Fi connection, whereas desktop computers are hardwired to the server.

Implementation Process

The implementation process addressed many aspects, including its organization, its rigors, and its duration of more than 1 year (Fig. 1). Once the decision to collaborate with Philips France was made, Philips Medical provided implementation guidelines that our ICCA's management team adapted to the hospital organization. The ICCA's management team included the head nurse, three other nurses including the nurse director of the unit, two physicians including the medical director of the unit, three engineers from the information technology department, one change manager, and one project manager. Various adaptations and training sessions were undertaken, with the ultimate long-term goal of implementing an EMR in the PICU. An important step in the implementation process was the configuration of new clinical processes for drug prescription and administration. Prior to configuration of the system's interface, ICCA's management team at the CHUSJ revised our clinical standards to incorporate the best actual practice in the EMR and maintain rigorous practice standards. This required literature reviews, discussions, and consensus on the best methods to be used in the PICU and incorporated into the EMR. Therefore, physicians' opinion helped configure admission and progress notes, nurses established how they wanted their interface, and other professionals were solicited to give their opinions and requests. It was necessary to build the best interfaces possible in terms of best practice standards and ease of use. The management team had to decide what checkboxes to display in EMRs, what multiple responses to enter in submenus, and how to divide EMRs in adequate sections, etc., to document the right amount and type of information into the patient's chart. Another example of preparation for implementation was in the rigorous work done to establish preprogrammed prescriptions by combining information found in different resources, such as adequate doses, concentrations, and routes in relation to weight and age.

Fig. 1.

Schematic presentation of the implementation process of an electronic medical record in a pediatric intensive care unit (ICCA, Philips Healthcare, Suresnes, France). This includes two simultaneous phases: information technology (IT) and clinical implementations. ADT, admission-discharge-transfer; GBM, biomedical engineering.

The installation and configuration of the servers began in August 2011 to connect monitors, ADT system, laboratory, mechanical ventilators, pumps, syringes, and the pharmacy interfaces to the ICCA software. There had to be an adaptation of EMRs consistent with the Quebec's health system and to the language/nomenclature caregivers commonly use. Philips visited the PICU several times in 2011 and 2012 to configure the EMR, teach the project management team of CHUSJ how to use the software correctly as a clinical tool, and make improvements in the software.⁴ In August 2012, 6 months prior to the “go live,” superusers participated in a 14-hour theoretical training given by Philips Medical and a 7-hour training given by an internal care consultant. That superuser team consisted of 14 nurses, 1 intensive care fellow, 3 respiratory therapists, and 5 other health care professionals (a physiotherapist, an occupational therapist, a nutritionist, a social worker, and a psychologist) for a total of 23 superusers. The five professionals' superusers were trained by a member of ICCA's management team at the CHUSJ rather than by Philips care consultant. From mid-August, superusers trained end-users for another 6 weeks. Because ICCA EMR has specific clinical documentation for each category of caregivers, the training was caregiver category oriented: nurses had two individual 2 hours sessions and then practiced with scenarios, whereas the respiratory therapists had 4 hours of training in a single session. Intensivists, residents and fellows had three 1-hour sessions, ward clerks had 1.5-hour training, and other professionals had an individual training conducted by superusers of each profession. ICCA's management team held 15 meetings with different specialties to elaborate an electronic self-training document organized as modules accessible via the hospital's intranet from mid-September to the present. It became a tool available for every employee within the CHUSJ and served primarily to train consultants.

On October 10, 2012, the PICU launched EMR's pilot implementation, which served as a test for the software rather than training for users. Two patients were double-charted by nurses, namely using paper and electronically, but were charted exclusively electronically by physicians. The other caregivers continued using paper records for these two patients. One patient was a long-stay patient, whereas several patients who stayed in the PICU subsequently undertook the role of the second patient. The two types of patient allowed assessment of the system performance for long- and short term stay patients. The other patients in the PICU were charted using the traditional paper records. This moment was crucial for fine-tuning and EMR configuration improvements before the “go live.” In December 2012, that is to say 1 month prior to the “go live,” training by specific themes was conducted and revised with users, including prescription entry, ADT, and progress notes. The “go live” or complete implementation in the PICU occurred January 8, 2013, and all patients in the PICU were exclusively in the EMRs. Concretely, this represents 20 beds equipped with EMRs on the 30 physical beds available, which is enough to cover all patients in the unit at most times of year. Since that moment, a continuous monitoring and change management was performed (“living lab” approach) with at least one superuser always present on the floor to help with and record users' difficulties with the software or hardware, regular meetings of the ICCA's management team that analyzes the barriers to implementation reported by the superusers, and implementation of the changes decided by the ICCA's management team, either with modifications of the configuration of ICCA EMR or adaptations of clinical process.

Assessment of Patient Safety

For the safety assessment we compared the number of severe incidents reported between January 13, 2013 and April 6, 2013 after EMR implementation to the same administrative period of the previous year in 2012 that had a similar workload. Severe incidents were defined according to the incident severity scale used in Sainte-Justine Hospital. Severe incident corresponded to an incident that has one of the following four definitions¹: (1) an incident that necessitates additional specialized care or treatments with increase duration of hospitalization,² (2) incident that results in permanent sequelae,³ (3) incident that necessitates lifesaving maneuvers,⁴ and (4) accident leading to death. To further identify the safety of the implementation of the electronic drug prescription, we analyzed in details the incident report pertaining to medications during the same periods of time. There were a total of 1,451 patient-days for the EMR study period and 1,115 patient-days for the control study period. Total number of medical incidents reported in these periods was recorded and medication incidents were classified in five categories of medication errors: drug stopped, drug not administered, inadequate delay/schedule/frequency, wrong dose administered, and wrong drug administered.

Assessment of Staff Comfort and Satisfaction

After implementation of the EMR in the PICU of CHUSJ, a survey was given to all users evaluating comfort in the use of the EMR after an interval of 1 week, 2 months, and 6 months after the “go live,” and was available to complete for 1 week. However, for the 6-month survey, an extra week and additional solicitation in the PICU were necessary to recruit an adequate number of respondents. The survey was also given a month after the “go live,” but it is not included in the study due to the low response rate. The survey that was identical at each time interval consisted of five affirmations: (1) I am comfortable in the general use of the EMR, (2) I am comfortable in the use of notes, (3) I am comfortable in the use of prescriptions, (4) I feel that oral communication is maintained within the team, (5) and I am generally satisfied with the EMR. Multiple choices available were ranked from “not at all” to “completely” on a five-point scale (1: not at all, 2: a little, 3: enough, 4: very much, 5: completely). Respondents had to provide their occupation and could add comments. At the 6-month survey, an open-ended question was added into the survey to collect positive comments: “Although we are still in an adaptation period, what are the advantages that you perceive to date?” The survey was distributed to staff by e-mails, the weekly PICU newsletter, and direct communication. The survey could be completed by paper or through the “Survey Monkey” web platform. Commentaries and responses to the survey were analyzed, synthesized, put into a progress report, and communicated to the PICU's personnel. All responses were collected anonymously.

Statistical Analysis Method

Data with percentages are presented as values and 95% confidence interval (CI). An analysis of variance (ANOVA) was performed for the comparison of the comfort and satisfaction questionnaire results at 1 week, 2 months, and 6 months. Relative risk of the modification of the incidence of medication incidents was calculated between the two periods before and after implementation. Statistical level of significance is p < 0.05.

Results

Safety

As far as the safety assessment, there were no severe incidents reported during the two 3-month interval of time. Between January 1 and March 24, 2012, there were 59 nonsevere medical incidents reported including 28 medication incidents while between January 13 to April 6, 2013, 31 nonsevere medical incidents were reported including 17 medication incidents. There were a total of 19.3 medication incidents per 1,000 patient-days before implementation of EMRs and 15.2 medication incidents per 1,000 patient-days after implementation of EMRs (Table 1). Assessment of incidence densities showed a nonsignificant reduction of observed events, except for incidents due to inadequate delays, schedule, and frequency that were not significantly increase with EMRs.

Table 1. PICU medication incidents before and after implementation of the electronic health record (EMR).

Type of medication incident	Medication incidents (per 1,000 patients-days [n])		Relative risk (95% CI)	p value
	Prior to EMR implementation	After EMR implementation
Medication discontinued	3.4 (5)	0.9 (1)	0.26 (0.03–2.23)	0.23
Medication not administered	5.5 (8)	1.8 (2)	0.33 (0.07–1.53)	0.16
Inadequate delay/schedule/frequency	2.8 (4)	6.3 (7)	2.28 (0.67–7.78)	0.19
Administration of wrong dose	2.8 (4)	1.8 (2)	0.65 (0.12–3.55)	0.62
Administration of wrong medication	4.8 (7)	4.5 (5)	0.93 (0.30–2.93)	0.90
Total	19.3 (28)	15.2 (17)	0.79 (0.43–1.44)	0.44

Abbreviations: CI, confidence interval; PICU, pediatric intensive care unit.

Note: The period before EMR implementation was between January 1, 2012 and March 24, 2012 and the period after EMR implementation was between January 13, 2013 and April 6, 2013.

Comfort

The target population of caregivers that used EMRs and could have answered the surveys is listed in Table 2, with the number of respondents for each period. The 6-month survey occurred during the summer and had a smaller number of respondents due to vacations (Table 2). When considering staff and evaluating the very much/completely responses for each statement between the three survey periods with a simple ANOVA, a significant improvement between week 1 and 6 months in the three statements: comfort with general use of the EMR (p < 0.001), use of progress notes (p = 0.002), and use of prescriptions (p = 0.002). In contrast between week 1 and 6 months there was no significant improvement concerning level of maintained verbal communication between caregivers and general satisfaction. At 6 months, 56.8% of caregivers are at least very comfortable with the EMR, whereas only 33.9% are very much or completely satisfied with the EMR (Table 3).

Table 2. Details of the staff working in the PICU during the survey periods (1 week, 2 months, and 6 months after electronic health record implementation).

Caregivers types		Week 1		2 mo		6 mo
		Total n	Respondents n (%)	Total n	Respondents n (%)	Total n	Respondents n (%)
Full-time PICU caregivers	Nurses	86	22 (26)	86	10 (12)	71	42 (59)
	Respiratory therapists	40	5 (13)	40	0 (0)	40	14 (35)
	Intensive care physicians	15	7 (47)	15	5 (33)	15	14 (93)
	Fellows	5	NA	5	NA	5	NA
	Attending physicians	10	NA	10	NA	10	Na
	Clerks	8	0 (0)	8	1 (13)	5	3 (60)
	Other	NA	–	NA	1 (–)	NA	1 (1)
Non-PICU caregivers	Consultant physicians, residents	NA	4 (–)	–	13 (–)	NA	26 (–)
	Professional (social worker, dietitian, nutritionist, …)	25	5 (20)	25	6 (24)	25	18 (72)
	Other (ex-engineer)	NA	0 (0)	NA	1 (–)	NA	0 (0)

Abbreviations: NA, not available; PICU, pediatric intensive care unit.

Table 3. Reponses of “agree very much” and “agree completely” to the questions of the survey on comfort and satisfaction at 1 week and 6 months after the electronic health record implementation in the PICU.

Questions of the survey	All caregivers working in the PICU (%) (95% CI)			Full-time PICU caregivers (%) (95% CI)		Part time PICU caregivers (%) (95% CI)		p valuea
	1 wk	6 mo	p value	1 wk	6 mo	1 wk	6 mo
I am comfortable in the general use of the EMR	26 (12–40)	57 (48–66)	< 0.001	32 (16–49)	76 (66–86)	0	25 (12–38)	<0.001
I am comfortable in the use of notes	40 (25–56)	67 (58–76)	0.002	50 (32–68)	85 (76–93)	0	38 (23–53)	<0.001
I am comfortable in the use of prescriptions	17 (5–30)	45 (35–55)	0.002	21 (6–35)	62 (50–74)	0	9 (0–19)	<0.001
I feel that oral communication is maintained in the team	20 (7–33)	33 (25–42)	0.11	15 (2–27)	30 (19–40)	43	40 (24–56)	0.27
I am generally satisfied with the EMR	26 (11–40)	34 (25–43)	0.34	26 (11–42)	42 (30–53)	17	20 (7–32)	0.015

Abbreviations: CI, confidence interval; EMR, electronic medical record; PICU, pediatric intensive care unit.

Note: Full-time PICU caregivers correspond to nurses, respiratory therapists, intensivists, clerks, and others, and non-PICU caregivers correspond to consultant physicians, residents, professionals, and others (see Table 1).

^aStatistical difference between full-time PICU and non-PICU caregivers at 6 months.

We considered full-time PICU staff which are nurses, respiratory therapists, intensive care physicians, and ward clerks, compared with external staff, which are consultant physicians, professionals who come to the PICU occasionally, and residents. After 6 months, the rates of internal staff who answered very/completely comfortable or satisfied are significantly higher than those of external staff. PICU caregivers have a high percentage of general comfort (75.7%), with notes (84.5%), with prescriptions (62.0%), and overall satisfaction (41.9%) compared with external staff and all respondents. Of the five statements, only verbal communication is affected similarly for PICU and non-PICU staff, with 40.0% of PICU staff and 29.7% of non-PICU staff assessing verbal communication very or completely maintained (Table 3).

Because nurses are the largest population of users of the EMRs, we underline their particular responses to the survey. Significant improvement in percentages of nurses that feel very or completely comfortable with general use of the EMR, use of progress notes, and use of prescriptions, and that are very or completely satisfied with the EMR is seen between the first week and the sixth month after implementation (Fig. 2). At 6 months more than 90% are at least generally satisfied “enough” and more than 20% feel that oral communication is not maintained in the team. The 198 commentaries provided by the survey are classified in Table 4 according to their main theme.

Fig. 2.

Distribution the responses of the pediatric intensive care unit nurses to the comfort, satisfaction and verbal communication questionnaire 1 week, 2 months, and 6 months after electronic medical record implementation. ICCA, IntelliSpace Critical Care and Anesthesia; NA, not applicable.

Table 4. Specific caregivers' comments to survey questionnaire on comfort and satisfaction at 6 months after EMR implementation in the PICU.

Positive	Number of respondents	Negative	Number of respondents
Readability and fluidity of the chart	55/118	Long process to fill in the record and to open the software	28/118
Time saving and less transcription	25/118	Technical problems (slow network, software bugs)	25/118
Improved access to the record	25/118	Problems with functionalities inherent to EMR	22/118
Centralization of information	12/118	Unfavorable ergonomics related to computer carts and mobile computers	18/118
New tools that reduce the risk of errors	3/118	Impaired verbal communication between stakeholders	13/118
Paper saving	1/118	Limited number of workstation availability	11/118
		Lack of drive, unfriendly software	8/118
		Transfer problems	6/118
		Less time spent with patients	4/118
		Increased printing at patient discharge	3/118
		Impaired confidentiality	1/118

Abbreviations: EMR, electronic medical record; PICU, pediatric intensive care unit.

Discussion

At the 6-month evaluation, the majority of everyday users were very or completely comfortable with the software, with a nonsignificant decrease in medical severe incident in comparison to a similar period of time, the previous year. After 6 months of use, ICCA satisfied 33.9% of all staff and 41.9% of PICU staff. There was a significant increase in percentages of very much or completely at ease with note writing, prescriptions, and general use of ICCA, from 40.5 to 67.3% (p < 0.001), 17.5 to 45.2% (p = 0.002), and 26.2% to 56.8% (p = 0.002), respectively, from the first week to 6 months after ICCA's implementation. At 6 months, 56.8% of users are very or completely comfortable with the general use of ICCA.

Immediately after system implementation, there was a low satisfaction rate that was due to the lack of comfort of users with the system and the integration of a new way of working. At the 6-month interval, users became habituated to work in the new environment and felt more comfortable. In short, satisfaction depended on multiple factors, including level of comfort, and even though the latter had improved significantly at 6 months, satisfaction was still far from our predicted goal of 70%.

There was a large gap between everyday users (internal staff) of the ICCA system as opposed to the occasional user (external staff). Several factors can explain this gap in satisfaction. Occasional users are less exposed to ICCA and operate mostly in other wards where paper charts are the predominant documentation tool; hence they have less practice and they maintained their work habits of paper charts. The 6 months were not spent exclusively working with ICCA as compared with the PICU users.

Studies about EMRs and computerized provider entry (CPOE) in the literature found satisfaction rates around 40 to 60% after 1 year or more experiencing their new system.^{5 6} Our result of 34.8% of satisfaction is reasonable when compared with literature and given the 6 months of ICCA's utilization. We thought satisfaction (happiness to work with the EMR) would rise to 70% after 6 months, but comfort (ease of EMR use) was the variable that improved. The implementation process is responsible for these encouraging results relative to ease with the software, by ensuring a detailed planning, optimal adaptations of the EMR to our care processes, good training and continuous adjustments, all of which contributed to an easier initiation for clinical users. Our implementation process lasted for 20 months to make it optimal, and indeed the literature states that implementation of a typical system in a hospital takes 18 months. Popernack's study had an 8-hour training session 2 months prior to the “go live” of a clinical information system in an ICU,⁶ study by Larrabee et al study also had 8 hours of training on a nursing information system in an acute care hospital,⁷ whereas Han et al asked for a mere 3-hour computer tutorial before implementation of CPOE.⁸ Training was longer in our case because a full transition from paper medical record to EMR including CPOE was planned. This training started 6 months prior to the “go live” with a 4-hour individual training in addition to various scenarios and modules. Also, the training by themes in the last month before implementation was an important factor in the implementation process. Training just before “go live” is beneficial, as it reviews important concepts, offers additional information, and ensures an appropriate use of the system.⁹ Our belief that 6 months only was necessary to achieve comfort in most users was due to our implementation plan that included progressively all PICU staff in the training (Fig. 1). The superusers contributed to an optimal implementation of the EMR by initiating their training first and more extensively, which allow them to be the predominant resource and ensures support for all users.⁶

Satisfaction and comfort are good indicators of user's knowledge about the software and the level of security provided by the system. The nonsignificant difference in satisfaction with time can be a fear that security is not optimized although no increase in medical severe incident was observed. The numerous training sessions provided for users aimed at building their knowledge about EMRs and their fluency to navigate through the software. User's level of comfort (56.5%) that increased significantly represents a comfort level in the use of the software and an adequate sense of security.

Comfort with notes, prescriptions, and in general evolved significantly over time. This reflected that users were getting used to the new system, and that continual adaptation to the users' needs engendered successful improvement in ICCA. Continual improvement could be achieved with the help of survey responses analyzed to highlight recurring comments and by taking necessary measures to address different problems. Communication of the results to users is important to highlight that their opinions are crucial, to involve them actively in the project and ultimately to have a shared vision and goal. The active participation of users through interaction with superusers certainly played a role in the successful implementation. “If an individual believes that the system is personally relevant, he will be more likely to form a positive attitude toward the system since attitudes are generally formed on the basis of beliefs.”¹⁰ “They need to see the value in it for them, and they need to participate actively in the organizational change.”¹¹ However, the system was still new in the PICU, and many adaptations had not yet been performed to correct identified problems, as configuration of the software can be quite long and arduous.

Nurses are of particular importance in analysis of the surveys. They use ICCA as a tool for their everyday professional life, and it is therefore essential that they feel at ease with it, first to optimize their workflow and to ensure maximal security for the patients. They are essential to the general organization of the PICU and are generally part of the early adopters due to time invested with the software.³ PICU nurses were very comfortable with the software after 6 months of use, with 81% of them being very or completely comfortable with the system, 90.5% with electronic notes, and 71.4% with the prescriptions. There is a significant difference between those results at 6 months compared with the lower percentages at 1 week. That is very encouraging and was above our expectations because of the short delay between implementation of the EMR. A substantial proportion of nurses felt that verbal communications were hampered, as only 13.6% at 1 week and 16.7% at 6 months felt that this aspect had been affected with ICCA. There is no significant improvement in verbal communication with time. These results correspond to the comments provided by different users throughout the different surveys. Satisfaction did not significantly increase either, and 50% of nurses were very or completely satisfied at 6 months postimplementation.

Residents are another group that have interesting results: 28% are comfortable with ICCA, 71% with writing notes, and 14% with the prescriptions. Moreover, 43% believe that verbal communication is maintained and 14% are very or completely satisfied with ICCA. Because they spend only a month or two in the PICU during their rotations, they lack the time to learn to use ICCA efficiently. Their workload was increased, because writing orders and daily notes in the EMRs is much more time consuming than in the paper records. EMR ensures legibility and clarity, which reduces time required to decode prescription, but increases time to write prescriptions due to the formatting norms when writing orders, which explains the small percentage of residents at ease with the use of the prescriptions.

Adoption and diffusion of a new technology follows an S curve, which can be used to interpret the adoption curve of an EMR in a hospital. Successively, early adopters, early majority, late majority, and finally laggards adopt the new procedure, which creates an S curve of adoption with the tipping point positioned at 20% of adoption, representing the peak acceleration in the adoption rate. When 70% of satisfaction is acquired, we are technically in the phase where the late majority is progressively accepting EMR, but our results of 34.8% of all users and 50.0% of nurses satisfied imply that we are still recruiting the early majority.^{12 13} It is normal that ICCA is maintained at this level after 6 months postimplementation. The learning curve corresponds to the first 6 months after implementation, where difficulties encountered are numerous and generate decreased satisfaction.⁹ During this lapse of time, measures concerning efficacy would not reflect the real power of the software, but rather the adaptation period of users. “With any new technology implementation there is a learning curve that must be conquered before users are operating at full speed.”¹¹

The effect of ICCA on error reporting was assessed by incident reports, which allow direct observation of the impact that adaptation phase to EMRs generates toward patients' security. By examining medication incident reports before and after ICCA's implementation, we wanted to provide a measure that would indicate if there was increased danger for a patient hospitalized in a newly equipped unit. ADE prevention study group established the rate of adverse drug events in ICUs at 19.4 per 1,000 patient-days.^{14 15 16 17} This number is similar to the 19.3 medication incidents per 1,000 patient-days before implementation of ICCA in our study. After implementation, there is a nonsignificant reduction in medication incidents to a rate of 15.2 medication incidents per 1,000 patient-days. Inadequate delays, schedules, and frequency were not significantly increased, however, probably because of the presence of two medication administration documents due to implementation of EMR with its own electronic document in addition to the document generated by the pharmacy. Caregivers need to ensure consistency between the two separate documents. There are also printing difficulties when transferring patients from the PICU to other wards. Orders written in ICCA are difficult to read and also present include recently stopped prescriptions as inactive which pollutes the prescriptions.

User comments highlighted in Table 4 give the main reasons why users are not satisfied or comfortable with ICCA. Issues can be classified as easy or difficult to address. Technical and physical problems are the easiest to correct. The limited number of workstations can be corrected by purchasing more carts; dealing with ergonomics concerns; loss of WIFI connectivity, sensitivity, and weight; and cart's monopolization of space can be improved. Specific problems to the software, such as functionalities and log-in, are more complicated to alter, taking into account that the software is limited in its configuration and that the process can be arduous and time consuming. The most difficult problems are those that cannot be solved by technical improvement, but by users' involvement. Verbal communication between stakeholders is a subject that is invariably affected by EMR, because information is accessible via computers and tablets that motivate stakeholders to send their recommendations and observations electronically from satellite sites, decreasing encounters between caregivers. The preservation of verbal communication between caregivers depends on users' efforts to physically visit the patient's bedside or call whenever an important change is made. There are also many complaints about reduced time spent with patients and increased time spent on computers, but most caregivers are satisfied by the opportunity of documenting notes and writing prescriptions from any location and not necessarily by the bedside. This tension can be corrected by finding a balance between both communicating in person and the freedom of documentation from satellite locations. The printing of the documents is cumbersome, but it is the only way at this moment to ensure appropriate transfer of patients from PICU to the other units who have not moved to an EMR. The main problems identified by users were the ones that were easily addressable by technical correction.

Readability and fluidity are the most positive aspects of ICCA reported by stakeholders. Effort spent on decrypting caregivers' handwriting can be oriented to other tasks. Information is categorized and placed automatically in specific sections, which provides an efficient navigation through data and easier research of information. Manual transcription is diminished with EMRs, saving precious time for caregivers. Vital signs, fluid balance, and syringe dosing are automatically entered in ICCA, and are easily found in emergency situations. Although availability of workstation is an issue, access to the record is enhanced by the fact that any computer with ICCA can access any patients' files. Many caregivers can be on a single chart simultaneously, eliminating time spent looking for the chart that was previously transported throughout the PICU. Stakeholders can also enter data and orders from a decentralized location. This is useful for consultants, professionals, and in contexts of infection but has a direct repercussion on time spent with the patient. Another advantage of ICCA identified by users is the centralization of information in a unique virtual place. Laboratories, actual medication, consultant notes, treatment plan, tendencies over several days, prescription guidelines such as weight-dose functionality, and most common doses prescribed are incorporated in the EMR.

A limitation to our study is that the few respondents who filled out the survey in the first week and the second month represent less than 50% of users. Survey fatigue is inevitable when a survey is given several times during a year. While the median response rate for an annual survey is 25%, it drops to 15% when it is sent repeatedly.¹⁸ Respondents were self-selected, and this bias attracts people who have strong positive or negative feelings relative to the survey's subject, in this case the EMR.⁵ People also tend to criticize and to give more negative than positive comments. Although it is an excellent method to trigger improvement, it gives the impression that people are dissatisfied with the system. Moreover, people who are satisfied may not take the time to answer surveys, and by their nonparticipation they contribute to nonrepresentative samples. In addition to small samples, the fact that the study was performed in a single PICU of one hospital diminishes the ability to generalize these results. This project is dynamic and spread over a long period, with unexpected events and continual improvement; hence multiple modifications occurred between surveys and affected the results.

Assessment of medication incidents is somewhat limited, as it is based on voluntary report rather than direct observations, a more accurate and objective method. A study of voluntary reports does not eliminate the bias of errors knowingly not declared and of caregivers unaware of errors they have done.¹⁹ Moreover, in January 2013, an administrative decision reduced the number of beds by 30% in the ICU for budgetary reasons, going from 20 to 14 patients. The lower number of patients could be responsible for part of the downward changes in incident report rates.

The methodology to distribute the same questionnaire at different moments after implementation of an EMR allows direct comparison, assessment of short- and long-term evolution, and gives strength to the study. The high response rate for the 6-month survey makes the sample representative and compensate for the smaller samples obtained at 1 week and 2 months. Surveys are very powerful tools to identify progress, potential areas of improvement, and establish related action plans. Our survey consisted of five-graded questions, one additional open question for the 6-month survey, and a section for comments.

In conclusion, the implementation of an EMR in PICU is a long process that started almost 2 years before the “go live,” was still ongoing at 6 months, and that is finished at 2 years. After the “go live,” a closed monitoring of the barriers and difficulties with rapid adaptation (“living lab” approach) helped in the success of the implementation but did not improve significantly caregivers' satisfaction, the first 6 months of this dramatic change in practice.

“Organizations that simply automate their existing inefficient paper processes will realize far fewer gains than those that make the commitment to reengineer.”¹¹