Attitudes and Perceptions of Pediatric Residents on Transitioning to CPOE

Summary

Background

Many resident physicians have experienced transitioning from traditional paper documentation and ordering to an electronic process during their training.

Objective

We sought to investigate the attitudes and perceptions of residents related to implementation of computer provider order entry (CPOE) and clinical decision support (CDS).

Methods

Pediatric residents completed web-based surveys prior to CPOE implementation and at 6 months and 12 months after implementation. The survey assessed resident attitudes and perceptions related to CPOE and the use of CDS tools. Additionally, at 6 and 12 months, residents were asked how electronic medical record (EMR) resources might impact future career decisions.

Results

Prior to CPOE implementation, 70% of residents were looking forward to CPOE, but 28% did not want to transition from paper ordering. At 12 months post-implementation, 80% of residents favored CPOE over paper ordering and only 3.33% wished to revert to paper ordering. Residents reported an increase in time needed to enter admission orders 6-months after CPOE implementation. By 12 months post-implementation, there was no significant difference in perceived time to complete admission orders when compared to pre-CPOE responses. Most residents (91.67%) identified that overall EMR resources were an important factor when considering future employment opportunities. The most important factors included the degree of EMR implementation, technology resources and the amount of support staff. The least important factors included patient portal access and which EMR product is used.

Conclusions

Overall, residents demonstrated a preference for CPOE compared to traditional paper order entry. Many residents remained unaware of CDS tools embedded within CPOE at the 12 month follow-up, but a majority of residents did find them helpful and felt more knowledgeable about current guidelines. EMR resources, including degree of EMR implementation, technology resources and support staff are likely to be important factors as residents take future employment opportunities into consideration.

1. Background

Use of electronic medical records (EMRs) has accelerated over the past decade [1], due in part to recommendations by organizations such as the Institute of Medicine [2] and the Joint Commission [3]. EMRs have also been shown to offer improved accessibility, legibility and flexibility as well as a potential for reduction in medication errors [4]. Additionally, clinical decision support (CDS) within EMRs can support adoption of clinical practice guidelines [5], particularly when guidance is customized to both the provider and the patient [6].

Understanding how resident physicians differ from experienced clinicians can aid in CDS design and new implementations. Most published research about implementations is focused on computer provider order entry (CPOE). Early implementations at academic centers were notoriously difficult, and some experiences associated with poor clinical outcomes [7]. Medical students and residents have less clinical experience which is associated with higher rates of errors [8], and provides a different challenge to CDS design. Additionally, some early studies raised concerns about negative impacts on medical student education [9]; however, there is still a lack of strong data to support the use of EMRs as educational tools [10]. Although academic centers have a higher rate of EMR use, overall pediatricians have a lower adoption rate [11].

At academic centers, resident physicians usually enter the bulk of physician orders, and therefore are well positioned to benefit from and advise upon CDS. Some literature suggests that medical students are not well prepared to provide information for such EHR regulations as meaningful use [12]; however, many residents in graduate medical education may train in a variety of health systems frequently, and may be more amenable to change. Additionally, adoption among ambulatory practices show that EMR use is inversely associated with physician age [13], suggesting that current trainees are more likely to consider EMR use part of their practice when they graduate.

As our institution transitioned from traditional paper orders to computer physician order entry (CPOE), we had a unique opportunity to measure the impact of this change on residents. We chose to focus on resident physicians since they are the group writing most of the orders in clinical patient care at our facility.

During design of pediatric CPOE, faculty and residents at our pediatric hospital helped design approximately 365 electronic order sets (Powerplans™) addressing admission orders for simple and complex illness, medical and surgical problems, and other groups of orders. CPOE content was also designed to support experienced clinicians as well as resident physicians. A wide range of CDS techniques were used to include the suggestions from general pediatric evidence-based guidelines, including order omission, orders embedded or pre-selected, linked reference text, and notes to the physician. The CDS techniques were used in admission order sets in an effort to encourage best practice as well as improve consistency in ordering practices. An example of the bronchiolitis admission order set is seen in ▶ Figure 1.

Fig. 1.

Example of the bronchiolitis admission order set generated for CPOE

Preparing residents for CPOE included many methods. All pediatric residents completed web-based training as well as classroom education. Some pediatric residents were also trained as “superusers” prior to implementation and strategically placed to staff all impacted areas. CPOE was implemented across all medical units simultaneously (including pediatric intensive care units, neonatal intensive care units, emergency department and medical wards) in August 2012. The implementation included on-site support by IS staff for 2 weeks and a central command center. Support continued with a more limited onsite team and phone support after the first 2 weeks.

2. Objectives

We sought to investigate the attitudes and perceptions of residents related to the implementation of CPOE and CDS before implementation, after 6 month of use, and after 12 months of use.

3. Methods

This study was completed using a repeat cross-sectional design survey of pediatric residents at a 276-bed tertiary care pediatric hospital in the Midwest US. In August 2012, two weeks prior to implementation of CPOE, a web-based, IRB-approved, survey was distributed to all 146 pediatric residents in the academic center. Residents surveyed included categorical pediatrics, internal medicine and pediatrics, emergency medicine and pediatrics, and pediatrics and psychiatry. Residents complete >90% of their inpatient pediatric rotations at this site.

For all surveys, residents were invited to participate via an email survey link. Recruitment included 2 reminder emails for each survey and responses were anonymously collected. There were no incentives used to encourage participation.

The surveys were created by the authors for this implementation project and included a variety of multiple choice questions, areas for free response and Likert scale queries (see supplementary online Appendix 1 and 2). Survey questions measured residents’ attitudes and perceptions about transitioning to CPOE and ordering practices, including the time residents perceive is needed to place admission orders, awareness of clinical decision support and its perceived utility. Residents were also encouraged to leave free text comments regarding their general attitudes about CPOE and CDS.

Follow-up surveys were distributed 6 months and 12 months after CPOE implementation and included additional questions about resident awareness and perceptions of CDS as well as how EMR resources might impact future career decisions. Comparisons were made between respondents at each point in time using descriptive statistics. Responses on perceived order entry time and frequency of being paged after entering orders were also analyzed using a series of two-proportion z-tests.

4. Results

Of the 146 residents surveyed, 43.8% (n=64) of residents responded to the pre-CPOE survey, 37.6% (n=55) responded to the 6-month post-CPOE survey, and 43.2% (n=63) responded to the 12 month survey. The group demographics are shown in ▶ Table 1.

Table 1.

Demographics of respondents for all surveys

	Pre CPOE	6 month after CPOE	12 month after CPOE
PGY-1s	18 (28.6% of pre-CPOE cohort)	12 (21.8% of 6-month cohort)	26 (41.3% of 12-month cohort)
PGY-2s	13 (20.6% of pre-CPOE cohort)	19 (34.6% of 6-month cohort)	14 (22.2% of 12-month cohort)
PGY-3s	17 (27% of pre-CPOE cohort)	17 (30.9% of 6-month cohort)	18 (28.6% of 12-month cohort)
>PGY-3	15 (23.8% of pre-CPOE cohort)	7 (12.7% of 6-month cohort)	5 (7.8% of 12-month cohort)
Total responses	64 (43.8% of invited)	55 (37.6% of invited)	63 (43.2% of invited)

4.1 Resident attitudes regarding CPOE transition

Prior to CPOE implementation, 70.3% of residents reported looking forward to the transition. In the survey 6 months after implementation, 48% of residents reported that, overall, they liked CPOE while 42% disliked it. A minority (16%) of respondents reported that that they would return to paper-based orders.

At 12 months after implementation, 80% of residents favored CPOE over paper ordering and only 3.33% wished to revert to paper ordering. Some (28.33%) respondents felt CPOE had improved patient care overall, and over half of the residents (55%) responded that CPOE had improved some aspects of patient care. 8.33% felt patient care was unaffected by CPOE, and no respondents felt it had worsened care.

4.2 Perceived changes in workflow

On the pre-CPOE survey, residents were asked to estimate the time needed to complete admission orders for most patients. The majority of residents reported perceived time needed to enter admission orders as 5–10 minutes (▶ Table 2). At 6 months, there was a significant drop in the 5–10 minute estimate (p=0.03), and more residents reported that admission orders took more than 15 minutes to complete (p<0.01). At 12 months, there was no significant difference between the perceived time needed to complete admission orders when compared to the pre-CPOE survey responses (p>0.2).

Table 2.

Perceived time needed to enter admission orders.

Time to enter admission orders	Pre-CPOE	6 months Post-CPOE		12 months Post-CPOE
	% responses (n)	% responses (n)	P value*	% responses (n)	P value*
<5 minutes	14.06 % (9)	8.16% (4)	0.33	17.24% (10)	0.63
5–10 minutes	53.13% (34)	32.65% (16)	0.03	41.38% (24)	0.2
10–15 minutes	26.56% (17)	32.65% (16)	0.48	29.31% (17)	0.74
>15 minutes	6.25% (4)	26.53% (13)	<0.01	12.07% (7)	0.27

*P values reflect comparison of 6 month data to Pre-CPOE and 12 month data to Pre-CPOE.

Residents were also asked to estimate how frequently they received pages from either pharmacy or nursing after entering orders both before and after CPOE. There was an initial increase in reported frequency of pages 6 months after CPOE implementation with 44% of residents reporting that they “Often” received a page after entering orders, compared to only 15.6% reporting this prior to CPOE (p<0.01) (▶ Table 3). By 12 months, the perceived frequency of paging had returned to levels not significantly different than what was described prior to CPOE implementation.

Table 3.

Frequency of pages perceived by residents before and after implementation of CPOE

Paged by Pharmacy or Nursing after entering orders	Pre-CPOE	6 months Post-CPOE		12 months Post-CPOE
	% responses (n)	% responses (n)	P value*	% responses (n)	P value*
Never	1.6% (1)	0% (0)	0.37	0% (0)	0.33
Sometimes	82.8% (53)	56% (28)	<0.01	72.4% (42)	0.17
Often	15.6% (10)	44% (22)	<0.01	27.6% (16)	0.11

*P values reflect comparison of 6 month data to Pre-CPOE and 12 month data to Pre-CPOE.

4.3 Clinical decision support awareness and utility

In the 6 month survey, 48% of residents reported finding CDS tools helpful and 32% felt that CDS tools improved their knowledge about certain guidelines (▶ Table 4). Forty percent of residents responded that they had not noticed the CDS tools. At 12 months, CDS tools were reported as useful by 55.17% of residents and 50.88% felt more knowledgeable because of them. In the assessment after 1 year, 36.21% of residents were not sure what CDS tools were and had not noticed them.

Table 4.

Resident perception of clinical decision support tools (CDS).

	% Responses at 6 months Post-CPOE (n)	% Responses at 12 months Post-CPOE (n)
I find the clinical decision support tools useful	48% (24)	55.2% (32)
I do not find the clinical decision support tools useful	14% (7)	8.6% (5)
What are clinical decision support tools? I have not noticed them!	38% (19)	36.2% (21)
I feel more knowledgeable about current guidelines for treatment of certain patients	32% (16)	50.9% (29)
I do not feel that the clinical decision support tools have influenced the way I care for my patients	26% (13)	7% (4)

4.4 Impact of EMR on career planning

In the 12 month follow-up survey, most of the residents (91.67%) identified that overall EMR resources were an important factor when considering future employment opportunities (▶ Table 5). The majority of residents responded that the degree of EMR implementation (88.33%), support staff availability (88.33%) and hardware availability (78.33%) were important factors. A smaller majority (54.24%) of residents responding reported that the specific EMR product would influence their choice, 45.76% responded that the specific EMR product in place was not an important factor. Additionally, 55% of residents reported that patient portal access was not an important factor.

Table 5.

Impact of EMR resources on resident’s future career planning

	Important	Unimportant
Overall EMR Resources	91.67%	8.33%
Fully implemented EHR	88.33%	11.67%
Support staff	88.33%	11.67%
Technology resources	78.33%	21.67%
Customization to preferences	66.67%	33.33%
EMR product available	54.24%	45.76%
Patient portal available	45%	55%

5. Discussion

During the widespread EMR implementations of the last decade, the elements of a successful CPOE implementation have been well described [14]. Physician satisfaction and comfort with EMR and CPOE specifically has been well described, with the majority of studies identifying both EMR use and satisfaction as inversely related to physician age [15]. Our surveys of pediatric residents throughout the first year of implementation indentified changes in attitudes related to CPOE and EMRs during this time of rapid change, as well as their perceptions as they moved through their training.

CPOE has been shown to reduce the amount of time (“turn around time”) to deliver medications, and improve pharmacy efficiency [16, 17]; however, physician efficiency has presented more challenges. In studies prior to widespread CPOE adoption, Bates et al suggested that physician time spent writing orders was increased when moving from paper to CPOE [18]. Since then, several studies have shown that CPOE and paper orders may be time neutral [19, 20]. In our study we did find that residents needed more time to enter orders 6 months after CPOE implementation. By 12 months after implementation, however, the responses were not significantly different from those prior to CPOE implementation, indicating that the residents had adjusted to the change in ordering practices and improved efficiency back to their initial baseline when using paper orders.

Predesigned order sets and clinical decision support have been identified as part of successful CPOE implementation [21], in part because of the impact on physician efficiency. However even order sets for evidence based management of common diagnoses use can be vary by diagnosis and venue (emergency department, medical ward and intensive care) [22].

Six months after CPOE implementation, only 48% of residents responded that they liked using CPOE. However, despite their initial concerns, by 12 months after CPOE implementation residents were more positive in their attitudes about CPOE with 80% responding that they would not choose to utilize paper orders if given the option. Similar trends in user satisfaction have been noted previously [23]. There are likely to be many reasons for this change, but it most likely represents an enhanced familiarity and comfort with the new ordering practices. Many residents found the CDS embedded within CPOE to be helpful and felt more knowledgeable about certain clinical practice guidelines because of them. Additionally, nearly all of the residents indicated that EMR resources and degree of EMR implementation would be important factors when considering future employment opportunities. These resident attitudes about CPOE and EMRs may reflect a new element of institutional resources as physicians consider health IT and EMR resources that are available at potential employers.

With recent surveys of established physicians indicating a high level of frustration and unhappiness with EMR use in regards to overall job satisfaction [24], the pressure is high for EMR vendors and onsite IT teams to meet clinical needs. While a majority of resident respondents stated that technology resources and EMR availability will impact their career planning; just over half reported that the specific EMR product would be an influence on their choice. These results could suggest that the next generation of physicians will have high expectations for EMRs that support clinical practice. Having trained during a period of implementation, they will be more aware of the impact of EMRs than previous generations, and therefore more likely to factor EMR implementation and readiness into their career decisions. While the specific EMR platform seems to be less important, it appears that, the commitment of private practices, health systems, and hospitals to health IT and EMRs may impact their ability to recruit physicians in the future.

The study does have limitations. As this was a voluntary survey conducted with our residents who experienced the CPOE implementation, the collected sample size (37–43%) may not have a full representation of the eligible population. As with any assessment conducted before and after major changes, perceptions from inexperienced users can be difficult to compare to that provided by more knowledgeable, informed users.

Our surveys were customized to our institutional needs and were not independently validated. We also did not normalize results for individual medical units or patient complexity, so perceived time to enter orders could have varied depending on what clinical setting a resident had worked in prior to the survey. Survey time spanned two different academic years and did not follow individuals. Thus, the group of residents surveyed in the 12 month survey differed (i.e. PGY-3 and above in the pre-CPOE cohort were absent in subsequent surveys), making it difficult to make direct comparisons of the responses between these surveys. Finally, given the study design, resident attitudes and perceptions were not correlated with direct observation and are thus subject to recall bias.

Despite these limitations, we feel that these results illustrate an emerging area of interest for residents in training with increasing awareness of how the EMR can be used as a tool to improve patient care. Experienced physicians are learning at the same time as their students, and instruction on the best design for EMR as well as the ease of use continue to evolve. Future challenges will be to prioritize the options available in CDS and incorporate EMR use into medical education curriculums. Given the wide array of screening, safety and best practice guidelines, clinical informaticians and medical educators must work together to provide EMR support for the most critical elements. Increasing understanding of EMR design and use early in medical education will be critical to improving efficiency in the physician workflow.

6. Conclusions

This study demonstrated that although pediatric residents perceived inefficiency in workflow in the first 6 months of CPOE; twelve months after implementation, residents came to prefer CPOE to traditional paper ordering practices and perceived efficiency improved. Residents also identified EMR resources and degree of implementation as important factors in selecting future employers.

As electronic medical records have become part of practicing medicine, medical education will need to adapt curricular changes to support the new skills necessary to function effectively with technology. As implementations of EMRs continue to be completed, physicians graduating and moving into the workforce will have expectations of functionality as well as support resources to use these tools effectively.