Skip to main content
Applied Clinical Informatics logoLink to Applied Clinical Informatics
. 2015 Dec 2;6(4):698–715. doi: 10.4338/ACI-2015-06-SOA-0067

Successful Implementation of Clinical Information Technology

Seven Key Lessons from CPOE

GA Gellert 1,, V Hill 2, K Bruner 1, G Maciaz 1, L Saucedo 1, L Catzoela 1, R Ramirez 1, WJ Jacobs 1, P Nguyen 3, L Patel 4, SL Webster 1
PMCID: PMC4704039  PMID: 26767065

Summary

Objectives

To identify and describe the most critical strategic and operational contributors to the successful implementation of clinical information technologies, as deployed within a moderate sized system of U.S. community hospitals.

Background and Setting

CHRISTUS Health is a multi-state system comprised of more than 350 services and 60 hospitals with over 9 000 physicians. The Santa Rosa region of CHRISTUS Health, located in greater San Antonio, Texas is comprised of three adult community hospital facilities and one Children’s hospital each with bed capacities of 142–180. Computerized Patient Order Entry (CPOE) was first implemented in 2012 within a complex market environment. The Santa Rosa region has 2 417 credentialed physicians and 263 mid-level allied health professionals.

Methods

This report focuses on the seven most valuable strategies deployed by the Health Informatics team in a large four hospital CHRISTUS region to achieve strong CPOE adoption and critical success lessons learned. The findings are placed within the context of the literature describing best practices in health information technology implementation.

Results

While the elements described involved discrete de novo process generation to support implementation and operations, collectively they represent the creation of a new customer-centric service culture in our Health Informatics team, which has served as a foundation for ensuring strong clinical information technology adoption beyond CPOE.

Conclusion

The seven success factors described are not limited in their value to and impact on CPOE adoption, but generalize to – and can advance success in – varied other clinical information technology implementations across diverse hospitals. A number of these factors are supported by reports in the literature of other institutions’ successful implementations of CPOE and other clinical information technologies, and while not prescriptive to other settings, may be adapted to yield value elsewhere.

Key words: CPOE/health informatics implementation, CPOE adoption, clinical information technology implementation, health information technology implementation

1. Objective

We seek to describe the most valuable strategies that the Health Informatics (HI) team in one large CHRISTUS region, Santa Rosa, deployed to achieve strong CPOE adoption, along with critical lessons learned. The seven success factors described are not limited in their value to and impact on CPOE adoption, but generalize to – and can advance success in – other clinical information technology implementations.

2. Background and Setting

Computerized Patient (or Provider/Physician) Order Entry (CPOE) is a technology used by clinicians to directly and digitally enter pharmacy, laboratory, radiology and other orders into a computer system or mobile device, which are then transmitted electronically to the respective department or service for execution. The order is documented in a digital, structured and computable format for multiple safety and other uses [1]. CPOE technology supports standardized, evidence-based and legible orders, and through Clinical Decision Support (CDS), can improve quality and safety by reducing medication and other errors at various stages of the order management process by promoting the use of evidence-based treatments and avoiding redundant testing [2–9]. CPOE can also accelerate the ordering process and delivery of care, improve efficiency, and reduce the number of individuals required to participate in the clinical workflow, thereby reducing care delays, adverse events, and errors due to miscommunication and illegibility [10–16].

However, it is clear that for many physicians, CPOE is one of the largest and potentially most dislocating changes in clinical practice and workflow in a generation. Reports abound of failed or troubled attempts to implement CPOE technology, producing high clinical end user resistance and dissatisfaction [17–29]. Furthermore, many of the early adopters of CPOE were academic medical centers, and/or hospitals that had engineered self-built CPOE systems [30]. Many community hospitals, which have adopted CPOE with the support of $19.2 billion in federal financial incentives created by the Health Information Technology for Economic and Clinical Health (HITECH) Act in 2009 [31], have little accumulated experience to guide their approach, which may differ substantially from that of academic hospitals [30].

CHRISTUS Health is a non-profit multi-state system of more than 350 services and 60 hospitals with over 9 000 physicians. CPOE was first implemented in 2012 within a complex market environment where competitor hospitals had not yet implemented, creating a safe harbor for physicians seeking to avoid CPOE adoption. The Santa Rosa region of CHRISTUS Health, located in greater San Antonio, Texas is comprised of three adult general hospital facilities and one Children’s hospital. CHRISTUS Santa Rosa-New Braunfels is licensed for 142 beds and usually operates with 100 occupied beds. CHRISTUS Santa Rosa-Westover Hills has a bed capacity of 150 beds which is typically exceeded. CHRISTUS Santa Rosa-Medical Center is licensed for 178 beds and has an average daily census of 74 (increasing to 110 during winter months). CHRISTUS Santa Rosa-Children’s Hospital of San Antonio has 180 beds with an average daily census of 100 occupied beds.

The Santa Rosa region has 2 417 credentialed physicians and 263 mid-level allied health professionals. The three adult facilities are community hospitals, and while not academic medical centers, have residency affiliations and 260 residents who train and rotate through the facilities annually across a number of specialties (about 50 residents are present in the region at any one time). Children’s Hospital of San Antonio is an academic medical center affiliated with Baylor College of Medicine. While resident physicians are younger and often more “tech savvy” than their older colleagues, the demography of the physician community in each facility is similar to U.S. physicians as a whole. Nonetheless, our younger physicians generally adopted CPOE more rapidly and easily, and tended to complain about CPOE and resist adoption less than physicians 50 years old or greater. CHRISTUS Health Santa Rosa contracts with several physician specialty groups for services and few physicians are employed directly by the region. Contracts exist with hospitalists and ED physicians. Other specialties during the CPOE go-live were participating partners in a number of risk/responsibility sharing payment incentive arrangements in which CPOE was one of a number of performance metrics to be achieved (among other quality, efficiency and patient satisfaction objectives).

The Electronic Health Record (EHR) deployed by CHRISTUS Health is MEDITECH Client Server Version 5.66. Facilities in the region did not implement CPOE simultaneously, but in a phased and overlapping manner over the first eight months of the first implementation year (2012). CPOE was launched in a serial manner, one regional facility at a time. In addition, each hospital did not launch across the entire facility at once, but was phased in service line by service line, with a specific plan for each determined well in advance of the go-live and in close collaboration with facility clinical and administrative leadership, who identified the most logical and least disruptive go-live process in terms of patient flow in the facility and overall multidisciplinary clinical workflow.

Prior to CPOE implementation (and ongoing), we trained super users to provide at the elbow, in person support of CPOE navigation to clinical end users on most service lines and units. These individuals were mostly nurses and unit clerks. Super users were recruited and trained on most service lines, and lent support to physicians on their respective service lines during the launch period and ongoing for maintenance support. We employed a standard process for intensive support of clinical end users during CPOE launch, and then engaged a subsequent transition to maintenance support usually 4–6 weeks after the initial go-live.

During the first four weeks of each facility go-live, Health Informatics deployed intensive team resources drawn from across the entire region and all facilities to support the acute launch period of newly adopting service lines. This involved most of the Health Informatics team, including leadership and clinical informaticists dedicated to other facilities or activities, and the creation of a schedule of overlapping shifts across day, evening and night shifts. During the first go-live weeks we hired, as needed, additional outside resources (mostly college students) and trained them to act as super users and provide CPOE support.

After 4–6 weeks, this level of resource intensity was gradually reduced so that each facility moved into a routine or maintenance CPOE support process, with one dedicated full time facility Clinical Informaticist (CI) physically on site to support physicians at the elbow, and to identify and overcome specific problems as needed during day shift. In addition 1–2 full time associates from the Information Management department were/are dedicated to each facility to respond to access, log in, computer workstation, mobile device, printer, and related technology infrastructure, maintenance or equipment issues. The Clinical Informaticists and Information Management team members round in their facility individually multiple times daily, and when possible together, and can be contacted by any end user or super user in the hospital through a local facility wireless communication device. A Help Desk is available telephonically as well 24/7 and is often used during the evening and night shifts to manage end user issues and lend continuing CPOE support around the clock. Following the go-live period, nursing supervisors and other facility leaders collected paper orders and placed them away under lock and key for use only during planned or unplanned downtimes where the impact involves our downtime solution also being inaccessible. However, some providers continue to bring their own paper orders or sets into the facility, so while removing paper orders was helpful it did not enable the complete elimination of paper.

Each Clinical Informaticist was integrated into their facility team at all levels and across all departments. For example, CIs were/are free to attend and present when clinical information technology updates or important messaging are appropriate at Medical Executive Committee Meetings, Active Medical Staff Meetings, departmental meetings, bed board, and physician partnership meetings. Regular dedicated Health Informatics meetings with multidisciplinary physicians were established during each facility’s go-live to facilitate, improve and remove barriers to CPOE adoption. In addition, facility Clinical Informaticists could be invited to participate in regularly scheduled or issue meetings between regional Health Informatics leaders and c-suite executive facility leaders (CEO, CMO and CNO) as determined by specific agenda items. These meetings are discussed in depth shortly.

We utilize in our adoption process (and here report) CPOE use rate percentages as extracted from Horizon Business Insight, McKesson Performance Analytics. ►Table 1 details the serial role out of CPOE in the Santa Rosa region by facility and month/year, indicates when each facility first achieved or exceeded a total facility CPOE use rate of 80%, and how many months were required to attain that threshold. CDS was adopted in the form of order sets within CPOE and the usual alerts for drug duplication, allergy, contraindications, etc.

Table 1.

Month/Year of CPOE launch and achievement of facility use rate of 80%

Facility Month/Year CPOE Launch Month/Year 80% CPOE Use Rate Months Required to Achieve 80%+ CPOE Use Level
CSR-Westover Hills March 2012 February 2013 11
CSR-New Braunfels May 2012 May 2014 24
CSR-Medical Center August 2012 February 2014 18
CSR-Children’s Hospital of San Antonio November 2014 January 2015 2

As of this August 2015 writing, all four facilities in the region have achieved sustained month over month facility CPOE use rates above 80%. Three facilities are performing at the 87–88% level month over month, and one at 82–84%. For calendar 2015, we have identified a sustained CPOE performance goal of 90% for three facilities and 85% for the fourth. One facility has already reached the 90% overall use rate threshold (CSR-Medical Center).

Figure 1 illustrates the adoption curve for each facility by year and quarter. In addition, because each facility launched at different times over several years, each curve denotes what the overall facility CPOE use rate was at 6, 12, 18, 24 and 30 months from first implementation in that facility.

Fig. 1.

Fig. 1

CHRISTUS Santa Rosa CPOE adoption trend by calendar quarter

We review the seven most important elements or factors contributing to the successful adoption of CPOE in these three adult facilities and Children’s Hospital from project launch in early 2012 to present (►Table 2). While these elements involved specific new process generation, collectively they created a new customer-centric service culture in Health Informatics as a foundation for ensuring strong CPOE adoption. Our success in using these principles is demonstrated by these facilities’ high and sustained overall CPOE use rates, and their positioning to achieve 90%+ use rates in 2015.

Table 2.

Seven strategic and operational contributors to successful implementation of clinical information technology

Strategic/Operational Success Factor
1. Recognize that Health Informatics is a customer service endeavor, and pursue customer-centered service excellence relentlessly.
2. Establish and maintain recurrent Health Informatics dedicated meetings with facility leaders, and with clinical end users, and ensure that building trust-based relationships with these partners is a central team focus and objective.
3. Include clinical information technology adoption as an obligation and required performance metric in physician contracts.
4. Engage regional executive leadership in a sustained manner, and secure their commitment to – and full co-ownership of – Health Informatics implementation strategies and deliverables.
5. Ensure that the facility Clinical Informaticist is fully integrated into the hospital team across departments and levels of the organization.
6. Sustain high visibility and accessibility of the Health Informatics team among clinicians and facility leaders at all levels and across departments.
7. Recruit a prominent physician leader to serve as a “Physician Health Informatics Liaison“ or champion in each facility, and leverage this individual to advance physician engagement and continuous EHR, CPOE and service improvement.

3. Results: Seven Critical Contributors to Successful Clinical Information Technology Implementation

We have assumed that the use of strategies such as providing clinical end user training in advance of go-live, or the training and deployment of super users on each service line are standard and fairly commonplace. Thus our review of critical contributors to successful implementation does not include these standard strategies and rather focuses on discrete differentiating factors which contributed significantly to our CPOE success that may not be ubiquitous.

3.1 Recognize that Health Informatics is a Customer Service Endeavor: Relentless Pursuit of Customer-Centered Service Excellence

As noted above, adoption of CPOE is perhaps the single greatest change in the delivery of healthcare to impact clinicians in a generation, and it is helpful if all parties – hospital and system executives as well as clinicians – recognize the significant dislocations in workflow that may result. The imperative to insert the scientific evidence base and clinical care standardization into care delivery via CPOE adoption is driven by the 210 000–440 000 Americans a year whose deaths are associated with care delivery-related errors [32]. By any definition, this is an injury epidemic.

One of the greatest challenges that Health Informatics teams face in advancing CPOE adoption among providers is that the urgent need to attack the care safety and clinical effectiveness challenges is not met with Electronic Health Record (EHR) and CPOE products that are intuitive, easy and fast to use [33–34]. EHR and CPOE products are still advancing in terms of usability and effective workflow. This has in turn produced another epidemic of clinical end user (particularly physician) dissatisfaction with EHRs and CPOE across the nation and different vendor EHR products [33–34].

Thus hospital leaders and HI departments are challenged with facilitating adoption of an unpopular – but essential – healthcare technology, as hospitals and end users wait for (and motivate) EHR manufacturers to make the investments required to advance the usability of their products [34–35]. CHRISTUS Health Informatics responded to this challenge using a number of strategies, but central was to ensure that every member of the HI team pursues their duties with a foundational understanding that we are engaged in customer service delivery in every end user encounter.

The highest priority customers of Health Informatics in any hospital or system are the patients cared for. However, because Informatics’ particular contribution to healthcare delivery is achieved through the clinical information technology used by clinicians, they are Informatics’ immediate proximal customer, and the vehicle through which HI value to patients and their care is conveyed. We consider our patient customers as integral with their clinicians, and refer to them as a single entity – Health Informatics’ “customers.”

For Health Informatics, “customer-centricity” translates into three practices that derive from traditional customer service endeavors in other industries and sectors:

  • Respect and exploit the intelligence that our clinician customers bring from their care delivery and end user experience, and recognize their importance as partners of Health Informatics in a process of continuous EHR improvement. Our experience suggests that even where clinicians’ dispassion for informatics products is high it is possible to leverage that dissatisfaction by engaging them in product and service improvement. When clinicians become negative and impatient with the current status of EHRs it is critical to recall the early, rudimentary state in the evolution of many of our products [33–35]. We need clinical end users not only to utilize these products despite poor perceived usability, but to help identify and drive improvements we can execute ourselves and, as necessary, forward to our EHR manufacturer for resolution (for incorporation into a subsequent generation product).

  • Clinicians are our customers and “the customer is never wrong“– they may be misinformed and/ or require education about proper technology use or its limitations – but they are not wrong. This would be true if our EHRs operated as well as contemporary smart phone technology, for example, but our adherence to this principle in health care must be greater when we are “selling“ (i.e., implementing and supporting) EHR products with perceived poor usability [33–35].

  • Clinicians’ concerns and issues must be recognized and responded to, even if the specific request does not fall under Health Informatics’ aegis but under that of another department. In practice this translates into every HI team member managing every clinician question, concern and request that is not HI-related by ensuring that the issue is communicated to the department under whose aegis a concern or problem falls. Dismissing or ignoring a clinician’s issue because “it’s not my job“ is anathema to such a Health Informatics service culture.

As noted above, HI should regard end users not merely as customers requiring service excellence, but as foundational partners in efforts to identify and execute opportunities for continuous improvement of our EHR/CPOE products. Without clinicians’ engagement and input, our ability to improve our products is vastly diminished and as such, HI is dependent on our clinicians’ good will. The above principles are the basis for all HI team members’ behavior and reflected in our facilities’ strong CPOE adoption and effective relationships with providers.

Since our regional CPOE go-live, a major new competitor entered the market of one of our live facilities and competed for physician alignment. Physicians have since reported that, while our EHR product’s usability is regarded as about the same as that of the new competitor, the other facility’s HI customer service/physician support is much inferior. Physicians report that our focus on high quality HI customer service is greatly valued by them. Thus our support of clinicians has become a positive competitive differentiator.

3.2 Engage Recurrent Health Informatics Dedicated Meetings and Build Relationships of Trust with Facility Leaders and Clinical End Users

Early in CPOE implementation HI established recurrent, dedicated informatics meetings at each facility with clinical end users and facility clinical and administrative leaders. In the early CPOE go-live period these meetings were weekly, then as adoption improved became biweekly or monthly (at two facilities are now bimonthly). The meetings became an exchange of concerns, issues and ideas on how to improve usability and facilitate physician CPOE adoption, and served as a reporting vehicle to identify and troubleshoot specific individual or specialty CPOE navigational issues and compliance challenges. A line listing or dashboard of end user reported issues, requests and problems to resolve was generated at the initial meeting, maintained and continually updated. The CPOE Issues Dashboard was distributed at each clinical end user-HI team meeting, and posted online on our HI intranet so that physicians and other clinical end users could monitor issue resolution. This also ensured HI accountability in responding to concerns articulated by clinical end users.

The value of these meetings to clinical end users, facility leaders and HI has been significant. The engagement, trust and sense of co-ownership engendered by the meetings have contributed substantially to our successful adoption of CPOE. These meetings also served a larger purpose – to motivate performance and develop standard best practices to be replicated across facilities in the region (and eventually throughout the enterprise) to improve CPOE performance at all sites.

In addition, regional health informatics leaders, including the regional Chief Medical Information Officer (CMIO), round regularly at all live facilities to meet informally with clinical end users as they work, and to check in and speak with the facility leaders whenever visiting the facility. This enabled frequent ad hoc and private face to face time to assess and discuss CPOE adoption progress and specific issues. The cultivation of personal, trust-based relationships with facility leaders has been a central focus of the HI team at all levels, and of inestimable value to CPOE adoption.

3.3 Inclusion of Clinical Information Technology Adoption Requirements and Performance Metrics into Physician Contracting

Several of our facilities struggled with CPOE adoption early on, achieving low overall facility adoption rates in the 55–70% even a year or longer after their go-live. This changed significantly when HI worked with regional and facility executives to ensure that a CPOE performance/use metric was included in physician group contracts. Inclusion of CPOE performance as a contractual obligation had a powerful – perhaps essential – impact. When any specialty is contracted by the region for clinical services, HI works closely with regional and facility leadership to identify an achievable minimum CPOE performance target, as well as one that exceeds minimum requirements. CPOE use objectives are linked to physician incentive payments for multiple critical physician specialty groups: ED physicians; hospitalists; various surgical specialties; nephrology and cardiology.

Contracting a performance objective helped foster CPOE adoption substantially, and was central to our success in achieving consistent 82–88% month-over-month CPOE overall facility use rates in all live facilities. Hospitalists and ED physicians comprise 58–74% of total order issuance in the three adult facilities in this region, and ensuring CPOE compliance within these specialties set the foundation for overall facility success. The power of physician contracting for CPOE performance is well illustrated by the history of CPOE adoption at one of our facilities: after declining to renew a contract with one hospitalist group (who were poor CPOE adopters) and recruitment of another with demonstrated CPOE experience, a 24–28% improvement in the facility’s overall CPOE use rate occurred rapidly (2–3 quarters). All physicians within this group subsequently achieved 90–94% sustained individual CPOE use rates.

3.4 Regional Executive Leadership Engagement, Commitment and Full Co-Ownership of Health Informatics Strategies and Deliverables

The establishment of a Clinical Information Technology Steering Committee – comprised of regional executive leadership and HI leaders – has been a key contributor to our successful adoption of CPOE. The Committee meets monthly, is sponsored by our regional CEO and co-chaired and managed by regional HI leadership. The meetings are attended consistently by all pertinent regional leaders (CEO, CMO, CNE, COO, CFO), and as needed by facility leadership teams. The 90 minute meeting moves rapidly through a typically packed agenda, and has enabled regional HI leaders to ensure that regional executive leadership is updated – and aligned with – all critical HI activities and deliverables.

The Steering Committee enables and empowers HI regional leaders to drive complex or obstinate issues and problems to the highest level of authority and decision-making in the region. Invariably, this has resulted in acceleration of stagnated processes and improvements in facility administrative and clinical leadership engagement (and compliance). The Steering Committee has been essential in facilitating HI achievement of its organizational objectives and delivery of value. However, this meeting was not an immediate success: 1–2 quarters (3–6 meetings) were required to demonstrate its importance and value to executive leaders, and for leadership to come to recognize – and trust – the capabilities and judgment of the regional HI leadership team.

As noted, when our first three facilities went live serially in the greater San Antonio market, few other commercial hospital systems had yet begun CPOE implementation. In our competitive local market, where we are a smaller actor and the rare non-profit one, we were about to challenge a primary driver of our revenue – our aligned physicians – to adopt a technology user interface that was in a more developmental, early stage than they may have encountered previously in their careers or personal lives. The challenges presented by EHR usability among physicians have become the focus of increasing national discussion and advocacy [33–35].

The reality was that our facilities risked losing aligned physicians – and thus revenue – from physicians decreasing, or eliminating altogether, the business they brought to us because they disliked and were unwilling to adopt CPOE. Prior to the go-live and during the first 18 months our facilities went live, a key concern in the dialogue between regional HI and executive leadership was what might result if our commercial competitors were delayed for an extended period in their own CPOE launches – effectively creating a “safe harbor” where our aligned physicians could shift their business in order to avoid adopting CPOE. This would possibly cause our revenue to decline substantially.

Executive leaders decided nonetheless, and in spite of these risks, to proceed with the launch of CPOE. They did so for one overriding reason: it was in the best interest of our patients. It required considerable courage for executive leadership to focus on this imperative given the uncertainties and financial risks associated with CPOE adoption in our market, and then to proceed. Similar challenges will face leaders in subsequent implementations of clinical information technology, and hopefully past successes will ensure continued bold decision-making that positions the interest of patients above all else.

3.5 Robust Facility Integration of the Clinical Informaticist into the Hospital Team

CHRISTUS Santa Rosa leadership made a seminal decision – and investment – at the outset of the CPOE go-live: to dedicate a full-time, on-site Clinical Informaticist (CI) to each live facility. However, placing a CI in each facility did not guarantee engagement of that CI by the facility’s executive and service line leadership teams, or by clinical end users. Demonstrating the value provided – and building trust – required some months, but soon each facility CI was an integral member of the multidisciplinary hospital care team.

It is difficult to overstate the value and impact of this single decision (and wise use of resources) on our region’s success with CPOE adoption. We are convinced that there is no better way to build relationships of trust and truly collegial collaboration with both providers and facility leaders at all levels than having a skilled, fully dedicated and passionate CI based and “living in the house“ with all the customers and stakeholders that HI serves. However, the latter is necessary but not sufficient, in our view, to ensure facility CPOE adoption – or successful implementation of other clinical information technology. In addition, facility leadership must be educated and encouraged (when necessary directed by regional executive leadership) to genuinely welcome and fully integrate this individual into the facility team.

This process often required 2–4 months to achieve, with close attention from regional HI leadership to ensure that CI integration is actually (and appropriately) occurring. Across the region, facility CPOE use rates passed the 70–75%+ level when true and robust integration of the CI into the hospital team was achieved. We suspect that a strong relationship between facility adoption success and level of facility CI integration exists and is a key to successful HI implementation.

In addition to each facility having a dedicated full time Clinical Informaticist, we engaged and trained super-users, typically nurses or unit clerks, to lend at the elbow support to physicians navigating CPOE. Prior to and ongoing throughout the implementation, we endeavored to ensure that there was at least 1–2 super-users on each service line in each facility. These super-users were trained to deal with the most commonly occurring end user impediments and issues, and they would contact and engage the CI when unable to assist a provider or colleague due to the complexity of the problem.

3.6 Visibility and Accessibility of Health Informatics Team to Clinicians and Facility Leaders

In our experience, the facility CI has been indispensable in the front line facility HI adoption effort day in, day out. We believe that CI performance is one of the most important predictors – along with aggressive contracting and facility leadership engagement – of a facility’s success in CPOE adoption. Each CI has been selected to ensure, insofar as possible, that s/he has the essential professional and personal skills to shepherd successful CPOE and other clinical information technology adoption. Well integrated into the hospital team, the CIs actively round multiple times each day, attend various facility meetings of clinicians, and are on call to providers, nurses and other end users throughout the hospital (through email, mobile telephone, or a wireless, voice-activated intra-facility communication device).

The Regional HI leadership team also spends their time predominantly within our region’s hospitals in order to attend important meetings with different segments of our end user/customer community, to meet with facility leaders, to work with the facility Health Informatics Physician Liaison (next section), and with the facility CI. HI leaders regard visiting each facility regularly to round and spend time with the facility CI as one of their most important activities and responsibilities. In this manner, HI leaders secure ongoing, real time updates from each facility CI and can pose an essential recurring question: what can we do to help this CI be maximally effective in their job? In our experience if the facility CI is not working successfully in each facility, there can be no success in CPOE adoption.

In addition to the above, HI leaders attend various meetings in each facility to educate, communicate, and advocate HI’s adoption strategies and tactics. Our attendance at every pertinent facility meeting would be extremely difficult and leave little time to do much else. Recognizing that it is impossible to attend every facility meeting where a health informatics issue or question is raised, meetings were often divided according to our respective credentials as a nurse and a physician (with both leaders attending all meetings dedicated fully to clinical information technology and CPOE adoption).

Our clinical and administrative leadership fully appreciated the revolution in healthcare delivery and workflow – positive and disruptive – that CPOE adoption brought and have been unfailingly supportive of HI leaders. Alignment with and engagement by the Regional CMO and CNE was another contributor to adoption success, and these executives lent their political and personal influence to CPOE adoption. As HI regional leaders articulated CPOE adoption objectives, strategies and tactics to these executives, they have – reliably and effectively – represented HI’s objectives in various meetings that HI leaders were unable to personally attend.

3.7 Recruitment of a Facility Physician Leader to Serve as “Physician Health Informatics Liaison“ and Champion

As noted, a key element facilitating our success in partnering with physician communities was the creation of dedicated HI monthly meetings at each facility, where physicians can articulate requests and problems encountered with CPOE and hear updates on HI’s continuous improvement efforts. Discussions at physician end user meetings focus on a mix of CPOE clinical content (order sets), navigation, and service support (plus end user “tips“ to accelerate physician workflow in CPOE). These Physicians Clinical Information Technology/CPOE Meetings are hosted by HI leaders and consistently attended by facility administrative and clinical leaders and members of the regional Information Management team. Scheduled at a time when physicians can attend, with a meal provided, significant numbers of high CPOE using physicians participate (usually 8–18+ at each facility). Physician contracts enabled certain physicians to bill and be remunerated for the hour during the first 1–2 years of the CPOE go-live, which facilitated attendance.

To build trust and engage true co-ownership of the meeting with each facility’s physician community, a respected physician leader within the facility was recruited to chair the meeting and serve as a point person in the facility for physician CPOE concerns. This “Physician Health Informatics Liaison” was selected with input and approval of facility leaders, and resources were secured to pay for up to eight hours of work per month on HI related issues. The Physician HI Liaison serves as a local physician leader to complement the activities of HI leaders, to act as a vehicle for communication of physician CPOE issues to and from HI, and to advocate for CPOE adoption among peers and colleagues in their facility. The Physician HI Liaisons augment HI’s surveillance of physician CPOE adoption problems, obstacles and areas of needed improvement. Each has become a critical ally of the regional CMIO and the facility CI, with whom productive and valuable working relationships developed.

The right person must be selected for this important role. Professionally, the physician must be a strong adopter of CPOE (preferably with an 80–85%+ personal use rate). Specialty is not a consideration in any one facility, but we seek a diverse specialty representation across the region. Our current cadre of Liaisons includes a general surgeon, a family medicine physician, and a hospitalist, each serving a one year (renewable) term. This constellation of important specialties for successful CPOE adoption was deliberate. Another desired specialty would be an ED physician.

Personality factors weigh equally in selecting the right Physician HI Liaison. The individual must be senior enough (and respected clinically) to command attention from colleagues, and to add value to HI efforts to secure physician engagement and compliance. But the HI Liaison should be young enough (in mind if not chronology) to possess the energy to make an HI contribution to the hospital beyond their clinical one, and evidencing a well informed interest in clinical information technology. Interestingly, all four Physician HI Liaisons that have served thus far in our region have been aged 30–45. A gregarious, inclusive and easy-to-collaborate-with personality is desirable. Our Physician HI Liaisons met these criteria, and their impact on CPOE adoption in each facility has been substantial.

The most important personal qualification for a Physician HI Liaison is an appropriate attitude toward clinical information technology. This physician must recognize the transformative value of clinical information technology to improving clinical outcomes and patient safety, yet also appreciate our early phase of EHR/CPOE technology evolution on a journey of continuous improvement and optimization. Physician HI Liaisons must be fully committed to genuine partnership with HI leaders and the facility CI – one of trust and candor – and are usually people who want to “be part of the solution, not part of the problem,” willing to “light a candle rather than curse the darkness.” Selection of the right facility Physician HI Liaison is of real consequence. The wrong person in the role can undermine rather than optimize HI work processes and deliverables. Our current physician liaisons are regarded by stakeholders as highly effective and valuable to CPOE adoption (and to future clinical information technology implementations).

4. Discussion

The above described strategic and operational contributors to successful clinical information technology implementation focus on diverse organizational processes: building and maintaining a service culture and processes within Health Informatics; engaging and aligning with facility, regional, and enterprise leaders through regular meetings and updates on implementation status and challenges; use of contracting for clinical information technology adoption as a discrete performance metric within physician contracts whenever possible; full co-ownership of Health Informatics processes and deliverables by leadership at all levels; robust integration of the facility clinical informaticist into the hospital team; high visibility and accessibility of the Health Informatics team (including team leaders) at the facility and service line level; and recruitment of a physician leader to serve as a Physician Health Informatics Liaison and champion in each facility. It would be arbitrary to rank these contributors against each other in terms of relative importance for a successful implementation: each has distinct and significant value; many are additive and mutually reinforcing.

We believe that physician contract stipulation of a CPOE performance metric was a prominent and powerful factor in facilitating physician adoption, particularly among hospitalists, where a pay for performance stratification differentiated between 90% and higher CPOE use versus less than 90%. Our ED physicians also had a CPOE performance expectation built into their contracts. Thus with these two specialties, which together constitute greater than 60% of total order issuance at the three adult facilities, it appears that contracting for CPOE performance had value. Similar objectives were included in other specialty agreements with physicians in the adult facilities that shared payment and risk with the facility.

However, one must be careful not to overstate this connection or imply a linear causation. Firstly, there was a selection bias in that we queried all companies competing for physician contracts in these specialties as to their past experience using CPOE, and only those with a history of past adoption were typically advanced in our proposal assessment process. In addition, the hospitalist and ED physician groups within these hospitals tended to be populated with somewhat younger physicians than other specialties, and though we have not assessed this statistically, we have observed that younger physicians adopt CPOE more readily and easily than their older colleagues. Further, we did not routinely have to rigidly “enforce” or bring particular attention to the contracted CPOE performance metric at an individual physician level, in part because this was left to the management of these contracted companies once lower performing physicians were identified, but also because just having the metric in the background as a stated performance expectation seemed to foster a culture of adoption by itself.

Table 1 and ►Figure 1 illustrate that the four facilities in the CHRISTUS Santa Rosa region utilized quite variable periods of time to achieve an 80% overall facility CPOE use rate. The stand out facility in terms of CPOE adoption pace was clearly Children’s Hospital of San Antonio, which required but two months to reach and then exceed an 80% overall facility CPOE use rate. Notable is the fact that this facility did not launch CPOE until 31 months after the first facility in the region. The unique nature of patient care provided and the associated care challenges faced by pediatricians contributed in general to a very strong safety and quality culture/focus among Children’s Hospital’s nurses and providers long before CPOE was launched there. Thus in its baseline clinical culture, Children’s Hospital always regarded and welcomed CPOE as a positive implementation to improve the quality and safety of patient care, rather than as a burden that slows or interferes with clinical workflow, as was more evident in our adult facilities.

The reasons for the late or delayed launch at Children’s Hospital are multiple and complex. The institution was in the midst of major physical and organizational transformations to upgrade and expand its facilities and secure a new academic partner, respectively. From a Health Informatics viewpoint, it became clear from our CPOE launch in the region’s adult facilities that the CPOE module we had utilized within our MEDITECH EHR would require specific modifications to enable specialized functionality to treat pediatric patients – and their unique clinical characteristics – effectively and safely (e.g., dose-volume calculations, order strings, etc.).

This activity, as well as development of an entirely new suite of pediatric order sets (since the 250+ adult order sets were not applicable), both necessitated and facilitated extensive and integral involvement of clinical leaders across specialties in Children’s Hospital CPOE planning and go-live. Thus, by the November 2014 CPOE launch when all the above (and much other) work in the facility had been completed, the physician and nursing community of the Children’s Hospital, which as a pediatric academic medical center had a baseline of much greater receptivity and less resistance than our most receptive adult facility, were primed, energized and more than ready to engage CPOE and achieve strong adoption. This is evidenced unequivocally by the fact that the facility broke the 80% overall facility use rate threshold in a mere two months, and since has rapidly approached and will soon breach the 90% level.

Among the adult hospitals, CSR-Westover Hills progressed to the 80% use threshold more rapidly than its peers (11 vs. 24 and 18 months respectively). We have no single factor to which we can attribute this difference. The median age of physicians at this facility may be somewhat younger than the other two adult facilities. However, the most powerful contributor to Westover Hills’ strong performance is the fact that facility leadership was very engaged and aggressively messaging an expectation of (and support for) CPOE adoption. Several specialties entered risk/benefit sharing contractual arrangements with the hospital, and CPOE performance was one of a number of performance metrics linked to physician incentive payments (though this was also the case at the other two adult facilities). In addition, we recruited and deployed our first Physician Health Informatics Liaison at this facility, a leader in the surgery department, and this individual engaged and performed formidably in communicating the CPOE adoption imperative to his colleagues throughout the facility, and in identifying needed CPOE refinements for the Health Informatics team.

Several aspects of CPOE adoption at CSR-Medical Center merit mention. For over a year after launch, this facility struggled to get its overall use rate above the 50–60% level. Three major changes then occurred which facilitated rapid improvement. New facility leadership arrived and engaged intensively with Health Informatics to increase CPOE use. In addition, the facility’s hospitalist group completed their contract and a request for proposal was issued for a new contract period. Leadership selected another hospitalist group for the facility with demonstrated past CPOE adoption experience, and a CPOE performance metric was stipulated in the incentives component of the contract. Many of the new hospitalists recruited were 10–15 years younger than the prior group, but moreover almost all had used various EHR CPOE modules in the past.

At this time we also recruited the chief hospitalist of the facility to serve as the facility’s Physician Health Informatics Liaison, and this individual engaged passionately with the Informatics team, with her hospitalist colleagues, and with other physicians in the facility on a variety of issues, including continuous improvement of order sets and CPOE functionality, as well as the adoption imperative. Over the ensuing nine months, the facility CPOE use rate increased steadily month over month to the 80% threshold, and once achieved, continued incremental use rate growth. As of this writing in August 2015, the facility broke the 90% use rate threshold.

CSR-New Braunfels had steady improvement of its use rate over time since launch, but its adoption curve has generally been less aggressive than the other facilities in this region and characterized by long periods of little or no growth and occasional use rate regression. This physician community shared many of the characteristics of other adult facilities in the region, and it was generally highly engaged in CPOE adoption, as was its strong leadership team. The only differences from the other facilities relative to CSR-New Braunfels has been a slower inclusion of a specific CPOE performance level in the facility’s contract with hospitalists, and entrenched resistance to adoption in one high volume specialty. These factors explain most of the gap in performance. The facility employed all of the seven success factors as were employed at the three other higher performing facilities.

However two years into the CPOE go-live, a notable differentiating feature emerged. CSR-New Braunfels was located in an ex-urban market with few nearby competitors. During the CPOE go-live, a new competitor facility in the local market was first announced, marketed heavily to physicians, and then launched. While it is not possible to attribute any specific changes in CPOE performance to these events, it was evident to regional, facility and Health Informatics leaders that physicians would be comparing their opportunities to work at each facility and, prominent in their assessment, would be their EHR and CPOE user experience and HI support. We learned that while this new facility also implemented CPOE from inception, the use policy was, as reported by physicians, considerably less rigorous as existed in our facility. This may have lent a more tentative character to physicians’ commitment to adopt CPOE at CSR-New Braunfels in ensuing years. However, as of this writing, very little physician attrition has occurred at our facility, and CPOE use rates have again begun to climb to the mid-80s. Further, physicians have provided feedback that while CPOE and EHR at the competitor facility are somewhat more navigable, they much preferred the strong at the elbow support and willingness to engage recommendations for desired refinements integral to our customer service orientation.

Our presentation of seven key factors contributing to successful CPOE adoption in the CHRISTUS Santa Rosa region is not an exhaustive or comprehensive checklist of recommendations and valuable strategies to achieve successful implementation. Hospital facilities and health systems in the U.S. and beyond have reported other success factors, including pre-implementation usability testing with end users, advance CPOE training and support, and effective deployment of super users [36]. We utilized many of these strategies as well in our CPOE implementation, but have assumed that these best practices are commonplace if not ubiquitous among hospital peers across the nation. Thus our focus here has been on success factors that, from our experience and discussions with colleagues elsewhere, may be less common and have differentiated our strategy and operational focus.

Other institutions have reported similar CPOE adoption success by focusing on a number of the factors described. Wu et al. emphasized that active physician involvement at multiple institutional levels improves end user understanding of the necessity and course of imminent changes and was key to successful adoption [36]. In addition, the authors noted their experience, aligned with our own, that a de facto partnership with facility physicians benefited greatly from specific physician end user input [36]. Cooley et al. [37] also emphasized the importance of ongoing physician involvement and input, yet noted the added value derived by ensuring the involvement of all affected ancillary personnel.

Strong leadership by and engagement of clinicians were reported as integral to successful implementation by Poon et al. [38]. Ozdas and Miller noted that sociocultural factors may dominate in determining the success of CPOE implementation and should govern technical factors [39]. High quality change communication and management can overcome resistance to CPOE adoption among even older physicians as described by Muslin and colleagues [40], and underlying most of our reported success factors as a central tenet and process feature has been engagement and a routinization of effective communication between clinicians and HI, and between clinicians and administrative leaders about CPOE in all its aspects.

Ingebrigtsen et al. describe a need for strong clinical leaders with past experience implementing information technology (IT) project management and informatics, a long term commitment to clinical IT and belief in its value [41]. Such leaders are motivated to adopt new information technology, and can sustain confidence and stability through the dislocations often accompanying adoption by engaging proactive leadership behaviors and partnerships with Health Informatics [41].

As Muslin et al. have emphasized, it is critical that physicians and other clinicians be engaged as active participants and co-creators – rather than as passive receivers or implementers – of a new technology [40], and the rigorous give and take of our routine dedicated HI meetings with clinicians in each facility strongly support this recommendation. Co-creation is often equated with customization. In our experience, however, while enabling a degree of customization among our co-creating clinical end users was/is desirable, the reality is that, given the current limitations of most EHR and CPOE products, customization must occur within the limitations of the product, and within the parameters of the evidence base. This understanding among physician end users must underlie the collaborative relationship with Health Informatics if it is to be successful.

Muslin et al. note that the perceptions of individual physicians regarding a technology are largely formed by the behavior, attitudes and statements of their colleagues, rather than by the exhortations of managers [40, 42]. Perceptions, beliefs and attitudes about the value of a particular technology are shaped through information exchanges and become shared within a physician or other end user community, yet this belief and perceptual content can be divorced from the technologies’ actual merits or deficits [40, 42, 43]. That is, resistance may exist even when the technology is sound. Our seven principles and recommended practices recognize these realities and leverage them toward advancing CPOE adoption. Further, we suggest they extend to and are fungible across other clinical information technologies that Health Informatics is yet to embrace and implement.

Simon and colleagues [30] cited the importance of clear governance and decision-making that involves clinicians as critical stakeholders, as well as education and training and live in-person support as key contributors to successful implementation. This finding closely parallels our experience, as does their observation that successful implementation hinged on the ability of strong clinical leaders to address and manage clinicians’ perceptions and fear of change [30]. Simon et al. note the imperative to anticipate consequences and to have a process ready to address them, one expression of our “customer service” orientation toward clinical end users and use of frequent, regular meetings with clinicians that are fully dedicated to capturing and responding to their actionable CPOE concerns, requests and issues [30].

Simon et al. also reported the importance of identifying and supporting a “champion” among each affected group of clinicians (physicians, nurses and pharmacists) [30]. While our role titling may differ in calling such physician champions “Health Informatics Physician Liaisons,” we concur that great value is added by these individuals in facilitating rich and effective communication and advocacy both to and from clinicians, and with administration [30]. In our facilities, other clinicians such as nurses and pharmacists, and administrative personnel were welcomed and frequently attended our dedicated HI meetings with physicians. Our experience also confirms these authors’ conclusion that whether or not Meaningful Use incentives achieve their objectives depends very much on the extent to which organizations can assimilate and apply such critical lessons [30].

In a systematic and comprehensive review of factors contributing to successful EHR implementation in hospitals, Boonstra et al. identified a number of recurring factors that validate our findings [44]. Among these were decisive and full leadership backing, active management involvement and support, participation of clinical staff in implementation processes, establishment of a multidisciplinary implementation group to deal with EHR issues as they arise, overcoming physician resistance by involving front line users and addressing their concerns, identifying clinical champions, and an organizational culture that supports collaboration and fosters teamwork [44]. These findings align well with the seven factors we describe and which were pivotal in achieving adoption success within our facilities.

5. Conclusion

Different hospitals have diverse physician/clinical communities, institutional contexts, financial resources, and management processes, so these seven recommendations are not intended to be prescriptive, or purported to be applicable everywhere. Each factor has contributed substantially to our CPOE success in hospitals located in a large metropolitan region, and will be replicated in future implementations. The seven factors have enabled:

  1. Achievement of high levels of CPOE adoption in all live facilities (82–88% current facility use rate levels month over month).

  2. Positioning the region to achieve facility use rates at the 90%+ level over the next 6–12 months.

  3. Creation of a clinical end user customer-centric service process, and a favorable customer experience and impact that are highly regarded and valued by providers and leaders, and which serve as a competitive differentiator in our market.

  4. Partial reduction of substantial initial physician dissatisfaction with clinical information technology. EMR and CPOE were leading sources of dissatisfaction in physician surveys after the first year of the CPOE go-live, due to the perceived poor usability of current EHR products. Reported dissatisfaction has since decreased but remains too high.

  5. Cultivating regional and facility leaders’ perception of Health Informatics as a highly effective and valuable department.

  6. Establishing and maintaining a satisfying and gratifying work environment for Health Informatics associates.

  7. Most importantly, maximizing Health Informatics’ contribution to the safety and best possible evidence-based care of our patients – our true customer in the end, who matters the most.

While the seven strategic and operational success factors discussed here were first employed in the context of CPOE adoption, we have found that these best practices generalize to other health informatics implementations. Each factor has become integral to our ongoing organizational service culture within Health Informatics, and enables the delivery of maximum value by our department.

We believe these factors are fungible and can generalize to many other hospitals and care delivery settings, with adaptation as needed and appropriate. In our experience, the processes were as integral to achieving Health Informatics implementation success as was the selection of the right technology vendor or software. While time and resource consuming, few among our Health Informatics team would suggest that the return on investment in these processes was not substantial, and even pivotal, in achieving successful implementation.

Acknowledgements

We greatly appreciate the partnership, contributions and passion for excellence of the following individuals: P. Carrier, J. Martin MD, P. Toney RN, MSN, K. Haynes, J. Gillian MD, J. Bourgeois, E. Graves, J. Gerard MD, G. Wilson MD, S. Pangburn MD, A. Hilger MD, V. Broumand MD, C. Bowe, A. Lambert RN, J. Pendon MD, M. Barnett MD, M. Williams, J. Wesson, D. Hobbs RN, E. Briggs MD, T. Curran MD; W. Albrecht MD, M. Tilly MD; L. Stewart PharmD and her team, A. Gest MD, B. Sosa, H. Garza RN, D. Ortiz RN, C. Evans RN, G. Smith, K. Hazlett PharmD, C. Santiago, J. Scott, J. Irving, J. Noreiga, K. Padilla, J. Harrison RN, S. Watson, K. Carlton RN, K. Hagerman RN, MSN, M. Delgado RN, L. Duelm, K. Robey, D. Walls, K. Vickers, D. Ruiz, and N. Gellert.

Footnotes

Statement of Clinical Relevance

This report focuses on the seven most valuable strategies deployed by the Health Informatics team in a four hospital CHRISTUS region in order to achieve strong CPOE adoption, along with critical success lessons learned along the way. While these elements involved specific de novo process generation to support operations, collectively they represent the creation of a new customer-centric service culture in our Health Informatics team as a foundation for ensuring strong clinical information technology adoption. The seven success factors described are not limited in their value to or impact on CPOE adoption, but generalize to – and can foster success in – varied other clinical information technology implementations within diverse hospitals.

Conflict of Interest

The authors have no conflicts of interest in the completion of this applied research and report.

Human Subject Protections

No human subjects were engaged in the completion of this work, and no identified personal health information was used in the analyses reported.

References

  • 1.CPOE for medication orders. 2010. http://www.cms.gov/Regulations-and-Guidance/Legislation/EHRIncentivePrograms/downloads/1_CPOE_for_Medication_Orders.pdf [Google Scholar]
  • 2.Bates DW, Leape LL, Cullen DJ, Laird N, Petersen LA, Teich JM, Burdick E, Hickey M, Kleefield S, Shea B, Vander Vliet M, Seger DL. Effect of computerized physician order entry and a team intervention on prevention of serious medication errors. J Am Med Assoc 1998; 280(15):1311–1316. [DOI] [PubMed] [Google Scholar]
  • 3.Bates DW, Teich JM, Lee J, Seger D, Kuperman GJ, Ma’Luf N, Boyle D, Leape L. The impact of computerized physician order entry on medication error prevention. J Am Med Inform Assoc 1999; 6(4):313–321. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Kuperman GJ, Bobb A, Payne TH, Avery AJ, Gandhi TK, Burns G, Classen DC, Bates DW. Medication-related clinical decision support in computerized provider order entry systems: a review. J Am Med Inform Assoc 2007; 14(1):29–40. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Khajouei R, Jaspers MW. The impact of CPOE medication systems’ design aspects on USABILITY, workflow and medication orders. Methods Inf Med 2010; 49(1): 3. [DOI] [PubMed] [Google Scholar]
  • 6.Georgiou A, Prgomet M, Paoloni R, Creswick N, Hordern A, Walter S, Westbrook J. The effect of computerized provider order entry systems on clinical care and work processes in emergency departments: a systematic review of the quantitative literature. Ann Emerg Med 2013; 61(6):644–653. [DOI] [PubMed] [Google Scholar]
  • 7.McKibbon KA, Lokker C, Handler SM, Dolovich LR, Holbrook AM, O’Reilly D, Tamblyn R, Hemens BJ, Basu R, Troyan S. The effectiveness of integrated health information technologies across the phases of medication management: a systematic review of randomized controlled trials. J Am Med Inform Assoc 2012; 19(1):22–30. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Georgiou A, Prgomet M, Markewycz A, Adams E, Westbrook JI. The impact of computerized provider order entry systems on medical-imaging services: a systematic review. J Am Med Inform Assoc 2011; 18(3):335–340. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Kaushal R, Jha AK, Franz C, Glaser J, Shetty KD, Jaggi T, Middleton B, Kuperman GJ, Khorasani R, Tanasijevic M. Return on investment for a computerized physician order entry system. J Am Med Inform Assoc 2006; 13(3):261–266. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Weiner M, Gress T, Thiemann DR, Jenckes M, Reel SL, Mandell SF, Bass EB. Contrasting views of physicians and nurses about an inpatient computer-based provider order-entry system. J Am Med Inform Assoc 1999; 6(3):234–244. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Oren E, Shaffer E, Guglielmo B. Impact of emerging technologies on medication errors and adverse drug events. Am J Health Syst Pharm 2003; 60(14):1447–1458. [DOI] [PubMed] [Google Scholar]
  • 12.Maslove DM, Rizk N, Lowe HJ. Computerized physician order entry in the critical care environment: a review of current literature. J Intensive Care Med 2011; 26(3):165–171. [DOI] [PubMed] [Google Scholar]
  • 13.Khajouei R, Wierenga PC, Hasman A, Jaspers MWM. Clinicians satisfaction with CPOE ease of use and effect on clinicians’ workflow, efficiency and medication safety. Int J Med Inform 2011; 80(5):297–309. [DOI] [PubMed] [Google Scholar]
  • 14.Mir C, Gadri A, Zelger GL, Pichon R, Pannatier A. Impact of a computerized physician order entry system on compliance with prescription accuracy requirements. Pharm World Sci 2009; 31(5):596–602. [DOI] [PubMed] [Google Scholar]
  • 15.Leung AA, Keohane C, Amato M, Simon SR, Coffey M, Kaufman N, Cadet B, Schiff G, Zimlichman E, Seger DL, Yoon C, Song P, Bates DW. Impact of vendor computerized physician order entry in community hospitals. J Gen Intern Med 2012; 27(7):801–807. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Devine EB, Hansen RN, Wilson-Norton JL, Lawless NM, Fisk AW, Blough DK, Martin DP, Sullivan SD. The impact of computerized provider order entry on medication errors in a multispecialty group practice. J Am Med Inform Assoc 2010; 17(1):78–84. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Aarts J, Koppel R. Implementation of computerized physician order entry in seven countries. Health Aff (Millwood) 2009; 28(2):404–414. [DOI] [PubMed] [Google Scholar]
  • 18.Bates DW, Kuperman GJ, Teich JM. Computerized physician order entry and quality of care. Qual Manag Health Care 1994; 2(4):8–27. [PubMed] [Google Scholar]
  • 19.Lee F, Teich JM, Spurr CD, Bates DW. Implementation of physician order entry: user satisfaction and self-reported usage patterns. J Am Med Inform Assoc 1996; 3(1):42–55. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Sittig DF, Stead WW. Computer-based physician order entry: the state of the art. J Am Med Inform Assoc 1994; 1(2):108–123. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Connolly C. Cedars-Sinai doctors cling to pen and paper. Washington Post; March 21, 2005. [Google Scholar]
  • 22.Massaro TA. Introducing physician order entry at a major academic center: I. Impact on organizational culture and behavior. Acad Med 1993; 68(1):20–25. [DOI] [PubMed] [Google Scholar]
  • 23.Massaro TA. Introducing physician order entry at a major academic center: II. Impact on medical education. Acad Med 1993; 68(1):25–30. [DOI] [PubMed] [Google Scholar]
  • 24.Sittig DF, Krall M, Kaalaas-Sittig J, Ash JS. Emotional aspects of computer-based provider order entry: a qualitative study. J Am Med Inform Assoc 2005; 12(5):561–567. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Ash JS, Gorman PN, Lavelle M, Stavri PZ, Lyman J, Fournier L. Perceptions of physician order entry: results of a cross-site qualitative study. Methods Inf Med 2003; 42(4):313–323. [PubMed] [Google Scholar]
  • 26.Ash J, Fournier L, Stavri P, Dykstra R. Principles for a successful computerized physician order entry implementation. AMIA Annu Symp Proc 2003; 2003: 36–40. [PMC free article] [PubMed] [Google Scholar]
  • 27.Karsh BT, Weinger MB, Abbott PA, Wears RL. Health information technology: fallacies and sober realities. J Am Med Inform Assoc 2010; 17(6):617–623. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Carayon P, Cartmill R, Blosky MA, Brown R, Hackenberg M, Hoonakker P, Hundt AS, Norfolk E, Wetterneck TB, Walker JM. ICU nurses’ acceptance of EHR. J Am Med Inform Assoc 2011; 18(6):812–819. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Ammenwerth E, Schnell-Inderst P, Siebert U. Vision and challenges of evidence-based health informatics: a case study of a CPOE meta-analysis. Int J Med Inform 2010; 79(4): e83-e88. [DOI] [PubMed] [Google Scholar]
  • 30.Simon SR, Keohane CA, Amato M, Coffey M, Cadet B, Zimlichman E, Bates DW. Lessons learned from implementation of computerized provider order entry in 5 community hospitals: a qualitative study. BMC Med Inform Decis Mak 2013; 13: 67. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.The American Recovery and Reinvestment Act: HR1; 2009. http://frwebgate.access.gpo.gov/cgi-bin/get-doc.cgi?dbname=111_cong_bills&docid=f:h1enr.pdf [Google Scholar]
  • 32.James JT. A new, evidence-based estimate of patient harms associated with hospital care. J Patient Saf 2013; 9(3):122–128. [DOI] [PubMed] [Google Scholar]
  • 33.Friedberg MW, Chen PG, Van Busum KR, Aunon Chau Pham FM, Caloyeras JP, Mattke S, Pitchforth E, Quigley DD, Brook RH, Crosson FJ, Tutty M. Factors affecting physician professional satisfaction and their implications for patient care, health systems, and health policy. Rand Corporation 2013. [PMC free article] [PubMed] [Google Scholar]
  • 34.American Medical Association. Improving Care: Priorities to improve electronic health record usability. Chicago, IL: American Medical Association; 2014. [Google Scholar]
  • 35.Gellert GA, Ramirez R, Webster SL. The rise of the medical scribes industry: Implications for electronic health record improvement. JAMA 2015; 313(13):1315–1316. [DOI] [PubMed] [Google Scholar]
  • 36.Wu RC, Abrans H, Baker M, Rossos PG. Implementation of a computerized physician order entry system of medications at the University Health network – physicians’ perspectives on the critical issues. Healthc Q 2006; 9(1):106–109. [PubMed] [Google Scholar]
  • 37.Cooley TW, May D, Alwan M, Sue C. Implementation of computerized prescriber order entry in four academic medical centers. Am J Health Syst Pharm 2012; 69(24):2166–2173. [DOI] [PubMed] [Google Scholar]
  • 38.Poon EG, Blumenthal D, Jaggi T, Honour MM, Bates DW, Kaushal R. Overcoming barriers to adopting and implementing computerized physician order entry systems in U.S. Hospitals. Health Aff (Millwood) 2004; 23(4):184–190. [DOI] [PubMed] [Google Scholar]
  • 39.Ozdas A, Miller RA. Care provider order entry (CPOE): a perspective on factors leading to success or to failure. Yearb Med Inform 2007: 128–137. [PubMed] [Google Scholar]
  • 40.Muslin IS, Vardaman JM, Cornell PT. Fostering acceptance of computerized physician order entry: insights from an implementation study. Health Care Manag (Frederick) 2014; 33(2):165–171. [DOI] [PubMed] [Google Scholar]
  • 41.Ingebrigtsen T, Georgiou A, Clay-Williams R, Magrabi F, Hordern A, Prgomet M, Li J, Westbrook J, Briathwaite J. The impact of clinical leadership on health information technology adoption: Systematic review. Int J Med Inform 2014; 83(6):393–405. [DOI] [PubMed] [Google Scholar]
  • 42.Fulk J, Steinfield CW, Schmitz J, Power JG. A social information processing model of media use in organizations. Commun Res 1987; 14(5):529–552. [Google Scholar]
  • 43.Orlikowski WJ, Hofman JD. An improvisational model for change management: the case of groupware technologies. Sloan Manag Rev 1997; 38: 11–21. [Google Scholar]
  • 44.Boonstra A, Versluis A, Vos JFJ. Implementing electronic health records in hospitals: A systematic literature review. BMIC Health Serv Res 2014; 14: 370. [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from Applied Clinical Informatics are provided here courtesy of Thieme Medical Publishers

RESOURCES