Abstract
Background
Computerized physician order entry (cpoe) systems allow for medical order management in a clinical setting. Use of a cpoe has been shown to significantly improve chemotherapy safety by reducing the number of prescribing errors. Usability of these systems has been identified as a critical factor in their successful adoption. However, there is a paucity of literature investigating the usability of cpoe for chemotherapy and describing the experiences of cancer care providers in implementing and using a cpoe system.
Methods
A mixed-methods study, including a national survey and a workshop, was conducted to determine the current status of cpoe adoption in Canadian oncology institutions, to identify and prioritize knowledge gaps in cpoe usability and adoption, and to establish a research agenda to bridge those gaps. Survey respondents were representatives of cancer care providers from each Canadian province. The workshop participants were oncology clinicians, human factors engineers, patient safety researchers, policymakers, and hospital administrators from across Canada, with participation from the United States.
Results
A variety of issues related to implementing and using a cpoe for chemotherapy were identified. The major issues concerned the need for better understanding of current practices of chemotherapy ordering, preparation, and administration; a lack of system selection and procurement guidance; a lack of implementation and maintenance guidance; poor cpoe usability and workflow support; and other cpoe system design issues. An additional three research themes for addressing the existing challenges and advancing successful adoption of cpoe for chemotherapy were identified:
The need to investigate variances in workflows and practices in chemotherapy ordering and administration
The need to develop best-practice cpoe procurement and implementation guidance specifically for chemotherapy
The need to measure the effects of cpoe implementation in medical oncology
Conclusions
Addressing the existing challenges in cpoe usability and adoption for chemotherapy, and accelerating successful migration to cpoe by cancer care providers requires future research focusing on workflow variations, chemotherapy-specific cpoe procurement needs, and implementation guidance needs.
Keywords: Chemotherapy, cpoe, computerized physician order entry, usability
1. INTRODUCTION
Computerized physician order entry (cpoe) has been shown to significantly improve chemotherapy safety by reducing the number of prescribing errors1–7. Since the early 2000s, research has established a knowledge base for successful cpoe implementation and ongoing use. Some of the critical success factors include the design and usability of the cpoe system8–12, standardization of the ordering process13,14, seamless integration with other health information systems and user workflows12,15–17, effective training and support10,14,16,18, support from leadership10,14,16, collaborative project management14,16, and effective ongoing maintenance and support16,19.
Although the usability of cpoe systems has been identified as a critical success factor, the usability of existing cpoe systems seems to be far from optimal. Usability can be defined as the extent to which a product can be used by specific users to achieve specific goals, with effectiveness, efficiency, and satisfaction in a specific context of use20. Some of the commonly reported cpoe usability issues include an excessively complex and unintuitive user interface21– 25, a cluttered or poorly organized display, lack of safeguards21,24, and inflexible or inefficient user interaction21,23–25.
Furthermore, there is a paucity of literature investigating the usability of cpoe systems for chemotherapy, despite studies that illustrate its importance. Corrao et al.26 found that, for most users, a cpoe system implemented without an initial usability evaluation raised issues of efficiency and satisfaction that could have been identified and resolved through usability testing before implementation. Khajouei et al.27 compared the ordering efficiency of oncologists using a cpoe system in a laboratory setting with and without the use of standard order sets. Those authors found that, although ordering efficiency can be improved by integrating standard order sets into a cpoe system, efficiency can be significantly impaired by usability problems.
Knowledge about the experiences of cancer care providers in implementing and using a cpoe system for chemotherapy is relatively limited. A handful of studies describe the methods used for implementing a cpoe system for chemotherapy6,14,19,28. Harshberger et al.29 reported improved user satisfaction and better completeness of chemotherapy orders and documentation after a cpoe system was implemented to replace paper-based charts at a large multi-site teaching hospital, and Brockstein et al.30 described the impact of the cpoe implementation at the same institution on documentation, communication, operations, quality improvement, and research. At a large Dutch teaching hospital, Pirnejad et al.31 found that a user requirement–driven and process-oriented cpoe system development process and proximity of the development site to the implementation site resulted in a preference by the chemotherapy and hematology clinicians for a home-grown cpoe system over a commercially available hospital-wide cpoe system.
To address the foregoing knowledge deficits, we conducted a mixed-methods qualitative study involving a workshop and a survey. The survey aimed to understand the adoption and use of cpoe by Canadian cancer care providers. The findings provided the context for discussions during the workshop. The workshop aimed to identify and prioritize knowledge gaps in chemotherapy cpoe adoption and usability literature and to establish a research agenda for bridging the knowledge gaps so as to improve clinical practice.
2. METHODS
2.1. Workshop Design
A purposive sample of 30 experts and representatives of Canadian cancer care providers was established using the literature, online searches, and provincial liaisons from the Systemic Therapy Safety Committee of the Canadian Association of Provincial Cancer Agencies. Participants included medical oncologists, oncology pharmacists, oncology nurses, informaticians, human factors engineers, patient safety researchers, policymakers, and hospital administrators from across Canada, with participation from the United States.
The workshop took place November 28, 2012, at Toronto General Hospital, Toronto, Ontario. The first half of the workshop was designed to provide participants with context for the topic of concern. Two invited experts presented their knowledge about the use of cpoe in medical oncology and their recent research activities in the field. Their talks were followed by a presentation about the survey of Canadian cancer care providers.
The second half of the workshop consisted of two breakout sessions designed to identify the knowledge and practice gaps related to cpoe adoption and usability. During the first breakout session, participants whose organizations were currently using a cpoe system for ordering chemotherapy (“current cpoe users”) were asked to identify the challenges they experienced associated with adopting and using a cpoe system. Participants who were not using a cpoe system (“future cpoe users”) were asked to discuss the challenges associated with planning for cpoe system implementation, because many of the future cpoe users came from organizations that had started planning for implementation of a cpoe system. The researcher group was asked to discuss the knowledge gaps in chemotherapy cpoe usability.
Immediately after the first breakout session, two study investigators reviewed the verbatim transcripts of the issues presented to identify emergent categories. An open coding exercise was conducted for each of the issues, and then higher-level themes were derived based on the relationships between the categories. Themes were member-checked by presenting them to the workshop participants and revising them based on the resulting feedback.
The participants were asked to further refine and prioritize the themes to derive priority research questions. After the group discussions, a representative from each group reported the findings to the rest of the participants. Based on the research topics presented, the study investigators recorded the key research themes.
2.2. Survey Design
The survey consisted of 33 questions for current cpoe users and 7 questions for future cpoe users. The survey for current cpoe users aimed to understand the status of cpoe usage, the type of cpoe system used, the procurement and implementation process used, the challenges experienced during cpoe implementation and use, and plans related to cpoe. The survey for future cpoe users focused on understanding the methods currently used by their organizations for communicating chemotherapy orders and on eliciting their plans related to cpoe. The questions were developed based on a review of the literature on cpoe usability and adoption and on consultations with a medical oncologist, human factors engineers, and a patient safety researcher. The final survey was approved by the research ethics board at the University Health Network (uhn reb 12-0488-AE). The survey and literature review served to inform the activities and discussions of the workshop.
The survey was distributed by provincial liaisons from the Canadian Association of Provincial Cancer Agencies Systemic Therapy Safety Committee. An online survey tool was used to administer the survey (http://www.surveymonkey.com: Survey Monkey, Palo Alto, CA, U.S.A.). Because the degree of over-sight and regulation of each cancer agency varies from province to province in Canada, the liaisons were encouraged to forward the survey request to appropriate personnel at individual cancer care facilities within the organization, while ensuring that the survey would be completed by a single individual in each facility. Responses to the survey were collected from October 11, 2012, to November 9, 2012.
3. RESULTS
3.1. Workshop Findings
The first breakout session and the discussions that followed provided insights into the challenges and unmet needs that current cpoe users experienced in implementing and using their cpoe system and that future cpoe users were experiencing in planning for their cpoe adoption (summarized in Table i). The major themes were:
Need for a better understanding of current practices
System selection and procurement issues
Implementation and maintenance issues
System usability issues
Other system design–related issues
TABLE I.
Themes of challenges and unmet needs related to adoption of computerized physician order entry (cpoe), as reported by workshop participants
| Themes of challenges and unmet needs | Description |
|---|---|
| Need for better understanding of current practices and standardization | |
| Large variations seem to exist in workflows and practices around chemotherapy ordering and administration. These variations are not currently well understood. Successful adoption of cpoe requires identifying the sources of workflow variations and eliminating those variations and practices that could be standardized or that are unnecessary. Overall, there needs to be guidance for better standardization of practices in systemic therapy (for example, standardizing body). | |
| System selection and procurement issues | |
| End users of chemotherapy cpoe systems are often not involved in the cpoe system product selection process. As a result, end users often have to adapt to a system, chosen by management, that does not meet their patient care and workflow needs. There needs to be a strategy (for example, organizational governance structure) ensuring that all levels of stakeholders, including end users, are involved in the system selection and procurement process. | |
| It is often challenging to perform a comprehensive evaluation of a cpoe product to ensure that it meets all functional criteria in a meaningful manner. | |
| Implementation and maintenance issues | |
| There is a lack of guidance, specific to chemotherapy providers, for cpoe implementation planning and ongoing usage. Such guidance should include recommendations for necessary resources, workflow modifications, training needs, practice changes, and so on. This could ensure that providers reap the maximum benefits. | |
| Transitioning from a paper-based system to a computerized system requires buy-in from end users, which might take time. A hybrid of paper-based tools and a cpoe system could be in use for a certain period of time. Risks associated with the utilization of a hybrid system need to be understood and managed. | |
| cpoe system usability issues | |
| Existing cpoe systems tend to require too many steps to accomplish clinical tasks and are too complex to use. Such systems make it challenging to train health care professionals who are not necessarily inclined to technology; training relies on memory, which is one of the least effective risk mitigations. | |
| Existing cpoe systems do not always provide appropriate feedback about the system state to users (for example, no feedback about the status of an order submitted). | |
| Existing systems tend to overload screens with information, such that key information is buried in large amounts of irrelevant information. | |
| The iterative and dynamic nature of chemotherapy ordering is not well supported by existing cpoe systems. | |
| cpoe systems should be equipped with more safeguards to minimize the chance that errors will occur and to prevent errors from propagating downstream all the way to the patient. | |
| cpoe systems should provide better support for interprofessional communication and task coordination. | |
| Other issues related to cpoe system design | |
| Existing cpoe systems do not interface with many other health information technology systems in place (for example, pharmacy information system, patient scheduling system, radiation oncology systems). This lack of interoperability results in the use of error-prone workarounds and processes (for example, manual transcription). | |
| New needs concernign oral chemotherapies are not well understood, nor appropriately supported by existing cpoe systems. | |
The second breakout session led to the establishment of 3 key research themes (Table ii) that address the knowledge gaps and sources of the existing challenges.
TABLE II.
Key research themes
| Index | Research theme description |
|---|---|
| Theme A | There is a need to understand variances in workflows and practices involved in chemotherapy ordering and administration, including the extent and drivers of the variances. |
| Theme B | There is a need for a computerized physician order entry (cpoe) procurement and implementation tool specifically for chemotherapy. |
| Theme C | There is a need for better measures of the effects of cpoe implementation in medical oncology (for example, provider efficiency) to establish evidence for advancing adoption of cpoe by cancer care providers. |
3.2. Survey Findings
Twenty-four organizations from ten provinces completed the survey, including 7 representatives of provincial cancer agencies and 17 representatives of individual cancer care facilities (including community clinics, community hospitals, and academic hospitals). Thirteen organizations from six provinces were current cpoe users, and eleven organizations from six provinces were future cpoe users.
The current cpoe users reported having experienced various implementation challenges, including these:
Users learning to use the system (11 organizations)
Integration of the cpoe system with typical user workflows (9 organizations)
Persistence of paper-based tools (9 organizations)
Physician resistance to adoption of the cpoe (6 organizations)
More specifically, current cpoe users reported that they had experienced issues related to cpoe system usability (Table iii) and workflow integration. Furthermore, 11 current cpoe users reported that their organization had experienced a patient safety incident related to cpoe use. Table iv presents the factors that were reported to have contributed to those incidents.
TABLE III.
Types of usability issues experienced by current users of computerized physician order entry (cpoe) systems
| cpoe system components associated with usability issues | Organizations reporting issues (n=13) |
|---|---|
| Alerts and warnings | 9 |
| Documentation and data entry components | 8 |
| Safeguards (forcing functions and constraints) | 7 |
| Navigating within the user interface | 7 |
| Drop-down lists, menus, options | 5 |
| Information layout and organization | 5 |
| Visual cues, icons, and guidelines | 3 |
TABLE IV.
Factors contributing to patient safety incidents reported by current users of computerized physician order entry (cpoe) systems
| Contributing factor | Organizations reporting this contributing factor (n=13) |
|---|---|
| Wrong dose selection error in cpoe | 7 |
| Drug delivery scheduled for the wrong time (that is, date) in cpoe | 7 |
| Change to an order in cpoe did not get communicated | 5 |
| Failure to process an order | 5 |
| User ignoring an important alert or warning in cpoe | 5 |
| Duplicate medical order error in cpoe | 4 |
| Manual data entry error | 4 |
| Wrong drug selection error in cpoe | 3 |
| Wrong route of administration selection error in cpoe | 2 |
| Order made to a wrong patient in cpoe | 2 |
| Other (wrong regimen selected, wrong body weight) | 2 |
4. DISCUSSION
Our workshop and survey results show that the Canadian cancer care providers that have adopted a cpoe system for chemotherapy have experienced a variety of issues ranging from product selection to maintenance. In particular, poor cpoe system usability seems to be a major problem experienced by current cpoe users. The survey responses indicated that a patient safety incident related to cpoe use occurred in most organizations. Considering that some of our respondents might have been hospital administrators, who are aware mainly of major incidents, the results probably underrepresent the frequency of the problems that occurred. The workshop participants also reported various usability issues, including excessively complex and unintuitive user interfaces, lack of safeguards to minimize the potential for and impact of errors, lack of appropriate feedback, cluttered and unorganized information displays, and lack of support for interprofessional communication and task coordination.
Despite the rather broad range of usability issues experienced by the current cpoe users, little guidance is available to ensure that cancer care providers can select a cpoe system with good usability. The American Society of Clinical Oncology has published a number of articles to help cancer care providers select and implement an electronic health record (ehr) system13,15,32–34. In particular, Clinical Oncology Requirements for the EHR sets out the functional requirements, clinical data elements, and interoperability requirements that should be considered. Those requirements were adapted by the U.S. Certification Commission for Healthcare Information Technology as its oncology ehr certification criteria35. Although the foregoing documents are necessary and useful, they are not focused on the usability of cpoe systems.
With regard to generic ehr systems, a few guidelines for evaluating their usability have been developed by notable medical technology organizations such as the Healthcare Information and Management Systems Society and the U.S. National Institute of Standards and Technology36–40. However, the foregoing documents do not address the unique needs of health care providers in a specific setting.
Some efforts have been made to establish ehr usability guidance that is more specific to primary care41,42 and to pediatric patient care43, but such guidance does not seem to exist for chemotherapy. To the best of our knowledge, the only guidance document in medical oncology that discusses at least some aspects of cpoe system usability is the best practice guideline for systemic-therapy cpoe recently published by Cancer Care Ontario44.
Closely related to the usability of cpoe systems are the existing workflows for ordering, preparing, and administering chemotherapy. An important aspect of system usability is whether the workflows modelled by the cpoe system match user practice needs40. The workshop participants acknowledged that there are wide variations in workflows for chemotherapy, making integration of a cpoe system into user workflows difficult. The negative effects of unnecessary variances in workflows are only exacerbated when a cpoe system is implemented. Hence, the American Society of Clinical Oncology EHR Workgroup emphasizes the need for standardization of workflows and chemotherapy regimens before implementation of an ehr13,15. Nevertheless, insufficient understanding has been developed about the workflow variations that currently exist in chemotherapy ordering, preparation, and administration processes; about the reasons for those variations; and about which variations are required and should be supported by a cpoe system, and which are unnecessary and pose risks to patient safety. To help cancer care providers streamline their workflows, the American Society of Clinical Oncology EHR Workgroup suggests some best principles to follow13,15. However, cancer care providers are demanding more detailed and evidence-based guidance that they can readily apply.
Cancer care providers are also seeking step-by-step guidance (a roadmap) that would lead them to successful adoption of a cpoe system and allow them to reap its maximum benefits. A number of tools are available for implementing a general ehr, including guidelines from the U.S. Department of Health and Human Services, the American College of Physicians, and the American Medical Association45–47. However, compared with other domains, chemotherapy is unique in that chemotherapy treatments involve complex multidrug regimens that often encompass drugs with narrow therapeutic indices, complex dose calculations and adjustments, and unique documentation requirements14. Guidelines specifically designed for cancer care providers to address those unique challenges are therefore necessary.
5. CONCLUSIONS
Computerized physician order entry technology offers the potential to significantly improve patient safety and the efficiency and quality of chemotherapy care. However, cpoe usability factors have contributed to patient safety incidents in cancer care organization across Canada. Those incidents involved wrong dose selections, drug deliveries at the wrong time, cpoe order communication breakdowns, and other issues. Many of the difficulties experienced by Canadian cancer care providers were also related to learning how to use the cpoe system, integrating the system with typical user workflows, and dealing with consequent user resistance to adoption and with persistence of paper-based tools. To start addressing usability problems with cpoe systems, the oncology community should take the actions necessary to fill the knowledge gaps concerning the prevalence of workflow variations in chemotherapy practice across organizations. Developing that knowledge will help to inform best practices and improve the fit between clinical practice and cpoe usability design. The improved synergy between practice and cpoe design will facilitate successful adoption of cpoe systems, inform comprehensive procurement and implementation guidance, and lead to the realization of the intended patient safety benefits.
6. ACKNOWLEDGMENTS
The authors thank the Canadian Institutes of Health Research for funding this work. The authors also gratefully acknowledge the contributions of the workshop participants: Peter Yu, Palo Alto Medical Foundation, Mountain View Centre, CA, U.S.A.; Caba Gordos and Roxanne Dobish, Alberta Health Services, AB; Julie Baisley, Allan Blair Cancer Centre, SK; Helen Anderson, Karen Janes, and Susan Walliser, BC Cancer Agency, BC; Larry Broadfield, Cancer Care Nova Scotia, NS; Esther Green, Cancer Care Ontario, ON; Dhali Dhaliwal and Venetia Bourrier, CancerCare Manitoba, MB; Mélanie Mornea, Direction Québécoise du cancer, QC (observer); Sharon Smith, Eastern Health, NL; Marc-Alexandre Vincent, Hôpital de la Cité-de-la-Santé, QC (observer); Eshwar Kumar, New Brunswick Cancer Network, NB; Liz Dobbin and Philip Champion, PEI Cancer Treatment Centre, PE; Janice Kolbinson, Saskatchewan Cancer Agency, SK; Flay Charbonneau, Sunnybrook Health Sciences Centre, ON; and Rachel White and Deborah Chan, University Health Network, ON.
The authors also thank Winston De Armas and Aarti Mathur for assistance with the study.
7. CONFLICT OF INTEREST DISCLOSURES
JJ, ST, and JAC are employed by Healthcare Human Factors at the University Health Network, which has a consulting relationship with Elekta AB.
8. REFERENCES
- 1.Kim GR, Chen AR, Arceci RJ, et al. Error reduction in pediatric chemotherapy: computerized order entry and failure modes and effects analysis. Arch Pediatr Adolesc Med. 2006;160:495–8. doi: 10.1001/archpedi.160.5.495. [DOI] [PubMed] [Google Scholar]
- 2.Voeffray M, Pannatier A, Stupp R, Fucina N, Leyvraz S, Wasserfallen JB. Effect of computerisation on the quality and safety of chemotherapy prescription. Qual Saf Health Care. 2006;15:418–21. doi: 10.1136/qshc.2005.016808. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Huertas Fernández MJ, Baena–Cañada JM, Martínez Bautista MJ, Arriola Arellano E, García Palacios MV. Impact of computerised chemotherapy prescriptions on the prevention of medication errors. Clin Transl Oncol. 2006;8:821–5. doi: 10.1007/s12094-006-0138-1. [DOI] [PubMed] [Google Scholar]
- 4.Dubeshter B, Walsh CJ, Altobelli K, Loughner J, Angel C. Experience with computerized chemotherapy order entry. J Oncol Pract. 2006;2:49–52. doi: 10.1200/JOP.2.2.49. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Collins CM, Elsaid KA. Using an enhanced oral chemotherapy computerized provider order entry system to reduce prescribing errors and improve safety. Int J Qual Health Care. 2011;23:36–43. doi: 10.1093/intqhc/mzq066. [DOI] [PubMed] [Google Scholar]
- 6.Chen AR, Lehmann CU. Computerized provider order entry in pediatric oncology: design, implementation, and outcomes. J Oncol Pract. 2011;7:218–22. doi: 10.1200/JOP.2011.000344. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Cheng CH, Chou CJ, Wang PC, Lin HY, Kao CL, Su CT. Applying hfmea to prevent chemotherapy errors. J Med Syst. 2012;36:1543–51. doi: 10.1007/s10916-010-9616-7. [DOI] [PubMed] [Google Scholar]
- 8.Chan J, Shojania KG, Easty AC, Etchells EE. Does user-centred design affect the efficiency, usability and safety of cpoe order sets? J Am Med Inform Assoc. 2011;18:276–81. doi: 10.1136/amiajnl-2010-000026. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Jaspers MW, Peute LW, Lauteslager A, Bakker PJ. Pre–post evaluation of physicians’ satisfaction with a redesigned electronic medical record system. Stud Health Technol Inform. 2008;136:303–8. [PubMed] [Google Scholar]
- 10.Boonstra A, Broekhuis M. Barriers to the acceptance of electronic medical records by physicians from systematic review to taxonomy and interventions. BMC Health Serv Res. 2010;10:231. doi: 10.1186/1472-6963-10-231. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Scanlon M. Computer physician order entry and the real world: we’re only humans. Jt Comm J Qual Saf. 2004;30:342–6. doi: 10.1016/s1549-3741(04)30039-0. [DOI] [PubMed] [Google Scholar]
- 12.Koppel R, Metlay JP, Cohen A, et al. Role of computerized physician order entry systems in facilitating medication errors. JAMA. 2005;293:1197–203. doi: 10.1001/jama.293.10.1197. [DOI] [PubMed] [Google Scholar]
- 13.Shulman LN, Miller RS, Ambinder EP, Yu PP, Cox JV. Principles of safe practice using an oncology ehr system for chemotherapy ordering, preparation, and administration, part 1 of 2. J Oncol Pract. 2008;4:203–6. doi: 10.1200/JOP.0847501. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Hoffman JM, Baker DK, Howard SC, Laver JH, Shenep JL. Safe and successful implementation of cpoe for chemotherapy at a children’s cancer center. J Natl Compr Canc Netw. 2011;9(suppl 3):S36–50. doi: 10.6004/jnccn.2011.0131. [DOI] [PubMed] [Google Scholar]
- 15.Shulman LN, Miller RS, Ambinder EP, Yu PP, Cox JV. Principles of safe practice using an oncology ehr system for chemotherapy ordering, preparation, and administration, part 2 of 2. J Oncol Pract. 2008;4:254–7. doi: 10.1200/JOP.0857501. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Ash JS, Fournier L, Stavri PZ, Dykstra R. Principles for a successful computerized physician order entry implementation. AMIA Annu Symp Proc. 2003;2003:36–40. [PMC free article] [PubMed] [Google Scholar]
- 17.Horsky J, Kaufman DR, Patel VL. When you come to a fork in the road, take it: strategy selection in order entry. AMIA Annu Symp Proc. 2005;2005:350–4. [PMC free article] [PubMed] [Google Scholar]
- 18.Granlien MF, Hertzum M, Gudmundsen J. The gap between actual and mandated use of an electronic medication record three years after deployment. Stud Health Technol Inform. 2008;136:419–24. [PubMed] [Google Scholar]
- 19.Sklarin NT, Granovsky S, O’Reilly EM, Zelenetz AD. Electronic chemotherapy order entry: a major cancer center’s implementation. J Oncol Pract. 2011;7:213–18. doi: 10.1200/JOP.2011.000266. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Schoeffel R. The concept of product usability: a standard to help manufacturers to help consumers. ISO Bulletin. 2003. Mar, pp. 5–7.
- 21.Abramson EL, Patel V, Malhotra S, et al. Physician experiences transitioning between an older versus newer electronic health record for electronic prescribing. Int J Med Inform. 2012;81:539–48. doi: 10.1016/j.ijmedinf.2012.02.010. [DOI] [PubMed] [Google Scholar]
- 22.Chan J, Shojania KG, Easty AC, Etchells EE. Usability evaluation of order sets in a computerised provider order entry system. BMJ Qual Saf. 2011;20:932–40. doi: 10.1136/bmjqs.2010.050021. [DOI] [PubMed] [Google Scholar]
- 23.Ghahramani N, Lendel I, Haque R, Sawruk K. User satisfaction with computerized order entry system and its effect on workplace level of stress. J Med Syst. 2009;33:199–205. doi: 10.1007/s10916-008-9180-6. [DOI] [PubMed] [Google Scholar]
- 24.Maslove DM, Rizk N, Lowe HJ. Computerized physician order entry in the critical care environment: a review of current literature. J Intensive Care Med. 26:165–71. doi: 10.1177/0885066610387984. 011. [DOI] [PubMed] [Google Scholar]
- 25.Schoville RR. Work-arounds and artifacts during transition to a computer physician order entry: what they are and what they mean. J Nurs Care Qual. 2009;24:316–24. doi: 10.1097/NCQ.0b013e3181ac8378. [DOI] [PubMed] [Google Scholar]
- 26.Corrao NJ, Robinson AG, Swiernik MA, Naeim A. Importance of testing for usability when selecting and implementing an electronic health or medical record system. J Oncol Pract. 2010;6:120–4. doi: 10.1200/JOP.200017. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Khajouei R, Peek N, Wierenga PC, Kersten MJ, Jaspers MW. Effect of predefined order sets and usability problems on efficiency of computerized medication ordering. Int J Med Inform. 2010;79:690–8. doi: 10.1016/j.ijmedinf.2010.08.001. [DOI] [PubMed] [Google Scholar]
- 28.Levy MA, Giuse DA, Eck C, et al. Integrated information systems for electronic chemotherapy medication administration. J Oncol Pract. 2011;7:226–30. doi: 10.1200/JOP.2011.000259. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Harshberger CA, Harper AJ, Carro GW, et al. Outcomes of computerized physician order entry in an electronic health record after implementation in an outpatient oncology setting. J Oncol Pract. 2011;7:233–7. doi: 10.1200/JOP.2011.000261. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Brockstein B, Hensing T, Carro GW, et al. Effect of an electronic health record on the culture of an outpatient medical oncology practice in a four-hospital integrated health care system: 5-year experience. J Oncol Pract. 2011;7:e20–4. doi: 10.1200/JOP.2011.000260. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Pirnejad H, Niazkhani Z, Aarts J, Bal R. What makes an information system more preferable for clinicians? a qualitative comparison of two systems. Stud Health Technol Inform. 2011;169:392–6. [PubMed] [Google Scholar]
- 32.American Society of Clinical Oncology (asco) Oncology EHR Vendor Directory 4th Quarter 2009. Alexandria, VA: ASCO; 2009. [Google Scholar]
- 33.American Society of Clinical Oncology (asco) Clinical Oncology Requirements for the EHR. Alexandria, VA: ASCO; 2009. [Google Scholar]
- 34.American Society of Clinical Oncology (asco) Selecting an electronic health record for your practice. J Oncol Pract. 2007;3:318–20. doi: 10.1200/JOP.0767501. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Certification Commission for Healthcare Information Technology (cchit) Certified 2011 Oncology EHR Certification Criteria. Chicago, IL: CCHIT; 2011. [Google Scholar]
- 36.Schumacher R, Lowry S. Customized Common Industry Format Template for Electronic Health Record Usability Testing. Gaithersburg, MD: National Institute of Standards and Technology; 2010. [Google Scholar]
- 37.Schumacher R, Lowry S. NIST Guide to the Processes Approach for Improving the Usability of Electronic Health Records. Gaithersburg, MD: National Institute of Standards and Technology; 2010. [Google Scholar]
- 38.Lowry S, Quinn MT, Ramaiah M, et al. Technical Evaluation, Testing, and Validation of the Usability of Electronic Health Records. Gaithersburg, MD: National Institute of Standards and Technology; 2012. [Google Scholar]
- 39.Belden JL, Grayson R, Barnes J. Defining and Testing EMR Usability: Principles and Proposed Methods of EMR Usability Evaluation and Rating. Chicago, IL: Healthcare Information and Management Systems Society; 2009. [Google Scholar]
- 40.Arellano P, Bochinski J, Boone E, et al. Selecting an EHR for Your Practice: Evaluating Usability. Chicago, IL: Healthcare Information and Management Systems Society; 2010. [Google Scholar]
- 41.Johnson CM, Johnston D, Crowle PK, et al. EHR Usability Toolkit: A Background Report on Usability and Electronic Health Records. Rockville, MD: Agency for Healthcare Research and Quality; 2011. [Google Scholar]
- 42.Armijo D, McDonnell C, Werner K. Electronic Health Record Usability: Evaluation and Use Case Framework. Rockville, MD: Agency for Healthcare Research and Quality; 2009. [Google Scholar]
- 43.Lowry S, Quinn MT, Ramaiah M, et al. A Human Factors Guide to Enhance EHR Usability of Critical User Interactions when Supporting Pediatric Patient Care. Gaithersburg, MD: National Institute of Standards and Technology; 2012. [DOI] [Google Scholar]
- 44.Kukreti V, Cheung A, Theriault M, et al. Computerized Prescriber Order Entry for Systemic Treatment: Best Practice Guideline. Toronto, ON: Cancer Care Ontario; 2012. [Google Scholar]
- 45.United States, Department of Health and Human Services . HealthITgov > For Providers and Professionals > How to Implement EHRs [Web resource] Washington, DC: n.d. [Available at: http://www.healthit.gov/providers-professionals/ehr-implementation-steps; cited August 21, 2013] [Google Scholar]
- 46.American Medical Association (ama) AMA > Advocacy > Advocacy Topics > Health Information Technology [Web resource] Chicago, IL: AMA; n.d. [Available at: http://www.ama-assn.org/ama/pub/advocacy/topics/health-informationtechnology.page; cited August 21, 2013] [Google Scholar]
- 47.American College of Physicians . Home > Running a Practice > Health Information Technology > Electronic Health Records > Roadmap [Web resource] Philadelphia, PA: [Available at: http://www.acponline.org/running_practice/technology/ehr/roadmap/; cited August 21, 2013] [Google Scholar]