Abstract
Background:
Diagnosis requires that clinicians communicate and share patient information in an efficient manner. Advances in electronic health records (EHR) and health information technologies have created both challenges and opportunities for such communication.
Methods:
We conducted a multi-method, focused ethnographic study of physicians on general medicine inpatient units in two teaching hospitals. Physician teams were observed during and after morning rounds to understand workflow, data sharing and communication during diagnosis. To validate findings, interviews and focus groups were conducted with physicians. Field notes and interview/focus group transcripts were reviewed and themes identified using content analysis.
Results:
Existing communication technologies and EHR-based data sharing processes were perceived as barriers to diagnosis. In particular, reliance on paging systems and lack of face-to-face communication among clinicians created obstacles to sustained thinking and discussion of diagnostic decision-making. Further, the EHR created data overload and data fragmentation, making integration for diagnosis difficult. To improve diagnosis, physicians recommended replacing pagers with two-way communication devices, restructuring the EHR to facilitate access to key information, and improving training on EHR systems.
Conclusions:
As advances in health information technology evolve, challenges in the way clinicians share information during the diagnostic process will rise. To improve diagnosis, changes to both the technology and the way in which we use it may be necessary.
INTRODUCTION
The process of making a medical diagnosis is complex, time consuming and includes the potential for error at many stages [1, 2, 3]. To arrive at the correct diagnosis, clinicians must gather vital data; communicate with, and share this information among several different providers; and integrate and synthesize large amounts of information in a short time. In general, this process is iterative and happens multiple times for one patient with the addition of new test results or data influencing the list of potential diagnoses. The evolution of diagnosis can be particularly challenging in an academic hospital, where physicians care for several complex patients simultaneously and may have never interacted with these patients prior to their admission. Furthermore, many of these physicians are in training and thus less facile with patient care or how best to use technologies to facilitate diagnosis in the hospital.
Historically, the information used for formulating a diagnosis was almost exclusively gathered at the bedside and through face-to-face communication. However, health information in in hospitals today is gathered primarily through a series of interactions with electronic data systems and communication exchanges with various providers. The widespread use of electronic health records (EHRs) in hospitals – up from 9% in 2008 to 96% in 2015 [4] -- along with the increase in laboratory and medical imaging information systems, computerized provider order entry systems, and electronic patient portals have fundamentally changed the way clinicians communicate and process patient data. Today, diagnosticians must filter, integrate, and interpret information from diverse sources. Some studies suggest the EHR may facilitate this process through better information sharing and timely access to data [5, 6, 7, 8]. However, studies also raise concerns ranging from fragmentation of data, information siloes, and data overload [9, 10, 11, 5]. For example, some studies have shown that EHRs influence physician workflow, workload, and communication practices [12]; while others suggest that more computer time, means less patient time [13] leading physicians to be perceived as less effective communicators [14]. Furthermore, we found a paucity of research examining these issues in an academic hospital setting. These gaps suggest limited real world understanding of how clinicians communicate, access data, and formulate diagnosis in a new information technology (IT) environment.
Therefore, we performed an ethnographic study to understand communication and data sharing between providers in inpatient units utilizing an EHR during diagnostic evaluations. We sought the opinions and perspectives of physicians at two affiliated teaching hospitals to understand diagnosis-related challenges and strategies for improvement.
MATERIALS AND METHODS
Study Design
We chose a focused ethnographic approach to help elucidate perspectives held by individuals operating in a specific context, namely healthcare [15, 16]. Embedding ourselves within the inpatient setting allowed us to capture diagnoses in its natural environment and in a way that is currently lacking in the literature [17].
Setting, Participants and Recruitment
We selected teams on teaching services based on their schedules on general medicine units, and asked attending physicians to allow us to observe and shadow during and after morning rounds when provider communication and EHR use were heavy. Following these observations, we invited members of the team to participate in follow-up focus groups or one-on-one interviews. Teams consisted of a medical attending (a senior-level physician that oversees the team), one senior resident (a second or third-year trainee), two interns (a trainee in their first year), and between one to four medical students. Teams were followed for a one-week period, Monday through Friday, so that a variety of work routines, including call days (i.e., admitting new patients to the hospital) and non-call days (i.e., days without admitting activities), could be observed.
Data Collection
Observations
A multi-disciplinary group of researchers including clinicians (e.g., attending physicians, nurses) and non-clinicians (e.g., qualitative researchers, social scientists) conducted observations of teaching teams during morning rounds (0800 to 1200) and in the afternoons following rounds (1300 to 1700). In most observations, two researchers were paired, one with a clinical background and one non-clinician, to observe one physician team at a time. During afternoon observations on non-call days, residents and interns spent most of their time together in their team room; thus, two researchers could observe communication and information sharing among the team members. During on-call days, researchers split and individually followed the senior resident and interns as they admitted new patients or cared for those already hospitalized. Attendings were not usually present during afternoon observations.
Data were collected using unstructured field notes [18]; however, to ensure that each member of the observation team collected the same type of information, observers were provided with a data template and asked to record information and behaviors related to information retrieval and sharing. For our study, information retrieval included any modality where data regarding patient care was resourced (e.g., EHR, imaging) while information sharing included verbal communication and activities writing H+Ps, progress notes and discharge summaries, or entering clinical orders.
Focus Groups & Interviews
At the end of weekly observations, we conducted focus groups with the senior resident and interns and one-on-one interviews with the attendings from each team. We interviewed attendings separately to ensure power differentials did not influence discussions. Experienced qualitative methodologists (JF, MH, MQ) and the study PI (VC) used open-ended, semi-structured questions to guide and facilitate discussions. The focus groups (average length=52 minutes, range= 44–61 minutes) and interviews (average length=52 minutes, range= 32–79 minutes) were held in hospital meeting rooms. These sessions were designed to augment observations and to validate the findings and views of the observers [19]. Interviews and focus groups were recorded and transcribed for analysis. As a token of appreciation, a $30 gift card was offered for participating in the interview/focus group discussions. All data were gathered between January 2016 and November 2016.
Data Analysis
Observations
Observation notes from rounding and afternoon shadowing were read and analyzed by two members of the study team (JF, MQ) expertise using a process of immersion and crystallization to identify patterns and themes related to communication and data sharing [20].
We specifically chose notes from afternoon shadowing sessions only to examine the flow of activities related to communication and interaction with the EHR. We focused on afternoon sessions because: (a) most patient care activities happen after rounds in the afternoon; (b) these periods tend to be largely unstructured (in contrast to morning rounds) and centered on communicating information within and across care teams; and (c) data retrieval, ordering, and interactions with the EHR were greatest during this time. Therefore, observation notes from afternoon sessions were reviewed to identify: 1) communication modes used by physicians (e.g., face-to-face communication, pager, telephone call); 2) type of interactions with the EHR (e.g., entering notes, ordering tests); 3) roles of participants engaged in communication (e.g. resident, intern); and 4) the content of the information communicated. To understand data flow and sources of data retrieval, we used notes from afternoon observations to construct a diagram depicting the activity flow for a “typical” afternoon for one intern, based on the average number of patients managed by the team across all afternoon observations. We chose intern activities to diagram because they performed the bulk of data-gathering and patient management tasks. For the selected afternoon, data on each element enumerated above, along with corresponding field notes, were entered into Excel, and a diagram was constructed from that data.
Focus Groups and Interviews
Three members of the research team (JF, MQ, SW) reviewed a sample of the interview and focus group transcripts, met, and agreed on a preliminary coding scheme. Two qualitative researchers (MQ, JF) then independently read all transcripts and conducted a focused content analysis to identify clinician perspectives on challenges and suggestions for improvement in communication and data sharing during the process of diagnosis. Preliminary codes were further refined during the analysis and additional inductive codes emerged [21]. Data were aggregated and organized under each code and common themes with exemplary quotes were identified. Researchers met several times to verify coding, discuss any discrepancies, and reach agreement.
Ethical and Regulatory Oversight
This study was approved by the Institutional Review Boards at the University of Michigan Health System (HUM-00106657) and the VA Ann Arbor Healthcare System (1-2016-010040).
RESULTS
Data from a total of eight inpatient teaching teams (comprised of 8 attendings, 8 senior residents, 15 interns and 22 medical students) representing a total of 231 hours of observation was used for this analysis. Correspondingly, eight one-on-one interviews were conducted with the participating attendings and eight focus groups were held with the resident/intern teams (Figure 1).
Overall, physicians felt that the constraints of existing communication and EHR technologies made integration and interpretation of data for diagnosis challenging. Further complicating the diagnostic process, these interactions occurred within a system of fragmented workflow, both in time and space, during which physicians managed multiple patients simultaneously (Figure 2). For example, across the afternoons observed, physician teams managed an average of 9 patients (range 6 to 13) during a single shift. During each shift, physicians routinely switched from communicating about one patient to another using various modes including phones, pagers, face-to-face communication, and the EHR. Physicians spent the bulk of their time with members of their team (residents, interns, and medical students), working on computers. Phones and pagers were used to communicate with other clinicians to coordinate treatment and discharge plans. Physicians (especially interns) interacted heavily with the EHR to write progress notes, complete discharge paperwork, enter consults, order tests and imaging, get results, and look at these results themselves and with other team members. Face-to-face communication between clinicians most commonly consisted of unplanned conversations among interns, the resident, or medical students related to patient care tasks or activities.
Common themes from interviews, focus groups, and observations are listed in Table 1 and summarized below. The following factors related to clinician communication and the EHR emerged as presenting the greatest challenges to diagnosis: 1) use of one-way communication tools such as pagers; 2) a lack of regular face-to-face communication regarding diagnosis; and 3) fragmentation of information and data overload in the EHR.
Table 1:
Domains | Challenges | Suggested Improvements |
---|---|---|
Communication |
One-way communication tools. Using pagers, the EHR, or other one-way modes is inefficient, causes delays, and doesn’t allow participants to convey context or have a discussion Constant pages. The large number of pages interrupts thinking required for diagnosis and patient care Non-prioritized pages. Frequent pages related to routine patient care can lead to ‘alert fatigue’ Inadequate face-to-face communication. Limited opportunity to discuss and think through diagnosis, including with nurses and specialty consultants |
Replace current pagers. Instead use only two-way communication devices such as phones or pagers that have text-back features ‘Batch’ pages and emails. For messages related to routine care, messages should be grouped together and sent at one time Prioritization of critical pages. Implement a new system that allows physicians to triage most critical pages and read others at a later time More discussion and conversation is needed. Clinicians need to make the time to talk face-to-face with each other about diagnosis |
EHR Data Sharing |
Fragmentation of Information. Gathering and receiving diagnostic information was fragmented Data overload. Physicians face information and data overload in EHR Not designed for diagnosis. EHR does not support thinking about diagnosis Limited access. EHR access may be limited or not available during morning rounds when diagnosis is discussed Lack of knowledge and efficacy. Physicians may not be aware of how to use the EHR to facilitate diagnosis. |
Restructure EHR. Restructure to support diagnosis and decision making Integrate data. Improve the integration of patient data (labs, imaging, test results, notes) to make it easier to see the full clinical picture and display in more clinically relevant ways Reduce data overload. Consider eliminating auto-populating options and creating alerts or notifications of new test results or updates Streamline. Improve process for ordering, collecting and expediting labs/test results Use laptops, computers on wheels, tablets, phones, or other technologies to access to EHR during morning rounds Physician trainings. Are needed on how to better use EHR to improve diagnosis |
One-Way Communication Tools
Physicians frequently used one-way communication tools, including pagers and the EHR, to discuss patient care plans. They reported that these tools were inefficient and did not support the rapid exchange and synthesis of information needed for diagnosis. The inefficiency of pagers was universally cited as a challenge.
“When you talk to somebody…[it] takes 20 seconds; but paging takes more time. They need to find a computer…to read your page and then they have to do whatever the page says and then they need to go find a computer in order to page you back.” (Intern W6H)
Physicians also reported that ‘one-way’ forms of communication do not allow back-and-forth discussions with other clinicians, an aspect felt critical to synthesize accurate and timely diagnosis. In fact, physicians indicated this could be an important source of errors.
“It’s hard with electronic records, people just post notes and then we don’t always talk … they will post a note and then like something may not make sense. You send them a page, ‘hey, what about this?’ And they send you a page back… it’s not even a full dialogue. So, making sure that we still …. talk in person or discuss things is important.” (Resident W1)
As a solution, some physicians suggested implementing a 2-way communication system that allows the user to text back responses immediately. Others said physicians and members of the care team should carry cell phones to talk directly with other clinicians, or find time to meet and discuss clinical care in person.
“My number one priority…I would really like to see the beepers go away and for us to have a much better communication system. I think it is a terribly outmoded technology that is causing us a lot of difficulties.” (Attending W3)
Physicians stressed that any new system of asynchronous communication should allow users to ‘triage’ messages by classifying them as low or high priority, in order to reduce distractions and cognitive load. They also suggested that low-priority communication be funneled or batched together to be handled later so as to not interrupt the diagnostic process.
“So when you get a page, 99% of the pages you get start out with ‘routine’ and when the nurses are putting in pages they can say it is ‘routine’, ‘urgent’ or ‘ASAP.’ Like I said about 99% are ‘routine’ but it’s not like it does anything different…it still beeps, I still have to look at it….it still alerts you and distracts you from whatever you’re doing.” (Attending W5F)
Inadequate Face-to-Face Communication
Creating more opportunities for face-to-face communication was reported as an important means to facilitate diagnosis. For example, we observed (and residents and interns reported), that they appreciated off-the-cuff conversations related to diagnosis in team rooms but that they often did not have as much time as they would like for these types of discussions.
“It’s always great to bounce ideas off of [others], you know especially more experienced [team] members and also just, in general, having people to talk through things.” (Resident W1)
“If I have questions, comments, or want to discuss [things I may not understand], I will turn to my left and say, hey, what do you think about this? Is it already missing this diagnosis? Are we missing a lab we should be ordering?” (Intern W6D)
Supportive, sustained in-person dialog with the team was lacking during our observations. During afternoon sessions, we observed only two instances of prolonged discussion of diagnosis that included the interns, resident, and attending or primary teams with consultants. While physicians reported that they sometimes use morning rounds as a time to discuss diagnosis, they also acknowledged that there was not enough time or opportunity during rounds for in-depth discussions. Rather, we observed that rounds were often hurried or focused on treatment and management rather than diagnosis.
“I think [having a discussion is better] that way you can discuss things, like with your consultants or your nurses … You get to understand their thought process rather than just like ‘here are my recommendations’. You get a better understanding why they think what they think and what their diagnostic evaluation is…” (Resident W1)
Fragmentation of Information and Data Overload in EHR
Physicians described several challenges to use of the EHR. The EHR presented physician notes, imaging and laboratory results, or other diagnostic tests in a fragmented fashion that required additional time and cognitive work to identify, sort, integrate and apply to the patient at hand. For example, the EHR often limited views to an array or list of pieces of data (e.g., lab test results, imaging) one at a time, rather than an integrated tabulation to facilitate diagnosis. As a result, some data pieces could be missed or ignored.
“So if you want to look at the x-ray, you have to go into the x-ray results, you can’t be looking at that and be looking at the labs …you can only look at one thing at a time” (Attending W5F)
“Learning (with the EHR) becomes very scattered I think because you have all this information and you basically go look up one little piece on treatment or diagnosis and you never like really sit there and digest the whole thing.” (Intern W5S)
Additionally, one physician stated that the EHR’s auto-populating function promoted data overload leading to clinician fatigue and frustration. Physicians recognized that while auto-populating data or using templates may initially save time, it made it difficult to recognize which data are relevant on a particular day or find data needed for diagnosis.
“You auto populate tons and tons of crap. And, sure, that looks like you know a lot about your patients but it’s not really reflective of what’s going on that day or what the heck’s been going on in the past.” (Attending W5F)
Along with suggestions related to replacing or improving the current paging system and increasing face-to-face discussion, several improvements to the EHR were proposed. These included: rethinking or disabling auto-populate features; streamlining and integrating health information such as organizing data by relevancy, temporality, or disease condition; and organizing diagnostic information in more clinically relevant ways. For example, one physician suggested that a template in which interns would write an assessment or plan immediately following a history and physical on a new patient would expedite thinking about diagnosis; the plan could then be pulled into the note to further refine and evolve diagnostic thinking. Physicians also said they needed a better way to access electronic data during morning rounds. One physician found that accessing the EHR and laboratory reports through their cell phone was helpful.
“One thing I actually found to be helpful is using … my phone. That was a nice way to pull it up… and to be able to pull it up on the fly. I mean I can’t do everything the same as on the computer but I think it’s ….a valuable tool.” (Attending W6H)
This attending physician added that most EHRs are capable of helping to facilitate diagnosis, but that physicians need specialized training on how to use these systems more efficiently.
“I found you can customize it and I don’t think that many people are knowledgeable enough to customize it … there are all these things you can do with it that most people don’t know is there.” (Attending W6H)
DISCUSSION
Diagnostic error remains one of the most critical problems facing healthcare. Changes in the way clinicians communicate and share information have far-reaching implications on the diagnostic process. Our study found that frontline physicians in two academic medical centers viewed current information storing, sorting and sharing processes as contributors to diagnostic inefficiency and a potential source of error. The main challenges reported included the use of one-way communication tools, which created interruptions and did not facilitate in-depth discussion; 2) a lack of face-to-face clinician discussion about diagnosis; and 3) an EHR that was perceived to be difficult to use, included fragmented information, and caused data overload. Suggestions for improvement included 1) replacing pagers with two-way communication tools;2) restructuring the current EHR to better facilitate diagnosis; 3) improve access to electronic patient data; and 4) enhance physician training on the EHR.
The landmark 2016 report by the National Academy on Medicine, Improving Diagnosis in Health Care, concluded that while health IT has great potential to improve the quality of health care overall, it often creates additional risks and barriers that are particularly concerning to the process of diagnosis. In particular, poor EHR design can lead to inefficiency and breakdowns in communication and information flow [22]. Our findings echo both the report conclusions and additional literature highlighting diagnostic challenges associated with use of EHRs, including auto-populate features and templates that can “bury” data needed for diagnosis. Additionally, burdensome interfaces that detract from critical thinking time and a general lack of innovations built into EHR systems may hinder diagnostic evaluations [23]. Our research also supports recent calls for a fundamental redesign of the EHR by focusing on appropriate organization and display of data [24], the continued importance of in-person conversation between clinicians [25], and other information technology-based interventions that facilitate better information management and communication during the diagnostic process [26, 6].
However, our paper also contributes to this new and emerging research in important ways. First, our study illustrates the gravity and challenges associated with current information sharing processes in the hospital. As viewed through the lens of residents and trainees, lack of efficient communication and EHR complexities clearly impair diagnostic reasoning. Second, we offer new and potentially innovative solutions that come from physicians directly grappling with data overload and fragmentation. Although some of these solutions are technical (e.g., use of two-way communication tools), it is pertinent to note that a call for social interventions that include more face-to-face time for diagnosis also emerged during our study. Finally, our study provides ways EHR vendors may consider improving their software platforms, including changes to the user interface as well as training programs to improve diagnostic decisions.
We believe that recent calls for applying a human factors and industrial engineering process approach in health IT design [27, 28] are both timely and appropriate. More broadly, findings presented in this paper support our related work previously published in this journal calling for comprehensive approaches to improving diagnosis that address both system-level factors (e.g., hospital paging and EHR systems) and cognitive-level problems (e.g., faulty data processing) [29] in conjunction with one another, rather than in silos.
Our study has some limitations. First, it is possible that the process of being observed altered physician behavior and dialogue. However, we believe that a potential Hawthorne effect was mitigated by use of focused ethnography and embedding ourselves within teams for protracted periods [30]. Second, study team members did not accompany physicians into patient rooms, as the focus of the study was on clinician communication and behaviors. It is possible that valuable information was missed during patient examination and consultation. Finally, observations and focus groups occurred with residents from the same residency training program, which may limit the generalizability of the results.
Our study also has notable strengths. First, we utilized an intensive approach for data collection spanning over 200 hours of observation, followed by in-depth focus groups and interviews. Next, inclusion of observers from medical, cognitive and social science realms allowed for unique insights, interpretation and theme generation. Finally, we included the perceptions and views of frontline clinicians who are most involved in diagnosis, which is lacking in current literature. Incorporating their perceptions and suggestions for improvement may result in better informed, and thus more effective, interventions to prevent errors.
Diagnosis requires efficient processes for data sharing, integration and interpretation. Once viewed as a panacea, communication and data technology tools now present distinct barriers to this quest. To combat the growing problem of diagnostic errors, old technologies must be reconsidered (i.e. face-to-face communication in lieu of pagers) and newer technologies (i.e. EHR) must be updated to promote usable and accessible information. Only with these measures can the full promise of modern EHRs and technology in preventing diagnostic errors be realized.
Acknowledgments
Funding: This project was supported by grant number P30HS024385 from the Agency for Healthcare Research and Quality. The funding source played no role in study design, data acquisition, analysis or decision to report these data. Dr. Chopra is supported by a career development award from the Agency of Healthcare Research and Quality (1-K08-HS022835–01). Dr. Krein is supported by a VA Health Services Research and Development Research Career Scientist Award (RCS 11–222). Dr. Singh is partially supported by Houston VA HSR&D Center for Innovations in Quality, Effectiveness and Safety (CIN 13–413). The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality or the Department of Veterans Affairs.
Footnotes
Conflicts of Interest: None declared for all coauthors
References:
- 1.Graber ML, Franklin N, Gordon R. Diagnostic error in internal medicine. Arch Intern Med 2005;165(13):1493–9. [DOI] [PubMed] [Google Scholar]
- 2.Singh H, Meyer AN, Thomas EJ. The frequency of diagnostic errors in outpatient care: estimations from three large observational studies involving US adult populations. BMJ Qual Saf 2014:bmjqs-2013-002627. [DOI] [PMC free article] [PubMed]
- 3.Graber ML. The incidence of diagnostic error in medicine. BMJ Qual Saf 2013:bmjqs-2012-001615. [DOI] [PMC free article] [PubMed]
- 4.Henry J., Pylypchuk Y., Searcy T. & Patel V. (May 2016). Adoption of Electronic Health Record Systems among U.S. Non-Federal Acute Care Hospitals: 2008–2015 ONC Data Brief, no.35. Office of the National Coordinator for Health Information Technology: Washington DC. [Google Scholar]
- 5.Graber ML, Byrne C, Johnston D. The impact of electronic health records on diagnosis. Diagnosis 2017;4(4):211–23. [DOI] [PubMed] [Google Scholar]
- 6.Singh H, Naik AD, Rao R, Petersen LA. Reducing diagnostic errors through effective communication: harnessing the power of information technology. J Gen Intern Med 2008;23(4):489–94. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Sittig DF, Singh H. Electronic health records and national patient-safety goals. Mass Med Soc; 2012. [DOI] [PMC free article] [PubMed]
- 8.Bates DW, Gawande AA. Improving safety with information technology. N Engl J Med 2003;348(25):2526–34. [DOI] [PubMed] [Google Scholar]
- 9.Verghese A, Shah NH, Harrington RA. What This Computer Needs Is a Physician: Humanism and Artificial Intelligence. JAMA 2018. [DOI] [PubMed]
- 10.Perrem LM, Fanshawe TR, Sharif F, Plüddemann A, O’Neill MB. A national physician survey of diagnostic error in paediatrics. Eur J Pediatr 2016;175(10):1387–92. [DOI] [PubMed] [Google Scholar]
- 11.Palojoki S, Pajunen T, Saranto K, Lehtonen L. Electronic health record-related safety concerns: a cross-sectional survey of electronic health record users. JMIR Med Inform 2016;4(2). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Asan O, Chiou E, Montague E. Quantitative ethnographic study of physician workflow and interactions with electronic health record systems. Int J Ind Ergon 2015. September 1;49:124–30 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Asan O, D. Smith P, Montague E. More screen time, less face time–implications for EHR design. J Eval Clin Pract 2014. December;20(6):896–901. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Street RL Jr, Liu L, Farber NJ, Chen Y, Calvitti A, Zuest D, et al. Provider interaction with the electronic health record: the effects on patient-centered communication in medical encounters. Patient Educ Couns 2014. September 1;96(3):315–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Higginbottom G, Pillay JJ, Boadu NY. Guidance on performing focused ethnographies with an emphasis on healthcare research. Qual Rep 2013;18(9):1–6. [Google Scholar]
- 16.Savage J. Participative observation: Standing in the shoes of others? Qual Health Res 2000;10(3):324–39. [DOI] [PubMed] [Google Scholar]
- 17.Chopra V, Harrod M, Winter S, Forman J, Quinn M, Krein S, et al. Focused Ethnography of Diagnosis in Academic Medical Centers. J Hosp Med 2018;13:E1–E5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Mulhall A. In the field: notes on observation in qualitative research. J Adv Nurs 2003;41(3):306–13. [DOI] [PubMed] [Google Scholar]
- 19.Johnson M, O’Hara R, Hirst E, Weyman A, Turner J, Mason S, et al. Multiple triangulation and collaborative research using qualitative methods to explore decision making in pre-hospital emergency care. BMC Med Res Methodol 2017;17(1):11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Borkan J. Immersion/crystallization. Doing qualitative research 1999;2:179–94. [Google Scholar]
- 21.Patton M. Qualitative Research and Evaluation Methods, 3rd edn Sage Thousand Oaks, California: 2002. [Google Scholar]
- 22.National Academies of Sciences, Engineering, and Medicine. Improving diagnosis in health care National Academies Press; 2016. January 29. [Google Scholar]
- 23.Upadhyay DK, Sittig DF, Singh H. Ebola US Patient Zero: lessons on misdiagnosis and effective use of electronic health records. Diagnosis 2014;1(4):283–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Schiff GD, Bates DW. Can electronic clinical documentation help prevent diagnostic errors? N Engl J Med 2010;362(12):1066–9. [DOI] [PubMed] [Google Scholar]
- 25.Coiera E. When conversation is better than computation. J Am Med Inform Assoc 2000;7(3):277–86. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.El-Kareh R, Hasan O, Schiff GD. Use of health information technology to reduce diagnostic errors. BMJ Qual Saf 2013;22(Suppl 2):ii40–ii51. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Asan O, Carayon P. Human Factors of Health Information Technology—Challenges and Opportunities 2017: 255–257. [DOI] [PMC free article] [PubMed]
- 28.Patel VL, Kannampallil TG. Human factors and health information technology: current challenges and future directions. Yearbook of medical informatics 2014;9(1):58. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Gupta A, Harrod M, Quinn M, Manojlovich M, Fowler KE, Singh H, Saint S, Chopra V. Mind the overlap: how system problems contribute to cognitive failure and diagnostic errors. Diagnosis 2018. September 25;5(3):151–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Monahan T, Fisher JA. Benefits of ‘observer effects’: lessons from the field. Qual Res 2010;10(3):357–76. [DOI] [PMC free article] [PubMed] [Google Scholar]