Abstract
Objective
To characterize question types that residents received on overnight shifts and what information sources were used to answer them.
Materials and Methods
Across 30 overnight shifts, questions asked of on-call senior residents, question askers’ roles, and residents’ responses were documented. External sources were noted.
Results
158 of 397 questions (39.8%) related to the plan of care, 53 (13.4%) to medical knowledge, 48 (12.1%) to taskwork knowledge, and 44 (11.1%) to the current condition of patients. For 351 (88.4%) questions residents provided specific, direct answers or visited the patient. For 16 of these, residents modeled or completed the task. For 216 questions, residents used previous knowledge or their own clinical judgment. Residents solicited external information sources for 118 questions and only a single source for 77 (65.3%) of them. For the 118, most questions concerned either the plan of care or the patient's current condition and were asked by interns and nurses (those with direct patient care responsibilities).
Discussion
Resident physicians serve as an information system and they often specifically answer the question using previous knowledge or their own clinical judgment, suggesting that askers are contacting an appropriately knowledgeable person. However, they do need to access patient information such as the plan of care. They also serve an educator role and answer many knowledge-related questions.
Conclusions
As synchronous verbal communications continue to be important pathways for information flow, informaticians need to consider the relationship between such communications and workflow in the development of healthcare support tools.
Keywords: Information systems, Information seeking behavior, Information resources, Communication
Background and significance
In healthcare, problems with information management are often associated with technology, and proposed solutions frequently involve the move from one technologically based system to another,1 2 as illustrated by the transition from paper to electronic medical records (EMR).3 While technology provides a means for augmenting human cognitive capacity in domains faced with a burgeoning set of pertinent information,4 as a whole, managing information is not simply a matter of data entry, storage and retrieval. Complex processes such as communication, aggregation, summarization, and conversion are necessary in order to turn information into knowledge that is useful for healthcare providers.5 While technology to support processes such as communication is improving, its capability is not a replacement for communication and healthcare providers continue engage in verbal communications.3 5–11 Furthermore, there is a growing body of evidence in healthcare of the inefficiencies and hazards resulting from the implementation of inappropriate, ‘technology-heavy’ solutions that exacerbate human information management problems.12–15
Understanding cognitive and behavioral as well as technological aspects of information processing is critical to designing solutions, given that both humans and technology frequently perform such functions.1 5 For example, humans interact with other information systems, both human and non-human, to find, share, and exchange information. Despite being critical aspects of patient care, such responsibilities are generally embedded within the healthcare system, are not clearly defined or outlined by organizational policies, which tend to prioritize the more obvious, explicable responsibilities associated with specific occupational roles (eg, physicians diagnose, pharmacists fill orders, etc.), and have remained relatively unsupported by information technology.5 6 Many EMR systems are designed directly to support patient care tasks by collecting, storing, aggregating, and making accessible a vast amount of patient data, but are not designed to support communication and information exchange between people or between people and other information systems.
Objectives
Resident physicians are an example in which the role as a human information system serving the information needs of others is critical to patient care. The goal of this study is to understand two specific aspects related to this role: what information is requested from residents and what information sources residents access, if any, when formulating an answer. This understanding can help inform the design of the EMR and other interventions that support the information needs of healthcare providers.
Methods
Study setting
The institutional review board of the University of Virginia approved this study and all participants gave informed consent. The study was conducted on two adjacent general pediatric acute care wards at the University of Virginia Children's Hospital, a university-based tertiary care hospital with a medium-sized pediatric residency training program where the general pediatrics service consisted of three first-year pediatric and/or family medicine residents (PGY-1), also called interns, two third-year pediatric residents (PGY-3), also called senior residents, and a pediatric attending physician. At the time of the study, the hospital used the medical information system (MIS) as the computerized provider order entry (CPOE) system. CareCast was the EMR system.
Each patient admitted to the ward was assigned to both an intern and third-year senior resident. The intern served as the patient's primary care provider while the senior resident supervised. The daytime care team consisted of two second or third year pediatric residents (PGY-2 or PGY-3), three interns (PGY-1), and the attending physician. The day team rounded together every morning. Night coverage was shared by eight residents: five on the primary care team (three interns and two PGY-3 senior residents on general ward rotations), and the cross-cover team (two PGY-2 senior residents and one intern on other pediatric rotations).
Data collection
This study focused on the time period beginning with the 16:00 h week-day patient handover sessions until the ensuing ‘on call’ shift. Each ‘observation session’ included all data collected between the end of the 16:00 h handover and 23:00 h; the 30 observations sessions were 210 h in total. For each observation session, an analyst silently observed the 16:00 h handover and then afterwards shadowed the on-call senior resident until 23:00 h. Observation sessions also partly overlapped with the hospital's visiting hours for patients (9–21 h daily). During this time family members also had the opportunity to interact with residents and ask questions.
For each observation session, the analyst documented all of the verbal questions that the senior resident was asked, and the role of the question asker. If the resident did not provide a specific response that included the answer to the question, the analyst noted what behavior occurred, if any.
The analyst could ask the resident to repeat a question and/or answer information if necessary. When there were telephone conversations, the analyst asked the resident to repeat any information that he could not overhear.
Any external source residents solicited for information were also noted. If the resident solicited more than one source, the sequence in which the resident solicited each source was also noted. The external sources available to residents for soliciting information included:
Other human sources (physicians, nurses, and others): phone calls, face-to-face communications.
Medical chart: daily progress notes, nursing and therapy interactions with the patient, attending notes, admission notes.
Other non-chart documents: cross-cover notes (written if there is a significant change in the treatment of a patient).
CPOE: used to place orders and view laboratory results.
EMR system: historical data such as past laboratory results, previous care, and admissions.
Sign-out sheet (handoff/handover sheet): printed list of patients and handover specific information elements.
To structure the data collection process, the analyst recorded all observations using a custom data collection worksheet (figure 1). To support data analysis, the worksheet data were entered into a custom Microsoft Access database.
Figure 1.
Data collection worksheet. CPOE, computerized provider order entry; EMR, electronic medical record; MIS, medical information system.
Question categorization determination
Two medical faculty members and one systems engineer reviewed and categorized each question independently using a content categorization scheme (table 1) from earlier work.16 While there was an assessment as to whether the response was or was not a direct answer to the question, there was no assessment of the correctness of the direct answers provided by the residents. After independently categorizing the set of questions, the group convened to determine a single final content category for each question. Consensus on the final category was defined using agreement between at least two of the three reviewers. Group discussion was encouraged to help reach consensus.
Table 1.
Question content categorization
| Category | Description |
|---|---|
| Plan of care (eg, How much oral nutrition should the patient get when he returns from the operating room?) | Both the immediate plan of care for the next 12–24 h for a patient, as well as the long term plan (‘big picture’) beyond 24 h and for a patient's entire hospital course |
| Rationale (eg, Why is the patient on oxygen?) | Rationale for the overall plan of care or specific elements of the plan of care, or the reason(s) they are under consideration |
| Current condition (eg, What is the patient's blood pressure?) | Most recent information on the patient's current condition |
| Past condition (eg, Do you know what has caused his recent respiratory outbreaks/cystic fibrosis respiratory flare-ups?) | Concerns the patient's previous condition (before the last 4 h) |
| Medical knowledge (eg, Can viral infection of the cerebrospinal fluid kill someone?) | General medical knowledge that could be found in a medical handbook, textbook, or online source, such as the dosages for morphine |
| Technical knowledge (eg, How do you add a patient to CareCast?) | Related to the use of specific hospital computer systems such as computerized order entry systems or the EMR |
| Medical practice (eg, How do you palpate a newborn's liver?) | The implementation of medical knowledge, such as how to carry out medical procedure |
| Taskwork knowledge (eg, Where do we go to get bilirubin lights?) | How to complete tasks relevant to the work at hand other than those concerning computer or technical systems (eg, ‘at what time does the pediatric intern begin his call shift?’) |
| Task in-progress (eg, Did you put in the urine analysis order for the patient?) | Activities or tasks which are currently in-progress for a patient. process check questions (eg, ‘have we done _____ for the patient yet?’) |
| Request (eg, Can you order another unit of blood for the patient?) | Request to complete a task or some type of activity or action |
EMR, electronic medical record.
Data analysis
Data are presented based on the method in which residents responded to questions: responded by doing/showing, responded with previous knowledge or clinical judgment only, and responded by soliciting information from external sources.
Summary and count data are reported for question categories, question askers, and external sources solicited. In addition, pair-wise summary and count data are presented for question category by question asker and question category by information source solicited. Order data for questions in which multiple information sources were solicited are also presented.
Results
Sample
Over a 6-week period, there were 30 observation sessions involving 10 individual senior residents (five PGY-2 and five PGY-3). A subset of the 10 residents was observed on multiple occasions due to varying schedules.
Question categorization
Senior residents were asked a total of 399 questions; total questions asked per observation session ranged from four to 30 (mean 13.3, median 13.5). The analysts determined a consensus category for 397 of the 399 questions (two questions were categorized as ‘unknown’ and are not included in subsequent analyses) (table 2).
Table 2.
Questions by category, question source, and question status (if direct answer to question)
| Source | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| All | Intern | Nurse | Family | Student | Primary care team | Cross-cover team | Other | |||||||||
| Category | n | An | n | An | n | An | n | An | N | An | n | An | n | An | n | An |
| Plan of care | 158 | 145 | 44 | 40 | 63 | 60 | 30 | 25 | 7 | 6 | 9 | 9 | 5 | 5 | – | – |
| Rationale | 16 | 13 | 2 | 2 | 3 | 2 | 4 | 4 | 4 | 3 | 1 | – | 2 | 2 | – | – |
| Current condition | 44 | 34 | 10 | 8 | 7 | 6 | 6 | 5 | 8 | 5 | 7 | 7 | 5 | 2 | 1 | 1 |
| Past condition | 21 | 19 | 8 | 8 | 3 | 2 | 3 | 3 | 5 | 5 | 2 | 1 | – | – | – | – |
| Subtotal | 239 | 211 | 64 | 58 | 76 | 70 | 43 | 37 | 24 | 19 | 19 | 17 | 12 | 9 | 1 | 1 |
| Medical knowledge | 53 | 47 | 8 | 6 | 5 | 5 | 23 | 22 | 13 | 12 | 1 | 1 | 2 | – | 1 | 1 |
| Taskwork knowledge | 48 | 41 | 20 | 18 | 6 | 4 | 8 | 7 | 5 | 4 | 5 | 4 | 2 | 2 | 2 | 2 |
| Medical practice | 14 | 13 | 4 | 4 | 2 | 2 | 6 | 5 | 2 | 2 | – | – | – | – | – | – |
| Technical knowledge | 13 | 11 | 10 | 8 | – | – | – | – | 1 | 1 | 1 | 1 | 1 | 1 | – | – |
| Subtotal | 128 | 112 | 42 | 36 | 13 | 11 | 37 | 34 | 21 | 19 | 7 | 6 | 5 | 3 | 3 | 3 |
| Task in-progress | 18 | 16 | 8 | 8 | 4 | 4 | 1 | – | – | – | 4 | 3 | 1 | 1 | – | – |
| Request | 12 | 11 | 1 | 1 | 10 | 9 | 1 | 1 | – | – | – | – | – | – | – | – |
| Subtotal | 30 | 27 | 9 | 9 | 14 | 13 | 2 | 1 | – | – | 4 | 3 | 1 | 1 | – | – |
| Total | 397 | 350 | 115 | 103 | 103 | 94 | 82 | 72 | 45 | 38 | 30 | 26 | 18 | 13 | 4 | 4 |
An, answer.
Close to half of the questions addressed the plan of care and its rationale (158 of 397, 39.8% and 16 of 397, 4.0%, respectively). Another 32.2% addressed knowledge requests (medical knowledge, taskwork knowledge, medical practice, and technical knowledge); 16.4% addressed the patient's current or past condition.
Resident responses to questions
The residents always acknowledged the questions asked. For questions for which the resident did not provide a direct answer to the question, the responses are categorized based on categories emergent from the data: (a) did not know the answer to the question, stated so, and did not seek more information, (b) indicated that the question had already been answered and did not provide additional information, (c) asked another person, sometimes the asker, to purse the answer, (d) referred the asker to another person, (e) stated that he or she would search for the answer at a later time, or (f) stated that he or she would ask another person at a later time.
For 46 questions of the 397 questions (11.6%), the resident did not provide or try to provide a specific answer to the actual question. For 21 of these 46 questions, residents stated that they did not know the answer to the question and did not seek more information. Fifteen of the 21 related to taskwork knowledge (6 of 21, 28.6%), the plan of care and the rationale for it (5 of 21, 23.8%), or general medical knowledge (4 of 21, 19.0%). For example in one case a family member asked ‘… you don't really know why my son has a fever or when he'll get discharged?’ and the resident answered that he did not know.
For the other 25 of the 46 questions, the question was already answered or a strategy was posed for getting the answer. For two questions, the resident stated that the question had already been answered and did not provide additional information. For five questions, the resident asked another person, sometimes the asker, to purse the answer. For example a medical school student asked where would be the best place to look for a patient's grandmother's phone number and the resident asked the nurse to help. When asked how to calculate a dose of Caspofungin, a lipopeptide antifungal drug, the resident responded that the asker should enter the question into an internet search engine.
In three cases, the resident referred the asker to another person. For example when an intern asked if the diabetic patient's insulin pump program should be changed to correct for her glucose level or whether the team wait for the midnight measurement, the resident asked the intern to consult with endocrinology.
For seven questions (three related to the patient's current condition, two to medical knowledge, one to the plan of care and one to the rationale for the plan), the resident stated that he or she would search for the answer at a later time. For example, an intern asked why the team was stopping the patient's feeds and the resident did not know the rationale for this plan. The resident stated that he would look into the issue at a later time.
For another eight questions (six related to the plan of care and two to the patient's current condition), the resident stated that he or she would ask another person at a later time. For example, when the intern asked whether the patient can leave after the MRI is completed, the resident stated that he would have to consult pediatric neurology. For example, when a family member asked when the child will be able to eat and drink again, the resident answered that in the morning he would discuss the situation with the attending physician who was not currently on the wards.
For the remaining 351 of the 397 questions, the resident provided or attempted to provide a specific, direct answer to the question. In one case, the resident went to see the patient because the intern noticed that the patient was bleeding where her catheter tube was inserted and asked the resident to come check it. For 16 questions, residents responded by modeling or completing a task. These questions were primarily technical questions (7 of 16, 43.8%), and requests (3 of 16, 18.8%). Six of the seven (85.7%) technical questions were asked by an intern and all three requests arose from a nurse. For example when asked by an intern how to set up discharge orders for a patient using MIS, the resident walked the intern through the steps.
For 216 questions (61% of the 351), residents’ responded using their prior knowledge or clinical judgment without soliciting information from external sources. For example, when a medical student asked if a patient has any renal problems, the resident immediately answered without checking any sources. With respect to the content, 103 questions were related to the plan of care, 42 to medical knowledge, and 29 to taskwork knowledge.
External information sources
Residents solicited information from external sources in response to 118 questions (table 3). Questions about the plan of care most often led residents to solicit information from external sources, and they accounted for 42.4% (50 of 118) of all questions in which residents solicited external sources for information. Questions about the current condition of patients (16.1%) and taskwork knowledge (13.6%) were also answered by accessing outside sources. Of the remaining categories, none accounted for more than 10% of the questions in which residents solicited external sources for information.
Table 3.
Questions by category when soliciting external sources
| At least one external source | Only one external source | More than one external source | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Category | Total (%) | An (%) | Unan (%) | Total (%) | An (%) | Unan (%) | Total (%) | An (%) | Unan (%) |
| Plan of care | 50 (42.4) | 47 (39.8) | 3 (2.5) | 26 (22.0) | 25 (21.2) | 1 (0.8) | 24 (20.3) | 22 (18.6) | 2 (1.7) |
| Current condition | 19 (16.1) | 16 (13.6) | 3 (2.5) | 14 (11.9) | 12 (10.2) | 2 (1.7) | 5 (4.2) | 4 (3.4) | 1 (0.8) |
| Taskwork knowledge | 16 (13.6) | 11 (9.3) | 5 (4.2) | 14 (11.9) | 10 (8.5) | 4 (3.4) | 2 (1.7) | 1 (0.8) | 1 (0.8) |
| Medical knowledge | 10 (8.5) | 8 (6.8) | 2 (1.7) | 5 (4.2) | 4 (3.4) | 1 (0.8) | 5 (4.2) | 4 (3.4) | 1 (0.8) |
| Past condition | 9 (7.6) | 8 (6.8) | 1 (0.8) | 7 (5.9) | 6 (5.1) | 1 (0.8) | 2 (1.7) | 2 (1.7) | – |
| Rationale | 6 (5.1) | 4 (3.4) | 2 (1.7) | 4 (3.4) | 3 (2.5) | 1 (0.8) | 2 (1.7) | 1 (0.8) | 1 (0.8) |
| Task in-progress | 5 (4.2) | 5 (4.2) | – | 5 (4.2) | 5 (4.2) | – | – | – | – |
| Request | 1 (0.8) | 1 (0.8) | – | – | – | – | 1 (0.8) | 1 (0.8) | – |
| Technical knowledge | 1 (0.8) | 1 (0.8) | – | 1 (0.8) | 1 (0.8) | – | – | – | – |
| Medical practice | 1 (0.8) | 1 (0.8) | – | 1 (0.8) | 1 (0.8) | – | – | – | – |
| Total | 118 (100) | 102 (86.4) | 16 (13.6) | 77 (65.3) | 67 (56.8) | 10 (8.5) | 41 (34.8) | 35 (29.7) | 6 (5.1) |
An, answered; Unan, unanswered.
Residents solicited one external information source in response to 77 of 118 questions (65.3%), indicating that for the majority of questions, the residents knew where to find the answer when not immediately knowing it. Fifty-four of these 77 questions (70.1%) related to the plan of care, the current condition of a patient, or taskwork knowledge. For these 54 questions, the most frequently solicited external sources were phone calls to other physicians (22 of 54, 40.7%), and sign-out sheets (10 of 54, 18.5%) generated using a custom handoff of care tool17 and then optionally annotated when providing patient care. Progress notes, queries to nurses, and calls to other non-physicians were less frequently solicited sources. Residents provided a specific answer to the question for 67 of the 77 questions (87.0%) when only one external source was solicited for information, most often after calling other physicians (24 of 77, 31.2%), checking sign-out sheets (11 of 77, 14.3%), checking progress notes (seven of 77 questions, 9.1%), or asking a nurse (six of 77, 7.8%).
In response to 41 of the 118 questions (34.7%), residents solicited information from more than one external source and they provided a specific, direct answer to the question for 33 (80.5%) of these 41. With respect to content, 24 of the 41 (58.5%) questions related to the plan of care, and residents provided specific answers for 22 of these (91.7%).
Residents most often called other physicians, checked the EMR (CareCast), or consulted sign-out sheets as the first source when soliciting more than one source of information (table 4). When residents called other physicians first but did not receive a direct answer to the question, they generally solicited only one additional external source (table 4).
Table 4.
External sources by order solicited
| Order solicited (N=41 questions) | |||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| First source | Final source | ||||||||||||||||
| 1st (N=41) | 2nd (N=15) | 3rd (N=1) | 2nd (N=26) | 3rd (N=14) | 4th (N=1) | ||||||||||||
| Information source | n | % of 41 | n | % of 15 | % of 41 | n | % of 1 | % of 41 | n | % of 26 | % of 41 | n | % of 14 | % of 41 | n | % of 1 | % of 41 |
| Phone call—other physician | 7 | 17.1 | 1 | 6.7 | 2.4 | – | – | – | 5 | 19.2 | 12.2 | 6 | 42.9 | 14.6 | – | – | – |
| MIS | 1 | 2.4 | 7 | 46.7 | 17.1 | – | – | – | 5 | 19.2 | 12.2 | 2 | 14.3 | 4.9 | 1 | 100 | 2.4 |
| Carecast | 7 | 17.1 | 1 | 6.7 | 2.4 | 1 | 100 | 2.4 | 3 | 11.5 | 7.3 | – | – | – | – | – | – |
| Chart—progress note | 5 | 12.2 | 2 | 13.3 | 4.9 | – | – | – | 2 | 7.7 | 4.9 | 2 | 14.3 | 4.9 | – | – | – |
| Sign-out sheet | 7 | 17.1 | – | – | – | – | – | – | 2 | 7.7 | 4.9 | – | – | – | – | – | – |
| Other non-chart document | 1 | 2.4 | 1 | 6.7 | 2.4 | – | – | – | 4 | 15.4 | 9.8 | 1 | 7.1 | 2.4 | – | – | – |
| Nursing chart | 4 | 9.8 | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – |
| Chart—attending note | 1 | 2.4 | 2 | 13.3 | 4.9 | – | – | – | 1 | 3.8 | 2.4 | – | – | – | – | – | – |
| Phone call—other non-physician | 2 | 4.9 | – | – | – | – | – | – | – | – | – | – | – | – | – | – | – |
| Nurse | – | – | 1 | 6.7 | 2.4 | – | – | – | – | – | – | 1 | 7.1 | 2.4 | – | – | – |
| Clinical judgment | 6 | 14.6 | – | – | – | – | – | – | 4 | 15.4 | 9.8 | 2 | 14.3 | 4.9 | – | – | – |
| Total | 41 | 100 | 15 | 100 | 36.6 | 1 | 100 | 2.4 | 26 | 100 | 63.4 | 14 | 100 | 34.1 | 1 | 100 | 2.4 |
MIS, medical information system.
For six of the questions (14.6%), residents first provided information using their previous or clinical judgment, but followed this by soliciting information from an external source (table 4). In all six of these instances, residents provided an answer to the question immediately after consulting the first external source. Sometimes they first sought information and then provided a clinical judgment. For six of the 41 questions (14.6%), residents first solicited information from one or two external sources yet used their clinical judgment to formulate their final response (table 4). Specific answers to the questions were given for five of six of these questions (83.3%). Interestingly, for 25% (11 of 44) of questions that related to the current condition of a patient, residents did not use their own clinical judgment to answer the question nor did they physically go and check on the patient as a source for that information.
Soliciting more than three external sources occurred only once (table 4). In general, residents rarely solicited information from attending physician notes (table 4).
Question askers
The majority of questions were asked by individuals with previous medical training and patient care responsibilities, namely first-year residents (interns) and nurses (table 1). Few questions were asked by the primary team, by ‘cross covering’ physicians or by consulting physicians. Most of these questions concerned the plan of care and/or the current condition of patients. Questions asked by interns and nurses accounted for the majority of requests and questions about in-progress tasks.
With respect to knowledge questions, interns asked the majority of questions pertaining to taskwork knowledge as well as the most technical questions. Patients, family members and medical students asked the greatest number of questions, and the majority of these questions related to medical or technical knowledge and medical or taskwork knowledge.
Discussion
Residents as information systems
In addition to providing patient care, resident physicians serve as an information system that receives and responds to questions from other healthcare providers as well as from patients and their families. In this study, residents generally gave some type of response, even if the acknowledgment was ‘I don't know’. While similar studies have shown that residents do not always attempt to answer clinical questions, most have focused on residents’ response to their own clinical questions as opposed to the questions they are asked by others.1 18–20 This study showed that when queried by others, residents do acknowledge the question asker.
In this study, residents responded to most questions, often by immediately providing their previous knowledge or own clinical judgment, suggesting that question askers are soliciting information from a reliable and effective information source (ie, residents who can usually provide an answer). When residents do not immediately respond by providing the requested information, they often solicited information from one or more external sources of information. In some cases, they asked the seeker to find the answer from a more knowledgable source (often a different healthcare provider) and at other times they committed to seeking the information themselves but at a future time as the information would be available or accessible later.
Overall, other physicians are the external information sources from which residents most frequently solicit information, particularly when they are seeking information about the plan of care or the current condition of a patient. These results match a growing body of literature reflecting that physicians engage in synchronous verbal communications when patient status data are required.7–11 20 Questions about the plan of care or current condition were usually asked by someone whose responsibilities might include tasks directly related to patient care (interns, nurses, medical students, and family members). All of these individuals were responsible in some way for patient care activities at the time the question was asked. Residents were able to respond to the majority of questions and to provide specific answers to the questions using previous knowledge or their own clinical judgment without needing to solicit external sources of information. However, the ‘plan of care’ is abstract, role specific, and not generally well characterized in the medical record.21 This may be why residents often consult other human sources when they are asked questions about the plan of care, and why such questions are sometimes left unanswered.20
Similarly, residents consulted external sources of information when they were asked questions about the current condition of patients. This can be critical information when senior residents are engaged in medical decision-making and providing patient care. For 25% (10 of 44) of questions that related to the current condition of a patient, residents did not use previoius knowledge or their own clinical judgment to answer the question nor did they check on the patient as a source for that information. In fact, checking on the patient was never observed in this study as a source of information utilized by residents (the only time that a resident went to the patient was when specifically asked to check on a patient who was bleeding where her catheter tube was inserted). This has previously been described in academic medical settings where members of the care team frequently exchange patient information between each other in order to avoid unnecessary disturbance to their patients.1
External information sources
People (generally other physicians), and electronic sources (specifically the CPOE system), were the information sources that most often led to an answer when residents did not immediately know the answer. Phone calls to other physicians were the most frequent external sources of information for residents. Interestingly, residents rarely sought information from daily progress notes or attending physician notes in the medical record, similar to results of other researchers who note that such documents are designed largely to provide evidence of care to support billing.23 As mentioned, some researchers have suggested that clinicians may have preferences for interactive communications. An alternative explanation may be that senior residents have previously learned they will not find the information they need from paper and electronic sources of information. No matter what the reason, these results raise concerns about the utility of information systems that marginalize verbal interactions, as is the case of EMR systems. Verbal communication is clearly very important for residents when providing and exchanging information.
This study shows that humans as well as electronic and paper systems are utilized by residents when soliciting information in response to the questions of others. Our data suggest some sources of information are more useful to residents than others, but little is known about how ‘useful’ these information sources are or how they influence patient care outcomes. Our data suggest ‘usefulness’ appears to be situation dependent. It is possible that where residents look first for information is the source they expect or assume the information they are looking for should be. As nurses, medical students and other health professionals have access to EMR systems, it is possible that they may be asking questions of the resident after discovering that the information was not there. Therefore, the residents might not consult the sources very often because they know in advance that the information/answer is not likely to be there. Interestingly for questions they did not answer, residents rarely used their sign-out sheet (not available to medical students and other health professionals) as the first external source of information. Some potential directions for this work included further assessments of the impact of external information sources on residents’ task efficiency and the development of metrics for quantitatively assessing information systems with respect to characteristics such as ‘usefulness’.
Limitations and future work
All of the observations were collected by a single individual. Errors of observation are always possible. Even though the analyst could ask the resident to repeat a question and/or answer information if necessary, future studies should employ multiple data collection analysts to minimize any data collection bias.
The total questions asked per observation period ranged from four to 30. This variability may be due to the approachability of each resident as some may be more receptive to questions than others. Future work should investigate this issue as barriers to gaining information.
The current analysis does not account for the factors related to the question asker with the exception of the role. For example, the question asker may frame the question in different ways based on features of the asked resident. No analysis was conducted to address details about the question asker or the question asker–resident interaction. Future studies should use more comprehensive ethnographic approaches to address these interactions.
Our data do not provide detail regarding the nature of residents’ clinical judgment. Therefore, we can only speculate about the means by which residents acquired this information or where it originated. Audio-recordings of the handovers immediately preceding the observation sessions in this study might provide some additional insight, given that the primary objective of patient handover is the accurate transfer of information relevant to patient care.16–17 Perhaps there is a relationship between the information communicated at handover and the information that residents later provided using their clinical judgments when responding to questions they were asked during their call shifts. Such a relationship could be evidence of the effect of handover on patient care activities. It is also possible that there are other sources, perhaps many others, from which residents solicit information that were not observed during this study. Therefore, as some authors have suggested,2 future studies in this area should account for a broad spectrum of potential information sources available to residents.
Conclusion
Verbal communications continue to be an important way that information is transferred between healthcare providers in the inpatient setting. Residents are asked many questions during their overnight shifts and are able to answer many of them without consulting external sources. When external sources are consulted, synchronous communications such as phone calls to other healthcare providers are common in addition to checking written and electronic sources. As synchronous communications such as phone calls continue to be important pathways for information flow, informatics researchers and designers need to consider the relationship between such communications and workflow in the development of healthcare decision support tools.
Acknowledgments
The authors would like to acknowledge the pediatric house staff at the University of Virginia, without whom this work could not have been completed. The authors would also like to thank the anonymous reviewers who provided substantive comments on an earlier version of this manuscript.
Footnotes
Funding: This work was supported in part by the National Library of Medicine under grant T15LM009462 and from the Graduate Medical Education Innovation Program at the University of Virginia. The content is solely the responsibility of the authors and does not necessarily represent the official views of the University of Virginia, the National Library of Medicine or the National Institutes of Health. None of these organizations had any influence in the design and conduct of the study, the collection, management, analysis, and interpretation of the data; or preparation, review, or approval of this manuscript.
Competing interests: None.
Ethics approval: The health systems research institutional review board of the University of Virginia approved this study.
Patient consent: Obtained.
Provenance and peer review: Not commissioned; externally peer reviewed.
References
- 1.Gorman PN. Information needs of physicians. J Am Soc Info Sci 1995;46:729–36 [Google Scholar]
- 2.Forsythe DE, Buchanan BG, Osheroff JA, et al. Expanding the concept of medical information: an observational study of physicians’ information needs. Comput Biomed Res 1992;25:181–200 [DOI] [PubMed] [Google Scholar]
- 3.Stead WW, Lin HS, eds. Computational technology for effective health care: immediate steps and strategic directions. Committee on engaging the computer science research community in health care informatics; National Research Council. Washington, DC: National Academies Press, 2009 [PubMed] [Google Scholar]
- 4.Stead WW, Starmer JM. Beyond expert-based practice. Institute of Medicine (IOM). Evidence-Based Medicine and the Changing Nature of Health Care: 2007 Annual Meeting Summary. Washington, DC: National Academies Press, 2008:94–105 [Google Scholar]
- 5.Detmer DE. Engineering information technology for actionable information and better health: Balancing social values through desired outcomes, complementary standards and decision-support. Info Knowledge Syst Manag 2009;8:107–18 [PubMed] [Google Scholar]
- 6.Davenport TH. Saving IT's soul: human-centered information management. Harv Bus Rev 1994;72:119–31 [Google Scholar]
- 7.Alvarez G, Coiera E. Interdisciplinary communication: an uncharted source of medical error? J Crit Care 2006;21:236–42 [DOI] [PubMed] [Google Scholar]
- 8.Coiera E. When conversation is better than computation. J Am Med Informatics Assoc 2000;7:277–86 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Coiera E, Tombs V. Communication behaviours in a hospital setting: an observational study. BMJ 1998;316:673–6 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Coiera EW, Jayasuriya RA, Hardy J, et al. Communication loads on clinical staff in the emergency department. Med J Aust 2002;176:415–18 [DOI] [PubMed] [Google Scholar]
- 11.Spencer R, Coiera E, Logan P. Variation in communication loads on clinical staff in the emergency department. Ann Emerg Med 2004;44:268–73 [DOI] [PubMed] [Google Scholar]
- 12.Sittig DF, Ash JS. Clinical information systems: overcoming adverse consequences. Burlington, Massachusetts: Jones & Bartlett Learning, 2009 [Google Scholar]
- 13.Sittig DF, Classen DC. Safe electronic health record use requires a comprehensive monitoring and evaluation framework. J Am Med Assoc 2010;303:450–1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Karsh B, Weinger MB, Abbott PA, et al. Health information technology: fallacies and sober realities. J Am Med Inform Assoc 2010;17:617–23 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Carayon P, Bass EJ, Bellandi T, et al. Socio-technical systems analysis in health care: A research agenda. IIE Trans Healthc Syst Eng 2011;1:145–60 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Waggoner-Fountain LA, Tomney BA, DeVoge JM, et al. Questions asked and answers located: what really happens on-call after sign-out. PAS 2007-Pediatric Academic Societies 2007:61 E-pub [Google Scholar]
- 17.DeVoge JM, Bass EJ, Sledd RMD, et al. Collaborating with physicians to redesign a sign-out tool: An iterative, multifaceted approach with users—even busy ones—can yield a satisfying and efficient product. Ergon Des 2009;20.– [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Chase HS, Kaufman DR, Johnson SB, et al. Voice capture of medical residents’ clinical information needs during an inpatient rotation. J Am Med Inform Assoc 2009;16:387–94 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Coumou HCH, Meijman FJ. How do primary care physicians seek answers to clinical questions? A literature review. J Med Libr Assoc 2006;94:55–60 [PMC free article] [PubMed] [Google Scholar]
- 20.Covell DG, Uman GC, Manning PR. Information needs in office practice: are they being met? Ann Intern Med 1985;103:596. [DOI] [PubMed] [Google Scholar]
- 21.Mueller FF, Kethers S, Alem L, et al. From the certainty of information transfer to the ambiguity of intuition. Proceeding OZCHI ‘06 Proceedings of the 18th Australia conference on Computer-Human Interaction: Design: Activities, Artefacts and Environments, 2006:63–70.10.1145/1228175.1228189 [Google Scholar]
- 22.Borowitz SM, Waggoner-Fountain LA, Bass EJ, et al. Resident sign-out: a precarious exchange of critical information in a fast paced world. In: Henriksen K, Battles JB, Keyes MA, Grady ML, eds. Advances in patient safety: new directions and alternative approaches. Vol. 2: Culture and redesign. AHRQ Publication Nos. 08–0034 (1–4) Rockville, MD: Agency for Healthcare Research and Quality, July 2008 [PubMed] [Google Scholar]
- 23.Brown PJ, Borowitz SM, Novicoff W. Information exchange in the NICU: what sources of information do physicians prefer to use. Int J Med Inform 2004;73:349–55 [DOI] [PubMed] [Google Scholar]