Comparing the information seeking strategies of residents, nurse practitioners, and physician assistants in critical care settings

Abstract

Objective

Critical care environments are information-intensive environments where effective decisions are predicated on successfully finding and using the ‘right information at the right time’. We characterize the differences in processes and strategies of information seeking between residents, nurse practitioners (NPs), and physician assistants (PAs).

Method

We conducted an exploratory study in the cardiothoracic intensive care units of two large academic hospitals within the same healthcare system. Clinicians (residents (n=5), NPs (n=5), and PAs (n=5)) were shadowed as they gathered information on patients in preparation for clinical rounds. Information seeking activities on 96 patients were collected over a period of 3 months (N_Res=37, N_NP=24, N_PA=35 patients). The sources of information and time spent gathering the information at each source were recorded. Exploratory data analysis using probabilistic sequential approaches was used to analyze the data.

Results

Residents predominantly used a patient-based information seeking strategy in which all relevant information was aggregated for one patient at a time. In contrast, NPs and PAs primarily utilized a source-based information seeking strategy in which similar (or equivalent) information was aggregated for multiple patients at a time (eg, X-rays for all patients).

Conclusions

The differences in the information seeking strategies are potentially a result of the differences in clinical training, strategies of managing cognitive load, and the nature of the use of available health IT tools. Further research is needed to investigate the effects of these differences on clinical and process outcomes.

Introduction

Critical care settings are prototypical information-intensive environments.^{1 2} Providing optimal care in such settings requires the effective synchronization of a highly engaged, interprofessional team. Assembling such a team is often hampered by staffing shortages, and recent reports show that only one in three intensive care unit (ICU) patients in the USA is treated by a board-certified intensivist.³ These shortages are further exacerbated by the 24/7 physician staffing guidelines,^{4 5} reduced resident work hours,^{6 7} and increased volume of critically ill patients.⁸ Increasingly, non-physician providers (nurse practitioners (NPs) and physician assistants (PAs); collectively referred to as advance-practice practitioners, affiliate providers, or mid-level practitioners), are working under the supervision of attending physicians and are effectively mitigating some of these issues.⁹

The performance of NPs and PAs in critical care settings has been investigated on a number of outcome measures including length of stay,^10–12 mortality rates,¹³ readmission rates,^{14 15} costs of care,^{16 17} time-savings for physicians,¹² rate of infections,¹⁸ and efficiency of mechanical ventilation.¹³ In academic critical care units, NPs and PAs often work interchangeably with residents in achieving comparable performance outcomes, and potentially better overall outcomes.^{16 19 20} Specifically, they may provide: better continuity of care as they do not often rotate off-service, better detection of subtle changes in patient conditions (eg, delirium),²¹ better communication with nurses, other providers, and patient family, and involvement in the education of nurses and clinical research.^{22 23} Even though their skills and effectiveness have been considered valuable for critical care practice, little is known about the processes and strategies they use in order to achieve comparable or better outcomes. We sought to characterize the differences in information seeking strategies of residents, NPs, and PAs, which may significantly impact patient care activities, use of health information technology, and clinical workflow.

Information seeking in critical care environments is a challenging task that is impeded by cognitive limitations such as memory capacity, information overload, and technological limitations.²⁴ Physical constraints such as distributed, redundant, and fragmented organization of information also lead to difficulties in tracking and tracing required information.^25–27 While there is significant empirical research on the challenges of meeting clinicians’ information needs^28–30 and on the types of information resources used by clinicians,^31–37 little is known about the process and strategies of information seeking or the inherent differences between clinical providers with different levels of expertise and training. While general differences in search choices have been described—for example, the types of information sought,³⁸ or the usual sources of information³⁹—much less is known about how clinicians make these choices and the potential implications of these choices.

Given the limited prior research on information seeking strategies, both within and across professional groups, we utilized an exploratory approach, predicated on characterizing the temporal sequences of information seeking activities and identifying the differences across residents, NPs, and PAs. Studying the processes and activities is key to understanding the cognitive underpinnings of human information seeking behavior in clinical settings, and can lead to the development of empirically grounded theories of human behavior. While preliminary, the differences in information seeking strategies have important implications for the management of clinical workflow, and the cognitive load on clinicians during clinical work activities.

Method

Study setting

The study was conducted in the cardiothoracic ICUs (CT-ICUs) of two large academic hospitals within the same healthcare system (henceforth referred to as CT-ICU (A) and CT-ICU (B)). CT-ICU (A) had 18 beds that were split between overlying floors. Both floors had similar patient loads (including acuity and number of patients). However, care providers differed by location—on an alternating monthly schedule, NPs and PAs provided care to half of the unit while residents provided care for the other half. An attending physician oversaw both groups and was present during morning rounds. The second site, CT-ICU (B), had 12 beds that were managed exclusively by NPs and PAs, who worked under the supervision of attending physicians. Both ICUs shared a common pool of attending physicians.

In both ICUs, a combination of verbal, electronic, and paper-based data was used for gathering and organizing information before morning rounds. While other clinical personnel (fellows and medical students) were sometimes involved in the aggregation of patient information, NPs, PAs, or residents performed a majority of this work.

Participants

Fifteen participants were recruited from an available pool of clinicians who worked at the ICUs during the study period (n=5 participants in each clinician group; n_total=15). The recruited participants were shadowed (see details in the data collection section) on a following shift that they worked. In terms of their information seeking for patient care activities, residents, NPs, and PAs performed similar tasks and had similar responsibilities—including collection and aggregation of information, updating records, and performing patient care activities. The institutional review board of the university approved the study, and verbal consent was obtained from all participants.

Data collection

Fifteen clinicians (n=5 in each group) were shadowed in order to collect data on their information seeking activities on 96 patients (N_Res=37, N_NP=24, N_PA=35), over 15 sessions, spread over a period of 3 months. Of the 15 data collection sessions, eight were conducted in CT-ICU (A) and seven in CT-ICU (B). A trained researcher with a doctoral degree in anthropology collected all data (LKJ).

Observation

Before the shadowing sessions, preliminary observations were conducted for 10 h to develop an understanding of the workflow activities and to create an exhaustive list of information sources used during information seeking. Field notes and informal discussions were used to characterize the sources of information that were being utilized. Sources were then grouped into six mutually exclusive categories including electronic and paper documentation, imaging systems, patient exam, verbal interactions with clinicians and patient family, and personal sources of information (table 1). Subcategories that were used with greater frequency were further defined, and less frequent sub-sources were dropped from the initial list.

Table 1.

Various information sources that were used during the information seeking process

Information sources	Description	Sub-sources
Electronic records (A)	Patient information that is recorded on electronic health records	Powerchart (Labs, IView), Emtek (vitals, respiratory)
Paper records (B)	Patient information that is recorded on paper charts	Progress notes, census sheet, printed EKG
Imaging systems (C)	Electronic images such as X-rays, ultrasounds	Centricity, Siemens
Verbal interactions (D)	Interactions with clinical colleagues and other individual such as pharmacists and patient family to gather patient related information	Consultations, teaching, family consultation
Patient exam (E)	Sources that the provider used during a physical examination of the patient	Exam, ventilator, drips, heart monitors
Personal sources (F)	Personal sources that were used for retrieving or verifying information	Reference books, websites and other smartphone-based apps

Shadowing

Shadowing was the primary method of data collection. Shadowing involves a researcher collecting data on a single participant and closely following that participant over an extended period of time. We have previously utilized shadowing techniques with success to collect data on physician activities,⁴⁰ physician workflow,⁴¹ interruptions,⁴² handoffs,⁴³ and information seeking.²⁴ In this study, we shadowed clinicians (residents, NPs and PAs) to capture data on the sources of information they used and the time spent collecting information from these sources.

For each shadowing session, the researcher would follow a participant, capturing the information sources the participant used and the time he or she spent using these sources. An iPad-based application, UObserve,⁴⁴ was used to record the data through a single-click mechanism to capture the start and end time of usage for each source of information (eg, Emtek) using a pre-defined list of sources and sub-sources. Additional field notes were also collected. Clinicians were shadowed until data on all patients were collected or until the beginning of clinical rounds. Shadowing sessions that significantly varied from anticipated workflows due to death of a patient, unforeseen patient events (eg, a patient resuscitation event; sudden cardiac arrest for a patient), or incomplete sessions due to grand rounds were excluded from this data set (three such sessions were removed from the study). Each session lasted approximately 4 h, and there was approximately 60 total hours of shadowing data.

Data analysis

Shadowing data was used to identify the differences in the information seeking strategies between residents, NPs, and PAs. As there was no previous research on the process of information seeking behavior or on the differences in the information seeking behavior among clinicians with different training, we utilized an exploratory data analysis approach.⁴⁵ Exploratory data analysis relies on summarizing data, through an open-ended and visually oriented transformation of data, without using a priori hypotheses or theoretical statistical models.

In order to characterize the strategies that clinicians used for information seeking, we used a temporally oriented exploratory sequential analysis method. Sequential analysis approaches have been used to identify temporal co-occurring patterns of human interaction.^46–52 Such an approach provided a viable mechanism not only for tracing the nuances of interactive behavior, but also for capturing repeated patterns of human behavior for contextually driven, insightful design or training interventions. The purpose of the sequential transition analysis was twofold: first, given the temporal nature of the information seeking activities, transitions between information seeking events provide a basis for characterizing the activities that contribute to the clinician's information seeking strategy. Second, it provides an efficient approach for characterizing and visualizing (eg, through transition figures) patterns of information seeking activities.

Time-stamped data included the accessed information source (eg, electronic source), sub-source (eg, Powerchart), specific aspects of the accessed sub-source (eg, labs), patient identifier (eg, patient bed number), and time spent using a source. A custom statistical processing application was developed for retrieving time-stamped data, organizing it into temporal event streams and transition probability matrices (TPM), and performing comparative sequential analysis.

Data organization and synthesis of sequences

First, data was converted into event streams. We considered the smallest unit of access as an event (ie, the sub-source that was accessed, eg, iView). The related sub-source and source (ie, Powerchart, Electronic Records) were automatically linked using cross-referenced event streams. Each of these streams was generated for each type of clinician (ie, resident, NP, and PA). The event sequences provide a chronological sequence of information source use. For example, a sequence EACD is a sequence of four information source use events: E (patient exam) → A (elec. records) → C (imaging systems) →D (verbal interactions). Event streams were used for computing number of patients attended, time spent for each patient, time spent on each source access, and frequency of source access. For this paper we used only the event streams related to information source access and their use.

Transition probability matrices

The next phase involved converting the event streams into TPM. A TPM provides a frequency of counts of transitions from one event to another. For example, for a TPM of information sources, each cell provides the frequency of transitions between two categories of sources (eg, frequency of transitions between electronic records and verbal interactions). A transition represents an instance where a clinician followed the access of an information source (eg, electronic record) with another information source (eg, paper record).

An information–source–use TPM was generated for every session, for each clinician, and then aggregated into a single matrix for each type of clinician (ie, one aggregated TPM each for residents, NPs, and PAs). The frequency of each transition was computed from the event streams generated in the previous phase of data analysis and was used to compute the probability of each transition.

Sequential analysis for commonly occurring patterns

We also used the sequential pattern analysis (SPA) approach to identify repeated patterns of information seeking activities. For this, we utilized a combination of the information source event stream and the TPM. Our SPA algorithm computed all 2-, 3-, and 4-step sequences that occurred at a greater than a pre-specified threshold value (set at 25%). We then identified the maximally repeating patterns—those sequences that did not have significantly repeating sub-sequences. For example, a maximally repeating pattern, ABD, would not be contained in any other longer sequences.

Results

The results are organized in the following manner. First, we provide an overview of the time spent on various information seeking activities. Second, we use transition analysis to characterize and visualize the probabilities of transitions between the information sources. This analysis is used to draw hypotheses regarding the information seeking strategies of residents, NPs, and PAs. Third, we utilize probability and pattern analysis to provide supporting evidence for the proposed hypotheses.

Time spent on information seeking activities

The number of patients attended (per day) by NPs was significantly fewer (M=4.8/day) than by either residents (M=7.0/day) or PAs (M=7.4/day). On average, residents spent less time on the information seeking activities per patient (M=88.3 s/patient/day; total time=3265 s) than either NPs (M=115.6 s/patient/day; total time=2775 s) or PAs (M=137.2 s/patient/day; total time=4803 s).

We also computed the total time spent on the various information sources. In general, residents and PAs spent more time accessing information from electronic records than NPs (Total_Res=1355.46 s, Total_NP=892.8 s, Total_PA=1397.3 s); PAs also spent more time on personal sources (eg, smartphones for searching for information) (Total_Res=71.6 s, Total_NP=0, Total_PA=540.6 s), verbal interactions with other clinicians (Total_Res=745.4 s, Total_NP=908.1 s, Total_PA=1404.9 s) and for patient exams (Total_Res=624.8 s, Total_NP=605.5 s, Total_PA=912.8 s) than residents or NPs. The time spent accessing paper records and imaging systems was fairly similar across the three groups of clinicians (see figure 1).

Figure 1.

Average time spent across the various information sources during the information seeking process among residents, nurse practitioners (NPs), and physician assistants (PAs).

In terms of the frequency of information access, we found that residents primarily used electronic records (48% of their total information access instances) in their information seeking process, followed by verbal interactions (24%), patient exam (17%), information from imaging systems (5%), and paper records (6%). NPs utilized verbal communication for information seeking (38%), followed by electronic records (31%), patient exam (21%), imaging systems (8%), and paper records (2%). PAs used electronic records (38%) and verbal communication (34%) as their main sources of information, followed by patient exam (21%), paper records (4%), and imaging systems (3%).

Transitions between sources and patients

Next, we used transition analysis to characterize and visualize patterns of information seeking activities. Based on the TPM, we computed the probability of transitions across the various sources. Figure 2 shows the transitions between the various sources across residents, NPs, and PAs. The size of the circles represents the frequency of use of a source, while the arrows depict the direction of transition from one source to another. The strength of the lines (dotted, thin, and thick) depicts the probability of a transition. We evaluated the within-source and between-source transitions across the three groups of clinicians.

Figure 2.

The switching between the various sources (depicted as circles) and their relative probability of transition (depicted by arrow direction and value directly above) is shown. The size of the circles shows the frequency of the use of a source; the thickness of the arrows shows the relative probability of switching from one source to another. For example, for residents (A), the probability of transitioning (ie, switching) from ‘imaging systems’ to ‘electronic records’ is 0.6. The square dashed lines show the prominent transitions from the residents and the physician assistants.

In terms of the between-source transitions, residents had three transitions (out of a possible 10 transitions) that had nearly 50% or greater probability of occurring (ie, greater than random chance): Imaging Systems → Electronic Records (0.6), Patient Exam → Verbal Interactions (0.53), and Paper Records → Electronic Records (0.49). In contrast, there was only one significant within-source transition: Electronic Records (0.72).

For NPs, only one between-source transition had close to 50% probability: Imaging Systems → Electronic Records (0.48). However, three (out of a possible four) within-source transitions were relevant: Paper Records (0.83), Electronic Records (0.58), and Verbal Communication (0.56). The other source, Patient Exam, was also approaching 50% (0.45). Transition patterns, in the case of PAs, were in between that of residents and PAs: there were no between-source transitions and only one within-source transition, Electronic Records (0.59) that was significant. Based on the analysis of transitions, we hypothesized that there are potential differences in the information switching (or use) sequences between residents, NPs, and PAs.

Information switching strategies

In order to evaluate the differences in the information source switching sequences, we computed the conditional probability of information source switching along with patient switching. Patient switching refers to the situation where a clinician chooses a different patient to gather clinical information during their information seeking process. For example, a clinician is gathering information on patient P1. After a set of activities, the clinician switches to patient P2 and gathers their information. Such transitional instances where a clinician switches from one patient to the next were categorized as patient switching events.

PS—patient is switched (or changed) by the clinician;

SS—information source is switched or changed by the clinician;

The conditional probability, P(SS|PS) gives the probability of information source switching, given that the clinician has already switched to a different patient. Lower values for P(SS|PS) would indicate more frequent switching between patients than between information sources—a pattern that would indicate information seeking for multiple patients on the same (or similar) type of information (eg, looking up X-ray information for several patients at a time). We call this strategy a source-based information seeking strategy. In contrast, higher values for P(SS|PS) indicate a patient-based information seeking strategy in which information from multiple sources is gathered for a single patient. We found that the conditional probability, P(SS|PS), was highest for residents (p=0.58), followed by NPs (p=0.37) and PAs (p=0.34). This provides potential evidence for a predominantly source-based information seeking strategy by NPs and PAs, and a predominantly patient-based information seeking strategy by residents.

Commonly occurring sequences

In addition to the transitions between information sources, we also identified maximally repeating sequences—that is, those sequences that do not appear in any longer sequences. We found that longer sequences occurred during the residents’ information seeking process, providing possible further evidence of their patient-based information seeking strategy: Verbal Interactions—Patient Exam—Electronic Records (34%) and Verbal Interactions—Electronic Systems—Imaging Systems—Electronic Records (25%). In other words, residents demonstrated an information seeking strategy predicated on aggregating information for a single patient at a time. In contrast, we found no support for longer sequences among NPs or PAs (see table 2), potentially showing a more distributed and fragmented, source-based information seeking approach.

Table 2.

Percentage support for each of the commonly occurring sequences

	% Support
Resident
Verbal—patient exam—elec. records	34
Verbal—elec. records—imaging systems—elec. records	25
Nurse practitioner
Patient exam—verbal	28
Elec. records—verbal	27
Physician assistant
Elec. records—verbal	30
Verbal—patient exam	29

Discussion

Based on an exploratory analysis of information seeking strategies of residents, NPs, and PAs, we found that the likely predominant mechanism of information seeking for residents was a patient-based strategy in which all relevant information for each patient was aggregated from multiple information sources. In contrast, NPs and PAs were likely to use a source-based strategy in which information on multiple patients was gathered from individual sources. Given the limited literature on information seeking strategies or on the differences in information seeking strategies between professionals, our approach was inherently exploratory—drawing on an iterative analysis of temporal information seeking activities. While it is beyond the scope of this paper to compare the effectiveness of these approaches in terms of patient or performance outcomes, it is important to characterize the implications of these strategies for critical care work activities, given their impact on case management, clinician cognition, and workflow.

In the rest of this section, we explicate the cognitive effects of these two strategies, while acknowledging that external influences such as clinical training, differences in skills, and knowledge of the information systems may have an influence on them. While further research is required to establish the effects of the strategies on clinical or process outcomes and other external influences, we discuss the cognitive underpinnings of each strategy based on observed differences.

Patient-based information seeking

The patient-based information seeking strategy of residents relied on a systematic process of aggregating information about each patient under a resident's care. Such a strategy was predicated on developing a detailed understanding of each patient by capturing the patient's current status and changes in conditions, and developing a detailed assessment and plan of care. A systematic process of this nature is predicated not only on developing a holistic picture of a patient's conditions, but also on identifying latent problems that are unlikely to be revealed in an overview-based approach. The strategy is also dependent on following a structured process of information seeking whereby available sources of information are utilized with less reliance on other care providers (eg, nurses).

While it is expected that such a strategy is potentially comprehensive in terms of patient diagnosis and management, this bottom-up approach to information seeking has potential disadvantages, including over-reliance on recorded information. Given the structured process of information seeking and the reliance on recorded patient information, this strategy is likely to require cognitive support tools that aid the clinician in sifting through the information, and highlighting critical information leading to effective diagnosis and assessment.

Source-based information seeking strategy

Source-based information seeking was predominant among NPs and PAs (more in the case of NPs than PAs), and was likely aimed at developing a global view of the ICU—that is, an overall status of all the patients in the ICU. Such an approach is especially useful in rapidly identifying those patients who have certain critical clinical parameters as outliers, or those who are trending towards being outliers (eg, identifying the sickest patients based on their blood gas levels or those that have an increasing heart rate over the last 24 h). In other words, the source-based approach is predicated on a top-down reasoning approach, in which sicker patients requiring more attention are identified based on a set of relevant clinical parameters (eg, lab results), thereby assisting the clinician in allocating his or her time and resources appropriately. Similar models, referred to as portfolio models,⁵³ have been purported as efficient mechanisms for clinician resource allocation of patient care activities. NPs and PAs using this strategy seemed to rely on informal communication with other clinicians (eg, nurses) to complement their understanding of current patient conditions.

Given the reliance on overview-based information (eg, graphs), a source-based strategy is likely to be effective while using enriched external representations such as graphs or images that provide an easy mechanism for visualizing clinically relevant parameters across several patients. This strategy becomes cognitively taxing and often unsustainable when the supporting evidence is in text formats (eg, notes). Additionally, a global strategy that focuses on identifying outliers increases the cognitive load on the clinicians for diagnostic reasoning about individual patients, potentially leading to a fragmented understanding of a patient's condition. While it can be argued that such an understanding may not be necessary in the case of every patient, it increases the likelihood of missed information.

Effects of task switching on cognitive load

Although information-seeking strategies of residents, NPs, and PAs involve frequent shifts (ie, switching) between tasks, their cognitive effects are fundamentally different. Prior research on task switching in psychology and cognitive science has characterized a phenomenon—referred to as switch cost—whereby responses tend to be slower and often more error prone immediately following a task switch.^{54 55} While we did not explicitly measure or model the switch cost, we describe the potential switch cost implications for each strategy. In the patient-based strategy, residents focused on developing a complete understanding of patient status, drawing on multiple sources. In this case, each ensuing source is considered as part of a continuum of tasks (referred to as a ‘task set’) and as such is not involved in task switching within each patient (ie, minimal switch cost).

Source-based strategy involves task switching at each change of source, thus increasing the switch cost. This switch cost is likely to increase the cognitive load and the potential for errors. Though we did not explicitly measure the actual switch costs or the errors, it is interesting to note that task switching strategies are potentially tied to the level of expertise of the clinicians; residents, given their limited expertise, relied on a strategy that minimized their cognitive load by reducing the number of task switches. Meanwhile, NPs and PAs, given their longer tenure in the ICU and clinical experience, successfully utilized a more cognitively demanding strategy that provided them with a better understanding of the overall status of the entire ICU. The decreased per-patient information seeking time for residents (M_Res=88.3 s/patient/day), as compared to NPs (M_NP=115.6 s/patient/day) and PAs (M_PA=137.2 s/patient/day), can be potentially attributed to decreased switch costs.

While further research is required to investigate the causal underpinnings of residents’, NPs’, and PAs’ information seeking behavior, including their educational and training influences, it is important to acknowledge that differences do exist, and that these differences can potentially have significant impact on ICU management and workflow.

Study limitations

We would like to point out several limitations of our study. First, this study was conducted with a small sample of participants (five clinicians in each group). Nevertheless, we gathered uninterrupted, detailed data on the information seeking activities of clinicians for 96 patients, giving a more robust sample for analysis. Second, though the overall task responsibilities for residents, NPs, and PAs were similar, there are some inherent differences: residents are viewed as trainees in the ICU with limited clinical experience (usually 2–3 years after medical school), and have short-term rotations in specific ICUs. In contrast, NPs and PAs are assigned to ICUs and have a longer tenure within the same ICU. Therefore, given their trainee role, residents had an added impetus to have a comprehensive understanding of their assigned patients, which could have potentially led to their structured, patient-based information seeking strategy. Third, we did not evaluate the effects of information seeking strategy on patient-related outcomes (eg, mortality) or other efficiency measures (eg, completeness of information). This was primarily due to the exploratory nature of the study and the limited prior work on the nature and structure of information seeking practices. Fourth, the differences in strategies are also likely because of the differences in education and training—residents rely on medical school training; NPs follow a nursing model of training; and PAs have a part of their training in a medical school environment. Further research is needed to establish the effects of educational background on clinical work practices related to information seeking. Fifth, it is possible that the presence of a researcher shadowing the clinicians may have had some influence on the clinician behavior. In order to avoid this, the researcher (LKJ) spent a significant amount of time at the research sites acclimatizing with the environment and possible participants prior to the start of the study. As such, we believe, the observer effect may have been minimal. Sixth, while efforts were taken to ensure similarity between the two sites where data were collected, the use of two ICU sites may also have had some effect on the information seeking practices. Finally, the fewer number of patients seen by the NPs (than residents or PAs) during the data collection days may have contributed to their identified strategies. However, we had conducted extensive observations prior to, and after the data collection where we observed the strategies that were reported in this paper.

Conclusion

One of the barriers for effective decision-making for clinicians in an information-intensive critical care environment is the difficulty in finding and retrieving the necessary patient-related information. Within the constraints of the clinical environment, clinicians develop processes and strategies that they believe optimize their information seeking efficiency, and increase performance outcomes (eg, quicker and better decision-making). Based on a comparative study, we showed inherent differences in the predominant strategies utilized by residents and affiliate providers. These differences have implications for design of health information technology that supports clinicians’ activities, developing cognitive aids to managing memory overload during work activities, and the management of ICU workflow.