Towards a Hybrid Method to Categorize Interruptions and Activities in Healthcare

Summary

Objective

Interruptions are known to have a negative impact on activity performance. Understanding how an interruption contributes to human error is limited because there is not a standard method for analyzing and classifying interruptions. Qualitative data are typically analyzed by either a deductive or an inductive method. Both methods have limitations. In this paper a hybrid method was developed that integrates deductive and inductive methods for the categorization of activities and interruptions recorded during an ethnographic study of physicians and registered nurses in a Level One Trauma Center. Understanding the effects of interruptions is important for designing and evaluating informatics tools in particular and for improving healthcare quality and patient safety in general.

Method

The hybrid method was developed using a deductive a priori classification framework with the provision of adding new categories discovered inductively in the data. The inductive process utilized line-by-line coding and constant comparison as stated in Grounded Theory.

Results

The categories of activities and interruptions were organized into a three-tiered hierarchy of activity. Validity and reliability of the categories were tested by categorizing a medical error case external to the study. No new categories of interruptions were identified during analysis of the medical error case.

Conclusions

Findings from this study provide evidence that the hybrid model of categorization is more complete than either a deductive or an inductive method alone. The hybrid method developed in this study provides the methodical support for understanding, analyzing, and managing interruptions and workflow.

1. Introduction

Interruption of an activity is known to have a negative impact on human performance for physicians [1–7], registered nurses (RNs) [8, 9], pharmacists [10], pilots [11–13], nuclear power plant operators [14], software engineers [15, 16], and office workers [17, 18]. This information has been derived from observational studies and the analysis of error reports. In each of these cases an unplanned change in workflow occurred.

Introduction of information technologies has certainly contributed to an increase in interruptions in workflow. In healthcare for example in healthcare, the introduction of an electronic medical record (EMR) presents interruptions and changes workflow for physicians and RNs because of alarms, alerts, and pop-up messages. These features were previously not part of a paper medical record but now require that the physician or RN interrupt workflow to accommodate. The mobile telephone is yet another information technology that contributes to interruption in workflow for physicians and RNs. The physician or RN is instantly accessible and open to interruption by anyone who calls. As more information technologies are introduced into the clinical environment a need exist to understand the interruption in workflow. Understanding how interruptions contribute to human error has been limited because there has not been a standard method used to analyze and classify interruptions.

The systematic and unbiased review of non-numeric data such as field notes is a challenging process during the data analysis phase of qualitative research studies. These challenges are felt by those using qualitative research methods to study issues in health informatics [19–23]. Large volumes of notes are recorded during complex observations. The researcher is faced with the question of how to organize the data by developing a classification system a priori to the observation or allow the classification system to emerge from the recorded data [24]. The a priori method requires that recorded data fit a predetermined classification system. This deductive strategy may limit new categories from forming or force the data to be classified using an inappropriate category.

In contrast, an emerging classification scheme arises inductively in response to the data that has been recorded as recommended in Grounded Theory [25]. Grounded Theory was developed by Glaser and Strauss [25] to rigorously and systematically explore qualitative data. Grounded Theory encourages data analysis to begin as soon as data collection begins. Grounded Theory is based on two components:

• line-by-line coding

• constant comparison

Line-by-line coding involves the study of each individually recorded incident, and constant comparison is a flexible strategy of creating and integrating categories during the analysis of qualitative data. Each new observation is compared with each previously coded observation for fit. If a new category is discovered, it is easily added. The flexibility of the method is a positive attribute for a preliminary framework. The technique relies on the researcher’s skill and ability to order the data through the identification of concepts, themes, and patterns found within the data.

In this paper, a third method was developed. The method is called the Hybrid Method to Categorize Interruptions and Activities (HyMCIA). The HyMCIA involves the hybridization of a deductive a priori classification framework with the provision of adding new categories discovered inductively in the data using Grounded Theory. The HyMCIA method uses both deductive and inductive processes to analyze qualitative data. HyMCIA was tested by analyzing data collected during an ethnographic study of healthcare professionals working in a Level One Trauma Center. The purpose of the ethnographic study was to observe, first-hand, healthcare professionals as they performed their usual duties in order to understand the tasks they performed as well as interruptions in workflow. A Level One Trauma Center was the chosen site because such a dynamic environment is intense, life-critical, interruption-laden, and stressful. Understanding the effects interruptions have on such an environment is important for improving healthcare quality and patient safety. This paper describes how to develop a method for categorizing for activities and interruptions with an explanation of as to why a particular level is appropriate. We test the validity and reliability of the categories by categorizing a medical error external to the study.

2. Methods

2.1 Developing a deductive framework to categorize activities

A series of interviews were held at the study site with domain experts in Emergency Medicine from the study site in order to obtain a list of activities performed there. A physician and two registered nurses (RNs) participated in developing an activity list for the Level One Trauma Center. The physician and RNs verified the list of duties for completeness. The preliminary list of duties performed by healthcare workers is shown in Table 1.

Table 1.

Self-reported activities performed by physicians and RNs in a Level One Trauma Center

Activities Performed by Physicians	Common Activities	Activities Performed by RNs
Dictating history and physical
Physician-specific issues		Start IVs Draw blood Perform treatment Data entry Patient and family education Assist physicians
	Communication Patient assessment Documentation

In Table 1 self-reported specific activities performed by healthcare professionals are identified. The physicians’ column contains duties performed only by physicians. The RNs’ column lists those duties performed by only RNs. The center column presents duties that both physicians and RNs perform. Table 1 was used as a deductive preliminary framework of activities from which to begin coding the field notes.

2.2 Forming an inductive framework to categorize interruptions

The workflow in healthcare or other settings can be described and analyzed at various levels of granularity forming a hierarchy. For example, we can describe workflow at a very high level of goals: take care of patients, ensure quality of care, monitor medical errors, etc. Or we can describe workflow at a very low level of motor actions: move index finger to “u”, then move the mouse to the icon “print”, then click, and so on. Both of these levels are relevant to specific purposes. But for the study of interruptions and their impact on clinicians’ performance, they are not the right level. What is needed is a level that has the basic properties relevant to meaningful performance in workflow. This is the level of activities. We call the abstract level “Superordinate Level”, the detailed level “Subordinate Level”, and the inclusive level “Basic Level” [26] .

This hierarchy has been established in the cognitive studies of concepts and categories by deductive methods. The study of categories has focused primarily on tangible objects with physical attributes. An object denotes an entity such as person, place, or thing with specific characteristics. Rosch [26] maintains that categories are two-dimensional structures with a vertical and horizontal orientation. The vertical axis represents a three-tiered hierarchical structure:

• Superordinate

• Basic

• Subordinate

The highest level of abstraction is designated as the Superordinate Category. The category is the most exclusive, contains the most distinctive features of the concept, and has fewer attributes in common with other categories. In contrast, the Subordinate Category has the most typical attributes of an entity assigned to the category. The Basic Category is at mid-level in the hierarchy. It is the most inclusive level of classification, the level that is most useful, and the one most often used by people when designating a category. The Basic Level Category is an entity that is highly significant psychologically for various cognitive functions when dealing with concepts and categories. It is the level that is most informative, first learned, consistent across cultures, and which has a prototypical set of features and actions shared among members at this level [27]. For example, we could categorize “furniture” as an entity of the Superordinate Category, “chair” as an object of the Basic Category, and “rocking chair” an instance of the Subordinate Category.

Conversely, the horizontal axis corresponds to the internal structure of the category. This axis is characterized by the typicality of members represented in the category.

First and foremost, to be classified as interruption, there must be the intrusion of a secondary, unplanned, and unexpected task into the primary task. The recipient of the interruption must suspend the current task in order to perform the interruption task, resulting in discontinuity during the process of performing the primary task [28]. In contrast, an activity that is not interrupted does not incur the intrusion of a secondary task or discontinuity suggesting continuity in the performance of the activity.

Rosch [26] asserts that prototypes are the most typical entities and are perceived as having more attributes in common with other objects in the same category and less with things in another category. As such, membership in a category depends both on the degree of abstraction and the prototypicality.

The study of categories has focused primarily on tangible objects with physical attributes. An object denotes an entity such as a person, place, or thing with specific characteristics. Rosch [26] and other researchers [29–31] have extended the study of categorization to non-concrete entities such as events. Rosch extended the three-tiered hierarchical category formation to form categories for events using a top-down configuration. An event is “a discrete bound temporal unit” [26, p. 43]. In our current study we consider an interruption an event based on operational definitions of interruption published in the healthcare literature. From these definitions of interruptions, a three-tiered hierarchy as outlined by Rosch, begins to form. Definitions from the healthcare literature show that physicians and RNs perform activities within the context of a clinical setting. An activity is the most general and all-inclusive term used in the definitions and therefore belongs in the Superordinate Category. More specifically, the definition of interruptions, either implicitly or explicitly, states that the physicians or RNs were engaged in an activity prior to receiving an interruption. The activity in progress can be labeled the primary task and the interruption as the secondary task. Although we have previously labeled the activity in progress as the primary task and the interruption as the secondary task, the definitions from the healthcare literature do not use the phrase secondary task but instead use the concept interruption. Interruption becomes the label for the Basic Category. The Subordinate Category of interruption is shown as specific instances in which healthcare workers become recipients of an interruption in the form of an unplanned, secondary task. Rosch’s method is useful in the formation of categories but does not provide a specific strategy to analyze qualitative data such as field notes collected during observational studies. This requires an inductive approach such as methods found in Grounded Theory.

According to Glaser and Strauss [25], categories have properties or attributes as well as two levels of abstraction. At the lower level, categories emerge early in data analysis. In contrast, higher level categories tend to emerge later in the analysis. This indicates a bottom-up strategy for category formation. The two levels of abstraction suggest a two-tiered category hierarchy. The lower category level is similar to Rosch’s Subordinate Category while the higher level is analogous to the Superordinate Category. In this arrangement, activities would be represented at the more abstract level while specific instances of activities would be represented at the lower level. The two-level hierarchy does not accommodate Rosch’s Basic Category, which is used to represent a mid-level of abstraction. Because Rosch’s framework and Grounded Theory have a similar category structure and hierarchical arrangement, they significantly contributed to the formation of HyMCIA. HyMCIA is a more complete and flexible framework by which to study qualitative data.

2.3 Data acquisition

Data collection was accomplished using a convenience sample of five physicians and eight RNs who were shadowed during their scheduled shifts. Shadowing is a qualitative technique that involves following a person as they go about their activities. A total of 13 sessions provided sufficiently rich data for the analysis of work flow processes and interruptions. While following the target, the observer records the actions and interactions in which the target engages. In most cases there was no attempt to ask the target for clarification of the actions observed. This ‘interruption’ could lead to serious patient harm in a functioning ED. However, the target sometimes spontaneously offered clarification of actions and interactions. In this specific study, we limited shadowing to routine sessions of physicians and RNs who gave written informed consent. Activities were recorded in one-minute intervals and began once the subject had completed the informed consent. The observers, using Tablet PCs, recorded all observations on a semi-structured field note form. This form was developed in Microsoft Word^®, and was revised over six iterations using feedback from the observers and a domain expert in Emergency Medicine. The final iteration of the form was loaded onto Tablet PCs for use during data collection. An example of the form is shown in Fig. 1.

Fig. 1.

Data collection form

The Tablet PCs were equipped with handwriting recognition software. This feature supported direct data entry into Microsoft^® Word.

The observers typically worked in teams of two and used direct observation with note-taking for activities performed by the study subject. Two observers were used in most observations to maximize the capture of interruptions in the fast-paced environment. Observer 1 is an RN with 26 years’ experience in healthcare and is competent in human factors. Observer 2 is also a human factors expert with 6 years’ experience but has had no training as a healthcare professional. Observer 3 is an RN faculty member with dual expertise in critical care and human factors. Observer 4 is a doctoral student in health information sciences and is competent in human factors but has had no training as a healthcare professional. Each observer received 30 minutes training using the data collection tool prior to beginning the actual data collection. The observers recorded their data independently and did not interact with each other for the purpose of clarifying observations just as they had, in general, refrained from interacting with the study subjects. This allowed for independent data capture which was used for inter-rater reliability at a later time. The ED staff provided a 30-minute orientation to acquaint the observers with the ED.

2.4 Analyzing the data

Sixty hours and fourteen minutes of field notes had been collected by shadowing eight RNs and five physicians working in a Level One Trauma Center. The field notes had been recorded by observers who entered a beginning and ending time for each activity performed by the healthcare professionals during a scheduled shift. Analysis of the data occurred concurrent with data collection. The process of data analysis was supported by using NVivo^© [32]. The software supports the analysis of non-numeric data or data that is not easily transformed into numbers. The software is useful in identifying themes, trends, and patterns in the data through the creation of categories and associated attributes. The data analysis protocol is described in the following section.

2.5 Data analysis protocol

The following protocol was used to analyze the field notes collected in the ED:

• 1. Each observation session that had been recorded as a Word^® document was converted into a rich text format (RTF) document.

• 2. The RTF document was imported into an NVivo^© [32] project.

• 3. An example of an imported observation in NVivo^© [32] is depicted in Fig. 2.

Fig. 2.

An example of a field note for activities performed by an RN working in the ED

This figure shows the activities performed by an RN working in the ED. The observers recorded observations on a nearly minute-by-minute basis.

• 4. Each time-stamped observation was reviewed for what activity had been performed, who performed the activity, if any other person(s) was involved and their role, the location where the activity was performed, and any other attributes that would make the category mutually exclusive from other categories.

• 5. The activity was identified and color-coded using the coder feature available in NVivo^© [32] which supports the use of 14 different colors.

• 6. Each attribute was identified and color-coded using the coder feature available in NVivo^© [32]. This process occurred for each activity that had been recorded in the field notes.

• 7. Each activity that was identified was compared to all previous observations as well as to the activity list to determine if that activity had occurred before or if it was a new activity. If it was discovered to be a new activity, it was added to the activity list.

• 8. Each time-stamped observation was analyzed to determine if the activity had been interrupted.

• 9. If the activity had been interrupted, it was categorized as an interruption. Each activity that was interrupted was color-coded.

• 10. Upon completion of all observations, the categories were reviewed, refined, and structured using Rosch’s three-tiered hierarchy.

3. Results

3.1 Hierarch of activities

A three-tiered hierarchy of activities was formed through the systematic study of field notes collected during the observation of physicians and RNs working in a Level One Trauma Center. The categories had been developed using a bottom-up approach, an inductive approach which is found in Grounded Theory [25]. A partial list of activities is shown in Fig. 3.

Fig. 3.

A partial list of activities performed by physicians and RNs

The specific activities were assigned to the lowest position in the hierarchy. They were designated as “Subordinate Categories”. A complete list of the Subordinate Categories is presented in Table 2.

Table 2.

A deductive - and – inductive - derived list of activities performed by physicians and RNs in a Level One Trauma Center

Activities Performed by Physicians	Common Activities	Activities Performed by RNs
	a priori Categories
Dictating history and physical
Physician-specific issues		Start IVs Draw blood Perform treatments Data entry Patient and family education Assist physicians
	Communication Patient assessment Documentation
Activities Performed by Doctors	Post Study Categories Common Activities	Activities Performed by RNs
Rounds Delayed communication Blocked communication Looks for Report Wait for Observe (i.e., present but not performing an activity)
	Interact with technology Clerical duties Teach Respond to (i.e., acknowledge) Provide information Receive information Perform procedure Request information Personal hygiene (i.e., hand washing) Delegate tasks Leave area Request assistance Retrieve supplies Break time
		Transport Discharge patient Set up room Safety Patient need Housekeeping Medicate Retrieve information Perform treatment Assist

In this study we have both deductively and inductively identified an extensive list of activities performed by physicians and RNs. The list of activities has been expanded from the initial self-reported list of activities and illustrates the usefulness of hybridization. The activities identified in this study are similar to those reported by Hollingsworth, Chisholm, Giles, Cordell, and Nelson [33]. The initial list of activities for this study has been updated to include some new activities such as interacting with technology, addressing safety issues, delegating tasks, leaving the unit, and delineating clinician communication patterns.

The identification of attributes facilitated making the category mutually exclusive. For example, the following properties were assigned to the category of Interacts with Technology and are shown in Fig. 4.

Fig. 4.

Category of Interacts with Technology

For an activity to be categorized as Interacts with Technology, the subject would have to be observed using a device such as an infusion pump or computer. From the field notes, additional data could be coded for the category including such attributes as the specific technology, the person using it, and its location.

Using a top-down approach, the Superordinate Category was formed by arranging the specific activities into a general category called Activity. This more inclusive category included any activity performed by a healthcare professional.

However, neither the Superordinate nor Subordinate differentiated as to whether an activity was performed with or without interruption. This separation is depicted in NVivo^© [32] as illustrated in Fig. 5.

Fig. 5.

Activities designated as interrupted and uninterrupted

The separation of activities into interrupted and uninterrupted led to the formation of the Basic Level Category. Specific instances of interruptions were organized into Subordinate Categories for interruptions. The Subordinate Categories for interruptions are found in Fig. 6.

Fig. 6.

Subordinate categories of interruption

The specific categories of interruption are:

1. Intended Recipient - the person to be interrupted.

2. Unintended Recipient - not the intended recipient of an interruption; i.e., receiving a phone call that was incorrectly dialed.

3. Indirect Recipient – the incidental recipient of an interruption; i.e., talking with a person who was interrupted that resulted in the suspension of the conversation.

4. Self-interruption – a person independent of another person suspends an activity to perform another; i.e., while walking stops abruptly and talks to another person.

5. Distraction – briefly disengaging from a task.

6. Organizational Design – disruption in workflow caused by flaws in the physical layout of the workspace.

7. Artifacts Not Available – disruption in workflow caused by a need to procure supplies and equipment not available in the workspace, thereby causing a disruption in workflow.

8. Initiator – the originator of an interruption.

Naming of the categories for interruption differs from that used for designating an activity. Some categories of interruption are designated by the role assigned to the subject in the interruption event. The Category Recipient of an interruption has the attributes revealed in Fig. 7.

Fig. 7.

The Subordinate Category of Recipient

The Subordinate Category of Recipient exemplifies the vertical and horizontal dimensions of a category. The category, Recipient, indicates the vertical dimension of the category and is assigned to the Subordinate Category in the Activity hierarchy. The properties of the category form the horizontal dimension. Both dimensions are required in order to categorize an interruption.

Two other types of interruption were identified while categorizing the data. First, there is Interruption by Organizational Design in which the physical layout of the workspace causes a break in workflow. An interruption of this type causes the physician or RN to leave their current workspace because completion of that activity must be carried out in another location such as the stat lab. All other activities they would perform are suspended until the person returns to their original workspace. A second type of interruption is known as Artifacts Not Available. Supplies or equipment are not available in the workspace. The recipient of this type of interruption must suspend the activity and retrieve the needed supply or equipment. These two new types of interruption finalize, up to this point, the categorization for interruptions.

4. Discussion

In this project we have extended the use of Rosch’s concept hierarchy to categorize activities and interruptions. Activity became the name for the Superordinate Category. This category made no distinction as to whether an activity was or was not interrupted. Clearly, additional specificity was needed to code and classify an activity. This resulted in the creation of a Basic Category which was used to classify whether or not an activity had been interrupted. This category was divided into those activities that were performed without interruption and those that had been interrupted. These categories were designated Uninterrupted and Interrupted. To code an interruption for more detail, specific instances of interruptions were identified and formed the Subordinate Categories.

One criticism of qualitative studies is the lack of generalizability attributed to issues related to validity and reliability. Generalizability can be enhanced by increasing the “representativeness” of the sample entities (participants) with respect to their parent or target populations. Marshall and Rossman [34] argue that external validity can be achieved when the researcher uses the original framework to show how the data was collected and how concepts and models guided data collection and analysis. This process defines the theoretical parameters of the study. The theoretical parameters then become the criteria for determining generalizability to other studies.

Reliability in qualitative research differs from that of quantitative research. Marshall and Rossman [34] assert that reliability in qualitative research is achieved by keeping notes of the rationale that guided design, review of procedures, protocols, decisions, and retrievable data. To test the validity and reliability of the method and the categories developed in this project, we coded the following medical error reported to AHRQ WebM&M.[35].

A 55-year-old man with acute myelogenous leukemia and several recent hospitalizations for fever and neutropenia presented to the emergency department (ED) with fever and hypotension. After assessment by the emergency physician, administration of intravenous crystalloid and empiric broad-spectrum antibiotics, the patient was assessed by his oncologist. Based on the patient’s several recent admissions and the results of a blood culture drawn during the last admission, the oncologist added an order for Diflucan (fluconazole) 100 mg IV to cover a possible fungal infection. Because intravenous fluconazole was not kept in the ED, the nurse phoned the pharmacy to send the medication as soon as possible. A 50 ml bottle of Diprivan (propofol, an intravenous sedative-hypnotic commonly used in anesthesia) that had been mistakenly labeled in the pharmacy as “Diflucan 100 mg/50 mL” was sent to the emergency department. Because the nurse also worked in the medical intensive care unit, she was quite familiar with both intravenous Diflucan and Diprivan. When a glass bottle containing an opaque liquid arrived instead of the plastic bag containing a clear solution that she expected, she thought that something might be amiss. As she was about to telephone the pharmacy for clarification, a physician demanding her immediate assistance with another patient distracted her. Several minutes later, when she re-entered the room of the leukemia patient, she forgot what she had been planning to do before the interruption and simply hung the medication, connecting the bottle of Diprivan to the patient’s subclavian line. The patient’s IV pump alarmed less than one minute later due to air in the line. Fortunately, in removing the air from the line, the nurse again noted the unusual appearance of the “Diflucan” and realized that she had been distracted before she could pursue the matter with the pharmacy. She stopped the infusion immediately and sent the bottle back to the pharmacy, which confirmed that Diprivan had mistakenly been dispensed in place of Diflucan. The patient experienced no adverse effects presumably – he received none of the Diprivan, given the air in the line, the infusion time of less than a minute, and the absence of clinical effect (Diprivan is a rapidly-acting agent). Nonetheless, the ED and pharmacy flagged this as a potentially fatal medication error and pursued a joint, interdisciplinary root cause analysis, which identified the following contributing factors: (i) Nearly 600 orders of medication labels are manually prepared and sorted daily; (ii) Labels are printed in “batch” by floor instead of by drug; (iii) The medications have “look-alike” brand names; (iv) A pharmacy technician trainee was working in IV medication preparation room at the time; and (v) The nurse had been “yelled at” the day before by another physician - she attributed her immediate and total diversion of attention in large part to her fear of a similar episode. (Reprinted with permission of AHRQ WebM&M).

In this example, the nurse performed a number of activities. The activities were color-coded following the previously determined rules. The activities categorized in this case were consistent with activities we have identified at the Subordinate Level such as communicating, assessing, observing, interacting with technology, and medicating.

No new activities were identified in reviewing the error report. This shows the utility of the method that we have developed to categorize activities. It also shows that the appropriateness of the level at which the activities are categorized.

In this case, the nurse was the direct recipient of an interruption. This interruption was assigned to the recipient category and color-coded following the previously determined rules.

A second concern of qualitative research relates to small sample size. Typically, sample size in a direct observational study is limited to between five and ten subjects. This number of subjects has been determined to provide an adequate amount of data. According to Glaser and Strauss [25], because data analysis occurs concurrent with data collection, data analysis can be stopped when no new categories are found during data analysis. In this study, no new categories were found after shadowing eight RNs and five physicians.

Ultimately, we have developed a hybrid method to systematically categorize activities and interruptions. An a priori framework, a pre-determined classification method, was used to order recorded data for activities performed in the ED. The framework was developed deductively through a review of the literature and interviews with domain experts. Additional categories of activities were identified inductively through the analysis of field notes. Specific activities performed by the clinicians were assigned to the Subordinate Category in the Activity hierarchy. The specific category name indicated an action being performed. These specific duties could be grouped at the Superordinate Category level to indicate any duty performed by the clinician. The Basic Category was subdivided into Interrupted and Uninterrupted Activities. The labels assigned to designate interruptions indicated the role of the person involved in the interruption except for Interruptions by Organizational Design and Artifacts Not Available. All the categories were deemed to be mutually exclusive. The validity of the categories identified in the study was tested using a medical report in which a nurse was interrupted. All activities and interruptions could be classified using the previously developed categories. No new categories for activities or interruptions were identified. Although HyMCIA was developed by analyzing activities performed in the ED, it is expected that it can and will be extensible to categorize activities and interruptions in other departments within a healthcare setting.

5. Conclusion

Understanding how new technologies such as EMRs and mobile telephones in the clinical environment contribute to interruption and changes in workflow requires systematic study. The method of data analysis of interruption and workflow developed in this study provides the methodical support to understand those changes. Based on those findings, strategies can be devised to reduce and mitigate the negative effects of interruptions.