Using observation to better understand the healthcare context

Abstract

Despite potential to capture rich contextual information, observation remains an underutilized data collection method. This paper provides a practical guide for using observation to understand complex issues in healthcare settings. Observation is a qualitative data collection method comprised of viewing and documenting phenomena in the usual environment. Drawing on our recent work using observation to better understand personal protective equipment use among healthcare personnel, we describe when to consider collecting data via observation, how to prepare and perform observation, and how to analyze resulting data. Observation data are documented in field notes that contain detailed information about setting, participants, and activity associated with the topic of interest. Field notes can be analyzed alone or triangulated with other types of data using theoretical or conceptual frameworks or by identifying themes. Observation is a valuable data collection method for health services researchers to identify key components involved in a topic of interest, a vital step in forming relevant questions, measuring appropriate variables, and designing effective interventions. Used with other methods or alone, observation yields the detailed data needed to address context-specific factors across a wide range of healthcare research topics and settings.

Introduction

The complex processes involved in the organization and delivery of health care have been the focus of research for decades. While quantitative data remains central in healthcare research, qualitative methods are increasingly used to better understand complex healthcare issues. Data collected through semi-structured interviews and focus groups are now commonly found in the healthcare literature, including studies of provider behavior^1,2 and patient experience.^3,4

Qualitative observation is a means of data collection comprised of viewing and documenting what people do in their environment with the goal of understanding behavior in context.^5-7 The purpose of this paper is to encourage health services and other researchers to consider how observation can add to their study and demonstrate how to prepare for and conduct observation as part of a health services research study. We begin by providing a general introduction to observation and why to use it as a research method. Then, we present our pragmatic approach to developing an observation protocol, training observers, and employing a systematic process for using observation in a healthcare setting using our own study in which non-participant observation was used to better understand and characterize the circumstances surrounding personal protective equipment (PPE) (i.e., gowns, gloves, masks, and eye protection) use by health care personnel (HCP) in the hospital environment. While the precise content of observation training may vary based on research topic, question, and goal, we also present the training that we provided to our research staff as an in-depth example and practical guide on how to train observers—an area of the literature where a gap is noted.

Why use observation as a research method

Historically, observation has been used by anthropologists seeking to understand particular social phenomena through enthnographic research.⁸ The range of factors (e.g., personal, environmental, temporal, and organizational) that can be captured by observing a phenomenon as it is performed in context provides rich, empirical knowledge and results in detailed data about the topic or activity of interest.⁹ Moreover, usual behavior performed in the usual environment by the usual participants contains invaluable contextual information that may not be identified in a simulated scenario¹⁰ or may contain nuance not captured using other methods such as interviews. Often, people perform tasks and activities that are so routine and habitual that they do not recall their specific actions. The highly detailed information garnered through observation can be used to achieve important health services research goals by identifying exemplary practices, intervention points, and process improvement opportunities.¹¹

Observation should be considered when the research question involves developing a detailed understanding of an event, activity, process, or outcome of interest. Specifically, observation should be considered when establishing a descriptive account (e.g., step-by-step description), identifying factors that may affect its performance or outcome, or characterizing the surrounding patterns of behavior (e.g., common or divergent practices). In addition to capturing what comprises, affects, and produces the activity, process, or outcome of interest, observation can be used to characterize how and how often it occurs. For example, healthcare organizations often structure work through protocols. However, most protocols are not flexible or nuanced enough to fully reflect all of the factors HCP must consider when performing tasks. Observation allows a better understanding of how work is actually performed. Further, observation can be used to explore how tacit and explicit knowledge influence HCP behaviors while performing their work. For instance, HCP may engage in behavior they tacitly know will get the work done, but at the same time explicitly know that they are not following protocol. While observation alone may not provide a full understanding of a behavior, it allows these sorts of behavioral patterns to be uncovered along with the circumstances or context in which they may occur.

Observation is the ideal tool to capture detailed, context- rich information that is difficult to discern through other methods. With some consideration of appropriate use for the research question and analysis goals, observation data can be used in various ways and throughout a study. It can be used in the beginning to familiarize researchers with the environment/context and build rapport with potential participants. Observation is a powerful formative research tool not only early on, but throughout a study, as important physical, organizational, and interpersonal factors may be identified during these early observations that produce and situate findings, lead to modifications of study aims, or are used in the development of other data collection tools, such as interview guides or surveys. Using observation to inform interview guides and surveys allows researchers to ask more specific and context-relevant questions.

Observation data can also be used throughout a study to describe processes, experiences, and areas for intervention, as well as to improve practice, further organizational goals, and develop context-specific interventions. One way observation data can be used to improve practice is to report findings to the participating groups. Depending on the needs of the group, findings can be delivered in different ways, including written reports and summaries, meetings with key staff, and formal presentations.

Observational data can be analyzed to develop findings related to the research aims. It can also be analyzed in conjunction with other types of data, including data collected through other observational methods (e.g., shadowing), self-report methods (e.g., interviews and focus groups), or more structured methods (e.g., surveys). Triangulation through multiple methods can be essential to understanding participants’ views and experiences within specific settings.

How to use observation from start to finish

Once the decision has been made to use observation, a number of additional decisions then need to be made to determine specifically how observation will be used to achieve the goal of the research. In this section, we discuss these considerations, including observer type and role, data collection tools and approaches, observer training and data collection, and data analysis.

Our study: Observing PPE use among HCP

To provide a practical example, we present our experience using observation in a study designed to better understand the circumstances surrounding PPE use by HCP in the hospital setting.¹² The objective of our study was to better understand how HCP used PPE while caring for patients in the hospital setting and what factors influenced their behavior and decision-making. Our study took place at two acute care hospitals with non-participant observation conducted on medical/surgical wards, intensive care units, and an emergency department. The goal of observation was to characterize the contextual and behavioral factors that affect PPE use among HCP while caring for patients under contact and droplet precautions. The observation methods we describe apply specifically to our research aims, however, would be applicable in most healthcare settings and can be modified and used as a guide for other research projects.

Observer type and role: Participant and non-participant observation

Observers will interact with participants and the environment to some extent by their presence; however, researchers will need to decide between participant observation and non-participant observation.^13-15 Participant observers engage directly in the activity of interest. ^16-18 This approach should be considered when the goal of the research involves performing a task or activity in order to understand insider experience of an event, activity, or process.¹⁹ For example, in the context of our PPE study, a participant observer would have directly engaged in patient care by performing activities such as patient repositioning, taking vital signs, delivering medications, and documenting not only what they observed around them, but also their experience of donning/ doffing and providing care while wearing PPE.

Because covert methods may be needed to gain access to insider information, there are major ethical considerations involved in participant observation.¹⁹ In particular, if research personnel conceal their identity and the fact that they are conducting research in order to gather insider information, they have failed to obtain informed consent, therefore, violating medical research ethics.

In addition to ethical considerations, participating in the activity of interest also presents a pragmatic data collection challenge in which the participant observer cannot fully participate and record observation data simultaneously. The resulting data are highly dependent on memory, as documentation must be completed at a later time. For a portion of our observations, research staff donned PPE and entered patient rooms with HCP. Many observers noted partial or total distraction from observing, inability to document in real time while donning/doffing, and confusion about how to use PPE despite training. These challenges to observation would be exponential during the performance of patient care activities.

In the context of health services research, advanced training and clinical knowledge may be required to engage in participant observation. While exisiting knowledge can be useful to inform data through an expert lens, paradoxically, substantial familiarity may result in compromised data if details perceived as standard practice are omitted.^16,20,21 Further, while content expertise and direct engagement in the activity of interest allow the participant observer to interpret events or explain the motivation behind behavior, it may also interfere with the collection of impartial or non-confirming descriptive data. Considering the potential effects of their involvement in participant data collection, content experts can be instrumental in other parts of the study, including research question development, study planning, observer training, and data analysis and interpretation.

In contrast, non-participant observers accompany natural participants as they perform the activity of interest without engaging in the activity themselves.^7,13,22 The non-participant observer’s singular role is to collect research data by documenting what they see and hear as the natural participants (e.g., HCP) perform the activity of interest in the natural environment.^6,15 Without direct performance of the activity or process of interest, the non-participant observer cannot provide a first-person description or insider experience, but can instead capture the all-seeing, fly-on-the-wall perspective. Non-participant observation should be considered when the goal of the research involves establishing a thorough understanding of an event, activity, process, or outcome, including how and under what circumstances it happens, who is involved, and what factors are at play as it occurs in the natural setting.¹⁵

Non-participant observation can be conducted following targeted training in the method, as described in this manuscript, and therefore, advanced training or expertise in the topic area is not necessarily required. As such, researchers may consider candidates with limited experience (e.g., students and junior research staff), potentially increasing the number of observers a research budget is able to accommodate. More important than budgetary prudence is the quality of the data collected by trained non-participant observers without content or context expertise. In keeping with the paradox that close familiarity may diminish data quality—as with expert participant observation— the eye of an outsider may be valuable in capturing meaningful detail noticed in the unfamiliar. For instance, a clinician may observe HCP as they perform a series of tasks and recognize that patient vitals are being taken. While the clinician’s interpretation is correct, a possible consequence of familiarity is that valuable description of the activity may be reduced, overlooked, or excluded. It would be inaccurate to suggest that a clinician could not document the details of familiar activity; however, they may be more apt to overlook the details that are familiar and known to them. A non-clinician observer who is unfamiliar with the task may be more likely to notice and document nuanced details that otherwise are considered “standard” by someone who is familiar with the process being observed. The excerpt below shows a non-clinician, non-participant observer’s description as a medical assistant moves around a patient room taking their vitals:

Medical assistant (MA) takes temperature: takes portable thermometer unit from the wall, inserts in box of probe covers, hands end to patient who places in mouth. MA holds thermometer unit until beeps, takes end piece from patient, ejects end piece cover into trash behind chair, replaces probe into unit, puts thermometer unit back on wall behind the bed. MA then checks BP: takes cuff out of wall basket, picks up patient arm and wraps cuff around it, rests patient arm on chair arm, then hits the button on the monitor. MA takes cuff off patient, returns to wall basket. MA then attaches the pulse ox to patient’s finger to check pulse and touches patient hand with both hands to adjust position. After each vital sign, MA reports to RN1 who records using the computer that is at the end of the room. (Enteric Site 2 Unit A Inside [Date] Room 1 [Observer Initials])

With a great deal of description, the non-clinician, non-participant observer provides a vivid and specific narrative of HCP interaction with both the patient and the patient environment. We can determine what equipment was used and where it came from, which surfaces were touched, and the order of task sequence. Through this level of detailed data, we can better understand what we’ve observed through later interpretation during analysis. The detailed description allows us to recognize when protocols and guidelines are followed, identify any potentially important deviations from recommended practice, and determine problems or issues that participants experience with the observed process. In contrast, had interpretations been documented instead of the details (e.g., takes patient’s temperature, takes patient’s blood pressure), we lose our clear, turn-by-turn description of the MA’s work in the context of the patient room as it was done. Further, important details that can be used to better understand process issues and potential solutions may be lost to summary or interpretation, rendering data useless in problem identification and solving.

As such, the expected level of description for data collection should be made clear to observers during training, and, therefore, does not preclude content experts as observers. Indeed, depending on the research question, a content expert may be needed to provide the necessary degree of technical information about the focus of the observation. Nonetheless, in many situations, highly detailed data collection can be achieved by those without content expertise with less risk of inadvertent loss of description due to familiarity.

Our point is that while participant observation can be effectively used to better understand insider experiences, non-participant observation may be better suited to health services research, given the capacity to capture a holistic account of the complex issues in equally complex environments. Thus, to develop an understanding of PPE use among HCP during patient care within the usual physical and organizational context – a complex issue with many contributing factors – we selected non-participant observation for our study.

Data collection tools and approaches: Structured, unstructured, or semi-structured field notes

In addition to the role of the observer, the research goal should also be used to determine whether data collection will be structured, unstructured, or semi-structured.^13-15 While classically presented as a dichotomy in name, a continuum exists between unstructured and structured data collection. ^15,23 On one end of the continuum, unstructured data collection is used to establish the “how” or “what” of the topic using field notes to broadly record details and description of the larger context. Structured data collection, however, often focuses on “if” or “how often” an activity or behavior occurs, which can be done using a checklist or template and result in quantitative data.^6,8,23-25 Between these two approaches exists what may be more appropriately called “semi-structured data collection,” in which elements of both unstructured and structured methods are present. A semi-structured approach can be used to capture information that is descriptively independent (e.g., precaution status of patient or unit type), but may influence the phenomenon of interest (e.g., use or non-use of PPE).

To demonstrate the differences, a structured data collection tool such as a checklist would be used to document when HCP donned gloves before entering patient room, while fully unstructured field notes would be used to document the details of HCP activity as they provide patient care. But, as we did in our study, semi-structured field notes would contain both a structured section for precaution status of patient and unit type as well as unstructured detailed narratives of HCP activity surrounding PPE behavior. By selecting this combination approach from early stages of our study, we were able to maintain a highly organized dataset, the benefits of which were felt during analyses of observation data both alone and together with focus group data. Choosing the appropriate documentation method is dependent upon the research question and the information needed to fully understand the phenomenon under study.

Observer training and data collection

After observer type and data collection approaches are selected, it is vitally important that observers are trained on the data collection method, the study, and how the data will be used to address the research question before entering the field. Training is paramount to ensure consistency in data collection and the performance of observation and ideally includes both didactic instruction and experiential training in the setting where the observation will be conducted (i.e., in the field). In planning our study, senior research team members performed several preliminary observations to get an idea of what should be documented to meet our research goals. We then developed a training plan based on our experience conducting these early observations.

We began with didactic training to introduce the research assistants (nursing and pre-med university students and professional research staff) to our study and to observation as a research method. Research assistants entered the field with members of the study team for experiential observation training on the hospital units where they gained experience interacting with participants and collecting data. Lastly, observers worked together and with members of the study team to finalize field notes and hone observation skills. Observer training remained ongoing throughout our study.

Didactic training

Our initial training session began with a presentation by the principal investigator on the purpose, background, and specific aims of the study. Next, our qualitative methodologist—a medical anthropologist—introduced observation as a research method, including what observation is, why it is used, by whom it is used and under what circumstances as described in this paper. Once the method and study fundamentals had been introduced, training proceeded with more specific instruction on how to conduct observation for the purposes of our study, including what to observe and how to document the data. Observers also received training on what to do in the field, including how to explain the study to participants, obtain informed consent, and identify rooms for observation. A field note guide outlining these data collection requirements was developed and distributed to research assistants at the time of training (see Appendix).

Next, our training focused on documentation of observation data, which is done during observation by taking handwritten notes that are later typed to produce finalized electronic field notes. Examples of de-identified handwritten notes from past observation work were presented to show the data in this primary stage. We then outlined the basic information that should be gathered for each observation in the structured data collection portion of the semi-structured field note. The content of this information header will vary by study, but typically includes date, time, observer, and location. We planned to look at our resulting observation data by unit type (e.g., medical and surgical, intensive care, and emergency), observer position (e.g., inside or outside room), and isolation precaution type (e.g., contact, enteric, or droplet), so these additional location descriptors were included in our header (Figure 1).

Following the header, observers were asked to include a brief description of the physical environment. Observers were asked to note signs posted, presence of supplies, trash containers, other furniture, and medical equipment. In addition to what was present, we also asked that observers note the state of the area and whether things were missing that should be there:

General overview: The room is located at the end of the hallway. Contact Precaution (green) sign posted on open door. This room is next to another Contact (green) Isolation room. The cart is located in the hallway between the two rooms. Cart appears to be well organized with two blood pressure cuffs, two boxes of gloves, and [an alcohol-based hand rub] bottle on top. Drawers are closed. There is also a [germicidal wipe] (purple) tub hanging between the two rooms. It has no lid. There is an empty, medium, plastic-lined open trash container between the cart and the wall. (Contact Site 2 Unit B Outside [Date] Room 1 [Observer Initials])

In addition to training observers on what basic information should be included in field notes, it is also important to specify the parameters of data collection, including what should not be documented. In accordance with our study protocol approved by the institutional review boards (IRBs) at our two data collection sites (HUM00112340; IRB-2016-339), personally identifiable information (PII) and protected health information (PHI) were not to be collected during observation. Patient names, room numbers, and PHI were not to be noted in any study documentation, including field notes. Collection of PII was also restricted for HCP. Instead of using names, observers were asked to refer to all participants by role (e.g., patient, registered nurse [RN], medical doctor [MD], physical therapist [PT]) or “unknown” if their role was not apparent. To distinguish between individuals when more than one participant of the same role was present, we asked observers to assign one-time numbers to multiples (e.g., RN1, RN2, RN3). Examples were presented during training to show observers how field notes should look without PII or PHI. “The patient begins talking to RN1 as he is looking at the screen and typing.” (Contact Site 2 Unit B Outside [Date] Room 2 [Observer Initials])

Figure 1.

Descriptive field note header.

Following the basics of documentation, training should focus on the main area of interest to be captured through observation. Given that the goal of observation in our study was to better understand the contextual and behavioral factors that affect PPE use among HCP while caring for patients under contact and droplet precautions, observers were asked to notice and document the details of HCP behavior from the time the HCP approached a precaution room through the time they left. All patient care activities performed by HCP both inside and outside the patient room were observed and recorded in detail. Specifically, observation field notes included who was present, performance of hand hygiene, if and how PPE was put on and removed (i.e., donned and doffed), care tasks performed, surfaces touched, equipment used and transported, interactions between HCP, and possible contamination points when unprotected contact occurred between the HCP and patient or patient environment.

After establishing the focus of observation, it is important to establish the level of detail observers are expected to document to meet research goals. While some studies may need only a high-level account, other studies may require painstaking detail for analysis that could not be achieved with a list of activities. We indicated to observers that a great deal of detail was needed for our study in order to perform our planned analysis. To demonstrate the level of detail needed, we presented field note excerpts with rich detail from preliminary observations collected by senior research team members. It may also be useful for observers to see examples of field notes that do not include sufficient detail. Below, we present two different ways the same observed activity can be documented. Example 1 demonstrates sufficient detail, and Example 2 demonstrates insufficient detail:

Example 1: MD1 pulls gown forward from the front, breaking the shoulder seals. MD1 then pulls the gown at the waist to break the tie. While MD1 uses the gown to remove her gloves; her hands touch outside portions of the gown. As MD1 crumples the gown, part of a tie falls onto the ground. MD1 puts gown in trash then walks outside, stopping briefly in doorway to speak to RN1. MD1 runs hands through hair then sanitizes hands with [alcohol-based hand rub]. MD1 walks away. (Contact Site 2 Unit E Inside [Date] Room 1 [Observer Initials])

Example 2:MD1 removes PPE and performs hand hygiene.

Example 1 provides a detailed narrative of this physician’s PPE removal method, room exit, and performance of hand hygiene. In contrast, Example 2 indicates that two tasks were performed: PPE removal and hand hygiene. The detailed description in Example 1 captures the physician’s movements as she leaves the room, an interaction with a nurse, and two potential self-contamination opportunities: one while doffing gloves with gown and one by touching hair before performing hand hygiene. Example 2 did not provide sufficient detail to meet our project goals as no additional information was available for further interpretation.

Experiential training

As was the case for our study, it may be necessary to provide content- or context-specific training to observers in advance of data collection. Because some observation for our study would require observers to enter precaution rooms with HCP, didactic training concluded and experiential training began with basic practical instruction from site infection prevention and control liaisons on when and how to appropriately use PPE.

After didactic training, study team members accompanied observers on their first few field visits, providing guidance on how to explain the study to HCP, identify appropriate rooms to observe, and obtain verbal consent from HCP and patients. Observers began by working in pairs to develop data collection skills before proceeding with individual observation. Each observer was provided with a plastic storage clipboard with attached digital clock (used to include intermittent time stamps), paper, and several pens. Because observers entered some precaution rooms with HCP, we also provided disposable plastic bags for covering clipboards. An on-the-unit protocol sheet was created, which included a list of study units, staff meeting times, how to determine the assigned nurse for precaution rooms, reminders and tips for checking in and interacting with unit staff, a list of PPE resources, and instructions for properly cleaning documentation supplies (e.g., notepad, pens, and clipboard) after observation inside a precaution room. Observers also carried study information sheets for interested HCP and copies of verbal consent language for reference.

Collecting observation data

At the beginning of each observation period, observers checked in with the unit nurse manager and clerk to explain the study and that they were on the unit to observe PPE use by HCP. Observers then identified a precaution room to observe by the posted precaution signage and introduced themselves to HCP that appeared to be working with the patient. Observers explained the study, obtained verbal consent from HCP, and addressed any HCP concerns, a sequential process which took less than five minutes. HCP who entered the unit during the observation period were consented as they arrived to care for the patient (e.g., MDs and rounding teams). The most common concern from HCP was that our observers were monitoring hand hygiene compliance. While compliance to protocols can be paramount to appropriate health care and patient safety, interventions focused on behavior change must be informed by an understanding of why behaviors happen in order to appropriately target factors that influence these behaviors. Observation conducted in a systematic and non-judgmental fashion is a valuable method for developing this type of understanding. As such, our observers addressed HCP concerns by explaining that the goal of our research was to learn from them the challenges HCP face when using PPE while delivering patient care in order to develop interventions that improve PPE use to promote patient and HCP safety, with no punitive component or intent to catch or expose non-compliant behavior. The context-specific information would be used to develop interventions and recommendations that would reflect their existing work processes, environments, and challenges.

Given that the data collected would contain no identifiable HCP or patient information, verbal consent from HCP was deemed appropriate by the overseeing IRBs. Therefore, HCP concern about compliance monitoring was further addressed as part of the informed consent process in which observers explained that PII would not be collected, observation data would not be provided to supervisors or leadership, findings would be reported in aggregate, and that the observer would be focused on objectively describing how PPE is used, without opinions on such behavior, including perceived level of correctness. Ultimately, refusal to participate was infrequent.

Verbal consent from the patient was also obtained before the observer entered a patient room. Observers were instructed to move on to another precaution room if HCP or patients declined consent and to leave the area if they felt they were in the way of HCP or if they were asked to do so. As with HCP, patient refusal occurred infrequently and was typically associated with patient acuity. Observation was not performed if patients were unable to provide consent.

Once consent had been obtained, observers situated themselves directly outside a selected room and recorded all activity using handwritten field notes. A separate field note was created for each observed room, beginning with a new informational header and brief physical description. If there was no activity at the chosen room, observers were instructed in advance to conclude observation after a few minutes, record lack of activity in the field note, and locate another room to begin a new observation. Otherwise, length of each room observation was left up to the observer, depending on room activity. As a result, the number of individual room observations an observer could conduct within a single observation period—two to three hours on average—ranged widely from one room to 10 rooms.

Participant interaction

Observers may encounter several kinds of interactions during data collection that vary in relevance to the topic of research. During the observation period, participants may interact with their peers (e.g., other HCP), individuals in different roles (e.g., patients), or directly with the observer. Important insights can be gained by observing participant interactions, whether or not the dialogue is directly relevant to the topic of research.

Interactions comprised of dialogue unrelated to the topic of interest may be described broadly in field notes to capture the interaction as an important part of the story even though it is not the main feature. For instance, the specific content of a clinical conversation between a nurse and patient is not relevant within the context of our study. However, the circumstances in which the conversation takes place may be important, as in the following example from our study:

9:06 am: RN1 entered room, does not gel [perform hand hygiene], stands ~1 foot away from the patient and asks questions. RN1 leaves and does not gel [perform hand hygiene] after. RN1 does not touch anything in the room. (Enteric Site 2 Unit B Outside [Date] Room 1 [Observer Initials])

Although the particulars of the conversation were excluded, important information about the context of the interaction was captured in the description of the nurse entering and exiting the room without using PPE or performing hand hygiene.

Further context for observed behavior can be gained by documenting content-relevant dialogue, including perceptions, motivations, and intent. As such, observers were asked to document interactions pertaining to PPE within our data collection parameters (i.e., exclusion of all PII and PHI), including but not limited to HCP teaching each other how and when to use PPE, ease or difficulty of use, and reasons why it was or was not used. In the following example, a group of two physicians and two medical students are observed as they approach the room of a patient under enteric precaution:

MD2 gels [performs hand hygiene], did not see anyone else gel before putting on gloves. All put on gowns over lab coats, did not see if used thumb holes [in gown wrist], then glove. MD1 tied [gown] in front, MD2 tied in back. MD2 notes sign near door. They discuss cleaning before gloves. MD1 states “I didn’t know we need to clean before putting gloves on.” (Enteric Site 2 Unit A Outside [Date] Room 2 [Observer Initials])

The conversation between physicians as they prepared to enter this precaution room revealed an incomplete understanding of the protocol for proper use of PPE when entering the room of a patient under precaution. This type of contextual information from a conversation among peers during the observation period can be used to identify potential intervention opportunities or important areas for further study.

Although not engaged in the activity, it is acceptable for the non-participant observer to speak to participants. Observers can ask clarifying questions of participants, and participants often offer explanations of what they are doing without prompting.^6,24 Participant dialogue may go beyond task-specific content, offering valuable insight into common challenges, workarounds, or institutional protocols relevant to their work. An example from our study demonstrates a participant explaining the reasoning behind their behavior to the observer:

3:24 pm: RN removes mask and discards it in the trash, stating, “I’m further than 3 feet away from the patient and am not facing him so I can take my mask off without risk of breathing in any droplets.” (Droplet Site 2 Unit C Outside [Date] Room 1 [Observer Initials])

Participants may go beyond providing an explanation for their behavior by also telling the observer what additional factors influence their behavior. In the example above, the nurse effectively explains their behavior by citing the protocol that supports their mask removal when turned away three feet from the patient. A researcher can then compare this explanation to the written protocol to see if they align. The following example goes further by providing an explanation for a medical student’s (MS) incomplete understanding of PPE protocol:

MS2 states that they have been told different things about following precautions and that when they are just going inside the room to talk to the patient and do not plan on any patient care, they do not have to wear PPE. (Contact Site 2 Unit D Outside [Date] Room 4 [Observer Initials])

If the medical student’s comment had not been captured, this observer may have noticed that the participant appeared confused as they prepared to enter the contact precaution room, but the reason for their confusion would remain unknown. The direct statement that they had received inconsistent information regarding what constitutes appropriate PPE use provides important context for not only the medical student’s behavior observed in this session, but also for general PPE use among HCP and education at the site.

Not observed vs. Not performed

A challenge observers may face in the field is a limited ability to observe and record activity. Missed activity can happen for a variety of reasons, including simultaneous activity, visual barriers, or because tasks may have been started or performed completely outside the observed space. When observation is performed in real time without the aid of video or audio recording, it is to be expected that even the most diligent observers will sometimes miss activity or details. In this situation, the observer should indicate in their field note that activity was not observed:

Example 1: RN approaches door (did not see donning of PPE or hand hygiene because view was blocked) and walks in (does not touch door) wearing a gown and gloves. (Contact Site 2 Unit B Inside [Date] Room 3 [Observer Initials])

In example 1, the observer was not able to see the process by which the nurse donned PPE, but we can determine that it occurred prior to entering the observed space because the nurse enters wearing gown and gloves. Although the nurse in Example 1 may have performed hand hygiene when donning PPE prior to entry, these activities were not seen, which the observer clearly indicated in the field note.

Example 2: RN1 walks into the room wearing no PPE to pick up a solid white tube then sanitizes it with the wipe outside of the room. RN1 sets the tube on the computer surface outside then sanitizes her hands. (Contact Site 2 Unit E Outside [Date] Room 2 [Observer Initials])

Although hand hygiene and the process of donning PPE were not observed in either example, donning of PPE was not only not observed in Example 2, it was not performed, as indicated by the nurse entering the precaution room without PPE. There is an important difference between activity not included in field notes because it was not observed (Example 1) and because it was not performed (Example 2). Observers should specify in field notes that activity was missed to clearly distinguish not observed from not performed.

Simultaneous activity is another reason an observer’s ability to see and document may be limited. In the hospital setting, bedside rounds performed by medical teams can bring up to a dozen people into the observation area. When a large group is present, choosing one or two individuals to focus on allows the observer to capture detailed data without becoming overwhelmed by all the activity. As with task not observed versus task not performed, it is important for observers to include in their field notes that they have made a decision to focus on one or two members of a group:

A large medical team on rounds approaches room. There are 12 members of the team comprised of MDs, medical students, and one pharmacist. This observation focuses on one MD from the medical team. (Enteric Site 2 Unit E Outside [Date] Room 1 [Observer Initials])

This observer’s annotation of their focused field note situates the context of the observed MD’s behavior as a member of a team and also, importantly, clarifies that while other HCP were present and noticed, the field note was purposefully focused.

Post-observation protocol

A post-observation protocol outlines what to do after each data collection session. The specifics of the protocol may vary by project and phase of data collection, but the overall goal is to preserve the content of what was observed in high-quality data. The protocol may be more involved during early data collection, with observers meeting to compare field notes or submitting field notes for feedback from other team members as they develop observation skills.

The primary feature of a post-observation protocol takes place in the time directly following observation, when observers type their handwritten notes into an electronic document. The product of hours of focused observation will be pages of handwritten notes that likely contain shorthand, abbreviations, symbols, and mnemonics that make sense only to the observer who took them. It is therefore vitally important for observers to finalize field notes as soon as possible after observation, when details can be recalled easily.^5,24At this time, the handwritten notes are fleshed out into highly detailed, timestamped narratives, which are the data that observation yields (Figure 2). Because observation itself can be tiring, it may be tempting to delay this finalization process, potentially compromising data due to diminished recollection. To encourage prompt finalization, our observers were asked to block one to two hours following time in the field to complete this required part of data collection.

Ensuring data quality

Our post-observation protocol included additional steps during early data collection to support newly trained observers as they developed skill and comfort with the method. During the first three weeks of data collection following didactic training, observation was conducted in pairs. In addition to the development of observation technique in the field, paired observation provides a natural comparison through review and discussion of field notes recorded separately while observing the same activity. When reviewing together, observers can identify strengths and weaknesses in their own notes, as well as learn how their observation partner perceived and recorded the same activity. This additional paired practice contributes not only to enhancing observer skill, but also to improved data quality.

Figure 2.

Field note from contact precaution room observation.

During the paired observation period of our study, observers typed their handwritten notes into separate documents, annotating notes to reflect their comparison conversation. Examples of annotations included missed activity recorded by the other observer, details they had not thought to record, and information remembered during the comparison session. Paired data collection, both in the field and while finalizing field notes, can help hone skills and build confidence until observers have demonstrated their ability to proceed independently. Once individual data collection began, observation was conducted independently, and observers were given a maximum of 24 hours after observation to finalize field notes.

To ensure high quality data, the post-observation protocol during early data collection can also include the submission of electronic field notes for review and feedback from study team members (e.g., methodologists and content experts). Much like paired observation and review, providing feedback on typed field notes can be a temporary step in the after-observation protocol that is discontinued when observers consistently produce high-quality field notes.

For our study, observers submitted field notes for multiple rounds of review by two core team members experienced in obseration for the first three weeks of data collection. To help observers improve clarity of description and prompt better attention to detail for future observations, feedback was often posed as content-specific questions (e.g., Where did this stethoscope come from? What happened to this stethoscope after being used on patient?), demonstrated in Figure 3. Similarly, content-specific questions were used to help observers notice the omission of an action that should have been performed (e.g., Was hand hygiene performed after PPE doffed?). Through multiple rounds of review, research assistants honed their observation and data collection technique to include both what was and was not happening and learned the level of specificity expected in field notes and how to finalize field notes to produce coherent, detailed narratives.

Data management

While specifics will vary by project, data management tasks may also be included in the post-observation protocol. We outlined a file naming convention, field note submission requirements, and data security protocol in our after-observation protocol. Once comfortable with the data collection process, observers were asked to finalize and submit field notes within 24 hours of the observation period using a secure web-based platform (CTools). Once an electronic version has been created, handwritten notes should be handled in accordance with the approved IRB protocol (e.g., filed securely, shredded, etc). Although no PHI or PII were collected during our observation, all handwritten notes were shredded once an electronic copy had been created.

Figure 3.

Field note from contact precaution room observation with feedback from methodologist.

When observation is conducted in more than one place or under more than one condition, a highly-specific naming convention is useful for data organization and analysis. We identified important descriptive factors that could be used to meaningfully group field notes and applied them to our field note naming convention. For example, because our analysis plan involved comparison of HCP behavior and PPE use in contact, enteric, and droplet precaution rooms, we indicated precaution type in the file name for easy sorting. Other important descriptors we included in our naming convention were site, unit, position inside or outside patient room, date, and observer. In addition to organizing data for analysis, a highly-specific naming convention can help the study team recognize oversampling or undersampling by factor during active data collection, when observation can be more readily redirected. Finally, factors that distinguish between field notes should be included in the file name. For our study, multiple rooms at the same site, on the same unit, and with the same type of precaution were observed by the same person on a single date. To prevent confusion during analysis, observers assigned room numbers according to the order in which they were observed (e.g., Room 1, Room 2), and included the assignment in their field note header and at the end of the file name. While detailed, early adoption and continuous use of this convention resulted in a highly organized dataset throughout the study. The example below shows the descriptive information gathered in the field note header (Figure 1) applied as a naming convention.

Convention: Precaution Type_Site_Unit_Observation Position_Date_Initials_Room #

File name: Contact Site 1 Med Surg 2B Outside 01_03_21 XY Room 2

Observation data analysis

Observation data can be analyzed in a number of ways. Field notes can be analyzed alone as the only source or type of data or in conjunction with other data. Field notes can be coded and analyzed in much the same way as interview and focus group transcripts, through close reading of data and application of codes to facilitate identifying themes or patterns in the data.^26-28 Observation data can also be analyzed using a theoretical or conceptual framework.^26,29,30

We conducted a directed content analysis of our observation field notes (325 individual room observations across two sites) to identify and characterize breaches in transmission-based precautions that could result in selfcontamination or transmission during routine hospital care.¹² Our theoretical framework was based on the Systems Engineering Initiative for Patient Safety (SEIPS) model³¹ used to examine the work system, processes, and outcomes. Three members of our study team (authors) coded several field notes to ensure coding consistency, resolving discrepancy through discussion, followed by division of the remaining field notes for individual coding. NVivo qualitative software (version 10 QSR International Pty Ltd) was used to organize data and coding. Given the amount of observation data, NVivo was a useful platform to apply coding consistently, sort data, and compare findings. Code reports were generated after initial coding from which code application accuracy was confirmed and additional codes were identified.

Commonly observed potential self-contamination events and potential for transmission of an infectious agent were identified in review of the code reports. These events were then coded throughout the field notes and were further categorized as active failures such as violations, mistakes, or slips using an additional human factors model.^32,33 Active failures in PPE use across these three categories were examined to determine the contributing factors that should be addressed to reduce potential transmission during routine hospital care. Complete results of this analysis have been reported.¹²

We also analyzed observation data with data from eight HCP focus groups conducted as part of our study in order to understand the factors in HCP decisions whether or not to don PPE.³⁴ Two members of the study team read the first focus group transcript and began constructing a codebook inductively. The remaining transcripts were then independently coded by the two study team members who met to compare coding and reconcile any discrepancies through discussion. As new codes were identified, they were added to the codebook and applied to previously coded transcripts. Observation field note procedure for this analysis followed the same as focus group transcripts, with two team members initially reading several field notes to inductively develop codes, coding independently, and meeting to compare coding and reconcile any discrepancy. As with the content analysis, NVivo was used to organize and code our data. Once these steps were completed for focus group transcripts and observation field notes, data were compared to determine consistency between what we saw (observation data) and what HCP described to us (focus group data). We identified several factors that affected HCP use of PPE and ability to follow precaution guidelines, including perceived risk of certain organisms and perceived risk of contamination related to different care activities, as well as organizational and environmental factors.

In addition to validation of findings through comparison of observation and focus group data, or triangulation through multiple methods, we also looked for inconsistencies and disconfirming evidence between the two types of data. Because the goal of analyzing these data together was to better understand HCP decisions related to PPE use, inconsistencies between observation and focus group data offered additional insight on both intent and selfawareness of behavior surrounding PPE use and were described in our findings. For example, focus group participants reported different circumstances in which they may have experienced a potential exposure or selfcontamination (i.e., unprotected contact with a patient or surface in a precaution room), typically following an unexpected prompt (e.g., a phone/pager ringing, urgent patient care task needed, or patient initiating contact). In addition to these prompted behaviors, we also identified HCP self-contamination in our observation data that were the result of automatic behavior (e.g., scratching face, touching hair, and leaning against surfaces in the patient room), suggesting that HCP self-awareness may be a factor in self-contamination during different types of behavior. Full results of this analysis have been reported.³⁴

Discussion

Observation is a valuable, but potentially underutilized, method in health services research. Observation should be considered when research questions call for an in-depth understanding of current practices in highly complex settings. Data gathered through observation may be fundamental to understanding how complex processes work in practice, forming relevant research questions and designing context-appropriate interventions. This paper provides a framework and practical example for developing a study protocol and using observation from start to finish in health services research.

Although the literature at the time of the study had identified concerns about PPE compliance using quantitative data, our analysis provided an in-depth understanding of the factors that influenced HCP behaviors and, in turn, compliance with recommended PPE practices. Establishing a thorough understanding of current PPE use was integral to the ultimate objective of developing strategies to enhance effective PPE use and improve patient and HCP safety. Moreover, we were able to provide specific insights and recommendations to each of the study facilities for helping HCP use PPE more effectively.

Non-participant observation performed by research personnel and students was well-suited for our research goals. With our dataset, we were able to better understand the nuances of the HCP experience including the habitual behaviors and the workarounds they have to use to make their work possible. Findings from this study have been published in leading journals in the fields of infectious disease,³⁵ internal medicine,¹² and infection control;^34,36,37 presented at annual research meetings via posters, a podium presentation, and an invited talk, and awarded one of 10 most notable American Journal of Infection Control papers in 2020. We reported our findings locally to both participating facilities (an academic medical center and a VA medical center) as well as to national partners to inform future infection control protocol and PPE design. Further, given the current COVID-19 pandemic, it is exceedingly important that we understand the use of PPE in practice to develop and implement interventions that support and protect HCP in their work along with patients.

We took steps in our study to address the limitations of this method. Observation is cost and resource intensive because staff must be trained thoroughly in observation data collection and topic-specific content. Time can also constrain the amount of data collected, as each observer may focus on one situation (e.g., one isolation room) at a time. For our study, observation was performed by 11 individuals over nine months. While the cost of personnel time is high, we were able to collect a great deal of data in this time period compared to having fewer observers over a longer period. We were also able to leverage our budget for more observers by hiring a mix of six professional research staff and five nursing and pre-med university students. Although contextual data are captured through observation, observers will not always understand the behavior or rationale behind the behavior observed. For this reason, our study also included focus groups where further explanation could be gained through questions and discussion.

In some cases, observers may find it difficult to record what they see without intervening in the observed activity. Also, ethical considerations may arise regarding observer obligation to intervene when an error or act with serious consequences (e.g., injury, death) is observed. While our observers were trained to report objectively on what they observed without interference or participation, we developed a protocol for what observers should do if they observed a serious patient safety issue. Finally, while the Hawthorne effect is often discussed as a limitation of observational research, this effect can be mitigated through clear communication with study participants.³⁸ In keeping with the literature on the effect of the observer on participants, we found that HCP typically forgot we were there after a short time or engaged us in conversation, sometimes narrating what they were doing as they worked.⁷ These reactions to our presence suggest that participants resumed usual behavior during periods of observation.

Conclusions

Health services researchers use many data collection methods to address a variety of healthcare topics. Incorporating observation can establish the key components involved in the topic of interest, a vital step in asking relevant questions, measuring appropriate variables, and designing effective interventions. Whether used to inform another method, used alongside another method, or as the primary research method, observation can be used to describe what, when, where, and how something is happening across a wide range of healthcare research topics.

With this paper as a guidance document for planning and training study team personnel, we assert that observation is an accessible research method through which a solid groundwork of content- and context-specific knowledge can be established. Done right, observers also gain participant trust and foster buy-in by demonstrating commitment to understanding what really happens from the people who live it. It is upon these rich data and relationships with participants that appropriate, informed interventions can be designed and implemented. The investment of time, training, and effort into gathering fundamental knowledge of the content and context in which implementation will happen is rewarded with greater effectiveness of eventual interventions and conservation of resources that may otherwise have been misused designing and implementing off-target interventions.

Takeaways

Based on our success in collecting and analyzing observation data, we offer a few suggestions to keep in mind when you are planning this type of research. Consider using observation when your research question involves better understanding behaviors in context. Build in time for training and feedback because observation is an acquired, systematic data collection skill. Prepare and provide written resources to help observers navigate the field, including your data collection requirements and contact information for senior team members in case questions arise, (See Appendix for our field note guide.) Review data as you go to iteratively improve your observation tool and/or process. Consider what other perspectives may be needed to address your research question, and include other methods like interviews or focus groups for robust data and findings.

Funding Statement

Funding: This work was supported by funding from the Centers for Disease Control and Prevention, Prime contract no. 2002011-42039, Task Order 0007; Centers for Disease Control and Prevention, 1 U54 CK000456-01; and supported by the VA Health Services Research & Development Service (RCS 11-222 for SLK). The funding source played no role in study design, data acquisition, analysis or decision to report these data. The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs or the U.S. Federal Government.