Missed Opportunities: The Development and Testing of Standard Precaution Case Vignettes

Abstract

Standard precautions are a recommended set of actions health care workers should take to prevent health care-associated infections and potentially infectious exposures. However, they are not reliably practiced, many opportunities are missed, and a substantial discrepancy between workers’ stated performance and actual performance exists. This article presents findings from developing and testing standard precaution case scenarios to enhance nursing knowledge. Infusion nursing is high risk and entails highly complex care in dynamic settings. Challenges may exist when integrating this advanced care with fundamental standard precaution requirements. For this reason, findings from this study have implications for infusion nursing practice.

In modern health care, patients require seamless integrated interdisciplinary clinical care under highly dynamic conditions. This care increasingly includes infusion therapy.^1–4 Optimal clinical outcomes require highly reliable performance by all health care workers for every aspect of care, from the most basic tasks to the most technologically advanced.^5,6 To produce highly reliable, optimal outcomes, health care workers must have an adaptive skill set and situational awareness to integrate advanced practices with basic standards of care.^2–3,6 When care is not well choreographed, health care workers may experience sharps injuries and bloodborne pathogen exposures and patient health care-associated infections (HAIs) may occur.^6–8

The most recent estimates suggest that 3.7/100 health care workers, including 1 in 25 registered nurses, suffer an occupational bloodborne pathogen exposure or needlestick injury annually with direct and indirect costs of $300 million. The majority of these, 56% to 88%, are also preventable.^9–12 HAIs remain a key patient safety indicator and a substantial public health problem, affecting approximately 750 000 patients a year and resulting in 75 000 estimated deaths. An estimated 70% of these are preventable, with attributable costs of $6.7 billion in US hospitals.^13,14

A primary prevention strategy recommended by the Centers for Disease Control and Prevention more than 2 decades ago are standard precautions, the base in a hierarchy of controls designed to protect health care workers and patients.¹⁵ Specific standard precaution components include: hand hygiene; the use of appropriate personal protective equipment; the safe use and disposal of sharps; patient placement and linen and waste management; and decontamination of the environment and patient care equipment. This set of recommended actions is applicable to all providers caring for patients in any setting and establishes a minimum standard of care for health care workers and organizations.¹⁵ Standard 19.1 of the Infusion Therapy Standards of Practice explicitly states: “Standard precautions are used during all infusion procedures that potentially expose the clinician to blood and body fluids, secretions, excretions except sweat, non-intact skin, and mucous membranes and may contain transmissible infectious agents.”^16(pS41) Despite its importance however, adherence to standard precautions remains grossly suboptimal, and actions that are indicated are followed by health care workers less than 50% of the time.^17–20

To explore reasons for the poor adherence, a set of tools was developed, adapted, and tested to survey nurses on the patient safety climate, reported standard precaution practices, and collected observational data on standard precaution adherence.²¹ Observers were trained to use a Standard Precaution Observation Tool (SPOT), using a series of vignettes in 5 hospitals. A substantial gap was found between the respondents’ reported and revealed preference, as the majority of the 140 nurses surveyed (94%) reported always or often adhering to standard precautions while the revealed preference demonstrated standard precautions adherence was 62%.²¹

This discrepancy between the behaviors providers report and the behaviors they are observed doing is concerning, given the high burden of preventable health care workers sharps injuries, bloodborne pathogen exposures, and HAIs. Reasons for this gap and poor standard precaution adherence are not completely clear, although there is evidence that standard precaution education, knowledge, and adherence may be related.^18–20 Unfortunately, this education often is provided in undergraduate training only, initially at clinical hire through didactic training, and annually by means of checklists affirming provider competency.^18,20,22 These approaches may result in poor understanding of standard precautions, knowledge decay over time, and an inability to translate didactic knowledge to “real-world” practice settings. Notably, there are no available, easy-to-use, realistic, valid, and reliable tools to assess knowledge and the recognition of risks and expected actions related to standard precaution behaviors. Therefore, no data exist to inform the development of new approaches, such as targeted ongoing educational interventions. The purpose of this study was to evaluate the use of the vignettes and an observational tool as an educational measure of standard precaution knowledge and recognition.

METHODS

This was a cross sectional pilot study to: 1) examine if the vignettes and observational tool and have sufficent reliability and validity properties to assess health care workers’ knowledge and recognition of standard precaution indications and actions, and 2) describe the baseline knowledge of participants. Ethical approval from the Institutional Review Board was obtained, and permissions were acquired to conduct the research at each hospital.

SETTING AND SAMPLE

A convenience sample of 5 medical surgical units from 4 hospitals in the mid-Atlantic region of the US which participated in the 2015 parent study were selected.²¹ Nurses who worked at least 16 hours a week in a direct-care capacity on those units for at least 12 months were eligible to participate. The aim was to recruit approximately 10 nurses from each unit, for a total of as many as 50 nurses, sufficent for this exploratory study. No identifying information was collected from the participants.

MEASURES

The SPOT was designed to measure observed adherence to components of standard precautions in hospital settings for hospital infection prevention and occupational health surveillance purposes.²¹ The instrument captures data on health care worker and patient interactions to record when a standard precaution action was indicated and if it was completed or missed. Ten standard precaution indications in the categories of hand hygiene, linen management, and the use of sharps and personal protective equipment, (eg, hand hygiene before patient contact, donning gloves, and immediately disposing of sharps after use) were included. Data were analyzed to yield a percentage of either correct adherence or missed opportunities for standard precaution actions.²¹ The instrument has demonstrated content and face validity and interrater reliability testing. Each indication was defined for the user of the tool and available during use and testing.

DATA COLLECTION

Nurses who worked on the units were recruited by various methods as appropriate to their specific work setting and in consultation with their unit manager (eg, unit huddles, flyers, or staff meetings). The research team coordinated with unit nurse managers to schedule dates and times of the pilot vignettes assessment. All data were collected during work hours on the hospital units at workstations or breakrooms to capitilize on the “real-world” dynamics that influence health care workers cognition and actions. Available and willing participants were provided instructions for the tool and were asked to read the vignettes and use the tool to indicate whether a standard precaution action was indicated and performed or missed. It was anticipated the time to complete the activity would not be longer than 10 minutes. All data were collected between June 2016 and October 2016.

ANALYSIS

The following were used in the selection of vignettes: interrater reliabiliy by percent agreement and Cohen’s kappa coefficient among raters using criteria of >80% agreement; statistically significant kappa statistics (P <.05); readability; appropriateness for educational testing; and a minimal duration to complete to yield a reduced set of vignettes, with no more than a 15-minute response burden. To evaluate nurses’ knowledge and recognition, staff nurses were tested using the vignettes and the SPOT to calculate the percentage of nurses who correctly identified whether a standard precaution action was indicated and if it was performed.

RESULTS

Selection of Vignettes

A series of 40 vignettes describing health care worker-patient interactions had been developed, based on clinical experience and field observations, to test standard precautions knowledge using the SPOT.²¹ Two nurse reseachers pretested them for educational assessment. Scoring was reviewed and discussion regarding readability, appropriateness, and the validity of the vignettes occurred until consenseus was reached, yielding 20 suitable vignettes for psychometric testing. The resulting 20 vignettes were subject to content, usability, feasibility, and interrater reliability testing by end users, staff nurses (n = 2) and clinical nurse educators (n = 2), who were 2015 pilot study liaisons and therefore familiar with the tool and the study design.

For each scenario, we examined percent agreement and kappa statistics between each set of educators and staff nurses. Eighteen of 20 vignettes demonstrated statistically significant kappa statistics (p <.05) and percent agreement >70% for both sets of raters. Nine vignettes were selected using criteria of: substantial to almost perfect agreement within and between sets of raters (>75%), statistically significant Cohen’s kappa statistics (p <.05), readability, appropriateness for educational testing, and minimal duration to complete. This set of nine vignettes included 45 indications across all standard precaution categories with appropriate actions performed or missed by the health care workers in the scenario; the answer key included 31 actions appropriately performed yielding an adherence score of 68.9%. Examples of retained and dropped vignettes follow:

Example Retained Scenario

“A nurse and a nursing assistant enter a patient’s room. The nurse uses hand sanitizer and dons gloves. The nursing assistant dons gloves. They boost the patient up in bed, touching bedside controls and linens. Then, they both remove gloves and throw them in the garbage. The nursing assistant uses hand sanitizer and leaves the room. The nurse pulls the television closer to the patient and pushes the bedside table away from the bed. She then assists the patient’s roommate out of the bathroom and back to bed.” In this example, the scenario met the criteria of substantial to almost perfect agreement of rating the number of standard precaution indications and actions taken within and between sets of raters demonstrated by statistically significant (p < .05) Cohen’s kappa statistics (κ = .75 for clinical educators and κ = .85 for frontline nurses), assessment of readability, appropriateness for educational testing, and minimal duration to complete.

Example Dropped Scenario

“A respiratory therapist enters a patient’s room to obtain a sputum sample. His hands are full of equipment, which he places on the bed while he goes to wash his hands in the sink. He dons a facemask, gown and gloves then prepares the sterile suction kit. After he has obtained the sample he pulls down his mask so he can speak to the patient, then removes his gown and gloves, throwing them in the garbage. He uses hand sanitizer and exits the room.” In this example, the scenario did not met the criteria of substantial to almost perfect agreement of rating the number of standard precaution indications and actions taken within and between sets of raters as demonstrated by Cohen’s kappa statistics (κ = .42, p < .05 for clinical educator agreement and κ = .01, p =.45 for frontline staff agreement), indicating only 1 set met statistical significance and both demonstrated only slight to moderate agreement. Specifically, differences in both the number of indications (some identified 6 indications whereas others identified 8) and actions taken (raters identified 4, 5 and 6 actions taken) were noted, and therefore, this scenario was dropped from further testing.

Standard Precaution Knowledge and Recognition

Forty nurses completed the SPOT using the final set of 9 vignettes. The first scenario was omitted from analyses as it was the test scenario. Missing responses were present in 3 and they were dropped from analyses, yielding 37 for analyses. In the full set of vignettes with 45 indications and 31 actions performed appropriately, or scenario standard precautions adherence of 68.9%, respondents identified a mean adherence score of 65.1% (SD = 0.10), with a range of 35.7 to 83.3% adherence. Of the 37 nurses who completed the assessment, 27 (73%), scored 80% or greater, and of those 7 scored 90% or greater, identifying if a standard precaution action was indicated. Twenty-five of the 37 (67%) scored 80% or greater if an action was taken, of those 23 scored 90% or higher.

Analyses at the item level (each of the 45 indications) found the correct number of indications (within one point) identified ranged between 29.7% to 100% and the correct number of actions taken identified ranged between 45.9% to 100%. These scores varied by complexity of scenario (eg, number of indications per scenario). Unit scores indicate 4 of 5 units had an aggregated score of 90% or greater for knowledge of indications and 3 of 5 units scored greater than 90% of recognizing when action was taken. The internal consistency reliability of tool and vignettes by Cronbach alpha was α = 0.87. Open ended comments from the participants’ further support validity. Respondents also reported the vignettes were “life-like,” “realistic,” “the tool takes a minute to get used to,” and is “easy to use.”

DISCUSSION

In prior work, the research team developed a library of 40 vignettes, based on clinical experience, and an answer key to train hospital-based observers to collect standard precautions adherence data. A substantial gap between health care workers reported performance and observed performance was discovered. This prompted the question: “Do providers know what they should be doing, and can they recognize if it has been completed with regards to standard precautions?” As a step to rapidly translate new evidence into practice, with the goal of reduced health care worker infectious exposures and patient HAIs, the research team refined and adapted the vignettes to evaluate nurses’ knowledge of standard precautions indications and actions. Results indicated that the vignettes and the SPOT have demonstrated properties of reliability and validity for use as educational tools in “real-world” clinical settings to quickly gauge nursing knowledge and recognition of conditions under which certain standard precautions protective actions should be and are taken. Assessment results indicate that ample opportunity for improvement exists.

It is concerning that approximately one-third of the nurses in the study could not identify when a standard precaution was indicated or action taken, even using a cut-off of 80%, and when using a one-dimensional written scenario. This inability to discern actions cognitively may have consequences on the ability to discern when and if appropriate behaviors are performed in the dynamic, live, clinical setting. Patient safety culture literature suggests individual performance may also have a ripple effect across the unit to either engender or prevent poor performance through teamwork, peer feedback, and role modeling.^5,6 This may be evident in our findings that individuals who performed well gravitated together and worked on units that performed well.

While this study included direct care nurses in hospitals, standard precautions recommendations apply to all providers, and implications for infusion nursing professionals as members of the health care team are evident. Infusion nursing is high risk and entails highly complex care in dynamic settings. Challenges may exist when integrating this advanced care with the fundamental standard precaution requirements. Further, maintaining situational awareness, understanding the actions of others and features of the environment, comprehending meaning and risks, and anticipating next steps, are necessary for highly reliable and safe care.^1–3,6 For example, direct care nurses provide around the clock care to patients receiving infusion therapy, and thus their actions can impact both the infusion treatment efficacy and patient susceptibility to health care associated infections. As members of the health care team, infusion nursing professionals must uphold not only their professional standards individually but maintain situational awareness of the complex care environment to assure patient and occupational safety.

Future work can include expanded use of the vignettes and SPOT in clinical training and assessment, development of vignettes for distinct and diverse provider groups and settings (such as home care), and development of interventions to encourage dialogue and role-playing. Considering the increasingly poor performance as the vignettes became increasingly complex, interactive training modalities, such as simulation, that incorporate varying complexity may be needed to enhance situational awareness skills and foster standard precautions adherence. Simulation is an educational approach and strategy to improve cognitive, psychomotor and affective outcomes of the learner.^23,24 Vignettes are designed to demand interventions of the learner that require critical thinking and prioritization, this is of paramount importance as providers must integrate the “basics” of standard precautions with the increasingly complex nature of clinical care.^23–25 For settings without simulation centers or access, the vignettes and testing may be useful as online or interactive learning modules.

By example, this educational content was presented at the Infusion Nurses Society’s (INS’) Annual Meeting in 2018, and using voluntary, anonymous online polling with the audience, similar baseline knowledge of when standard precautions were indicated, and when actions were taken or missed using these scenarios was found to the study sample. Out of 75 attendees, 62% of participants identified if an action was indicated and 76% if the action was taken. Qualitatively, the potential value and impact of such a standard precautions intervention on nursing knowledge is evident through open-ended evaluation responses. Participants noted: “disparity between perceived compliance and observed compliance”; “rethought what I have learned from past and new ways that health care personnel are missing many opportunities to follow standard precautions;” and “we need to be much more aware and vigilant.” The most beneficial learning impact was: “scenario reviews”; “new way of looking at previous methods for precautions”; “review and observation of others for best practice”; “I can apply new info at work”; and “poll (eg, vignettes) actually made you think more about each step of precautions.”

LIMITATIONS

Test scores include those both under- and overcounted, (eg, responded with less or more indications or actions than the answer key) or either answer was categorized as incorrect. Important fundamental differences in cognition and therefore educational needs may exist. In this study, the research team tested knowledge based on reading comprehension of short (several sentence) vignettes in English only. Differences may exist among nurses who speak and read English as a primary or secondary language. However, the research team did not think this was a substantial limitation as all nurses had passed state licensing board tests in English language only. Demographic data from this sample were not collected and it was assumed that the sample reflected the original pilot distribution, though important differences may exist. All testing occurred during the day shift and results across shifts may differ.

CONCLUSIONS

Standard precaution knowledge and recognition can be reliably measured using these vignettes and the SPOT. The research team found that substantial knowledge deficits exist in this sample of nurses. The inability of nurses to recognize standard precaution indications and actions cognitively may have consequences on the ability to discern when and if appropriate behaviors are performed in practice, and therefore limit the ability to provide performance feedback to peers. These tools may be a useful adjunct to educational programs that encourage dialogue, roleplaying, situational awareness, and action. Further, these tools may be useful for educators, unit managers, infection preventionists, and occupational safety and risk committees to periodically assess and reinforce best practices. Future research to tailor vignettes to specific provider specialties, such as infusion nursing, and test in diverse settings, where risk perception may differ from acute care, such as home care and long-term care, is needed.

Biographies

Authors Biographical Sketch:

Amanda J. Hessels, PhD, MPH, RN, CIC, CPHQ, FAPIC is an Associate Research Scientist at Columbia University School of Nursing, NY, NY jointly appointment as a Nurse Scientist at Hackensack-Meridian Health, Ann May Center for Nursing, Neptune, NJ. Dr. Hessels’ research endeavors center on advancing the science of infection prevention practices, particularly standard precautions.

Susan H. Weaver, Phd, RN, CRNI, NEA-BC is a Nurse Scientist at the Hackensack-Meridian Health Ann May Center for Nursing, Neptune, NJ. Dr. Weavers’ program of research is on the evening and night administrative supervisors and their impact on nurse and patient safety.