Effects of tall man lettering on the visual behaviour of critical care nurses while identifying syringe drug labels: a randomised in situ simulation

Abstract

Background

Patients in intensive care units are prone to the occurrence of medication errors. Look-alike, sound-alike drugs with similar drug names can lead to medication errors and therefore endanger patient safety. Capitalisation of distinct text parts in drug names might facilitate differentiation of medication labels. The aim of this study was to test whether the use of such ‘tall man’ lettering (TML) reduces the error rate and to examine effects on the visual attention of critical care nurses while identifying syringe labels.

Methods

This was a prospective, randomised in situ simulation conducted at the University Hospital Zurich, Zurich, Switzerland. Under observation by eye tracking, 30 nurses were given 10 successive tasks involving the presentation of a drug name and its selection from a dedicated set of 10 labelled syringes that included look-alike and sound-alike drug names, half of which had TML-coded labels.

Error rate as well as dwell time, fixation count, fixation duration and revisits were analysed using a linear mixed-effects model analysis to compare TML-coded with non-TML-coded labels.

Results

TML coding of syringe labels led to a significant decrease in the error rate (from 5.3% (8 of 150 in non-TML-coded sets) to 0.7% (1 of 150 in TML-coded sets), p<0.05). Eye tracking further showed that TML affects visual attention, resulting in longer dwell time (p<0.01), more and longer fixations (p<0.05 and p<0.01, respectively) on the drug name as well as more frequent revisits (p<0.01) compared with non-TML-coded labels. Detailed analysis revealed that these effects were stronger for labels using TML in the mid-to-end position of the drug name.

Conclusions

TML in drug names changes visual attention while identifying syringe labels and supports critical care nurses in preventing medication errors.

Key messages.

What is already known on this topic

• Tall man lettering (TML) is a promising approach to distinguishing look-alike, sound-alike drugs in order to avoid medication errors. Although TML is recommended by several institutions, the evidence on the use of TML and its influence on visual attention is limited.

What this study adds

• In a simulated setting, the use of TML reduced the error rate of critical care nurses while identifying drug names on syringe labels. Eye tracking data further revealed that TML in syringe labels changed visual attention compared with labels without TML.

How this study might affect research, practice or policy

• Eye tracking allowed the investigation of subtle differences in visual behaviour relating to the position of TML in drug names, which might help to improve drug label designs and to reduce medication errors in intensive care units.

Background

Due to the fast-paced and complex environment, patients in hospitals and especially intensive care units (ICUs) are prone to harm resulting from drug incidents.^1–4 One identified reason for the occurrence of drug incidents are so-called look-alike, sound-alike (LASA) drugs with similar drug names leading to mistakes and misidentifications.⁵ A survey among anaesthetists has suggested that label misidentifications of potentially harmful drugs occur frequently in anaesthesiological settings.⁶ Furthermore, high costs of overall medication errors including label misidentifications have been described.⁷

One specific approach to avoiding medication misidentifications is a particular form of label text enhancement called ‘tall man’ lettering (TML), which describes the capitalisation of distinct text parts in drug names to facilitate differentiation from LASA counterparts.⁸ In 2001, the Food and Drug Administration of the USA (FDA) promoted the FDA Name Differentiation Project and recommended the evaluation of the use of TML.⁹ However, despite being recommended by several institutions such as the German Interdisciplinary Association for Intensive Care and Emergency Medicine,¹⁰ to date, the evidence on the use of TML remains unclear, in part conflicting and mostly based on conventional observation studies.^{8 11–14}

Eye tracking as a gaze analysis technique has been used in hospitals to assess visual patterns in different real-life and simulated study settings^15–23; it has clear potential to help investigate and understand the effect of TML in drug labels.

In light of the current gap in the literature and limited available evidence in relation to the use of TML, the aim of this study was to investigate the visual behaviour of ICU nurses while identifying syringe labels of commonly used drugs. We hypothesised that the use of TML reduces the error rate and changes visual attention while identifying drug syringe labels.

Methods

Study design and population

This was a simulated, prospective eye tracking study conducted at the interdisciplinary ICU of the University Hospital Zurich, Zurich, Switzerland, between August 2020 and December 2020. The study was designed as a randomised in situ simulation, providing a realistic drug selection scenario in a standard ICU patient treatment room.

Participants were recruited by the study team from a pool of certified ICU nurses (>18 years old) employed at the hospital. Participation was voluntary and subject to written consent of the participants. Random sampling of participants was conducted by stratified randomisation to ensure a representative population in terms of participant age and ICU experience. Stratified randomisation was conducted by the study team prior to approaching possible participants. If participants declined to participate, suitable replacements were identified by the study team (figure 1, study flow diagram). Exclusion criteria were visual disturbances including lack of stereoscopic vision, monocular vision and achromatopsia. Based on comparable eye tracking studies,^15–23 a participant number of 20–30 was considered adequate and targeted. Prior to the eye tracking data collection, the recruited participants were randomised as illustrated in figure 1. To minimise biases and to provide a realistic scenario, data collection was conducted during daytime.

Figure 1.

Study flow diagram. Each participant was given 10 sets of syringes in the form of five TML-coded and five non-TML-coded sets. The order of the 10 sets was randomly permuted by block randomisation. TML, tall man lettering. Figure created by the authors and approved by all authors.

Label design and syringe sets

For the experiment, uniformly sized 10 mL Braun Injekt (B Braun Medical, Sempach, Switzerland) syringes were used. All syringe labels were designed in compliance with the international standard defined in ISO 26825.²⁴ All syringes were uniformly filled with water.

In total, 200 syringes were labelled, 100 with TML-coded labels and 100 with their non-TML-coded counterparts. The syringes were divided into sets of 10 syringes, which resulted in 10 sets with TML-coded labels and 10 identical sets with non-TML-coded labels. Each set included syringes labelled with LASA drug names (figure 2), either in the form of LASA pairs or trios. The allocation of LASA pairs/trios to the sets was performed in order that all 10 sets with TML-coded labels included different LASA pairs/trios and all 10 sets with non-TML-coded labels included different LASA pairs/trios. A complete list of the chosen LASA pairs/trios is provided in online supplemental table 1. Photographic illustrations of exemplary syringe sets are demonstrated in online supplemental figure 1 (TML coded) and online supplemental figure 2 (non-TML coded). At our ICU, all drugs chosen for this study are normally prepared on-site by the nursing staff in charge (including handling of ampoules, preparation and labelling of syringes). The labels used in clinical practice are typed and printed.

Figure 2.

Example of a LASA pair of drug names represented as non-TML-coded and TML-coded syringe labels: Ephedrin with TML coding in the middle of the drug name (left) and Epinephrin with TML coding at the start (right). LASA, look-alike, sound-alike; TML, tall man lettering. Figure created by the authors and approved by all authors.

Study task and experiment protocol

The experiment started after a 30-minute period of familiarisation with the eye tracking glasses and a standardised three-point calibration. If participants were myopic or hyperopic, correction glasses provided by the manufacturer of the eye tracking device were used.

During the experiment, each participant was repeatedly asked to identify and select a syringe labelled with the correct prescribed medication from one of the predefined sets. The prescription was given by presenting a drug name and a concentration via the in-hospital Patient Data Management System. As soon as a participant selected a syringe, it had to be placed on a predefined surface, which ended the experiment. If a participant was not able to select a syringe, he or she had to check a box, which also ended the experiment. The selection of a syringe differing from the correct one was considered an error, which was taken into account for the calculation of the error rate.

The allocation of syringe sets to the participants is illustrated in figure 1 and online supplemental figure 3. Each participant was successively given 10 sets of syringes in the form of five TML-coded and five non-TML-coded sets. The order of the 10 sets was randomly permuted by block randomisation by the study team (number of participants: 30, block size: 10) (online supplemental figure 3). Participants were blinded to the set numbers used.

After completion, the participants were asked to fill out a post-experiment questionnaire.

Post-experiment questionnaire

The questionnaire had to be completed using paper and pencil. The time to complete it was limited to 10 min. It collected data on baseline characteristics (gender, age and professional experience), and seven further questions on individual experience with TML-coded labels (familiarity, perceived helpfulness and effectiveness) and on participants’ personal assessment of study limitations (realism, difficulty and disturbance) had to be answered. Familiarity and helpfulness of TML were assessed dichotomously (yes/no). The remaining questions (except baseline characteristics) were analysed using a scale ranging from 1 to 10 (1=totally disagree, 10=totally agree).

Eye tracking data collection and analysis

For data collection, SMI ET Glasses 2 Wireless (SensoMotoric Instruments, Teltow, Germany) were used. Eye tracking was executed at a sampling rate of 60 Hz. Over all distances, the angle of view was measured with an accuracy of 0.5°. The scene video was recorded with a resolution of 960×720 pixels at 30 fps. Eye tracking data were processed using the SMI BeGaze V.3.6 software (SensoMotoric Instruments, Teltow, Germany) and the SMI algorithm for fixation determination. Ocular fixations during the experiments were manually assigned to the areas of interest (AOIs).

Eye tracking data were evaluated by an AOI analysis with each syringe label defined as an AOI. Data analysis included all fixations on the AOIs as well as all saccades between these fixations (including the saccades entering and leaving the AOI). Visual fixations on other areas were not subject to analysis and thus excluded.

Based on the preprocessed data, four common AOI measurements were computed: (1) dwell time (cumulated time spent on the AOIs, including blinks and saccades), (2) fixation count (cumulative number of fixations on the AOIs), (3) fixation duration (cumulated time of all fixations on the AOIs) and (4) number of revisits (the number of times the gaze returns to an AOI which had been looked at before).

Study outcomes

The study compared error rate and eye tracking measurements (dwell time, fixation count, fixation duration and number of revisits) between tasks with TML-coded and non-TML-coded labels. All eye tracking measurements were further applied to compare visual behaviour on TML-coded labels with respect to the positioning of TML within the label (TML at the start vs mid-to-end position of the drug name).

The post-experiment questionnaire gave additional insights into the participants’ baseline characteristics and their experience with TML coding in the context of this study.

Statistical analysis

Data were modelled by means of linear mixed-effects models, considering eye tracking measurements as dependent variables and either presence of TML coding or TML position as fixed effects while accounting for subject random effect. Additionally, a drug name fixed effect was incorporated into the model without interaction term, as inclusion of the latter into the model did not add information. The null hypothesis, formulated as the absence of effect for a specific comparison, was assessed by means of a likelihood ratio test considering whether all spline parameters were equal to zero.

Statistical analysis was performed via a fully scripted data management pathway using the R environment for statistical computing, V.4.0.2. A two-sided p<0.05 was considered statistically significant. Values are given as means with SD, medians with interquartile rooms (IQRs) or counts and percentages as appropriate.

Results

Participants

In total, 34 possible participants were approached (figure 1). Four nurses declined to participate. A representative random sample of 30 ICU nurses (8 male, 22 female) were included (table 1). Data from all 30 participants were analysed.

Table 1.

Data derived from post-experiment questionnaire

Baseline characteristics
Gender
Male	8 (26.7%)
Female	22 (73.3%)
Age (years)	36 (28–45)
Total professional experience (years)	17.5 (6–23)
Professional experience on ICU (years)	6 (3–17)
Experience with TML
Familiar with TML
No	19 (63.3%)
Yes	11 (36.7%)
TML was helpful
No	18 (60%)
Yes	12 (40%)
Potential to make errors with TML*	7 (5–8)
Limitations
Experiment was realistic*	9 (8–10)
Drugs were realistic*	9.5 (7–10)
Experiment was difficult*	2.5 (2–4)
Disturbed by glasses*	1.5 (1–4)

Data expressed as numbers (percentages) or median (IQR), where appropriate.

Table created by the authors and approved by all authors.

*Marks a subjective/self-assessed characteristic on a scale ranging from 1 to 10 (1=totally disagree, 10=totally agree). The potential to make errors with TML refers to subjective perception of participants to make errors even with TML.

ICU, intensive care unit; TML, tall man lettering.

The participant group was characterised by a median age of 36 years (IQR: 28–45) and a professional experience of 17.5 years (IQR: 6–23). Median professional experience on the ICU was 6 years (IQR: 3–17). No participant was excluded during the study. No harm or unintended effects occurred during the eye tracking data collection.

Error rate

Throughout the study, participants selected the wrong syringe in only 1 of 150 sets with TML-coded labels, but in 8 of 150 sets with non-TML-coded labels.

This corresponds to an error rate of 0.7% with TML and 5.3% without TML. Statistical evaluation showed a significant difference in the number of errors (p<0.05). Effect sizes are provided in online supplemental table 2.

The eight errors made in non-TML-coded labels were clustered around seven participants. One participant made two mistakes. Errors occurred across a wide range of professional experience (range 2–34 years). Four of the seven participants making errors with non-TML-coded labels were familiar with TML and three of these seven participants considered TML to be helpful.

TML versus non-TML-coded syringe labels analysed by eye tracking

Eye tracking data revealed differences in the visual attention provided on TML-coded and non-TML-coded labels (figure 3). Within the study, dwell time was significantly longer on TML-coded labels than on non-TML-coded labels (p<0.01). This result was consistent with an overall higher fixation count (p<0.05) and significantly longer fixation duration (p<0.01) when using TML. Effect sizes and detailed p values are provided in online supplemental table 2.

Figure 3.

Point-range plots depicting the isolated effect of eye tracking measurements depending on the presence of TML coding: (A) dwell time, (B) fixation count, (C) duration of fixation and (D) number of revisits. Effects were modelled by means of a linear mixed-effects model accounting for drug name as fixed effect and subject as random effect. The point reflects the point estimate and the bar represents the 95% CI of the effect estimation. Significance levels: p≥0.05—NS, *p<0.05, **p<0.01, ***<0.001. TML, tall man lettering. Figure created by the authors and approved by all authors.

We further found that during interaction with TML-coded labels, participants visually revisited labels which had already been looked at more often. Figure 3D shows that working with TML-coded labels increased the occurrence of revisits with a median of seven revisits. Statistical evaluation confirmed a significant difference in number of revisits between TML-coded and non-TML-coded labels (p<0.01).

TML at the start versus mid-to-end position in drug names analysed by eye tracking

The comparison of visual attention depending on the position of the tall man letters in the drug name showed significant differences between TML at the start and TML in the mid-to-end of the label (figure 4). We found longer dwell times (p<0.001), higher fixation counts (p<0.001), longer fixation duration (p<0.001) and higher numbers of revisits (p<0.01) for labels using TML at a mid-to-end position in contrast to TML at a start position. Effect sizes and detailed p values are provided in online supplemental table 2.

Figure 4.

Box plots depicting the results of eye tracking measurements depending on the position of TML coding on syringe labels: (A) dwell time, (B) fixation count, (C) duration of fixation and (D) number of revisits. Significance levels: p≥0.05—NS, *p<0.05, **p<0.01, ***p<0.001. Figure created by the authors and approved by all authors.

Questionnaire

The results of the post-experiment questionnaire demonstrated that only 37% of the participants were familiar with TML coding before the study (table 1). Furthermore, 60% of the participants declared that, in their opinion, TML was not helpful in preventing medication errors. The potential for still making mistakes despite using TML was even rated as 7 on a scale ranging from 1 to 10 (1 indicating no potential for making mistakes, 10 indicating maximal potential for making mistakes).

Discussion

The aim of this study was to test whether the use of TML reduces the error rate and to examine effects on the visual attention of critical care nurses while identifying syringe labels. Further, we also tested whether the position of TML (at the beginning of the label vs mid-to-end position) had an effect on the performance of individual participants.

Our main results demonstrate that (1) the use of TML reduces the error rate of critical care nurses when identifying drug names on syringe labels, (2) TML changes visual attention compared with non-TML-coded labels and (3) visual attention differs according to the positioning of TML within syringe labels.

The concept of LASA drugs as potential sources of error was described almost 50 years ago.⁵ Since then, diverse studies have assessed strategies to minimise medication selection errors; however, they have produced conflicting evidence as to whether the use of TML is beneficial or not.8 12–14 25 26 Our study showed that the use of TML in syringe labels led to a lower error rate compared with the use of non-TML-coded labels under simulation conditions. This result suggests that the use of TML ensures improved and more accurate reading by participants, underscoring the potential role of TML to improve patient safety. Similar results were also obtained in an observational study by DeHenau et al, where TML led to a more frequent successful detection of graphically presented changes on a screen.⁸ The fact that error rate is an accepted metric which has also been used in other studies with TML²⁶ adds practical relevance to our finding, which was obtained from a realistic study environment. Further studies in designs suitable for proof of effectiveness are needed to confirm our results, as has also been highlighted in a recent systematic review.¹²

Overall, TML changed visual attention compared with non-TML-coded labels (figure 3). A central associated finding was the increased dwell time, fixation duration and fixation count on labels where TML was used compared with non-TML-coded labels (figure 3), all parameters reflecting the importance and relevance of AOIs in a specific context. Despite longer and more intense reading intervals on TML labels (compared with non-TML-coded labels), these findings suggest that the use of TML was associated with an accurate reading quality by the participants, as also reflected in the lower error rate. In fact, participants not only seem to fixate the correct target longer and more intensely, but also the corresponding misleading LASA drugs, if TML is used. TML thus not only facilitates the detection of correct syringes, but also differentiation of LASA pairs/trios, which has high clinical relevance. Our results also showed that the number of revisits to labels with TML was significantly higher than without TML (figure 3). Revisits are known to be indicators for behavioural patterns related to controlling or monitoring.^{27 28} Since the use of TML in the study led to more frequent cross-checks between the labels in a syringe set, we further suggest that TML highlights the differences between drug names, which mandates a more intensive comparison prior to the definite selection of a syringe. Thus, TML coding seems to ease the identification and comparison of LASA pairs/trios by shifting visual attention to the characteristic parts of drug names.

We further analysed the eye tracking data with respect to the position of TML in the drug name. According to our results, dwell time, fixation count, fixation duration and number of revisits were all higher with TML at a mid-to-end position compared with a position at the beginning of the name, suggesting that participants need even more time to identify the corresponding labels. These results can be explained by the assumption that the participants’ natural behaviour involves comparing the first letters of the drug names to identify the right syringe. In this case, TML at the start of the name would primarily not significantly change the distribution of visual attention. If, instead, TML is used at a mid-to-end position, participants are stimulated to shift their visual attention from the first letters to the capital letters in the middle or end of the name, with an associated more elaborate reading process. However, whether this has an impact on the error rate or is clinically relevant remains unclear and should be investigated in further studies. Moreover, future studies should address whether increased experience of participants affects complexity and time needed to process TML-related information.

The post-experiment questionnaire revealed that most participants were sceptical of the effectiveness of TML coding on syringe labels. Taking into account that fewer errors occurred when using TML in the study, this is a surprising result. It can be assumed that participants who accomplished their tasks without any errors have not experienced the benefit of TML, but only the perceived disadvantage of more complex and time-consuming information processing. This aspect should be considered in staff training when TML coding is rolled out in ICUs.

An advantage of the study is its pragmatic design, which might easily be transformed into further real-life studies investigating gaze patterns of professionals while focusing on syringe labels. Although the participants stated that the study was already characterised by a high level of reality, even more realistic studies (eg, studies that analyse gaze patterns of professionals during the administration of real drugs to patients) could reveal further insights contributing to enhanced patient safety in the future. Clearly, such studies would need to be designed carefully in order not to risk any patient harm.

In this study, eye tracking proved to be a helpful tool that (compared with conventional observational or video analyses) can offer valuable data for an objective and reliable analysis of visual attention. Eye tracking allowed the investigation of subtle differences in visual behaviour relating to the position of TML in the drug names, which, to our knowledge, no comparable study has evaluated so far in ICU professionals. Thus, our study also shows that future eye tracking research has the potential to further improve the design of syringe labels and to ease their readability. Additionally, eye tracking studies also need to address sustainability and possible long-term effects of TML with regard to medication errors and patient safety, which are so far unclear.

Our study also has to account for some limitations. First, gaze patterns represent cognition only to a limited extent. There are aspects of cognition that remain undeterminable by eye tracking. Second, the study was conducted in a simulated design, which should be considered when addressing generalisability. However, it was performed in real and fully operational ICU facilities with common everyday equipment, providing a realistic scenario. Third, the participant number was relatively low, which could be increased in future studies. Fourth, differing word length or the pronunciation of the chosen drug names might have introduced biases as it is currently not well known whether word length or pronunciation affects the effectiveness of TML. This should be addressed in further comparable studies. Moreover, the choice of defined drug names by the study team prior to the experiments might have influenced the data and led to a selection bias. However, the participants considered the choice of drugs to be realistic in the post-experiment questionnaire. Finally, external validity of our findings might be impaired due to the fact that, according to the post-experiment questionnaire, only one-third of participants were familiar with TML before the experiment.

Conclusion

In conclusion, TML reduced the error rate and changed the visual attention of critical care nurses when identifying syringe labels.

Data availability statement

Data are available upon reasonable request. The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Ethics statements

Patient consent for publication

Not required.

Ethics approval

The study was approved by the local ethics committee (Kantonale Ethikkommission Zurich, BASEC ID REQ 2017-00798). All participating ICU nurses agreed to participate voluntarily.