Abstract
Hand hygiene practice in hospitals is unfortunately still widely insufficient, even though it is known that transmitting pathogens via hands is the leading cause of healthcare-associated infections. Previous research has shown that improving knowledge, providing feedback on past behaviour and targeting social norms are promising approaches to improve hand hygiene practices. The present field experiment was designed to direct people on when to perform hand hygiene and prevent forgetfulness. This intervention is the first to examine the effect of inducing injunctive social norms via an emoticon-based feedback system on hand hygiene behaviour. Electronic monitoring and feedback devices were installed in hospital patient rooms on top of hand-rub dispensers, next to the doorway, for a period of 17 weeks. In the emoticon condition, screens at the devices activated whenever a person entered or exited the room. Before using the alcohol-based hand-rub dispenser, a frowny face was displayed, indicating that hand hygiene should be performed. If the dispenser was subsequently used, this picture changed to a smiley face to positively reinforce the correct behaviour. Hand hygiene behaviour in the emoticon rooms significantly outperformed the behaviour in three other tested conditions. The strong effect in this field experiment indicates that activating injunctive norms may be a promising approach to improve hand hygiene behaviour. Theoretical and practical implications of these findings are discussed.
Introduction
Healthcare-associated infections are a rising threat to global health systems. Infections acquired while receiving healthcare can cause severe impairments to patients’ physical and mental health, increase death rates, and add massive costs to health systems [1,2]. According to one estimate, over four million patients suffer from healthcare-associated infections in Europe annually. Of these cases, approximately 37,000 patients die as a direct consequence [3]. Due to the rise of antimicrobial resistant organisms, this number is likely to increase in the future [4]. Contact with contaminated healthcare workers’ hands has long been known as the main vehicle for transmitting pathogens that cause healthcare-associated infections [2]. In addition, patients’ and visitors’ hands have been recently identified as a severe risk factor for the spread of germs [5,6].
Improving hand hygiene has proven to be an effective method of decreasing infection rates, as well as the spread of antimicrobial resistant organisms [7–9]. However, hand hygiene practice in healthcare is still largely insufficient. On average, healthcare personnel in industrialised countries around the world clean their hands in only 40% of the cases required by guidelines [10]. Patients’ and visitors’ hand hygiene behaviour is estimated to be even lower [11,12].
Psychological factors influencing hand hygiene behaviour
From a theoretical standpoint, Ajzen’s theory of planned behaviour [13] is the most widely used model to explain hand hygiene behaviour [14]. The theory postulates that behaviour–in this case, hand hygiene–is driven by the intention to perform this action. The intention is predicted by three factors: first, attitude, which is formed by knowledge and beliefs about hand hygiene and its outcomes. Second, subjective norms, shaped by a person’s perception of how others think about hand hygiene. Third, perceived behaviour control, which is composed of beliefs about the ease or difficulty of performing hand hygiene [15]. Some researchers have reported positive correlations between all three factors and hand hygiene behaviour among healthcare personnel [16–19]. However, in other studies, not all factors correlated with hand hygiene [15,20,21]. No single variable emerged as being particularly influential above and beyond the others.
Scholars have indicated that focusing solely on one theory is too narrow of an approach [22,23]. Therefore, researchers have examined predictors for non-compliance with hand hygiene guidelines among healthcare workers within the theoretical domains framework, which is based on a range of behavioural theories [24]. They found three theoretical domains to predict more than three-quarters of non-compliance: memory/attention/decision making (i.e., forgetting, being distracted or prioritising another task); knowledge (i.e., lack of knowledge about guidelines); and environmental context/resources (i.e., lack of time or availability of products) [23].
To sum everything up, several variables have been identified as predictors for hand hygiene behaviour in hospitals. The quintessence of the research suggests the following: 1) Hand hygiene products need to be available; 2) People need to know when to perform hand hygiene; 3) The environment should be designed to prevent forgetfulness and support hand hygiene behaviour; and 4) Social norms should encourage hand hygiene behaviour. Alcohol-based hand-rub products–the gold-standard for hand hygiene–are widely accessible in Western hospitals [25]. However, further research is needed to establish a solid theoretical foundation concerning the timing of hand hygiene behaviour, environmental design, and the role of social norms.
Previous interventions to improve hand hygiene behaviour
Numerous interventions have been previously implemented to increase hand hygiene compliance with guidelines among healthcare personnel [26–33]. This volume of literature signifies the importance, but also the difficulties, associated with finding approaches to shift behaviour in this domain. One challenge with interventions is that they only modestly improve hand hygiene practice. For instance, relying solely on signs and posters seems to have no sizeable effect on improving hand hygiene behaviour [34,35]. Even when an intervention substantially increases compliance rates, the positive effect often lasts only for a short period of time. This is especially true after on-off education-based programmes, where compliance rates have been found to fall back to pre-intervention levels [34,36]. However, previous studies targeting the three aforementioned factors (i.e., knowledge of when to perform hand hygiene, reminders to prevent forgetfulness, and social norms) have shown promising results.
One intervention to improve knowledge via reoccurring, goal-oriented and reinforcing personal feedback, led to significantly higher hand hygiene compliance over three years compared to a control condition [37]. However, the programme was labour-intensive, which negatively affected implementation. Automated electronic monitoring with feedback systems has been suggested as an alternative to direct observation and personal feedback, as this approach decreases workload and eliminates observation biases [32]. The main benefit is that these systems can indicate when to perform hand hygiene at all times. One study used visual and auditory prompts at alcohol-based hand-rub dispensers, as a reminder to perform hand hygiene whenever someone passed by [38]. While the outcomes were promising, additional evidence is needed to verify the study’s results. More advanced systems have multiple sensors in the patient area to monitor hand hygiene opportunities. Here, healthcare workers need to wear badges, which register hand hygiene opportunities and events (i.e., using an alcohol-based hand-rub dispenser). The badges operate as indicators of when to perform hand hygiene, providing real-time feedback about hand hygiene status via light [39,40] or sound [41]. The greatest benefit of extensive electronic monitoring systems is that they offer constant, non-punitive, and direct feedback, which has been shown to be the most effective form of feedback [42]. However, such systems have limitations: 1) Wireless systems might interfere with medical equipment; 2) The systems are expensive and need maintenance; and 3) Healthcare workers are not comfortable with being constantly monitored [43]. Results of the effects of extensive systems on hand hygiene compliance have been mixed but promising [43]. Overall, interventions aimed to increase knowledge on when to perform hand hygiene via reminders and feedback have been shown to improve hand hygiene behaviour in hospitals. However, all attempts showed various limitations, and there is still room for improvement. As these systems are very expensive, it should be sufficiently tested which exact factors have a positive effect on hand hygiene behaviour prior to their implementation.
Previous research has also successfully targeted social norms to improve hand hygiene behaviour in hospitals. One study found that adding social norm elements (e.g., setting team norms and role modelling) to a traditional intervention (including education, reminders and group feedback) further improved hand hygiene compliance among hospital workers [44]. However, this approach is labour intensive, which makes it hard to uphold over time. Another intervention used a remote video auditing system to improve hand hygiene compliance by publicly sharing performance feedback [45]. The adherence rate increased from 6.5% to 81.6%, but the intervention was expensive because of the cost associated with camera installation and the ongoing evaluation of video footage. Additionally, video surveillance would not be acceptable in many countries and, as mentioned before, people feel uneasy about being constantly monitored.
Previous campaigns have typically used role modelling and peer pressure to activate social norms (e.g., [44]). However, there might be less labour-intensive ways to target social norms. Research has shown that displaying images of eyes or eye-like illustrations increases pro-social behaviour through the feeling of being observed. This phenomenon is known as the watching eyes effect, and it has been found in lab-based studies (e.g., [46]) as well as in field experiments (e.g., [47,48]). However, it should be noted that a recently conducted meta-analysis examining the surveillance effect on generosity raised scepticism on its robustness [49]. Besides this, the watching eyes effect has been previously tested on hand hygiene behaviour; in a series of experiments, pictures of human eyes were placed at hand-rub dispensers in hospitals [50–52]. Two [50,52] out of the three studies found that dispensers with images of eyes were used more frequently than dispensers without such illustrations, while the other [51] found no significant difference. One possible, but controversially discussed, explanation for the watching eyes effect could be that eye cues make people more inclined to follow salient local or so-called descriptive norms (i.e., ‘I should do what everybody else is doing’) ([53,54] but see [47] for opposing view). This norm-based approach might explain some of the mixed findings in the hand hygiene studies, as descriptive norms could vary between healthcare facilities. Additionally, scholars have argued that utilising descriptive norms on their own can backfire, because descriptive norms can differ considerably from injunctive norms (i.e., ‘I should do, what ought to be done’) [55,56]. Therefore, if the eye-like image makes descriptive norms salient, and a healthcare professional or layperson thinks that most other people do not perform hand hygiene, they would be more likely to copy this bad behaviour to comply with the norm. A simple solution for this problem was offered in the context of energy consumption [56]: Using emoticons as a form of injunctive normative feedback to prevent the boomerang effect, since the images convey social disapproval (frowny face) or approval (smiley face) very clearly. Negatively and positively valanced emoticons have also been successfully used in behavioural change settings to reduce food waste [57], water consumption [58] and speeding [59]. However, to our knowledge, they have never been applied in interventions to improve hand hygiene behaviour in hospitals. In summary, utilising social norms to change behaviour has been shown to be successful, but relying on descriptive norms might backfire. Therefore, it should be investigated if targeting injunctive norms might prove to be a more reliable approach in this highly sensitive field.
The present study
An electronic monitoring and feedback system was installed on top of alcohol-based hand-rub dispensers inside patient rooms in a German hospital, in order to measure room traffic and display visual cues. Only entrances and exits were monitored via motion sensors, which has been shown to capture around 80% to 85% of all possible hand hygiene opportunities in patient care [43]. This has three main benefits: First, it is an adequate but cost-efficient approach compared to more extensive systems. Second, healthcare staff find entrance/exit monitoring more acceptable than being under constant surveillance [60]. Third, it includes patients and visitors in addition to healthcare personnel, as recommended by German guidelines for hand hygiene in healthcare facilities [25]. The last point is a strong advantage of the present study, as many previous interventions only targeted healthcare professionals.
The objective of the present study was to test whether or not visual reminders with the potential to activate social norms affect hand hygiene behaviour in hospital patient rooms. Based on the literature reviewed, three research questions were formulated:
Research question 1: Does the display of emoticons, which is assumed to induce injunctive norms by providing instant feedback about current hand hygiene status, improve hand hygiene behaviour in patient rooms?
Research question 2: Can images of human eyes, which potentially make descriptive norms salient, improve hand hygiene practice in patient rooms? (Replication of the watching eyes effect)
Research question 3: Are visual cues displaying emoticons more effective than visual cues displaying human eyes in changing hand hygiene behaviour?
Materials and method
Design
This field experiment was conducted at the Rottal Inn Hospital in Eggenfelden, Germany. In consultation with the hygiene specialists at the hospital, eight out of the sixteen patient rooms within the trauma surgery unit were randomly chosen. This unit was selected for the pilot intervention because it usually has the highest patient turnover rate, a diverse mix of patients (e.g., age, gender and disease patterns) and is a high-risk area for post-surgical infections. These rooms can be accessed by healthcare staff, patients and visitors. The eight rooms are nearly identical in design, with two patient beds and an ensuite bathroom. Each room only has one dispenser, but there is another one in the hallway next to each room’s entrance. Ethical approval for the study was obtained from the UCL Research Ethics Committee. Both the Ethics Committee and the hospital's management team waived the requirement for informed consent as the experiment was of a passive nature. At no point did we collect any personal data (e.g., names, gender) of patients and/or medical staff. Our experimental design and research protocol did not influence existing processes, necessary to allow the smooth operation of the hospital.
Manipulations
This study had a baseline and an intervention phase. The latter consisted of four conditions (see Fig 1), which are described below. To aid discoverability and possible synthesis of the results with other interventions of health psychology studies, the relevant components of the Behaviour Change Technique Taxonomy v1 (BCT-T-v1) [61] are outlined within each condition description. 1) Screen-Emoticons: In the emoticon condition, the screen at the device turned on whenever the motion sensor was activated (i.e., when people entered or left the room), displaying a frowny face. The frowny turned into a smiley when the hand-rub dispenser was used. The frowny stayed activated for 25 seconds unless the dispenser was used, in which case the smiley was also displayed for 25 seconds. After either 25-second period, the screen switched off again. The Screen-Emoticons condition was expected to attract attention to the fact that hands should be cleaned at this point and prevent forgetfulness through visual cues (BCT-T-v1: prompts/cues). It was supposed to activate injunctive norms via the frowny face by providing direct feedback about the hand hygiene status for the user (wrong behaviour: sad face; right behaviour: happy face; BCT-T-v1: feedback on behaviour and social reward).
2) Screen-Neutral: In the first neutral control condition, the screen turned on whenever the motion sensor was activated, displaying a neutral image of a flower that changed to a slightly different flower when the dispenser was used (the same logic as in the Screen-Emoticon condition). The changing image was also supposed to act as a reminder for hand hygiene by attracting attention but not activating social norms (BCT-T-v1: prompts/cues).
3) Picture-Neutral: In the second neutral control condition, there were no changing screens. Instead, a stationary image of a flower was presented at all times. In the Picture-Neutral condition, neither social norms were activated nor feedback provided (BCT-Tv-1: prompts/cues).
4) Picture-Eyes: The setup of the eye condition was identical to the Picture-Neutral condition, except that an image of human eyes was presented. The picture was a visual cue to remind people about hand hygiene, and was intended to make social norms salient through the feeling of being observed. No feedback about hand hygiene status was provided (BCT-Tv1: prompts/cues and social comparison).
All images were displayed in black and white to avoid the influence of any colour. Also, the emotional quality of all pictures was pre-tested in a survey, as has been done in other studies regarding hand hygiene (e.g., [62]). The participants rated to what extent the images elicited different emotions on a five-point scale, from 1 (not at all) to 5 (very much). The measure of the emotion guilt was of special interest, as it was expected to be a suitable proxy for the pictures’ ability to activate social norms in the hospital. The survey data showed that the picture of the frowny and both eye images had significantly higher guilt-ratings than the three other pictures, which we interpreted as being suitable candidates to induce social norms in the experiment (for statistical analysis see S1 Appendix).
Measures
The dependent variable was the percentage (ratio) of hand hygiene events (HHE, i.e., using the hand-rub dispenser) among all motion sensor activations, which covers every entrance and exit of the rooms. Electronic devices were installed on top of the existing dispensers to measure room traffic, HHE, and display the images. Those devices (see Fig 2) were white plastic boxes, which included: 1) Infrared motion sensors with a sensor range of 60cm to register people passing by the dispenser; 2) Screens to display the above-mentioned images, or frames for the stationary pictures; 3) Magnetic sensors to measure dispenser usage (i.e., pressing the lever); and 4) Computers to process the information. The monitors and stationary pictures within the devices were five inches (11cm x 6.5cm) in size and in good visual sight from the entrance. However, the overall number of motion sensor activations is not a perfect proxy for room entrance and exit, as everyone who uses the bathroom within a patient room passes the sensor. Thus, the total number of activations also includes the traffic between beds and bathrooms.
The data collected included a time measure, the counts of motion sensor activations (i.e., entering and leaving the patient room or bathroom), and HHE (i.e., usage of dispenser, which counts as HHE independent of how many times the lever was pressed). When the motion sensor was activated, a period of 25 seconds started. In each sensor period, it was measured whether or not a HHE occurred or not. The motion sensor ignored all movements within this period of 25 seconds so that the initial picture could be presented.
Procedure
Prior to the study set-up, staff and patients were informed about the experiment. The study lasted 17 weeks. In the first eight weeks, the baseline HHE ratio was measured within the eight rooms. During the baseline phase, all screens were off, and there were no images on the devices without screens. A nine-week intervention phase followed immediately afterwards. At the start of the intervention phase, two rooms each, making up the total of eight rooms, were randomly allocated to one of the four conditions (Screen-Emoticons, Screen-Neutral, Picture-Eyes, Picture-Neutral). The images presented on the screens in the Screen-Emoticons and Scree-Neutral conditions remained the same during the entire intervention phase. However, the pictures in the conditions with the stationary devices (Picture-Eyes and Picture-Neutral) were changed after the first month of the intervention: Flower 1 was replaced with Flower 2, and Eyes 1 was replaced with Eyes 2. This was done to control for minor differences in the emotional quality of the eye images, and to include both images of the neutral stimuli (Flower 1 and Flower 2) within the stationary devices. The data was collected on regular occasions.
Results
In total, 65,907 hand hygiene opportunities (motion sensor activations) and 3,340 HHE were registered over the entire study period. The data was arranged to provide one data point with the average of hand hygiene opportunities and HHE for every week. This approach should have led to a total of 17 data points per room–eight at the baseline phase and nine at the intervention phase. However, in the baseline phase, devices in three of the eight rooms encountered technical issues. Data from two weeks are missing in one of the Screen-Neutral rooms and data from six weeks are missing in one of the Screen-Emoticon and one of the Picture-Eyes rooms respectively. In addition, the motion sensor activation data for one room of the Screen-Neutral condition in the intervention phase was not recorded for four weeks, even though all sensors and the screen worked. HHE data was documented without problems. In total, 8.8% of the data were missing. Missing data were replaced using the multiple imputation tool available from SPSS 25 by creating 20 imputed data sets, which has been recommended in the literature [63]. Both motion sensor activations and hand hygiene events over the 17 weeks were included in the imputation model. The average of the pooled results was then taken as the estimate of the missing value for the analysis.
Hand hygiene event ratio
It was tested whether or not there were any differences between the conditions regarding the dependent variable HHE ratio. HHE ratio per week was calculated by dividing the number of HHE by the number of motion sensor activations, and multiplying by 100 (i.e., calculating the percentage of HHE overall motion sensor activations). The eight weeks of data from the baseline phase were combined to form the baseline measure, and the nine weeks of data from the intervention phase were divided into two timepoints (intervention timepoint 1 included the data from May, and intervention timepoint 2 the data from June) to see if the intervention’s effect was stable over time. The data was clustered in this manner because both room traffic and hand hygiene behaviour are affected by contextual factors on a short-term basis. These factors included fluctuations in room occupancy, patients’ need for care, and seasonal influences such as holidays. While these aspects might bias the hand hygiene event ratio in the short run, they should be randomized out over a longer period. It should be mentioned that there were temporal changes on a weekly basis. However, as mentioned, these fluctuations cannot be interpreted without considering all contextual factors. The HHE data were analysed using a two-way repeated-measures ANOVA with phase (baseline vs intervention timepoint 1 vs intervention timepoint 2) as the within-subjects factor and condition (Screen-Emoticons vs Screen-Neutral vs Picture-Eyes vs Picture-Neutral) as the between-subjects factor. HHE ratio was normally distributed, as assessed by Normal Q-Q Plot. No studentized residual had a value greater than ±3, therefore, this implied that there were no outliers in the data set. The assumption of sphericity was met according to the Mauchly’s test of sphericity, χ2 = 4.498, p = .106. The ANOVA revealed a significant interaction between the phase of the experiment and the four conditions on HHE ratio, F(6, 8) = 11.128, p = .002, = .893. There was also a significant main effect for phase, F(2, 8) = 70.721, p < .001, = .946, and a main effect for condition, F(3, 4) = 19.969, p = .007, = .937. Fig 3. shows the mean ratios for all conditions between the timepoints.
To further investigate whether or not the HHE ratio differed between conditions in the baseline and intervention phase (intervention timepoint 1 and intervention timepoint 2), three separate one-way ANOVAs were conducted. No statistically significant difference in HHE ratio between the four conditions at the baseline phase was found, F(3, 4) = 2.065, p = .248, = .608. Accordingly, the Tukey HSD post hoc analysis found no mean HHE ratio differences between the four groups Screen-Emoticons (M = 4.180, SD = 0.711), Screen-Neutral (M = 2.738, SD = 0.755), Picture-Eyes (M = 3.429, SD = 0.448) and Picture-Neutral (M = 3.424, SD = 0.265).
There was a significant difference in HHE ratio between the conditions at the first timepoint during the intervention phase, F(3, 4) = 29.797, p = .003, = .957. The post hoc test revealed that the Screen-Emoticon group (M = 12.518, SD = 0.118) had significantly higher means than Screen-Neutral (p = .005), Picture-Eyes (p = .005) and Picture-Neutral (p = .005). There were no significant differences between Screen-Neutral (M = 5.280, SD = 1.278), Picture-Eyes (M = 5.506, SD = 0.592) and Picture-Neutral (M = 5.511, SD = 1.172). This means that the average HHE ratio at 12.5% in the emoticon condition was statistically higher than in the other three conditions (Screen-Neutral: 5.3%, Picture-Eyes: 5.5% and Picture-Neutral: 5.5%) at intervention timepoint 1.
HHE ratio also varied significantly between the conditions at the second timepoint in the intervention phase, F(3, 4) = 8.145, p = .035, = .859. According to the Tukey HSD test, Screen-Emoticons (M = 9.041, SD = 1.653) differed significantly from Picture-Neutral (M = 4.928, SD = 0.367, p = .037) and marginally significantly from Screen-Neutral (M = 5.634, SD = 0.256, p = .067) and Picture-Eyes (M = 5.613, SE = 0.666, p = .066), while there was no mean difference between the other conditions. These results mean that the HHE ratio of the Screen-Emoticon group at 9.0% at intervention timepoint 2 was still significantly higher than in the Picture-Neutral rooms at 4.9%, and slightly higher than in the Screen-Neutral at 5.6% and Picture-Eyes at 5.6%.
To test if the HHE ratio differed within the conditions over time, a repeated measures ANOVA over the data, split into conditions, was conducted. There was a significant effect of phase on the HHE ratio for the Screen-Emoticons condition, F(2, 2) = 44.702, p = .022, = .978. The HHE ratio in the Screen-Emoticons group increased from 4.2% at baseline phase to 12.5% at intervention timepoint 1, which is almost a three-fold surge in hand hygiene behaviour. Also, the 9.0% HHE ratio at the second intervention timepoint was still higher than the 4.2% HHE ratio for the baseline measure. At intervention timepoint 2, the dispensers in the emoticon rooms were used more than twice as often as during the baseline measure. However, HHE ratio decreased from 12.5% at intervention timepoint 1 to 9% at intervention timepoint 2. The post hoc analysis did not reveal any significant effects for the Screen-Emoticons condition. A significant simple main effect for phase on HHE ratio was found for the Screen-Neutral condition as well, F(2, 2) = 19.109, p = .050, = .950. The HHE ratio in the Screen-Neutral condition increased from 2.7% at baseline phase to 5.3% at intervention timepoint 1. The HHE ratio remained comparatively stable within the intervention phase (5.2% and 5.6% respectively). Again, the post-hoc analysis did not show any significant results. No significant effect of phase on HHE ratio was found for the Picture-Eyes condition, F(2, 2) = 6.703, p = .130, = .870, or the Picture-Neutral condition, F(2, 2) = 9.390, p = .096, = .904.
Discussion
In the present study, we explored if targeting social norms improves hand hygiene behaviour in a hospital. The most important finding was that the emoticon-based feedback system significantly increased the usage of hand-rub dispensers in patient rooms, in comparison to the three other tested conditions. This result indicates that the activation of injunctive norms through emoticons, which potentially act as indicators to perform the behaviour to prevent forgetfulness, is an effective approach to improve hand hygiene behaviour.
The primary aim of the data analysis was to investigate the HHE ratio differences between the four different conditions, Screen-Emoticons, Screen-Neutral, Picture-Eyes and Picture-Neutral, over the three phases (baseline, intervention timepoint 1 and timepoint 2). The analysis showed that there were no baseline differences in HHE ratio among the four groups. This implies that prior to the intervention, the rooms in the four conditions did not differ in their usage of the dispensers. This changed after the implementation of the intervention. The dispensers in the rooms with screens displaying a frowny/smiley were used more than twice as often as dispensers in the other patient rooms (i.e., the other three conditions) over the first month after implementation. In the second month, dispensers in the Screen-Emoticons rooms were still used more regularly than the ones in the other three conditions on a descriptive level, but the mean difference was smaller, and only compared to the Picture-Neutral condition was this difference statistically significant.
These results can be used to answer the three postulated research questions. First, in the present experiment, the display of emoticons indeed improved hand hygiene behaviour in the patient rooms. As indicated by previous research [56], it was assumed that emoticons would activate injunctive norms (i.e., ‘I should do, what ought to be done’) by providing instant feedback about a person’s hand hygiene status. The appearance of the frowny face likely acted as an indicator for people that hand hygiene behaviour should occur at this point, and that non-compliance was unacceptable. After using the dispenser, the screen changed from a frowny to a smiley face. This positively reinforced the hand hygiene behaviour performed by showing that this was the desired thing to do. Both aspects might have helped to prevent forgetfulness, which is particularly important for patients and visitors, who receive no formal training. It appears as if injunctive feedback was the main driver for the improvement of HHE ratio in this study. The hand hygiene behaviour in the Screen-Neutral condition did not improve as substantially as in the Screen-Emoticon condition. This suggests that the positive effect in the Screen-Emoticons condition cannot solely be explained through attracting attention via the changing screens. Thus, research question 1 was affirmed in general. However, the positive effect of the emoticon-based feedback system lessened over time. More research is needed to identify the main reason for this decline and find ways to mitigate the reduction in effectiveness.
Presenting eye-like stimuli did not improve hand hygiene behaviour in the patient rooms. Therefore, the watching eyes effect was not found in the present study, and research question 2 must be negated. This is in accordance with other studies, which have also failed to replicate the watching eyes effect (see for instance [49]). It has been suggested that the surveillance-effect might depend on the number of other people present in the situation. For instance, two experiments found no strong positive effect of eye-stimuli on donations when a supermarket was highly frequented with customers [64,65], and two other studies found similar results on littering behaviour ([66,67] but see [47] for an opposing result). In our experiment, there were usually several people present in the patient rooms. If the eye-cue indeed activates a search for references about how other people behave and no other hospital employee, patient or visitor cleans their hands, skipping hand hygiene might be accepted as the appropriate norm. This could be one possible explanation for why the watching eyes effect was not found. A second explanation could be that our images looked rather feminine, and were drawings instead of real photographs. In some previous studies, the watching eyes effect was only found for pictures of sombre looking male eyes (e.g., [52] but see [53] for an opposing result). On the other hand, the watching eye effect has even been found in experiments using so-called eye spots [46,64], which are not images of real human eyes but can be interpreted as eyes. A third—and according to the literature increasingly more plausible—explanation would be that eye cues indeed do not notably influence human behaviour [49].
Lastly, it was expected that visual cues displaying emoticons, which potentially activate injunctive norms by providing instant feedback, would improve hand hygiene behaviour more effectively than eye images, which we believe to make descriptive norms salient. The data supported this assumption, and thus, research question 3 can be answered in the affirmative. This result builds on the findings of previous research [56], which also found that targeting injunctive norms was more robust in changing people’s behaviour than descriptive norms.
The present experiment is the first study to test the effect of activating injunctive norms through an emoticon-based feedback system on hand hygiene behaviour in a hospital. To our knowledge, none of the studies investigating the effectiveness of emoticons in traffic–which from a technical standpoint are similar to the present experiment–tested underlying psychological mechanisms. But, it is plausible that the psychological principles are the same. Therefore, the results of this experiment contribute to the theoretical understanding of how injunctive norms might modify behaviour in other research areas as well. Our results add promising and novel findings about the effectiveness of applying injunctive norms in health behaviour change interventions to the literature.
This study had several limitations. First, the sample size of eight patient rooms was small. The limited number of rooms impacted the data analysis because it inhibited the assumptions of homogeneity of variance and equality of covariance. Despite the small sample size, the results could be interpreted due to the strength of the measured effects. Therefore, subsequent studies should include more rooms. Second, some technical problems occurred. As this study was a pilot intervention, technical challenges were to be expected. However, the missing data points had no severe impact on the general experiment. Third, it is difficult to compare the results from this experiment with other interventions to increase hand hygiene behaviour for two reasons. Firstly, in the present study, hospital staff, patients and visitors were not distinguishable. Therefore, it is not possible to infer if the intervention affected each group of people differently. However, most other studies focused only on hospital employees. Secondly, the motion sensor activations were not a perfect proxy for hand hygiene opportunities. One reason for this is that traffic to and from the patients’ bathroom are included in that data. Another reason is that the motion sensors ignored all movements within a 25 second period after activation. If more people entered the room simultaneously, entered and left the room within this period, or lingered right in front of the dispenser, the log file would only show one motion action and connect all HHE occurring within this period to this activation. Therefore, HHE ratios measured in this study must not be misunderstood as a compliance rate, which is usually the dependent measure to evaluate interventions. The final limitation is concerned with the quality of hand hygiene. As with all electronic measures of hand hygiene, no assertion about the quality of hand hygiene behaviour (i.e., if enough hand-rub was used and if it was properly spread over the hands), as well as about the necessity to perform a HHE (e.g., if a nurse entered the room without touching the patient and the patient’s environment), can be made.
In conclusion, this work presents preliminary evidence that inducing injunctive norms via emoticons can improve hand hygiene behaviour in hospitals. The emoticon-based reminders indicated to people that hand hygiene behaviour should be performed via a frowny emoticon. Consequently, a smiley face positively reinforced adequate behaviour (i.e., using the dispenser). Hand hygiene rates in the emoticon-condition significantly outperformed the three other conditions. However, the positive effect seems to be rather temporary. Therefore, future research should be conducted to find optimal ways on how the positive effect of triggering injunctive norms can be employed to sustainably increase hand hygiene compliance in healthcare facilities. These findings might also be applied to other healthcare-related behaviour change settings.
Supporting information
Acknowledgments
We thank Karsten Weder from the Rottal Inn Hospital in Eggenfelden for his support in organising the experimental setup in the hospital. Thanks to Laura Woods for proofreading this version. Also, the authors would like to thank two anonymous reviewers for their helpful comments and advice.
Data Availability
All relevant data are within the paper and its Supporting Information files.
Funding Statement
This work was supported by the German Research Foundation (DFG) within the funding programme Open Access Publishing.
References
- 1.WHO. Report on the Burden of Endemic Health Care-Associated Infection Worldwide. Clean Care is Safer Care. Available from http://www.who.int/gpsc/country_work/burden_hcai/en/2011.
- 2.WHO. WHO Guidelines on Hand Hygiene in Health Care: A Summary. Available from http://www.who.int/gpsc/5may/tools/9789241597906/en/2009.
- 3.ECDC. Annual Epidemiological Report on Communicable Diseases in Europe. Available from https://ecdc.europa.eu/sites/portal/files/media/en/publications/Publications/0812_SUR_Annual_Epidemiological_Report_2008.pdf2008.
- 4.ECDC. Antimicrobial resistance surveillance in Europe. Annual report of the European Antimicrobial Resistance Surveillance Network (EARS-Net). Available from https://ecdc.europa.eu/en/publications-data?f[0]=publication_series%3A2542#show-more2015.
- 5.Birnbach DJ, Rosen LF, Fitzpatrick M, Arheart KL, Munoz-Price LS. An evaluation of hand hygiene in an intensive care unit: Are visitors a potential vector for pathogens? J Infect Public Health. 2015;8(6):570–574. doi: 10.1016/j.jiph.2015.04.027 [DOI] [PubMed] [Google Scholar]
- 6.Istenes N, Bingham J, Hazelett S, Fleming E, Kirk J. Patients’ potential role in the transmission of health care-associated infections: Prevalence of contamination with bacterial pathogens and patient attitudes toward hand hygiene. Am J Infect Control. 2013;41(9):793–798. doi: 10.1016/j.ajic.2012.11.012 [DOI] [PubMed] [Google Scholar]
- 7.Allegranzi B, Pittet D. Role of hand hygiene in healthcare-associated infection prevention. J Hosp Infect. 2009;73(4):305–315. doi: 10.1016/j.jhin.2009.04.019 [DOI] [PubMed] [Google Scholar]
- 8.Barnes SL, Morgan DJ, Harris AD, Carling PC, Thom KA. Preventing the transmission of multidrug-resistant organisms: Modeling the relative importance of hand hygiene and environmental cleaning interventions. Infect Control Hosp Epidemiol. 2014;35(9):1156–1162. doi: 10.1086/677632 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Sroka S, Gastmeier P, Meyer E. Impact of alcohol hand-rub use on meticillin-resistant Staphylococcus aureus: An analysis of the literature. J Hosp Infect. 2010;74(3):204–211. doi: 10.1016/j.jhin.2009.08.023 [DOI] [PubMed] [Google Scholar]
- 10.Erasmus V, Daha TJ, Brug H, Richardus JH, Behrendt MD, Vos MC, et al. Systematic review of studies on compliance with hand hygiene guidelines in hospital care. Infect Control Hosp Epidemiol. 2010;31(3):283–294. doi: 10.1086/650451 [DOI] [PubMed] [Google Scholar]
- 11.Birnbach DJ, Nevo I, Barnes S, Fitzpatrick M, Rosen LF, Everett-Thomas R, et al. Do hospital visitors wash their hands? Assessing the use of alcohol-based hand sanitizer in a hospital lobby. Am J Infect Control. 2012;40(4):340–343. doi: 10.1016/j.ajic.2011.05.006 [DOI] [PubMed] [Google Scholar]
- 12.Ardizzone LL, Smolowitz J, Kline N, Thom B, Larson EL. Patient hand hygiene practices in surgical patients. Am J Infect Control. 2013;41(6)487–491. doi: 10.1016/j.ajic.2012.05.029 [DOI] [PubMed] [Google Scholar]
- 13.Ajzen I. The theory of planned behavior. Organ Behav Hum Decis Process. 1991;50(2):179–211. doi: 10.1016/0749-5978(91)90020-T [Google Scholar]
- 14.Godin G, Bélanger-Gravel A, Eccles M, Grimshaw J. Healthcare professionals' intentions and behaviours: A systematic review of studies based on social cognitive theories. Implement Sci. 2008;3:36 doi: 10.1186/1748-5908-3-36 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.O'Boyle CA, Henly SJ, Larson E. Understanding adherence to hand hygiene recommendations: The theory of planned behavior. Am J Infect Control. 2001;29(6):352–360. doi: 10.1067/mic.2001.18405 [DOI] [PubMed] [Google Scholar]
- 16.Eiamsitrakoon T, Apisarnthanarak A, Nuallaong W, Khawcharoenporn T, Mundy LM. Hand hygiene behavior: Translating behavioral research into infection control practice. Infect Control Hosp Epidemiol. 2013;34(11):1137–1145. doi: 10.1086/673446 [DOI] [PubMed] [Google Scholar]
- 17.Whitby M, McLaws M-L, Ross MW. Why healthcare workers don't wash their hands: A behavioral explanation. Infect Control Hosp Epidemiol. 2006;27(5):484–492. doi: 10.1086/503335 [DOI] [PubMed] [Google Scholar]
- 18.McLaws M-L, Maharlouei N, Yousefi F, Askarian M. Predicting hand hygiene among Iranian health care workers using the theory of planned behavior. Am J Infect Control. 2012;40(4):336–339. doi: 10.1016/j.ajic.2011.04.004 [DOI] [PubMed] [Google Scholar]
- 19.Pittet D, Simon A, Hugonnet S, Pessoa-Silva CL, Sauvan V, Perneger TV. Hand hygiene among physicians: performance, beliefs, and perceptions. Ann Intern Med. 2004;141(1):1–8. doi: 10.7326/0003-4819-141-1-200407060-00008 [DOI] [PubMed] [Google Scholar]
- 20.Pessoa-Silva CL, Posfay-Barbe K, Pfister R, Touveneau S, Perneger TV, Pittet D. Attitudes and perceptions toward hand hygiene among healthcare workers caring for critically ill neonates. Infect Control Hosp Epidemiol. 2005;26(3):305–311. doi: 10.1086/502544 [DOI] [PubMed] [Google Scholar]
- 21.Sax H, Uckay I, Richet H, Allegranzi B, Pittet D. Determinants of good adherence to hand hygiene among healthcare workers who have extensive exposure to hand hygiene campaigns. Infect Control Hosp Epidemiol. 2007;28(11):1267–1274. doi: 10.1086/521663 [DOI] [PubMed] [Google Scholar]
- 22.Cane J, O’Connor D, Michie S. Validation of the theoretical domains framework for use in behaviour change and implementation research. Implement Sci. 2012;7:37 doi: 10.1186/1748-5908-7-37 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Fuller C, Besser S, Savage J, McAteer J, Stone S, Michie S. Application of a theoretical framework for behavior change to hospital workers’ real-time explanations for noncompliance with hand hygiene guidelines. Am J Infect Control. 2014;42(2):106–110. doi: 10.1016/j.ajic.2013.07.019 [DOI] [PubMed] [Google Scholar]
- 24.Michie S, Johnston M, Abraham C, Lawton R, Parker D, Walker A. Making psychological theory useful for implementing evidence based practice: A consensus approach. BMJ Qual Saf. 2005;14(1):26–33. doi: 10.1136/qshc.2004.011155 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.KRINKO. Händehygiene in Einrichtungen des Gesundheitswesens. Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz. 2016;9:1189–1220. doi: 10.1007/s00103-016-2416-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Schweizer ML, Reisinger HS, Ohl M, Formanek MB, Blevins A, Ward MA, et al. Searching for an optimal hand hygiene bundle: A meta-analysis. Clin Infect Dis. 2013;58(2):248–259. doi: 10.1093/cid/cit670 [DOI] [PubMed] [Google Scholar]
- 27.Luangasanatip N, Hongsuwan M, Limmathurotsakul D, Lubell Y, Lee AS, Harbarth S, et al. Comparative efficacy of interventions to promote hand hygiene in hospital: Systematic review and network meta-analysis. BMJ. 2015;351:h3728 doi: 10.1136/bmj.h3728 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Gould DJ, Moralejo D, Drey N, Chudleigh JH. Interventions to improve hand hygiene compliance in patient care. Cochrane Database Syst Rev. 2010;9 doi: 10.1002/14651858.CD005186.pub3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Huis A, van Achterberg T, de Bruin M, Grol R, Schoonhoven L, Hulscher M. A systematic review of hand hygiene improvement strategies: A behavioural approach. Implement Sci. 2012;7:92 doi: 10.1186/1748-5908-7-92 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Allegranzi B, Sax H, Pittet D. Hand hygiene and healthcare system change within multi-modal promotion: A narrative review. J Hosp Infect. 2013;83:3–10. doi: 10.1016/S0195-6701(13)60003-1 [DOI] [PubMed] [Google Scholar]
- 31.Srigley J, Corace K, Hargadon D, Yu D, MacDonald T, Fabrigar L, et al. Applying psychological frameworks of behaviour change to improve healthcare worker hand hygiene: A systematic review. J Hosp Infect. 2015;91(3):202–210. doi: 10.1016/j.jhin.2015.06.019 [DOI] [PubMed] [Google Scholar]
- 32.Srigley J, Gardam M, Fernie G, Lightfoot D, Lebovic G, Muller M. Hand hygiene monitoring technology: A systematic review of efficacy. J Hosp Infect. 2015;89(1):51–60. doi: 10.1016/j.jhin.2014.10.005 [DOI] [PubMed] [Google Scholar]
- 33.Shlomai NO, Rao S, Patole S. Efficacy of interventions to improve hand hygiene compliance in neonatal units: A systematic review and meta-analysis. Eur J Clin Microbiol Infect Dis. 2015;34(5):887–897. doi: 10.1007/s10096-015-2313-1 [DOI] [PubMed] [Google Scholar]
- 34.Naikoba S, Hayward A. The effectiveness of interventions aimed at increasing handwashing in healthcare workers: A systematic review. J Hosp Infect. 2001;47(3):173–180. doi: 10.1053/jhin.2000.0882 [DOI] [PubMed] [Google Scholar]
- 35.Birnbach DJ, Rosen LF, Fitzpatrick M, Everett-Thomas R, Arheart KL. A ubiquitous but ineffective intervention: Signs do not increase hand hygiene compliance. J Infect Public Health. 2017;10(3):295–298. doi: 10.1016/j.jiph.2016.05.015 [DOI] [PubMed] [Google Scholar]
- 36.von Lengerke T, Schulz-Stübner S, Chaberny IF, Lutze B. Psychologie der Händehygiene-Compliance: Von der Motivation zum Verhalten. Krankenhaushygiene up2date. 2016;11(2):135–150. doi: 10.1055/s-0042-107371 [Google Scholar]
- 37.Fuller C, Michie S, Savage J, McAteer J, Besser S, Charlett A, et al. The Feedback Intervention Trial (FIT)—improving hand-hygiene compliance in UK healthcare workers: a stepped wedge cluster randomised controlled trial. PLoS One. 2012;7(10):e41617 doi: 10.1371/journal.pone.0041617 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Venkatesh AK, Lankford MG, Rooney DM, Blachford T, Watts CM, Noskin GA. Use of electronic alerts to enhance hand hygiene compliance and decrease transmission of vancomycin-resistant Enterococcus in a hematology unit. Am J Infect Control. 2008;36(3):199–205. doi: 10.1016/j.ajic.2007.11.005 [DOI] [PubMed] [Google Scholar]
- 39.Storey S, FitzGerald G, Moore G, Knights E, Atkinson S, Smith S, et al. Effect of a contact monitoring system with immediate visual feedback on hand hygiene compliance. J Hosp Infect. 2014;88(2):84–88. doi: 10.1016/j.jhin.2014.06.014 [DOI] [PubMed] [Google Scholar]
- 40.Marra AR, Camargo TZS, Magnus TP, Blaya RP, Dos Santos GB, Guastelli LR, et al. The use of real-time feedback via wireless technology to improve hand hygiene compliance. Am J Infect Control. 2014;42(6):608–611. doi: 10.1016/j.ajic.2014.02.006 [DOI] [PubMed] [Google Scholar]
- 41.Fisher DA, Seetoh T, May-Lin HO, Viswanathan S, Toh Y, Yin WC, et al. Automated measures of hand hygiene compliance among healthcare workers using ultrasound: Validation and a randomized controlled trial. Infect Control Hosp Epidemiol. 2013;34(9):919–928. doi: 10.1086/671738 [DOI] [PubMed] [Google Scholar]
- 42.Hysong SJ, Best RG, Pugh JA. Audit and feedback and clinical practice guideline adherence: making feedback actionable. Implement Sci. 2006;1:9 doi: 10.1186/1748-5908-1-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Conway LJ. Challenges in implementing electronic hand hygiene monitoring systems. Am J Infect Control 2016;44(5):7–12. doi: 10.1016/j.ajic.2015.11.031 [DOI] [PubMed] [Google Scholar]
- 44.Huis A, Schoonhoven L, Grol R, Donders R, Hulscher M, van Achterberg T. Impact of a team and leaders-directed strategy to improve nurses’ adherence to hand hygiene guidelines: A cluster randomised trial. Int J Nurs Stud. 2013;50(4):464–474. doi: 10.1016/j.ijnurstu.2012.08.004 [DOI] [PubMed] [Google Scholar]
- 45.Armellino D, Hussain E, Schilling ME, Senicola W, Eichorn A, Dlugacz Y, et al. Using high-technology to enforce low-technology safety measures: The use of third-party remote video auditing and real-time feedback in healthcare. Clin Infect Dis. 2011;54(1):1–7. doi: 10.1093/cid/cir773 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Haley KJ, Fessler DM. Nobody's watching?: Subtle cues affect generosity in an anonymous economic game. Evol Hum Behav. 2005;26(3):245–256. doi: 10.1016/j.evolhumbehav.2005.01.002 [Google Scholar]
- 47.Bateson M, Callow L, Holmes JR, Roche MLR, Nettle D. Do images of ‘watching eyes’ induce behaviour that is more pro-social or more normative? A field experiment on littering. PLoS One. 2013;8(12):e82055 doi: 10.1371/journal.pone.0082055 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48.Bateson M, Nettle D, Roberts G. Cues of being watched enhance cooperation in a real-world setting. Biol Lett. 2006;2(3):412–414. doi: 10.1098/rsbl.2006.0509 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.Northover SB, Pedersen WC, Cohen AB, Andrews PW. Artificial surveillance cues do not increase generosity: Two meta-analyses. Evol Hum Behav. 2017;38(1):144–153. doi: 10.1016/j.evolhumbehav.2016.07.001 [Google Scholar]
- 50.Beyfus TA, Dawson NL, Danner CH, Rawal B, Gruber PE, Petrou SP. The use of passive visual stimuli to enhance compliance with handwashing in a perioperative setting. Am J Infect Control. 2016;44(5):496–499. doi: 10.1016/j.ajic.2015.12.021 [DOI] [PubMed] [Google Scholar]
- 51.Kuliga S, Tanja-Dijkstra K, Verhoeven F. Wa(tc)sh out! The effects of cues of being watched on implicit activation of norms and hand disinfection behaviour. In Haans A, van Gennip DA, Ham P, de Kort YAW, Midden CJA, editors. Proceedings of Envionment 2.0: Proceedings of the 9th Biennial Conference on Environmental Psychology; 2011 Sep 26–28. Eindhoven, The Netherlands. Eindhoven University of Technology; 2011. Available from http://proceedings.envpsych2011.eu/index.html.
- 52.King D, Vlaev I, Everett-Thomas R, Fitzpatrick M, Darzi A, Birnbach DJ. “Priming” hand hygiene compliance in clinical environments. Health Psychol. 2016;35(1):96–101. doi: 10.1037/hea0000239 [DOI] [PubMed] [Google Scholar]
- 53.Nettle D, Harper Z, Kidson A, Stone R, Penton-Voak IS, Bateson M. The watching eyes effect in the Dictator Game: It's not how much you give, it's being seen to give something. Evol Hum Behav. 2013;34:35–40. doi: 10.1016/j.evolhumbehav.2012.08.004 [Google Scholar]
- 54.Oda R, Kato Y, Hiraishi K. The watching-eye effect on prosocial lying. Evol Psychol. 2015;13:1–6, doi: 10.1177/1474704915594959 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 55.Cialdini RB, Reno RR, Kallgren CA. A focus theory of normative conduct: Recycling the concept of norms to reduce littering in public places. J Pers Soc Psychol. 1990;58(6):1015–1026. doi: 10.1037/0022-3514.58.6.1015 [Google Scholar]
- 56.Schultz PW, Nolan JM, Cialdini RB, Goldstein NJ, Griskevicius V. The constructive, destructive, and reconstructive power of social norms. Psychol Sci. 2007;18(5):429–434. doi: 10.1111/j.1467-9280.2007.01917.x [DOI] [PubMed] [Google Scholar]
- 57.Nomura H, John PC, Cotterill S. The use of feedback to enhance environmental outcomes: A randomised controlled trial of a food waste scheme. Local Environ. 2011;16(7):637–653. doi: 10.1080/13549839.2011.586026 [Google Scholar]
- 58.Schultz PW, Messina A, Tronu G, Limas EF, Gupta R, Estrada M. Personalized Normative Feedback and the Moderating Role of Personal Norms. Environ Behav. 2016;48(5):686–710. doi: 10.1177/0013916514553835 [Google Scholar]
- 59.Ariën C, Cornu J, Brijs K, Brijs T, Vanroelen G, Jongen EM, et al. Measuring the impact of digital information displays on speed: A driving simulator study. In: Transportation Research Board. The 92nd TRB Annual Meeting 2013: The 2012 Annual Meeting Compendium of Papers; 2013 Jan 13–17; Washington D.C., U.S.: Transportation Research Board. 2013, p. 1–17. Available from https://doclib.uhasselt.be/dspace/bitstream/1942/14625/1/TRB_2013_Ari%C3%ABnetal.pdf.
- 60.Ellingson K, Polgreen PM, Schneider A, Shinkunas L, Kaldjian LC, Wright D, et al. Healthcare personnel perceptions of hand hygiene monitoring technology. Infect Control Hosp Epidemiol. 2011;32(11):1091–1096. doi: 10.1086/662179 [DOI] [PubMed] [Google Scholar]
- 61.Michie S, Richardson M, Johnston M, Abraham C, Francis J, Hardeman W., et al. The behavior change technique taxonomy (v1) of 93 hierarchically clustered techniques: Building an international consensus for the reporting of behavior change interventions. Ann Behav Med. 2013;46(1): 81–95. doi: 10.1007/s12160-013-9486-6 [DOI] [PubMed] [Google Scholar]
- 62.Porzig-Drummond R, Stevenson R, Case T, Oaten M. Can the emotion of disgust be harnessed to promote hand hygiene? Experimental and field-based tests. Soc Sci Med. 2009;68(6):1006–1012. doi: 10.1016/j.socscimed.2009.01.013 [DOI] [PubMed] [Google Scholar]
- 63.Graham JW, Olchowski AE, Gilreath TD. How many imputations are really needed? Some practical clarifications of multiple imputation theory. Prev Sci (2007);8, 206–213. doi: 10.1007/s11121-007-0070-9 [DOI] [PubMed] [Google Scholar]
- 64.Powell KL, Roberts G, Nettle D. Eye images increase charitable donations: Evidence from an opportunistic field experiment in a supermarket. Ethology. 2012;118(11):1096–1101. doi: 10.1111/eth.12011 [Google Scholar]
- 65.Ekström M. Do watching eyes affect charitable giving? Evidence from a field experiment. Exp Econ. 2012;15(3):530–546. doi: 10.1007/s10683-011-9312-6 [Google Scholar]
- 66.Ernest-Jones M, Nettle D, Bateson M. Effects of eye images on everyday cooperative behavior: A field experiment. Evol Hum Behav. 2011;32(3):172–178. doi: 10.1016/j.evolhumbehav.2010.10.006 [Google Scholar]
- 67.Bateson M, Robinson R, Abayomi-Cole T, Greenlees J, O’Connor A, Nettle D. Watching eyes on potential litter can reduce littering: Evidence from two field experiments. PeerJ 3:e1443 doi: 10.7717/peerj.1443 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
All relevant data are within the paper and its Supporting Information files.