Experimental Pretesting of Hand-Washing Interventions in a Natural Setting

Abstract

Objectives. We pretested interventions derived from different domains of behavior change theory to determine their effectiveness at increasing hand washing with soap in a natural setting.

Methods. We installed wireless devices in highway service station restrooms to record entry and soap use. Two text-only messages for each of 7 psychological domains were compared for their effect on soap-use rates. We collected data on nearly 200 000 restroom uses.

Results. The knowledge activation domain was most effective for women, with a relative increase in soap use of 9.4% compared with the control condition (P = .001). For men, disgust was the most effective, increasing soap use by 9.8% (P = .001). Disgust was not significantly better than the control condition for women, nor was knowledge activation for men. Messages based on social norms and social status were effective for both genders.

Conclusions. Our data show that unobtrusive observation of behavior in a natural setting can help identify the most effective interventions for changing behaviors of public health importance. The gender differences we found suggest that public health interventions should target men and women differently.

Health promotion is expensive, with large sums invested worldwide in trying to change people's behavior. Yet many behavior-change campaigns prove to be largely ineffective, such as those targeting binge drinking,¹ diet,^2,3 adolescent sexual practices,^4,5 or washing hands with soap.^6,7 It would be helpful if the components of a behavior-change campaign were pretested for effectiveness before rolling out the campaign at scale.⁸ Despite the potential advantages of early pretesting, relatively few public health interventions have employed this tactic.

Hand washing with soap (hereafter referred to as “hand washing”) has been ranked the most cost-effective intervention for the worldwide control of disease.^9,10 It could save more than a million lives a year from diarrheal diseases¹¹ and prevent respiratory infections,¹² the 2 biggest causes of child mortality in developing countries. In developed countries, hand washing is important in preventing the spread of viral infections, including norovirus,¹³ rotavirus,¹⁴ and influenza^15,16; it also prevents the spread of hospital-acquired infections such as methicillin-resistant Staphylococcus aureus and Clostridium difficile.¹⁷ A recent study that we conducted in the United Kingdom showed that a quarter of commuters sampled had fecal bacteria on their hands.¹⁸ However, we still do not know the most effective way of encouraging hand washing.

A common problem with studies of behavior is that self-reporting can be unreliable. Hand washing, like many health-related behaviors, is socially desirable and morally laden and hence tends to be vastly overreported in interviews.^19,20 An alternative to self-reporting is direct observation. However, methods such as structured observation of hand washing can also provide biased estimates of real behavior because being watched can affect hand-washing practice.²¹ Unobtrusive monitoring of hand-washing behavior could provide an unbiased estimate of natural behavior as well as a means of measuring the efficacy of different interventions.

Advocates argue that theory-based interventions have been shown to be more effective than other interventions.^22,23 However, there are many theories to choose from, each with several component domains and each capable of being instantiated in many forms. The epidemiological gold standard for testing interventions, the randomized controlled trial, can only compare a few interventions against a control. Comparing interventions simultaneously is a cost-effective means of maximizing the effects of a program and of avoiding expensive mistakes in large-scale programs. It could also be used in theory development, because the performance of interventions derived from different theoretical domains can be compared.

For this study, we pretested the effects of messages targeting a range of theoretical domains important to behavior change by comparing many different interventions simultaneously to determine which were the most effective.⁸ Previous work consolidated the many theoretical constructs within psychological theories of behavior into types of domains, such as knowledge, behavioral regulation, and motivation, for practical use in designing health interventions.²⁴ This work provided a theoretical foundation for our study, supplemented by a review of empirical knowledge from formative research studies about the motivations for hand washing, which showed that the determinants of hand washing include disgust, comfort, affiliation (or social norms, the desire to conform with what others do or with what appears to be socially acceptable behavior), and habit.^25,26 Disgust²⁷ and social norms^28,29 (elicited by the visible presence of other people in public restrooms) have been shown to increase hand washing in other studies. Although health education is a common strategy, it has been shown to be ineffective at changing hygiene behaviors.^30,31 One recent study showed that interventions relying on disgust were more effective than educational messages at increasing hand washing.³²

METHODS

The restrooms of a highway service station in England provided a natural setting in which to observe the typical hand-washing behavior of a large volume of people. Because the study took place during the summer holidays (July–September 2008), service station users included families, young people, and the elderly, as well as the usual weekday business clientele, thus representing a wide spectrum of the traveling public in England.

Study Design

Baseline recording of restroom entry and soap use indicated that hand-washing rates varied over the course of the day, so for randomization purposes, we split the day into 4 time blocks of 6 hours each and displayed individual messages for 1 hour at a time. Within each time block, the baseline rate of hand washing remained relatively constant. Each message was displayed once in every 3-day period, in each of the 4 time blocks. Within these constraints, the hour in which a particular message was displayed was randomly determined. Each message was displayed for a mean of 44 hours (range = 39–49 hours; the large range resulted from random loss of data caused by occasional technical faults).

The interventions consisted of 18 types of exposure—7 experimental domains (each with 2 messages) and 2 different control conditions—which took the form of text-only messages displayed on an electronic dot matrix screen over the entryway to the 2 restrooms, in full view of people entering. Each control message was shown for the equivalent time of 2 messages, for reasons of statistical power. During a treatment, everyone who used the restroom in that hour could see the same message as they entered. The messages were in capital letters, and they flashed for the duration of their presentation to attract attention (except the blank control condition, when no message was displayed).

Procedures

Data were collected anonymously with wireless sensors: infrared sensors at the entryway recorded the number of people entering and leaving the public restroom, and sensors inside soap dispensers recorded each soap use. Data from these wireless sensors allowed us to calculate the ratio of people entering the restroom to those who used soap. Unlike in studies that measured the volume of soap used,^33,34 this method allowed us to calculate the ratio of soap users over short periods, allowing the randomization of many interventions.

As a test for exposure to the intervention, a researcher approached 111 people leaving the restrooms (73 men and 38 women) at different times of day and days of the week to see whether they recalled seeing the message. They were asked, “Did you see an electronic sign hanging from the ceiling at the entrance to the toilets? If so, what did it say?” The answers to the second question were counted as correct even if the reported phrase was not exactly the same as the message, as long as the respondent captured the basic meaning.

To check the reliability of the infrared sensors, a researcher counted the number of people entering and leaving the restrooms in 20 time slots, each lasting 30 minutes, at different times of day and at different days of the week. We compared these results with those from the sensors and calculated the discrepancy by dividing the difference between the 2 measures by the number of people determined by the researcher.

The pilot data indicated that we could expect 80 presentations (i.e., display of a message over a 1-hour period) per domain, giving 80% power to detect a 3% increase (absolute) in the ratio of soap use to entry ratio for men and 8% for women, compared with the control condition.

Development of Intervention Messages

Interventions were text-only messages (so as to minimize the opportunity for variability in message impact and effectiveness attributable to graphic elements). The messages were limited to a maximum of 48 characters by the display technology and all included the word soap. The domains and their definitions were chosen at a specially convened workshop of 13 health psychologists and behavior change and marketing experts. First, we presented the group with empirical data about hand hygiene²⁷ and with the 11 domains from a paper consolidating behavior change theory.²⁶ The experts then selected 7 domains for pretesting that they judged to be most relevant for this behavior:

• Knowledge of risk. Inform people about a fact they may not know, which holds a danger for them (e.g., water alone will not kill germs, you need soap).

• Knowledge activation. Remind people of what they know already or convince them of the importance of what they know. Unusual presentation can serve to activate this existing knowledge by increasing processing of the message.

• Norms or affiliation. Raise concern for social judgments on people's hygiene behaviors because of the knowledge that others might be concerned with standards for acceptable behavior.

• Status or identity. Help people to feel that hand washing—or more broadly, cleanliness and being hygienic—is an important aspect of their self-image.

• Comfort. Emphasize positive sensory qualities of having clean hands.

• Disgust. Trigger the arousal of a “yuck” response.

• Cue. Provide people with a behavioral rule triggered by an object in the environment or an event (e.g., I'm coming out of a restroom) that encourages them to use soap after using the toilet.

Two messages were tested for each domain. One control condition had no message displayed, and the other was a positive control that simply stated, “Wash your hands with soap.” A list of potential intervention messages for each domain was drawn up at the workshop, and further messages were developed by a professional creative agency. The final set of messages to be pretested was determined by e-mail voting by the workshop attendees and is shown in the second column of Table 1. The 18 exposures comprised 14 intervention messages and 2 control conditions, which were repeated twice for reasons of statistical power.

TABLE 1.

Mean Soap Ratios and Significance Levels Across Different Messages in an Intervention to Encourage Hand Washing: United Kingdom, July–September 2008

Soap Ratio	Message	Domain	Relative Increase From Control Condition, %	P
Men
0.312	Blank	Blank control (Ref)	…	…
0.321	Blank	Blank control (Ref)	…	…
0.325	Toilet germs soap hands clean	Knowledge activation	2.7	.488
0.328	Sticky hands? Get that soapy smoothness	Comfort	3.6	.35
0.331	Water doesn't kill germs, soap does	Knowledge of risk	4.4	.267
0.337	Don't be a dope, wash with soap	Status/Identity	6.5	.09
0.338	Wsah yuor hnads wiht saop	Knowledge activation	6.7	.065
0.339	Shake hands confidently—Wash with soap	Norms/Affiliation	6.9	.09
0.339	Wash your hands with soap	Positive control	7.0	.067
0.341	See sink? Use soap	Cue	7.5	.073
0.341	Washing hands with soap avoids 47% of disease	Knowledge of risk	7.6	.033
0.342	Toilet—sink—think: soap	Cue	7.9	.029
0.346	Soap it off or eat it later	Disgust	9.3	.012
0.347	Wash your hands with soap	Positive control	9.4	.017
0.349	Don't be a dirty soap dodger	Status/Identity	10.3	.013
0.350	Don't take the loo with you—wash with soap	Disgust	10.3	.005
0.354	Soap adds a fresh touch	Comfort	11.7	.004
0.355	Is the person next to you washing with soap?	Norms/Affiliation	12.1	.001
Women
0.620	Soap adds a fresh touch	Comfort	−4.8	.143
0.648	Blank	Blank control (Ref)	…	…
0.654	Blank	Blank control (Ref)	…	…
0.662	Toilet—sink—think: soap	Cue	1.7	.562
0.663	Soap it off or eat it later	Disgust	1.9	.565
0.670	Shake hands confidently—wash with soap!	Norms/Affiliation	2.9	.375
0.680	Sticky hands? Get that soapy smoothness!	Comfort	4.4	.173
0.683	Don't be a dope, wash with soap	Status/Identity	4.9	.155
0.691	See sink? Use soap	Cue	6.2	.078
0.694	Washing hands with soap avoids 47% of disease	Knowledg of risk	6.5	.051
0.702	Don't be a dirty soap dodger	Status/Identity	7.7	.014
0.705	Don't take the loo with you—wash with soap	Disgust	8.3	.013
0.707	Wash your hands with soap	Positive control	8.6	.015
0.709	Wsah yuor hnads wiht saop	Knowledge activation	8.9	.007
0.709	Wash your hands with soap	Positive control	8.9	.005
0.714	Toilet germs soap hands clean	Knowledge activation	9.7	.004
0.722	Is the person next to you washing with soap?	Norms/Affiliation	10.9	.001
0.723	Water doesn't kill germs, soap does	Knowledge of risk	11.1	.001

Statistical Analysis

For each 1-hour interval during which a message was displayed, we calculated the ratio of soap dispenser uses to the number of people entering the facility (separately for the men's and women's facilities). This ratio was our measure of the proportion of people washing their hands. Soap dispenser uses within 5 seconds of each other were treated as a single use. Data were discarded for the first 5 minutes of each hour to avoid including the soap use of people who had seen the message of the previous hour in the calculation for the soap-use ratio for the new message. (Their entry was still included in the totals for the previous hour, but this splitting of different activities by 1 person was inevitable with any cutoff point.)

For each intervention message, we calculated the ratio of soap users as the average of the soap-use ratios of all hours during which the message was displayed, weighted by the number of entries per hour. We compared the effects of different messages on the soap-use ratio with the blank control through univariate linear regression weighted by the number of entries per hour. The ratio of soap users within each hour-long treatment was the dependent variable; the particular message displayed was the predictor variable in the model. To investigate the social effect in more detail, we also looked at the correlation between the ratio of soap users and the number of people in the restroom. We used Stata, version 10 (StataCorp LP, College Station, TX) for all analyses.

RESULTS

We collected 32 days of data, measuring more than 108 000 male restroom uses and more than 90 000 female restroom uses. The mean number of male users per day was 3691 (SD = 579) and of female users was 3096 (SD = 1132).

The weighted average of the soap ratios by domain for men and women are shown in Table 2. The ratio of soap users during control periods (when the screen was blank) was 0.317 for men and 0.651 for women.

TABLE 2.

Mean Soap Ratios and Significance Levels Across Different Domains of Behavior-Change Theory in an Intervention to Encourage Hand Washing: United Kingdom, July–September 2008

Domain	Soap Ratio	Relative Increase, %	P
Men
Blank control (Ref)	0.317	…	…
Knowledge activation	0.333	5.1	.093
Knowledge of risk	0.336	6.0	.044
Comfort	0.341	7.5	.02
Cue	0.341	7.7	.014
Positive control	0.343	8.2	.01
Status/Identity	0.343	8.3	.012
Norms/Affiliation	0.347	9.6	.003
Disgust	0.348	9.8	.001
Women
Blank control (Ref)	0.651	…	…
Comfort	0.654	0.6	.832
Cue	0.674	3.5	.178
Disgust	0.683	5.0	.078
Status/Identity	0.692	6.4	.021
Norms/Affiliation	0.698	7.3	.008
Knowledge of risk	0.706	8.6	.003
Positive control	0.708	8.9	.002
Knowledge activation	0.711	9.4	.001

Note. Domains are ranked in descending order.

Most domains showed a small but significant increase in soap-use ratio when compared with the blank control; however, the pattern of results was very different for men and women. For men, disgust and norms were the most effective (with disgust corresponding to a 9.8% relative increase compared with the control; P = .001), followed by status, positive control, cue, and comfort; knowledge of risk and knowledge activation were the least effective.

For women, all domains showed an increase; however, some were not significantly better than the control condition. Knowledge activation, positive control, and knowledge of risk were the most effective (with knowledge activation corresponding to a 9.4% increase compared with the control condition; P = .001), followed by norms and status. Disgust and cue were only slightly better than the control condition, and comfort only marginally so.

Knowledge activation was the top-performing domain for women but was ineffective for men. Disgust triggered the highest response in men but produced no significant response in women. Norms and status were effective for both genders (although slightly better for men), as was the positive control condition. When we compared the different domains with each other, rather than with the control condition, we found no statistically significant differences.

We also looked at the results broken down by message (Table 1). The only message that performed well in both genders was the normative message, “Is the person next to you washing with soap?” which resulted in a 12.1% relative increase in hand-washing ratio among men and a 10.9% increase among women compared with the control condition. The only comfort message to perform significantly better than the control condition in men was, “Soap adds a fresh touch.” However, this message performed worse (although not significantly) than the control condition for women. Among the knowledge-based messages (i.e., knowledge of risk and knowledge activation), messages including the word germs were the most successful for women but the least successful for men.

In some cases, 2 messages from the same domain caused different degrees of change in behavior. Three of these differences within domains were statistically significant: the effect of comfort messages in both women (P = .027) and men (P = .087) and norms messages for women (P = .048). These statistically significant findings may have resulted from real differences in effectiveness or in some cases from chance, because of our multiple statistical comparisons. However, it should be noted that in the data for men, the 2 randomizations of the positive control message also performed differently, even though they were exactly the same message. This random variation may account for some of the apparent differences between messages within a domain.

Messages based on norms were effective at increasing soap use for both genders, which indicates at least an implicit concern for how others perceive one's hygiene behavior. This could be manifested in increased levels of hand washing when there are more people in the restroom. The correlation between the soap ratio and the number of people in the restroom in a given hour is plotted in Figure 1. (Although this was a measure of the intensity of restroom use rather than the number of people in the restroom when a person was washing his or her hands, it can be assumed that there was a relationship between the number of people using the restroom in an hour and the number present at a given time during that hour.) Figure 1a shows the data for men, which indicate a trend for the soap ratio to increase during hours when there are more people in the restroom (r = 0.31; P < .001). The association was minimal in the data for women (Figure 1b; r = 0.12; P = .002).

FIGURE 1.

Ratio of restroom users washing their hands with soap to number of people in the restroom among (a) men and (b) women: United Kingdom, July–September 2008.

Note. For men, r = 0.31; P < .001. For women, r = 0.12; P = .002. Horizontal lines are linear regressions through the data.

We further investigated this social effect, to see whether it differed by domain. The correlation coefficient for the ratio of people using soap against the number of people in the restroom in a given hour was calculated for each domain (but not the control condition) and plotted against the mean soap ratio for that domain (Figure 2). We found a clear positive correlation, showing that the more effective the domain, the greater the social effect it had—that is, effective messages were even more effective during periods when the restrooms were busier (r = 0.75; P = .02). We again found no similar correlation in women (r = −0.55; (P = .16).

FIGURE 2.

Social effect correlation coefficient among men for each domain plotted against the mean soap ratio: United Kingdom, July–September 2008.

Note. r = 0.75; P = .002.

When questioned after visiting the restroom, 33% of men and 35% of women reported seeing the sign; 23% of men and 29% of women could remember characteristic aspects of the message displayed. The test of the infrared sensors showed that the discrepancy in the data ranged from 2% to 19% for men and from 2% to 25% for women. The discrepancy was larger at busy times and greatest between noon and 2 pm.

DISCUSSION

We designed our study to pit interventions based on various domains against one another. Our study was thus novel in both purpose and method: to our knowledge it was the first to experimentally assess several theoretical domains for their relative ability to increase hand washing. Our study demonstrated that messages can be assessed in a natural context for their ability to increase hand washing. It allowed for comparisons between theoretical domains of behavior change and messages that were not directly compared before. Unobtrusive monitoring allowed us to avoid the biases inherent in structured observation or self-reporting of behavior and proved a reliable means of data collection.

In general, we found that most of the interventions increased levels of hand washing compared with the blank control condition. Unexpectedly, we also found that men and women responded to health promotion messages in different ways. Men but not women responded well to disgust-based messages; women but not men responded well to messages based on activating preexisting knowledge about the dangers of failing to wash hands. Both genders responded well to messages based on norms. Indeed, the only message that was effective for both genders was the norms message, “Is the person next to you washing with soap?” This was the most effective message for men and the second most effective for women. However, replication is needed, because these were post hoc rather than planned comparisons.

Considering the gender differences we observed, the relatively low effect of disgust messages on women's behavior was surprising, because women have been shown to have a higher disgust sensitivity than men.^26,35,36 The suggestion that disgust—a powerful motivation arising from strong feelings—may not be elicited effectively in a text-only message cannot explain this finding, because disgust messages were effective for men. However, women's responses to knowledge-based messages have not previously been compared with their responses to motivational messages, so it is possible that women respond to disgust but that they respond even more strongly to knowledge-based messages or that the mention of germs in the knowledge messages elicited more disgust than did the disgust-based messages.

The social effect for men (more hand washing occurred when men's restrooms were busier) was stronger for the more effective messages; that is, the presence of other people in the restroom made successful messages more successful. It is probable that the subliminal knowledge that others were likely to be thinking about hand washing was stronger for the more successful messages and therefore prompted more hand washing. We did not observe this social effect in women.

Other studies have shown that the visible presence of another person in the restroom increases hand-washing rates among both men²⁸ and women.²⁹ This agrees well with our finding that messages based on social norms were effective for both genders but not with the lack of correlation in our data between soap use and the number of women using the restrooms. One explanation could be that women were already practicing a relatively high baseline rate of hand washing, so the women who might have been affected by the presence of other people might already have been soap users.

Limitations

Overall, the differences in effectiveness between messages and domains were relatively small, which may be explained by the message display. Although the sign was in a prominent location and the message was flashing, only one quarter of women and one third of men in the exposure survey reported seeing it. Despite this low rate of exposure, the most effective messages increased hand-washing rates by more than 10% (relative increase). Our power calculations did not take account of this low exposure rate; hence we did not have sufficient power to test whether certain domains were significantly more effective than others. Future studies should improve the display design or run the study for longer, to increase effect sizes and study power. However, because our data highlighted less successful approaches for each gender, these could be omitted in future studies, which could then focus on fewer interventions.

The use of many messages also leads to problems of multiple testing. We did not use the Bonferroni adjustment for multiple comparisons because of the high incidence of false negatives, preferring a more descriptive approach to the data rather than an overreliance on P values and significance levels.³⁷

Electronic counts incorporate their own biases—in the case of our study being less reliable at busy periods, because the wide entryway to the restrooms at the site allowed more than 1 person to cross the sensors at once. We eliminated these effects by randomizing treatments over time of day. However, it is likely that some of the association between the number of people in the restroom and soap use (Figure 1) may be explained by underestimates in the entry counts at peak times.

Further work is needed to assess the extent to which the messages selected were represented by the theoretical domains that we intended to test or whether they also worked via a different domain. For example, a knowledge activation message mentioning germs may also elicit disgust. Future work will refine the messages and determine whether they represent the intended domains. Finally, our equipment was unable to record the length or effectiveness of the hand washing, an issue that could also be addressed in future studies.

Conclusions

Our study demonstrated that different interventions can be experimentally pretested in a natural setting, that hand washing can be measured unobtrusively, and that hygiene behavior can be changed with a single exposure to a text-based message. To our knowledge, this study was the first to pretest the effect of many different interventions on an important public health behavior. Because trials usually compare a single intervention with a control condition, knowledge about the relative efficacy of different approaches is limited. The kind of information provided by this design should therefore be welcome to public health practitioners.

Knowledge about the basic motivators of hand washing in the general population, and how they may vary between different groups, is vital for the development of effective hand-washing campaigns aimed at controlling the spread of diseases such as pandemic influenza. Our study demonstrated how a novel first step in the development of hand-washing programs—pretesting potential interventions—can generate options that are more likely to be effective. More work is needed to refine the domains and text messages into polished interventions, but we have made a first step in investigating which theoretical domains have the biggest effect on hand-washing behavior and whether different population groups have different responses. A similar method of unobtrusive observation should also work for other kinds of behaviors important to public health, such as smoking cessation and alcohol moderation campaigns.