Observed and self-reported COVID-19 health protection behaviours on a university campus and the impact of a single simple intervention

Abstract

During the COVID-19 pandemic, most data on adherence to health protective behaviours were collected via a self-report. We quantified the discrepancy between self-report data and discretely observed behaviour in a sample of university staff and students. We assessed the prevalence of cleaning hands, wearing a face-covering and maintaining distance from others. We also tested whether additional signage reminding people that these behaviours were mandatory improved observed adherence. Prevalence estimates based on self-report were higher than those based on observations. Signage was associated with improvements for observed behaviours (all χ² ≥ 6.0, P < 0.05). We caution that self-reported data can produce misleading adherence rates.

Introduction

During the COVID-19 pandemic, the public were asked to adapt to their behaviour in an attempt to slow the spread of SARS-CoV-2.^1,2 Efforts to understand adherence to these changes, and factors associated with adherence, largely relied on the use of self-reported data gathered in cross-sectional surveys, often with low response rates or self-selected samples.³ Whereas these surveys can provide useful insight into trends over time, the accuracy of their data is unclear, particularly for behaviours that may occur many times a day, in different settings and are therefore prone to recall error.^4,5 In this study, we tested the accuracy of such cross-sectional surveys at a large university in London. We compared data gathered in this way with data based on discrete observation of behaviours. We also tested whether a simple intervention (installation of clear signage) could improve adherence to these behaviours. The behaviours explored were mandatory at the university: cleaning hands, wearing a face-covering and maintaining social distancing upon entering a building.

Methods

We used two study designs. First, we used an online cross-sectional survey to measure self-reported adherence to the three behaviours. Second, we used direct covert observation of behaviour at the only entrance to the main campus building of the university on two consecutive days. Day 1 consisted of the control condition. On day 2, we installed a large sign at the entrance reminding people about the university rules on COVID-19 protective behaviours. Survey responses were obtained between 1 December 2020 and 22 March 2021. The observational study took place during the term time on 15 and 16 March 2021.

For the survey, an invitation to participate was disseminated via a bi-weekly newsletter advertising research studies within the university. This is sent to all students and staff. The survey consisted of 16 items. Among other things, we asked:

• How often do you wear a mask when entering the [university] campus buildings?

• How often do you practice hand-hygiene measures when entering the [university] campus buildings?

• How often do you practice social distancing guidelines when within the [university] campus buildings?

Response options consisted of ‘always’, ‘often’ ‘sometimes’ ‘rarely’ and ‘never’.

For the observational study, a single observer was based outside the sole entrance to the main campus building on two consecutive days. On day 2, we erected a sign immediately outside the entrance stating the mandatory policy for mask wearing, hand-hygiene and social distancing within the building. (Supplementary Figures 1 and 2). Except for a 20-min period at 3 p.m., in which a short break was taken by the observer, every person who entered the building between 9 a.m. and 5 p.m. was observed.

Participants were observed for three behaviours upon entering the building: hand-hygiene, adequate wearing of a face-covering and adequate distancing from others. Hand-hygiene was defined as use of hand-sanitizer or gel, or use of a hand-washing station. Adequate face-covering was defined as the wearing of any covering or mask that covered both the mouth and nose of the participant. Adequate distancing was defined as remaining more than 2m away from others within the building. In some instances, distancing could not be assessed, for example, if an individual entered the building alone and the entrance area was empty. If groups entered the building together and were not distanced from each other on approach before entering the building, they were assumed to be a ‘bubble’ or household. Under guidance in place at the time, this negated the requirement for distancing. In order to keep the observations discrete, participants were not approached by the observer and no identifiable or personal information was collected.

We used χ² tests to assess differences between self-reporting ‘always’ performing a behaviour and levels of observed behaviour on day 1. We also used χ² tests to assess differences between levels of observed behaviour on days 1 and 2 in order to evaluate the impact of the signage intervention.

Results

In total, 252 participants responded to the cross-sectional survey, 311 people were observed on day 1 and 375 people on day 2. Table 1 shows the number reporting each of the three behaviours in the survey and the number of people performing each behaviour on days one and two. Many more people reported ‘always’ practicing hand-hygiene (68.3%, 95% confidence interval 62.1–70.4%) or socially distancing (49.2%, 42.9–55.6%) in the survey than was observed on day 1 (16.1%, 12.2–20.6% and 7.6%, 3.5–13.9%, respectively), but the difference between self-reported and observed rates of face-covering was smaller (89.6%, 85.2–93.1% vs 82.3%, 77.7–86.4%). All comparisons of observed versus self-reported behaviour were significant (all χ² ≥ 6.0, P < 0.05). Occurrence of all behaviours was significantly higher on day 2 of the study than day 1 (all χ² ≥ 13.3, P < 0.0005).

Table 1.

Self-reported vs observed hand-hygiene, face-covering use and social distancing

Self-reported behaviour						Observed behaviour
Behaviour	Always (n (%, 95% confidence interval))	Often (n (%, 95% confidence interval))	Sometimes (n (%, 95% confidence interval))	Rarely (n (%, 95% confidence interval))	Never (n (%, 95% confidence interval))	Day 1 (control) n/sample size (% (95% confidence interval))	Day 2 (signage) n/sample size (% (95% confidence interval))	Difference between self-reports of ‘always’ engaging in behaviour and day 1 observation	Difference between day 1 and 2 observations
Practices hand-hygiene when entering	172 (68.3%, 62.1% to 74.0%)	46 (18.3%, 13.7% to 23.6%)	11 (4.4%, 2.2% to 7.7%)	5 (2.0%, 0.7% to 4.6%)	18 (7.1%, 4.3% to 11.1%)	50/311 (16.1%, 12.2% to 20.6%)	104/375 (27.7%, 23.3% to 32.6%)	χ2 = 158.7, p = < 0.00001	χ2 = 13.3, p = 0.0003
Adequate mask wearing when entering	225 (89.6%, 85.2% to 93.1%)	8 (3.2%, 1.4% to 6.2%)	0 (0%, 0.0% to 1.5%)	1 (0.4%, 0.1% to 2.2%)	17 (6.8%, 4.0% to 11.0%)	256/311 (82.3%, 77.7% to 86.4%)	374/375 (99.7%, 98.5% to 99.9%)	χ2 = 6.0, p = 0.013	χ2 = 68.8, p < 0.00001
Maintains 2 m distancing from others	124 (49.2%, 42.9% to 55.6%)	82 (32.5%, 26.8% to 38.7%)	18 (7.1%, 4.3% to 11.1%)	8 (3.2%, 1.4% to 6.2%)	20 (7.9%, 4.9% to 12.0%)	9/119 (7.6%, 3.5% to 13.9%)	79/144 (54.9%, 46.4% to 63.2%)	χ2 = 70.0, p < 0.00001	χ2 = 65.5, p < 0.00001

Discussion

Our data demonstrate that self-report surveys commonly used to assess behavioural changes during the COVID-19 pandemic can substantially overestimate levels of adherence. This was particularly the case for hand-hygiene and social distancing. It is possible that this reflects the influence on self-report of recall or social desirability. Rates of self-reported face-covering use, still higher than observed rates, were more accurate. This may be because, as a clearly visible action, wearing a face-covering is more likely to be enforced or noticed by others. Our study also demonstrated that observational methods can be readily used to evaluate the impact of simple interventions to improve adherence. Our findings are consistent with other evidence on the efficacy of the use of notices and other environmental prompts for activating behaviour changes, including hand-hygiene.⁶ Where an accurate assessment of adherence to behavioural recommendations is required, observation should be the preferred method.

There are several limitations to this work. Although many studies in this area use self-selected samples, we did not use the quota sampling or weighting that are also common in this field. Surveys that use these methods tend to find high self-reported adherence to hand-hygiene and distancing behaviours nonetheless, suggesting this does not account for the rates that we observed.⁷ The observational study was limited to a single university building. Other populations or buildings may have had higher adherence. Difficulties assessing whether people were in household bubbles made it challenging to assess distancing and results based on these data should be taken as tentative. Strengths of this work include the use of an observational method to assess behaviour, something that has been remarkably rare during the COVID-19 pandemic⁸ and the comparison of observational data against the type of survey data that are regularly used to inform academic research in this field.

Data availability

Data Available on request from authors.

Conflict of interests

GJR participated in the UK Government’s Scientific Advisory Group for Emergencies and its subgroups and is acting as a legal expert witness in a case involving Bayer.

Funding

This study was funded by the National Institute for Health Research Health Protection Research Unit (NIHR HPRU) in Emergency Preparedness and Response, a partnership between the UK Health Security Agency, King’s College London and the University of East Anglia. The views expressed are those of the author(s) and not necessarily those of the NIHR, Public Health England or the Department of Health and Social Care.

Author Statements

Ethics

This study was approved by King’s College London’s BDM Research Ethics Subcommittee (Reference: HR-20/21-21752).

Rachel Davies, Dr

John Weinman, Professor

G. James Rubin, Professor