Evaluating the feasibility and acceptability of a safety protocol to mitigate SARS-CoV-2 transmission risks when participating in full-capacity live mass events: a cross-sectional survey and interview-based study

Abstract

Objective

Investigate the feasibility and acceptability of a novel COVID-19 safety protocol combining professionally witnessed home-based videoed pre-event testing and a data-driven risk assessment model that was piloted at the Standon Calling Festival in July 2021.

Design

Observational study using a sequential explanatory mixed-methods design involving a survey, personal interviews and group discussions with a cross section of participants.

Setting

Standon Calling Festival, Hertfordshire, England.

Participants

4726 adults who attended Standon Calling and consented to participate in the study.

Results

Nearly a quarter (23.1%; 1093) attendees (women 65%, men 35%) responded to the postevent survey. Eleven participants were interviewed before thematic saturation was reached. The majority (81.0%) of respondents found the at-home testing protocol convenient and of reasonable cost (73.6%). Confidence in the test result was enhanced due to professional-supported videoing (76.2%), whereas 72.6% had confidence in the security of the data. Videoed self-testing helped 45.0% of respondents to feel more confident in their lateral flow testing technique. The majority (85.5%) felt safer at the event and 93.7% agreed that the protocol did not interfere with their enjoyment of the event. Themes generated from interviews showed that the protocol could be applied to other disease areas and events, but there were concerns that over-reliance on test results alone could lead some people to have a false sense of security around the safety of the live event.

Conclusions

Our study showed that a protocol that combines professionally witnessed home-based videoed pre-event testing is highly acceptable and feasible, and it can inform decision making and support the safe reopening of live mass events at full capacity. Although COVID-19 is now considered endemic in the UK, this protocol can be of value for other countries where the live events industry remains heavily impacted. Risk modelling should be tested and evaluated at future events to further increase the robustness of this protocol.

Strengths and limitations of this study.

• This was the first study evaluating the feasibility and acceptability of a COVID-19 safety protocol for the safe reopening of live mass events at full capacity.

• We used a sequential explanatory mixed-methods design, whereby quantitative findings from an online questionnaire informed the collection of contextual data from interviews and one focus group discussion.

• We surveyed 1093 respondents who had attended the mass live event and surfaced assumptions from 11 participants to explore barriers and drivers for routine adoption of witnessed rapid lateral flow testing.

• We were unable to evaluate the utility of risk modelling as a decision tool because the event had already been given permission to go ahead; however, it was successfully conducted in real time, and its projections reflected the number of COVID-19 cases measured by Test and Trace.

Introduction

As part of the public health response to limit the spread of COVID-19, live events were suspended in the United Kingdom (UK) for several months since the first national lockdown in March. The lockdown negatively impacted the country’s economy and the mental health of individuals, calling for the deployment of innovative strategies to help reopen live events safely while enabling venues to operate at near or full capacity.1 2

Under the auspices of the UK government’s Events Research Programme (ERP), a small number of legally permitted live events took place during lockdown between April and July 2021, with the aim of investigating the risk of COVID-19 transmission to inform the reopening of live mass events. Attendees were required to show proof of a negative lateral flow test (LFT), double vaccination or natural immunity when entering the venue.3 Findings showed that COVID-19 transmission was in line with or below community level at most events.4 However, risk of transmission was highest at outdoor unseated events and may have been influenced by individual behaviour before and during the event, vaccination coverage, event size and duration as well as the prevalence of COVID-19 at the time of the event, suggesting that findings from the ERP may not be generalisable to all live events.

To account for contextual differences between events, and to further increase the effectiveness and usefulness of pre-event testing for safe reopening of live events at full capacity, the CAPACITY protocol combined mass pre-event LFT testing and a data-driven risk assessment model based on levels of immunity (vaccination status) in the event population.1 5 The premise was that at-home, professionally verified LFT testing, combined with ticket validation and real-time risk modelling of transmission at the event would catch asymptomatic infected ticket holders before travelling to the event and support public health authorities to make informed, data-driven decisions around the overall safety of the event. During ticket purchase, customers completed a risk questionnaire and purchased an LFT to be taken within 48 hours of entering the venue. The test was videoed and validated by a professionally trained testing control officer based on the assessment of the sample collection method. The test result was linked to a QR (Quick Response) code that would generate a go/no-go signal to attend the event. The risk prediction model would estimate the number of infectious people at the event, their infectivity and the risk of transmission at the event, based on parameters such as community background prevalence, characteristics of the venue and of the audience attending the event, including their vaccination status. By providing event-specific risk estimates, this protocol had the potential to inform ticketholders, event organisers, public health authorities and health systems to guide decision-making as to whether the live event should go ahead as planned.6

The CAPACITY protocol was trialled in July 2021 at Standon Calling Festival (SCF), an independent boutique music festival that takes place annually in Hertfordshire. Although SCF was cancelled in Summer 2020, the festival resumed in Summer 2021 at full capacity (~15 000 people), in part because the proposed CAPACITY safety protocol helped to assuage concerns around event safety at the time and was, therefore, the first event to be held outside of the ERP.7 8

The aim of our study was to investigate the views and perceptions of attendees at SCF on the feasibility and acceptability of the CAPACITY protocol using a mixed-methods design.

Methods

Study design

We adopted a sequential explanatory mixed-methods design, whereby quantitative data collection and analysis were followed by the collection and analysis of contextual, qualitative data.9 In the first phase of the study, we collected quantitative data through an online questionnaire. The analysis of quantitative data informed the second phase of the study, during which we collected and analysed data from in-depth semistructured interviews and a focus group discussion to further explore and expand the quantitative findings from the first phase of the study.

Quantitative methodology

Participants

Adults who attended SCF in July 2021 and gave permission to be contacted by the research team during ticket purchase were eligible to participate in the survey and were contacted via email (n=4726). The survey was voluntary, and participants were free to decide whether they wanted to take part.

Data collection

Participants were surveyed anonymously in the first 2 weeks of November 2021 through an online questionnaire to capture their views and experiences regarding the safety protocol put in place at SCF to minimise the spread and exposure to COVID-19. The survey was developed in collaboration with CERTIFIC and Standon Calling event organisers, including input from the Patient and Public Involvement group of the NW London Appied Research Collaborative, ensuring that questions were clear and in plain English. The survey was validated following two rounds of beta testing by members of the study team who also tested the usability and technical functionality prior to distribution. The survey was distributed on 3 November 2021 via web link on the Qualtrics platform, with one reminder sent the following week. The full survey is included in online supplemental file 1.

The online survey comprised of 17 total items organised in five different blocks. The first captured the sociodemographic characteristics of participants, including age, gender and ethnicity, as well as their COVID-19 vaccine status and frequency of COVID-19 testing at home before the event. The second block of questions captured participant feedback on pre-event videoed COVID-19 testing, focusing on the convenience of the procedure, data security, intrusiveness, cost and its impact on their confidence to self-test correctly. The third block investigated the experience and sense of safety of participants at the event, and the extent to which they adopted other protective measures against COVID-19. The last two set of questions sought to determine whether respondents tested positive to COVID-19 within 2 weeks after attending the event, explored general views on pre-event safety measures and whether respondents would recommend videoed testing for future events. The survey included multiple-choice, Likert 5-point scale and yes/no questions. Participants were asked to give consent at the start of the survey and were free to change their response up until the point of submitting the survey. All data were stored on a password-encrypted database which only the study team had access to.

Data analysis

Sociodemographic characteristics of respondents and responses to survey questions were analysed using routine descriptive statistical methods with STATA/SE V.17.0.

Qualitative methodology

Participants

Interviewees were randomly selected among survey respondents who voluntarily provided their contact details to be approached for a follow-up interview. Potential interviewees were contacted via email. All participants who agreed to be interviewed were asked to complete a consent form before the interview, and all gave verbal consent at the start of the interview.

Data collection

Semistructured personal interviews and a focus group discussion were conducted through Microsoft Teams between December 2021 and January 2022 by two researchers. The research team developed an interview guide with open-ended questions that was informed by findings from the online questionnaire, seeking to further explore the experience of participants at SCF, to capture the enablers and barriers to participate in COVID-secure live events, and to investigate attitudes and perceptions on pre-event safety measures and medically certified home-based testing. The semistructured interview guide is included in online supplemental file 2. Thematic saturation was achieved after seven in-depth personal interviews and one focus group discussion with four participants (n=11 total participants), each lasting between 30 min and 60 min. Interviews were recorded, auto-transcribed, manually checked and pseudonymised.

Data analysis

Interview transcripts were analysed inductively through thematic analysis by manually identifying codes and subsequently through the formation of themes. Two researchers reviewed the transcripts and agreed on the emergent themes, which were further discussed with the research team. Once emergent themes were categorised and considered in context, quotes from transcripts reflecting key themes were captured.

Ethics

The survey was anonymous, and respondents were not asked to provide personal details unless they voluntarily expressed to be contacted for a follow-up interview. Participants were free to withdraw from the survey or interview at any time. A lottery to receive free tickets to the next SCF was provided by the event organisers as an inducement for ticket holders to participate in the evaluation. Interview data were pseudonymised.

Patient and public involvement

No patient was involved in this study.

Results

A total of 1093 respondents completed the survey, and contextual data were collected by interviewing 11 participants. Results from the survey and interviews are reported in the following sections.

Survey

Results from the survey were reported using the CHERRIES (Checklist for Reporting Results of Internet E-Surveys) Checklist.10 A total of 1093 participants completed the survey out of the 4726 that were contacted (response rate 23.1%). Characteristics of survey respondents are summarised in table 1, and survey results are reported in online supplemental table S1.

Table 1.

Sample size and characteristics of respondents of the survey.

	N	%
Total	1093	100.0
Gender
Female	711	65.5
Male	374	34.5
Age
15–24	122	11.3
25–39	290	26.7
40–54	483	44.6
≥55	186	17.4
Ethnicity
White	1030	95.5
Mixed/multiple ethnic groups	28	2.6
Asian/Asian British	8	0.7
Black/African/Caribbean/Black British	2	0.2
Other	11	1.0
COVID-19 vaccine status before the event
Fully vaccinated (two doses)	832	76.2
Partially vaccinated (one dose)	220	20.2
Not vaccinated	41	3.6
Routine COVID-19 testing at home before the event
At least once a week	504	46.2
Not regularly	457	41.8
Never	131	12.0

Overall, most respondents agreed or strongly agreed that videoed pre-event testing was convenient (81.0%, figure 1) and not intrusive (82.8%), and about three-quarters of respondents agreed or strongly agreed that the cost of the test was reasonable (73.6%). A significant proportion of respondents agreed or strongly agreed that they were not concerned with the security of their data (72.6%), and about three-quarters of respondents agreed or strongly agreed that they felt confident in their results knowing that the pre-event testing was videoed (76.2%).

Figure 1.

Percentage of respondents who agreed or disagreed with various statements about videoed pre-event COVID-19 testing, safety and experience (n=1093).

Nearly half the respondents (45.0%) felt that having completed a videoed LFT increased their confidence to self-test correctly. Less than 1% of respondents (0.4%) felt they were still not confident to self-test correctly.

Most respondents agreed or strongly agreed that knowing that everyone at the event had been tested made them feel safer (85.5%, figure 1). The large majority disagreed or strongly disagreed that pre-event testing interfered with how much fun they had at the event (93.7%).

About half of respondents reported having regularly washed their hands during the event (51.2%), making it the most common form of prevention against COVID-19. One-third of respondents reported having socially distanced to some extent (33.4%), although only a small proportion reported having maintained social distancing at most times during the event (8.1%). More than half of respondents reported not having worn a face mask or covering at all throughout the event (54.7%).

Almost all respondents reported not having tested positive for COVID-19 within 2 weeks of attendance to SCF (1053 respondents, 95.5%). Only 38 respondents (3.5%) tested positive for COVID-19 within 2 weeks of attendance to the event.

Most respondents agreed or strongly agreed that pre-event testing reduces the risk of catching COVID-19 (87.1%, figure 1); however, nearly all respondents (97.5%) agreed or strongly agreed that it cannot reduce the risk to zero. A significant proportion of respondents agreed or strongly agreed that other events should adopt videoed testing at home (76.4%). About a third of respondents agreed that they would have valued to receive a personal risk score of catching COVID-19 at the event (35.6%), while another third was neutral about this (34.8%). The majority of respondents (89.4%) agreed or strongly agreed that home-based testing was preferred to at-the-door testing.

Most respondents (93.7%) reported that the requirement of at-home videoed rapid testing would not influence their decision to attend future events.

Interviews

By interviewing a subset of survey respondents, we were able to further explore the findings from the online questionnaire. The characteristics of interviewees are summarised in table 2.

Table 2.

Sample size and characteristics of interviewees

	N (%)	%
Total	11	100.0
Gender
Female	6	54.5
Male	5	45.4
Age
15–24	2	18.2
25–39	1	9.1
40–54	1	9.1
≥55	7	63.6
Ethnicity
White	9	81.8
Mixed/multiple ethnic groups	1	9.1
Other	1	9.1

Although the risk modelling was successfully conducted in real time and its projections were largely in line with the number of COVID-19 cases that emerged from SCF as measured by Test and Trace, it was not used at the event because permission for it to go ahead had already been granted. Therefore, we were unable to interview public health authorities to assess the utility of the risk modelling as a go/no-go decision tool.

Three main themes emerged from the data: overall impressions of respondents on the safety protocol, particularly on its feasibility, acceptability and the learning associated with it; experience of respondents at the event, including the impact of the safety protocol on their sense of safety and the use of other protective measures against COVID-19; general views on the value and relevance of pre-event safety procedures and on medically certified home-based testing.

Overall impressions on the safety protocol

All respondents were positively impressed with the safety protocol. They agreed that it was feasible, efficient and straightforward, and they could not identify major drawbacks.

I was really impressed. It was cheap. It was easy.… It worked really well, I was very pleased with it from start to finish. And at the time I thought it was a great idea. … I can't think of a drawback from the testing process (P7, age 55).

For many respondents, this was the first event they were able to attend since the beginning of the COVID-19 pandemic, and, therefore, found this procedure acceptable for the festival to take place. None of the participants reported concerns regarding the intrusiveness of the test or the way personal data were handled. Respondents agreed that the cost of the test was marginal with respect to the overall price of the ticket, and, therefore, accessible, even for families or large group bookings. Some participants in the older age groups feared to experience technological difficulties while carrying out the videoed test. Nevertheless, they were willing to do it and were able to successfully complete the procedure.

Although the safety protocol implied that videoed self-tests were reviewed by a professional, one respondent had doubts that all videos were being checked, given the large amount of festival attendees and the short time window between the test submission and the start of the festival. However, another respondent believed that there was someone manually checking the videos given that test results were not made available immediately.

The fact that you had to give it in, but the time scales were such that you wouldn't necessarily get replies at certain points made me believe that there was a real person watching them all. … And that gave me a reassurance (P2, age 53).

We asked participants about the impact of videoed testing on their confidence in self-testing. The majority did not notice any change in confidence in their self-testing technique after carrying out a videoed test, since they were used to testing regularly before attending the event. However, one respondent said that the videoed aspect of it might have had a positive impact on her self-testing technique:

The animation person was doing it and then you kind of follow, so you know you're doing the right thing and it probably did influence my tests afterwards because I'd seen that and it was in my memory (P3, age 28).

Experience at the event

Respondents believed that the protocol helped to minimise the spread of COVID-19. This contributed to create a feeling of safety and reassurance among participants at the event.

I was very aware of the fact that nothing was going to stop a certain amount of infections taking place, but it minimised, and that to me was the main point. … I felt much safer there, I felt much happier (P1, age 57).

Several respondents stressed that they felt safer at the event knowing that everybody had been tested and that the protocol implied people had to strictly follow a set of instructions to be able to attend the event.

The general belief and understanding that this was a COVID free zone was partly reinforced by the fact that I believe that everyone had been properly screened and proven to be COVID free within the days before entering the venue, and the videoed testing supported that (P7, age 55).

Some respondents attributed the added sense of safety to the robustness to videoed testing, which to many seemed harder to forge compared with other testing protocols, for example, those that require to upload a picture of the test result.

When asked about other preventive measures against COVID-19 adopted at the event, most respondents agreed that it did not feel necessary to wear a mask or to socially distance given that everyone had been tested, and that the event was outdoors. However, they all reported using hand sanitiser throughout the festival.

General views on pre-event safety procedures and medically certified home-based testing

We asked respondents whether they would recommend this safety protocol for future events. While the majority agreed that videoed testing was very relevant at the time of SCF, they thought it may not be as necessary moving forward, given the difference of the current circumstances compared with last summer, including the availability of vaccines and the increased frequency in self-testing among the general population.

I thought it’s perfect for last Summer, but … I think we're way past now this kind of filming people to do an LFT … Last summer we didn't really understand the disease to the same way that we do now, and loads of people weren't vaccinated. Now, everyone has had the opportunity to be vaccinated. … I don't feel like we should be protecting people to the same extent because people should be protecting themselves (P9, age 46).

Participants were also asked whether they would find it useful to receive a personalised COVID-19 risk score before attending an event, calculated based on their vaccine status, that of other attendees, and the characteristics of the venue. While some would find it useful, the majority thought that a high-risk score would not stop them from going to an event. This may be because people would have taken other precautions or made their own considerations about the consequences and losses involved.

We asked respondents about their views on the potential of medically certified home-based testing for other diseases, such as influenza, diabetes or HIV. The majority of respondents agreed that home-based testing could be useful, as long as tests are easy to carry out and people have the right skills and confidence to do them.

I think home testing is a wonderful step forward. … Most people would really appreciate being able to do those things at home … as long as it’s an easy process and you have confidence in the process and confidence in your ability to do it so that the result is accurate (P1, age 57).

Other participants stressed that having the opportunity to test for everything may become excessive and lead to negative consequences, resulting in people not taking sufficient responsibility, ignoring social norms, and even in the worsening of health outcomes.

I think also there’s the danger of promoting a false sense of security through test for too many things, so people then take no notice of the normal nor the social norms. Another thing that reduced the rate of flu last winter was the fact that we were socially isolating, washing our hands and doing all the other things, so you know perhaps it’s almost a reason for not testing for flu, you know, make sure that people take responsibility (P10, age 71).

Discussion

The aim of our study was to explore the views and perceptions of individuals who attended SCF in July 2021 in relation to the safety protocol that was implemented to minimise transmission of COVID-19 at the event. Our findings show that the protocol was highly acceptable, accessible, feasible and reliable. Participants agreed that videoed testing was straightforward, not intrusive and inexpensive, and it did not negatively impact their experience at the event. Videoed testing was helpful to improve the ability to self-test and increased confidence in test results. Additionally, by interviewing a subgroup of survey respondents, we found that participants felt safe at the event knowing that everyone had followed a robust testing protocol that contributed to minimise the spread of COVID-19 and did not feel it was necessary to follow additional safety measures at the event such as social distancing or mask wearing. The protocol of at-home, professionally validated self-testing linked to ticketing, won the Full Production Award at the UK Events Management Awards in December 2022, with judges saying:

‘Delivering live festivals on the back of a pandemic and in the midst of a pingdemic was a truly challenging set of circumstances. Achieving success was a considerable feat and is one which set parameters within which other festivals could operate. Standon Calling 2021 was a risk but definitely one worth taking!’ (https://eventproductionawards.com/eventproductionawards2022/en/page/2022-winners).

In this evaluation, some respondents mentioned that although videoed testing was very relevant at the time of SCF, it may not be as necessary for future events given significant improvements in the COVID-19 pandemic in the UK since last summer. Participants also reported that having access to personalised risk scores would not extensively influence their decision to attend an event. Most respondents agreed that medically certified home-based testing has the potential to be used for other diseases if tests are easy to carry out and patients feel confident enough to do and interpret them. However, the ability to test excessively might negatively impact individual responsibility and people’s respect of social norms.

Home-based self-testing is widely used in healthcare. Studies have shown that self-testing for chronic disease management, such as diabetes, or diagnostics, such as HIV, is highly accepted among patients. Most patients can easily perform these tests, obtain accurate results and correctly interpret them.11–14 There are several advantages associated with home-based self-testing, including reduced costs of care, increased availability and uptake and patient empowerment.13 15 Concerns with self-testing are often associated with user errors in performance or interpretation of results, leading patients to report the wrong outcome and change their behaviour accordingly.12 14 16 This can be avoided by ensuring that self-tests are simple and easy to use, and that patients are provided with better instructions to carry out and interpret tests correctly.17 Our findings show that professionally witnessed home-based videoed testing is feasible and reliable and has potential to be implemented in other health spheres to increase confidence among patients in disease self-management and further support correct interpretation of results.

Although the majority of respondents believed that the CAPACITY protocol was very relevant at the time of SCF, it may not be as useful for the current UK context given that, at the time of writing, all restrictions in relation to COVID-19 are close to being lifted.18 However, this safety protocol may be relevant in countries where the live event industry has been heavily affected by the recent rise in COVID-19 infections associated with the Omicron variant. For example, several live events have been cancelled in Australia,19 while in New Zealand, they are currently capped at 100 people, who have to prove they have been vaccinated in order to gain entry to the event.20 In these contexts, the CAPACITY protocol may be a useful strategy to support the safe reopening of live mass events at full capacity, while minimising transmission of COVID-19.

Limitations

Our study was the first to evaluate the feasibility and acceptability of a safety protocol that involves professionally witnessed home-based videoed pre-event testing to minimise COVID-19 transmission and support the safe reopening of live mass events at full capacity. Our findings, however, have some limitations.

The survey and interviews were conducted several months after the event took place; therefore, the study may be subjected to both recall and/or selection bias. Although our study sample was largely representative of the event population, interviewees were a small, self-selected subset of survey respondents. It is plausible that individuals who agreed to be interviewed may have had greater awareness of and interest in the research topic compared with those who did not participate, which may be an indication of selection bias. Although it is not possible to predict how selection bias might have influenced the findings of the research, if survey respondents were more likely to be those that were interested in technology and novel innovations or capable of responding to the electronic survey, then it may have led to a spruriously positive assessment of the protocol.

Furthermore, the way in which participants responded to the safety protocol may have been influenced by the audience composition, given that SCF is a family-friendly, boutique festival. The study is of only one event, in a specific context, that is characterised by attracting families, often with young children. The attendees that responded to the survey were predominantly aged 40–54 (45%), of white ethnicity (95%) and women (65%) and this reflected the characteristics of the attendees at the event as a whole. Although the numbers were too small for direct comparison, the interviewees were also largely of the same age, gender and ethnicity. However, we acknowledge the possibility that attendees at other types of events might have a different experience of the protocol. For example, it is possible that attendees at an opera, a cinema or a heavy metal rock concert, would have different views of at-home, professionally videoed LFT. It was not possible to compare the experience of the protocol in other settings and we were not able to assess the generalisability of the survey and interview findings, as this was the only live event to use the CAPACITY protocol, and the first live event to be held outside of the ERP. However, there were no themes or responses that suggested that the protocol was either so inconvenient, unpalatable or ineffective, that other events, of a ticketed nature, could not consider using it. Clearly, the public health imperative, which in this case was the COVID-19 pandemic, needs to exist to justify the use of the protocol.

Finally, although the risk modelling successfully predicted the number of COVID-19 cases arising from the event in real time as measured by Test and Trace, we were unable to evaluate the utility of risk-modelling as a go/no-go decision tool because it was a fast-moving environment and the event had already been given permission to go ahead. However, operationally the modelling was possible and has potential to support decision-making at future events.

Conclusion

Our study has shown that a safety protocol involving professionally witnessed home-based videoed pre-event testing for COVID-19 can support the safe reopening of live mass events at full capacity and is highly acceptable and feasible. While it may not necessarily be relevant for the current UK context, it can be useful for other countries where live events are currently suspended or are taking place at reduced capacity due to high numbers of COVID-19 infections. Risk modelling has the potential to further increase the robustness of this protocol and should be tested and evaluated at future events.

Data availability statement

All data relevant to the study are included in the article or uploaded as supplementary information.

Ethics statements

Patient consent for publication

Not applicable.

Ethics approval

The study received ethical approval from the Imperial College London Ethics Committee (ICREC 21IC6705). Participants gave informed consent to participate in the study before taking part.