Did COVID-19 surveillance system sensitivity change after Omicron? a retrospective observational study in England

Abstract

Background

During the COVID-19 pandemic in England, increases and falls in COVID-19 cases were monitored using many surveillance systems (SS). However, surveillance sensitivity may have changed as different variants were introduced to the population, due to greater disease-resistance after comprehensive vaccination programmes and widespread natural infection or for other reasons.

Methods

Time series data from ten epidemic trackers in England that were available Sept 2021-June 2022 were compared to each other using Spearman correlation statistics. Least biased and most timely SS in England were identified as ‘best’ standard epidemic trackers, while other COVID-19 tracking datasets we denote as complementary trackers. We compared the best standard trackers with each other and with the complementary trackers. Correlation calculations with 95% confidence intervals were made between complementary and best standard epidemic trackers. We tested the hypothesis that correlation with the best trackers was especially poor during transition periods when Delta, Omicron BA.1 and Omicron BA.2 sublineages were each dominant. Daily ascertainment percentages of incident cases that each SS detected during each variant’s dominance were calculated. We tested for statistically significant (at p < 0.05) differences in the distribution of the ascertainment values during each COVID-19 variant’s dominance, using Welch’s oneway ANOVA.

Results

Spearman rho correlation was significantly positive between most complementary and the best trackers over the whole period. There was no apparent visual indication that correlations were especially poor during transition period from Delta to BA.1. There were falls in correlation in the transition period from BA.1 to BA.2 but these falls were relatively small compared to correlation fluctuations over the full period. Ascertainment was highest in the Delta period for complementary systems against the least biased tracker of incidence. Ascertainment was statistically different between the three variant-dominant periods.

Conclusions

From September 2021 to June 2022, complementary SS generally reflected case rises and falls. Ascertainment was highest in the Delta-dominant period but no complementary tracker was highly stable. Factors other than which variant was dominant seem likely to have affected how well each tracker reflected true case rises and falls.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12879-025-11120-0.

Introduction

Surveillance systems (SS) were critical during the COVID-19 pandemic to manage, monitor and forecast health care demand, disease burden and the optimal timing of social distancing measures [1, 2]. COVID-19 SS facilitated identification of high-risk groups [3–6], informed modelling studies [7–10] and helped identification of distinct syndromic features of COVID-19 compared to other influenza-like illnesses [1, 11, 12]. To foster surveillance resilience and meet diverse information needs, multiple SS operated simultaneously in many jurisdictions, including in England UK [13].

As the UK COVID-19 activity progressed during the pandemic, it seems likely that the sensitivity of SS changed driven by multiple and complex pressures/factors. Universal vaccination programmes [14, 15] and widespread COVID-19 infections [16] are likely to have resulted in higher population resistance to developing symptomatic COVID-19 infection as well as reduced severity when symptoms were present. One key change in the pandemic occurred with the arrival of the Omicron (B.1.1.529) variant in England in November 2021. Omicron was associated with increased transmission rates [17] which were likely to affect surveillance sensitivity and specificity. Epidemiological evidence demonstrated that although Omicron sublineages were more transmissible than earlier COVID-19 variants [18], infection with Omicron variants were also associated with much milder illness [19]. Omicron sublineages becoming dominant coincided with reduced incubation periods [20] but higher risk of reinfection [18]. Laboratory evidence [21] indicated that Omicron had superior replication competence in the upper respiratory tract but poorer replication capacity in the lungs compared to earlier COVID-19 variants, potentially leading generally to more transmissible but milder illness and thus altered case presentation in infected persons.

Using SS effectively and efficiently means understanding the strengths and limitations of each individual system [22, 23]. How the sensitivity of SS may change as pandemics develop is a key aspect of understanding these limitations. In addition to estimates of total cases, SS are relied upon to provide timely information about when case counts are rising or falling. If an SS is dependent on disease severity, then when typical case presentation changes, system sensitivity is likely to change also. Understanding of what proportion of cases an SS has identified could be faulty if sensitivity abruptly changes. In this retrospective observational study of COVID-19 SS in England, UK, we identified three best standard epidemic trackers that yielded daily estimates of COVID-19 case incidence and prevalence and compared these sources with case count estimates with each other and with other, complementary surveillance systems (CompSS). We apply simple and replicable methods to document how sensitive many SS were to estimating case counts in England when the dominant COVID-19 variant changed from Delta to Omicron BA.1 and then to Omicron BA.2.

Methods

In this study we use nine SS datasets that were previously described at length [13] as well as one additional surveillance dataset newly described here: case counts of persons hospitalised primarily for COVID-19. Below we provide a brief recap of each surveillance dataset used in this study, which are also summarised in Table 1.

Table 1.

Best and complementary epidemic trackers

Source	Short name	Description	Primary purpose of dataset(s)
Government testing framework			Epidemic control and monitoring
	P12	Pillar 1 & 2 case counts
Randomly chosen households for sampling	ONSincid	ONSCISE incidence estimates	Estimate new infections, and prevalence ; epidemic monitoring
	ONSprev	ONSCISE prevalence estimates
NHS hospitalisation records	HospAdm	New admissions to hospital of patients who tested positive for COVID-19	Hospital services demand
	C19PRinH	Count of persons who were in hospital primarily because of COVID-19 infection
Syndromic datasets held by UKHSA, derived from actual health care usage records	EDSS	Emergency Department attendances	Syndromic surveillance
	GPIH	Consultations with GPs in usual opening hours
	111calls	NHS 111 telephone calls
	111web	NHS 111 website assessments
The ZOE App	ZOE	Symptom tracker application	Identify symptoms associated with COVID-19 infection status

Notes: GP = General practice surgeries; NHS = National Health Service; ONSCISE = Office for National Statistics Coronavirus infection Survey for England; UKHSA = United Kingdom Health Security Agency

Best standard epidemic trackers

We identified three potential ‘best standard’ epidemic trackers (BestET) for tracking cases in the English COVID-19 epidemic. The two least-biased case count estimators were daily case count estimates published by the Office for National Statistics (ONS) following random and stratified household sampling (Coronavirus Infection Survey for England, CISE). ONSCISE modelling allowed for demographic variations between the sampled and general population [24] to generate estimates of both incidence (ONSincid; new case count estimates) and prevalence (ONSprev: estimates of total concurrently infectious cases). The ONSCISE had high detection rates for asymptomatic cases because participants were swabbed at random, and high stability because data collection methods did not change and the modelling approach changed relatively little over time. However, the ONSCISE was not timely because of a typical > 10 day delay between swab collection and reporting date. ONS incidence estimates were not generated between 20 June and 19 November 2022 inclusive.

The most timely COVID-19 epidemic tracker was a combined count of laboratory-confirmed cases from two testing frameworks deployed during the COVID-19 epidemic in England. During our monitoring period, COVID-19 tests were available to health care workers and persons presenting with medical needs as part of what was designated a Pillar 1 testing framework. Tests to confirm COVID-19 infection were also available to wider members of the public with or without symptoms under a testing framework denoted as Pillar 2 tests. Combined case counts from Pillar 1 and 2 (P12) testing were widely used as most up-to-date estimates of daily total confirmed case counts during the English COVID-19 epidemic. P12 swab results [25, 26] were typically published < 24 h after sample collection [27], and thus were very timely and had potentially high stability. P12 data strictly reflect incidence not prevalence. P12 data were likely to under-detect asymptomatic cases, because people who lacked symptoms had less reason to take a test. We therefore regard P12 data as a third ‘best standard’ epidemic tracker, and the most timely epidemic monitoring tracker, especially of symptomatic infection.

Complementary surveillance systems/datasets

This study compares the three BestET with seven other potential COVID-19 against CompSS concurrently available in England. We describe whether each dataset predominantly reflected incidence or prevalence or potentially both. The government website coronavirus.data.gov.uk provided counts of persons newly admitted to National Health Service (NHS) hospitals with SARS-CoV-2 positivity (HospAdm). Another epidemic tracker based on health service usage is counts of hospital inpatients in England who had COVID-19 as the primary reason for their hospitalisation (C19PRinH), which information was available from 18 June 2021 onwards. Cases in the HospAdm dataset were only counted once per infectious episode so align with COVID-19 incidence. However, hospitalisations while infected with Covid could be multiple for the same infectious episode in a single patient, and as a result the C19PRinH dataset could reflect incidence or prevalence.

We used four CompSS which were based on national syndromic SS maintained by the UK Health Security Agency (UKHSA) real-time syndromic surveillance service. These syndromic CompSS describe patients accessing the NHS 111 telephone health advice (111calls) and NHS 111 online health assessments (111web) services; emergency department attendances (EDSS) and general practitioner in-hours (GPIH) consultations [2]. COVID-19-specific syndromic indicators for each of these syndromic CompSS were developed in 2020 [1]. All of these health service-record based indicators and counts of SARS-CoV-2 positive persons in hospital were typically reported within 24 h so were timely. They were consistently defined during this monitoring period and thus had high stability. However, these CompSS were unlikely to capture asymptomatic or minimally symptomatic cases. These UKHSA datasets could reflect incidence or prevalence.

The final CompSS what we used in our analysis was not based on actual health care demand but rather voluntary symptom-reporting. In a private sector project, the Zoe project (ZOE) produced COVID-19 incidence estimates [28]. ZOE was a nutritional and wellbeing digital application in development before 2020 which was adapted to support daily COVID-19 symptom tracking. ZOE produced estimates of community incidence that were derived from their own models that incorporated counts of their App users who reported new case status, with adjustments for demographic imbalances compared to the general population and the most recent ONSCISE estimates [29]. The precise algorithm(s) for how ZOE generated its incidence estimates and the completeness of the data collected are not published which means that stability of ZOE and its ascertainment biases are unknown. There were many ZOE model changes in their incidence estimates during the UK COVID-19 epidemic [29]. Previous research [13] found that in 2020–2021 the ZOE data correlated highly with the least biased and most timely epidemic trackers used in this article.

Where available, we used uncorrected data (as first published) from each CompSS, because minimally verified data is what real-world decision makers usually have access to while an epidemic is active. We calculated correlation statistics between the BestET and the CompSS for both the full monitoring period and sub-periods. All SS and datasets apply to England only. More information about the data used in this study, cleaning, additional processing and sources is in [13].

Monitoring period

We compared the datasets from 1 September 2021, until 19 June 2022. This period encompasses > 90 days before Omicron became dominant in England, the rapid transition from Delta to Omicron dominance in sequenced swab samples in late 2021, and transitions between several different Omicron waves before reporting of ONSCISE incidence estimates was paused on 20 June 2022.

COVID-19 variants

We consider SS sensitivity to track infections with regard to three main COVID-19 variants and their sublineages: Delta, Omicron BA.1 and Omicron BA.2. In the UK COVID-19 epidemic, the Delta variant was dominant among genetically sequenced test samples from 24 May to 14 December 2021. Omicron variant group BA.1 and sublineages dominated samples from 14 December 2021 through 20 February 2022, after which Omicron BA.2 sublineages were most sequenced samples until 7 June 2022. From 7 to 19 June 2022 no single SARS-Cov-2 variant dominated sequenced samples in England.

Analysis: quantitative comparisons

To compare BestET and CompSS, we undertook visual and statistical comparisons that were simple and replicable. We separate some of the analysis between incidence and prevalence comparisons. The case counts suggested by BestET least biased and most timely best standard epidemic trackers were plotted in time series with CompSS, to visually discern if there was close correspondence. The two BestET that indicated incidence were compared to each other to see if the correlation between the most timely BestET (P12) and the least biased BestET (ONSincid) changed over time or when dominant variants changed. Spearman rho correlation statistics were calculated with 95% confidence intervals for the full monitoring periods. Spearman rho was appropriate because the time series had strongly nonparametric distributions.

We wanted to test whether correlations between CompSS and BestET were especially low around the period when the dominant variant changed, from Delta to BA.1 or from BA.1 to BA.2 dominance. For this test, we calculated Spearman rho on day 16 in 31 day duration moving time windows, for six CompSS and P12 with ONSincid over the full period. The same was done for 7 CompSS against ONSprev. The Spearman rho value for day 16 in the moving window was plotted against calendar dates and with reference to which variant was dominant. These plots were visually assessed for apparent dips in correlation in transition periods.

Daily ascertainment percentages for incidence were also calculated and reported as median values with IQR, to see how many cases each system seemingly detected daily, compared to the best standard SS. The resulting datasets (ascertainment percentages) were mostly non-parametric and lacked homoscedasticity between groups separated by variant dominance. We therefore tested for statistically significant (at p < 0.05) differences in the distribution of the ascertainnment values during each COVID-19 variant’s dominance, using Welch’s oneway ANOVA tests. Rstudio 2022.02.0 (R version 4.1.1) was used to generate plots and statistics.

Results

Overview

Plots of each CompSS (rows) are shown against the BestET (columns) in Supplementary File S1. Two exemplar time series are reproduced as Fig. 1a and b in this article. Each individual figure has the complementary series plotted against left axis, and BestET against right axis, with approximate alignment for their peak values. These time series were smoothed (7 day moving average on central date) in these visualisations except the ONSincid and ONSprev. Raw (not smoothed) values were used for statistical comparisons and analysis. The period when each variant was dominant is indicated by multi-coloured lines at the bottom of each chart (green for Delta, cyan for BA.1 and purple for BA.2). Duration of dominance by each variant was : Delta for 104 days, BA.1 for 69 days and BA.2 for 107 days. There is strong visual similarity between many time series: many rises and falls happened about the same time. Comparisons between time series in all the figures are quantified statistically in Table 2.

Fig. 1.

Pillar 1 and 2 case count time series overlain with ONS estimates of incidence and prevalence. Notes: P12 is scaled to left side axis, ONS dataset counts are scaled to each right vertical axis. Dates are 1 September 2021 to 19 June 2022

Table 2.

Spearman Rho correlations between SS with 95% confidence intervals

SS
ONSincid	rho	1
	ci	1 to 1
ONSprev	rho	0.93	1
	ci	0.91 to 0.94	1 to 1
P12	rho	0.53	0.57	1
	ci	0.45 to 0.59	0.49 to 0.64	1 to 1
HospAdm	rho	0.79	0.91	0.65	1
	ci	0.73 to 0.83	0.87 to 0.93	0.56 to 0.72	1 to 1
C19PRinH	rho	0.35	0.56	0.65	0.74	1
	ci	0.24 to 0.45	0.46 to 0.63	0.55 to 0.72	0.67 to 0.79	1 to 1
ZOE	rho	0.91	0.94	0.36	0.80	0.39	1
	ci	0.89 to 0.93	0.92 to 0.96	0.27 to 0.44	0.74 to 0.84	0.29 to 0.47	1 to 1
EDSS	rho	0.42	0.49	0.83	0.65	0.74	0.28	1
	ci	0.30 to 0.52	0.40 to 0.58	0.78 to 0.87	0.56 to 0.71	0.68 to 0.79	0.16 to 0.38	1 to 1
GPIH	rho	0.03	0.06	0.48	0.22	0.33	-0.06	0.48	1
	ci	-0.10 to 0.15	-0.06 to 0.17	0.40 to 0.57	0.11 to 0.33	0.22 to 0.43	-0.18 to 0.053	0.38 to 0.56	1 to 1
111calls	rho	0.26	0.27	0.44	0.23	0.34	0.12	0.54	-0.11	1
	ci	0.15 to 0.35	0.17 to 0.37	0.32 to 0.53	0.11 to 0.34	0.24 to 0.44	0.00 to 0.23	0.44 to 0.63	-0.22 to 0.02	1 to 1
111web	rho	-0.18	-0.17	0.34	-0.07	0.23	-0.34	0.49	0.15	0.79	1
	ci	-0.27 to -0.07	-0.28 to -0.07	0.23 to 0.44	-0.18 to 0.053	0.12 to 0.34	-0.44 to -0.24	0.39 to 0.57	0.03 to 0.26	0.73 to 0.83	1 to 1
		ONSincid	ONSprev	P12	HospAdm	C19PRinH	ZOE	EDSS	GPIH	111calls	111web

Notes: See Methods for explanation of surveillance system (SS) names. rho = Spearman rho estimate, possible range is -1 to + 1, ci = 95% confidence interval for rho. Font colours with respect to correlation: Green: 95% confidence interval ≥ 0.7; black = 95% confidence interval is entirely > 0 and but lower bound of CI is < 0.70; grey = 95% confidence interval crosses zero; orange 95% confidence interval < 0

Table 2 shows whole period correlation (Spearman rho with 95% confidence intervals) between BestET and CompSS. Most correlations were significant at p < 0.05, meaning there was evidence of correlation, mostly positive. HospAdm and ZOE had the highest positive correlation (95% confidence intervals > 0.70) with the BestET ONSincid and ONSprev, but were both much less correlated with P12 (95%CI < 0.70). EDSS was the CompSS most correlated with the most timely tracker (P12; rho 0.83, 95%CI 0.78–0.87). P12, EDSS and C19PRinH were each positively correlated with all other SS data. Given many patients tested under the Pillar 1 framework were attenders to ED, it is not surprising that P12 correlated highly with EDSS. GPIH and 111web had the weakest relationships with ONSincid or ONSprev or other trackers, except that the 111web data strongly correlated with 111calls counts (rho 0.79, 95%CI 0.73–0.83).

Best epidemic trackers compared to each other

Figure1a and 1b in this article show the P12 time series with ONSincid and ONSprev. Figure 1a and b are useful to focus on because they suggest that ascertainment by P12 compared to ONSincid declined over time. Ascertainment was higher during Delta or BA.1, but lower during BA.2 time periods. When the ascertainment ratio was calculated for P12 compared to ONS incidence the values were 42.5 (95%CI 37.0-49.1) during Delta, 30.39 (95%CI 23.9–38.1) in BA.1 and 8.93 (95%CI 7.0-11.3) in the BA.2 time period. This varying ascertainment relationship seems like an important reason why the Spearman rho correlation statistic over the full monitoring period between P12 and the ONSincid time series is positive but not high (rho 0.53, 95%CI 0.45–0.59). In our previous research for the same trackers in the period September 2020 to December 2021, this correlation was much higher (rho 0.94, 95%CI 0.92–0.95) [13].

Ascertainment of complementary compared to bestet

Table 3 gives subperiod ascertainment percentages of cases detected for CompSS incidence and prevalence trackers, compared to case counts reported by BestET, and separated by which variant was concurrently dominant. Data are reported as median ascertainment ratio (IQR). GPIH is not included because GPIH data were available as rates per 100,000 registered patients, and as such aren’t directly suitable for calculating ascertainment ratios.

Table 3.

Ascertainment (as %) during variant-dominant periods

Complementary SS	Comparison with ONS prevalence
	Delta	BA.1	BA.2
C19PRinH	0.48 (0.4, 0.6)	0.21 (0.2, 0.2)	0.19 (0.2, 0.2)
EDSS	0.04 (0.0, 0.0)	0.02 (0.0, 0.0)	0.01 (0.0, 0.0)
111calls	0.27 (0.2, 0.3)	0.09 (0.1, 0.1)	0.10 (0.1, 0.2)
111web	0.29 (0.3, 0.3)	0.06 (0.1, 0.1)	0.07 (0.1, 0.1)

	Comparison with ONS incidence
	Delta	BA.1	BA.2
P12	42.5 (37.0, 49.1)	30.4 (23.9, 38.1)	8.93 (7.0, 11.3)
HospAdm	0.87 (0.8, 1.1)	0.55 (0.4, 0.6)	0.48 (0.4, 0.7)
ZOE	75.6 (67.6, 84.8)	53.1 (43.7, 66.4)	68.6 (54.4, 110.5)
EDSS	0.42 (0.4, 0.5)	0.16 (0.1, 0.2)	0.11 (0.1, 0.1)
111calls	3.15 (2.8, 3.6)	0.86 (0.7, 1.1)	1.00 (0.6, 1.7)
111web	3.35 (2.8, 3.8)	0.64 (0.5, 0.8)	0.74 (0.4, 1.2)

	Comparison with P12 incidence
	Delta	BA.1	BA.2
HospAdm	2.13 (1.9, 2.5)	1.83 (1.2, 2.3)	7.06 (3.5, 8.8)
ZOE	176.4 (154, 207)	162.9 (121, 277)	949.4 (451, 1411)
EDSS	1.03 (0.9, 1.3)	0.52 (0.5, 0.6)	1.40 (0.8, 1.8)
111calls	6.90 (6.1, 9.1)	2.80 (2.2, 4.1)	12.02 (5.4, 22.2)
111web	7.72 (6.1, 9.1)	2.08 (1.6, 2.8)	8.79 (3.9, 16.8)

Notes: Values are for entire monitoring period, ascertainment percentage median (IQR). Tests for between group differences (in rows) using Welch’s one-way ANOVA were all significant at p < < 0.001

Ascertainment of prevalence or incidence suggested by P12 or CompSS compared to the ONS estimates was lower in BA.1 period compared to Delta dominant period. However, using ONSCISE outputs as Best ET, differences in ascertainment between the BA.1 period and the BA.2 period were mostly small. A noticeable exception is that P12 had very much lower ascertainment of the least biased incidence estimates in the BA.2 period (median 8.93%, IQR 7.0-11.3%) than in BA.1 or Delta dominant periods (medians respectively 30.4% and 42.5% and IQR respectively 23.9-38.1% and 37.0-49.1%). Ascertainment of total cases for each CompSS compared to P12 was highest during BA.2 and lowest during the BA.1 period.

Because the ascertainment data were calculated daily, it was possible to statistically compare the distribution of daily ascertainment ratios during the dominance of each variant using Welch’s one-way ANOVA. This test yields an F-statistic with p-values; all between group differences (for ascertainment ratios across rows/variants in Table 3) were significant at p < 0.001 for the underlying data summarised in Table 3. This statistical finding documents that no complementary SS was highly stable (consistent) in case detection rate across all three variant periods.

Moving 31 day period correlation between best and complementary SS

The panels in Figs. 2a and 3g plot the individual Spearman rho correlation statistics for complementary SS, compared to the two least biased epidemic trackers (ONSincid and ONSprev) in moving time windows. The same time series are plotted together in Supplementary File S1, Figures S2 and S3. The supplementary plots evidence that new hospital admissions for patients with Covid strongly related to ED attendances syndromically assigned as Covid-19, as indicated by high visual correspondence between these two time series on Figure S2. Otherwise, for most time series, they do not very obviously follow similar trajectories as each other on Figures S3 and S4. The plotted correlation values are for day 16 within each moving 31 day duration window. Concurrent dominant COVID-19 variant is also indicated by a colour-coded line that intersects the vertical axis at 0.5.

Fig. 2.

Moving 31 day correlations, complementary trackers against least biased incidence tracker (ONSincid). Notes: Y-axis is Spearman rho; month on x-axis (1 September 2021 to 19 June 2022). Line to colour code concurrent variant dominance in each point intersects 0.5 on the vertical axis

Fig. 3.

Moving 30 day correlations, complementary trackers against least biased prevalence tracker (ONSprev). Notes: Y-axis is Spearman rho; month on x-axis (1 September 2021 to 19 June 2022). Line to colour code concurrent variant dominance in each point intersects 0.5 on the vertical axis

Visually, on all the graphs the Spearman rho varies considerably over the study period with multiple cycles of low correlation followed by periods of higher correlation. This variability appears greater for complementary SS based upon hospital data (e.g. EDSS, C19PRinH, HospAdm) than for other SS. Correlations with ONSprev are generally higher than with ONSincid. In many complementary SS, these dips to lower correlation appear to happen about the same time and don’t tend to occur at the same time as changes in the dominant variant. For instance, on multiple systems there appears to be a dip in correlations around November-December 2021 and another dip around February 2022 and the start of June 2022. HospAdm and EDSS correlate consistently well with ONSprev after February 2022 and apart from two short periods, ZOE correlates well with ONSprev over the entire study period.

We hypothesised that changes in correlation would concur with changes in the dominant variant. However, visually, during the transition period from Delta to Omicron BA.1 (December 2021), correlations were rising or remained consistently high, there was no apparent dip. Around the time of variant transition from BA.1 to BA.2 (late February 2022), all complementary trackers had poor (generally declining) correlation with the ONSincid and ONSprev. However, these BA.1 to BA.2 dips were mostly small or similar to other fluctuations in correlation over the monitoring period shown in Figs. 2 and 3 panels.

Discussion

In a previous comparison of SS [2020–2021; 13] we found that P12 closely tracked the least biased indicator (ONSincid), with a whole-period Spearman rho of 0.95 (95%CI 0.92–0.95). Here we focus upon the period where the dominant COVID-19 variant changed from Delta to Omicron BA.1 and then to Omicron BA.2. We found a much lower whole-period correlation between P12 and ONSincid (0.53 (95%CI 0.45–0.59)) and the ascertainment ratio dropped from 42.9% at the start of the period when the Delta variant dominated to 8.9% by the time BA.2 became dominant. There are several likely reasons for reduction in sensitivity over time. It seems likely that fewer people chose to get tested in the late UK epidemic period, but each surveillance system also exhibited increased variability in January-June 2022. Bajaj et al. [30] found that counts of Pillar 2 PCR tests taken/100 persons per day noticeably fell between 11 January and 31 March 2022 when rapid antigen (lateral flow device) tests were still available for free but no-cost access to Pillar 2 PCR tests was withdrawn. Other reasons for declines in epidemic tracker sensitivity possibly include an increasingly vaccinated population experiencing mostly re-infections with variants (Omicron) which have been associated with less severe symptoms than Delta and earlier variants. Less severe symptoms would result in lower usage of health services, fewer cases detected through surveillance and reduced sensitivity of Pillar 1 tests [31]. Attitudes about individual duty to contribute to infection control probably also changed. On 21 February 2022 the UK announced a “Living with Covid” plan [32, 33] which meant withdrawal of all free COVID-19 antigen or PCR tests by 31 March 2022, making it both harder to obtain tests and also seemingly less important to know one’s own case status. When the BestET were compared to CompSS, like in our previous analysis, ZOE was particularly correlated with ONSincid and ONSprev. ZOE also maintained relatively high ascertainment ratios with ONSincid and ONSprev. This is to be expected because ZOE incorporated recent ONS data to generate their estimates.

Some CompSS that we used had ascertainment ratios lower than found in our previous study of 2020–2021 data. Their correlations with BestET also reduced over time in our current analysis. This is consistent with an infection that is becoming declining importance as a public health risk. By 11 February 2022 around 71% of the English population had been infected at least once with SARS-CoV-2 [34]. COVID-19 vaccine distribution and uptake were fast and high in England: > 90% of English adults age > 65 + received at least one COVID-19 vaccination by 17 May 2021 [35]. Vulnerable persons were encouraged to be vaccinated again repeatedly from autumn 2021 onwards. Hybrid immunity (achieved after both vaccination and natural infection) is widely believed to lead to milder clinical presentation for subsequent COVID-19 infections [36–40]. As a result, many if not most cases detected in our study period could have been from reinfections or vaccinated individuals. Most of CompSS in our analysis were highly sensitive to symptomatic illness and had low sensitivity to minimally symptomatic infections. This is likely to be explain the reduced ascertainment of EDSS, HospAdm and C19PRinH in comparison to ONSprev and ONSincid in contrast to 2020–2021 [13]. It may also help explain the additional reduction in ascertainment amongst these SS as the study period moves from Delta to BA.1 to BA.2.

Some CompSS had an especially large reduction in ascertainment rates (e.g. 111calls). A plausible explanation is that in addition to changing severity, altered behaviour played a role. In February 2022 the UK government announced a “Living with Covid” plan [32, 33] that described lifting COVID-19-linked legal restrictions on social contact and testing requirements. The plan meant cessation of free laboratory tests to confirm COVID-19 infection (such as rapid antigen and rtPCR tests) from March 2022; as a result, the Pillar 1 & 2 case detection strategy changed. Our analysis showed that other SS lacked stability over the period 1 September 2021 to 19 June 2022. The transition to “Living with Covid” seems likely to have reduced the number of people seeking health care advice for and/or reporting COVID-19 symptoms. This may help to explain the reductions for CompSS of less severe illness (111calls and 111web) suggestive of fewer individuals seeking medical advice for an illness perceived as a lower potential health threat. Hence it is unsurprising that sensitivity reduced over this time period when the dominant variant changes from Delta to BA.1 and BA.2.

In spite of lower ascertainment over time, it is notable that several of these trackers have high correlation with ONSincid and ONSprev over the full period (see Table 2). Most notable is HospAdm which correlated with ONSincid and ONSprev at mean values 0.79 and 0.91 respectively. EDSS and C19PRinH have what could be considered moderate correlation with both ONS indicators (mean correlations ranging from 0.35 to 0.56, full confidence intervals always above zero). In Table 2, other CompSS, such as 111calls and 111web and GPIH, which seem likely to be most sensitive to the least severe COVID-19 infections, generally correlated poorly with ONS trackers. One reason may be because for many of the SS systems, health conditions are assigned probabilistically, based on clinical presentation rather than being laboratory-confirmed [41]. Increasing amounts of minimally symptomatic COVID-19 may complicate this process. Furthermore, for most of 2020–2021, there was low circulation of respiratory viruses other than SARS-CoV-2 which made the COVID-19 syndrome assignment especially reliable. However, during our study period there were large resurgences in cases of non-COVID-19 respiratory virus infections in many high income countries, including England [42–44]. Hence, some patients infected with other respiratory viruses may have been syndromically assigned as COVID-19 while some patients with multiple infections (e.g., influenza and COVID-19) may have been assigned only one of these conditions. These imprecise assignments may have contributed to lower correlations especially for presentations by patients with mild symptoms.

It was notable that lower correlation with ONSincid and ONSprev did not associate with transition between variants. Across systems there were some periods when correlation coefficients appeared particularly poor (e.g. November 2021); identifying the reasons for that requires separate research.

Strengths and limitations

Our analysis is novel by robustly quantifying that SS sensitivity for COVID-19 changed and varied in England in 2021–2022. This happened among multiple epidemic trackers. The purpose of surveillance data is to support decision-making that can facilitate rapid introduction of interventions and thus potentially less mortality and morbidity. Optimal use of surveillance data means understanding strengths and weaknesses of individual systems and indicators, and which trackers are best suited to support specific decisions made in real time. The value of complementary surveillance systems is both confirmatory and to reduce risk of over-reliance on individual trackers which may have unrecognised instability or limitations. Complementary systems have the potential to monitor and track disease burden across the breadth of the health care service, ensuring that all severities of presentation are captured. Identifying declines in sensitivity and instability, including temporal coincidence with some specific policy and public health changes, can inform better prospective decision-making during a future pandemic. We hypothesised why falls in surveillance sensitivity may have happened, and also propose that such changes in surveillance sensitivity may be expected as a normal development during an epidemic. We note that choices about which surveillance systems to implement are likely to at least partly depend on specific epidemic management strategies and priorities with corresponding cost-benefit evidence which is outside our study’s remit.

Our analysis benefited from large and comprehensive datasets that described access to the NHS, which is the main health care provider for the English population. We also accessed estimates made by the ZOE project for COVID-19 incidence. Some CompSS such as HospAdm and C19PRinH were priority epidemic trackers because they informed resource allocation and as such were subject to priority verification. These indicators were consistently defined during this monitoring period and thus had high stability. However, because these definitions were developed before or when Delta was dominant, the surveillance systems may have had higher sensitivity to pre-Omicron variants. We did not assess if correlations improved by adjusting correlation using delays (lags) between time series, such as if increased visits to the NHS111 website correlated with rises in hospital admissions 7–10 days later. Our previous analysis [13] found that allowing for 0–14 day offsets did not much improve correlation between the same epidemic tracker time series that were used in this article. Our comparison is limited to only specific datasets. We have not considered many possible other surveillance datasets that could merit consideration as epidemic trackers, such as infected-person counts derived from wastewater sampling for SARS-CoV-2 virus. Jones et al. (2025) [45] undertook a thorough catchment-by-catchment comparison between ONSCISE infection counts and geographically coincident prevalence derived from waste water samples in England from July 2020 to December 2021. Case count estimates derived from wastewater data were not available during the full time period that we wanted to monitor. Jones et al. concluded that while SARS-CoV-2 concentrations in wastewater in each catchment generally correlated well with ONSCISE prevalence, the corelations noticeably declined in the period July to December 2021. Jones observed that this deterioration in correlation coincided with younger persons becoming proportionally dominant among detected cases, the general population becoming highly vaccinated and (in the last month of their monitoring), sudden emergence of the Omicron variant.

Conclusion

From 1 September 2021 to 19 June 2022, CompSS greatly reduced in the proportion of COVID-19 cases they were detecting. This decrease was greatest in SS and indicators particularly focussed on less severe cases (e.g. presenting to the NHS 111 telephone helpline) which may reflect lower engagement with healthcare for COVID-19-like symptoms. However, despite this fall many indicators of more acute (and severe) COVID-19 presentations (e.g. HospAdm) continued to correlate well with the least biased epidemic trackers indicating that they generally reflected true trends in COVID-19 activity. For all CompSS, correlations tended to cycle over time indicating periods when they were all useful measures of COVID-19 trends. Factors other than which variant was dominant seem likely to have affected how well each tracker reflected true rises and falls in COVID-19 infections.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Abbreviations

111calls

Counts of calls to the NHS111 telephone advice service

111web

Counts of website visits to the NHS111 web advice service

ANOVA

Analysis of variance

BestET

Best epidemic trackers

C19PRinH

Covid-19 was primary reason for being in hospital

CompSS

Complementary surveillance systems

COVID-19

Coronavirus disease 2019

GPIH

General practice in hours services, surveillance system

EDSS

Emergency department surveillance system

HospAdm

Hospital admissions

NHS

National health service

ONS

Office for National Statistics

ONSCISE

Office for National Statistics coronavirus infection survey for England

ONSincid

ONSCISE estimate of incidence

ONSprev

ONSCISE estimate of prevalence

P12

Count of COVID-19 cases confirmed under Pillar 1 and 2 test frameworks

Sept

September

SS

Surveillance systems

UK

United Kingdom

UKHSA

United Kingdom health security agency

ZOE

Zoe application data / Zoe application data custodians

Author contributions

PRH and IRL conceived of the study. PHR and JB extracted data; PRH, IRL and JB designed the data analysis strategy. JB and IRL wrote the draft protocol. IRL and JB designed the visualisations. JB assembled the datasets, analysed the data, wrote the first draft of the manuscript and assembled revisions. RAM and AJE facilitated access to and interpretation of the syndromic surveillance data. All authors commented on the draft manuscript, have read and approved of the final manuscript.

Funding

This work was funded and authors were financially renumerated by the National Institute for Health and Care Research (NIHR) Health Protection Research Unit (HPRU) in Emergency Preparedness and Response at King’s College London in partnership with the UK Health Security Agency (UKHSA) in collaboration with the University of East Anglia. AJE is also affiliated with the NIHR HPRU in Gastrointestinal Infections at the University of Liverpool. The views expressed are those of the author(s) and not necessarily those of the NHS, NIHR, UEA, any HPRU, Department of Health or UKHSA. The funders had no role in writing the manuscript. Authors were not precluded from accessing data that supported this study and authors accept responsibility for this submission.

Data availability

None of the original datasets were collected by ourselves and therefore they should not be redistributed by us. Zoe application, ONSCISE, COVID-19 hospital admissions and Pillar 1 & 2 data all have been accessible in the public domain. Applications for requests to access relevant anonymised data included in this study held by UKSA should be submitted to the UKHSA Office for Data Release at: https://www.gov.uk/government/publications/accessing-ukhsa-protected-data/accessing-ukhsa-protected-data ). R scripts used for analysis and to generate plots are available at https://github.com/JuliiBrainard/SpotOmicron/tree/main.

Declarations

Ethics approval and consent to participate

The research adhered to the Declaration of Helsinki 2024. The anonymised health data used in this study were either published (freely and fully in the public domain) or collected and aggregated as part of the public health function of the UK Health Security Agency (UKHSA). UKHSA has access to a range of data sources under Regulation 3 (Health Protection) of The Health Service (Control of Patient Information) Regulations 2002. Under the terms of that access, internal review by the.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

the UKHSA Research Support and Governance Office agreed that this study fell outside the remit for ethical review. This decision was in accordance with the revised guidance in the Governance Arrangements for Research Ethics Committees that was released in September 2011.