Utility of the Peterborough Public Health COVID-19 rapid antigen test self-report tool: Implications for COVID-19 surveillance

Abstract

Background

The ongoing coronavirus disease 2019 (COVID-19) pandemic has necessitated novel testing strategies, including the use of rapid antigen tests (RATs). The widespread distribution of RATs to the public prompted Peterborough Public Health to launch a pilot RAT self-report tool to assess its utility in COVID-19 surveillance. The objective of this study is to investigate the utility of RAT using correlations between RAT self-report results and other indicators of COVID-19.

Methods

We investigated the association between RAT results, PCR test results and wastewater levels of nmN1N2 severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) genes (to infer COVID-19 levels) using Pearson’s correlation coefficient. Percent positivity and count of positive tests for RATs and polymerase chain reaction (PCR) tests were analyzed.

Results

The PCR percent positivity and wastewater were weakly correlated (r=0.33, p=0.022), as were RAT percent positivity and wastewater nmN1N2 levels (r=0.33, p=0.002). The RAT percent positivity and PCR percent positivity were not significantly correlated (r=-0.035, p=0.75). Count of positive RATs and count of positive PCR tests were moderately correlated (r=0.59, p<0.001). Wastewater nmN1N2 levels were not significantly correlated with either count of positive RATs (r=0.019, p=0.864) or count of positive PCR tests (r=0.004, p=0.971).

Conclusion

Our results support the use of RAT self-reporting as a low-cost simple adjunctive COVID-19 surveillance tool, and suggest that its utility is greatest when considering an absolute count of positive RATs rather than percent positivity due to reporting bias towards positive tests. These results can help inform COVID-19 surveillance strategies of local public health units and encourage the use of a RAT self-report tool.

Introduction

The ongoing resource pressures in the coronavirus disease 2019 (COVID-19) pandemic have warranted the implementation of novel testing strategies. For example, limits to polymerase chain reaction (PCR) testing (i.e. available only for only high-risk individuals in late 2021) posed a new challenge for accurately monitoring COVID-19 case counts ((1)). Wastewater surveillance has proven useful in addressing surveillance gaps ((2–4)); however, limitations exist in the interpretation of wastewater data.

Self-administered rapid antigen tests (RATs) can detect the viral proteins that cause COVID-19 in as little as 15 minutes ((5)), and is an alternative method employed by the Government of Ontario to monitor the virus’ spread. In December 2021, the Ontario government announced the distribution of RATs for elementary and secondary school students and staff ((6)), followed by the expansion of the RAT rollout to pharmacies and grocery stores in February 2022 ((7)). This program continues until present, with publicly funded and privately purchased RATs widely available. The provincial distribution of RATs prompted Peterborough Public Health (PPH) to launch a pilot surveillance project asking local residents to voluntarily and confidentially report their RAT results via an online self-report tool. The objective was to monitor an approximate percent positivity among those who reported their results using the online self-report tool to contribute to community COVID-19 surveillance.

Peterborough Public Health is the only public health unit globally that has launched a RAT self-report tool. The objective of this study is to investigate the utility of this tool using correlations between RAT self-report results and other indicators of COVID-19, specifically PCR test results and wastewater levels of COVID-19. This is the first study to investigate the utility of a RAT self-report tool, and the results will contribute robust evidence towards the role of RAT self-report results in COVID-19 surveillance.

Methods

Wastewater samples were collected and analyzed by Trent University in Peterborough. Wastewater levels of COVID-19 were measured in the region’s primary municipal wastewater source, the City of Peterborough, using a normalized average of viral genes N1 and N2 (nmN1N2) ((8)).

The PCR test results were obtained from the Ontario Ministry of Health and Long-Term Care COVID-19 Testing Dashboard ((9)). The RAT results were obtained from the PPH online RAT self-report tool ((10)). The PCR and RAT results from Peterborough City and County were included in the analysis while only the wastewater data from the City of Peterborough were included.

This study did not require ethics approval as the data we analyzed is routinely collected by PPH for COVID-19 surveillance and is not reported on an individually identifiable basis.

Analysis

The relationship between RAT results, PCR results, and wastewater levels of COVID-19 was analyzed using Pearson’s correlation. Both percent positivity as well as count of positive tests for RATs and PCR tests were analyzed. The data obtained for all three indicators were collected from December 17, 2021 to April 30, 2022; dates with missing data were excluded. All data were analyzed using R statistical software ((11)). The RAT, PCR test and wastewater data were screened for outliers and log-transformed to meet the assumption of a normal distribution for Pearson’s correlation. A weak correlation was defined as a value between 0.2 and 0.39, while a moderate correlation was defined as between 0.4 and 0.59 ((12)).

Results

Descriptive results

From December 17, 2021 to April 30, 2022, 4,571 RAT self-report results were recorded and 2,138 responses were reported as positive. Figure 1 summarizes the count of positive and negative RATs, Figure 2 summarizes the percent positivity of PCR tests, RATs and wastewater nmN1N2 levels and Figure 3 summarizes the count of PCR tests, RATs and wastewater nmN1N2 levels.

Figure 1.

Seven-day rolling average of reported positive and negative rapid antigen test results, by date, in the Peterborough Public Health region, from December 17, 2021 to April 30, 2022

Figure 2.

Percent positivity of the polymerase chain reaction and rapid antigen tests and wastewater nmN1N2 levels by date, in the Peterborough Public Health region, from December 17, 2021 to April 30, 2022

Abbreviations: nmN1N2, normalized N1 and N2 severe acute respiratory syndrome coronavirus 2 genes; PCR, polymerase chain reaction; RAT, rapid antigen test; WW, wastewater

Figure 3.

Count of positive polymerase chain reaction and rapid antigen tests and wastewater nmN1N2 levels by date, in the Peterborough Public Health region, from December 17, 2021 to April 30, 2022

Abbreviations: nmN1N2, normalized N1 and N2 severe acute respiratory syndrome coronavirus 2 genes; PCR, polymerase chain reaction; RAT, rapid antigen test; WW, wastewater

Analytical results

The PCR percent positivity and wastewater levels were weakly correlated (r=0.33, p=0.022), as were RAT percent positivity and wastewater (r=0.33, p=0.002); these results were statistically significant. The RAT percent positivity and PCR percent positivity were not significantly correlated to each other (Table 1).

Table 1. Correlation matrix of rapid antigen and polymerase chain reaction test percent positivity and wastewater nmN1N2 levels from December 17, 2021 to April 30, 2022^a.

Indicators of COVID-19	RAT % positivity	PCR % positivity	Wastewater (nmN1N2)
RAT % positivity	1.0	N/A	N/A
PCR % positivity	-0.035 (p=0.75)	1.0	N/A
Wastewater (nmN1N2)	0.33 (p=0.002)	0.33 (p=0.022)	1.0

Abbreviations: COVID-19, coronavirus disease 2019; nmN1N2, normalized average of viral genes N1 and N2; N/A, not applicable; PCR, polymerase chain reaction; RAT, rapid antigen test

^a Pearson correlation, r

Count of positive RATs and count of positive PCR tests were moderately correlated (r=0.59, p<0.001); this result was statistically significant. Wastewater was not significantly correlated to count of positive RATs or count of positive PCR tests (Table 2).

Table 2. Correlation matrix of count of positive RAT, count of positive PCR, and wastewater nmN1N2 levels from December 17, 2021 to April 30, 2022^a.

Indicators of COVID-19	Count of positive RAT	Count of positive PCR	Wastewater (nmN1N2)
Count of positive RAT	1.0	N/A	N/A
Count of positive PCR	0.59 (p<0.001)	1.0	N/A
Wastewater (nmN1N2)	0.019 (p=0.864)	0.004 (p=0.971)	1.0

Abbreviations: COVID-19, coronavirus disease 2019; nmN1N2, normalized average of viral genes N1 and N2; N/A, not applicable; PCR, polymerase chain reaction; RAT, rapid antigen test

^a Pearson correlation, r

Discussion

Our study is the first to investigate the utility of a RAT self-report tool for COVID-19 surveillance. We found that both PCR percent positivity and RAT percent positivity were weakly correlated with wastewater levels of COVID-19, and that the count of positive RAT and count of positive PCR tests were moderately correlated. We did not find a significant correlation between PCR percent positivity and RAT percent positivity or the count of positive RATs and count of positive PCR tests with wastewater levels. A possible explanation for these results is that not everyone who contracts COVID-19 obtained a RAT or PCR test, as many cases were asymptomatic or testing access criteria had changed. In addition, a significant proportion of people with active COVID-19 infections shed the virus in their feces, sometimes before their symptoms start; this contrasts with the time lag associated with RAT and PCR testing ((13)). Previous studies have also indicated the incidence and persistence of viral shedding through feces even after a negative nasopharyngeal swab ((14)). These temporal differences, along with a lack of testing in some active cases, may offer explanation for why there was no significant correlation observed between the counts of positive PCR tests or RATs with wastewater levels.

Strengths and limitations

A key strength of our study is that we used data collected during the peak in cases during the Omicron wave in Ontario. Heightened public awareness of COVID-19 transmission as well as the launch of the PPH RAT self-report tool contributed to a high number of RAT self-report submissions and an overall robust dataset. An additional strength was our investigation of the utility of this RAT self-report tool compared with multiple indicators of COVID-19. This quantified the relative utility of the RAT self-report tool using correlations and suggested that, in a complex landscape of COVID-19 surveillance, RAT self-reporting may present a helpful adjunctive surveillance tool at a minimal cost and administrative burden to public health organizations.

The main limitations in our study are the biases in reporting COVID-19 cases through both PCR testing and the RAT self-report tool. As mentioned, the restrictions on PCR testing to only high-risk individuals in late 2021 likely resulted in a significant under-representation of actual COVID-19 cases in the PCR data that we analyzed. In addition, the voluntary nature of self-reporting a RAT result introduces inaccuracies, as the PPH self-report tool likely only captured a fraction of actual positive and negative RAT results. Intermittent efforts to raise awareness of RAT self-reporting may result in fluctuations when the baseline reporting rates we observed were fairly low. A second limitation relates to availability of RATs. The RATs will continue to be distributed free of charge for the foreseeable future. If free tests are no longer readily available, the likely respondents would be further restricted to those who could afford or choose to purchase RATs. The role of differential positive reporting bias in the RAT self-report data necessitated the analysis of absolute numbers of positive RATs and trends over time as opposed to traditional measures such as percent positivity. This ultimately informed the decision to base the PPH COVID-19 Risk Index indicator on RAT based on the number of positive reports and continues to be assessed for utility over time.

The use of Pearson’s correlation coefficient in our analysis also presents limitations; for example, the exclusion of potential confounding variables ((15)). In the future, a more robust statistical analysis could be conducted that would account for extraneous variables.

A final limitation in our study is that although the RAT and PCR test results were collected from the entire county, the wastewater data we analyzed was collected only from the City of Peterborough. Although these samples may have differed from other areas in the county, the City of Peterborough represents the majority of the PPH population and can, therefore, be used as a proxy for the PPH region in a geographically small region such as ours.

Implications for policy

The weak correlation that we found between RAT percent positivity and wastewater, as well as PCR percent positivity and wastewater nmN1N2 levels, suggests that RAT percent positivity can be used in COVID-19 surveillance; however, the limitations of this surveillance method must be explored. Further assessment through additional waves of COVID-19 is needed to understand temporality in RAT self-reporting signals. For instance, corroborative information of RAT percent positivity, PCR percent positivity and increasing wastewater nmN1N2 levels may be useful in reporting on changes in community transmission risk. Indeed, this is the reason that PPH has incorporated RAT testing into its COVID-19 Community Risk Index ((16)).

The moderate correlation between count of positive RAT and count of positive PCR tests suggests that the utility of RAT self-report results is greatest when using the count of positive RATs compared to percent positivity as a tool for COVID-19. In the event of further changes to PCR testing eligibility continuing to decrease, maintaining access to RATs and RAT self-reporting could be a complementary surveillance tool. These results could provide evidence in favour of the use of RAT self-report results in COVID-19 surveillance and support the potential for public health units across Canada to develop local RAT self-report tools.

With this being said, multiple surveillance tools are required to obtain the most accurate picture of pandemic risk. For example, PPH has developed a COVID-19 Risk Index to inform community members about the current risk for COVID-19 transmission (16)). Six indicators (case rate, hospitalizations, deaths, PCR test positivity, positive RAT count, wastewater surveillance) are weighted to present an overall risk level with accompanying risk guidance. A key consideration in the use of COVID-19 surveillance tools such as the PPH COVID-19 Risk Index is to ensure public awareness of the complexities of COVID-19 surveillance; for example, adequate communication around limitations to interpreting each indicator is necessary.

In addition, some limitations surrounding reporting bias might be effectively addressed through collaboration between the public funder of RATs and local public health agencies; for example, exploring the potential for linking RAT promotion, distribution and surveillance through a coordinated mechanism, such as including a QR code on the test package linking to a survey.

Implications for research

We identified reporting biases as a risk to the use of data from RAT self-reports, yet further research is required to determine how public health units can encourage community members to report their RAT results. Additionally, further research on alternative surveillance tools is important as we progress through the pandemic given decreased access to PCR testing and reliability of their results to measure COVID-19 infections in the community. Finally, additional research is warranted to investigate which indicators are the most sensitive to change. From a visual analysis of our data (Figure 3), wastewater was the most sensitive COVID-19 indicator as it was the first to increase—coinciding with the start of wave 6 (March–April 2022). The RAT results have the next greatest sensitivity, with PCR being the least sensitive as it was the last indicator to show an increase. Further research evaluation is required to assess the sensitivity of each indicator to enable the earliest detection of an increase in COVID-19.

Conclusion

The ongoing COVID-19 pandemic has necessitated the use of numerous indicators and testing strategies to approximate levels of COVID-19 infections. Our results provide evidence that may support the use of RAT self-reports in COVID-19 surveillance. The utility of RAT self-reports is likely greatest when considering count of positive RAT tests (when public availability of RATs is constant over time) rather than percent positivity due to reporting biases. The results from our study may inform the COVID-19 surveillance strategies of local public health units and encourage the use of a RAT self-reports as an adjunctive surveillance tool.