Letter
Ladhani and coworkers reported in this journal that false positive SARS-CoV-2 assays may cause serious complications for elderly care homes.1 In their report they presented a number of such false-positive cases that had a significant impact on the care homes, including a temporary unnecessary isolation of vulnerable residents and a loss of workforce leading to reduced care provision. Ladhani and colleagues concluded that “repeated unnecessary interventions are also likely to be detrimental to the long-term mitigation strategy in care homes, have significant resource implications and impact on the wellbeing of residents and staff”1. The authors further conclude that “in addition, there is the danger of behavioural fatigue so that, when strict infection control measures are required in a genuine outbreak, recommended measures may not be adhered to.”1
Taking the experience of Ladhani et al.1 into account we here present evidence that the problem of false positive SARS-CoV-2 testing goes even beyond these latter risks, especially if the screening strategy is based on latera flow antigen tests (rapid tests).
A recent German strategy to overcome the COVID-19 pandemic is the broad usage of SARS-CoV-2 lateral flow tests (LFT) for rapid antigen testing. These assays are intended to detect SARS-CoV-2 in the upper airways of infected people and are approved for the usage in symptomatic patients.
While some initial reports on the usage of lateral flow antigen assays based on controlled clinical studies were promising,2, 3, 4 the real-world performance of LFTs remains controversial.5, 6, 7, 8
Nevertheless, the Federal testing strategy in Germany includes an off-label usage of these assays for liberal screening of asymptomatic people9 due to the probability of infection establishment with the intention to detect and isolate infected individuals as soon as possible and since 8th March the state bears the incurred costs for at least one rapid antigen test per week for German citizens.9 These direct cost are accompanied by indirect costs as the off-label usage may result in a high rate of false positive results, which might be associated with a high economic burden. This can be calculated on the basis of data we have obtained for the City of Cologne in May 2021. Between 1st and 31st of May a total number of 1.245.962 rapid antigen assays were performed in approximately 800 test centers certified for rapid antigen testing according to recent emergency edicts by the German government.
The daily rates of positive and false positive lateral flow assays are shown in Fig. 1 (a and b). In summary, out of the 1.245.962 rapid antigen tests performed in the city of Cologne in May 2021 a total number of 2.906 specimens were tested positive by lateral flow assays, of which only 1.345 could be confirmed by PCR, thus the overall false-positive rate was 46.28%. 52% (1561/3060) of all positive cases identified during May 2021 in Cologne were identified by rapid antigen screening. It can be excluded that there is any bias regarding test providers or assay used, because positive rapid antigen tests were uniformly reported from all over the city, as cross checked by postcodes (data not shown). Considering that this number probably includes persons who used a test reasonably (e.g. because of possible contact) and thus have a higher pre-test probability and persons who would had made a PCR test otherwise, this number is a conservative estimate for the effectiveness of liberal antigen screening.
Fig. 1a.
Daily incidence of newly detected SARS-CoV-2 infections per 100.000 people, total registered positive rapid antigen tests and false positive rapid antigen tests in Cologne (May 2021).
On the basis of the data it is not possible to calculate other diagnostic accuracy measures than the positive predictive values (PPV), as only positive tests were verified, i.e. negative lateral flow tests were not confirmed by PCR testing. The overall PPV for May is PPV = 0.53, whilst the PPV for the last week of May with already lower incidence is PPV = 0.47. This indicates that with decreasing pre-test probability the PPV likely further decreases (Fig. 1a, Fig. 1b ). This assumption is confirmed by the fact that at the end of May the percentage of positive cases correctly identified by LFTs decreases to 39% (313/796).
Fig. 1b.
False positive and registered positive rapid antigen tests in a 7-day interval in May 2021 (Cologne, Germany).
To get insights into the economic burden of rapid antigen testing we calculated the direct costs arising from the performed tests and the indirect costs arising from the subsequent quarantine. The associated costs of false negative tested persons could not be included because no data on false-negatives were available. The assumptions for resource use and prices are listed in Table 1a . As the data on new infections (R-value) and hospitalization rate were highly variable in the last months we decided to use the moving average before implementation of rapid antigen testing for the R-value, hospitalizations, respectively.
Table 1a.
Resource usage, and prices/costs used for the calculation, based on data available for 2019 and/or 2020.
Based on the demographic data available for Cologne the cost were initially calculated and then extrapolated to Germany based on the number of citizens and the average household size. For this purpose, we assumed that the average incidence as well as quarantine and discharge management apply to the entire Federal Republic of Germany. However, this simplification may cause a bias as in other parts of Germany the incidence at that time, the contact tracing, and the diagnostic turn-over times may vary, especially in rural areas. Consequently, the resulting numbers should only be considered as rough approximations.
The costs to be reimbursed for the assays performed in Cologne in May summed up to 22.427.316 €. Depending on the length of preventive quarantine before availability of PCR-test results, the total costs for the rapid antigen testing strategy ranged from 24.742.608 € (minimum = 22.427.316 € test costs plus 2.315.292 € productivity loss) to 27.057.899 € (maximum = 22.427.316 € direct test costs plus 4.630.583 € productivity loss) (Table 1b ). Merely assuming three days quarantine the indirect costs of quarantine due to false negative results were about 1.607.399 €. Based on this cost, it can be estimated that the “cost per identified case” were 16.592 €, the “cost per infection avoided” were about 11.061 €, and the “cost per hospitalization avoided” were about 110.610 €.
Table 1b.
Analyses of costs arising from rapid antigen tests, Cologne.
| 3 days quarantine | 2 days quarantine | 4 days quarantine | |
|---|---|---|---|
| Test-cost | 22.427.316 € | na | na |
| Productivity loss total | 3.472.937 € | 2.315.292 € | 4.630.583 € |
| Productivity loss false negative | 1.607.399 € | 1.071.599 € | 2.143.198 € |
na: not applicable.
When the costs are extrapolated to the entire Federal Republic of Germany, the costs ranged between 1.884.178.754 € (minimum =1.714.084.799 € test costs plus 170.093.955 € productivity loss) to 2.054.272.708 € (maximum =1.714.084.799 € test costs plus 340.187.909 € productivity loss) (Table 1c ).
Table 1c.
Extrapolation of costs analyses arising from rapid antigen tests, Germany.
| 3 days quarantine | 2 days quarantine | 4 days quarantine | |
|---|---|---|---|
| Test-cost | 1.714.084.799 € | na | na |
| Productivity loss total | 255.140.932 € | 170.093.955 € | 340.187.909 € |
| Productivity loss false negative | 118.088.284 € | 78.725.523 € | 157.451.045 € |
na: not applicable.
Noticeable, all cost data refer to May 2021. In the view of the subsequently declining incidence and consequently higher rate of false positives, the cost per infection and hospitalization avoided are likely higher in June when the incidence was even much lower.
From a pure laboratory and diagnostic point of view it has to be concluded that the usage of an antigen testing strategy with a false detecting rate of about 50% is unacceptable, although false positive results have no direct negative effect on the healthy individual. Concurrently, from the public health perspective it is indisputable to ideally identify all positive cases to curb possible outbreaks, but therefore, the most important test accuracy measure in the current phase of the pandemics is the negative predictive value.
Our data suggest that the use of rapid antigen testing only appears appropriate in high prevalence/incidence situations, because a (too) low prevalence may increase the risk of false-positive results leading to unnecessary quarantine and high economic burden. Although the Covid-19 prevalence was quite high (compared to target incidences foreseen by the governmental guidelines) in May in Cologne the cost per hospitalization avoided were 110.610 €. Therefore, as the prevalence and the morbidity from Covid-19 decreases due to vaccination coverage and seasonal effects, and in parallel the results of false positive results increases, the negative consequences from false positive results become more and more important. Consequently, the question arises whether a universal population screening of asymptomatic people is a reasonable measure or if the antigen testing strategy should be considered more strongly to the incidence to increase the PPV and to improve cost-effectiveness, especially as PCR-based test strategies appear feasible in low incidence situations regarding laboratory capacities in western countries.
Moreover, the effectiveness of a universal antigen screening has to be considered against alternative screening strategies. Taking into account that further infection waves can be expected for the upcoming autumn and winter season and that morbidity from Covid-19 further decreases there appears to be a need to find other, more valid and more targeted testing strategies. These strategies must be superior to rapid antigen assays regarding diagnostic test accuracy and thus are more cost-effectiveness. An example of a promising concept is the so-called “lolly-assays” performed in high-risk settings with high personal on interior contact settings as actually performed in schools of North-Rhine Westphalia. For these assays entire school classes are tested twice weekly, while the individual children have to suck on a swab for 30 s before all class swabs are collected in a single transport vehicle with stabilizing transport buffer and then are PCR tested by a pool testing specialized laboratory. Positive pools are individually resolved and lead to single testing of all pupils in the positive class. This concept could easily be expanded to other settings which high transmission rates such as workplaces.
Declaration of Competing Interest
The authors declare that no competing interests exist related to this submission.
References
- 1.Ladhani S.N., Chow J.Y., Atkin S., Brown K.E., Ramsay M.E., Randell P., et al. Regular mass screening for SARS-CoV-2 infection in care homes already affected by COVID-19 outbreaks: implications of false positive test results. J Infect. 2021;82(2):282–327. doi: 10.1016/j.jinf.2020.09.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Lindner A.K., Nikolai O., Kausch F., Wintel M., Hommes F., Gertler M., et al. Head-to-head comparison of SARS-CoV-2 antigen-detecting rapid test with self-collected nasal swab versus professional-collected nasopharyngeal swab. Eur Respir J. 2021;57(4) doi: 10.1183/13993003.03961-2020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Kruger L.J., Gaeddert M., Tobian F., Lainati F., Gottschalk C., Klein J.A.F., et al. The Abbott PanBio WHO emergency use listed, rapid, antigen-detecting point-of-care diagnostic test for SARS-CoV-2-Evaluation of the accuracy and ease-of-use. PLoS One. 2021;16(5) doi: 10.1371/journal.pone.0247918. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Mockel M., Corman V.M., Stegemann M.S., Hofmann J., Stein A., Jones T.C., et al. SARS-CoV-2 antigen rapid immunoassay for diagnosis of COVID-19 in the emergency department. Biomarkers. 2021:1–10. doi: 10.1080/1354750X.2021.1876769. biochemical indicators of exposure, response, and susceptibility to chemicals. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Osterman A., Baldauf H.M., Eletreby M., Wettengel J.M., Afridi S.Q., Fuchs T., et al. Evaluation of two rapid antigen tests to detect SARS-CoV-2 in a hospital setting. Med Microbiol Immunol. 2021;26(3):213–220. doi: 10.1080/1354750X.2021.1876769. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Schildgen V., Demuth S., Lusebrink J., Schildgen O. Limits and opportunities of SARS-CoV-2 antigen rapid tests: an experienced-based perspective. Pathogens. 2021;10(1) doi: 10.3390/pathogens10010038. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.James A.E., Gulley T., Kothari A., Holder K., Garner K., Patil N. Performance of the BinaxNOW coronavirus disease 2019 (COVID-19) antigen card test relative to the severe acute respiratory coronavirus virus 2 (SARS-CoV-2) real-time reverse transcriptase polymerase chain reaction (rRT-PCR) assay among symptomatic and asymptomatic healthcare employees. Infect Control Hosp Epidemiol. 2021:1–3. doi: 10.1017/ice.2021.20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Dinnes J., Deeks J.J., Berhane S., Taylor M., Adriano A., Davenport C., et al. Rapid, point-of-care antigen and molecular-based tests for diagnosis of SARS-CoV-2 infection. Cochrane Database Syst Rev. 2021;3 doi: 10.1002/14651858.CD013705.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Bundesministerium für Gesundheit/Federal Ministry of Health, Germany, 2021 [access 25 June 2021], Available from: https://www.bundesgesundheitsministerium.de/coronavirus/nationale-teststrategie.html.