Factors Affecting SARS-CoV-2 Test Discordance in Skilled Nursing Facilities

Abstract

Objectives

Reverse transcription polymerase chain reaction (PCR) and antigen tests for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) are sometimes discordant. We evaluated the discordance between antigen and PCR tests sampled in skilled nursing facilities (SNFs) to assess the relationship of symptom presence, timing between tests, and the presence of a facility outbreak.

Design

Observational study using electronic health record data.

Setting and Participants

Residents of 306 SNFs in 23 states, operated by 1 company.

Methods

We identified all rapid antigen and PCR tests conducted in study SNFs as of January 10, 2021, and classified whether symptoms were present and whether the facility was in outbreak at time of testing. We calculated the proportions of antigen tests with discordant follow-up PCR results conducted no more than 2 days after the antigen test.

Results

Of the 171,280 antigen tests in 34,437 SNF residents, 20,991 (12.3%) were followed by a PCR test within 2 days. A total of 1324 negative antigen tests were followed by a positive PCR result, representing 0.8% of all antigen tests and 6.3% of repeated antigen tests; while 337 positive antigen tests were followed by a negative PCR result, representing 0.2% of all antigen tests and 1.6% of repeated antigen tests. Discordance more often occurred when residents were symptomatic at time of antigen testing, during known facility outbreaks, and when the antigen test was compared with a PCR test done within 2 days vs 1 day.

Conclusions and Implications

Overall, discordance between SARS-CoV-2 antigen and PCR tests was low. Discordance was more common when the individual was symptomatic at time of antigen testing and during facility outbreaks. This suggests that a testing strategy which couples widespread use of antigen tests with clinical thresholds to conduct follow-up confirmatory PCR testing appears to perform well in SNFs, where timely and accurate SARS-CoV-2 case identification are critical.

The development, availability, and distribution of accurate and timely severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) tests have been critical factors for mitigating viral transmission over the course of the pandemic. The reverse transcription-polymerase chain reaction (PCR) offers a high sensitivity and specificity test, even in the early stages of an infection. However, it is expensive and results can be delayed, especially when demand is high.¹

In May 2020, the first SARS-CoV-2 antigen test in the US was approved under emergency use authorization for symptomatic individuals,² and many are now approved for screening of asymptomatic persons.³ Compared with PCR tests, antigen tests have similar specificity but reduced sensitivity,⁴ particularly in early infection. However, antigen tests can produce results in as little as 15 minutes,⁵ at a much lower cost than PCR tests.^{6 , 7} Consequently, antigen testing has expanded access to SARS-CoV-2 testing and early detection of cases.

In September 2020, the Centers for Medicare and Medicaid Services distributed an initial supply of rapid antigen tests to skilled nursing facilities (SNFs) nationally.⁸ Rapid antigen tests are now widely used in SNFs for surveillance and diagnostic testing of staff and residents. Following Centers for Disease Control and Prevention current guidance,⁹ SNF clinicians often pair antigen tests with confirmatory PCR tests when there is high clinical suspicion for infection, particularly if the initial antigen test is negative.

However, data are limited on the frequency of discordance between antigen and PCR test results in the SNF setting. In this study, we examined discordance between antigen and PCR test results from a large sample of SNFs, and this association with the presence of symptoms, timing between tests, and a concurrent outbreak.

Methods

We used electronic health record (EHR) data from 306 SNFs in 23 states, all owned and operated by Genesis HealthCare, one of the largest long-term care providers in the US. Key data elements used included a detailed testing file which records test date, type, brand, and result for all SARS-CoV-2 tests conducted; a daily census to identify daily disposition of residents; and nursing documentation to identify resident symptoms.^{10 , 11}

The Genesis testing policy, aligned with Centers for Disease Control and Prevention guidance,⁹ allowed for rapid antigen tests for both screening and diagnostic purposes, and recommended follow-up confirmatory PCR whenever the index of suspicion for SARS-CoV-2 infection was high, particularly if the initial antigen test was negative. We identified all rapid antigen and PCR tests conducted as of January 10, 2021, months before the alpha variant became the dominant circulating SARS-CoV-2 strain.¹²

Nurses recorded new symptoms identified through daily resident assessments in structured change in condition notes.¹³ We used these to classify residents as symptomatic if new SARS-CoV-2-related symptoms were documented in the 5 days prior to the test. We classified the test as conducted during an outbreak if at least 1 resident in the building had a new positive antigen or PCR test in the prior 14 days.

All analyses were descriptive. We calculated the proportions of antigen tests which had discordant follow-up PCR results using two numerators: (1) the number of positive antigen tests with a negative follow-up PCR test; and (2) the number of negative antigen tests with a positive follow-up PCR test. We defined a follow-up PCR test as one which was done within 1 day or 2 days of the antigen test. We placed these numerators over 2 denominators: (1) the total number of antigen tests conducted (ie, those with and without a follow-up PCR test); and (2) the number of antigen tests after which a follow-up PCR test was done.

We used 2 different denominators because repeated tests have a higher pre-test probability of discordance because front-line clinicians more likely will conduct a follow-up PCR test when they suspect infection despite an initial negative antigen test, or disbelieve a positive result when it significantly impacts other residents (like dictating group quarantine). As such, we would expect higher rates of discordance among tests that were repeated vs all tests conducted. We cannot calculate true ‘false positive’ or ‘false negative’ rates for the antigen tests because most did not have a follow-up PCR done as a gold-standard comparator, and solely using the subset of tests that were repeated would produce biased estimates. Also, because antigen and follow-up PCR tests were asynchronous, we cannot validly directly compare them, as viral titers and antigen expression dynamically change from one day to the next.

We compared test discordance rates based on 3 factors: (1) whether the resident was symptomatic at time of antigen test; (2) whether the PCR test was repeated within 1 or 2 days of the antigen test; and (3) whether the antigen test was conducted during a facility outbreak.

The Brown University institutional review board approved this study.

Results

As of January 10, 2021, 34,437 unique SNF residents underwent 171,280 antigen tests. The distribution of tests by brand was as follows: 104,994 (61.2%) BD Veritor, 57,430 (33.5%) Abbott BinaxNOW, 8797 (5.1%) Quidel, and 59 (0.03%) Abbott ID NOW.

Table 1 shows discordance rates stratified by resident symptom status at time of antigen test, and whether a follow-up PCR was done within 1 or 2 days. Antigen tests were mostly conducted when individuals were asymptomatic (98.2%). PCR tests followed 8.6% of antigen tests within 1 day, and 12.3% within 2 days. These occurred more often when the individual was symptomatic at the time of antigen testing. Of antigen tested individuals, 20.4% of those with symptoms were followed by a PCR test within 1 day, while only 8.4% of those without symptoms were followed by a PCR test within 1 day.

Table 1.

Discordance Between Rapid Antigen and Follow-Up PCR Tests, by Symptom Status and Time to PCR Test

	PCR within 1 D of Antigen Test			PCR within 2 D of Antigen Test
	Overall	Symptomatic	Asymptomatic	Overall	Symptomatic	Asymptomatic
Total antigen tests, n	171,280	3052	168,228	171,280	3052	168,228
Antigen tests with a follow-up PCR test, n (%)	14,720 (8.6%)	623 (20.4%)	14,097 (8.4%)	20,991 (12.3%)	761 (24.9%)	20,230 (12.0%)
Negative antigen tests with positive follow-up PCR test, n	808	58	750	1324	79	1245
% of total antigen tests with discordant PCR	0.5%	1.9%	0.4%	0.8%	2.6%	0.7%
% of repeated tests with discordant PCR	5.5%	9.3%	5.3%	6.3%	10.4%	6.2%
Positive antigen tests with negative follow-up PCR test, n	312	13	299	337	15	322
% of total antigen tests with discordant PCR	0.2%	0.4%	0.2%	0.2%	0.5%	0.2%
% of repeated tests with discordant PCR	2.1%	2.1%	2.1%	1.6%	2.0%	1.6%

Symptomatic vs asymptomatic determined based on whether the resident had new onset symptoms in the 5 days prior to testing. Data as of January 10, 2021.

An initially negative antigen test was more often followed by a discordant positive PCR test within a day when the person was symptomatic than asymptomatic at the time of antigen testing (9.3% vs 5.3%, respectively). This was still a small proportion of total antigen tests conducted (ie, 1.9% of all antigen tests in symptomatic individuals and 0.4% in asymptomatic individuals with an initial negative antigen result and a positive PCR result within 1 day. Discordance was slightly higher when the antigen test was compared with a PCR test done within 2 days.

A positive antigen result followed by a discordant negative PCR test occurred less often, and differences by symptom status were smaller. Among antigen tests with a follow-up PCR test within 1 day, only 2.1% (13/623) in symptomatic individuals and 2.1% (299/14,097) in asymptomatic individuals had a positive antigen result followed by negative PCR result. This represented 0.4% (13/3,052) of all antigen tests done on symptomatic individuals and 0.2% (299/168,228) of all tests on asymptomatic individuals.

Table 2 and Figure 1 show discordance rates stratified by whether the facility was experiencing an outbreak at the time of antigen testing. Follow-up PCR tests were more often conducted during outbreaks. Of the 105,627 antigen tests done during outbreaks, 11,577 (11.0%) had a follow-up PCR test. By comparison, of the 65,442 antigen tests conducted outside of an outbreak, only 3143 (4.8%) had a follow-up PCR test.

Table 2.

Discordance Between Rapid Antigen and Follow-Up PCR Tests for Antigen Tests Done during a Facility Outbreak vs No Outbreak

	No Outbreak			Outbreak
	Overall	Symptomatic	Asymptomatic	Overall	Symptomatic	Asymptomatic
Total antigen tests, n	65,442	1121	64,321	105,627	1931	103,696
Antigen tests with a follow-up PCR test, n (%)	3143 (4.8%)	184 (16.4%)	2959 (4.6%)	11,577 (11.0%)	439 (22.7%)	11,138 (10.7%)
Negative antigen tests with positive follow-up PCR test, n	11	0	11	797	58	739
% of total antigen tests with discordant PCR	0.02%	0%	0.02%	0.8%	3.0%	0.7%
% of repeated tests with discordant PCR	0.3%	0%	0.4%	6.9%	13.2%	6.6%
Positive antigen tests with negative follow-up PCR test, n	16	0	16	296	13	283
% of total antigen tests with discordant PCR	0.02%	0%	0.02%	0.3%	0.7%	0.3%
% of repeated tests with discordant PCR	0.5%	0%	0.5%	2.6%	3.0%	2.5%

All vendors, repeat PCR within 1 day. Symptomatic vs asymptomatic determined based on whether the resident had new onset symptoms in the 5 days prior to testing. Outbreak is defined as 1+ new resident SARS-CoV-2 case(s) in the facility in the 14 days prior to the test. Data as of January 10, 2021. Note, total antigen tests here = 171,069 vs. 171,280 in Table 1 due to some missing daily census data needed to determine outbreak status at time of testing.

Fig. 1.

Discordance between rapid antigen and follow-up PCR tests for antigen tests done during a facility outbreak vs no outbreak (All vendors, repeat PCR within 1 day). Symptomatic vs asymptomatic determined based on whether the resident had new onset symptoms in the 5 days prior to testing. Outbreak is defined as 1+ new resident SARS-CoV-2 case(s) in the facility in the 14 days prior to the test. Data as of January 10, 2021.

Discordance rates were higher during outbreaks. During outbreaks, there were 797 negative antigen tests followed by a positive PCR test within 1 day, representing 6.9% of repeated tests and 0.8% of total antigen tests. By contrast, outside of outbreaks, only 11 negative tests were followed by a positive PCR test, representing 0.3% of repeated tests and 0.02% of total antigen tests.

Positive antigen tests followed by negative PCR tests were also somewhat more common during outbreaks. There were 296 positive antigen tests followed by negative PCR tests during outbreaks, representing 2.6% of repeated antigen tests and 0.3% of total antigen tests. Outside of outbreaks, there were only 16 positive antigen tests followed by negative PCR tests, representing 0.5% of repeated tests and 0.02% of total antigen tests.

Discussion

In this study, we examined discordance between SARS-CoV-2 antigen and PCR tests conducted in a large sample of SNFs through early winter 2021. Overall, we observed low rates of discordance, meaning that clinicians can have confidence when using antigen testing for surveillance and diagnostic testing, particularly when paired with a testing strategy that incorporates low clinical thresholds to determine whether a follow-up PCR test is needed. That we observed higher relative frequencies of discordance among antigen tests that were repeated, compared with all antigen tests conducted, suggests that front-line clinicians were fairly adept at flagging antigen results that warranted follow-up confirmatory PCR testing.

Timing between the initial antigen test and follow-up PCR was an important factor affecting discordance. We observed slightly higher frequencies of negative antigen results with discordant positive PCR results when we compared with PCR tests conducted within 48 vs 24 hours of the antigen test. This is consistent with some of the known complexities of SARS-CoV-2 testing. Viral shedding with SARS-CoV-2 infection varies over time, generally peaking near the time of symptom onset (end of incubation) or 4‒7 days after initial inoculation (end of latency).¹⁴ PCR tests can detect viral RNA earlier and later than when antigen tests can detect antigen.¹⁵ This highlights the importance of maintaining a high clinical index of suspicion, informed by exposure risk and symptom presentation, for assessing whether the initial antigen test may have been done too early or late to detect infection, thus prompting a confirmatory PCR test.

Although PCR tests may be the gold standard for detecting SARS-CoV-2 infection, antigen tests have the key advantage of detecting virus when transmission risk is greatest. This, coupled with the fact that antigen results are available within minutes, makes them very useful for immediate clinical decisions at the point of care. Our findings suggest that in the setting of a known outbreak, SNF clinicians must be more cautious in their interpretation of an initial negative antigen test result and maintain lower thresholds for quarantining and triggering a follow-up PCR test, particularly if the person was symptomatic at time of antigen test. During surges when PCR testing demand is high and results may be delayed by days, serial antigen testing in high risk individuals (those with symptoms or suspected exposure) may be an alternative strategy. Outside of known outbreaks, clinicians can have greater confidence in the accuracy of antigen test results, including for asymptomatic testing which has been an area of concern for some health departments.

We observed low frequencies of negative antigen tests followed by discordant positive PCR results. However, they were slightly more common during outbreaks, which could potentially be related to the higher volume of tests done. The potential for false positives has been another area of concern with the antigen tests, particularly when they first became available.¹⁶ However, our findings suggest that they are still relatively rare events, meaning that SNF clinicians should treat an initial positive antigen as presumptive.

Some important limitations should be noted. First, because we only had confirmatory PCR tests on a small subset (12.3%) of the antigen tests, we were unable to measure actual false negative or false positive rates. Estimating these measures in a small subset would produce artificially elevated false positive and false negative rates, given that confirmatory PCR was only performed when there was a clinical suspicion of an inaccurate antigen test result. The purpose of our study was not to estimate the true performance of antigen tests, but instead to estimate their performance in a real-world setting, as they are currently being used in US nursing homes. In addition, we used testing data only through January 2021, prior to the Delta and Omicron variants, and when vaccines were just initially becoming available. Although we would expect the general trends we observed to hold true in the current period, SNF clinicians must now consider an added layer of complexity regarding individual vaccination status and transmission risk of the different variants, when making decisions around confirmatory PCR testing.

Conclusions and Implications

In summary, a testing strategy that couples widespread use of antigen tests with clinical thresholds to conduct follow-up confirmatory PCR testing appears to perform well in SNFs where timely and accurate SARS-CoV-2 case identification are critical.