Abstract
Objectives
The aim of this work was to estimate the conditioned probability for the diagnosis of SARS-CoV-2 infection with reverse transcription polymerase chain reaction (RT-PCR), viral antigen rapid diagnostic tests (Ag-RDT), and antibody detection tests depending on the prevalence in the specific healthcare settings in Spain in 2020, and on the pre-test probability (PTP) according to the clinical situation, age and unknown or close contacts of the patient.
Material and methods
Performance parameters of tests were obtained from literature. Prevalence data and PTP were obtained from Spanish sources and a survey, respectively. The post-test probability is the positive predictive value (PPV) when test is positive. For negative result, we also calculated the probability of having the infection (false negatives).
Results
For both RT-PCR and viral Ag-RDT, the lowest PPV values were for the population screenings. This strategy proved to be useful in ruling out infection but generates a high number of false positives. At individual level, both tools provided high PPV (≥ 97%) when the PTP values are over 35%. In seroprevalence studies, though the specificity of IgG alone tests is high, under low seroprevalence, false positives cannot be avoided. Total antibodies tests are useful for diagnosis of COVID-19 in those doubtful cases with RT-PCR or Ag-RDT tests being repeatedly negative.
Conclusions
The interpretating of results depends not only on the accuracy of the test, but also on the prevalence of the infection in different settings, and the PTP associated to the patient before performing the test.
Keywords: SARS-CoV-2, RT-PCR, antigen rapid diagnostic tests, antibody detection, testing strategy
Abstract
Objetivos
En este trabajo estimamos la probabilidad condicionada del diagnóstico de infección por SARS-CoV-2 con RTPCR, pruebas de antígenos virales (Ag-RDT) y pruebas de detección de anticuerpos, en función de la prevalencia en España en diferentes ámbitos durante 2020, y de la probabilidad pre-test (PPT) según la situación clínica, edad y contactos del paciente.
Material y métodos
Los parámetros de rendimiento de las pruebas se obtuvieron de bibliografía. Los datos de prevalencia y PPT se obtuvieron de fuentes españolas y de una encuesta, respectivamente. La probabilidad post-test es el valor predictivo positivo (VPP) cuando la prueba es positiva. Para el resultado negativo, también calculamos la probabilidad de tener la infección (falsos negativos).
Resultados
Tanto con RT-PCR como con Ag-RDT, los valores más bajos de VPP se detectaron en los cribados poblacionales, que demostraron ser útiles para descartar la infección, pero generan muchos falsos positivos. A nivel individual, ambas pruebas proporcionaron un VPP ≥ 97% cuando los valores de PPT son superiores al 35%. En estudios de seroprevalencia, aunque la especificidad de las pruebas de IgG sola es alta, si la seroprevalencia es baja, no se pueden evitar falsos positivos. Además, las pruebas de anticuerpos totales pueden ayudar al diagnóstico de COVID-19 en aquellos casos dudosos con pruebas de RT-PCR o Ag-RDT repetidamente negativas.
Conclusiones
La interpretación de los resultados depende no sólo del rendimiento de las pruebas, sino también de la prevalencia de la infección en diferentes ámbitos, y de la PPT asociada al paciente antes de realizar la prueba.
Palabras clave: SARS-CoV-2, RT-PCR, pruebas de diagnóstico rápido de antígenos, detección de anticuerpos, estrategia diagnóstica
INTRODUCTION
The interpretation of a microbiological test result depends both on the performance of the test, as assessed by its intrinsic characteristics of sensitivity and specificity, and on the pre-test probability or estimate of the baseline infection risk of each patient prior to ordering the test [1].
Since SARS-CoV-2 emergence, there has been an unprecedented race to develop diagnostic tests for detection of this virus, both directly (reverse transcription polymerase chain reaction, RT-PCR, and viral antigen rapid diagnostic tests, Ag-RDT) and indirectly (serological antibody detection tests). Due to the urgent need for diagnostic tests, most of the commercialized tests have been granted with emergency use authorisation by regulatory agencies (CE-IVD in Europe and EUA in the USA). This type of authorisation is based exclusively on analytical performance under ideal conditions with positive and negative sample controls [2]. RT-PCR is currently considered as the gold standard test for the diagnosis of COVID-19 by the WHO [3]; however, marketed RT-PCR tests use different extraction reagents and amplify different genomic regions of the virus, which affects the sensitivity of the test and makes results interpretation rather challenging. Moreover, most commercial tests have not adequately estimated the sensitivity and specificity in routine clinical practice [4], and most microbiology laboratories used consecutively or simultaneously the available RT-PCR, AgRDT, and antibody detection tests depending on the moment of the pandemic and the availability of viral transport media, reagents and consumables.
On the other hand, as for any laboratory test, the reliability of results obtained with microbiological tests differs depending on the pre-test probability of the patient and/or the prevalence of the disease in the particular setting from which the test is requested. When the pre-test probability and/or the prevalence of disease decrease, false positives are more likely to occur, and when the pre-test probability and/or the prevalence of disease increase, false negatives increase, as well.
The aim of this work was to estimate the conditioned probability for the diagnosis of SARS-CoV-2 infection with RTPCR, Ag-RDT and antibody detection tests depending on the prevalence in the specific healthcare settings in Spain in 2020 and on the pre-test probability according to the clinical situation, age and unknown or close contacts of the patient during the pandemic.
MATERIAL AND METHODS
Diagnostic performance of the diagnostic tests. Performance parameters of RT-PCR, viral Ag-RDT and antibody detection were obtained from literature. Only systematic reviews were considered.
RT-PCR. Three systematic reviews provide similar sensitivity percentages, ranging from 86 to 89% with overlapping 95% confidence interval (95% CI) [5-7]. Out the three studies, we selected that by Kim et al. [5] to calculate the conditional probabilities because the specificity reported (99%) is in accordance with most authors, who assume a false positive rate <1% [8,9]. The study by Kim et al. is a meta-analysis that included 1,502 patients from 19 studies, with a sensitivity of 89% (95% CI: 81%-94%) and specificity of 99%. The calculated positive and negative likelihood ratios (LR) were 89 and 0.11, respectively.
Viral Ag-RDT. The average sensitivity value of 56% (95%CI: 29.5%-79.8%) and specificity of 99%, (95%CI: 98%-99.9%) from the systematic review by Dinnes et al. [10] were used. The calculated positive and negative LRs were 56 and 0.44, respectively. Although the sensitivity demonstrated in asymptomatic screenings was lower than those in symptomatic case studies and contact tracing studies, the sensitivity was similar in the three subject groups when high viral loads are detected by RTPCR (threshold cycle value, Ct< 25) [11,12]. Therefore, in order to facilitate simulations, we used the same sensitivity and specificity mean values in all health care settings.
Detection of IgG and total antibodies (IgG and IgM). In seroprevalence surveys to estimate the prevalence of detectable antibodies resulting from infection in a community, IgG assay is recommended because it persists for long time after infection. The estimated sensitivity and specificity of IgG tests were 90% (95%CI: 88.5-91) and 99 (95%CI: 98.6-99.1), respectively [13]. The calculated positive and negative LRs were 90 and 0.1, respectively. On the other hand, total antibodies (IgG and IgM) may help diagnose COVID-19 cases in patients with a high clinical suspicion and repeatedly negative RT-PCR testing. Sensitivity and specificity values of total antibodies assays were obtained from the study carried out by Fox et al. [13]. Due to sensitivity variation along time course of infection [14], sensitivity values at week 3 after onset (91%, 95%CI: 88%-93.2%) and at week 5 after onset (94.3%, CI 95%: 93-95.5%) were selected. The specificity was 99% (95%CI: 99.6%-99.9%). The calculated positive and negative LRs were 455 and 0.09 for week 3, and 471 and 0.06 at week 5 after onset.
Prevalence data. Prevalence data were obtained from Spanish sources. In all cases data were from 2020, and different settings were considered.
Population screening. Firstly, we selected the screening carried out from October 7th to 10th 2020 in Azkoitia (Guipúzcoa), which included 3,069 subjects aged from 17 to 60 years old; 35 positives were detected (1.14%) [15]. Then we also used the positivity rate of different population screenings in the Basque Country, which was < 2% [15].
Primary care centres. Data from the study carried out by Albert et al. were used [16]. Between 2nd September and 7th October 2020 this prospective study enrolled 412 patients with clinical suspicion of COVID-19 attending primary care centres of the Clínico-Malvarrosa Health Department in Valencia (Spain). An Ag-RDT performed well as point of care for early diagnosis of COVID-19 in primary healthcare centres and was confirmed by RT-PCR. The prevalence estimated ranged from 5% to 10% at the time of study.
Nursing homes. Data from a test-based screening carried out at the Vall d’Hebron Hospital, a tertiary hospital in Catalonia, Spain, were considered. In that study, carried out during April 10th –24th 2020, 69 nursing homes with a total census of 6,714 persons were evaluated (previous laboratory-confirmed cases of COVID-19 were excluded). Overall, 768 (23.9%) residents and 403 (15.2%) staff members tested positive for SARS-CoV-2 [17]. RT-PCR was used as diagnostic test.
Hospital Emergency Service. Prevalence data in the Emergency Service of the Araba University Hospital in Vito-ria-Gasteiz between 18th – 31th March 2020 were used. Prevalence ranged from 35% to 50% of the patients attended [18]. RT-PCR was used as diagnostic test.
Seroprevalence. We used the ENE-COVID study, a nationwide, population-based seroepidemiological study, which was carried out between 27th April and 11th May 2020. Individuals from 50 Spanish provinces and the two autonomous cities were included. A total of 51,958 immunoassay analyses were done. The overall seroprevalence was 4.6%, and the seroprevalence of the health care occupational sector was 10% [19].
Pre-test probability. Pre-test probability was obtained from a survey of healthcare professionals who estimated the probability of SARS-CoV-2 infection based on clinical status [20]. The subjects were classified according to age (20-30 years and 60-70 years) and whether or not they had had close contacts. Clinical signs and symptoms included in the survey were: 1. None, 2. Odynophagia and nasal congestion, 3. Odynophagia, nasal congestion and anosmia/ageusia, and 4. Odynophagia, nasal congestion, anosmia/ageusia, fever and body weakness. In accordance with clinical experience, survey results confirmed that pre-test probability increased with increasing prevalence, patient age, documented exposure to the virus in the medical record and clinical signs and symptoms intensity.
In addition, pre-test probability was also estimated from studies that compared the diagnostic test accuracy of total antibodies (IgG and IgM) among patients with varying degrees of clinical suspicion for COVID-19 and negative RT-PCR throughout the course of their illness [21,22].
Estimation of the post-test probability. From the calculated LRs, the prevalence (pre-test probability) was converted into the post-test probability (probability of the patient having the infection after the diagnostic test) according to a previously reported method [23-25]. This method includes the following steps:
Calculation of the pre-test odds: pre-test odds = prevalence / (1 – prevalence)
Calculation of the post-test odds: post-test odds = pre-test odds × LR
Calculation of the post-test probability: post-test probability = post-test odds / (1 + post-test odds)
The post-test probability is the Positive Predictive Value (PPV) when test is positive, and we also estimated the probability of no infection (1-PPV). When the result is negative, the probability of not having the infection is the Negative Predictive Value (NPV). For negative result, we also calculated the probability of having the infection (1-NPV) and a threshold of 5% has been established, below which it is reasonable to consider the person as uninfected (e.g., permission to visit an elderly relative) [2].
The 95% CIs of the post-test probabilities were calculated by Miettinen’s method. When Miettinen’s method could not be applied, we used the first-order approximation of Taylor’s development [25].
RESULTS
Post-test probability for diagnosing infection was estimated in different settings (population screenings, primary care centres, nursing homes, and hospital emergency service) assigning a corresponding prevalence value to each of them. Table 1 shows the post-test probability calculated from the estimates of accuracy of RT-PCR and Ag-RDTs and from the prevalence values in the different settings. With both test methods, the lowest PPV values were for the population screenings: for prevalence of 1%, the PPV was 47% with RT-PCR and 36% with Ag-RDTs. Confident intervals show that PPV could be actually as low as 8% with RT-PCR, and 4% for Ag-RDTs when the prevalence is 1%. Both methods proved to be useful in ruling out infection in population screenings; however, and according to the confidence intervals, > 90% of positives may be false positives (1-PPV). Results also show the gradual increase of PPV as prevalence values increase, with the highest PPV for the hospital emergency service. Regardless the test method, when prevalence is ≥ 15%, PPV was > 94% for the nursing homes and the hospital emergency service, and therefore, RT-PCR and Ag-RDTs are more useful to confirm infection.
Table 1.
Conditional probability for the diagnosis of SARS-CoV-2 infection depending on the prevalence in different settings presuming a sensitivity of 89% and a specificity of 99%, LR +/- 89/0.11 (RT-PCR)[5] and sensitivity of 56% and a specificity of 99%, LR +/- 56/0.44 (Ag-RDT) [10].
| Prevalence in 2020 | |||||||||
|---|---|---|---|---|---|---|---|---|---|
| 1% | 2% | 5% | 10% | 15% | 25% | 35% | 50% | ||
| Population screenings in the Basque Country [15] | Primary Care Centres in Valencia [16] |
Staff members | Residents | Hospital Emergency Service | |||||
| Nursing Homes in Barcelona [17] | HUA Vitoria-Gasteiz [18] | ||||||||
| Sensitivity/ Specificity (%) |
If positive (+) result | Post-test probability (%) | |||||||
| 89/99 (RT-PCR) | Infection P (PPV) No infection P (1-PPV) |
47 (8-90) 53 (10-92) |
64 (19-93) 35 (6-81) |
82 (41-97) 17 (3-59) |
91 (60-98) 9 (1.5-40) |
94 (70-99) 6 (1-30) |
97 (81-99.5) 3 (0.5-19) |
98 (86-100) 2 (0.3-14) |
99 (90-99.9) 1 (0.1-10) |
| 56/99 (Ag-RDT) | Infection P (PPV) No infection P (1-PPV) |
36 (4-88) 64 (12-96) |
53 (11-91) 47 (9-89) |
75 (29-95) 25 (4-71) |
86 (48-98) 14 (2-52) |
91 (59-98) 9 (1.5-41) |
95 (72-99) 5 (0.7-28) |
97 (79-100) 2 (0.3-14) |
98 (85-100) 1.8 (0.2-15) |
| If negative (-) result | Post-test probability (%) | ||||||||
| 89/99 (RT-PCR) | Infection P (1- NPV) No infection P (NPV) |
0.1 (0-4) 99.9 (96-100) |
0.2 (0-4) 99.8 (96-100) |
0.6 (0.1-5) 99.4 (95-100) |
1.2 (0.2-6) 98.8 (94-99.8) |
1.9 (0.5-7) 98.1 (92-99.5) |
3.6 (1-10) 96.4 (90-99) |
6 (2-14) 94 (86-98) |
10 (4-21) 90 (79-95) |
| 56/99 (Ag-RDT) | Infection P (1- NPV) No infection P (NPV) |
0.4 (0-5) 99.6 (95-100) |
0.9 (0.1-5) 99.1 (95-100) |
2.3 (0.7-8) 97.7 (92-99) |
5 (2-11) 95 (89-98) |
7 (3-14) 93 (85-96) |
13 (7-22) 87 (78-93) |
19 (12-29) 81 (71-88) |
31 (21-42) 69 (58-79) |
P: probability; PPV: positive predictive value; NPV: negative predictive value; LR +/-: Positive/negative likelihood ratio.
The probability of having the infection whether the result is positive or negative is expressed in % (95% CI). In bold, 1-NPV < 5% (threshold for ruling out infection). HUA: Araba University Hospital, Vitoria-Gasteiz.
Table 1 also shows the NPV estimation, whose values for both tests decreased as prevalence increased. As an example, when the prevalence is 50% in the hospital emergency service, confident intervals show that the probability of having the infection when the result is negative (1- NPV or false negatives) is 21% with RT-PCR and 42% with Ag-RDTs.
Table 2 features the post-test probability for RT-PCR and Ag-RDTs depending on the pre-test probability estimated considering clinical situation, age, and contacts. Regardless of age, contacts, and pre-test probability, post-test probability was similar for both test methods. PPV was ≥ 97 % in all cases except for asymptomatic with unknown contacts (pre-test probability ≤ 5%). For subjects with close contacts, PPV was always close to 100%. In general, the probability of infection if the result is negative (1-NPV or false negatives) is higher for the Ag-RDTs than for the RT-PCR. Confident intervals indicate that the probability of false negatives is as low as 5% with RT-PCR and 8% for Ag-RDTs for asymptomatic with unknown contacts over 60 years of age (pre-test probability of 5%). False negatives increased as the pre-test probability increased. Moreover, if the result is negative, the probability of infection is higher if the patient has close contacts than if they have unknown contacts, regardless of age and the test method.
Table 2.
Conditional probability for the diagnosis of SARS-CoV-2 infection depending on pre-test probability by clinical situation, age and unknown or close contact [20], presuming a sensitivity of 89% and a specificity of 99%, LR +/- 89/0.11 (RT-PCR)[5] and a sensitivity of 56% and a specificity of 99%, LR +/- 56/0.44 (Ag-RDT) [10].
| Asymptomatic | Odynophagia + nasal congestion | Odynophagia + nasal congestion + anosmia/ageusia | Odynophagia + nasal congestion + anosmia/ageusia + fever + body weakness | Asymptomatic | Odynophagia + nasal congestion | Odynophagia + nasal congestion + anosmia/ageusia | Odynophagia + nasal congestion + anosmia/ageusia + fever + body weakness | |||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Unknown contact | 20-30 years old | 60-70 years old | ||||||||||||||||||||
| Pre-test probability | 3% | 35% | 80% | 85% | 5% | 50% | 80% | 90% | ||||||||||||||
| RT-PCR | Ag-RDT | RT-PCR | Ag-RDT | RT-PCR | Ag-RDT | RT-PCR | Ag-RDT | RT-PCR | Ag-RDT | RT-PCR | Ag-RDT | RT-PCR | Ag-RDT | RT-PCR | Ag-RDT | |||||||
| Infection P (PPV) if positive (+) result | 74 (27-95) |
63 (17-93) |
98 (86-100) |
97 (79-100) |
99.7 (94-100) |
99.6 (91-100) |
99.8 (95-100) |
99.7 (92-100) |
82 (41-97) |
75 (29-95) |
99 (90-100) |
98 (85-100) |
99.7 (94-100) |
99.6 (91-100) |
99.9 (95-100) |
99.8 (93-100) |
||||||
| Infection P (1- NPV) if negative (-) result | 0.3 (0-4) |
1.4 (0.3-6) |
6 (2-14) |
19 (12-29) |
31 (17-49) |
64 (51-75) |
39 (22-58) |
72 (58-82) |
0.6 (0.1-5) |
2.3 (0.7-8) |
10 (4-21) |
31 (21-42) |
31 (17-49) |
64 (51-75) |
50 (30-70) |
80 (67-89) |
||||||
| Close contact | 20-30 years old | 60-70 years old | ||||||||||||||||||||
| Pre-test probability | 40% | 60% | 87% | >95% | 45% | 77% | 94% | >95% | ||||||||||||||
| RT-PCR | Ag-RDT | RT-PCR | Ag-RDT | RT-PCR | Ag-RDT | RT-PCR | Ag-RDT | RT-PCR | Ag-RDT | RT-PCR | Ag-RDT | RT-PCR | Ag-RDT | RT-PCR | Ag-RDT | |||||||
| Infection P (PPV) if positive (+) result | 98 (88-100) |
97 (81-100) |
99 (92-100) |
99 (88-100) |
99.8 (95-100) |
99.7 (92-100) |
99.9 (96-100) |
99.9 (93-100) |
99 (89-100) |
98 (83-100) |
99.7 (94-100) |
99.5 (91-100) |
99.9 (95-100) |
99.9 (93-100) |
99.9 (96-100) |
99.9 (93-100) |
||||||
| Infection P (1- NPV) if negative (-) result | 7 (3-16) |
23 (15-33) |
14 (7-27) |
40 (29-52) |
43 (25-63) |
75 (61-85) |
73 (47-89) |
91 (80-97) |
8 (3-18) |
27 (18-38) |
27 (15-44) |
60 (47-72) |
63 (40-82) |
87 (75-94) |
73 (47-89) |
91 (80-97) |
||||||
P: probability; PPV: positive predictive value; NPV: negative predictive value. LR +/-: Positive/negative likelihood ratio. Post-test probability is expressed in % (95% CI).
Table 3 displays the post-test probabilities for IgG test based on seroprevalence, and pre-test probability of total antibodies (IgG and IgM) among patients with varying degrees of clinical suspicion for COVID-19 and negative RT-PCR throughout the course of their illness. In seroprevalence studies, although the specificity of IgG alone tests is high (around 99%), when seroprevalence is low, false positives, which tend to overestimate infection numbers, cannot be avoided. With a seroprevalence of 5%, false positives reach 17%, while true negatives account for more than 99% of negative results. On the other hand, the available total antibodies tests can help the diagnosis of COVID-19 in those doubtful cases with RTPCR or Ag-RDT tests repeatedly negative. For instance, in a patient with a high clinical suspicion (persistent symptoms) and repeatedly negative RT-PCR testing (40% pre-test probability), the probability of having the infection reaches values around 100% when the total antibody test is positive at week 3 after onset, and thus the infection would be confirmed. In a second patient with mild symptoms lasting for more than a month of evolution (low pre-test probability), a negative total antibody test practically rules out the infection, while a positive total antibody test, according to the confidence intervals, could yield false positives (1-PPV) of up to 30%.
Table 3.
Conditional probability of diagnosis of SARS-CoV-2 infection with assays targeting IgG depending on seroprevalence (sensitivity of 90% and a specificity of 99%, LR +/- 90/0.1) [13] or pre-test probability of total antibodies (IgG and IgM) among patients with varying degrees of clinical suspicion for COVID-19 and either negative RT-PCR throughout the course of their illness, presuming a sensitivity of 91% and a specificity of 99.8%, LR +/- 455/0.09 at week 3 after onset and a sensitivity of 94.3% and a specificity of 99.8%, LR +/- 471/0.06 at week 5 after onset [13].
| Seroprevalence [19] | Pre-test probability [22] | |||
|---|---|---|---|---|
| 5% | 10% | 40% | 10% | |
| Seroprevalence study ENE-COVID, 27 April - 11 May 2020 |
Unvaccinated old patient with diabetes presents with low-grade fever and mild cough 15 days prior, beginning 5 days after attending a family reunion | Young patient previously healthy and vaccinated presents with 5 weeks of debilitating fatigue and difficulty concentrating. The patient informs 2 days of a mild sore throat shortly before the onset of current symptoms. | ||
| If positive (+) result | Assays targeting IgG alone sensitivity 90%, specificity 99% |
Total antibodies, sensitivity 91%, specificity 99.8% (week 3 after onset) |
Total antibodies, sensitivity 94.3%, specificity 99.8% (week 5 after onset) |
|
| Infection P (PPV) No infection P (1-PPV) |
83 (41-97) 17 (3-59) |
91 (60-98.5) 9 (1.5-39) |
99.7 (90-100) 0.3 (0-10) |
98.1 (69-100) 1.9 (0.1-31) |
| If negative (-) result | ||||
| Infection P (1- NPV) No infection P (NPV) |
0.5 (0.1-5) 99.5 (95-100) |
1 (0.2-6) 99 (94-100) |
6 (2-14) 94 (86-98) |
0.6 (0.1-5) 99.4 (95-100) |
P: probability; PPV: positive predictive value; NPV: negative predictive value. LR +/-: Positive/negative likelihood ratio. The probability of having the infection whether the result is positive, or negative is expressed in % (95% CI).
DISCUSSION
In this work, we present the estimated conditional probability for RT-PCR, Ag-RDT and antibody detection diagnostic assays, in various setting and clinically relevant real-life situations using real data of SARS-CoV-2 prevalence in Spain in 2020. It is well known that false positive and false negative results cannot be completely avoided, despite different strategies to minimize them [22, 26-27].
In 2020, population screenings were very frequent in Spain, and RT-PCR was the diagnostic test used, which is not a screening test. These screenings generally provided a prevalence less than 1% [15]. For this prevalence value, and according to our results, the probability of not having infection if the result is negative is very high (> 99.8%). However, it is very important to take into account, that this strategy generates a high number of false positives. As an example, assuming 89% sensitivity and 99% specificity of RT-PCR, for every 50,000 people screened, we would detect 940 positive results, 495 of them may be false positives (53%). With a lower sensitivity (56%), Ag-RDT would have detected 775 positives, 64% of them may be false positives. On the one hand, population screenings generate unnecessary quarantines, economic losses associated with people who should not have been isolated and consume enormous human and material resources. On the other hand, population screenings with molecular tests led to the saturation of microbiology laboratories, hindering the rapid response to the tests requested from settings with high prevalence, such as hospital emergencies, nursing homes, symptomatic patients and exposed and / or vulnerable people. Processing large numbers of samples within a short period impairs the normal work-flow of microbiology laboratories.
Based on the fact that diagnostic tests are not perfect and can be quite inaccurate, it is particularly important to determine how well diagnostic tests rule out infection. According to our data, using RT-PCR, the probability of having the infection when the result is negative (1-NPV) is lower than 5%, even if the pre-test probability is as high as 30%, which leads the subject to be considered uninfected. In this case, these results give confidence to both staff and visitors of elderly people in nursing homes. By using Ag-RDT, which have lower sensitivity than the RT-PCR, the post-test probability may remain below the 5% threshold if the prevalence is ≤ 10%. In settings of high prevalence, such as the hospital emergency services during the first pandemic wave, a positive result with either diagnostic tools, would confirm the infection. On the contrary, a negative result, even using the most sensitive test (RT-PCR), would not rule out infection if the pre-test probability is high. In this situation, the clinician should consider it a false negative and the repetition of the test should be proposed.
To know the impact of pre-test probability on the viral Ag-RDT and RT-PCR results, the clinical situation, the age and exposure history of the patient must be considered. As expected, both diagnostic tools provided high PPV (≥ 97%) when the pre-test probability values are higher than 35%; this result indicates that they correctly classify almost infected individuals as positive. Only if the subject is asymptomatic with unknown contact, the PPV is lower, though always ≥ 63%. These results indicate that a high proportion of positive results in asymptomatic patients may be false positives, and the repetition of the test must be done to confirm the infection.
It is important to highlight that a single negative test result may not be informative if the pre-test probability is high. It is estimated that one patient with typical symptoms of COVID-19 (odynophagia, nasal congestion, fever and body weakness) has a pre-test probability of 90%. If they have a negative RT-PCR or Ag-RDT result, the probability of having the infection is 50% or 80%, respectively, depending on the diagnostic test. Even if the patient has two negative test results, there is still a risk of infection of 10% (RT-PCR) and 64% (Ag-RDT), data not shown. In this regard, Arévalo-Rodriguez et al. [28] estimated that out of every 100 tested subjects by RT-PCR, and assuming a prevalence of 50%, 1 to 27 cases would be misdiagnosed and, therefore, adequate clinical management would not be applied; repeated testing during hospitalization or additional testing for other diagnoses would be required. Our results agree with those of Arévalo-Rodriguez et al. In fact, considering the confident intervals, the probability of false negatives we estimated ranges from 4 to 21% with RT-PCR when the pre-test probability is 50%. Most authors consider RT-PCR as imperfect reference standard [27,29], even when used repeatedly, because it tends to underestimate the false negatives (RT-PCR is not done for all patients). Although clinical history, epidemiological data and imaging tests are considered jointly with the RT-PCR as a composite standard, bias are not avoided because the assessed test is part of the comparison standard. Therefore, there is a tendency to overestimate sensitivity. False negative cases have important implications for isolation and transmission risk of infected people, and a single negative test should not be used as a rule-out in patients with typical symptoms of COVID-19.
At the population level, antibody tests can be useful in estimating the proportion of people who have serum antibodies to SARS-CoV-2 as a result of a previous infection. However, the uncertainties of seroprevalence studies limit their usefulness for assessing the impact of both non-pharmacological interventions and vaccination campaigns [30]. It is known that protection against COVID-19 induced by infections and vaccines decreases over time. In addition, when many people are sampled, a large number of false positives will be detected if the prevalence is low, even if the test used has a high specificity. The prevalence estimated is not useful to distinguish a high percentage of asymptomatic people from a high level of false positives. In the latter case, the degree of prior infection will be overestimated, which may lead to the relaxation of control measures. It is generally accepted that the estimates provided by seroprevalence studies should be interpreted in conjunction with further information, such as confirmed cases, deaths and infectious disease models, to better understand the disease [31].
At the individual level, antibodies to SARS-CoV-2 are detected in almost all patients after the second week of symptom onset, and they may be useful when RT-PCR or Ag-RDT tests are negative in patients with clinically suspected COVID-19. In this work, we describe two cases with different pre-test probability, presented as clinical examples by the IDSA Diagnostics Committee [22]. In the first case, the patient with potential exposure and symptoms suggestive of COVID-19, presented a high risk of evolution to severe infection due to age, diabetes and not being vaccinated (high pre-test probability). This clinical presentation suggests cytokine release syndrome (CRS), which occurs 1-2 weeks after acute infection and where RTPCR negative tests have been described with some frequency [32]. A positive SARS-CoV-2 anti-N result would confirm the diagnosis in these patients, who could benefit from the establishment of early treatment with immunomodulators.
The second case involves patients with mild COVID-19 who did not undergo diagnostic tests and experienced sequelae several weeks after a paucisymptomatic infection (low pre-test probability) and had negative RT-PCR results at the time of consultation with the doctor. This presentation suggests post-acute sequelae of SARS-CoV-2 (PASC), which can deteriorate the quality of life of these patients. In addition to a positive anti-S SARS-CoV-2 antibody result due to vaccination, a positive or negative anti-N at week 5 after onset could confirm or rule out the diagnosis of PASC.
The study has several limitations. First, prevalence reported in the different settings was during the first pandemic waves in 2020, and it should be noted that when the prevalence of COVID-19 changes, the predictive values of the tests will also change. Second, Omicron variant is currently the global-dominated strain with multiple subvariants which demands a continuous evaluation of current detection methods. A lower sensitivity of Ag-RDT for the Omicron variant has been reported [33,34], although similar sensitivity with high viral loads (Ct < 25) between symptomatic and asymptomatic cases have been detected [35]. WHO recommends that Ag-RDT should be prioritized for use in symptomatic individuals meeting the case definition for COVID-19, and to test asymptomatic individuals at high risk of infection, including contacts and health workers, particularly in settings where RT-PCR testing capacity is limited [36]. However, some authors consider that a rapid, highly-specific but modestly-sensitive test for SARS-CoV-2 (sensitivity around 50%) such as Ag-RDT, still allows to identify a large proportion of infected individuals and at the same time reduce the isolation of uninfected contacts [37]. Third, the heterogeneity observed in the studies included in the different meta-analyses may affect the diagnostic performance indicators of the tests used for the diagnosis of SARS-CoV-2 infection. Several factors can alter these indicators, including disease prevalence, sample type (saliva, nasal swabs, nasopharyngeal swabs, pooled nose and throat swabs), study setting, symptom status, etc. However, regarding RT-PCR, Tsang et al. [38] in a meta-analysis subsequent to those used in this work, compared the diagnostic performance of different clinical samples collection methods and provided similar values of sensitivity (86%, 95%CI: 77%-93%) and specificity (99%, 95%CI: 96%-100%) for nasopharyngeal and nasal samples among individuals presenting in ambulatory care. Fourth, thresholds for ruling out infection, established in this study at 5%, may vary depending on the sensitivity of the test used and on the clinical-epidemiological needs (for instance, it should be lower in case of visits to hospitalized immunocompromised relatives).
In conclusion, by analyzing Spain prevalence and sero-prevalence data during the first waves in 2020, and the estimation of the pre-test probability from epidemiological and clinical data, we have confirmed that interpretating the result of a COVID-19 test depends not only on the accuracy of the test, but also on the prevalence of the infection in different settings, and the pre-test probability associated to the patient before performing the test.
ACKNOWLEDGEMENTS
*Members of the Araba University Hospital Group on COVID-19 Diagnostic Tests: Silvia Hernáez Crespo, Mª Luz Cordón Rodríguez, Amaia Aguirre Quiñonero, Mª Concepción Lecaroz Agara, Marina Fernández Torres, José Israel López Mirones, Mª Rosario Almela Ferrer, Jorge Arribas García, Ana Mª Sánchez Díaz and Izaskun Alejo Cancho.
FUNDING
None to declare
CONFLICT OF INTEREST
The authors declare no conflict of interest
References
- 1.Alvarez Ruiz S, Canut Blasco A, Rodeño Ortiz de Zárate E, Barbero Martínez I, Alonso Colmenares JI, Cortés Hernández Jet al. Medici-na basada en la evidencia. Aplicación a la medicina nuclear. Vertiente diagnóstica. Parte II [Evidence based medicine. Application to nuclear medicine. Diagnostic slope. Part II]. Rev Esp Med Nucl. 2001. Aug 20(5):393-412; quiz 413-5. Spanish. doi: 10.1016/s0212-6982(01)71982-9. [DOI] [PubMed] [Google Scholar]
- 2.Woloshin S, Patel N, Kesselheim AS. False Negative Tests for SARSCoV-2 Infection-Challenges and Implications. N Engl J Med. 2020. Aug 6;383(6):e38. doi: 10.1056/NEJMp2015897. [DOI] [PubMed] [Google Scholar]
- 3.Zhu Y, Zhang M, Jie Z, Tao S. Nucleic acid testing of SARS-CoV-2: A review of current methods, challenges, and prospects. Front Micro-biol. 2022. Dec 9;13:1074289. doi: 10.3389/fmicb.2022.1074289. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Antonelli G, Stefani S, Pistello M. SARS-CoV-2 diagnostics: Some reflections on current assays. Diagn Microbiol Infect Dis. 2021. Feb;99(2):115237. doi: 10.1016/j.diagmicrobio.2020.115237. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Kim H, Hong H, Yoon SH. Diagnostic Performance of CT and Reverse Transcriptase Polymerase Chain Reaction for Coronavirus Disease 2019: A Meta-Analysis. Radiology. 2020. Sep;296(3):E145-E155. doi: 10.1148/radiol.2020201343. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Floriano I, Silvinato A, Bernardo WM, Reis JC, Soledade G. Accuracy of the Polymerase Chain Reaction (PCR) test in the diagnosis of acute respiratory syndrome due to coronavirus: a systematic review and meta-analysis. Rev Assoc Med Bras (1992). 2020. Jul;66(7):880-888. doi: 10.1590/1806-9282.66.7.880. [DOI] [PubMed] [Google Scholar]
- 7.Jarrom D, Elston L, Washington J, Prettyjohns M, Cann K, Myles Set al. Effectiveness of tests to detect the presence of SARS-CoV-2 virus, and antibodies to SARS-CoV-2, to inform COVID-19 diagnosis: a rapid systematic review. BMJ Evid Based Med. 2020. Oct 1:bmjebm-2020-111511. doi: 10.1136/bmjebm-2020-111511. [DOI] [PubMed] [Google Scholar]
- 8.Mayers C, Baker K. Impact of false-positives and false-negatives in the UK’s COVID-19 RT-PCR testing programme. https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/895843/S0519_Impact_of_false_positives_and_negatives.pdf. Date: June 3, 2020. Date accessed: March 23, 2023.
- 9.García F, Melón S, Navarro D, Paño JR, Galán JC. Organización del diagnóstico de SARS-CoV-2 y estrategias de optimización. V 1.0. 14/10/2020. Galán JC, Paño JR. (editores). Documentos y recomendaciones. Sociedad Española Enfermedades Infecciosas y Microbiología Clínica (SEIMC). https://covid19.seimc.org/index.php/diagnostico-sars-cov-2/. Date: 2020. Date accessed: March 23, 2023. [Google Scholar]
- 10.Dinnes J, Deeks JJ, Adriano A, Berhane S, Davenport C, Dittrich S, et al. ; Cochrane COVID-19 Diagnostic Test Accuracy Group . Rapid, point-of-care antigen and molecular-based tests for diagnosis of SARS-CoV-2 infection. Cochrane Database Syst Rev. 2020. Aug 26;8:CD013705. doi: 10.1002/14651858.CD013705. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Agulló V, Fernández-González M, Ortiz de la Tabla V, Gonzalo-Jiménez N, García JA, Masiá Met al. Evaluation of the rapid antigen test Panbio COVID-19 in saliva and nasal swabs in a population-based point-of-care study. J Infect. 2020. Dec 9S0163-4453(20)30768-4. doi: 10.1016/j.jinf.2020.12.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Alemany A, Baró B, Ouchi D, Rodó P, Ubals M, Corbacho-Monné Met al. Analytical and clinical performance of the panbio COVID-19 antigen-detecting rapid diagnostic test. J Infect. 2021. Jan 7:S0163-4453(21)00004-9. doi: 10.1016/j.jinf.2020.12.033. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Fox T, Geppert J, Dinnes J, Scandrett K, Bigio J, Sulis Get al. ; Cochrane COVID-19 Diagnostic Test Accuracy Group . Antibody tests for identification of current and past infection with SARS-CoV-2. Cochrane Database Syst Rev. 2022. Nov 17;11(11):CD013652. doi: 10.1002/14651858.CD013652.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Watson J, Richter A, Deeks J. Testing for SARS-CoV-2 antibodies. BMJ. 2020. Sep 8;370:m3325. doi: 10.1136/bmj.m3325. [DOI] [PubMed] [Google Scholar]
- 15.Datos COVID-19 en Euskadi. https://www.euskadi.eus/gobierno-vasco/contenidos/noticia/2020_11_02_64303/es_64303/64303.html. Last accessed: March 23, 2023.
- 16.Albert E, Torres I, Bueno F, Huntley D, Molla E, Fernández-Fuentes MÁet al. Field evaluation of a rapid antigen test (Panbio™ COVID-19 Ag Rapid Test Device) for COVID-19 diagnosis in primary healthcare centres. Clin Microbiol Infect. 2020. Nov 13:S1198-743X(20)30697-2. doi: 10.1016/j.cmi.2020.11.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Borras-Bermejo B, Martínez-Gómez X, San Miguel MG, Esperalba J, Antón A, Martin Eet al. Asymptomatic SARS-CoV-2 Infection in Nursing Homes, Barcelona, Spain, April 2020. Emerg Infect Dis. 2020. Sep;26(9):2281–3. doi: 10.3201/eid2609.202603. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Servicio de Urgencias . Hospital Universitario Araba. Vitoria-Gasteiz. (Dr. Javier Oribe. Personnel communication).
- 19.Pollán M, Pérez-Gómez B, Pastor-Barriuso R, Oteo J, Hernán MA, Pérez-Olmeda M, et al. ; ENE-COVID Study Group . Prevalence of SARS-CoV-2 in Spain (ENE-COVID): a nationwide, population-based seroepidemiological study. Lancet. 2020. Aug 22;396(10250):535-544. doi: 10.1016/S0140-6736(20)31483-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Brooks, Z.; Das S.; Pliura, T. A “Clinician’s Probability Calculator” to convert pre-test to post-test probability of COVID-19, based on method validation from each laboratory. Preprints 2020, 2020120094 (doi: 10.20944/preprints202012.0094.v1). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Hanson KE, Caliendo AM, Arias CA, Englund JA, Hayden MK, Lee MJet al. Infectious Diseases Society of America Guidelines on the Diagnosis of COVID-19: Serologic Testing. Clin Infect Dis. 2020. Sep 12:ciaa1343. doi: 10.1093/cid/ciaa1343. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Colgrove R, Bruno-Murtha LA, Chastain CA, Hanson KE, Lee F, Odom John ARet al. Tale of the Titers: Serologic Testing for SARS-CoV-2-Yes, No, and Maybe, With Clinical Examples From the IDSA Diagnostics Committee. Open Forum Infect Dis. 2022. Dec 14;10(1):ofac674. doi: 10.1093/ofid/ofac674. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Fletcher RH, Fletcher SW. Epidemiología Clínica. 4ª edición. Barcelona: Wolters Kluwer, Lippincott Williams and Wilkins, 2008. [Google Scholar]
- 24.Brooks Z, Das S, Pliura T. Clinicians’ Probability Calculator to Convert Pre-Test to Post-Test Probability of SARS-CoV-2 Infection Based on Method Validation from Each Laboratory. EJIFCC. 2021. Jun 29;32(2):265-279. [PMC free article] [PubMed] [Google Scholar]
- 25.Primo J. Critical Appraisal Skills Programme. https://redcaspe.org/materiales/calculadoras/calculadoras para diagnóstico. Last accessed March 23, 2023.
- 26.Surkova E, Nikolayevskyy V, Drobniewski F. False-positive COVID-19 results: hidden problems and costs. Lancet Respir Med. 2020. Dec;8(12):1167-1168. doi: 10.1016/S2213-2600(20)30453-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Skittrall JP, Wilson M, Smielewska AA, Parmar S, Fortune MD, Sparkes Det al. Specificity and positive predictive value of SARS-CoV-2 nucleic acid amplification testing in a low-prevalence setting. Clin Microbiol Infect. 2021. Mar;27(3):469.e9-469.e15. doi: 10.1016/j.cmi.2020.10.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Arevalo-Rodriguez I, Buitrago-Garcia D, Simancas-Racines D, Zambrano-Achig P, Del Campo R, Ciapponi Aet al. False-negative results of initial RT-PCR assays for COVID-19: A systematic review. PLoS One. 2020. Dec 10;15(12):e0242958. doi: 10.1371/journal.pone.0242958. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Watson J, Whiting PF, Brush JE. Interpreting a covid-19 test result. BMJ. 2020. May 12;369:m1808. doi: 10.1136/bmj.m1808. [DOI] [PubMed] [Google Scholar]
- 30.Bendavid E, Mulaney B, Sood N, Shah S, Bromley-Dulfano R, Lai Cet al. COVID-19 antibody seroprevalence in Santa Clara County, California. Int J Epidemiol. 2021. May 17;50(2):410-419. doi: 10.1093/ije/dyab010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.McConnell D, Hickey C, Bargary N, Trela-Larsen L, Walsh C, Barry Met al. Understanding the Challenges and Uncertainties of Sero-prevalence Studies for SARS-CoV-2. Int J Environ Res Public Health. 2021. Apr 27;18(9):4640. doi: 10.3390/ijerph18094640. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Fontana LM, Villamagna AH, Sikka MK, McGregor JC. Understanding viral shedding of severe acute respiratory coronavirus virus 2 (SARS-CoV-2): Review of current literature. Infect Control Hosp Epidemiol. 2021. Jun;42(6):659-668. doi: 10.1017/ice.2020.1273. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Jüni P, Baert S, Corbeil A, Johnstone J, Patel SN, Bobos Pet al. Use of rapid antigen tests during the omicron wave. Science Briefs of the Ontario COVID-19 Science Advisory Table. 2022;3(56). 10.47326/ocsat.2022.03.56.1.0 [DOI] [Google Scholar]
- 34.Bayart JL, Degosserie J, Favresse J, Gillot C, Didembourg M, Djokoto HPet al. Analytical Sensitivity of Six SARS-CoV-2 Rapid Antigen Tests for Omicron versus Delta Variant. Viruses. 2022. Mar 22;14(4):654. doi: 10.3390/v14040654. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Regev-Yochay G, Kriger O, Mina MJ, Beni S, Rubin C, Mechnik Bet al. Real World Performance of SARS-CoV-2 Antigen Rapid Diagnostic Tests in Various Clinical Settings. Infect Control Hosp Epidemiol. 2022. Mar 2:1-20. doi: 10.1017/ice.2022.3. [DOI] [PubMed] [Google Scholar]
- 36.Antigen-detection in the diagnosis of SARS-CoV-2 infection Interim guidance 6 October 2021. WHO reference number: WHO/2019-nCoV/Antigen_Detection/2021.1. https://www.who.int/publications/i/item/antigen-detection-in-the-diagnosis-of-sars-cov-2infection-using-rapid-immunoassays. Last accessed: May 19, 2023.
- 37.Kennedy-Shaffer L, Baym M, Hanage WP. Perfect as the enemy of good: tracing transmissions with low-sensitivity tests to mitigate SARS-CoV-2 outbreaks. Lancet Microbe. 2021. May;2(5):e219-e224. doi: 10.1016/S2666-5247(21)00004-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Tsang NNY, So HC, Ng KY, Cowling BJ, Leung GM, Ip DKM. Diagnostic performance of different sampling approaches for SARSCoV-2 RT-PCR testing: a systematic review and meta-analysis. Lancet Infect Dis. 2021. Sep;21(9):1233-1245. doi: 10.1016/S1473-3099(21)00146-8. [DOI] [PMC free article] [PubMed] [Google Scholar]