A word of caution in interpreting COVID‐19 diagnostics tests

To the Editor,

Since the emergence of SARS‐CoV‐2, many diagnostic tests including molecular and serological assays have been developed and approved by the Food and Drug Administration (FDA) for diagnosis of COVID‐19. ¹ , ² However, concerns about the sensitivity and specificity of many diagnostic assays, especially the rapid tests, have been raised. Diagnostic tests with unacceptable rates of false positive and false‐negative results interfere with therapeutic management of patients and can have serious implications for public health authorities in the decision‐making process regarding COVID‐19 control.

In this context, understanding key concepts in terms of development and validation of diagnostic assays is crucial for correct interpretation of test results. The parameters for validating diagnostic tests include analytical sensitivity, analytical specificity, clinical sensitivity, clinical specificity, positive likelihood ratio, negative likelihood ratio, positive predictive value (PPV), negative predictive value (NPV), repeatability, reproducibility, and accuracy (Table 1). After initial development and optimization of a new COVID‐19 test, the performance of the assay should be assessed using a set of well‐defined clinical samples taking into account the required sample size. Ideally, data used for validation of the COVID‐19 test should be published in a peer‐reviewed publication to allow independent evaluation.

Table 1.

Diagnostic parameters analyzed during the development and clinical validation

Diagnostic parameter	Definition
Analytical sensitivity/limit of detection	Lowest concentration of the analyte that can be reliably detected by the assay.
Analytical specificity	Ability of the assay of not cross‐reacting with other pathogens.
Clinical sensitivity	Probability the test is positive when the infection is present.
Clinical specificity	Probability the test is negative when the infection is absent.
Positive likelihood ratio	Ratio between the probability of an infected person testing positive and the probability of an uninfected person testing positive.
Negative likelihood ratio	Ratio between the probability of an infected person testing negative and the probability of an uninfected person testing negative.
Positive predictive value (PPV)	Probability that the pathogen is present when the test result is positive.
Negative predictive value (NPV)	Probability that the pathogen is absent when the test result is negative.
Repeatability	Agreement between results of replicates of a sample both within and between runs of the same test in the same laboratory.
Reproducibility	Agreement between results of patient specimens assayed in different laboratories.
Robustness	Ability of the test to remain unaffected by minor variations that may occur during the testing process.
Accuracy	Overall probability that the patient is correctly diagnosed by the test.

Validation is essential for the development of a diagnostic test and requires a series of interrelated steps where the diagnostic test is experimentally standardized to detect the analyte (antibody, antigen, nucleic acid [DNA or RNA]), with precision and high accuracy. ³ Importantly, the new test should be compared side‐by‐side to a gold standard method that is used as a reference method to detect the pathogen. In the case of SARS‐CoV‐2, quantitative reverse transcription polymer chain reaction (RT‐qPCR) is considered the reference test for the laboratory diagnosis of COVID‐19 patients recommended by the World Health Organization (WHO). ¹

Recently, several studies have raised concerns about false‐negative RT‐qPCR results in patients with COVID‐19 disease during the pandemic course. ¹ , ⁴ In a recent study, Li et al. ⁴ tested specimens collected from 610 hospitalized patients from Wuhan, China, and found a high rate of false‐negative RT‐qPCR results. There are a number of reasons that may produce false‐negative results, including low viral load in the patient sample, inadequate storage during specimen transportation, laboratory error during sampling, low sensitivity of the diagnostic test, or the use of the unsuitable diagnostic mode according to date of sample collection post the onset of symptoms (molecular or serological approach). Molecular assays should also consider the presence of mutations/mismatches in primer/probe binding sites in the SARS‐CoV‐2 genome that might interfere with viral detection and produce false‐negative RT‐qPCR results. ⁵

In the last few months, a variety of serological assays have been designed to detect antibodies against different portions of the SARS‐CoV‐2 genome. The viral nucleocapsid protein (N) and spike protein (S) have been the preferred antigens for use in serology because of their high antigenicity. ⁶ Recent studies suggested that the receptor‐binding domain (RBD) of the viral spike protein (S1 subunit) is a major immunodominant epitope against which antibodies to SARS‐CoV‐2 are directed. The S1 subunit is more specific than S for the serological diagnosis of SARS‐CoV‐2 infection. ⁷ , ⁸

Many studies have also reported false‐negative results in serological tests. ⁹ , ¹⁰ , ¹¹ In this context, Tang et al. ¹⁰ evaluated the clinical performance of two serological tests (Abbott and EUROIMMUN [EI]) to detect SARS‐CoV‐2 using 103 samples from 48 patients with COVID‐19‐confirmed previously assayed by RT‐qPCR. They analyzed the diagnostic performance using specimens from different times after illness onset (<3, 3–7, 8–14, and ≥14 days). The results revealed that both serological tests had poor clinical sensitivity, especially when used during the early phase of COVID‐19 infection (≥14 days) generating false‐negative results. In another related meta‐analysis study, Castro et al. ⁹ evaluated the diagnostic performance of 16 commercial serological assays approved and registered by the Brazilian Health Regulatory Agency (ANVISA) for use in Brazil, the third country most severely affected by the COVID‐19 pandemic. The authors found a high rate of false‐negative results obtained from serological tests, mainly in patients in the first two weeks of COVID‐19 onset. Thus, the timing of disease onset is a critical factor when evaluating a molecular or a serological test. ⁸

Despite the exceptional efforts made by public institutions and private companies to rapidly develop COVID‐19 diagnostic tests in the past few months, diagnostic laboratories should be cautious in choosing the COVID‐19 to be used given the risks of inaccurate results. Finally, we suggest clinical validation under realistic conditions using patient samples collected from different times after the symptoms onset, different geographical locations, and different forms of the disease severity (asymptomatic, mild, and severe cases). After the independent validation and registration for diagnostic use—continuous surveillance and evaluation of performance features of the diagnostic assay are required to ensure and validate test results. This will enable health authorities, clinicians, and governments to make sound decisions aimed at controlling this devastating pathogen.