SARS-CoV-2 detection by reverse transcriptase polymerase chain reaction testing: Analysis of false positive results and recommendations for quality control measures

Abstract

Testing for SARS-CoV-2 has become a critical component for the management of the COVID-19 pandemic. Reverse transcriptase polymerase chain reaction (RT-PCR) assays are currently the predominate method for testing. Quality control (QC) measures utilize known positive and known negative controls to ensure the adequacy of extraction and RT-PCR steps but do not evaluate all components of testing. We have conducted a quality assurance review of our RT-PCR testing for COVID-19 to determine the rate of false positive results in asymptomatic patients and causes for these errors.

Design

We have developed a quality control procedure in which all specimens from asymptomatic unexposed persons with SARS-CoV-2 positive tests were retested. When a second test was “non-detected” a third test was performed and a root cause analysis of the erroneous result undertaken.

Results

In the study period, 24,717 samples were tested and 6251 were from asymptomatic patients. Of the 288 initial positive tests, 20 (6.9%) were negative on retesting. Review of cycle threshold curves, technologists’ records, location of specimen on testing plates and relationships with high viral load specimens was undertaken. Analysis revealed technologists’ errors (misplacement of specimen in testing plate or contamination) and cross contamination from high viral load specimens in adjacent wells of testing plates were common causes for false positive results.

Discussion

SARS-CoV-2 RT-PCR testing is associated with a small number of false positive results, most easily recognized in asymptomatic non-exposed patients. Implementation of a limited retesting protocol identifies clinically significant testing errors and allows review and improvement of laboratory procedures.

1. Introduction

The severe acute respiratory syndrome Coronavirus 2 (SARS-CoV-2) pandemic has presented major challenges to the global health care system, not least of which has been the availability of accurate wide spread testing of symptomatic and potentially exposed individuals. While a variety of testing methods have been identified, reverse transcriptase polymerase chain reaction (RT-PCR) for the detection of SARS-CoV-2 nucleic acids is currently the most widely used method and is generally regarded as most sensitive and specific. Testing is necessary for identification of infected patients and for contact tracing. A positive result often results in patient (and contacts) quarantine. Such quarantine may delay important medical interventions in those patients with serious co-morbidities (such as cancer) or result in patients being transferred to COVID-19 wards. If a test is falsely positive and a patient is placed in a COVID-19 ward, that increases the potential exposure to SARS-CoV-2 virus for that patient, with a possible increased risk for developing COVID-19. Both the delay of other potentially necessary medical interventions and the potential transfer to isolation wards with increased risk for contraction of the virus are undesirable results. Thus, minimizing false positive SARS-CoV-2 tests is highly desirable.

Xiao et al. [1] outlined a series of steps for optimal laboratory management and quality control for laboratories doing SARS-CoV-2 nucleic acid testing. They discussed the use of plasmids provided in the commercial test kits as negative and positive quality controls. They however observed batch to batch and box to box variations in the quality control products and reaction systems that in their experience lead to unreliable results. Their quality control methods focused predominately on the sensitivity of testing to detect low virus load cases, i.e. diminishing or eliminating false negative results. False positive results were not emphasized as a potential quality control issue in their report.

Layfield et al. [2] observed a small number of patients without symptoms of COVID-19 or known exposures to infected individuals, who had positive test results. Since these patients had no exposure history nor were they symptomatic for COVID-19 disease, the discordance between expected results and laboratory findings caused the authors to re-test their specimens. Layfield et al. [2] found that up to 19% of this cohort’s initial positive results were falsely positive based on the follow up testing. To help ensure the accuracy of our positive results we developed a protocol for repeat testing of all positive results in asymptomatic and unexposed patients ( Figs. 1 and 2) and in all cases where a specimen with a positive result was located in the 96 well test plate near another specimen with a high virus load. Herein we report the results of that testing protocol over an eight-week period.

Fig. 1.

Protocol for Reporting positive PCR Results for SARS-CoV-2 Testing in asymptomatic and non-exposed patients.

Fig. 2.

Root cause analysis for false positive results.

2. Materials and methods

The study underwent Institutional Review Board review and was determined to be exempt. Following the detection of a number of false positive COVID-19 RT-PCR tests, a quality control protocol was implemented as shown in Figs. 1 and 2. Our test order form requests clinical information including the presence or absence of symptoms and information on possible patient exposure to COVID-19. The study utilizing the quality control protocol was performed between September 14 and October 31, 2020. The quality control protocol requires that all positive results in asymptomatic apparently unexposed patients have retesting of those specimens. Additionally, the plate maps of all patients testing positive for SARS-CoV-2 are analyzed to determine if the specimen well was near another sample with a high viral load. High viral loads were defined as a cycle threshold (CT) less than 20. Positive cases with proximity to a high viral load specimen were retested. When found to be positive on retesting these specimens were reported as positive. When repeat testing was negative, the specimen was recorded as inconclusive and an additional specimen was requested. Each positive specimen which retested as negative had a second retest performed and a root cause analysis of the error undertaken.

All participants within this study presented themselves to one of three sites at the University of Missouri. One was a drive thru site adjacent to the molecular testing laboratory and two sites were associated with University of Missouri clinics. The participants were simply asked if they believed that they had been exposed to an individual with COVID-19. No information was obtained on the length of the exposure, the type of contact or the individuals that the participant believed were their contacts. At each of the sites, the testing was performed by either registered nurses or licensed vocational nurses who had undergone a training course on how to take nasopharyngeal and nasal swabs. This educational course was required for each of the providers obtaining specimens.

All samples were either nasopharyngeal or nasal swabs placed in transport media. In all cases, initial and subsequent sample testing used RNA extraction and RT-PCR testing performed on the King Fisher Flex/ABI 7500 Fast (Thermo Fisher, Waltham, MA) or the Siemens Versant SP/Quant Studio5 (Siemens, Munich Germany) systems. The reagents (TaqPath COVID-19) and (FTD SARS-CoV2) were supplied by the respective instrument manufacturers.

All testing was performed at the University of Missouri Clinical Lab in the Molecular Diagnostic Laboratory. This is a College of American Pathologists inspected and approved laboratory performing a variety of testing for viral infections (cytomegalovirus, herpes simplex virus and human papilloma virus) as well as mutations associated with a variety of malignancies (EGFR, KRAS, and BRAF). Four certified technologists normally working in the clinical molecular laboratory along with three technologists from the veterinary pathology molecular laboratory performed the testing. The four technologists from the clinical molecular laboratory were ASCP certified technologists and the three technologists drawn from the veterinary laboratory had worked in the veterinary molecular laboratory which had been certified by the corresponding certification organization.

The sensitivity limit for the Tag Path COVID-19 assay reagent on the King Fisher Flex/ABI 7500 Fast system was LOD 500 cp/ml. The sensitivity limit for the FTD SARS-CoV-2 assay reagent used with the Siemens versant SP/Quant Studio 5 was LOD 500 cp/ml. Cut point for the TaqPath COVID-19 assay was a CT value of less than 38 and for the FTD SARS-CoV2 the cut point for a positive result was CT value less than 40.

Two positive targets were required to designate a specimen as positive. Repeat testing involved both re-extraction and repeat RT-PCR.

When repeat testing of a positive case from an asymptomatic patient was negative, root cause analysis was performed (Fig. 2). This entailed review of the RT-PCR curve by the laboratory supervisor as the first step in the analysis. When the supervisor believed that the curves had been incorrectly interpreted the case curves were shown to a second senior technologist or the laboratory medical director for confirmation of the misinterpretation.

When reanalysis of the specimen curves confirmed the initial interpretation, the “plate map” was reviewed by the supervisor to determine if the specimen well was in close proximity to a specimen with a high viral load result. Specimens with a cycle threshold curve of twenty or below were considered high viral load specimens and the erroneous result of the false positive sample was ascribed to cross contamination from the high viral load specimen.

When curve and plate position analysis did not disclose a reason for the false positive result, the supervisor discussed the specimen handling with the technologist loading the 96 well plate. The supervisor observed the technologist’s technique for specimen pipetting and plate loading. Since the error was invariably recognized within twenty-four hours, technologists usually remembered the technical issues associated with plate loading.

3. Results

During the eight-week period (September 14, 2020 to October 31, 2020) when the quality control protocol was first fully implemented, 24,717 COVID-19 tests RT-PCR were performed. Of these, 6251 specimens were obtained from asymptomatic patients, and 288 of these were positive by the initial test. Replicate testing revealed 20 (6.9%) of these positive tests to be false positive results. In the group of asymptomatic patients, the overall false positive rate was 0.3%. Review of the cycle threshold curves, heat maps to demonstrate locations of specimens on test plates in relationship to other positive tests, and technologists’ records and interviews revealed that contamination due to specimen carryover or mispositioning of testing plates (technologist error), erroneous automated interpretation of RT-PCR curves, and contamination of the test specimen from adjacent high viral load specimens were likely causes of false positive results ( Table 1).

Table 1.

Documented causes of false positive RT-PCR results.

Probable Cause	Number of Cases (% false positive)
Pipetting error by technologist with sample well contamination or specimen misplacement	6 (30%)
Incorrect insertion of testing plate	4 (20%)
Cross contamination from high viral load specimen	4 (20%)
Incorrect automated curve interpretation	6 (30%)

In six cases, review of the curves demonstrated that the analysis software supplied with the instrument had scored the specimen as positive despite only one of the three markers coming up before cycle 40 and in each of the false positive cases the cycle number had been 36 or 37. Thus we believe that the specimen had been incorrectly marked as positive by the automated software and had not been properly checked by the laboratory staff. In four cases review of the plate map showed that the false positive specimens had been immediately adjacent to a high viral load specimen with a cycle threshold of less than 20. In two of the four cases, the cycle threshold of the adjacent well had been seventeen. In four cases the technologist had incorrectly inserted the plate into the instrument. All four erroneous results were from a single plate. Unfortunately, the positive and negative controls had mirror image specimens with results matching the controls so the plate insertion error was not noticed until the supervisor reviewed the plate and results. In a further four cases, discussions between the supervisor and the technologist disclosed that the technologist had been concerned by possible sample loading issues but had elected to continue to load and use the plate because of concerns over the cost of discarding the plate or impact on turnaround time. In these cases, the technologist was counseled and advised to discard plates where they had a reasonable doubt regarding the proper loading of the plate. In the final two specimens, observation of the technologists’ pipetting and loading techniques disclosed serious breaches of good practice including “dripping” of specimens around other wells or poor pipetting practice with potential cross contamination. All six technologist errors were associated with two technologists who had been transferred from the Veterinary Molecular Laboratory following the surge in testing load and had not previously worked in the Molecular Laboratory of the hospital clinical laboratory.

4. Discussion

RT-PCR testing for SARS-CoV-2 RNA is currently the preferred technique for testing of patients with suspected COVID-19 disease. Identification of patients positive for SARS-CoV-2 is vital for identification of those infected (and potentially able to infect others) and for contact tracing. These steps are highly important for management of the current COVID-19 pandemic. Much concern has been expressed regarding false negative diagnoses and false negative rates have been significant for some of the “rapid” testing techniques [3], [4]. False negative results have been recognized as a significant problem [3], [4]. A large systematic review has documented a reported false negative rate varying from 0.018 to 0.58 (1.8–58%) [5]. A less discussed but equally important issue is that of false positive diagnoses [6], [7].

False positive diagnoses have important implications for patient management. False positive results may lead to inappropriate quarantine, delay of other necessary medical treatment including operative interventions, and when a patient is in a long term or post-operative care facility a positive result may lead to transfer to a COVID-19 ward where the risk of infection for truly negative patients is elevated. These are important health care issues but are under-discussed in the available literature.

Many testing sites including those in the United Kingdom have seen an increase in the number of asymptomatic patients being tested for COVID-19 disease [6]. This increased testing of asymptomatic individuals reduces the pre-test probability of a positive test as the proportion of asymptomatic cases screened increases [6]. This is particularly so when the prevalence of viral infections in the population is low. Indeed, Surkova et al. [6] observed a significant decrease in positivity rates of swab specimens from April 2020 to July 2020. This change in positivity rates has occurred during a time in which laboratory testing professionals have invested great efforts to reduce turnaround times and improve test sensitivity. Less emphasis has been placed on the reduction of false positive rates. Early studies have estimated that the false positive rate may be between 0.8% and 4.0% [6].

Reduction of false negative results has received priority. The issue of false positive specimens has been de-emphasized during a time of increased testing for symptomatic or exposed patients. The consequences of false positive results are not trivial. Surkova et al. [6] recounted a number of potential consequences for false positive COVID-19 test results. These include false positive pre-operative tests resulting in unnecessary cancellation or postponement of necessary surgical procedures, financial losses incurred by the individual receiving a false positive test including those from self-quarantine, income loss, and cancellation of travel. Additionally, there is the psychological impact on the patient due to the misdiagnoses resulting in fear of infecting other individuals, enforced isolation and stigmatization [6]. Beyond the individual negative impacts, society is negatively affected due to misspent funding and misplacement of human resources for retesting and tracing, finding replacements in the work place for those quarantined, and business losses [6]. Additionally, over-estimating COVID-19 incidence and the extent of asymptomatic infection may result in inappropriate planning and misplaced assignment of health care and testing resources as well as misdirection of societal policies regarding lockdowns and school closures.

Surkova et al. [6] and Lin et al. [7] have drawn attention to technical problems including contamination during sampling procedures, contamination by RT-PCR amplicons, contamination of reagents, and sample cross contamination. Early on in the global testing experience, US Food and Drug Administration (FDA) issued a warning that cross contamination could result in unacceptably high false positive rates for COVID-19 RT-PCR testing. [8].

The current rate of false positive swab tests in the United Kingdom is unknown but preliminary estimates show that it could be between 0.8% and 4.0% [6]. More recent date suggests a false-positive rate of between 0.2% and 0.9% [9]. The higher end of these estimates indicates the potential for a significant number of false positive test results in areas with a low prevalence of the virus. This would negatively impact the positive predictive value of the test, reducing its value for surveillance and development of government policies. Currently, government policies do not provide provisions for retesting of individuals who test positive despite being asymptomatic and without known contact with diseased individuals [9]. Of equal importance is an issue of an overly sensitive test picking up patients with a minimal viral load. Currently, no data suggested that detection of low levels of viral RNA by RT-PCR equates with infectivity unless infectious viral particles have been confirmed with laboratory culture methods [10]. Such viral cultures are not standard and, in the USA, at least, are only recommended for biosafety level 3 facilities [10]. The issues of false positive tests and positive tests with minimal viral load become increasingly important as testing of asymptomatic and apparently unexposed individuals increases. Retesting of all potentially false positive results does not appear to be a viable option as some authorities will not accept a subsequent negative test [9].

In a review by Cohen [9], the percentage of positive results that are incorrect (calculated on the basis of the infection and false positive rates) was 39.89%. This calculation was for a 1% infection rate in a study population and a 0.5% false positive rate. This implies a significant problem with false positive results when test positivity rates are low [9]. Layfield et al. [2] discussed the findings of false positive results in a set of patients known by clinical history to be asymptomatic and without known exposure risk. In that study of approximately 21,000 COVID-19 tests, replicate tests revealed 30 of 155 (19%) positive tests from asymptomatic patients to have negative repeat tests. In the cases with adequate sample, a second repeat test was performed and confirmed the negativity of the sample in all such cases. The false positive rate for asymptomatic patients was 0.44% similar to the rate (0.5%) used in the Cohen study [9]. Review of the cycle threshold curves, the location of the specimens on the test plates, and the technologists’ records showed that contamination due to poor technologist technique and specimen location on the testing plate (near a high viral load sample), were common reasons for these false positive tests. The cause of the false positive test could not be determined in all cases. False positive results can be due to errors in the pre-analytic and analytic phases of testing. Pre-analytic issues include sample contamination by testing materials, improper sampling or testing technique, and improper specimen transport. These would not explain the false positive results in either the Layfield et al. [2] study or the current study because in these cases repeat analysis would remain positive. Our errors would have to have occurred during the analytic phase itself and not involve contamination of the primary specimen. A protocol was developed (Figs. 1 and 2) to identify potential false positive results within the asymptomatic group and initiate immediate retesting of those samples.

In the eight-week period between September 14, 2020 and October 31, 2020, the quality assurance protocol was in place. The specimen results were analyzed to determine the number of false positive specimens identified, the false positive rate and the causes of false positive tests.

At the time new protocol was put in place, a rigorous retraining and training program for technologists involved in COVID-19 testing was undertaken to ensure optimal specimen loading and processing techniques. Staffing changes were also made to increase the number of technologists with prior experience in a molecular laboratory division of a clinical laboratory. This training program resulted in a small decrease in the number and rate of false positive tests in the asymptomatic cohort. In the initial study, 30 of 155 positive tests (19%) were falsely positive yielding a false positive rate of 0.44% for asymptomatic patients. Following the increased training and staffing changes false positive results fell to 20 in 288 positive tests (7%). The false positive rate fell from 0.44% to 0.3%. Review of cycle threshold curves, heat maps to demonstrate possible carry over or cross contamination of specimen wells and test plates and technologists’ records revealed that specimen carry over, mispositioning of test plates along with erroneous automated interpretation of RT-PCR curves were recognizable causes of false positive RT-PCR results. In our testing facility, a false positive rate of approximately 0.3% has so far persisted despite rigorous training and quality control programs. Improved training would not impact the number or rate of errors due to cross contamination of specimens from adjacent high viral load sample wells nor would it have impacted erroneous interpretations by the software supplied with the testing equipment. Only ten of twenty (50%) false positive results were due to technologist error. Thus, improved training may have had significant impact on technologist performance. Our study demonstrated only the false positive rate in asymptomatic apparently unexposed individuals. The false positive rate in symptomatic and/or exposed individuals may differ and indeed may be impacted by the overall positive rate in the symptomatic or exposed population. In a recent study by Lin et al. [7] a false positive rate of 0.1% was reported. Their study was similar to ours with technical errors being the major cause of false positive results. We believe that a quality control program retesting all positive results in asymptomatic patients represents good practice to avoid the negative impact of false positive diagnoses in this subset of the patient population. Additionally, problems in testing identified in the asymptomatic cohort and processes to eliminate these “poor” practices should also result in a decrease in the false positive rate for all samples undergoing RT-PCR testing for COVID-19.

The current study has some weaknesses potentially limiting its universal applicability. First, the testing was performed when COVID-19 variants were relatively unimportant. Subsequently, a number of variants have developed which have different characteristics. Most notably, the Delta variant of COVID-19 appears to have increased infectivity. Despite changes in infectivity, this should have little impact on the occurrence of false positive diagnoses. The three probes utilized during our study are still the probes utilized in current RT-PCR testing. Genetic changes in the virus would more likely lead to increased numbers of false negatives rather than increasing numbers or percentages of false positives. Second, the testing was performed at a single site which may have technical characteristics not universally applicable to RT-PCR testing at other laboratories. The two instruments used are among the most common platforms for RT-PCR testing and hence issues with instrumentation as demonstrated in the present study are probably widely applicable. In support of this belief is the fact that 20% of our false positive diagnoses were due to cross contamination from high viral load specimens in nearby wells. This problem had been recognized by the Food and Drug Administration and was the subject of a letter issued by the FDA [8]. The same problem was also recognized by Lin et al. [7]. Similarly, incorrect automated curve interpretation was found to be an issue in 30% of false positive diagnoses. This automated system for curve interpretation was supplied by the manufacturers and is most likely an issue experienced at other laboratories. The training and experience of the laboratory technicians performing the testing might also vary from other testing centers. A high portion of our laboratory staff were certified, and the laboratory had recently undergone College of American Pathologists (CAP) inspection and certification. Thus, this would be unlikely to explain the issues with false positives. Moreover, our false positive rate is similar to that reported in other studies [6], [7], [9]. Based on this, the authors believe that training and overall competency of laboratory staff is not dissimilar from that characteristic of other testing laboratories. Finally, a number of RT-PCR testing instruments are currently available on the market. We use the instruments supplied by Thermo-Fisher and Siemens. These both use 96 well plates but do have slightly different instrument designs. We did not find a difference in false positive rates between the two instruments. Other instruments are available such as the Panther Fusion System by Hologic (Marlborough, MA). The design of the Panther Fusion System is different than the 96 well plates used by Thermo-Fisher and the Siemens systems. Hence, rates for false positive diagnoses as well as causes for false positive diagnoses may differ. The present study has not addressed this issue. The authors believe that these characteristics of our study do not negate the utility or importance of our findings, nor do they significantly impact the universality of our findings regarding issues for false positive results.

Author statement

Dr. Lester J. Layfield designed the study, helped collect the data, analyzed the data and wrote the manuscript. Simone Camp analyzed the RT-PCR curves for accuracy, collected data and reviewed and edited the manuscript. Dr. Bowers helped collect the data. Dr. Douglas C. Miller reviewed and edited the manuscript.

Conflict of interest

The authors have no conflicts of interest to report in relation to this manuscript.