Clinical evaluation of the Procleix SARS-CoV-2 assay, a sensitive, high-throughput test that runs on an automated system

Abstract

Testing is crucial in controlling COVID-19. The Procleix® SARS-CoV-2 assay, a transcription-mediated amplification nucleic acid test that runs on an automated system, was evaluated using inactivated virus and clinical samples. The sensitivity of the assay was assessed using heat-inactivated SARS-CoV-2 and compared to 3 other tests. Clinical validation utilized 2 sets of samples: (1) Nasal, nasopharyngeal and oropharyngeal samples (n = 963) from asymptomatic individuals, and (2) nasopharyngeal samples from symptomatic patients: 100 positive and 100 negative by RT-PCR. The Procleix assay had greater sensitivity (3-fold to 100-fold) than the comparators and had high specificity (100%) in asymptomatic subjects. In symptomatic patients, the Procleix assay detected 100% of PCR-positives and found 24 positives in the initial PCR-negatives. Eighteen of these were confirmed positive and 6 were inconclusive. These studies showed that the Procleix SARS-CoV-2 assay was a sensitive and specific tool for detecting COVID-19.

1. Introduction

The SARS-CoV-2 virus was identified as the cause of a pneumonia outbreak in Wuhan, China in late 2019 (Gorbalenya et al., 2020; Huang et al., 2020). In March 2020, COVID-19, the disease caused by SARS-CoV-2 infection, was declared a worldwide pandemic by the World Health Organization (World Health Organization, 2021). The rapid spread of this previously unknown virus created a critical need for novel therapeutic agents and testing strategies for SARS-CoV-2. Sensitive, specific testing is a crucial component of controlling the pandemic during vaccine rollout and pending identification of effective drug therapies (Del Rio and Malani, 2020; Paules et al., 2020; Tang et al., 2020).

Different types of tests for SARS-CoV-2 infection have been developed since the virus was discovered. These include those based on reverse transcriptase polymerase chain reaction (RT-PCR), transcription-mediated amplification (TMA), and antigen and antibody detection tests (MacKay et al., 2020; Resilience Health, 2020).

The Procleix SARS-CoV-2 assay, a TMA assay, runs on the fully automated Procleix Panther system. The Panther system generates the first test results in 3.5 hours, and then 5 results every 5 minutes. The system is capable of testing of about 1000 samples in 24 hours. This type assay could be crucial to testing, tracking and tracing strategies - essential public health activities to help control the spread of COVID-19, especially in the absence of widespread vaccination and proven therapeutics. Additionally, high-throughput testing will allow the rapid detection of positive samples that can be further analyzed to assess the spread of SARS-CoV-2 variants. The studies described in this paper were designed to assess the sensitivity and specificity of this automated assay system.

2. Materials and methods

2.1. Preparation of heat-inactivated SARS-CoV-2 virus panel

Heat-Inactivated SARS-CoV-2 Virus was obtained from BEI Resources (NR-52286; Manassas, VA, USA). This material was selected as the standard for this study because it is widely available. This heat-inactivated form facilitates safer shipping and reduces the risk to laboratory personnel. Serial dilutions and a negative control were made in viral transport medium to give a final range of 0 to 2000 copies/mL (0, 2, 6, 20, 60, 200, 600 and 2000 copies/ml). The panel was stored and shipped frozen and provided to laboratories in a blinded manner. The panel was tested using the Procleix SARS-CoV-2 Assay (which targets the nucleocapsid protein (N-) gene) on the Procleix Panther System (Grifols Diagnostic Solutions Inc, San Diego, USA). See Appendix A; Table A.1 for a summary of all tests used in this study.

On this instrument, results obtained from the negative and positive calibrators are used to determine the validity of the run and to establish the assay cutoff values for the internal control signal and the analyte signal. Reactive or nonreactive results for each specimen tested are provided by the instrument based on predetermined calculations of the cutoff values that are coded within the assay software.

The results from the Procleix assay were compared to 3 commercial assays and an in-house specificity TMA assay (n = 6 for all assays: see Appendix A; Table A.1 for assay summaries). The commercial assays were the Allplex™ 2019-nCOV Assay (Seegene, Seoul, Korea: polymerase chain reaction assay (PCR) targeting envelope protein- (E-), RNA-dependent RNA polymerase- (RdRP-), and N- genes), the Fast Track Diagnostic™ (FTD) SARS-CoV-2 Assay (Siemens Healthineers, Erlangen, Germany: PCR targeting open reading frame 1a/b- (ORF1a/b-) N- and spike protein (S-) genes) and the Viasure SARS-CoV-2 Assay (CerTest Biotec, Zaragosa, Spain: PCR targeting ORF1a/b- and N-genes). The in-house confirmatory assay (SARS-CoV-2 Transcription-Mediated Amplification (TMA) Confirmatory Assay) targets the S-gene.

2.2. Prospective testing of samples from asymptomatic individuals

For this prospective study, samples were collected from 127 asymptomatic volunteers who provided written informed consent to be tested for COVID-19 (Fig. 1). To qualify for the study, volunteers were required not be presenting any of the symptoms of COVID-19. Specifically, volunteers were required be free of the following symptoms: fever, dry cough, sore throat, trouble breathing or a loss of taste and smell (based on the European Center for Disease Prevention and Control case definition) (European Center for Disease Prevention and Control, 2020). In addition to lacking any of these symptoms, volunteers were required to not have been in direct contact with anyone in the last 2 weeks who had PCR-confirmed SARS-CoV-2 infection. Volunteers were also checked for the development of symptoms at least 1 week after the initial sample collection. Volunteers testing negative on their first test were asked to provide additional samples in the following weeks. The study was approved by the corresponding institutional review board and followed all applicable regulations and guidelines. Samples were collected between April 2020 and September 2020.

Fig. 1.

Flow diagram for samples from asymptomatic and symptomatic individuals.

Nasopharyngeal, nasal and oropharyngeal samples were collected using flocked swabs in viral transport medium (VTM): UTM® universal transport medium (Copan, Carlsbad, CA, USA), VICUM® virus medium (Deltalab Barcelona, Spain). Samples were stored according to the assay manufacturer's instructions.

If a sample gave a positive result on the initial test, the sample was re-tested with an in-house confirmatory assay based on N1 primers published by the US Centers for Disease Control and Prevention, 2020 purchased from Integrated DNA Technologies, Inc (Coralville, Iowa). Briefly, RNA was extracted from 200µL of respiratory specimen using QIAamp Viral RNA Mini Kit (Qiagen, Hilden, Germany) and was eluted in 25 µL. Twelve (12) µL of the eluate were used as template for the quantitative reverse transcription polymerase chain reaction (RT-qPCR) carried out with the TaqPath™ 1-Step RT-qPCR Master Mix (ThermoFisher, Waltham, MA, USA: PCR targeting the ORF1a/b-, N- and S-genes; see Appendix A: Table A.1 for assay summaries) using ABI7500 RT-qPCR equipment (ThermoFisher). A positive initial result followed by a positive result in any of the confirmatory assays was considered a confirmed positive result.

2.3. Retrospective testing of samples collected from symptomatic patients

The sample panel investigated in this study consisted of 200 nasopharyngeal and oropharyngeal samples collected (in the VTM described above) from patients at the Vall d'Hebrón Hospital (Barcelona, Spain). The study was approved by the Ethics Committee of Vall d'Hebrón University Hospital (Fig. 1). These samples were initially tested for SARS-CoV-2 by RT-PCR at the hospital. RT-PCR assays used were the Allplex™ 2019-nCOV Assay and the Cobas® SARS-CoV-2 Test (Roche Diagnostics, Basel, Switzerland: PCR targeting ORF1a/b- and E-genes; see Appendix A; Table A.1 for assay summaries). The 200 samples were collected between March 21, 2020 and April 15, 2020. These samples were collected from symptomatic patients at the Vall d'Hebron Hospital (or other centers attached to the hospital) presenting with symptoms that were compatible with COVID-19 and were, therefore, suspected of having the infection.

The samples were randomly selected without specific clinical criteria with the stipulation that 100 were positive and 100 were negative. Samples were later sent to Banc de Sang i Teixits, Servei Català de la Salut (Blood and Tissue Bank, Catalan Health Service), Barcelona, Spain to be tested using the Procleix SARS-CoV-2 assay on the Procleix Panther System (see section 2.4).

Results from the Procleix assay were compared with results from the original RT-PCR assay. When results from the Procleix assay were not in agreement with the original RT-PCR assay, the samples were re-tested. Re-testing was conducted using EasyMag Extraction (BioMerieux, Marcy l'Etoile, France) followed by the TaqMan™ 2019-nCoV Assay Kit v2 (ThermoFisher, Waltham, MA, USA). Samples that gave discordant results in the first re-test assay were subjected to a second re-test using an in-house confirmatory assay (SARS-CoV-2 TMA Confirmatory Assay).

For the Taqman assay, a result was considered positive if the cycle threshold (Ct) < 37. If Ct 37 ≤ Ct < 40, the result was considered inconclusive and the test should be repeated. If the Ct = 40 or was undetermined and the control (RNAseP assay) Ct < 40, the result was considered negative (ThermoFisher, 2019).

2.4. Sample analysis in the procleix panther system

All the samples from the sources described above were analyzed using the Procleix Panther System. Samples were homogenized by brief vortex mixing prior to analysis. Each bar-coded sample tube for analysis in the system contained 500 µL of Procleix specimen extraction buffer (SEB) and 500 µL of sample. Samples were gently mixed by pipetting (10X) while avoiding bubble formation. Prepared samples were processed on the Procleix Panther System according to the SARS-CoV-2 package insert (Grifols Diagnostics, 2020) and the Procleix Panther System operator's manual.

For the Procleix SARS-CoV-2 assay, samples were considered nonreactive if the signal/cutoff ratio (S/CO) was less than 1.0 and the internal control signal was above its cutoff value. Samples were considered reactive if the S/CO was greater than 1.0 and the signal was within the limits of the system. Additional testing was recommended when a specimen gave a reactive result and the S/CO was greater than 1.0 but less than 2.0 (Grifols Diagnostics, 2020). The units of measurement in this assay, relative light units (RLU), do not correspond to viral load in contrast to Ct values from RT-PCR assays.

2.5. Samples analyzed using other SARS-CoV-2 assays

Samples analyzed with the assay systems produced by other manufacturers were testing according to each individual manufacturer's instructions.

2.6. Data availability

All the relevant data that support the findings of this study are available within the article. Complementary data can be available from the corresponding author upon a reasonable request.

3. Results

3.1. Detection of heat-inactivated SARS-CoV-2

The Procleix SARS-CoV-2 assay was used to test heat-inactivated SARS-CoV-2 virus at over a range of 0 to 2000 copies/mL (Table 1 ). The Allplex 2019-nCoV, FTD SARS-CoV-2, Viasure SARS-CoV-2 and in-house TMA confirmatory assay were used to test the same panel. Results were reported as the number and percent positive results out of 6 replicates. The Procleix assay and the in-house confirmatory assay were the most sensitive showing 100% positivity at 20 copies/mL. In this study, the other assays were 3-fold to 100-fold less sensitive than the Procleix assay. The Procleix assay was 3-fold more sensitive than the FTD assay and 100-fold more sensitive than the Allplex assay. One false positive was also observed in this study. The Viasure assay gave a single positive response (n = 6) for the negative control (0 copies of the virus).

Table 1.

Percent reactivity results for the detection of heat-inactivated SARS-CoV-2 virus (NR-52286 from BEI resources, manassas, VA), diluted in viral transport medium (VTM) and provided blinded to testing laboratories.

	# Positive / # tested (% positive)
Copies / mL	Allplex™ 2019-nCOV assaya / Ct averageb	FTD™ SARS-CoV-2 / Ct averageb	Procleix® SARS-CoV-2 assay	SARS-CoV-2 TMA confirmatory assay	VIASURE SARS-CoV-2a/ Ct averageb
2000	6/6 (100%) / 36.6	6/6 (100%) / 32.2	6/6 (100%)	6/6 (100%)	6/6 (100%) /35.6
600	4/6 (67%) / 37.6	6/6 (100%) / 33.5	6/6 (100%)	6/6 (100%)	6/6 (100%) / 37.1
200	3/6 (50%) / 38.2	6/6 (100%) / 35.1	6/6 (100%)	6/6 (100%)	5/6 (83%) /38.5
60	0/6 (0%)	6/6 (100%) / 36.6	6/6 (100%)	6/6 (100%)	4/6 (67%) /38.5
20	0/6 (0%)	3/6 (50%) / 37.6	6/6 (100%)	6/6 (100%)	0/6 (0%)
6	0/6 (0%)	0/6 (0%)	3/6 (50%)	4/6 (67%)	1/6 (17%) /39.5
2	0/6 (0%)	0/6 (0%)	1/6 (17%)	1/6 (17%)	0/6 (0%)
0	0/6 (0%)	0/6 (0%)	0/6 (0%)	0/6 (0%)	1/6 (17%) / 38.6

^aConsidered positive if 1 specific target of SARS-CoV-2 genome was detected.

^bAverage of all cycle thresholds (Ct) obtained for this concentration. No Ct is obtainable for the TMA assays as they are isothermal.

3.2. Prospective testing of samples from asymptomatic volunteers

Testing of 963 nasal, nasopharyngeal and oropharyngeal swab samples from 127 asymptomatic individuals, gave 956 samples nonreactive results and 7 reactive results (Table 2 ). Five of the 7 initially reactive samples had a signal to cutoff ratio ≥1 and ≤2. These initially reactive samples were re-tested in duplicate, according to the manufacturer instructions, and found to be nonreactive. The remaining 2 samples, (signal/cutoff ratio 4.0 and 3.8) were tested by RT-PCR and confirmed to be positive. In this study, the initial specificity of the was determined to be 99.5% (95% CI 98.8%–99.8%) and 100% (95% CI 99.60%–100%) after re-test and/or confirmation.

Table 2.

Summary of results from testing 127 asymptomatic individuals with the procleix SARS-CoV-2 assay.

# Specimen types (Transport medium)	# Specimen tested	# Initial reactive	# Confirmed positive	# Nonreactive	% Specificity (95% CI)
Nasal Swabs (VTM)	213	2a	0	213	nd
Nasal Swabs (ITM)	122	0	0	122	nd
NP Swabs (VTM)	332	2	2b	330	nd
NP Swabs (ITM)	127	0	0	127	nd
OP Swabs (VTM)	169	3a	0	169	nd
Total	963	7	2	961	100 (99.60–100)

CI = SCORE confidence interval; nd = not determined; VTM = viral transport medium; ITM = inactivation transport medium; NP = nasopharyngeal; OP = oropharyngeal.

^aFive samples had initial S/CO ≥1 and ≤2, samples were re-tested in duplicate according to recommendation from instruction for use and were nonreactive, initial specificity was 99.5% (95% CI: 98.8%–99.8%).

^bConfirmed positive by repeat testing and by a RT-PCR assay using same primers and probes as N1 assay published by CDC (https://www.cdc.gov/coronavirus/2019-ncov/lab/rt-pcr-panel-primer-probes.html).

3.3. Retrospective testing of sample panel from 200 symptomatic patients

A sample panel from 200 patients at Vall d'Hebron Hospital was also tested with the Procleix SARS-CoV-2 assay (Table 3 ). This panel contained 100 positive and 100 negative samples as determined previously by RT-PCR (Cobas® SARS-CoV-2 and Allpex™ 2019-nCoV assays). For the samples shown to be positive by the initial RT-PCR test, all 100 were reactive with the Procleix assay. For the samples determined to be negative with the initial RT-PCR assay, 99 yielded a valid result with the Procleix assay and 1 sample was invalid and not re-tested. Twenty-four of the samples that were found negative in the initial RT-PCR assay were found to be reactive with the Procleix assay. These samples were re-tested as described above and results are in Table 4 . The re-testing confirmed that 18 of these 24 samples were indeed positive for SARS-CoV-2 by at least 1 confirmatory assay. Of these 18 samples, 11 were confirmed positive using the Taqman 2019-nCoV assay kit (version 2).

Table 3.

Summary of clinical results from 200 symptomatic individuals tested with the procleix SARS-CoV-2 assay.

Initial specimen statusa	# Specimen tested	# Valid	# Nonreactive	# Reactive	# Confirmed positive
Positive	100	100	0	100	N/A
Negative	100	99	75	24	18b

N/A = not applicable.

^aStatus determined by Cobas® SARS-CoV-2 Test (Roche) or Allplex™ 2019-nCoV Assay (Seegene).

^bConfirmed positive by TaqMan™ 2019-nCoV Assay Kit v2 (ThermoFisher) or using a SARS-CoV-2 TMA confirmatory assay.

Table 4.

Analysis of discrepant results between initial PCR assay and procleix assay.

Initial assay status	Procleix SARS-CoV-2 assay result	TaqMan™ 2019-nCoV assay kit v2 result	SARS-CoV-2 confirmatory TMA assay result	Final status	N
Negative	Reactive	Positive	NT	Confirmed positive	11
Negative	Reactive	Inconclusivea	NT	Confirmed positive	5
Negative	Reactive	Negative	Reactive	Confirmed positive	2
Negative	Reactive	Negative	Nonreactive	Undeterminedb	6

NT = not tested.

^aOnly 1 SARS-CoV-2 gene detected (N, ORF1a/b, S).

^bQuantities not sufficient for additional testing.

Five samples gave inconclusive results in the Taqman assay but because they were reactive in the Procleix SARS-CoV-2 assay, they were considered confirmed positive. Two samples were negative in the Taqman assay but were reactive in the in-house-confirmatory TMA assay. These samples were also counted as confirmed positive. The remaining 6 samples were negative in the Taqman assay as well as the in-house-confirmatory TMA assay and insufficient sample volume remained for additional testing.

This resulted in 92% agreement for the negative samples between the Procleix assay and the samples that tested negative for any RT-PCR assay or confirmatory TMA. When combined with the testing for the positive samples this yielded an overall agreement of 97% between the Procleix assay and the initial RT-PCR tests.

4. Discussion

For a COVID-19 diagnostic assay to be most useful in the current environment of the COVID-19 global pandemic, it must be proven to be reliable, specific and sensitive. Greater diagnostic and public health value can be associated with assays that are higher throughput, random access, (not batched), and more user friendly. The studies described in this paper have shown that the automated Procleix SARS-CoV-2 assay is sensitive and specific which could make it a valuable diagnostic tool for controlling COVID-19.

The Procleix Panther system and assays developed for this system were created for screening of donated blood, plasma, tissues and organs. Monoplex and multiplex assays have been developed to detect several transfusion-transmissible pathogens such as HIV-1, HIV-2, hepatitis B virus, hepatitis C virus, hepatitis E virus, West Nile virus, Dengue virus, Zika virus and parasites such as Babesia. These systems are highly specific and sensitive. They have been in use for blood screening for several years (initial CE-IVD marking in 2012).

In the current studies, the Procleix SARS-CoV-2 assay was shown to be more sensitive (3-fold to 100-fold) than comparator assays when tested against heat-inactivated SARS-CoV-2 virus. This greater sensitivity may allow pooling of samples when prevalence is low enough (Chong et al., 2020) which could further increase the throughput of this assay and laboratory capabilities. Testing of pooled plasma samples (6–16 samples/pool) demonstrated the ability of the Procleix SARS-CoV-2 assay to detect a small number of reactive samples with low levels of SARS-CoV-2 RNA in pooled plasma samples (Bakkour et al., 2020). Testing of pooled nonreactive respiratory samples spiked with known amounts of inactivated virus (8 samples/pool, which is estimated to be equivalent to a decrease of 3 in the Ct for PCR) demonstrated comparable assay sensitivity in pooled samples compared to single samples (Grifols Diagnostics, 2020).

The Procleix SARS-CoV-2 assay was also found to be highly specific in finding a small number (2) of positive samples among 961 samples collected from 127 asymptomatic volunteers. The initial testing was highly specific at 99.5% (95% CI: 98.8%–99.8%), but with re-testing of 5 reactive samples with low S/CO values (per the package insert instructions), these samples were found to be nonreactive and the specificity improved to 100% (95% CI: 99.60%–100%).

In testing of 200 samples from symptomatic individuals, the Procleix SARS-CoV-2 assay was able to correctly identify 100% of the samples previously found to be positive by RT-PCR. In addition, among the samples previously designated as negative by RT-PCR, 24 were found to be reactive in the Procleix assay. Of these 24 samples, 18 were found to be positive in confirmatory assays while the other 6 were not confirmed by any other method. This discrepancy may be due to the substantially higher analytical and clinical sensitivity of the Procleix assay, based on the 100% assay specificity seen in the asymptomatic population (Table 2).

When looking into test sensitivity, significant differences in the limit of detection (over 3000-fold) have been reported across molecular tests, (MacKay et al., 2020; US Food and Drug Administration, 2020) and when comparing different techniques, such as the lateral flow antigen test (US Food and Drug Administration, 2020). Test sensitivity may play a relevant role in disease control, recent evidence seems to indicate that increases in the limit of detection may lead to more false negatives, which may translate into less effective control of the pandemic (Gray et al., 2020; MacKay et al., 2020). Lateral flow antigen tests are rapid and less expensive but are much less sensitive than NAT systems and are only indicated for people in the first 5 to 7 days of COVID-19 symptoms and/or people in close contact to a confirmed positive individual (Abbott, 2021; US Centers for Medicare and Medicaid Services, 2020). For example, for the BinaxNOW Ag Card (Abbott Diagnostics, Scarborough, ME, USA), the manufacturer recommends that negative tests in persons with symptoms for more than 7 days be considered presumptive and may need to be followed up by a molecular assay (Abbott, 2021).

Antibody tests have the disadvantage of detecting exposure to the virus and not necessarily active infections. Consequently, antibody testing is not currently recommended for diagnosis of COVID-19, (Barakat et al., 2020; US Centers for Disease Control and Prevention, 2020) but could play a larger role in surveillance and determination of possible population immunity from prior infection and after vaccines are widely implemented (Wilson et al., 2012).

A potential disadvantage of NAT assays is the detection of SARS-CoV-2 RNA in respiratory samples after symptom resolution in the absence of replication-competent virus (Korean Disease Control and Prevention Agency, 2021; Li et al., 2020; Liu et al., 2020). This can lead to a positive result in individuals who have been recently infected but have recovered. Without determining virus infectivity, which can be a lengthy and laborious process, it can be very challenging to distinguish infectious virus from noninfectious RNA. As with most diagnostic tests, the results cannot be considered in isolation, but must be viewed in the larger context of the overall clinical picture for that individual.

Limitations of this study include the lack of sufficient sample in some instances to allow additional testing to resolve inconclusive results. Another limitation was the collection of patient samples from 1 site from patients residing in a limited geographic area. This could result in inadvertent selection of a particular variant of SARS-CoV-2 and limit the generalization of the study results.

In summary, the Procleix SARS-CoV-2 assay is a highly sensitive and specific test for the detection of SARS-CoV-2 RNA on a high-throughput, automated platform. This assay is currently in use in hospital laboratories. It represents a successful adaptation of NAT technology (widely used for screening donated blood and plasma) to create a useful tool for detection of SARS-CoV-2 in respiratory samples. With the high sensitivity of this system and potential pairing with an automated pooling instrument, it has the potential to significantly increase laboratory capacity for testing samples from asymptomatic and symptomatic populations. Based on its overall performance characteristics, the Procleix SARS-CoV-2 assay on the Panther system can play an important role in testing strategies aimed at controlling the current pandemic.

Acknowledgments

Ana Maria Ortega is acknowledged for her assistance with initiation of the study. Michael K. James, Ph.D. and Jordi Bozzo, Ph.D. are acknowledged for writing and editorial assistance.

Funding

This work was sponsored by Grifols Diagnostic Solutions, Inc.

Declaration of competing interest

LP, VB, LA-H, EE, AMM-M, AG and JML are employees of Grifols companies, including Grifols Diagnostic Solutions, Inc, which provided financial support for this study and is the manufacturer of the Procleix SARS-CoV-2 assay for the Procleix Panther System. The other authors declare that they have no conflict of interest.