Skip to main content
. 2021 Mar 24;2021(3):CD013705. doi: 10.1002/14651858.CD013705.pub2

Loeffelholz 2020.

Study characteristics
Patient Sampling Two‐group study to estimate sensitivity and specificity for diagnosis of active disease
‐ suspected patients referred for COVID‐19 testing at 7 sites according to the local criteria (n = 486); sampled to enrich for RT‐PCR‐positive specimens (not further described)
Recruitment: convenience (in addition, 1 site (LAC+USC) tested specimens from a 4‐day point prevalence survey of patients presenting with COVID‐19 symptoms)
Prospective or retrospective: retrospective
Number of samples (samples with confirmed SARS‐CoV‐2): 486 (220)
Patient characteristics and setting Setting: not stated
Location: 7 sites:
Johns Hopkins University, Baltimore;
LAC+USC Medical Centre, University of Southern California, Los Angeles;
Manchester University NHS Foundation Trust Manchester;
Mondor Hospital, Paris;
New York City Dept. Health and Mental Hygiene, NYC;
Niguarda Hospital, Milan;
University Hospital, Newark.
Country: USA, UK, France, Italy
Dates: 1 March‐2 April 2020
Symptoms and severity: not stated
Demographics: adults at all sites except New York City Dept. Health and Mental Hygiene and Niguarda Hospital where all age groups were tested (ages not stated)
Exposure history: not stated
Index tests Test name: Cepheid Xpert Xpress SARS‐CoV‐2 (RUO version, no product code reported)
Manufacturer: Cepheid Europe
Antigen target: nucleocapsid gene (N2) and the envelope gene (E) (RUO version also detects RdRp gene but this does not contribute to definition of positive)
Antibody: N/A
Test method: automated point‐of‐care PCR
Samples used: swabs (NP (n = 339), OP (n = 15), combined NP/OP in the same transport vial (n = 97)), and TA (n = 30):

  1. Baltimore ‐ 61 NP

  2. Los Angeles ‐ 88 NP

  3. Manchester ‐ 54 NP/OP, 11 NP

  4. Paris ‐ 68 NP

  5. NYC ‐ NP 11, OP 15, TA 30, NP/OP 43

  6. Milan ‐ 79 NP

  7. Newark ‐ 21 NP


Transport media: VTM (swabs), diluted in saline (TA). 1 site (Manchester) pretreated specimens with an equal volume (≥ 30‐< 50% (w/w)) of a guanidine hydrochloride buffer and heated at 80 °C
Sample storage: stored at −80 °C prior to index test, except at 1 site (University Hospital, Newark) where specimens were tested in real time, within 2 h by the Xpert test (n = 21).
Test operator: not stated; presume laboratory staff
Definition of test positivity: as per manufacturer: if both targets are detected, or if only N2 is detected, the test reports a positive result. If only the E target is detected the test reports a presumptive positive result "because this target is shared among some members of the sarbecovirus subgenus of coronaviruses". The RUO version of the test shows the amplification curves and PCR cycle threshold for all 3 genetic targets. The study reports that "The EUA test version cartridge contains the same reagents as the RUO cartridge. The only difference between the tests is the software which in the EUA version allows the user to see amplification curves and results for the N2 and E targets only".
Blinding reported: not stated
Timing of samples: not stated, presume on presentation
Target condition and reference standard(s) Reference standard: RT‐PCR (sites using each kit not reported, added by review team based on number of samples per site and per RT‐PCR kit)

  1. New York SARS‐CoV‐2 Real‐time Reverse Transcriptase (RT)‐ PCR Diagnostic Panel; NYC

  2. Quest SARS‐CoV‐2 rRT‐PCR (Quest Diagnostics, San Juan Capistrano, US); Los Angeles

  3. RealStar® SARS‐CoV‐2 RT‐PCR Kit 1.0 (Altona Diagnostics, Hamburg, Germany); Baltimore and Paris

  4. GeneFinder COVID‐19 Plus RealAmp Kit (ELITechGroup, Puteaux, France); Milan

  5. Allplex 2019‐nCoV Assay (Seegene, Seoul, SK); Milan

  6. Charité Virology (Berlin, Germany) (in‐house); Manchester

  7. Abbott RealTime SARS‐CoV‐2 Assay (Abbott, Des Plaines, US); Newark

  8. Simplexa COVID‐19 Direct (DiaSorin, Cypress, US); Newark


Definition of non‐COVID cases: yes (performed prior to index test)
Genetic target(s): different targets depending on RT‐PCR test used:

  1. New York Panel; N (N1, N2)

  2. Quest; N (N1, N3)

  3. RealStar ; S, E

  4. GeneFinderTM; RdRp, E, N

  5. Allplex ; RdRp, E, N

  6. Charité Virology; RdRp

  7. Abbott RealTime ; RdRp, N

  8. Simplexa; ORF1ab, S


Tie‐breaker methods (for discrepant results), included: Hologic Panther Fusion (San Diego, USA), Tib‐Molbiol LightMix Modular Wuhan Coronavirus E‐gene RT‐PCR (Roche, Basel, Switzerland); and the CDC assay (IDT primers and probes)
Samples used: as for index test
Timing of reference standard: as for index test
Blinded to index test: no storage; tested in real time
Incorporated index test: no
Flow and timing Time interval between index and reference tests: same samples but index performed after frozen storage for undefined period of time except at University Hospital, Newark where specimens were tested in real time, within 2 h by the Xpert test
All participants received same reference standard: no
Missing data: 4 Xpert Xpress test results were lost permanently due to a single instrument computer malfunction
Uninterpretable results: 1 Xpert Xpress test was invalid due to a cartridge error (inadequate sample volume)
Indeterminate results (index test) presumptive positive results on Xpert Xpress were not reanalysed by Xpert Xpress, but all discrepant results were reanalysed by a third RT‐PCR method
Indeterminate results (reference standard): specimens with inconclusive results by a test, and those with discrepant results between Xpert and the RT‐PCR tests were analysed by a third RT‐PCR method
1 FN result was inconclusive on Quest SARS‐CoV‐2, and negative on CDC RT‐PCR; re‐considered as TN
Of 11 FPs (including 1 presumptive positive on Xpert Xpress), 2 were negative on both New York SARS‐CoV‐2 and Panther Fusion (remained as FPs), and 9 were negative on in‐house RT‐PCR but positive on Roche RT‐PCR (reclassified as TP)
In addition, 12 specimens (8 NP, 4 NP/OP) were inconclusive by the NY (RT)‐ PCR Diagnostic Panel and considered positive for data analysis purposes in the study. Of these, 11 were positive by the Xpert test and 1 was presumptive positive (EUA version of Xpert test). In 4 of these only the N1 target was detected and in 8 only the N2 target was detected by the New York EUA method, all with Ct values > 36
One NP specimen was inconclusive by the Quest SARS‐CoV‐2 rRT‐PCR test and negative by the Xpert test. The Quest test reports inconclusive if only a single target (N1 or N3) is detected. They were unable to determine which target was detected by the Quest test. This specimen was negative by a tie‐breaker NAAT.
Unit of analysis: not stated; only samples reported
Comparative  
Notes Funding: not stated; presume funded by test manufacturer (see COI statement)
Publication status: accepted manuscript
Source: Journal of Clinical Microbiolobyogy
Author COI: the study was designed and supervised by the sponsor, Cepheid. Data were collected by investigators at each study site, and statistical analyses were performed by a Cepheid author. Cepheid authors wrote the first draft of the manuscript. All study authors vouch for the accuracy and completeness of the data reported.
Methodological quality
Item Authors' judgement Risk of bias Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled? No    
Was a case‐control design avoided? No    
Did the study avoid inappropriate exclusions? Unclear    
Did the study avoid inappropriate inclusions? Yes    
Could the selection of patients have introduced bias?   High risk  
Are there concerns that the included patients and setting do not match the review question?     High
DOMAIN 2: Index Test (Antigen tests)
DOMAIN 2: Index Test (Rapid molecular tests)
Were the index test results interpreted without knowledge of the results of the reference standard? Unclear    
If a threshold was used, was it pre‐specified? Yes    
Could the conduct or interpretation of the index test have introduced bias?   Unclear risk  
Are there concerns that the index test, its conduct, or interpretation differ from the review question?     Unclear
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition? No    
Were the reference standard results interpreted without knowledge of the results of the index tests? Yes    
Reference standard does not incorporate result of index test? Yes    
Could the reference standard, its conduct, or its interpretation have introduced bias?   High risk  
Are there concerns that the target condition as defined by the reference standard does not match the question?     High
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard? Yes    
Did all patients receive the same reference standard? Yes    
Were all patients included in the analysis? No    
Did all participants receive a reference standard? Yes    
Were results presented per patient? Yes    
Could the patient flow have introduced bias?   High risk