FDA authorized molecular point-of-care SARS-CoV-2 tests: A critical review on principles, systems and clinical performances

Abstract

Since the start of the COVID-19 pandemic, 10 manufacturers of molecular tests for SARS-CoV-2 have received Emergency Use Authorizations from the U.S. Food and Drug Administration for point-of-care or over the counter use. In this review, the working principle of these tests is described as well as the relevant characteristics (e.g. time-to-result and specimen type). The analytical (e.g. analytical sensitivity) and clinical performance (positive and negative percent agreement) and useability characteristics (e.g. cost, reusability and throughput) of these tests are compared and critically reviewed. Also details for relevant respiratory multiplex assays of these 10 manufacturers are discussed. Critical review of scientific literature on these authorized tests revealed that for many of these tests publications are almost or completely absent, with the exception of two systems. The Xpert Xpress has been thoroughly investigated and good performance has been reported, whereas ID NOW is also well-represented in literature, but has relatively low sensitivity.

1. Introduction

In December 2019 the first case of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) was reported in Wuhan, Hubei Province, China (Lu et al., 2020). The rapid increase and spread in reported cases triggered the World Health Organization to declare COVID-19 to be a global pandemic on March 11, 2020. To date, two years later, there are already over 267.5 million confirmed COVID-19 cases and over 5.2 million deaths reported globally (World Health Organization, 2021).

A large drawback of the current reverse transcriptase polymerase chain reaction (RT-PCR) tests are the long turnaround times (about 12–48 h from making an appointment for testing at a medical centre to result), because of both the method and the logistics involved. This is unfortunate, since quickly receiving test results is an important factor in preventing the further spread of the SARS-CoV-2 virus. Therefore, it is important to develop tests that do not require testing at large centralized or near-patient labs, which can be accomplished by point-of-care testing (POCT) technology. By the use of point-of-care (POC) tests, it becomes possible to test everybody quickly, without the need for complex logistics (Ravi et al., 2020). As of December 10, 2021, 422 tests and sample collection devices are authorized by the U.S. Food and Drug Administration (FDA) under emergency use authorizations. These include 291 molecular tests and sample collection devices, 90 antibody and other immune response tests and 41 antigen tests. There are 67 molecular authorizations that can be used with home-collected samples and there are 3 molecular over the counter (OTC) at-home tests (viz. Cue COVID-19 Test, Lucira Check-It COVID-19 Test and Detect Covid-19 Test) and 1 molecular prescription at-home (Lucira COVID-19 All-In-One Test Kit) test (U.S. Food & Drug Administration, n.d.-m, U.S. Food & Drug Administration, n.d.-q).

This review focuses on molecular tests that have received Emergency Use Authorisations (EUAs) by the FDA for POC use or OTC use before April 2022. Lab-based molecular tests that have authorization for use with home-collected samples are out of scope for this review since these are not a solution for the long turnaround times and complex logistics. In addition, non-molecular tests, such as antigen tests or the recently-approved POC Breathalyzer (U.S. Food & Drug Administration, n.d.-f) are out of scope. As of December 13, 2021, a total of 10 manufacturers of molecular tests for SARS-CoV-2 have received EUAs by the FDA for POC or OTC use for in total 14 different tests. These 14 tests will be discussed in this review. At first, the principle of the singleplex test systems is discussed (section 2). The system performance, expressed in system characteristics (e.g. time-to-result and specimen type), analytical and clinical performance, can be found in section 3. 3 of the 14 authorized tests are multiplex assays, in which the detection of SARS-CoV-2 is combined with the detection of other respiratory infections. These systems are discussed in section 4. After discussing the separate systems, a comparison of the critical aspects of the systems is given, including data of clinical trials, followed by a critical discussion of the pros and cons of the various systems (section 5) and conclusions.

2. Authorized molecular POC and OTC SARS-CoV-2 tests

The WHO developed the so-called ASSURED (Affordable, Sensitive, Specific, User-friendly, Rapid and robust, Equipment-free and Deliverable to end-users) criteria, which can be applied to POC tests for all kind of infectious diseases. Commercially available SARS-CoV-2 tests (both molecular and antigen tests) are available in high throughput batch-based format (usually in combination with large instruments) as well as in single-use portable sample-to-answer format (Loeffelholz and Tang, 2021). According to the ISO 22870:2016 standard POC (or near-patient) testing is defined as testing that is performed near or at the site of a patient with the result leading to possible change in the care of the patient. In principle this can also be at-home (ISO, n.d.). The EUA authorization by the FDA distinguishes between POC use or OTC (at-home) tests. For this review only the mentioned 14 tests with such a POC or OTC EUA to detect SARS-CoV-2 have been taken into account (U.S. Food & Drug Administration, n.d.-q).

On November 20, 2020 Lucira Health Inc received EUA as first molecular SARS-CoV-2 test for home use with their prescription home-use COVID-19 test. Later on, on March 5, 2021, Cue Health received EUA as first OTC molecular SARS-CoV-2 test. Also Lucira Health Inc received this EUA for OTC (home) use on April 9, 2021 (Sagentia Innovation, 2021). In a news release of the FDA on October 29, 2021 was announced that the Detect Covid-19 test received EUA for OTC as the third diagnostic molecular test (U.S. Food & Drug Administration, n.d.-n). The test is authorized for non-prescription home use according to the authorization letter (U.S. Food & Drug Administration, 2021c). The timeline of EUA authorized SARS-CoV-2 tests is shown in Fig. 1 . In the following subsections the operation and working principle/functionality of the singleplex tests is discussed.

Fig. 1.

Timeline of the EUA authorizations for POC and OTC by the FDA of molecular tests for SARS-CoV-2.

2.1. Accula

The Accula SARS-CoV-2 test from Mesa Biotech combines RT-PCR, which is targeting the N-gene, with a lateral flow immunoassay. The Accula Dock or the Silaris Dock must be used to perform the test (Fig. 2 A). This docking station controls the reaction temperatures, timing and fluid movement within the test cassette. The nasal or mid-turbinate/throat swab must be first inserted into a buffer vial. The subject can perform self-sampling (nasal swab), but this needs to be performed under supervision. Some sample is transferred from this vial to the test cassette by using a specially designed tiny bulb pipette. The test cassette contains all reagents and is placed in the dock. After about 30 min the result can be checked visually (Mesa Biotech, 2021; Ravi et al., 2020; Yu et al., 2021). The instructions for use can also be viewed on a video on YouTube (Mesa Biotech, 2019). There is a line for the internal positive process control, one for the internal negative process control and one for the actual SARS-CoV-2 test. A blue line, even a vague one, at the SARS-CoV-2 line indicates a positive result for SARS-CoV-2. If the negative process control line becomes blue, the test is invalid (and must be performed again). The positive process control is a non-infectious RNA bacteriophage to verify RNA extraction, reverse transcriptase amplification and detection. As negative control, to exclude false positive results due to nonspecific binding, a non-SARS-CoV-2 nucleic acid probe is used. External positive (high and low SARS-CoV-2) and negative control swabs can be used to show that the Accula SARS-CoV-2 test is working properly (Mesa Biotech, 2021; Ravi et al., 2020).

Fig. 2.

A) The Accula dock from Mesa Biotech with an Accula test cassette. Picture taken from: https://15toknow.com/tests. B) The cobas SARS-CoV-2 assay tube and cobas Liat Analyzer. Picture taken from https://diagnostics.roche.com/global/en/products/instruments/cobas-liat.html.

Mesa Biotech received their letter of authorization by the FDA on January 7, 2021. Their Accula SARS-CoV-2 test is authorized for use at the POC, i.e., in patient care settings operating under a Clinical Laboratory Improvement Amendments (CLIA) Certificate of Waiver, Certificate of Compliance, or Certificate of Accreditation (U.S. Food & Drug Administration, 2021a).

2.2. BioFire

On March 17, 2021 the BioFire Respiratory Panel 2.1 (RP2.1) from BioFire Diagnostics was granted the first marketing authorization using the De Novo review pathway by the FDA. With granting of the De Novo, the EUA for the RP2.1 was revoked, which was initially authorized for emergency use in May 2020 (U.S. Food & Drug Administration, n.d.-o, U.S. Food & Drug Administration, 2021e). The BioFire RP2.1-EZ got their letter of authorization for POC on August 30, 2021 (U.S. Food & Drug Administration, 2021b).

Since the BioFire assay is solely a multiplex test, the BioFire Respiratory Panel 2.1-EZ will be discussed in more detail in Section 4.1.

2.3. Cobas

The cobas SARS-CoV-2 nucleic acid test is developed to be used on the cobas Liat Analyzer (Fig. 2B) and is a RT-PCR test, to be used in combination with a nasal, nasopharyngeal (NP) or oropharyngeal (OP) swab, or a self-collected anterior nasal (nasal) swab. The Liat Analyzer automates and integrates sample purification, nucleic acid amplification and detection of the target sequence in about 20 min. The assay is targeting the ORF1 a/b region as well as the N-gene. The collected specimen is put in 3 mL of viral transport medium (VTM) or a saline-solution. With a transfer pipette the specimen is loaded into the cobas assay tube. After scanning the barcode on the tube, the tube entry door on the cobas Liat Analyzer will open automatically, whereafter the tube can be placed inside the Liat Analyzer and a test starts (Roche Molecular Systems, n.d.-b). An instruction video can be found on YouTube (Roche Molecular Diagnostics, n.d.-b). To monitor the processes of sample purification, nucleic acid amplification and to monitor the presence of inhibitors, an internal process control is present. Also external, negative and positive, controls are available (Roche Molecular Systems, n.d.-b).

Roche Molecular Systems received their letter of authorization for use at the POC for their cobas SARS-CoV-2 Nucleic acid test for use on the cobas Liat System (cobas SARS-CoV-2) on June 17, 2021 (U.S. Food & Drug Administration, n.d.-a). Roche also received a letter of authorization for POC for their multiplex test for SARS-CoV-2 & Influenza A/B on September 14, 2020 (U.S. Food & Drug Administration, n.d.-b). This multiplex test will be further discussed in section 4.1.

2.4. Cue

The Cue COVID-19 Test from Cue Health Inc. is for home and OTC use. The test cartridge is used in combination with the Cue Cartridge Reader and the Cue Health App (Fig. 3 A). The anterior nasal sample must be collected with the Cue Sample Wand. The Cue Health App makes use of pictures and videos to guide the user through the sample collection steps and to run a test. The cartridge must be inserted into the reader before the Cue Sample Wand is inserted in the cartridge, since the cartridge needs to be pre-heated before the test. After putting the Wand in the Cue Reader, the test will start and takes about 20 min. The Cue system makes use of isothermal amplification, targeting the N-gene (Cue Health Inc, 2021; Ravi et al., 2020; Yu et al., 2021). A brief instruction video can be found on the website of Cue (Cue Health Inc, n.d.-c).

Fig. 3.

A) The Cue COVID-19 Test from Cue Health Inc. with a cartridge pouch, the reader (with a cartridge with a sample wand inserted) and the app. Picture taken from:https://www.cuehealth.com/products/how-cue-detects-covid-19/. B) Exploded view of the test cartridge and the Cue Cartridge Reader (Sagentia Innovation, 2021).

An exploded view of the Cue Cartridge Reader is given in Fig. 3B. The system makes use of valves, which are thermally actuated. Since the device is based on gravity and capillary flow no pumps are required, there is only a user activated foil piercing part, used in combination with on-board fluid storage. The detection is based on electrochemistry (Sagentia Innovation, 2021).

On 26 March 2021Cue Health Inc. received their letter of authorization for their Cue COVID-19 test for use at the POC (U.S. Food & Drug Administration, n.d.-c). Prior to this letter of authorization, they received a letter of authorization for their Cue COVID-19 test for home and OTC use on March 5, 2021 (U.S. Food & Drug Administration, n.d.-d).

2.5. Detect

The Detect Covid-19 test (Fig. 4 A) from Detect Inc. makes use of reverse transcriptase loop-mediated isothermal amplification (RT-LAMP) and lateral flow strip technology, which is targeting the ORF1ab region of the SARS-CoV-2 genome. The disposable test tube, with collection buffer, is used in combination with the Detect Hub (which needs to be set aside for about 65 min after plugging in before running the test, presumably for system stabilization). After inserting the tube in the Detect Hub, the processing (amplification) of the sample will start automatically and will take 55 min. After amplification, the tube is inserted into the reader to perform the lateral flow assay, which takes about 10 min. The Detect app on a smartphone is used for the read-out of the result. A positive processing control is present that identifies nucleic acids from a human gene to verify that the sample collection, extraction, reagent integrity and test execution are all fine (Detect Inc, 2021).

Fig. 4.

A) The Detect Covid-19 test from Detect Inc. Picture taken from: https://www.nbcconnecticut.com/news/local/qa-how-a-ct-company-is-working-to-roll-out-a-new-at-home-testing/2613221/. B) The ID NOW COVID-19 assay with the Test Base, Sample Receiver, Transfer Cartridge, swab and the ID NOW instrument. Picture taken from https://www.rapidmicrobiology.com/news/instant-results-from-abbott39s-covid-19-point-of-care-test.

While the Detect Covid-19 test is designed as a molecular home test, the IFU are only available for healthcare providers. Detect Inc. got their letter of authorization by the FDA on October 28, 2021. The test is authorized for non-prescription home use (U.S. Food & Drug Administration, 2021c).

2.6. ID NOW

The ID NOW COVID-19 assay is developed to be used in combination with the ID NOW Instrument from Abbott (Fig. 4B). The test makes use of isothermal amplification, targeting the RdRp segment and can be used with an anterior nasal (nasal), NP or throat swab and giving the result within 13 min. A Sample Receiver (with elution/lysis buffer), a Test Base (comprising two sealed reaction tubes with lyophilized pellet) and a Transfer Cartridge (for transfer of the eluted sample to the Test Base) are needed to perform a test. The reaction tubes contain the reagents for the SARS-CoV-2 test, as well as for an internal control. First the Sample Receiver and the Test Base need to be inserted in the ID NOW Instrument. The sample is added to the Sample Receiver and via the Transfer Cartridge transferred to the Test Base, which initiates amplification (Abbott Diagnostics Scarborough, n.d.; Ravi et al., 2020). On the website of Abbott demo and training videos are available (Abbott, n.d.-a).

Abbott Diagnostics Scarborough got their letter of authorization by the FDA on August 27, 2021 for use at the POC (U.S. Food & Drug Administration, n.d.-e).

2.7. Lucira

The Lucira Check-It COVID-19 Test Kit (Fig. 5 A) is developed and authorized for non-prescription home use with self-collected anterior nasal (nasal) swab specimens. The Lucira Test makes uses of RT-LAMP, targeting two non-overlapping regions of the N-gene and gives results in less than 30 min. The colorimetric reaction is based on pH change occurring during a successful amplification reaction. The swab needs to be put in the elution buffer and subsequently the sample is lysed at room temperature. The sample vial is then placed on the test unit and the eluant resolubilizes lyophilized reagents. The read-out is done using optical and electronic elements in the test unit (Lucira Health Inc, n.d.-a). A demonstration video of the Lucira COVID-19 test is available on YouTube (Lucira Health, n.d.-b). A positive internal control and a lysis internal control is present (Lucira Health Inc, n.d.-a).

Fig. 5.

A) The Lucira Check-It COVID-19 Test Kit from Lucira Health Inc. Picture taken from:https://www.lucirahealth.com/. B) Exploded view of the Lucira test unit, including sample vial (Sagentia Innovation, 2021).

An exploded view of the Lucira device – sample tube and test unit – is given in Fig. 5B. Since the complete device is a disposable, Lucira minimized the number of parts and actuators. The device makes use of gravity and capillary flow and therefore no pumps are required, only a single user-activated valve is present. There is only a single PCB present, which contains the heater and optics. All liquid used is present in the sample vial, so there is no fluid storage – which would require vapour proof seals – on the test unit itself (Sagentia Innovation, 2021).

Lucira Health Inc got their letter of authorization for non-prescription home use for the Lucira Check-it COVID-19 test by the FDA on April 9, 2021 (U.S. Food & Drug Administration, n.d.-g). Prior to that, Lucira Health Inc received their letter of authorization by the FDA on November 17, 2020 for the Lucira COVID-19 All-In-One Test Kit for prescription home use and POC use (U.S. Food & Drug Administration, n.d.-h). Based on review of the documentation of the Lucira COVID-19 All-In-One Test Kit and the Lucira Check-It COVID-19 Test Kit and the reported performances therein, it was concluded that this is in fact the same test used for different applications. Since this does not change the characteristics and performance, they are discussed as one in this review.

2.8. Talis

The Talis One COVID-19 test from Talis Biomedical Corporation makes use of reverse transcriptase isothermal amplification, targeting the N- and ORF1ab-gene. The Talis One instrument automates and integrates sample metering, nucleic acid purification, nucleic acid amplification and detection. A nasal (mid-turbinate (MT)) swab is collected and put in the tube containing collection medium. With a pipette, part of the sample is transferred to the cartridge (Fig. 6 A) and after closing the sample inlet the cartridge is put in the instrument. The test result can be read-out on the instrument after about27 min. The Talis cartridge contains two wells with identical primers and detection probes of the SARS-CoV-2 assay and the third well contains primers and detection probes of the sample processing control (human β-actin) (Talis Biomedical Corporation, n.d.-b). A video is available on the website of Talis that shows the test process (Talis Biomedical Corporation, n.d.-c). External controls are available; the positive control swab contains cultured and inactivated SARS-CoV-2 isolate and human A549 cells, while the negative control swab only contains human A549 cells. The IFU on the website of the FDA only describe the specimen collection procedure of a nasal (mid-turbinate) swab by a clinician, i.e. no self-sampling is possible (Talis Biomedical Corporation, n.d.-b).

Fig. 6.

A) The Talis One COVID-19 test cartridge from Talis Biomedical Corporation. Picture taken from: https://talisbio.com/talis-one-covid-19-test-system/. B) The Visby from Visby Medical. Picture taken from: https://www.medicaldevice-network.com/news/visby-medical-funding-flu-covid-19-pcr-test/. C) The Xpert Xpress SARS-CoV-2 test cartridge from Cepheid. Picture taken from: http://www.oucru.org/hcwscreening/.

Talis Biomedical Corporation got their letter of authorization by the FDA on November 5, 2021. Their Talis One COVID-19 Test System is authorized for use at the POC, i.e. in patient care settings operating under a CLIA Certificate of Waiver, Certificate of Compliance, or Certificate of Accreditation (U.S. Food & Drug Administration, 2021d).

2.9. Visby

The Visby COVID-19 test, depicted in Fig. 6B, from Visby Medical combines RT-PCR with a lateral flow assay, targeting the N1-gene. The Visby is a disposable device that is used for sample preparation, complementary DNA production, amplification and detection. A NP, MT or nasal swab can be used stored in Universal Transport Medium (UTM)VTM and subsequently transferred to the Visby buffer tube using the Visby pipette. The inlet port is closed followed by three sequential button pushes, after which the device can be plugged in and then starts automatically. The test result can be read-out after 30 min and any shade of colour should be considered a spot and indicates a positive result for SARS-CoV-2 (Visby Medical, 2021a; Yu et al., 2021). The instructions for use can also be viewed on a video on Vimeo (Visby Medical, 2021b). A positive control spot is present on the Visby COVID-19 test to confirm that all elements in the test device are functioning properly. There is an electronic control mechanism that detects hardware, software and various user error failures. However, no further details are given. External positive and negative controls (NATrol SARS-CoV-2 external run controls by ZeptoMetrix) can be used to show that the Visby COVID-19 test is working properly (Visby Medical, 2021a).

On February 8, 2021 Visby Medical got their letter of authorization by the FDA. Their Visby medical COVID-19 test is authorized for use at the POC (U.S. Food & Drug Administration, n.d.-i).

2.10. Xpert Xpress

The Xpert Xpress SARS-CoV-2 test is developed by Cepheid and makes use of real-time PCR assays. The test must be used with the GeneExpert Instrument Systems, which automate and integrate sample preparation, nucleic acid extraction and amplification and detection of the target sequences. Besides this instrument, a computer with preloaded software is needed. As specimen a NP, OP, nasal or MT swab and/or nasal wash/aspirate can be used. After collection, the specimen is placed in a tube containing transport medium or saline. A transfer pipette is used to transfer the sample to the sample chamber of the cartridge (Fig. 6C), which is then loaded into the instrument, where sample work-up, amplification and detection take place (Cepheid, n.d.-e). This process can also be viewed on a video on YouTube (CepheidNews, n.d.). A sampling processing control and a probe check control are also present in the cartridge. The sampling processing control checks adequate processing of the sample and is used to monitor if there are potential inhibitors present in the reaction. It also ensures that the reaction conditions (temperature and time) are appropriate and that the reagents are functional. The probe check control verifies reagent rehydration, PCR tube filling and confirms that all reaction components are present in the cartridge and is used to monitor probe integrity and dye stability (Cepheid, n.d.-e).

Cepheid got their letter of authorization by the FDA on January 7, 2021. Their Xpert Xpress SARS-CoV-2 test is authorized for use at the POC, i.e. in patient care settings operating under a CLIA Certificate of Waiver, Certificate of Compliance, or Certificate of Accreditation (U.S. Food & Drug Administration, n.d.-l). In addition, the Xpert Xpress SARS-CoV-2/FLU/RSV and the Xpert Xpress CoV-2/Flu/RSV plus assays got authorized on October 1, 2020 and September 10, 2021, respectively (U.S. Food & Drug Administration, n.d.-j, U.S. Food & Drug Administration, n.d.-k). These multiplex tests will be further discussed in section 4.1.

3. System performance

3.1. System characteristics

An overview of the characteristics, like specimen type, amplification principle and time-to-result, of all systems is summarized in Table 3.

Table 3.

Characteristics of tests that received EUA by the FDA for POC or OCT use to detect SARS-CoV-2.

Name	Specimen (swab type(s))	Amplification method	Target(s)	Control(s)	App	Time-to-result	Re-useable reader	Ref.
Accula (Mesa Biotech)	Nasal or MT	RT-PCR + lateral flow	N-gene	Internal positive process control & Internal negative process control	No, visual read-out	∼30 min	Y	(Loeffelholz and Tang, 2021; Mesa Biotech, 2021; Ravi et al., 2020; Yu et al., 2021)
BioFire (BioMerieux)	NP	Nested multiplex PCR	19 targets, S-gene and M-gene	RNA process control & PCR2 control	No	∼45 min	Y	(BioMerieux, n.d.; Loeffelholz and Tang, 2021)
cobas (Roche Molecular Systems)	Nasal, NP or MT	RT-PCR	N-gene and ORF1ab	Internal process control	No	∼20 min	Y	(Loeffelholz and Tang, 2021; Roche Molecular Systems, n.d.-b)
Cue (Cue Health)	Nasal	Isothermal	N-gene	Human cellular material	Yes	∼20 min	Y	(Cue Health Inc, 2021; Donato et al., 2021; Loeffelholz and Tang, 2021; Ravi et al., 2020; Yu et al., 2021)
Detect (Detect Inc.)	Nasal	Isothermal: RT-LAMP + lateral flow	ORF1ab	Human gene	Yes (with IFU)	>60 min	Y	(Detect Inc, 2021)
ID NOW (Abbott)	Nasal, NP or OP	Isothermal: NEAR	RdRp	Internal control	No	≤13 min	Y	(Abbott Diagnostics Scarborough, n.d.; Loeffelholz and Tang, 2021; Ravi et al., 2020)
Lucira (Lucira Health)	Nasal swab	Isothermal: RT-LAMP	2 non-overlapping N-genes	Positive IC & Lysis IC	No, but digital ‘LUCI PASS’	≤30 min	N	(Loeffelholz and Tang, 2021; Lucira Health Inc, n.d.-a)
Talis (Talis Biomedical Corporation)	MT	Isothermal: RT-LAMP	N-gene and ORF1ab	Sample processing control	No	∼27 min	Y	(Talis Biomedical Corporation, n.d.-b)
Visby (Visby Medical)	Nasal, NP or MT	RT-PCR + lateral flow	N1-gene	Positive IC	No, visual read-out	∼30 min	N	(Loeffelholz and Tang, 2021; Visby Medical, 2021a)
Xpert Xpress (Cepheid)	Nasal, NP, MT or OP	RT-PCR	N2-gene and E-gene	Sampling processing control & Probe check control	No	≤45 min	Y	(Cepheid, n.d.-e; Loeffelholz and Tang, 2021; Ravi et al., 2020)

In general, the tests can be divided into two categories based on amplification principle: those that use a polymerase chain reaction (PCR) and those that use isothermal amplification protocols. Because the SARS-CoV-2 virus is a RNA-virus, reverse transcriptase (RT) is needed in both cases to initially convert the viral RNA into DNA which can be amplified. Specificity of the amplifications are ensured by the primer set(s) used for replication, which are carefully designed for the gene(s) of interest. It is noted that information regarding the primers in the commercial tests considered in this work is proprietary.

PCR is a very sensitive amplification protocol which relies on thermal cycling. During each cycle, the DNA is melted at high temperature and the resulting single strands are replicated by DNA polymerase at a lower temperature. Of the tests discussed in this review, 5 use PCR: Accula, Biofire, Cobas, Visby and Xpert Xpress.

Isothermal amplification does not require thermal cycling. Since the DNA is not melted during isothermal amplification, special strand-displacing DNA polymerases are used, which can operate on double-stranded DNA. Loop-mediated isothermal amplification (LAMP) is the most commonly used isothermal amplification protocol in SARS-CoV-2 detection and used by Detect, Lucira and Talis. Abbott used another isothermal amplification technique, NEAR amplification, in the ID NOW. The Cue COVID-19 Test also relies on an isothermal amplification technique, but information on the protocol cannot be found.

The 3 molecular tests that have EUA authorization for OTC at-home use tests (viz. Cue COVID-19 Test, Lucira Check-It COVID-19 Test and Detect Covid-19 Test) are all used in combination with a nasal swab for self-sampling, as can be seen in Table 3. Although these 3 are all based on an isothermal amplification method, the time-to-result differs significantly, with about 20 min for the Cue test to >60 min for the Detect test. The exact duration of the Detect test is unclear, since on the website Detect claims that the result is obtained in 1 h, but also that it takes 55 min to process the sample after which one has to wait 10 min before reading the result (Detect Inc, n.d.-b). In addition, in the IFU it is mentioned that the Hub must be set aside for about 65 min and that sample processing will take 55 min (Detect Inc, 2021). Of these 3, the Lucira is the only test that does not make use of an app (with IFU), but users can verify and report the result online with ‘LUCI PASS’ (Lucira Health Inc, n.d.-b).

Both Lucira and Visby make use of a completely disposable system, while the other 8 systems have a re-useable reader. Visby and Accula combine the RT-PCR amplification with lateral flow detection, resulting in read-out by eye. Also Detect is based on later flow detection, but the read-out is via an app (which performs image analysis). The Lucira device reports the test result by use of two indicator LEDs, which can be interpreted by the user directly, or with help of an online tool. For BioFire, cobas, ID NOW, Talis and Xpert Xpress the result can be directly read from the device. Cue uses their mobile application to report the test result.

3.2. Analytical performance

An overview of the analytical performance is given in Table 4. Some of the tests are solely a singleplex test, while others are multiplex tests (e.g. respiratory panel). The (analytical) performance of these multiplex tests are also given in Table 4 and are further discussed in section 4.1. The performance in terms of inclusivity, exclusivity and interfering substances is similar amongst all tests, as can be seen in Table 4. The analytical sensitivity (LOD) varies over a relatively wide range and is sometimes hard to compare between tests, since it is not always evident how to compare the units and whether or not dilution factors (e.g. due to sample preparation steps) are taken into account (Fung et al., 2020).

Table 4.

Analytical and clinical performance of tests that received EUA by the FDA for POC or OCT use to detect SARS-CoV-2 (manufacturer info, comparator assay details not provided).

		Analytical performance				Clinical performance
Name	Specimen (swab type)	Analytical sensitivity (LOD)	Analytical Reactivity/ Inclusivity	Analytical Specificity/Exclusivity (cross-reactivity)	Analytical Specificity (interfering substances)	Number of samples	PPA	NPA	Ref.
		≥ 95% confidence	In silico homology	No cross-reaction with	No interference with		CI 95%, vs. EUA comparator assay
Singleplex assays
Accula (Mesa Biotech)	NP	150 copies/mL	100%a	32 organisms In silico: Alignment with SARS-CoV-1	25 substances	50 52	95.8%b (78.9–99.9%) 100%c (39.8–100%)	100%b (86.8–100%) 100%c (92.6–100%)	(Mesa Biotech, 2021)
cobas (Roche)	NP	12 copies/mL	>99.97%d	27 organismse In silico: Alignment with SARS-CoV-1	No info	230 207	96.1%f (89.0–98.6%) 100%g (84.5–100%)	96.8%f (92.6–98.6%) 98.9%g (96.2–99.7%)	(Roche Molecular Systems, n.d.-b)
Cue (Cue Health)	Nasal	20 genome copies/sample wand	≥98.6%	31 organisms In silico: Alignment with SARS-CoV-1	23 substances	271	97.4% (86.5–99.5%)	99.1% (96.9–99.8%)	(Cue Health Inc, 2021)
Detect (Detect Inc.)	Nasal	313 copies/swabh	98.6%	31 organisms In silico: No alignment with SARS-CoV-1i	19 substancesj	112	90.9% (76.4–96.9%)	97.5% (91.2–99.3%)	(Detect Inc, 2021)
ID NOW (Abbott)	NP	125 genome equivalents/ mL	99.32%	37 organismse In silico: No alignment with SARS-CoV-1	No info	50	100%k (83.9–100%)	100% (88.7–100%)	(Abbott Diagnostics Scarborough, n.d.)
Lucira (Lucira Health)	Nasal	2700 cp/swab	Set 1: 95.9% Set 2: 98.35%	33 organisms In silico: Alignment with SARS-CoV-1	15 substances	404	91.7% (85.6–95.8%)	98.2% (95.8–99.4%)	(Lucira Health Inc, n.d.-a)
Talis (Talis Biomedical Corporation)	MT	500 copies/mL	≥99.4%	31 organisms In silico: No alignment with SARS-CoV-1	11 substancesl	77	100% (90.8–100%)	100% (91.0–100%)	(Talis Biomedical Corporation, n.d.-b)
Visby (Visby Medical)	NP	1112 genomic copies/mL	≥95%	31 organisms In silico: Alignment with SARS-CoV-1 and Bat SARS like coronavirus	13 substances	60	100% (88.6–100%)	100% (88.6–100%)	(Visby Medical, 2021a)
Xpert Xpress (Cepheid)	NP	0.0200 PFU/mL	E-gene: 99.14% N2-gene: 97.29%	39 organismse In silico: Alignment with SARS-CoV-1 and Bat SARS like coronavirus for E-gene	No info	90	97.8% (88.4–99.6%)	95.6% (85.2–98.8%)	(Cepheid, n.d.-e)
Multiplex assaysm
BioFire (BioMerieux)	NP	500 copies/mL	>99.99%	65 organismsn In silico: Alignment with Sarbecoviruses	43 substanceso	98	98% (89.3–99.6%)	100% (92.7–100%)	(BioMerieux, n.d.)
cobas SARS-CoV-2 & Influenza A/B (Roche)	NP	12 copies/mL	>98%p	24 organismse In silico: Alignment with SARS-CoV-1	No info	285 963	100%b (93.6–100%) 95.2%c (88.3–98.1%)	100%b (98.4–100%) 99.6%c (99.0–99.9%)	(Roche Molecular Systems, n.d.-a)
Xpert Xpress SARS-CoV-2/Flu/RSV (Cepheid)	NP	131 copies/mL	E-gene: 99.5% N2-gene: 99.2%	39 organismse In silico: Alignment with SARS-CoV-1 and Bat SARS like coronavirus for E-gene	17 substances	240	97.9% (88.9–99.6%)	100% (98.1–100%)	(Cepheid, n.d.-d)
Xpert Xpress CoV-2/Flu/RSV plus (Cepheid)	NP	138 copies/mL	E-gene: 99.4% N2-gene: 96.65% RdRP-gene: 99.05%	39 organismse In silico: Alignment with SARS-CoV-1 and Bat SARS like coronavirus for E-gene	17 substances	279	100% (94.5–100%)	100% (98.2–100%)	(Cepheid, n.d.-c)

^aIn silico analysis revealed that the forward primer is predicted to be bound to the mismatch template at the annealing/extension temperatures of the assay (90.3% homology).

^bRetrospective study.

^cProspective study.

^dFor both target regions simultaneously.

^eOnly in silico analysis.

^fIndividuals suspected of COVID-19.

^gAsymptomatic individuals.

^h800 copies/mL if all virus is transferred from the swab to the buffer.

ⁱGreater than 80% homology was only apparent for a single SARS-CoV-2 primer with Pneumocystis jirovecii and two primers with Candida albicans.

^jOnly biotin, dexamethasone and flunisolide in higher concentrations showed some false negatives.

^kFor 2x LOD target concentration.

^lOnly mucin and ayr nasal gel in higher concentrations showed some false negatives.

^mOnly the values for the SARS-CoV-2 assay are given.

ⁿOff-panel organisms.

^oOnly bleach can possibly interfere.

^pThe (non-significant) mismatches on one gene had 100% perfect match on the other gene.

3.3. Clinical performance

The definition of sensitivity, as used by the FDA, is the proportion of subjects with the target condition in whom the test is positive. The specificity of the test is defined as the proportion of subjects without the target condition in whom the test is negative. This data provides information on how often a new test is correct. Be aware that these numbers are only estimates, since it is only based on a subset of subjects from the intended use population (U.S. Food & Drug Administration, n.d.-p).

The sensitivity is calculated with the following equation: TP/(TP + FN) x 100%. The specificity is calculated with the following equation: TN/(FP + TN) x 100% (Table 1 ) (U.S. Food & Drug Administration, n.d.-p).

Table 1.

Possible outcomes when comparing a new test outcome to the reference standard outcome (U.S. Food & Drug Administration, n.d.-p).

	New test
Reference test	Positive	Negative
Positive	True positive (TP)	False negative (FN)

When a new test is compared to a non-reference standard, the terms TP, FP, FN and TN are no longer valid. To characterize the diagnostic accuracy of a test in such a case, usually the PPA and NPA are reported. The data provides information on how often the new test agrees with this non-reference standard (U.S. Food & Drug Administration, n.d.-p).

The positive percent agreement (PPA) and the negative percent agreement (NPA) are calculated with the following formulas, respectively: PPA = a/(a+c) x 100% and NPA = d/(b + d) x 100% (Table 2 ). Sometimes also the overall percent agreement (OPA) is reported. This OPA is calculated with the following formula: (a+d)/(a+b + c + d) x 100% (U.S. Food & Drug Administration, n.d.-p).

Table 2.

Possible outcomes when comparing a new test outcome to the non-reference standard outcome (U.S. Food & Drug Administration, n.d.-p).

	New test
Non-reference test	Positive	Negative
Positive	a	c
Negative	b	d

The PPA and NPA values must not be confused with the positive predictive value (PPV), which is the proportion of patients that test positive who have the target condition and the negative predictive value (NPV), which is the proportion of patients with a negative test result who do not have the target condition. The PPV and NPV are calculated with the following formulas: PPV = TP/(TP + FP) x 100% and NPV = TN/(TN + FN) x 100% (U.S. Food & Drug Administration, n.d.-p).

In this review, whenever possible, the PPA and NPA are reported, since these terms are also used by the FDA to determine the clinical performance of an assay. The PPA and NPA values for the tests reviewed in section 2 and section 4.1 as determined by the manufacturer can be found in Table 4 . Table 5 gives PPA and NPA values reported in independent, peer-reviewed publications. This performance data is further discussed in section 6.

Table 5.

Clinical performance of tests that received EUA by the FDA for POC or OCT use to detect SARS-CoV-2.

Name	Specimen	Number of samples	Clinical performance		Ref.
			PPA	NPA
			CI 95%, vs. EUA comparator assay
Accula (Mesa Biotech)	NP swab	100	68.0% (53.3–80.5%)	100% (92.6–100%)	(Hogan et al., 2020)
BioFire	Currently no peer-reviewed scientific literature available
Cobas (Roche)	NP swaba	524	94.6%	97.7%	(Mahmoud et al., 2021a)
	NP swab and salivab	79	100% (97.7–100%)	100% (97.7–100%)	(Tsang et al., 2021)
	NP swab	357	100% (97.7–100%)	97.4% (94.1–98.9%)	(Hansen et al., 2021)
	NP and oral swab	100	100% (92.4–100%)	96.4% (86.0–100%)	(Daum and Fischer, 2021)
	NP, OP and nasal swab	NAc	96.3% (83.6–99.3%)	99.8% (99.1–100%)	(Ulhaq and Soraya, 2021)
Cue (Cue Health)	Nasal swab	267	91.7%	98.4%	(Donato et al., 2021)
Detect (Detect Inc.)	Currently no peer-reviewed scientific literature available
ID NOW (Abbott)	NP swab	200	80.3% (71.9–87.1%)	100% (95.4–100%)	(Moore et al., 2020)
	NP swab	61	71.7%	100%	(Mitchell and George, 2020)
	Nasal swab	974	91.30% (70–98%)	100% (82–100%)	(Tu et al., 2021)
	NP swab	395	96.2%	98.7%	(Van et al., 2021)
	NP swab	686	95.2%	96.9%	(Mahmoud et al., 2021a)
	NP swab	113	73.9% (63.2–82.3%)	100% (83.4–100%)	(Smithgall et al., 2020)
	NP swab	182	53.3% (26.6–78.7%)	100% (97.8–100%)	(Thwe and Ren, 2020)
	Quality controld	23	70.6%	100%	(Aupaix et al., 2021)
	Respiratory specimens	N/Ae	78.6% (73.7–82.8%)	99.8% (99.2–99.9%)	(Dinnes et al., 2021)
	Respiratory specimens	N/Af	73.0% (66.8–78.4%)	99.7% (98.7–99.9%)
	Respiratory specimens	N/Ag	79% (69–86%)	100% (98–100%)	(Lee and Song, 2021)
	Nasal or NP swab	239	83.3%	97.2%	(Procop et al., 2021)
	Nasal swab	88	48% (30–67%)	100% (94–100%)	(Lephart et al., 2021)
	NP swab	108	87.7% (76–95%)	100% (93–100%)	(Dinnes et al., 2021; Lee and Song, 2021; Lephart et al., 2021; Procop et al., 2021; Serei et al., 2021; Ulhaq and Soraya, 2021; Zhen et al., 2020)
	Nasal swabd	105	60%	100%	(Serei et al., 2021)
	NP and nasal swab	N/Ah	91.6% (80.5–96.6%)	94.2% (70.8–99.1%)	(Ulhaq and Soraya, 2021)
Lucira (Lucira Health)	Currently no peer-reviewed scientific literature available
Talis (Talis Biomedical Corporation)	Currently no peer-reviewed scientific literature available
Visby (Visby Medical)	NP swab	78	95.1% (86.3–99%)	100% (80.5–100%)	(Renzoni et al., 2021)
	NP swab	100	96.7%	98.6%	(Katzman et al., 2021)
Xpert Xpress (Cepheid)	Respiratory specimens	N/Ai	99.1% (97.7–99.7%)	97.9% (94.6–99.2%)	(Dinnes et al., 2021)
	Respiratory specimens	N/Aj	100% (88.1–100%)	97.2% (89.4–99.3%)
	Nasal or NP swab	238	97.6%	93.0%	(Procop et al., 2021)
	NP swab	113	98.9% (92.9–99.9%)	92.0% (72.4–98.6%)	(Smithgall et al., 2020)
	Respiratory specimens	N/Ak	99% (97–99%)	97% (95–98%)	(Lee and Song, 2021)
	NP swab	104	98.1% (90.1–100%)	100% (94.2–100%)	(Stevens et al., 2020)
	NP swabl	26	100% (75–100%)	100% (75–100%)	(Lieberman et al., 2020)
	Nasal and NP swabj	103	100% (92–100%)	98% (91–100%)	(Moran et al., 2020)
	NP or OP swab	90	100% (94.0–100%)	100% (88.6–100%)	(Jokela et al., 2020)
	OP swab	285	96.1% (91.3–98.4%)	96.2% (90.9–98.6%)	(Hou et al., 2020)
	Saliva	40	100%	100%	(Vaz et al., 2021)
	NP, OP, NP/OP swabs and tracheal aspirates	481	99.5% (97.5–99.9%)	95.8% (92.6–97.6%)	(Loeffelholz et al., 2020)
	NP swab	38	100%	100%	(Dust et al., 2020)
	Nasal swab	88	100% (87–100%)	97% (87–99%)	(Lephart et al., 2021)
	NP swab	108	98.3% (91%–100%)	100% (93%–100%)	((Dinnes et al., 2021; Lee and Song, 2021; Lephart et al., 2021; Procop et al., 2021; Serei et al., 2021; Ulhaq and Soraya, 2021; Zhen et al., 2020))
	NP and nasal swab	N/Am	95.6% (84.9–98.8%)	96.4 (77.9–99.5%)	(Ulhaq and Soraya, 2021)

^aThe cobas Liat SARS-CoV-2 & Influenza A/B assay.

^bThe cobas Liat SARS-CoV-2 & Influenza A/B assay compared to the Xpress SARS-CoV-2 assay.

^cOverall PPA and NPA based on 2 studies.

^dAnalytical performance, compared to the Xpress SARS-CoV-2 assay.

^eOverall PPA and NPA based on 12 studies.

^fOverall PPA and NPA based on 4 studies, whereby the studies were restricted to be IFU-compliant.

^gOverall PPA and NPA based on 10 studies.

^hOverall PPA and NPA based on 4 studies, among which Procop et al. and Lephart et al. (Lephart et al., 2021; Procop et al., 2021).

ⁱOverall PPA and NPA based on 13 studies.

^jOverall PPA and NPA based on 2 studies, whereby the studies were restricted to be IFU-compliant.

^kOverall PPA and NPA based on 11 studies.

^lPPA and NPA not given in the article itself, but calculated by Dinnes et al. (2021)

^mOverall PPA and NPA based on 2 studies, i.e. Procop et al. and Lephart et al. (Lephart et al., 2021; Procop et al., 2021).

4. Additional and (multiplex) respiratory assays

Some of the EUA authorized POC devices were already on the market prior to the COVID-pandemic, operating with one or more assays other than SARS-CoV-2. Other brands added SARS-CoV-2 to an existing respiratory panel, providing a multiplex assay (usually a combination of SARS-CoV-2 and Influenza A (Flu A)/Influenza B (Flu B) and Respiratory Syncytial Virus (RSV)). In the following sections, these multiplex assays, as well as systems previously described (section 2) with multiple respiratory virus tests available on the market are discussed. Finally, other tests already available for use on these devices and envisioned additions are listed.

4.1. Respiratory multiplex assays

The BioFire RP2.1-EZ Panel is a nested multiplexed PCR test with the ability to identify 15 viral and 4 bacterial respiratory pathogens in patients suspected of respiratory tract infection. This test works with a pouch (cartridge) in which the user must inject hydration solution and sample combined with buffer (Fig. 7 ). The pouch is placed into an instrument, the BioFire FilmArray, in which purification, amplification and detection take place. The entire process takes about 45 min. The pouch also contains an RNA process control and a PCR2 control. External controls are available for all viruses and bacteria detected by the RP2.1-EZ test. The test outcome depends on a melt curve analysis and analysis of replicates. The test is positive for SARS-CoV-2 if either one or both of the target(s) on the S-gene or M-gene is/are detected. The analytical and clinical performance of the BioFire RP2.1-EZ assay for SARS-CoV-2 is given in Table 4 (BioMerieux, n.d.).

Fig. 7.

The BioFire pouch from BioMerieux. Picture taken from: https://www.biomerieux-usa.com/clinical/biofire-film-array.

As mentioned in section 2.3, Roche Molecular Systems received authorization from the FDA for both their SARS-CoV-2 assay and the multiplex SARS-CoV-2 & Influenza A/B assay (U.S. Food & Drug Administration, n.d.-j, U.S. Food & Drug Administration, n.d.-k). The multiplex test is also performed in combination with the cobas Liat Analyzer using the same steps as described in section 2.3. As in the singleplex test, for SARS-CoV-2 the ORF1 a/b region and N-gene are targeted. For Flu A and Flu B, a well-conserved region of the matrix gene and the non-structural protein gene are targeted, respectively. The internal process control is present to monitor the complete process from sample purification to amplification, including the presence of inhibitors. When the SARS-CoV-2 result is positive, the results for Flu A & B should be considered presumptive, since high viral loads of SARS-CoV-2 can inhibit the amplification and detection of Flu A & B virus RNA at lower concentrations. Likewise, high concentrations of Flu B can cause inhibition of SARS-CoV-2 detection. The performance of the multiplex for SARS-CoV-2 was claimed to be similar to that for the singleplex SARS-CoV-2 assay, see also Table 4 (Roche Molecular Systems, n.d.-a).

Besides the Xpert Xpress SARS-CoV-2 test, Cepheid also offers two EUA FDA authorized multiplex tests to be used on the GeneXpert Xpress System: the Xpert Xpress SARS-CoV-2/Flu/RSV (U.S. Food & Drug Administration, n.d.-j, U.S. Food & Drug Administration, n.d.-k) and the Xpert Xpress CoV-2/Flu/RSV plus (U.S. Food & Drug Administration, n.d.-j, U.S. Food & Drug Administration, n.d.-k). The test principle and workflow are the same as described in section 2.10. The difference between these two tests is the number of genes targeted for SARS-CoV-2: both tests target the E− and N2 genes, the Xpert Xpress CoV-2/Flu/RSV plus additionally targets the RdRp gene to increase sensitivity (Cepheid, n.d.-b). The system characteristics of these tests can be found in Table 4. Some competitive interference was observed for higher concentrations of the interfering strain (see (Cepheid, n.d.-c, Cepheid., n.d.-d) for more details).

4.2. Non-SARS-CoV-2 respiratory assays

In addition to the multiplex assay, Roche has a menu of tests for the cobas Liat System, such as Influenza A/B & RSV and Strep A (Roche Molecular Diagnostics, n.d.-a). Two other systems cannot (yet) be used for multiplex assays, but do offer separate tests for a number of respiratory viruses in addition to the SARS-CoV-2 test. The Accula system can be used for three separate PCR-based tests, the original two being Flu A/Flu B and RSV (Mesa Biotech, n.d.). Abbott offers molecular assays for the detection of Flu A/Flu B 2, Strep A 2 and RSV that can be used on the ID NOW (Abbott, n.d.-b).

4.3. Additional and future assays

Visby Medical already brought a sexual health test for women on the market, which is CLIA-waved and designed for the point-of-care use. Their SARS-CoV-2 test makes use of the same (disposable) device (Visby Medical, n.d.). For three of the systems with re-useable devices that currently only offer SARS-CoV-2 tests, there are intentions to expand the number of available tests. On their website, Cue Health states that tests for, among others, respiratory health, sexual health and cardiometabolics are under development (Cue Health Inc, n.d.-a). Detect Inc. is aiming for at-home tests for sexually transmitted infections and Strep Throat that can be run in the Detect Hub (Detect Inc, n.d.-b). In fact, Talis is currently working on expanding their test menu with a COVID-19/Influenza A/B multiplex assay and tests that are related to sexual and women's health (Talis Biomedical Corporation, n.d.-a). On their website, Lucira Health only mentions a SARS-CoV-2 assay that can be used in combination with a disposable reader and no future products are introduced (Lucira Health Inc, n.d.-b).

5. Clinical performance of authorized molecular POC and OTC SARS-CoV-2 tests

In this section, scientific reports about the previously discussed POC tests (section 2), as well as clinical trials concerning these devices are reviewed. The PPA and NPA values reported in these scientific reports are given in Table 5. These PPA and NPA values will be compared to the PPA and NPA stated in the IFUs of these tests (Table 4) in the discussion (section 6). Studies that only tested confirmed COVID-19 patients (Broder et al., 2020; Rhoads et al., 2020; Stokes et al., 2021) were excluded, since in that case no NPA can be calculated. Studies with only symptomatic or clinically suspected COVID-19 subjects (Cradic et al., 2020; Deslandes et al., 2021; Goldenberger et al., 2020; Jian et al., 2021; Serei et al., 2021; Wong et al., 2020; Zhen et al., 2020) were excluded as well (or used sparingly), because using only suspected cases might lead to bias, since a high number of positives is expected. Furthermore, studies that used different specimen types for the comparator assay and the tested EUA assay (e.g. nasal and NP swabs) (Basu et al., 2020; Harrington et al., 2020) were excluded, as were studies that made use of pooling (Berke et al., 2021), that did not report a clear PPA and NPA and/or used an improper reference standard (Basu et al., 2020; Jian et al., 2021; Jin et al., 2020), or that have not been peer reviewed (yet) (Chheda et al., 2021; Ghofrani et al., 2020; Kortüm et al., 2021; Leong et al., 2021; Mahmoud et al., 2021b; SoRelle et al., 2020).

In general, scientific literature about the ID NOW and Xpert Xpress assays is readily available, whereas for the cobas Liat and Cue assays only a few articles have been published. Not a single scientific paper concerning the Talis, BioFire (EZ panel), Detect or Lucira assays could be found. This is also reflected in the (reference lists of the) reviews from Dinnes et al. (2021), Hawthorne (Hawthorne and Harvey, 2021), and Tu et al. (Tu, Iqbal, & O'leary, 2021), and results in the large variation in the number of lines per test in Table 5.

5.1. Scientific literature discussing the clinical performance

The PPA and NPA values published in peer-reviewed papers are listed in Table 5 for each POC test, as well as the corresponding specimen(s) used and the number of tested samples. Also, a representative selection of scientific literature, focusing on the comparison of the EUA-listed tests, is discussed in more detail below.

Aupaix et al. performed a literature study, but also investigated the performance of the ID NOW themselves with a focus on the LOD of the system. An analytical sensitivity of 81 copies/mL was obtained in experimental settings (8/8 gave positive results). A regression model calculated an LOD of 64 copies/mL. The PPA and NPA were determined upon comparison with the Xpert Xpress SARS-CoV-2 assay with quality control samples. These samples were correctly assessed by the Xpert Xpress test (Aupaix et al., 2021).

Mahmoud et al. evaluated six different rapid test methods, among which were the Abbott ID NOW and the cobas Liat system (SARS-CoV-2 & Influenza A/B test). A total of 4981 subjects were tested with a standard RT-PCR method and one of the six rapid methods for the detection of SARS-CoV-2. The ID NOW and cobas Liat (together with the Genechecker system) had the best performance and agreement of all tested methods compared to the standard RT-PCR assay (Mahmoud et al., 2021a). Tsang et al. compared the cobas Liat system (SARS-CoV-2 & Influenza A/B test) with the Cepheid GeneXpert (Xpress SARS-CoV-2 assay) system. Posterior oropharyngeal saliva was collected from 70 people suspected of SARS-CoV-2 infection (symptomatic and asymptomatic). Also 9 NP swabs samples were part of the study of Tsang et al. They concluded that there is an excellent overall concordance of the cobas Liat SARS-CoV-2 & Influenza A/B assay and cepheid Xpress SARS-CoV-2 assay (Tsang et al., 2021). The cobas Liat SARS-CoV-2 & Influenza A/B and the Xpert Xpress SARS-CoV-2/Flu/RSV assays (and the RP2.1 assay from BioFire) were compared by Jian et al. While their goal was to analyse if a variant of concern could still be detected by these systems, they also concluded that these test systems have equal performance in terms of sensitivity and accuracy (Jian et al., 2021).

In contrast to other systems, the Xpert Xpress and the ID NOW are compared quite extensively in scientific literature (Dinnes et al., 2021; Lee and Song, 2021; Lephart et al., 2021; Procop et al., 2021; Serei et al., 2021; Ulhaq and Soraya, 2021; Zhen et al., 2020). The Xpert Xpress takes about 3 times as long as the ID NOW, but has much better performance. All reports found good performance (PPA and NPA >95%) for the Xpert Xpress and high NPA values (>95%) for the ID NOW. The sensitivity of the ID NOW was found to be significantly lower, however, especially for samples with low viral load, resulting in PPA values down to 48%.

5.2. Clinical trials

The status of clinical trials was investigated on ClinicalTrials.gov. The resulting impression is that even though the systems are available on the market, data from clinical trials is sparse at best. The Lucira COVID-19 All-In-One test kit was compared to the Hologic Panther Fusion SARS-CoV-2 RT-PCR assay in a clinical trial, which is completed. However, there are no results posted for this study (Clinical Trials.gov, n.d.-a). From a previous study in which the same assays were compared, the clinical trial with 101 participants is completed and results are submitted. A PPA (across all samples) of 94.1% was obtained and a NPA of 98% (Clinical Trials.gov, n.d.-b). A clinical trial with the goal to compare the ID NOW and the Accula tests was withdrawn, due to absence of adequate resources to perform the study (Clinical Trials.gov, n.d.-d). A study plan has been set up to test 2882 subjects under the age of 18 with a buccal swab to determine the clinical sensitivity of the Abbott ID NOW test. This study is now in the recruiting phase (Clinical Trials.gov, n.d.-c). Another two clinical trials, involving the ID NOW, have an active status, but are not recruiting (Clinical Trials.gov, n.d.-f, Clinical Trials.gov, n.d.-h). To evaluate the impact of the use of the ID NOW COVID-19 tests on the length of stay of a parturient in the delivery room, a clinical trial has been set up, which is currently in the recruiting phase (Clinical Trials.gov, n.d.-i). The Xpert Xpress SARS-CoV-2 test from Cepheid is used in a clinical trial as reference assay to be compared with Surface Enhanced Raman Scattering technology for the detection of SARS-CoV-2 infection. No results are published (yet), since the study is still in the active phase (but no recruiting) (Clinical Trials.gov, n.d.-e). The Xpert Xpress SARS-CoV-2 assay will also be used in a study of COVID-19 outbreak in a hospital. At the moment, the study is in the recruiting status (Clinical Trials.gov, n.d.-j). In another trial, which is in the recruiting phase, will the Xpert Xpress be used as comparator assay (Clinical Trials.gov, n.d.-g).

To the best of our knowledge no clinical trials involving the Talis, Visby, Cue, cobas Liat, BioFire RP2.1-EZ or Detect SARS-CoV-2 tests are reported at this moment (March 2022).

6. Discussion

At the moment (March 2022) 14 SARS-CoV-2 tests have received EUAs by the FDA for POC use or OTC use. In this section the performance of these systems is compared and discussed. Also, characteristics as costs and throughput are reviewed.

• •Clinical performance

Based on the data in the IFUs (see also section 3.3 and Table 4), most tests have a PPA and NPA of >95% (due to a few false positive or negative results) or even 100%. The wide confidence interval of the PPA for the Accula system is due to the limited number of positive samples (Mesa Biotech, 2021). The PPA of the Detect assay is 90.9%, which is due to 3 false negatives and only 30 true positives (Detect Inc, 2021). Also the Lucira has a PPA below 95% (i.e 91.7%), due to 11 false negatives and 121 true positives (Lucira Health Inc, n.d.-a). To determine the clinical performance of the ID NOW only samples with spiked SARS-CoV-2 RNA were used (Abbott Diagnostics Scarborough, n.d.) instead of real patient samples as is customary. This may have positively impacted their values for the PPA and NPA. For the Talis test an additional set of diluted specimens was tested to determine the clinical performance. With a total of 93 diluted specimens a PPA and NPA of 95.7% and 100% were obtained (Talis Biomedical Corporation, n.d.-b).

As mentioned (section 5.1), for 4 of the assays (i.e., BioFire Respiratory Panel 2.1-EZ, Detect Covid-19 test, Lucira (COVID-19 All-In-One Test Kit and/or Check-It COVID-19 Test and Talis One COVID-19 test) no scientific peer-reviewed literature is available (yet). For 4 other assays (viz. Accula SARS-CoV-2 test, cobas Liat tests (SARS-CoV-2 or SARS-CoV-2 & Influenza A/B), Cue COVID-19 Test and Visby COVID-19 test) only a limited amount (<5) of publications is available that discuss the clinical performance of these assays.

Also in the scientific literature for most studies a PPA and NPA of >95% is reported (see also Table 5). Remarkable is the low PPA of the Accula system, i.e. 68%. Especially for samples with a low viral load the positive agreement was low, meaning that the Accula is not as sensitive as the comparator assay (Hogan et al., 2020). The reported clinical performance of the cobas assays, is (close to) > 95% and in accordance with the claimed PPA and NPA values in their IFU (Roche Molecular Systems and cobas, n.d.-a, Roche Molecular Systems, n.d.-b).

For the Cue only 1 article could be found that discusses the clinical performance of this system, in which the PPA is reported to be relatively low. The Cue Health POC SARS-CoV-2 RNA test was evaluated in a drive-through COVID specimen collection centre with symptomatic and asymptomatic subjects with known COVID-19 exposure. For the comparator assay NP swabs were taken and tested with the Hologic Aptima SARS-CoV-2 assay on a Hologic Panther instrument or, when the laboratory exceeded capacity, a RT-PCR test on a Roche Light Cycler 480. To overcome the lack of a true reference method, a tie-breaker system was used. If a sample was negative by the Cue Health test, but positive by a laboratory method and the patient received testing by more than one reference method, the reference result was considered to be the result obtained by two of the three (Cue, Hologic, Lab RT-PCR) methods. A PPA was obtained of 91.7% or 95.7%, with or without one patient with no tie-breaker method, respectively (Donato et al., 2021).

The ID NOW shows a wide spread in reported PPA and NPA values in academic literature. While the NPA value is consistently >95%, the PPA value varies from 53.3% to 96.2%, with most articles reporting a PPA value < 95%. This means that with the ID NOW true positives are missed and labelled as negative, meaning the ID NOW is not as sensitive as the comparator assay(s). Fung et al. determined the analytical limits of several commercial high-throughput laboratory analysers, sample-to-answer systems and one point-of-care instrument (i.e., Abbott ID NOW). The experiments on the ID NOW gave an experimental LOD of 316 cp/mL and a probit LOD of 511 cp/mL, which is one of the highest LODs and the highest probit LOD of the tested methods (Fung et al., 2020). In another study, the performance of the ID NOW and cobas SARS-CoV-2 tests were compared with 52 specimens by Jin et al. Since 6 specimens tested positive with the cobas test and only 4 with the ID NOW assay, it was concluded that the cobas test is more sensitive, i.e. the ID NOW system is giving false negative results (Jin et al., 2020). The working of the ID NOW is based on the NEAR isothermal amplification technique. While this amplification method is extremely rapid in comparison to other amplification methods, the drawback is that non-specific products are overproduced. This results in limited amplification efficiency and sensitivity (Jayakody et al., 2021; Wang et al., 2018).

Although only two scientific publications could be found for the Visby test, the performance reported is good. Renzoni et al. compared the Visby COVID-19 test to the cobas 6800 RT-qPCR assay from Roche in a retrospective study using frozen SARS-CoV-2-positive NP samples. A sample was defined positive when the Ct-value was below 35 (determined previously on the cobas 6800). In total 61 positive and 17 negative samples were used. For the Visby, 3 false-negative results were obtained, all for samples with low viral loads. Renzoni et al. also determined the LOD of the Visby COVID-19 test, which was 10² copies/mL (Renzoni et al., 2021).

The reported performance for the Xpert Xpress tests is very high, since for most studies a PPA and NPA of >95% is reported. As can be read in section 5.1., due to the good performance of the Xpert Xpress SARS-CoV-2 assay this assay is also used as comparator assay (Aupaix et al., 2021; Craney et al., 2021; Granato et al., 2021; Mitchell et al., 2021; Tanida et al., 2020; Thwe and Ren, 2020).

• •User experience

In the review of Ravi et al. four tests are compared: Cue, Xpert Xpress, Accula and ID NOW. The isothermal amplification tests, Cue and ID NOW, were found to be easier to use, have shorter turnaround times and consume less power than the RT-PCR tests (Xpert Xpress and Accula). Of these four tests, the Cue system is considered the most promising for POC applications, because this system outperforms the other three tests by portability, ease of use and the app (Ravi et al., 2020).

• •Throughput

The throughput of the discussed tests is 1 test at a time per instrument, except for the GeneXpert system that is available in 2-, 4-, and 16-module configurations (Cepheid, n.d.-a; Jian et al., 2021; Zhen et al., 2020).

• •Costs

The three molecular tests that have EUA authorization for OTC at-home use tests are commercially available. The Lucira Check It COVID-19 Test Kit can be bought via the website of Lucira for $75 (Lucira Health, n.d.-a). The Detect Start Kit, with 1 test and 1 Hub, is available for $75. A single tests costs $49 and only the Hub is $39. The tests and Hubs are available on the website of Detect, although at the moment (March 2022), everything is sold out (Detect Inc, n.d.-a). The Cue Reader can be bought for $249 and a 3-pack and 10-pack tests for $195 and $617.50, respectively. There are also combinations packages available via the website for a Cue Reader and a 3-pack or 10-pack tests for $444 and $854.05, respectively (Cue Health Inc, n.d.-b). The price of a SARS-CoV-2 antigen test, which you can buy at a pharmacy or supermarket, is about $15 and a PCR test at private clinics costs $100 or (much) more in the United States of America. For comparison, in some European countries an antigen self-test costs only €3–5 (€1.00 ≈ $1.10). The price of a PCR test (executed in a lab or at the POC) in Europe varies between €40 and €100 or even more (Consumer Reports, n.d.; The Guardian, n.d.). This means that the price of a molecular at-home test is about 3–10 times that of an antigen test, although molecular tests are more sensitive than antigen tests (Dinnes et al., 2021).

It is noted that it is not straightforward to estimate/specify exact costs (per test) for POC tests, since for such tests also additional expenses need to be taken into account (e.g. for the required apparatus and cartridges, as well as logistics). Moreover, in some countries POC tests are embedded in national healthcare systems and health insurances, which further hampers accurate estimation of the exact costs per POC test.

• •Disposable

As mentioned in section 3.1, Lucira and Visby make use of a completely disposable system. For the other 8 systems, the test cassette/cartridge (including swab) is discarded. For all systems, this means quite some plastic waste, but this is comparable to, or less than, the waste produced by conventional lab tests (protective equipment, swab, …). However, the Lucira and Visby systems and the Cue cartridge also create electronic waste, with the corresponding negative environmental impact.

• •Other remarks

In March 2021 the FDA warned for potential false results with the cobas SARS-CoV-2 & Influenza test on the cobas Liat system. The assay tubes leak sporadically, resulting in an increased likelihood of false positive Flu B results. There were also some issues resulting in abnormal PCR cycling, leading to false positive results for all analytes (U.S. Food & Drug Administration, n.d.-r). Blackall et al. tested all positive cobas Liat tests also on another PCR platform for their quality assurance project and to confirm whether false positives are indeed an issue. In total 641 positive were retested, which resulted in an overall false positive rate of 9.6% (Blackall et al., 2021).

The IFU of the ID NOW described the use of VTM, however, it has been shown that using a swab in VTM negatively influenced the performance of the Abbott ID NOW (giving false negatives) (Abbott, n.d.-c; Fung et al., 2020; Jin et al., 2020; Mitchell and George, 2020; Procop et al., 2021; Smithgall et al., 2020). Smithgall et al. also compared the ID NOW with the Xpert Xpress (with the cobas SARS-CoV-2 assay on the 6800 platform as comparator assay). The Xpert Xpress showed 98.9% and 92.0% PPA and NPA, respectively. While the NPA with the ID NOW was 100%, the PPA was only 73.9%. Smithgall et al. concluded that the ID NOW is not performing well with samples with low viral concentration that are collected in viral or universal transport media (Smithgall et al., 2020). Therefore, it was decided to only use dry NP swabs for this test (Abbott, n.d.-c).

7. Conclusion

POC systems are a good way to reduce the time-to-result for SARS-CoV-2 detection, the main drawback of the current RT-PCR tests due to the methods and logistics involved. Of the 14 molecular tests for SARS-CoV-2 that received EUAs by the FDA for POC or OTC use, the Xpert Xpress performs the best (i.e., highest PPA and NPA values reported). The ID NOW performs the worst in terms of sensitivity, meaning that true positives are missed and labelled as negative. For some POC SARS-CoV-2 tests no (i.e., BioFire, Detect and Talis) or a limited amount of (i.e., Accula, cobas and Visby) scientific literature is available, making it hard to compare the clinical performance of these systems. Currently there are only 3 molecular OTC at-home tests (i.e., Cue COVID-19 Test, Lucira Check-It COVID-19 Test and Detect Covid-19 Test) and 1 molecular prescription at-home (Lucira COVID-19 All-In-One Test Kit) test. Based on the information in the IFUs, the Cue COVID-19 Test scores best on clinical performance compared to the Lucira and Detect tests, but is also the most expensive.

Due to use of POCT technology, test-outcomes are obtained faster and, in particular in case of OTC use, less infrastructure is required. These aspects motivate the massive world-wide usage of such tests for detection of SARS-CoV-2. In fact, due to frequent use of self-testing among the population during the COVID-19 pandemic, it appears that individuals are more disposed to perform easy-and-quick at-home tests. This, combined with the fact that users control themselves when to perform a test and for which purpose, and because home testing is more accessible and discrete, opens additional opportunities for at-home tests, e.g. for screening on and/or detection of sexual transmitted diseases (STD), provided that such tests are easy, quick, cheap and reliable.

Challenges for OTC and POC tests are reduction of the costs per test (the cheaper the better, for broad accessibility and utilization among populations), without compromising sensitivity and selectivity. In addition and for home testing especially, guaranteeing ease of use and reliability of the test result despite uncontrolled environments and untrained users can be complicated. For both ease of use, as well as reducing costs for users, development of multiplex tests, or at least multiple test types that can be used on a single device, is instrumental.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.