Evaluation of the Cue Health point-of-care COVID-19 (SARS-CoV-2 nucleic acid amplification) test at a community drive through collection center

Abstract

Point-of-care (POC) tests are in high demand in order to facilitate rapid care decisions for patients suspected of SARS-CoV-2. We conducted a clinical validation study of the Cue Health POC nucleic acid amplification test (NAAT) using the Cue lower nasal swab, compared to a reference NAAT using standard nasopharyngeal swab, in 292 symptomatic and asymptomatic outpatients for SARS-CoV-2 detection in a community drive through collection setting. Positive percent agreement between Cue COVID-19 and reference SARS-CoV-2 test was 91.7% (22 of 24); or 95.7% (22 of 23) when one patient with no tie-breaker method was excluded. Negative percent agreement was 98.4% (239 of 243), and there were 25 (8.6%) invalid or canceled results. The Cue COVID-19 test demonstrated very good positive and negative percent agreement with central laboratory tests and will be useful in settings where accurate POC testing is needed to facilitate management of patients suspected of COVID-19.

1. Introduction

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) causes a mild to severe respiratory illness with varied presentations including fever, cough, and fatigue (Guan et al., 2020). While most infections are asymptomatic or cause mild symptoms, individuals with underlying health conditions and the elderly are at highest risk of developing severe complications that require admission to an intensive care unit with a correspondingly high mortality rate of up to 26% (Grasselli et al., 2020). The disease has a high rate of transmission mainly through respiratory droplets or aerosolized secretions and viral load is highest at the time of symptom onset and potentially even before symptoms manifest (Pan et al., 2020). In fact, asymptomatic transmissions are possible and may account for as many as 30% to 40% of infections (Lavezzo et al., 2020; Oran and Topol, 2020). Therefore, rapid testing of patients with respiratory symptoms or confirmed exposures is critical for identifying cases of active infection so patient isolation and contact tracing can start expeditiously.

Since the SARS-CoV-2 pandemic started, the need for rapid and accurate detection of active infection has remained a critical clinical need. Central laboratory methods using nucleic acid amplification tests (NAAT), primarily reverse transcription polymerase chain reaction (RT-PCR), take several hours for analytical processing alone. When specimen delivery, sample preparation, and long testing queues are factored in, the total turnaround time using central lab testing is often 24 hours and can be as much as multiple days or even weeks if testing is sent to a reference laboratory. This extended time to result is not conducive to the rapid testing needs of certain clinical settings. Sample collection can also be a challenge. Specimens taken from the upper respiratory tract using a nasaopharyngeal swab (NPS) are considered optimal given that this location has the highest SARS-CoV-2 viral load (Zou et al., 2020). Sample collection of the NPS can be unpleasant for the patient and extremely high global demand for testing has resulted in a critical supply shortage of nasopharyngeal collection swabs and viral transport media (VTM).

To date, the Cue COVID-19 test is one of only 6 rapid point-of-care (POC) NAATs that have received emergency use authorization (EUA) by US Food and Drug Administration (FDA) for point of care use to detect SARS-CoV-2 (https://www.fda.gov/medical-devices/coronavirus-disease-2019-covid-19-emergency-use-authorizations-medical-devices/vitro-diagnostics-euas, accessed 10/19/2020). These tests are the Cue COVID-19 test, Abbott ID NOW, Cepheid Xpert Xpress SARS-CoV-2 test, Roche Cobas SARS-CoV-2 & Influenza A/B on the Cobas Liat System, Mesa BioTech Accula SARS-CoV-2, and BioFire Respiratory Panel 2.1-EZ.

Since these tests have received EUA for point of care use, they can be implemented without rigorous laboratory evaluation. Data collected for EUA studies usually consist of utilization of residual patient samples obtained from clinically ordered SARS-CoV-2 NAATs, or small prospective studies that are not submitted for peer-reviewed publication. While these studies can probe the analytical performance of test systems, they do not evaluate the clinical performance by end-users in a clinical setting. Cue Health was granted EUA on June 10, 2020 for its rapid, POC molecular system. The single-use test cartridge is packaged with a sample collection wand for specimen collection from both nostrils for direct test cartridge insertion, thus alleviating the need for separate swabs or VTM. We evaluated the performance of the Cue Health POC SARS-CoV-2 RNA test (Cue COVID-19 test) in a drive through COVID specimen collection center serving symptomatic patients and asymptomatic patients with known COVID-19 exposures in Southwest Minnesota.

2. Materials and Methods

2.1. Study population

This was a prospective study of adult outpatients referred for SARS-CoV-2 testing at temporary drive through COVID-19 specimen collection center operated by Mayo Clinic Health System-Mankato and located in Mankato, MN. Patients were referred for testing after nurse triage based upon symptoms, exposures, or other criteria in use at the time of the study (August 5–August 17, 2020). Patient health status was not collected at the time of testing. A limited retrospective chart review was performed after study completion for patients with positive reference test results to assess whether they presented with or without symptoms. The study protocol was approved by the Mayo Clinic Institutional Review Board.

2.2. Cue COVID-19 Test

The Cue COVID-19 test (Cue Health Inc., San Diego, CA) includes the Cue Cartridge Reader, the Cue COVID-19 test cartridge, a proprietary Cue Sample Wand nasal swab, and the Cue Health Mobile Application (Cue Health App downloaded from the Apple App Store). The Cue COVID-19 test utilizes isothermal NAAT technology for the qualitative detection of SARS-CoV-2 nucleic acids. The Cue COVID-19 test primers amplify the nucleocapsid (N) region of the SARS-CoV-2 virus. An internal control ensures the presence of human cellular material in the sample and proper assay execution including sample inhibition, amplification, and assay reagent function. If the internal control is not detected, the Cue COVID-19 Test will return a result of invalid. Each Cue COVID-19 test cartridge is packaged with a single-use, sterile Cue Sample Wand that is used for collection of a nasal sample from the inferior turbinate area of the human nose. The wand is comprised of a plastic wand handle and flocked tip. When the user inserts the Cue Sample Wand with nasal sample into the cartridge, the test automatically begins. Heating, mixing, amplification, and detection take place within the cartridge. The current flow from the electrodes provides a semiquantitative nanoampere measurement that is converted to a positive or negative result (based on a predetermined cutoff). The Cue COVID-19 test takes about 20 minutes from Sample Wand insertion to results. This test has received FDA EUA.

2.3. Cue test sample collection and testing

After informed consent, patients presenting for SARS-CoV-2 testing had a sample collected using the Cue Sample Wand. Nurses performing collections for institutional reference testing were trained to collect nasal specimens using the Cue Sample Wand, swabbing both nostrils with the same Sample Wand for 5 rotations against the nasal wall of each nostril. The Cue COVID-19 test is authorized for use with no training required. The Cue Health App provides instructional videos for self-training on how to collect a nasal sample using the Cue Sample Wand and how to run the test.

After the Cue nasal wand collection, nurses handed the Cue Sample Wand to one of two laboratory technologists performing testing for the study. Per the manufacturer's instructions for use, testing personnel immediately inserted the Cue COVID-19 test Cartridge into 1 of 6 Cartridge Readers used for the study, after allowing the cartridge to preheat for about one minute, to initiate testing.

2.4. Reference laboratory testing

After collection of the Cue nasal sample, nurses collected a NPS specimen using a sterile nylon fiber non-flocked nasopharyngeal swab. NPS were placed in 3 mL Phosphate-Buffered Saline (PBS) for transportation to the testing lab. After receipt in the Mayo Clinic Health System-Mankato laboratory, PBS specimen was added to Hologic Aptima lysis tubes for testing using the Hologic Aptima SARS-CoV-2 (Hologic, Marlborough MA) assay on a Hologic Panther instrument according to manufacturer's instructions. When the laboratory had exceeded capacity available on the Hologic Panther, PBS samples were referred to the Mayo Clinic Rochester laboratory for testing by a RT-PCR test on the Roche Light Cycler 480 (Rodino et al., 2020).

The Hologic Aptima test amplifies and detects two conserved regions of the ORF1ab gene in the same fluorescence channel, with amplification of either or both regions leading to a fluorescent signal. Amplification is by transcription-mediated amplification. Reporting of a positive specimen requires only one of the 2 targets to be detected. This test has received FDA EUA.

Reference testing was also performed with a laboratory-developed test using the TaqMan assay on a Roche Light Cycler 480 (LC480) performed in the Mayo Clinic Rochester laboratories. For LC480 testing, nucleic acid is first extracted on the bioMerieux easyMAG/eMAG, Hamilton STAR, or Roche MP96. Subsequently, the nucleic acid extracts are tested for the presence of target RNA. This TaqMan assay employs a reverse transcriptase reaction to convert RNA to complementary DNA followed by amplification of the nucleocapsid gene. A Taqman probe specific for SARS-CoV-2 RNA is labeled with the fluorophore FAM. The dye labeled probe allows for detection of SARS-CoV-2 virus in the corresponding detector channel of the LightCycler 480 instrument. The test has received FDA EUA.

2.5. Statistical analysis

The primary outcome was positive and negative percent agreement between Cue COVID-19 and reference SARS-CoV-2 RNA detection. Because we anticipated fewer positive (compared to negative) results, we employed a tie-breaker system for any sample with a positive result by a laboratory method but negative result by Cue Health. In these cases if patients received testing by more than one reference method within 14 days of study enrollment (e.g., multiple clinical orders for testing and different methods used), a tie-breaker system was used whereby the reference result (positive or negative) was considered to be the result obtained by two of the three (Cue COVID-19, Hologic Aptima, and laboratory-developed RT-PCR) methods. This approach is similar to that used in another recent study of SARS-CoV-2 molecular methods in defining positive results by multiple methods to be the reference definition of positive (Procop et al., 2020), and helps overcome the lack of a true reference method for SARS-CoV-2 RNA detection.

3. Results

A total of 300 patients were enrolled in the study. Eight patients were referred to another location for NPS collection after enrollment such that it was not possible to obtain a Cue Sample Wand nasal sample, resulting in 292 patients with paired Cue COVID-19 and reference NAATs. A total of 206 reference tests were performed using the Hologic Aptima system while 85 reference tests were performed by laboratory-developed RT-PCR. One patient withdrew consent after Cue testing and thus the reference method and information other than the reference test result could not be retrieved from the electronic medical record. Twenty four Cue results were invalid, and one result was cancelled when the Cue nasal swab would not insert into the cartridge. According to instructions for use, tests should be repeated in this situation, but this was not possible due to the study design (patients had left the drive through site before Cue testing completed, therefore, collection of a second nasal wand for retesting was not possible). The invalid/canceled rate for the initial test was 8.6% (25 of 292). Since the time of the study, Cue Health has lowered the cut-off value for the internal control (detects the presence of human cellular material in the nasal sample), such that 12 invalid results obtained during the study would now return a concordant negative result. With this change, we would have observed 13 (4.5%) invalid or canceled results during the study.

Twenty-two of 24 patients (91.7%) with a positive reference result had a positive Cue COVID-19 result; while 239 of 243 (98.4%) patients with a negative reference test had a negative Cue result (Table 1 ). Excluding 1 patient (patient #1, see below) for whom no tie-breaker test was available, the positive agreement was 95.7% (22 of 23). The overall concordance between Cue Health and reference testing was 97.8% among the 267 paired samples obtained. Assay performance could not be assessed separately in symptomatic and asymptomatic patients because this information was not collected during the study. A limited retrospective chart review of patients with positive reference results (n = 24) found that 19 patients were symptomatic at the time of sample collection, while 3 were asymptomatic and for 2 patients no information was available regarding reasons for testing. Given the indications for testing at the time of the study, it is likely that the majority of patients included in the study were symptomatic at the time of testing.

Table 1.

Comparison of Cue COVID-19 test to a reference method (Hologic Aptima or laboratory-developed RT-PCR test) using 267 paired samples.

	Number of samples with a reference result of:
Number of samples with a Cue result of:	Positive	Negative	Total
Positive	22	4	26
Negative	2a	239	241
Positive percent agreement	91.7%a
Negative percent agreement		98.4%
Total	24	243	267

^aOne discrepant positive reference sample did not have a tie-breaker method available, so positive percent agreement would be 22/23 (95.7%) excluding that sample.

Table 2 lists the details for patients with discrepant Cue Health, Hologic Aptima and/or laboratory-developed RT-PCR results. Patient #1 had a negative Cue COVID-19 result with a positive result by the Hologic Aptima method. This patient was enrolled in the study 23 days after symptom onset with symptoms including headache, diarrhea, sore throat and loss of smell. The patient also reported contact with a person with a laboratory-confirmed case of COVID-19. At the time of testing the patient reported some loss of smell but other symptoms had improved. No other testing was available for comparison.

Table 2.

Details of discordant results between Cue COVID-19, Hologic Aptima, and/or laboratory-developed RT-PCR assays.

Patient #	Cue result	Reference result (method)	RLU or Cp value	Other testing performed (days ± study enrollment)	Reference Consensus result
1	Negative	Positive (Hologic)	RLU 1139	None	Positive
2	Negative	Positive (Hologic)	RLU 1097	Negative LDT RT-PCR (1 day before enrollment)	Negative
3	Negative	Positive (LDT RT-PCR)	Cp 35.00	Positive Hologic (8 days before enrollment)	Positive
4	Positive	Negative (Hologic)	RLU 288	None	Negative
5	Positive	Negative (Hologic)	RLU 279	None	Negative
6	Positive	Negative (Hologic)	RLU 283	None	Negative
7	Positive	Negative (Hologic)	RLU 287	Negative Hologic (4 days after enrollment)	Negative

RLU = relative light unit, Hologic Aptima test results are based upon total RLU and the kinetic curve type; Cp = crossing point, a positive LDT RT-PCR has a Cp ≤ 40 and a valid amplification curve; LDT = laboratory-developed test; RT-PCR = reverse transcription polymerase chain reaction

Patient #2 had a pre-procedural positive Hologic Aptima test result performed 12 days prior to study enrollment and was also positive for SARS-CoV-2 antibodies using the Roche total antibody test. Repeat testing on the day before study enrollment using the laboratory-developed RT-PCR method yielded a negative result; while testing on the date of study enrollment by the Hologic Aptima method was positive. This patient subsequently tested positive by the Hologic Aptima method on an additional NPS sample obtained 3 days after study enrollment. The reference result for patient #2 was negative, as 2 of the 3 tests (Cue COVID-19 and laboratory-developed RT-PCR) were negative within 14 days of study enrollment. Although discussed here as a patient with discordant results, the reference consensus result is negative, therefore this patient is included among the 239 patients with negative agreement.

Patient #3 originally tested positive on the Hologic Aptima 8 days prior to study enrollment after experiencing symptoms of loss of smell and taste the day prior to initial testing (9 days prior to study enrollment). The day of study enrollment, the Cue test was negative with a positive laboratory-developed RT-PCR result showing a crossing point (Cp) value of 35.00 indicating a low viral load.

Patient #4 reported contact with a person with laboratory-confirmed COVID-19 and new onset headache and diarrhea on the day of study enrollment, and tested positive by Cue but negative by Hologic Aptima.

Patient #5 reported new onset diarrhea and cough on the day of study enrollment, and tested positive by Cue but negative by Hologic Aptima.

Patient #6 had a positive Cue with a negative Hologic Aptima result, but no additional information could be found in the electronic medical record on patient symptoms or reasons for testing. No additional testing information was available for patients #4-6.

Patient #7 had a positive Cue test with negative Hologic Aptima test on the day of study enrollment, and an additional negative test by Hologic Aptima 4 days after study enrollment. No information on symptoms or reasons for testing could be found in the medical record for patient #7.

4. Discussion

Rapid testing and result interpretation for SARS-CoV-2 performed near the patient has value in many clinical settings including emergency departments, pre-procedural locations, walk-in clinics, and environments such as long-term care facilities with many high risk patients. These locations may not have access to central laboratory equipment or personnel and have a need to triage or counsel the patient according to the results obtained, thus making a POC test the optimal solution. The evaluations of several rapid or POC test solutions for molecular detection of SARS-CoV-2 have been recently published (Basu et al., 2020; Creager et al., 2020; Hogan et al., 2020; Wolters et al., 2020; Zhen et al., 2020), however these studies have used mostly residual samples submitted for central laboratory testing and the testing was performed by in a controlled laboratory environment.

Here we describe the performance of the Cue COVID-19 test at a drive through screening location for patients with symptoms consistent with infection or who were asymptomatic but had a recent exposure to a confirmed SARS-CoV-2 patient. Compared to the reference result, the Cue test had an overall concordance of 97.8%. It should be noted that our study compared nasal swab testing on the Cue test to nasopharyngeal swab testing on the reference method. NPS collections are still considered the gold standard and are recommended by the Infectious Disease Society of America, however, some recent studies report sensitivities of NPS and nasal swab collections to be very similar (https://www.idsociety.org/practice-guideline/covid-19-guideline-diagnostics/ Accessed 10/19/2020.; Tu et al., 2020). The difference in sample collection source could account for some of the observed discrepant results. Patients with discrepant positive results (patients 1–3, Table 2) were tested 8 to 23 days after symptom onset. This may have contributed to discrepant test results due to either different RNA targets used in the assays or differing analytical sensitivity of the three tests.

A total of 8.6% of the initial Cue test results were either invalid or canceled and would have required patient retesting in accordance with the instructions for use. With recent changes in the cut-off for determining a valid internal control (i.e., the amount of signal required to give a result of either positive or negative) made since the time of the study, the invalid rate would be anticipated to be 4.5% in our study set. Recollection of samples and retesting would be expected to further reduce the invalid rate, but this was not possible during our study.

This study has several limitations to note. The study site was a community-based collection center where patients with a clinical order for SARS-CoV-2 testing were recruited. While the location was chosen based on a relatively high rate (∼10%) of positive test results, this approach limited the number of positive cases compared to studies including more positives from residual lab samples. At the time this study was conducted, the Cue COVID-19 test had not yet received FDA EUA for testing residual or banked specimens; and therefore, a prospective comparison was the only means to assess performance of the device. The advantage to this study design is that our findings likely reflect observed performance in a similar setting (overall positive rate 8.2%). While sample collection, a crucial step in infectious disease testing, was conducted by nursing staff as is typical in POC settings; to prevent backlogs in clinical specimen collection, the Cue test was performed by laboratory staff in the POC environment. Our study also did not have a method for resolving all discrepant results observed between the Cue and reference NAAT. Therefore, an incorrect reference method result cannot be ruled out. It was also not possible to perform a formal limit of detection study due to the design of the assay at that time. Lastly, patients with invalid/canceled results were not able to be retested as directed by the Cue instructions for use because study participants left the facility before POC testing was completed.

5. Conclusions

In summary, our study demonstrates that the Cue COVID-19 test using a nasal swab collection method is accurate (97.8% overall concordance), and is both sensitive and specific compared to central laboratory testing using an NPS collection. The test is easy to perform with minimal training or previous laboratory testing experience. Invalid results can occur and should be factored into patient testing workflows to allow for retesting by Cue or with another NAAT. Near-patient rapid testing at locations such as drive through SARS-CoV-2 testing sites can facilitate prompt disease identification and patient quarantine. These facilities are often staffed by nurses or other health care providers who are inexperienced with laboratory testing; therefore, the Cue Health test for SARS-CoV-2 can be considered a feasible solution to implement at sites requiring a POC solution.

Author contribution statement

Leslie Donato: Conceptualization, formal analysis, writing original draft, editing and reviewing. Vipul Trivedi: Supervision, resources, editing and reviewing manuscript. Angie Stransky: Supervision, resources, editing and reviewing manuscript. Artika Misra: Supervision, resources, editing and reviewing manuscript. Bobbi Pritt: Conceptualization, data curation, methodology, editing and review manuscript. Matthew Binnicker: Formal analysis, methodology, editing and reviewing manuscript. Brad Karon: Conceptualization, methodology, formal analysis, writing original draft, editing and reviewing manuscript.