Abstract
A comparison of rapid point-of-care serology tests using finger prick and venous blood was done on 278 participants. In a laboratory setting, immunoglobulin G (IgG) sensitivity neared 100%; however, IgG sensitivity dramatically dropped (82%) in field testing. Possible factors include finger prick volume variability, hemolysis, cassette readability, and operator training.
Keywords: COVID-19, diagnostic microbiology, public health, rapid point-of-care test, serology
As the number of coronavirus disease 2019 (COVID-19) cases continues to increase worldwide, the need for fast, easy-to-use, and accurate tests is urgent. Rapid point-of-care (rPOC) lateral flow tests measure serum antibodies to severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), and a number of these are currently available. Most cassettes are designed to detect separately and simultaneously immunoglobulin M (IgM) and immunoglobulin G (IgG) antibody types, and they have been used in a variety of studies to provide estimates of population seroprevalence. In acute cases, when a patient is repeatedly negative by polymerase chain reaction (PCR) but symptomatic, a highly sensitive and specific rPOC could be utilized as a diagnostic method for those with difficult venous access. rPOC can be also be used for surveillance purposes in hard-to-reach populations that have no access to laboratories, or to satisfy pretravel requirements. The literature suggests that most of the comparative evaluations on rPOC tests have been done in laboratory settings [1–3]. In order to evaluate the performance of rPOC in the field, the British Columbia Centre for Disease Control Public Health Laboratory (BCCDC PHL) conducted a comparative assessment of the performance of rPOC lateral flow assays in a laboratory (using venous blood samples) vs field (using fingerpick capillary blood) setting. Field testing was conducted in 2 long-term care facilities (LTCFs) affected by COVID-19 outbreaks [4].
We conducted initial laboratory-based evaluations with a total of 142 venous blood samples, with subsequent evaluation in the field on 278 capillary blood samples. Briefly, 3 rPOC products were screened in the laboratory using venous samples obtained from known COVID-19 patients at 0–7, 8–14, and >14 days post–illness onset (total n = 79), as well as prepandemic negative samples stored at BCCDC PHL tested for other serology before 2019, which included samples with seropositivity to other common pathogens, such as HIV, hepatitis C virus, syphilis, etc. (n = 63) (Table 1). Some of those negative samples, such as Toxoplasma IgM, West Nile virus IgM, and Chikungunya IgM, were selected because they are notorious for exhibiting nonspecific reactivity against many other pathogens (Table 2). All rPOC products used in this study could detect both IgM and IgG on the same cassette. Positive patients were confirmed by BCCDC PHL in-house laboratory-developed reverse transcription PCR [5]. Three of the products tested—Artron Diagnostics Inc. (Burnaby, BC) referred as Artron, BioCan Diagnostics Inc. (Coquitlam, BC) referred as BioCan and Rapid Response BTNX Inc. (Markham, ON) referred as BTNX—yielded very promising analytical performance with 91%–95% sensitivity and 93%–100% specificity (Table 1). All 3 assays demonstrated highest sensitivities when tested against serum taken >14 days post–illness onset. In terms of specificity, Artron detection of COVID-19 IgM cross-reacted with WNV IgG+, mumps IgM+, and Chikungunya IgM+ sera, and BTNX detection of COVID-19 IgG cross-reacted with Toxoplasma IgM + serum.
Table 1.
Laboratory Validation of rPOC COVID-19 Lateral Flow Cassette Performance
| Stratified by Timing of Measurement: Number of Days From Illness Onset | ||||||
|---|---|---|---|---|---|---|
| ≤4 to 7 | >7 to 14 | >14 | Average Performance of Total Samples | |||
| n = 11 | n = 28 | n = 32 | n = 79a | n = 63 | ||
| Isotype | Test Name | Sensitivity | Sensitivity | Sensitivity | Sensitivity | Specificity |
| IgM | Artron | 72.7 | 96.4 | 100 | 94.9 | 93.5 |
| IgG | Artron | 72.7 | 89.3 | 100 | 92.4 | 100 |
| IgM | BioCan | 72.7 | 89.3 | 96.9 | 91.1 | 100 |
| IgG | BioCan | 72.7 | 92.9 | 100 | 93.7 | 100 |
| IgM | BTNX | 72.7 | 96.4 | 93.8 | 91.1 | 100 |
| IgG | BTNX | 63.6 | 92.9 | 100 | 92.4 | 98.4 |
Abbreviations: COVID-19, coronavirus disease 2019; IgG, immunoglobulin G; IgM, immunoglobulin M; rPOC, rapid point-of-care.
aTotal N is not equal to the sum of individual subsets, as additional samples with unknown date of onset from illness were included.
Table 2.
Specificity Assessment of COVID-19 Point-of-Care Test Kits
| Cross-Reacted With | |||||||
|---|---|---|---|---|---|---|---|
| Arton | Bio-Can | BTNX | |||||
| Negative Samplesa | Samples | IgM | IgG | IgM | IgG | IgM | IgG |
| Presumed negativeb | 19 | 0 | 0 | 0 | 0 | 0 | 0 |
| Coronavirus seasonal | 2 | 0 | 0 | 0 | 0 | 0 | 0 |
| Coronavirus 229E | 1 | 0 | 0 | 0 | 0 | 0 | 0 |
| Coronavirus NL63 | 2 | 0 | 0 | 0 | 0 | 0 | 0 |
| Coronavirus HKU1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 |
| SARS-CoV-1 | 2 | 0 | 0 | 0 | 0 | 0 | 0 |
| Influenza A | 3 | 0 | 0 | 0 | 0 | 0 | 0 |
| Influenza B | 3 | 0 | 0 | 0 | 0 | 0 | 0 |
| RSV | 2 | 0 | 0 | 0 | 0 | 0 | 0 |
| HCV+ | 5 | 0 | 0 | 0 | 0 | 0 | 0 |
| Mumps IgM+ | 5 | 1 | 0 | 0 | 0 | 0 | 0 |
| WNV IgM+ | 2 | 1 | 0 | 0 | 0 | 0 | 0 |
| Toxoplasma IgM+ | 5 | 0 | 0 | 0 | 0 | 0 | 1 |
| Chikungunya IgM+ | 5 | 1 | 0 | 0 | 0 | 0 | 0 |
| RPR 1:512 (syphilis) | 2 | 0 | 0 | 0 | 0 | 0 | 0 |
| RPR 1:128 (syphilis) | 2 | 0 | 0 | 0 | 0 | 0 | 0 |
| RPR 1:32 (syphilis) | 2 | 0 | 0 | 0 | 0 | 0 | 0 |
| Total | 63 | 3 | 0 | 0 | 0 | 0 | 1 |
Abbreviations: COVID-19, coronavirus disease 2019; HCV, hepatitis C virus; IgG, immunoglobulin G; IgM, immunoglobulin M; RPR, rapid plasma reagin; RSV, respiratory syncytial virus; SARS-CoV-1, severe acute respiratory syndrome coronavirus 2; WNV, West Nile virus.
aSamples were selected to include those with possible nonspecific cross-reactivity; all samples were collected pre–November 2019, before SARS-CoV-2 virus was detected in British Columbia.
bPresumed negatives were selected from samples of prenatal and organ donor.
Based on laboratory performance, secure supply chain, and product cost, the Artron Diagnostics Inc. product was selected for a dual laboratory/field trial, with the field trial conducted in 2 LTCFs with confirmed COVID-19 outbreaks. In LTCFs, rPOC tests were performed by laboratory medicine technologists who were trained before conducting testing. The BioCan Diagnostics Inc. and BTNX products were also tested using only the venous blood samples collected at the LTCF.
Samples were collected from residents and staff at least 14 days after symptom onset (for known PCR-confirmed COVID-19 patients). Samples comprised those from known COVID-positive patients (PCR-confirmed) and from patients of “unknown” status (either PCR-negative or never tested by PCR). “Unknown” status patients were classified as “presumed positive” (consensus positive SARS-CoV-2 serology on all 4 high-throughput automated platforms: [(1) LIAISON® SARS-CoV-2 S1/S2 IgG (DiaSorin Canada Ltd, Mississauga, ON); (2) ARCHITECT SARS-CoV-2 IgG (ABbott DIAGNOSTICS, Mississauga, ON); (3) VITROS® Anti-SARS-CoV-2 Total (Ortho Clinical Diagnostics Canada, Markham, ON) and (4) SARS-CoV-2 Total Assay (Siemens health care limited, Oakville, ON)] and “presumed negative” (consensus negative SARS-CoV-2 serology on all 4 high-throughput automated platforms). Any samples with discrepant results on any of the 4 high-throughput automated platforms were excluded from the analysis.
In the field, we found that the finger prick–based sensitivity of the Artron rPOC test was overall inferior to that of venous blood in a laboratory setting (Table 3). Finger prick IgG sensitivity dropped to ~83% (specificity, 99%). When paired venous samples were tested in the laboratory on Artron cassettes, the sensitivity did improve to ~89%, but still did not reach that observed in the initial validation study (Table 1). IgM sensitivity in the field setting was higher than in the in-laboratory serum performance on paired samples (~67% vs ~58%), but both were markedly lower than in the initial validation study (Table 1). There was also a small drop in specificity for IgM in the field vs in the laboratory setting when conducted on paired samples (91.5% vs 92.6%). When BioCan and BTNX rPOC cassettes were trialed on a large subset (No. dependent on availability of cassettes) of the same venous samples, the performance was also inferior to that previously observed in the laboratory evaluation. Specifically, the sensitivity of both BioCan and BNTX cassettes in detecting COVID-19 IgM 14 days after symptom onset was markedly lower than that in the validation study (data not shown). Furthermore, additional BioCan and BTNX cassettes procured for the LTCF evaluation were noted to have variable appearance and inferior quality to that of the first batch trialed in the laboratory.
Table 3.
Field Trial of rPOC COVID-19 Lateral Flow Cassette Performance
|
|
Our results demonstrate poorer performance of rPOC assays under field settings relative to what can be achieved in the laboratory, possibly due to reduced standardization in blood inoculum in the field. Capillary blood inoculum may vary in volume with possible effects on sensitivity. The nature of capillary blood collection also predisposes the sample to hemolysis, which might interfere with test specificity. Variability of lighting in the field and operator training may further compound these effects. Our experience further highlights the instability of rapidly developed and produced COVID-19-related product supplies, which can have substantial batch-to-batch variations, depending on the manufacturer.
Acknowledgments
We would like to thank to all BCCDC PHL Laboratory technicians and allied health staff at LTCFs for their relentless work during the SARS-CoV-2 pandemic. The authors would also like to thank all manufacturers who participated in this study by providing rPOC kits free of charge. Karen Chu assisted in copy editing and proofreading the manuscript. The authors would also like to thank Dr. Karen Chu for critically reading this manuscript. Michael Donoghue provided project management, and the authors would like to thank him for his support.
Financial support. This work was funded by Genome BC on project COV-050.
Potential conflicts of interest. All authors: no reported conflicts of interest. All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.
Author contributions. All authors contributed equally to this work.
Patient consent. This study was authorized by the Provincial Health Officer and approved by the Clinical Research Ethics Board of the University of British Columbia (H20-01089). Long-term care residents capable of providing informed consent were asked before Health Authority staff attended the site to collect specimens if they agreed to participate in the validation work. For residents incapable of providing their own informed consent (the majority of residents), long-term care facility staff contacted the appropriate family members to obtain verbal consent. Long-term care facility staff obtained consent because of their preexisting relationship with residents and their family members. Health Authority staff (medical laboratory assistants for phlebotomy and medical laboratory technologists for point-of-care testing) were provided with the list of consented residents. At the time of specimen collection, the resident was asked if it was okay to collect the specimens. No samples were collected or testing performed for any resident who refused at this point, regardless of previous consent from the resident or family members.
References
- 1. Charlton CL, Kanji JN, Johal K, et al. Evaluation of six commercial mid to high volume antibody and six point of care lateral flow assays for detection of SARS-CoV-2 antibodies. J Clin Microbiol 2020; 58:e01361-20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Flower B, Brown JC, Simmons B, et al. Clinical and laboratory evaluation of SARS-CoV-2 lateral flow assays for use in a national COVID-19 seroprevalence survey. Thorax 2020; 75:1082–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Wu JL, Tseng WP, Lin CH, et al. Four point-of-care lateral flow immunoassays for diagnosis of COVID-19 and for assessing dynamics of antibody responses to SARS-CoV-2. J Infect 2020; 81:435–42. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Vijh R, Ghafari C, Hayden A, et al. Serological survey following SARS-COV-2 outbreaks at long-term care facilities in metro Vancouver, British Columbia: implications for outbreak management and infection control policies. Am J Infect Control 2020; S0196-6553(20)30927-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. LeBlanc JJ, Gubbay JB, Li Y, et al. COVID-19 Pandemic Diagnostics Investigation Team of the Canadian Public Health Laboratory Network (CPHLN) Respiratory Virus Working Group. Real-time PCR-based SARS-CoV-2 detection in Canadian laboratories. Clin Virol 2020; 128:104433. [DOI] [PMC free article] [PubMed] [Google Scholar]