Skip to main content
PMC home page
Wellcome Open Research logoLink to Wellcome Open Research
. 2021 Jan 21;6:9. [Version 1] doi: 10.12688/wellcomeopenres.16517.1

SARS-CoV-2 detection by a clinical diagnostic RT-LAMP assay

Michael D Buck 1,a,#, Enzo Z Poirier 1,b,#, Ana Cardoso 1,#, Bruno Frederico 1,#, Johnathan Canton 1, Sam Barrell 1, Rupert Beale 1, Richard Byrne 1, Simon Caidan 1, Margaret Crawford 1, Laura Cubitt 1, Sonia Gandhi 1,2,3, Robert Goldstone 1, Paul R Grant 4, Kiran Gulati 5, Steve Hindmarsh 1, Michael Howell 1, Michael Hubank 6,7, Rachael Instrell 1, Ming Jiang 1, George Kassiotis 1, Wei-Ting Lu 1, James I MacRae 1, Iana Martini 8, Davin Miller 5, David Moore 2,3, Eleni Nastouli 1,3,9, Jerome Nicod 1, Luke Nightingale 1, Jessica Olsen 1, Amin Oomatia 8, Nicola O'Reilly 1, Anett Rideg 1, Ok-Ryul Song 1, Amy Strange 1, Charles Swanton 1,2,3, Samra Turajlic 1,6, Mary Wu 1, Caetano Reis e Sousa 1,c; The Crick COVID-19 Consortium
PMCID: PMC8170534  PMID: 34095506

Abstract

The ongoing pandemic of SARS-CoV-2 calls for rapid and cost-effective methods to accurately identify infected individuals. The vast majority of patient samples is assessed for viral RNA presence by RT-qPCR. Our biomedical research institute, in collaboration between partner hospitals and an accredited clinical diagnostic laboratory, established a diagnostic testing pipeline that has reported on more than 252,000 RT-qPCR results since its commencement at the beginning of April 2020. However, due to ongoing demand and competition for critical resources, alternative testing strategies were sought. In this work, we present a clinically-validated procedure for high-throughput SARS-CoV-2 detection by RT-LAMP in 25 minutes that is robust, reliable, repeatable, sensitive, specific, and inexpensive.

Keywords: SARS-CoV-2, clinical diagnostic, RT-LAMP

Introduction

The current pandemic caused by novel coronavirus SARS-CoV-2, first detected in late 2019 in the province of Wuhan, China, has rapidly spread worldwide, infecting more than 68 million individuals as of 11 December 2020 1– 3. Infection with SARS-CoV-2 can lead to development of COVID-19, a disease associated with severe acute respiratory syndrome, responsible for hundreds of thousands of deaths globally. Controlling the spread of SARS-CoV-2 relies on the ability of healthcare systems to quickly identify infected individuals, which has mainly relied on RT-qPCR for viral RNA detection 4. International competition for commercial kits and reagents has negatively impacted the ability of many countries to scale up testing capacity to deal with the increased demand caused by rampant infection. Implementing RT-qPCR testing programs requires specialised laboratory equipment and reagents, presenting additional challenges.

In an effort to increase the diagnostic capacity for SARS-CoV-2 infection in the UK, the Francis Crick Institute, a biomedical research institute based in London, rapidly repurposed its staff and facilities in late March 2020 to serve as a clinical diagnostic testing facility through a partnership between a major local healthcare provider (University College London Hospitals National Health Services Trust) and an accredited clinical diagnostic laboratory (Health Services Laboratories, HSL), termed the CRICK COVID-19 Consortium (CCC) 5. The pipeline utilises a series of in-house buffers to first inactivate patient samples received from care homes and hospitals, and to then extract RNA before using a CE marked commercial kit to detect SARS-CoV-2 by RT-qPCR. Patient results are reported through a custom online web portal that interfaces with the reference laboratory 5. In order to avoid dependence on any singular testing methodology, to continue increasing testing capacity, and to provide a potential means to deliver diagnostics at the point-of-care, the CCC was also tasked with developing and validating alternative SARS-CoV-2 testing strategies.

Herein, we describe the use of loop mediated isothermal amplification PCR coupled with reverse transcription (RT-LAMP) as a robust method for SARS-CoV-2 detection in clinical specimens 6. The entire strand displacement and amplification procedure is carried out at a single temperature in less than 30 minutes, alleviating the need for a thermocycler qPCR system. Diagnostic tests utilising this technique have been developed for RNA viruses and other pathogens 7– 9, and there are now several works focusing on the use of RT-LAMP to detect SARS-CoV-2 10– 17. We set up a SARS-CoV-2 RT-LAMP assay using the WarmStart Colorimetric LAMP 2X Mastermix commercialised by New England Biolabs (NEB), which allows for visual assessment of DNA amplification. Alternatively, DNA amplification can be quantified on a real-time PCR machine by complexing the reaction with the DNA dye SYTO 9 10. We made use of primers developed by Zhang et al. targeting the nucleocapsid phosphoprotein (N gene) of SARS-CoV-2 10 and, in a parallel reaction, primers that detect human 18S rRNA to control for specimen quality 9.

Our results demonstrate that within the CCC pipeline, RT-LAMP can readily replace RT-qPCR as a means for detecting SARS-CoV-2 transcripts within RNA extracted from nose-throat swabs and endotracheal secretions/bronchoalveolar lavage fluid. RT-LAMP for the N gene shows absence of non-specific amplification and cross-reactivity with other human coronaviruses or respiratory viruses, and displays a sensitivity threshold almost equivalent to the gold standard RT-qPCR. Switching to RT-LAMP translates into a ten-fold decrease in total reagent cost and a potential four-fold increase in pipeline output. Additionally, we provide preliminary data suggesting that RT-LAMP can be performed without prior RNA extraction, allowing rapid and cost-effective testing that could potentially be extended to point-of-care. The entire workflow was validated under extended governance by public health authorities during the pandemic and inspected by a qualified UKAS assessor against GenQA guidelines to verify compliance to ISO 15189:2012 equivalent standard (US equiv. CAP/CLIA). As such, the procedure developed here is ready to be deployed for diagnostic testing of SARS-CoV-2.

Methods

Patient samples

In response to the pandemic, the Francis Crick Institute formed a partnership with University College London Hospitals (the Crick-COVID-Consortium) and set up a SARS-CoV-2 RT-qPCR testing pipeline (CCC pipeline). Nasopharyngeal swabs tested in this study were obtained from the CCC pipeline 5.

CCC pipeline procedures and HSL RT-qPCR

All the samples processing through the CCC pipeline was done in accordance with Standard Operating Procedures (SOPs) described recently 5, which can be found at https://www.crick.ac.uk/research/covid-19/covid19-consortium. A redefined reaction, running, and reporting SOP can be found in the Supplemental Methods ( Extended data 18).

RT-LAMP assay

RT-LAMP reaction was performed in a total volume of 15 µL, mixing 7.5 µL WarmStart Colorimetric LAMP 2X Master Mix (New England Biolabs, M1800), 1.5 µL of 10X primer mix, 1.5 µL SYTO 9 Green Fluorescent Nucleic Acid Stain (ThermoFisher Scientific, S34854), 1.5 µL of Nuclease-free water (Ambion, AM9932) and 3 µL of sample unless stated otherwise. 10X SYTO 9 solution at 5 µM in nuclease free water was prepared for a final concentration of 0.5 µM in the final RT-LAMP reaction. 10X primer mix was prepared from 100 µM desalted DNA primers obtained from Sigma Custom Oligos Service. 10X primer mixes for N gene and 18S contained FIP and BIP primers at 16 µM, F3 and B3 at 2 µM, LF and LB at 4 µM. RT-LAMP was ran following a SOP (cf. Extended data: Supplementary Methods 1 and 2 18) on a 7500, 7500 Fast, QuantStudio 3, 5, or 7 Real-Time PCR System (Applied Biosystems). A negative no template control (NTC) and a positive control (supplied by HSL, SARS-CoV-2 clinical sample) were included on every run. Experiments utilising laboratory grown SARS-CoV-2 were performed in containment level 3 at the Francis Crick Institute (FCI) by trained personnel according to health and safety guidelines.

We found that the assembled reaction mix was unstable when kept at 4°C for more than a few hours, and sensitive to freeze-thaw when kept at -20°C for more than a day. The RT-LAMP should be performed using freshly prepared reaction mix. The individual components should be stored at -20°C until use and avoid repeated freeze/thaw cycles.

Accreditation and governance

As outlined recently 5, the CCC was formed in partnership with HSL, a UCLH UKAS accredited lab, who already had a COVID-19 RT-qPCR test in scope. All samples were received and communicated by HSL under their accreditation and the CCC RT-LAMP assay was validated against their existing RT-qPCR test and the CCC’s validated RT-qPCR test, which uses a CE marked commercial kit (BGI). Given the urgent timeframe required to implement testing, it was not possible to secure official clinical laboratory accreditation for the FCI. However, full measures were taken to ensure that the CCC RT-LAMP test was evaluated, verified and performed for diagnostic use in an environment that adhered to equivalent international standards (ISO 15189:2012, US equiv. CAP/CLIA), overseen and audited by HSL. These measures were implemented under the advice and oversight of registered professionals from existing nearby ISO accredited medical laboratories, and included writing and following clinical diagnostic SOPs for every stage of the pipeline from sample reception, processing to result reporting by qualified clinical scientists prior to results being communicated to patients by HSL. Additional SOPs were followed for sample storage, disposal of materials, batch certification of reagents and incident reporting. Appropriate risk assessments, training and competency assessment procedures were established and documented. Record sheets were created to document the receipt, batch acceptance testing, and start/end of use dates for key reagents and consumables. An inventory of all key equipment was compiled which, where appropriate, included details of service and calibration records. Systems were also established for the control of all key documents (version implementation, distribution and acknowledgement), audit trailing (what samples were tested when, by whom, with what equipment and using which consumable/reagent batches), and the recording of all untoward incidents/issues (thus facilitating appropriate investigation, rectification and recurrence prevention). Samples were barcoded and tracked using the Crick Clarity library information management system (LIMS). All key documents are available at https://www.crick.ac.uk/research/covid-19/covid19-consortium. NHS governance was extended by a specific memorandum of understanding for diagnostic testing between UCLH and the FCI enabled by NHS England. Assurance of the pipeline was performed in collaboration with GenQA, following their checklist for non-accredited laboratories and the lab and CCC workflow were inspected by a qualified UKAS assessor against the GenQA guidelines to verify compliance to IS015189:2012 equivalent standard.

Results

RT-LAMP test validity

RT-LAMP was performed with 3 µL of RNA in a total reaction volume of 15 µL unless stated otherwise. Each sample was tested in two separate reactions, one with primers targeting the N gene of SARS-CoV-2, and the second targeting human 18S rRNA to control for specimen integrity and quality ( Figure 1A). The RT-LAMP mastermix contains a colorimetric pH indicator that turns from pink to yellow upon DNA amplification ( Figure 1B). In addition, we benchmarked the RT-LAMP method by measuring DNA amplification using a SYBR based dye in a real-time PCR machine. When measuring fluorescence every minute (‘cycle’), double stranded DNA accumulation follows a characteristic exponential amplification phase that eventually plateaus ( Figure 1C and 1D).

Figure 1. Validation of SARS-CoV-2 detection by RT-LAMP targeting the N gene.

Figure 1.

Open in a new tab

( A) The N gene of SARS-CoV-2 was targeted using primers designed by Zhang et al. 10, whilst the internal control implemented utilised primers for 18S rRNA initially published by Lamb et al. 9. ( B) Upon DNA amplification, a colorimetric pH dye in the reaction mix will change from pink to yellow if the target is present. Four SARS-CoV-2 positive and four SARS-CoV-2 negative patient samples are shown. ( C–D) Amplification curves from 24 patient samples assessed by SARS-CoV-2 (N gene) RT-LAMP ( C) or 18S ( D) in a Real-Time PCR machine. ( E) Dot plot of the SARS-CoV-2 Ct values from ( C–D) compared to Ct values derived from the Crick’s RT-qPCR diagnostic pipeline using the CE marked BGI kit. Ct values of ‘undetermined’ are plotted as “Ct = 40” (left y-axis) or “Ct = 25” (right y-axis) for illustrative purposes. Data are normalised by assay run/cycles determined by the two methods for comparative purposes. The clinical call from the reference laboratory is indicated above the graph.

An initial characterization of the technique was performed using 24 RNA samples purified from patient nasopharyngeal swabs, 12 of which were positive for SARS-CoV-2 as assessed by the CCC pipeline via RT-qPCR 5. RT-LAMP could detect SARS-CoV-2 in the 12 positive samples with no amplification detected in all 12 negative samples, displaying 100% concordance to our current clinical diagnostic platform ( Figures 1C and Figures 1E). Internal control signal was detected in all samples ( Figure 1D).

RT-LAMP background noise and specificity

The background signal of the SARS-CoV-2 RT-LAMP assay was assessed by running RNA elution buffer (7 samples) or nuclease free water (7 samples) no template controls (NTCs) performed 4 separate times by 4 different operators. Non-specific signal could be detected in one well of one experiment after 25 minutes ( Figure 2A). A similar experiment was performed for the 18S rRNA internal control RT-LAMP assay and revealed signal in NTCs after 20 minutes ( Figure 2B). Based on these NTC data, a detection threshold was set for each RT-LAMP assay which eliminated all false positives in the N gene RT-LAMP assay and 21 of 23 false positives in the 18S rRNA RT-LAMP assay ( Figure 2C). This translated to a maximum run time for each reaction: 25 minutes for N gene and 20 minutes for 18S ( Figure 1C). Implementing these criteria in a separate experiment yielded no false positives from 47 NTCs in the SARS-CoV-2 RT-LAMP protocol and 4.5% false positives in the 18S rRNA RT-LAMP assay (all with a time threshold > 19) ( Figure 2D–2E).

Figure 2. Optimisation of the RT-LAMP assay for accurate detection.

Figure 2.

Open in a new tab

( A–B) 7 samples of water and RNA elution buffer from the CCC pipeline were run 4 independent times by 4 distinct operators by SARS-CoV-2. ( A) or 18S ( B) RT-LAMP. ( C) Dot plot of ‘Ct’ values assessed in ( A–B). Assay endpoint detection threshold was set for 18S (dashed blue line) and SARS-CoV-2 (dashed orange line) based on these data. ( D–E) RT-LAMP amplification data from 47 samples of RNA elution buffer using the newly established assay endpoint of 25 min for SARS-CoV-2 ( D) and 20 minutes for 18S ( E).

( A–B) 7 samples of water and RNA elution buffer from the CCC pipeline were run 4 independent times by 4 distinct operators by SARS-CoV-2. ( A) or 18S ( B) RT-LAMP. ( C) Dot plot of ‘Ct’ values assessed in ( A–B). Assay endpoint detection threshold was set for 18S (dashed blue line) and SARS-CoV-2 (dashed orange line) based on these data. ( D–E) RT-LAMP amplification data from 47 samples of RNA elution buffer using the newly established assay endpoint of 25 min for SARS-CoV-2 ( D) and 20 minutes for 18S ( E).

We then asked if melting curves obtained from thermocycler runs could be used to help discriminate true positives from false positives. N gene and 18S rRNA RT-LAMP reactions each gave single peaks in melt curve plots ( Extended data: Figure S1A 18) and consistent melting temperature values ( Extended data: Figure S1B 18) over multiple experiments performed on positive and negative patient samples. Moreover, positive samples that amplified near the assay endpoint ( i.e., high time threshold) still displayed a clear melting curve peak consistent with being true positives ( Extended data: Figure S2A 18) and were distinguishable from false positives ( Extended data: Figure S2B 18). Therefore, when implementing the assay using a real-time PCR machine (thermocycler) with SYBR-based detection, incorporation of a traditional melting curve stage allows for elimination of false positives and increases reporting confidence for SARS-CoV-2 positive samples (see Extended data: Supplemental Methods 18).

To further assess the specificity of the N gene RT-LAMP assay, we first determined possible cross-reactivity with human RNA. 95 wells of RNA from a human cell line extracted by the CCC pipeline gave no signal in the assay ( Figure 3A). To assess the assay’s specificity for SARS-CoV-2, the RT-LAMP assay was performed on RNA purified from clinical samples from COVID-19 negative patients infected with a variety of RNA viruses, including seasonal coronaviruses strains 229E, NL63, OC43 and HKU1, MERS-CoV, Influenza A and B viruses, metapneumovirus (MPV), respiratory syncytial virus (RSV) or parainfluenza viruses type 3 and 4 (PIV3, PIV4). All samples gave a signal in the 18S rRNA RT-LAMP, confirming the presence of human RNA ( Figure 3B). With the N gene RT-LAMP, no signal was observed with any of the 14 distinct virus containing specimens tested, confirming that the assay is highly specific for SARS-CoV-2 ( Figure 3B).

Figure 3. SARS-CoV-2 RT-LAMP is highly specific.

Figure 3.

Open in a new tab

( A) SARS-CoV-2 RT-LAMP was performed on 95 wells of human cell line RNA extracted by the CCC pipeline. ( B–C) RT-LAMP targeting SARS-CoV-2 ( B) or 18S ( C) was performed on COVID-19 negative patient samples with positively identified with other viral infections, including human coronaviruses (HCoV), influenzas (Flu), respiratory syncytial virus (RSV), parainfluenzavirus (PIV), adenovirus (Adeno), metapneumovirus (MPV), rhinovirus (Rhino), and human enterovirus (H. Entero). The colorimetric read-out from the actual run is depicted below. NTC (-) and positive control (+).

RT-LAMP sensitivity and precision

In order to ascertain the sensitivity of the N gene RT-LAMP assay, RNA from a SARS-CoV-2 quantified gene copy number standard obtained from the UK National Institute for Biological Standards and Control (NIBSC) was extracted and assessed by limiting dilution. The results indicate that the limit of reliable detection of the N gene RT-LAMP assay is between 500–1000 copies of N gene ( Figure 4A and 4B) introduced in the assay. As the reaction is performed with 4.5 µL of sample, this assay presents a limit of detection of 1.1 - 2.2 × 10 5 N gene copies per mL. This was confirmed using 10-fold serial dilutions of RNA extracted from laboratory-grown SARS-CoV-2 quantified using the NIBSC standard ( Figure 4C). Notably, a highly linear response was observed, even in the presence of 1% Triton-X 100, which has been proposed as a means of inactivating SARS-CoV-2 19.

Figure 4. SARS-CoV-2 RT-LAMP is highly sensitive, robust, and precise.

Figure 4.

Open in a new tab

( A–B) NIBSC SARS-CoV-2 standard was serially diluted and the indicated number of copies in 4.5 µL was assessed by N gene RT-LAMP. Amplification curves shown with the limit of detection (L.O.D.) determined by the presence or absence of amplification following the depicted dilution in ( A) or via colorimetric read-out ( B). ( C) RNA extracted from laboratory grown SARS-CoV-2 was serially diluted 10-fold and assessed by RT-LAMP in the presence or absence of 1% Triton-X 100. ( D) A COVID-19 positive patient sample was RNA extracted independently 5 times through the CCC pipeline and subjected to 5 independent N gene RT-LAMP reactions. Assay precision for N gene and 18S was determined by calculating the coefficient of variation between Ct values observed (CV).

The RT-LAMP assay reproducibility and precision were determined by extracting RNA 5 times from a confirmed COVID-19 positive patient sample through the CCC pipeline and assessing by N gene and 18S RT-LAMP in 5 independent experiments, performed by two different operators ( Figure 4D). The coefficient of variation was 0.03151 for the N gene RT-LAMP analysis and 0.03595 for the 18S rRNA internal control. We also verified that RT-LAMP can be performed in a 384-well plate format using a QuantStudio 5 real-time PCR machine with equivalent results ( Extended data: Figure S3 18).

Clinical validation of RT-LAMP against two clinical diagnostic RT-qPCR assays

The RT-LAMP assay was benchmarked against the RT-qPCR methods used by the reference clinical diagnostic laboratory, HSL, and that used by the CCC validated clinical diagnostic platform by assessing 37 clinical samples processed in parallel by both laboratories with duplicate RT-qPCR analyses. RNA extracted by the CCC pipeline was then tested by RT-LAMP in duplicate runs. RT-LAMP detected 16 positives in both experiments, which was 100% concordant with results obtained by the CCC’s RT-qPCR assay using a CE marked kit from BGI and HSL’s N gene RT-qPCR assay. However, positives with Ct > 35 near the limit of detection (Ct = 37) of the BGI RT-qPCR assay, which were termed ‘borderline positives’, were not consistently detected by the N gene RT-LAMP assay ( Figure 5A). In an additional set of experiments, 71 clinical specimens in viral transport medium (VTM) stored at HSL were subjected to RNA extraction through the CCC pipeline and examined by HSL’s RT-qPCR and CCC’s RT-LAMP assay in duplicate ( Figure 5B). Again, samples with Ct < 39 in the HSL RT-qPCR analysis (which runs for 45 cycles as opposed to 40 in the BGI assay) were confidently detected by RT-LAMP, whilst samples with Ct values near the limit of detection (39 < Ct < 41), indicating very low levels of SARS-CoV-2 RNA, displayed inconsistent amplification by N gene RT-LAMP. Altogether, these data demonstrate that RT-LAMP performed on extracted RNA accurately detects SARS-CoV-2 in clinical samples as verified by a clinical diagnostic laboratory and a validated clinical diagnostic pipeline. However, samples with Ct values within two cycles of the limit of detection of either RT-qPCR assays, were not consistently identified in duplicate RT-LAMP runs, with detection for some samples in one run, but not always in the other. These results suggest that RT-LAMP is slightly less sensitive than RT-qPCR when performed with extracted RNA. However, it is important to bear in mind that the RT-LAMP assay was performed during these validation experiments with a third of the input RNA used by either RT-qPCR methods during these validation experiments (3 µL versus 10 µL), partly due to limitations in sample availability.

Figure 5. Clinical validation of SARS-CoV-2 RT-LAMP.

Figure 5.

Open in a new tab

( A) 37 patient samples were processed in parallel by HSL and the CCC pipeline and interrogated by HSL’s RT-qPCR (N gene) and the CCC’s BGI RT-qPCR (ORF1a) in duplicate. RNA left from the CCC pipeline was assessed in two separate experiments by N gene RT-LAMP. The graph indicates Ct values for HSL and CCC RT-qPCR runs by left y-axis and ‘Ct’ time thresholds for RT-LAMP via the right y-axis. Data are normalised by assay run/cycles determined by the two methods for comparative purposes. The clinical call from the reference laboratory is depicted above the summary table provided below. Positives (P) with low Ct values were reliably detected by RT-LAMP, whilst ‘borderline positive’ (samples with Ct values near the limit of detection of each RT-qPCR assay) were inconsistently detected, displaying no visible amplification at times – negative (N). ( B) 71 VTM samples from HSL were RNA-extracted through the CCC pipeline and interrogated by HSL RT-qPCR (45 cycles), or by two independent N gene RT-LAMP experiments (25 minutes

Discussion

Rapid and reliable detection of SARS-CoV-2 is required to efficiently diagnose infected individuals and to provide governments and health systems with guidance for treatment and quarantine strategies to reduce the risk of transmission. The CCC was formed to address a critical testing shortage in the London area, especially for frontline healthcare workers, with the additional goal of rapidly validating and disseminating SOPs for others to scale up their own diagnostic programs. The majority of testing is currently performed by RT-qPCR amplification of viral RNA obtained from nasopharyngeal samples. In the face of continued global demand and competition for reagents and resources and to minimise reliance on any singular testing strategy, we have outlined here procedures to utilise RT-LAMP as a cost-effective and high throughput alternative to RT-qPCR for detecting SARS-CoV-2 in clinical specimens.

Implementing this testing method would reduce the CCC’s operational costs ten-fold when accounting for all consumables (data not shown), provide the ability to scale up our output further ( Extended data: Figure S3 18), and decrease reporting turnaround time in comparison to our current method. We confirmed that the assay is highly specific for SARS-CoV-2 and displays lack of cross-reactivity with other respiratory viruses, including seasonal coronaviruses. Assay accuracy and robustness is denoted by an absence of false positives and the further introduction of a melt curve stage allows increased confidence in the identification and reporting of genuine positives. When performed with extracted RNA, the N gene RT-LAMP assay displays a sensitivity comparable to clinically approved RT-qPCR methods. Additionally, preliminary experiments demonstrate that RT-LAMP can be performed directly without RNA extraction, by inactivating virus-containing dry swabs in a detergent-based solution. However, these experiments did not make use of true nasopharyngeal swabs that are likely to contain material that can cause RNA degradation or inhibit the assay (unpublished observations).

Our methodology makes use of a real-time qPCR machine that measures DNA amplification using a SYBR based dye, allowing for real-time detection and standardised reporting. We did not test extensively if RT-LAMP could be assessed by the colorimetric indicator alone, although some of our data suggests that the colour change readout is concordant with the SYTO 9 dye results ( Figure 1, Figure 3 and Figure 4). If RT-LAMP were coupled with a colorimetric read-out and the need for RNA extraction could be obviated, the result would be a testing modality that tremendously reduces the cost and time for SARS-CoV-2 diagnostics and allow its application at point-of-care and in remote areas where sophisticated testing infrastructures currently do not exist.

Data availability

Underlying data

Open Science Framework: SARS-CoV-2 detection by a clinical diagnostic RT-LAMP assay; raw data for figures, https://doi.org/10.17605/OSF.IO/2XR8U 20.

Open Science Framework: SARS-CoV-2 detection by a clinical diagnostic RT-LAMP assay; raw data for supplementary figures, https://doi.org/10.17605/OSF.IO/YFCSW 21.

Extended data

Open Science Framework: SARS-CoV-2 detection by a clinical diagnostic RT-LAMP assay, supplementary figures and methods, https://doi.org/10.17605/OSF.IO/PAF36 18.

This project contains the following extended data:

  • -

    Figure S1. SARS-CoV-2 RT-LAMP and internal control 18S rRNA RT-LAMP display a consistent dissociation peak and melting temperature. (A) Dissociation curves are shown for 11 SARS-CoV-2 positive and 11 negative clinical samples along with 12 NTC wells tested by RT-LAMP N gene assay (top) and 18S assay (bottom). (B) Dot plot of the melting temperatures determined by 94 positive patient samples and 235 negative patient samples tested by RT-LAMP assay. Data are pooled from six separate experiments performed on 3 separate RT-PCR machines. The average melting temperature (Tm) is depicted by a dotted line in (A) and below the graph in (B) with the standard of deviation.

  • -

    Figure S2. Inclusion of a melt curve stage complements the ability of RT-LAMP to accurately distinguish true SARS-CoV-2 positive samples. (A) Amplification curves (top) and dissociation curves (bottom) are depicted from a range of positive samples for each assay along with a negative sample for reference. TP indicates a true positive call based on amplification above background plus consistent melting curve and Tm consistent with genuine positives. (B) Amplification curve (left) and dissociation curve (right) are depicted of positive samples and NTCs. TP indicates the same criteria outlined in (A), whereas FP denotes a false positive, which despite amplifying above background, does not have a melting curve and temperature consistent with genuine positives.

  • -

    Figure S3. RT-LAMP yields identical results across different RT-PCR instruments and 96 versus 384-well plate formats. Dot plot comparing results from RT-LAMP assays (N gene, top; 18S, bottom) on the same clinical positive and negative sample serially diluted 2-fold using an ABI 7500 Fast or QuantStudio 5 machine (96-well vs. 384-well plate format).

  • -

    Supplementary Method 1: RT-LAMP Mastermix Preparation

  • -

    Supplementary Method 2: RT-LAMP RNA Loading and Running RT-PCR Machine

  • -

    Supplementary Method 3: RT-LAMP Results Reporting

Data are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).

Acknowledgments

The authors wish to thank all of the CCC members and CAPTURE study members that have helped contribute reagents, advice, and time to this work.

The Crick COVID-19 Consortium (CCC): Jim Aitken 1, Zoe Allen 2, Rachel Ambler 1, Karen Ambrose 1, Emma Ashton 2, Alida Avola 1, Samutheswari Balakrishnan 1, Caitlin Barns-Jenkins 1, Genevieve Barr 1, Sam Barrell 1, Souradeep Basu 1, Rupert Beale 1, 3, Clare Beesley 2, Nisha Bhardwaj 1, Shahnaz Bibi 2, Ganka Bineva-Todd 1, Dhruva Biswas 3, Michael J. Blackman 1, 4, Dominique Bonnet 1, Faye Bowker 1, Malgorzata Broncel 1, Claire Brooks 2, Michael D. Buck 1, Andrew Buckton 2, Timothy Budd 1, Alana Burrell 1, Louise Busby 2, Claudio Bussi 1, Simon Butterworth 1, Fiona Byrne 1, Richard Byrne 1, Simon Caidan 1, Joanna Campbell 5, Johnathan Canton 1, Ana Cardoso 1, Nick Carter 1, Luiz Carvalho 1, Raffaella Carzaniga 1, Natalie Chandler 2, Qu Chen 1, Peter Cherepanov 1, Laura Churchward 6, Graham Clark 1, Bobbi Clayton 1, Clementina Cobolli Gigli 1, Zena Collins 1, Sally Cottrell 2, Margaret Crawford 1, Laura Cubitt 1, Tom Cullup 2, Heledd Davies 1, Patrick Davis 1, Dara Davison 1, Vicky Dearing 1, Solene Debaisieux 1, Monica Diaz-Romero 1, Alison Dibbs 1, Jessica Diring 1, Paul C. Driscoll 1, Annalisa D’Avola 1, Christopher Earl 1, Amelia Edwards 1, Chris Ekin 7, Dimitrios Evangelopoulos 1, 3, Rupert Faraway 1, 3, Antony Fearns 1, Aaron Ferron 1, Efthymios Fidanis 1 Dan Fitz 1, James Fleming 1, Bruno Frederico 1, Alessandra Gaiba 1, Anthony Gait 2, Steve Gamblin 1, Sonia Gandhi 1, 3, 6, Liam Gaul 1, Helen M. Golding 1, Jacki Goldman 1, Robert Goldstone 1, Belen Gomez Dominguez 2, Hui Gong 1, Paul R. Grant 7, Maria Greco 1, Mariana Grobler 2, Anabel Guedan 1, Maximiliano G. Gutierrez 1, Fiona Hackett 1, Ross Hall 1, Steinar Halldorsson 1, Suzanne Harris 1, Sugera Hashim 2, Lyn Healy 1, Judith Heaney 6, Susanne Herbst 1, Graeme Hewitt 1, Theresa Higgins 1, Steve Hindmarsh 1, Rajnika Hirani 1, Joshua Hope 1, Elizabeth Horton 1, Beth Hoskins 2, Catherine F. Houlihan 6, Michael Howell 1, Louise Howitt 1, Jacqueline Hoyle 2, Mint R. Htun 1, Michael Hubank 8, 9, Hector Huerga Encabo 1, Deborah Hughes 8, Jane Hughes 1, Almaz Huseynova 1, Ming-Shih Hwang 1, Rachael Instrell 1, Deborah Jackson 1, Mariam Jamal-Hanjani 3, 6, Lucy Jenkins 2, Ming Jiang 1, Mark Johnson 1, Leigh Jones 1, Nnennaya Kanu 3, George Kassiotis 1, Louise Kiely 2, Anastacio King Spert Teixeira 1, Stuart Kirk 7, Svend Kjaer 1, Ellen Knuepfer 1, 12, Nikita Komarov 1, 3, Paul Kotzampaltiris 5, Konstantinos Kousis 1, Tammy Krylova 1, Ania Kucharska 1, Robyn Labrum 5, Catherine Lambe 1, Michelle Lappin 1, Stacey-Ann Lee 1, Andrew Levett 7, Lisa Levett 7, Marcel Levi 6, Hon Wing Liu 1, Sam Loughlin 2, Wei-Ting Lu 1, James I. MacRae 1, Akshay Madoo 1, Julie A. Marczak 1, Mimmi Martensson 1, Thomas Martinez 1, Bishara Marzook 1, John Matthews 7, Joachim M. Matz 1, Samuel McCall 5, Laura E. McCoy 3, Fiona McKay 2, Edel C. McNamara 1, Carlos M. Minutti 1, Gita Mistry 1, Miriam Molina-Arcas 1, Beatriz Montaner 1, Kylie Montgomery 2, Catherine Moore 10, David Moore 3, 6, Anastasia Moraiti 1, Lucia Moreira-Teixeira 1, Joyita Mukherjee 1, Cristina Naceur-Lombardelli 3, Eleni Nastouli 6, 11, Aileen Nelson 1, Jerome Nicod 1, Luke Nightingale 1, Stephanie Nofal 1, Paul Nurse 1, Savita Nutan 2, Caroline Oedekoven 1, Anne O’Garra 1, Jean D. O’Leary 1, Jessica Olsen 1, Olga O’Neill 1, Nicola O'Reilly 1, Paula Ordonez Suarez 1, Neil Osborne 1, Amar Pabari 7, Aleksandra Pajak 1, Venizelos Papayannopoulos 1, Namita Patel 1, Yogen Patel 5, Oana Paun 1, Nigel Peat 1, Laura Peces-Barba Castano 1, Ana Perez Caballero 2, Jimena Perez-Lloret 1, Magali S. Perrault 1, Abigail Perrin 1, Roy Poh 5, Enzo Z. Poirier 1, James M. Polke 5, Marc Pollitt 1, Lucia Prieto-Godino 1, Alize Proust 1, Clinda Puvirajasinghe 2, Christophe Queval 1, Vijaya Ramachandran 2, Abhinay Ramaprasad 1, Peter Ratcliffe 1, Laura Reed 2, Caetano Reis e Sousa 1, Kayleigh Richardson 1, Sophie Ridewood 1, Fiona Roberts 1, Rowenna Roberts 2, Angela Rodgers 1, Pablo Romero Clavijo 1, Annachiara Rosa 1, Alice Rossi 1, Chloe Roustan 1, Andrew Rowan 1, Erik Sahai 1, Aaron Sait 1, Katarzyna Sala 1, Theo Sanderson 1, Pierre Santucci 1, Fatima Sardar 1, Adam Sateriale 1, Jill A. Saunders 1, Chelsea Sawyer 1, Anja Schlott 1, Edina Schweighoffer 1, Sandra Segura-Bayona 1, Rajvee Shah Punatar 1, Joe Shaw 2, Gee Yen Shin 6, 7, Mariana Silva Dos Santos 1, Margaux Silvestre 1, Matthew Singer 1, Daniel M. Snell 1, Ok-Ryul Song 1, Moira J. Spyer 3, Louisa Steel 2, Amy Strange 1, Adrienne E. Sullivan 1, Charles Swanton 1, 3, 6, Michele S.Y. Tan 1, Zoe H. Tautz-Davis 1, Effie Taylor 1, Gunes Taylor 1, Harriet B. Taylor 1, Alison Taylor-Beadling 2, Fernanda Teixeira Subtil 1, Berta Terré Torras 1, Patrick Toolan-Kerr 1, 3, Francesca Torelli 1, Tea Toteva 1, Moritz Treeck 1, Hadija Trojer 13, Ming-Han C. Tsai 1, James M.A.Turner 1, Melanie Turner 7, Jernej Ule 1, 3, Rachel Ulferts 1, Sharon P. Vanloo 1, 3, Selvaraju Veeriah 3, Subramanian Venkatesan 1, Karen Vousden 1, Andreas Wack 1, Claire Walder 2, Philip A. Walker 1, Yiran Wang 1, Sophia Ward 1, 3, Catharina Wenman 2, Luke Williams 1, Matthew J. Williams 1, Wai Keong Wong 6, Joshua Wright 1, Mary Wu 1, Lauren Wynne 1, Zheng Xiang 1, Melvyn Yap 1, Julian A. Zagalak 1, 3, Davide Zecchin 1, 11 and Rachel Zillwood 1

1The Francis Crick Institute, London NW1 1AT, UK

2Great Ormond Street Hospital for Sick Children NHS Foundation Trust, London WC1N 3JH, UK.

3University College London, London WC1E 6BT, UK

4London School of Hygiene & Tropical Medicine, London WC1E 7HT, UK

5National Hospital for Neurology and Neurosurgery, University College London Hospitals,

NHS Foundation Trust, London WC1N 3BG, UK

6University College London Hospitals, NHS Foundation Trust, London NW1 2PG, UK

7Health Services Laboratories, London WC1H 9AX, UK

8The Institute of Cancer Research, London SW7 3RP, UK

9The Royal Marsden Hospital, Surrey SM2 5NG, UK

10Public Health Wales, Heath Park, Cardiff CF14 4XW, UK

11University College London GOS Institute of Child Health, London WC1N 1EH, UK

12Department of Pathobiology and Population Sciences, Royal Veterinary College, University of London, London NW1 0TU, UK

13Royal Free London NHS Foundation Trust, London NW3 2QG, UK

CAPTURE study: Nicky Browne 1, Kim Edmonds 1, Yasir Khan 1, Sarah Sarker 1, Ben Shum 1, Tim Slattery 1, & Samra Turajlic 1, 2

1The Royal Marsden Hospital, Surrey, SM2 5NG, UK

2The Francis Crick Institute, London NW1 1AT, UK

A previous version of this article is published on medRxiv: https://doi.org/10.1101/2020.06.29.20142430.

Funding Statement

This work was supported by The Rosetrees Trust and The John Black Charitable Foundation, as well as the Francis Crick Institute, which receives its funding from the UK Research Medical Council (FC001169, FC001078), Cancer Research UK (FC001169, FC001078) and the Wellcome Trust (FC001169, FC001078). This research was funded in part by the Wellcome Trust (FC001136). E. Nastouli is funded by MRC, NIHR, GSK and H2020. S. Gandhi has an MRC Senior Clinical Fellowship. E.Z. Poirier and M.D. Buck are supported by EMBO Long-Term Fellowships (ALTF 536-2018 and ALTF 1096-2018) and Marie Skłodowska-Curie Individual Fellowships (832511 and 837951).

The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

[version 1; peer review: 2 approved with reservations]

References

  • 1. Zhu N, Zhang D, Wang W, et al. : A Novel Coronavirus from Patients with Pneumonia in China, 2019. N Engl J Med. 2020;382(8):727–33. 10.1056/NEJMoa2001017 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2. Wu F, Zhao S, Yu B, et al. : A new coronavirus associated with human respiratory disease in China. Nature. 2020;579(7798):265–9. 10.1038/s41586-020-2008-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3. Zhou P, Yang XL, Wang XG, et al. : A pneumonia outbreak associated with a new coronavirus of probable bat origin. Nature. 2020;579(7798):270–3. 10.1038/s41586-020-2012-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4. Yan Y, Chang L, Wang L: Laboratory testing of SARS-CoV, MERS-CoV, and SARS-CoV-2 (2019-nCoV): Current status, challenges, and countermeasures. Rev Med Virol. 2020;30(3):e2106. 10.1002/rmv.2106 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5. Aitken J, Ambrose K, Barrell S, et al. : Scalable and Resilient SARS-CoV-2 testing in an Academic Centre. Health Systems and Quality Improvement. 2020. 10.1101/2020.04.19.20071373 [DOI] [Google Scholar]
  • 6. Notomi T, Okayama H, Masubuchi H, et al. : Loop-mediated isothermal amplification of DNA. Nucleic Acids Res. 2000;28(12):E63. 10.1093/nar/28.12.e63 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7. Mori Y, Notomi T: Loop-mediated isothermal amplification (LAMP): a rapid, accurate, and cost-effective diagnostic method for infectious diseases. J Infect Chemother. 2009;15(2):62–9. 10.1007/s10156-009-0669-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8. Ahn SJ, Baek YH, Lloren KKS, et al. : Rapid and simple colorimetric detection of multiple influenza viruses infecting humans using a reverse transcriptional loop-mediated isothermal amplification (RT-LAMP) diagnostic platform. BMC Infect Dis. 2019;19(1):676. 10.1186/s12879-019-4277-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9. Lamb LE, Bartolone SN, Tree MO, et al. : Rapid Detection of Zika Virus in Urine Samples and Infected Mosquitos by Reverse Transcription-Loop-Mediated Isothermal Amplification. Sci Rep. 2018;8(1):3803. 10.1038/s41598-018-22102-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10. Zhang Y, Odiwuor N, Xiong J, et al. : Rapid Molecular Detection of SARS-CoV-2 (COVID-19) Virus RNA Using Colorimetric LAMP. Infectious Diseases (except HIV/AIDS). 2020. 10.1101/2020.02.26.20028373 [DOI] [Google Scholar]
  • 11. Rabe BA, Cepko C: SARS-CoV-2 detection using isothermal amplification and a rapid, inexpensive protocol for sample inactivation and purification. Proc Natl Acad Sci. 2020;117(39):24450–8. 10.1073/pnas.2011221117 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12. Dao Thi VL, Herbst K, Boerner K, et al. : A colorimetric RT-LAMP assay and LAMP-sequencing for detecting SARS-CoV-2 RNA in clinical samples. Sci Transl Med. 2020;12(556):eabc7075. 10.1126/scitranslmed.abc7075 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13. Fowler VL, Armson B, Gonzales JL, et al. : A highly effective reverse-transcription loop-mediated isothermal amplification (RT-LAMP) assay for the rapid detection of SARS-CoV-2 infection. J Infect. 2020; S0163-4453(20)30731-3. 10.1016/j.jinf.2020.10.039 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14. Pang B, Xu J, Liu Y, et al. : Isothermal Amplification and Ambient Visualization in a Single Tube for the Detection of SARS-CoV-2 Using Loop-Mediated Amplification and CRISPR Technology. Anal Chem. 2020;92(24):16204–16212. 10.1021/acs.analchem.0c04047 [DOI] [PubMed] [Google Scholar]
  • 15. Taki K, Yokota I, Fukumoto T, et al. : SARS-CoV-2 detection by fluorescence loop-mediated isothermal amplification with and without RNA extraction. J Infect Chemother. 2020; S1341-321X(20)30399-8. 10.1016/j.jiac.2020.10.029 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16. Yoshikawa R, Abe H, Igasaki Y, et al. : Development and evaluation of a rapid and simple diagnostic assay for COVID-19 based on loop-mediated isothermal amplification. PLoS Negl Trop Dis. 2020;14(11):e0008855. 10.1371/journal.pntd.0008855 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17. Lalli MA, Langmade SJ, Chen X, et al. : Rapid and extraction-free detection of SARS-CoV-2 from saliva by colorimetric reverse-transcription loop-mediated isothermal amplification. Clin Chem. 2020;hvaa267. 10.1093/clinchem/hvaa267 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18. Poirier E: SARS-CoV-2 detection by a clinical diagnostic RT-LAMP assay.2021. 10.17605/OSF.IO/PAF36 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19. Patterson EI, Prince T, Anderson ER, et al. : Methods of inactivation of SARS-CoV-2 for downstream biological assays. Microbiology. 2020. 10.1101/2020.05.21.108035 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20. Poirier E: SARS-CoV-2 detection by a clinical diagnostic RT-LAMP assay.2021. 10.17605/OSF.IO/2XR8U [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21. Poirier E: SARS-CoV-2 detection by a clinical diagnostic RT-LAMP assay.2021. 10.17605/OSF.IO/YFCSW [DOI] [PMC free article] [PubMed] [Google Scholar]
Wellcome Open Res. 2021 Mar 30. doi: 10.21956/wellcomeopenres.18193.r43084

Reviewer response for version 1

Leyi Wang 1, Vanessa Revindran-Stam 1

Till March 23, 2021, SARS-COV-2 global pandemic has caused infection of over 124 million and over 2.7 million deaths. It is critically important to mitigate transmission through detection and isolation. The current pandemic has made it widely important for fast, reliable and relatively cheap, alternative methods of detecting the SARS-CoV-2 infection. Numerous diagnostic methods for detection of virus genome and antigens have been developed. Molecular methods are most commonly used due to a higher level of sensitivity. The manuscript by Buck et al. reports evaluation of a clinical diagnostic RT-LAMP assay for detection of SARS-CoV-2. The authors performed sensitivity, specificity, and clinical validation. The authors used NEB LAMP reagent and one set of N gene primers and 18S primers reported from previous studies. They used 24 RNA samples with half positive for initial validation. Through testing water and RNA elution buffer, they optimized the assay with reduced reaction time to minimize background noise. They tested different species of human coronaviruses, and other human viruses and showed no cross reaction with non-SARS-CoV-2 viruses. They also determined the LOD of the assay and performed clinical validation on 37 and 71 samples tested by other labs. To address the false positive of RT-LAMP, evaluation of melting temperature was introduced in the procedure, however, there is no approach taken to address false negative due to the relative lower sensitivity compared with real-time qPCR.

Major comments:

  1. A conventional real-time PCR enables multiplex capability, how can the validated assay of running two separate PCRs for one sample improve the workflow? Based on the data (Figure 5A and 5B), it seems switching from real-time PCR to LAMP could potentially lose sensitivity, as RT-LAMP missed weak positives. In addition, the incubation time for 18S is shorter than that for SARS-CoV-2, which means not same procedure used and 18S may not serve well as an internal control. The abundance of 18S (Figure 1D) may not reflect the presence of any inhibition in the test. Please clarify.

  2. As two sets of N gene primers in the reference 10 (Zhang et al.), this study uses one set of them, have the authors tested both of them? While the N gene is highly conserved among different coronaviruses, what is the sensitivity/reliability of using one set of primers? Other publications have noted the use of multiple primer sets to increase sensitivity of the RT-LAMP assay and especially with the increasing discovery of variants of SARS-CoV, it would be crucial to include a combination of target genes.

  3. In the pages 3 and 9, the RT-LAMP assay procedure, 3 µL of the RNA sample is used, however, in page 8 of the manuscript file, and pages 3 and 5 of the Supplementary Method file, 4.5 µL of the RNA was used. It is kind of confused. What correct amount is used in the procedure?

  4. In the Introduction section, they mentioned that the LAMP assay alleviates the need for a thermocycler qPCR system, as the processing time on machine is less 30 minutes. The sentence needs to be rewritten. Instead using a dry bath in the Reference 10, this study still uses thermocycler. qPCR using the new fast chemistry (like TaqCheck) on QuantStudio, only takes 43 minutes on PCR machine. Therefore, the developed procedure may not help much to alleviate the need for thermocyclers.

  5. This study only has a very low number (37 and 71, a total of 108) of the samples used for clinical validation of RT-LAMP, compared with their large testing volume completed by RT-qPCR. This part needs including much more samples for the clinical validation.

  6. On numerous occasions, the authors refer to the RNA extraction through the CCC pipeline. There is missing extraction method in the materials and methods section.

Minor comments:

  1. In Page 3, the authors stated they provided the preliminary data about running RT-LAMP on samples without RNA extraction, where are these data?

  2. In the page 16 of Supplementary Method file, result interpretation table did not include possibility of real SARS-CoV-2 positive, but Internal control negative. Please clarify it.

  3. It is confused that Figure 1B has 4 positive and 4 negative, but Figure 1C-1E are 12 positive and 12 negative. Please include colorimetric readout result for all 12 samples. There are missing Ct values of these 12 positive samples determined by RT-qPCR.

  4. Figure 2: The pink dotted line in 2A, B, D and E should be identified.

  5. Figure 3, C was not labeled/marked for 18S graph. In addition, in the colorimetric run for the RT-LAMp 18S rRNA, the 5th well looks to be pink/red and not yellow, indicating that one sample did not give signal.

  6. For colorimetric read-out results, inconsistent data format was presented as Figure 1B was tubes containing the whole reaction, however, Figure 3B and 3C, and Figure 4 B were bottom part of tube, plate, or others? These need clearly indication in the figure legend.

  7. Figure 4, 1% Triton-X 100 was shown. It is unclear which step Triton-X 100 was used and what is the purpose of using it.

Is the rationale for developing the new method (or application) clearly explained?

Yes

Is the description of the method technically sound?

Yes

Are the conclusions about the method and its performance adequately supported by the findings presented in the article?

Yes

If any results are presented, are all the source data underlying the results available to ensure full reproducibility?

Yes

Are sufficient details provided to allow replication of the method development and its use by others?

Partly

Reviewer Expertise:

Molecular virology and clinical diagnostics.

We confirm that we have read this submission and believe that we have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however we have significant reservations, as outlined above.

Wellcome Open Res. 2021 Apr 28.
Enzo Poirier 1

We thank all reviewers for their constructive comments and suggestions. We have addressed their points with additional data and revisions to the text, as detailed below. We hope that the reviewers will find the revised manuscript improved and acceptable for publication.

Major comments:

 

  1. A conventional real-time PCR enables multiplex capability, how can the validated assay of running two separate PCRs for one sample improve the workflow? Based on the data (Figure 5A and 5B), it seems switching from real-time PCR to LAMP could potentially lose sensitivity, as RT-LAMP missed weak positives. In addition, the incubation time for 18S is shorter than that for SARS-CoV-2, which means not same procedure used and 18S may not serve well as an internal control. The abundance of 18S (Figure 1D) may not reflect the presence of any inhibition in the test. Please clarify.

    We have modified the manuscript to include further discussion of the advantages and disadvantages of the outlined RT-LAMP assay in comparison to conventional RT-qPCR.

    Switching from RT-PCR to RT-LAMP would result in a loss in sensitivity. However, the loss in sensitivity is less than what is experienced for currently approved antigen tests and the assay provides other advantages. Supplies of RT-LAMP reagents, especially in the beginning of the pandemic, were more readily available, easier to acquire and markedly less expensive, than RT-PCR reagents due to less demand and markedly less expensive. The assay can also be performed without a thermocycler if need be.

    In terms of the workflow, the assay can be run on either 96 or 384-well plate format with a thermocycler and both the SARS-CoV-2 and internal control reactions can be ran on the sample plate for the same amount of time as amplification results are only assessed at a defined data even if data is collected beyond that time point (e.g. N-gene assay, amplification before 25 minutes and 18S rRNA assay, amplification before 20 minutes).

  2. As two sets of N gene primers in the reference 10 (Zhang et al.), this study uses one set of them, have the authors tested both of them? While the N gene is highly conserved among different coronaviruses, what is the sensitivity/reliability of using one set of primers? Other publications have noted the use of multiple primer sets to increase sensitivity of the RT-LAMP assay and especially with the increasing discovery of variants of SARS-CoV, it would be crucial to include a combination of target genes.

    We did initial tests with different primer sets that were originally denoted in Zhang et al. and have included this data in Supplement Figure 1 with an explanation of our rationale for moving forward with the particular N gene primers that we used throughout the rest of our studies.

    In regards to current questions of reliability in light of new variants, Reviewer #1 had similar concerns and we have now included analyses that compare the mutational rate in the specific sequences that these primers target (see Supplemental Figure 2).

    As mentioned with Reviewer #1, we have now included discussion of various strategies, including the use of multiple primer pairs, to increase the sensitivity of the RT-LAMP assay in the revised manuscript.

  3. In the pages 3 and 9, the RT-LAMP assay procedure, 3 µL of the RNA sample is used, however, in page 8 of the manuscript file, and pages 3 and 5 of the Supplementary Method file, 4.5 µL of the RNA was used. It is kind of confused. What correct amount is used in the procedure?

    We apologise for the confusion: the standard RNA input amount is 4.5 µL, however some experiments were performed using 3 µL of input due to limitations in sample availability. Experiments that used 3 µL as opposed to the standard amount have been denoted in the corresponding figure legends and text.

  4. In the Introduction section, they mentioned that the LAMP assay alleviates the need for a thermocycler qPCR system, as the processing time on machine is less 30 minutes. The sentence needs to be rewritten. Instead using a dry bath in the Reference 10, this study still uses thermocycler. qPCR using the new fast chemistry (like TaqCheck) on QuantStudio, only takes 43 minutes on PCR machine. Therefore, the developed procedure may not help much to alleviate the need for thermocyclers.

    We have modified the specific sentence mentioned from the introduction and have expanded our discussion in the revised text of the advantages and disadvantages of RT-LAMP. There are still instances where access to a thermocycler may not be readily available and a procedure that can be carried out without one if necessary (RT-LAMP) would be a valued asset. We have mentioned this fact in our revised discussion.

  5. This study only has a very low number (37 and 71, a total of 108) of the samples used for clinical validation of RT-LAMP, compared with their large testing volume completed by RT-qPCR. This part needs including much more samples for the clinical validation.

    We acknowledge that increasing the number of clinical samples tested by RT-LAMP would allow more robust clinical validation of the assay’s performance. However, as seen in our response to Reviewer #1, point 7, the assay was ultimately not adopted for use at the institute. We have included discussion of this particular issue in the revised manuscript.

  6. On numerous occasions, the authors refer to the RNA extraction through the CCC pipeline. There is missing extraction method in the materials and methods section.

    We apologise for not including this information in the original submission and have amended the methods accordingly.

Minor comments:

 

  1. In Page 3, the authors stated they provided the preliminary data about running RT-LAMP on samples without RNA extraction, where are these data?

    We apologise that these data were not included in the original submission. They now appear in new Supplemental Figure 6.

  2. In the page 16 of Supplementary Method file, result interpretation table did not include possibility of real SARS-CoV-2 positive, but Internal control negative. Please clarify it.

    Please see our response to Reviewer #1, point 12, first paragraph.

  3. It is confused that Figure 1B has 4 positive and 4 negative, but Figure 1C-1E are 12 positive and 12 negative. Please include colorimetric readout result for all 12 samples. There are missing Ct values of these 12 positive samples determined by RT-qPCR.

    As the primary readout in Figure 1C-E was Syto9 amplification, we do not have the corresponding images demonstrating the colorimetric conversions.

    The Ct values from Figures 1C and 1D are displayed in Figure 1E (grey coloured dots, refer to left y-axis) as indicated in the figure legend.

  4. Figure 2: The pink dotted line in 2A, B, D and E should be identified.

    We have removed the pink dotted line, which referred to the threshold set for the assay, from the indicated figures as its depiction is not necessary.

  5. Figure 3, C was not labeled/marked for 18S graph. In addition, in the colorimetric run for the RT-LAMp 18S rRNA, the 5th well looks to be pink/red and not yellow, indicating that one sample did not give signal.

    We thank the reviewer for their careful eye and have corrected the corresponding figure legend.

    As for the 5 th well that appears lighter in hue compared to other samples, we have explained the discrepancy in our response to Reviewer #1 (see response 17, paragraph 3 and response 19)

  6. For colorimetric read-out results, inconsistent data format was presented as Figure 1B was tubes containing the whole reaction, however, Figure 3B and 3C, and Figure 4 B were bottom part of tube, plate, or others? These need clearly indication in the figure legend.

    We have amended the indicated figure legends as suggested to clarify how our colorimetric data are presented.

  7. Figure 4, 1% Triton-X 100 was shown. It is unclear which step Triton-X 100 was used and what is the purpose of using it.

    Please see our response to Reviewer #1, point 20.

Wellcome Open Res. 2021 Feb 15. doi: 10.21956/wellcomeopenres.18193.r42423

Reviewer response for version 1

Viet Loan Dao Thi 2, Michael Knop 1

Summary:

In the ongoing pandemic, alternative methods to detect SARS-CoV-2 infections at a scalable, faster, and cheaper level, are still in need. In this manuscript, Buck et al. report on the use of the NEB warmstart enzyme preparation for colorimetric reverse transcription loop-mediated isothermal amplification (RT-LAMP) in combination with primers published by NEB (Zhang et al.) for the amplification and detection of SARS-CoV-2 viral RNA in clinical samples. For detection, a colorimetric implementation of the RT-LAMP assay (based on a pH shift and color change of an indicator dye upon amplification of nucleic acids) along with a fluorescent dye for DNA detection was used. The assay was tested for COVID-19 diagnostics and the report includes data on sensitivity, specificity as well as a positive sample control reaction. All established procedures were set up in agreement with the requirements for patient diagnostics, including SOPs for all relevant procedures. The authors furthermore claim that the cost of the assays would be a fraction of the ones used for RT-qPCR.

Major points:

The authors use colorimetric LAMP reagents but then add a green fluorescent dye for the detection of the fluorescence signal using a light cycler throughout their experiments, with the exception of the few figure panels where they show color images of the reaction tubes? Is this correct?

Importantly, the authors advertise the speed of the RT-LAMP assay in the abstract (25 min). This is somewhat misleading to non-expert readers who might think that the total time of the procedure (from sample taking to result) takes only that long. But the time to isolate the RNA and to set up the assays also needs to be considered.

The preprint in its current form appeared online on July 2020, and the authors claim clinical validation of the assay, higher speed, and almost similar sensitivity to qPCR at much lower costs. Given these advantages, may I ask whether the assay has been used for SARS-CoV-2 diagnostics? If yes, for how many samples? It would be essential to present such data including statistics from validation of LAMP positives by qPCR. On the other hand, if the assay was not used for diagnostics, it would be good to carefully explain the reasons behind it; otherwise, this may raise concerns about the validity of some of the data and acclaimed features of the assay.

Average RT-qPCR assays using well-established primer sets meanwhile reach sensitivities where 10 molecules of the viral genome are detected in 95% of the cases, which for most RT-qPCR kits corresponds to a CT approx. = 37. This can be considered as standard. Is this given for the used PCR assay (CRICK BGI)?

The manuscript does not elaborate on the used RNA isolation procedures. This should be included in the methods section. Different RNA extraction protocols exist, either using silica matrix columns or magnetic beads. These protocols allow purification and simultaneous concentration of nucleic acids relative to the original specimen. For diagnostic analysis of nucleic acids in SWAB specimen, a concentration factor of 2 to 5 usually applies. This yields a LOD of approx. 500 or fewer viral RNA molecules per ml of patient specimen for an RT-qPCR that uses 10 μl input volume. The authors here claim a sensitivity of the RT-LAMP of 11000-22000 molecules per ml patient specimen using a SARS-CoV-2 standard (Figure 4), while they claim a sensitivity up to Ct=35 with patient samples (Figure 5a). A Ct=35 in an RT-qPCR assay corresponds roughly to 40 molecules per 10 μl purified RNA, while only 3 μl are used for the RT-LAMP, which would imply a sensitivity limit of approx. 15 molecules per reaction. This is inconsistent with the data shown in Figure 4 and with previous reports for the N-gene LAMP primer sets used here, which was reported to have a sensitivity in the range of 100 molecules per reaction (e.g. 1). A closer inspection of the data shown in Figure 5a indicates that all RT-LAMP positive samples have CT<<32 (Figure 4a), but not enough samples in the CT range of 25 – 35 are shown to confidently estimate a CT value that corresponds to the LOD. These issues should be clarified and, if necessary, faulty standard calibrations or additional means to calibrate the sensitivity should be used.

The authors represent a consortium that has generated more than 252000 RT-qPCR results since the beginning of the pandemic. Obviously, knowing the exact sensitivity limit of the RT-LAMP assay and the data of the RT-qPCR would permit an estimation of the false-negative rate of the RT-LAMP assay in a much more realistic scenario as compared to the limited number of samples presented here.

The assay makes use of one primer set only for SARS-CoV-2 detection. Meanwhile, many more primer sets have been described, and a publication by NEB scientists 2 furthermore presented means to improve the sensitivity of the assay by adding GuHCL to the reaction buffer and by using multiple primer sets in combination. This could be discussed and the benefits of using better primers/conditions to be estimated.

Specificity: The authors elude that the RT-LAMP assay does not cross-react with other viruses. If the authors really want to use this experiment as a valid assay to demonstrate the sensitivity of the RT-LAMP assay for SARS-CoV-2, they should provide evidence that the used samples indeed contain viral nucleic acids in sufficient numbers (> 10000 molecules per ml) of the patient specimen. Another issue that needs to be addressed is the relatively well-established propensity of RT-LAMP to produce false-positive results by spurious amplification (e.g. see 3). The frequency of these may be small, i.e. below 1%, and hence such cases escaped detection in this work. As shown by the authors, melting curves allow to identify those, but upscaling and the use of the colorimetric RT-LAMP will certainly bear the risk of false positives. Here, validation of positives by RT-qPCR could be used.

Additional points:

The authors mention twice preliminary data on an RNA-extraction free procedure. Where are these data? I suggest either adding the data or removing the corresponding sentences in the introduction and discussion.

The number of cells in a swab can be affected by many factors. Even in the absence of human cells/detectable housekeeping genomes, viral RNA can be or should be present in a positive sample in the form of cell-free, viral particles. How do the authors suggest proceeding if 18S rRNA is not detected in a sample? How does the detection of the housekeeping gene affect the analysis of the clinical samples (e.g. in Figure 5)?

For example, in Figure 3B. The fifth sample appears to be 18S rRNA negative, as the reaction is still pink. How is such a result taken into account?

The citations are not exhaustive. Many manuscripts have been published meanwhile that make use of the colorimetric RT-LAMP assay for COVID-19 diagnostics. In addition, a description of the major differences or similarities to previously published work is missing.

The manuscript would benefit from a report on the frequency of mutations in the binding sequences of the primer set used, which could lead to false negatives. This can easily be done using online resources ( https://www.gisaid.org or https://covidcg.org) and the great number of viral genomes sequenced in the UK.

The authors need to report on measures to avoid contamination by LAMP products. This is an important point that may be less relevant for a diagnostic setting (since people are aware of this issue), but in an academic environment, researchers may need a reminder, which could be given for example by pointing out relevant literature 4 or by indicating strategies to eliminate those in the first place (UNG/UDG), or both.

Specific comments to the Figures:

Figure 1: Suggest moving primer sequences to a separate table.

Figure 2: Panels D and E are redundant with Panels A and B as the threshold is indicated by the grey background (just add an explanation of the threshold to the legend). Panel C is also redundant and difficult to read, the information is already in A and B. Give an explanation of the false negatives on the 18S control, which already give fluorescent signals at 20 mins into the LAMP reaction. The primer design seems suboptimal as spurious amplification usually occurs at later time points. At what time point did the authors analyze the colorimetric read-out for 18S, e.g. in Fig 3B. Please define up to when is the reaction reliable.

Suggest moving the melting curves from S1A to the main figures. When calculating sensitivity thresholds-does the melting curve change anything? Does it improve the sensitivity?

Figure 3B, right panel: All different samples are just blue lines. Like this, it is impossible for the reader to understand which line corresponds to which sample. As the coloration is somewhat different between the samples as well as the time points where the fluorescent signal becomes positive, it would be good to see if some sort of correlation exists or not.

Figure 4 C: I do not understand the reason behind treating already purified SARS-CoV-2 RNA with Triton X. This only makes sense in an extraction-free setting.

Figure 4 D: When determining the RT-LAMP assay precision, why extracting the RNA 5 times independently and then running the assay? I expect higher variations from the RNA extraction than from the RT-LAMP assay itself, which should be highly stable and reproducible on purified samples unless the sample contains only very few amounts of target RNA and the detection can become stochastic.

Is the rationale for developing the new method (or application) clearly explained?

Yes

Is the description of the method technically sound?

Yes

Are the conclusions about the method and its performance adequately supported by the findings presented in the article?

Yes

If any results are presented, are all the source data underlying the results available to ensure full reproducibility?

Yes

Are sufficient details provided to allow replication of the method development and its use by others?

Yes

Reviewer Expertise:

Molecular virology and diagnostics.

We confirm that we have read this submission and believe that we have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however we have significant reservations, as outlined above.

References

  • 1. : Rapid Molecular Detection of SARS-CoV-2 (COVID-19) Virus RNA Using Colorimetric LAMP. medRxiv.2020; 10.1101/2020.02.26.20028373 10.1101/2020.02.26.20028373 [DOI] [Google Scholar]
  • 2. : Enhancing Colorimetric LAMP Amplification Speed and Sensitivity with Guanidine Chloride. bioRxiv.2020; 10.1101/2020.06.03.132894 10.1101/2020.06.03.132894 [DOI] [PubMed] [Google Scholar]
  • 3. : High-surety isothermal amplification and detection of SARS-CoV-2, including with crude enzymes. bioRxiv.2020; 10.1101/2020.04.13.039941 10.1101/2020.04.13.039941 [DOI] [Google Scholar]
  • 4. : Amplicon contamination in labs masquerades as COVID19 in surveillance tests. medRxiv.2020; 10.1101/2020.12.08.20244525 10.1101/2020.12.08.20244525 [DOI] [Google Scholar]
Wellcome Open Res. 2021 Apr 28.
Enzo Poirier 1

We thank all reviewers for their constructive comments and suggestions. We have addressed their points with additional data and revisions to the text, as detailed below. We hope that the reviewers will find the revised manuscript improved and acceptable for publication.

Major points:

 

  1. The authors use colorimetric LAMP reagents but then add a green fluorescent dye for the detection of the fluorescence signal using a light cycler throughout their experiments, with the exception of the few figure panels where they show color images of the reaction tubes? Is this correct?

    Yes, as the reviewers have stated, we used colorimetric reagents combined with a green fluorescent DNA intercalating dye with amplification read-out by using a real-time PCR machine. Some figure panels feature the colorimetric conversion from these experiments.

  2. Importantly, the authors advertise the speed of the RT-LAMP assay in the abstract (25 min). This is somewhat misleading to non-expert readers who might think that the total time of the procedure (from sample taking to result) takes only that long. But the time to isolate the RNA and to set up the assays also needs to be considered.

    We thank the reviewer for their comment and have excluded the mention of the assay run time to help avoid confusion between the run-time of the RT-LAMP assay versus total procedure time from sample collection, RNA extraction, RT-LAMP, and results reporting.

  3. The preprint in its current form appeared online on July 2020, and the authors claim clinical validation of the assay, higher speed, and almost similar sensitivity to qPCR at much lower costs. Given these advantages, may I ask whether the assay has been used for SARS-CoV-2 diagnostics? If yes, for how many samples? It would be essential to present such data including statistics from validation of LAMP positives by qPCR. On the other hand, if the assay was not used for diagnostics, it would be good to carefully explain the reasons behind it; otherwise, this may raise concerns about the validity of some of the data and acclaimed features of the assay.

    We thank the reviewer for their suggestion and have modified the discussion to explain the reasons why the assay was not ultimately adopted at our institute.

  4. Average RT-qPCR assays using well-established primer sets meanwhile reach sensitivities where 10 molecules of the viral genome are detected in 95% of the cases, which for most RT-qPCR kits corresponds to a CT approx. = 37. This can be considered as standard. Is this given for the used PCR assay (CRICK BGI)?

    The LOD for the BGI’s Real-Time Fluorescent RT-PCR kit for detecting 2019-nCov (SARS-CoV-2) is 100 copies/mL, which is consistent with standard viral genome detection from other RT-qPCR kits (https://www.bgi.com/global/molecular-genetics/2019-ncov-detection-kit/).

  5. The manuscript does not elaborate on the used RNA isolation procedures. This should be included in the methods section. Different RNA extraction protocols exist, either using silica matrix columns or magnetic beads. These protocols allow purification and simultaneous concentration of nucleic acids relative to the original specimen.

    We apologise for not including the information in our original submission as the complete RNA isolation/purification procedure utilised by the institute’s RT-qPCR pipeline had already been published in its entirety. However, we have now included a brief description in the revised methods.

  6. For diagnostic analysis of nucleic acids in SWAB specimen, a concentration factor of 2 to 5 usually applies. This yields a LOD of approx. 500 or fewer viral RNA molecules per ml of patient specimen for an RT-qPCR that uses 10 μl input volume. The authors here claim a sensitivity of the RT-LAMP of 11000-22000 molecules per ml patient specimen using a SARS-CoV-2 standard (Figure 4), while they claim a sensitivity up to Ct=35 with patient samples (Figure 5a). A Ct=35 in an RT-qPCR assay corresponds roughly to 40 molecules per 10 μl purified RNA, while only 3 μl are used for the RT-LAMP, which would imply a sensitivity limit of approx. 15 molecules per reaction. This is inconsistent with the data shown in Figure 4 and with previous reports for the N-gene LAMP primer sets used here, which was reported to have a sensitivity in the range of 100 molecules per reaction (e.g. 1). A closer inspection of the data shown in Figure 5a indicates that all RT-LAMP positive samples have CT<<32 (Figure 4a), but not enough samples in the CT range of 25 – 35 are shown to confidently estimate a CT value that corresponds to the LOD. These issues should be clarified and, if necessary, faulty standard calibrations or additional means to calibrate the sensitivity should be used.

    We apologise for the confusion and have clarified our discussion of the assay’s LOD, including the acknowledgement that a true determination of the LOD/false negative rate would require additional clinical specimen testing. We thank the reviewer also for their careful analysis of Figure 5A. Samples that had Ct > 33, not 35 (a typo), were not consistently detected, which would be more consistent with the existing LOD determinations.

    We have also modified the text to more explicitly state that the assay is less sensitive than RT-qPCR.

  7. The authors represent a consortium that has generated more than 252000 RT-qPCR results since the beginning of the pandemic. Obviously, knowing the exact sensitivity limit of the RT-LAMP assay and the data of the RT-qPCR would permit an estimation of the false-negative rate of the RT-LAMP assay in a much more realistic scenario as compared to the limited number of samples presented here.

    We acknowledge that knowing the exact sensitivity and running the RT-LAMP assay in parallel against RT-qPCR for the same samples would allow a true estimation of the false-negative rate of the RT-LAMP assay. However, as stated above in response 3, the assay was ultimately not adopted and therefore, we can only speculate. We have now included some discussion of this particular issue in the revised text.

  8. The assay makes use of one primer set only for SARS-CoV-2 detection. Meanwhile, many more primer sets have been described, and a publication by NEB scientists 2 furthermore presented means to improve the sensitivity of the assay by adding GuHCL to the reaction buffer and by using multiple primer sets in combination. This could be discussed and the benefits of using better primers/conditions to be estimated.

    In the first months of the pandemic during the initiation of our studies, we tested several LAMP primers specific for SARS-CoV-2 as outlined in Ref 1. We moved forward with our particular primers because they did not display aberrant non-specific amplification seen with the other primers. We have now mentioned this in the revised text and included these data  in new Supplement Figure 1.

    As the reviewer has suggested, we have expanded our discussion of possible ways to optimise the assay.

  9. Specificity: The authors elude that the RT-LAMP assay does not cross-react with other viruses. If the authors really want to use this experiment as a valid assay to demonstrate the sensitivity of the RT-LAMP assay for SARS-CoV-2, they should provide evidence that the used samples indeed contain viral nucleic acids in sufficient numbers (> 10000 molecules per ml) of the patient specimen.

    We have included this information now in Figure 3B and Supplement Figure 4.

  10. Another issue that needs to be addressed is the relatively well-established propensity of RT-LAMP to produce false-positive results by spurious amplification (e.g. see 3). The frequency of these may be small, i.e. below 1%, and hence such cases escaped detection in this work. As shown by the authors, melting curves allow to identify those, but upscaling and the use of the colorimetric RT-LAMP will certainly bear the risk of false positives. Here, validation of positives by RT-qPCR could be used.

    We agree that samples suspected of being false-positives (e.g. late amplification) could be ruled out by validation with RT-qPCR and have included this suggestion in the text.

Additional points:

 

  1. The authors mention twice preliminary data on an RNA-extraction free procedure. Where are these data? I suggest either adding the data or removing the corresponding sentences in the introduction and discussion.

    We apologise for this oversight and have described the data now in the corresponding text for new Supplemental Figure 6.

  2. The number of cells in a swab can be affected by many factors. Even in the absence of human cells/detectable housekeeping genomes, viral RNA can be or should be present in a positive sample in the form of cell-free, viral particles. How do the authors suggest proceeding if 18S rRNA is not detected in a sample? How does the detection of the housekeeping gene affect the analysis of the clinical samples (e.g. in Figure 5)? For example, in Figure 3B. The fifth sample appears to be 18S rRNA negative, as the reaction is still pink. How is such a result taken into account?

    We acknowledge that SARS-CoV-2 could be present in samples in cell-free, viral particles that may not be associated with human cells. However, to avoid potential confusion during diagnosis between the following two scenarios: whether a patient’s sample may have been inadvertently contaminated during processing with SARS-CoV-2 versus detection of SARS-CoV-2 from a genuine nasopharyngeal swab, we explicitly state in the supplemental methods that a SARS-CoV-2 positive result would only be made if there was sufficient amplification in the internal control.

    The analysis of the housekeeping gene should not affect the analysis of SARS-CoV-2 as they are carried out in separate reactions (excluding any possible errors associated with loading of the RNA input into the reaction).

    The sample in Figure 3B was 18S rRNA positive and was one of the last samples to amplify (around 18 minutes) as seen in the corresponding amplification plot (now denoted by asterisk). One of the reasons we utilised a thermocycler for detection of Syto9 incorporation was to more accurately quantitate amplification during the assay because as the reviewer points out, sometimes a change in sample colour is not always obvious to the naked eye.

  3. The citations are not exhaustive. Many manuscripts have been published meanwhile that make use of the colorimetric RT-LAMP assay for COVID-19 diagnostics. In addition, a description of the major differences or similarities to previously published work is missing.

    We agree that many studies since our original submissions have been published and now reference a recent review that compares and contrasts various RT-LAMP assays for COVID-19 diagnostics that have been developed. We have revised our discussion to include a comparison of our assay against previous work.

  4. The manuscript would benefit from a report on the frequency of mutations in the binding sequences of the primer set used, which could lead to false negatives. This can easily be done using online resources (https://www.gisaid.org or https://covidcg.org) and the great number of viral genomes sequenced in the UK.

    We have undertaken an analysis of the frequency of mutations present in the binding sequences of the primers we utilised as the reviewers have suggested. These new data are included in Supplemental Figure 2 and the corresponding revised text.

  5. The authors need to report on measures to avoid contamination by LAMP products. This is an important point that may be less relevant for a diagnostic setting (since people are aware of this issue), but in an academic environment, researchers may need a reminder, which could be given for example by pointing out relevant literature 4 or by indicating strategies to eliminate those in the first place (UNG/UDG), or both.

    We have mentioned in the revised discussion the need and strategies to avoid contamination by LAMP products, including the relevant literature that the reviewer has brought our attention to.

Specific comments to the Figures:

 

  1. Figure 1: Suggest moving primer sequences to a separate table.

    For increased visibility, we have decided to continue highlighting the primers utilised in Figure 1.

  2. Figure 2: Panels D and E are redundant with Panels A and B as the threshold is indicated by the grey background (just add an explanation of the threshold to the legend). Panel C is also redundant and difficult to read, the information is already in A and B. Give an explanation of the false negatives on the 18S control, which already give fluorescent signals at 20 mins into the LAMP reaction. The primer design seems suboptimal as spurious amplification usually occurs at later time points. At what time point did the authors analyze the colorimetric read-out for 18S, e.g. in Fig 3B. Please define up to when is the reaction reliable.

    We have removed the grey background in Figures 2A and 2B and removed Figure 2C as suggested. Since Panels 2D and 2E were from separate experiments done in Figures 2A and 2B utilising the new detection threshold, we have chosen to keep them within the main figure.

    We acknowledge that the 18S rRNA specific primers we adopted from a previous study were not exhaustively assessed in our own studies and may not be optimally designed, which may account for the exhibition of spurious amplification. However, the inclusion of the melting curve analyses should help distinguish these false positives from true positives.

    The colorimetric reactions were visualised either after 25 minutes of amplification (N-gene SARS-CoV-2) or 20 minutes of amplification (18S rRNA). Although the images are included alongside the amplification plots (corresponding to Syto9 dye incorporation), positive amplification by Syto9 read-out was more sensitive than the subjective colour conversion read-out as explained in response 12.

  3. Suggest moving the melting curves from S1A to the main figures. When calculating sensitivity thresholds-does the melting curve change anything? Does it improve the sensitivity?

    We have done as the reviewer has suggested and modified Figure 2 to include the data featured in Figure S1A.

    The inclusion of the melting curve analyses does not improve the sensitivity of the reaction (e.g. carrying out amplification for longer periods in the hopes of amplifying low copy number positives), but merely aids in distinguishing true amplification from false amplification. Samples that have copy numbers below the LOD do not appear to amplify with longer incubation times (data not shown).

  4. Figure 3B, right panel: All different samples are just blue lines. Like this, it is impossible for the reader to understand which line corresponds to which sample. As the coloration is somewhat different between the samples as well as the time points where the fluorescent signal becomes positive, it would be good to see if some sort of correlation exists or not.

    For the one sample that had late 18S rRNA amplification and concurrently, suboptimal colour change, we now denote this particular amplification curve and plate well with an asterisk in Figure 3B to assist in the visualisation of the correlation between the two analyses and cite this in the figure legend.

  5. Figure 4 C: I do not understand the reason behind treating already purified SARS-CoV-2 RNA with Triton X. This only makes sense in an extraction-free setting.

    The reviewer is correct that the rationale in using Triton-X treated RNA would be applicable for an extraction-free setting. We have included new data in Supplementary Figure 6 that explains the reasoning for having performed the experiment depicted in Figure 4C.

  6. Figure 4 D: When determining the RT-LAMP assay precision, why extracting the RNA 5 times independently and then running the assay? I expect higher variations from the RNA extraction than from the RT-LAMP assay itself, which should be highly stable and reproducible on purified samples unless the sample contains only very few amounts of target RNA and the detection can become stochastic.

    One of the criteria that needed to be met for approving the assay for clinical diagnostics (in collaboration with the accredited clinical diagnostic laboratory and hospital partners) was to insure that for any one sample going through the entire pipeline (including RNA extraction from the original swab sample to final read-out), the result would need to be highly reproducible / precise. Therefore, we performed the exact same assessment our institute used to validate their use of a CE marked RT-qPCR assay, which was to process a nasopharyngeal sample that had an accompanying clinical diagnosis for COVID-19, 5 times from start to finish (lysate to result).

Associated Data

    This section collects any data citations, data availability statements, or supplementary materials included in this article.

    Data Availability Statement

    Underlying data

    Open Science Framework: SARS-CoV-2 detection by a clinical diagnostic RT-LAMP assay; raw data for figures, https://doi.org/10.17605/OSF.IO/2XR8U 20.

    Open Science Framework: SARS-CoV-2 detection by a clinical diagnostic RT-LAMP assay; raw data for supplementary figures, https://doi.org/10.17605/OSF.IO/YFCSW 21.

    Extended data

    Open Science Framework: SARS-CoV-2 detection by a clinical diagnostic RT-LAMP assay, supplementary figures and methods, https://doi.org/10.17605/OSF.IO/PAF36 18.

    This project contains the following extended data:

    • -

      Figure S1. SARS-CoV-2 RT-LAMP and internal control 18S rRNA RT-LAMP display a consistent dissociation peak and melting temperature. (A) Dissociation curves are shown for 11 SARS-CoV-2 positive and 11 negative clinical samples along with 12 NTC wells tested by RT-LAMP N gene assay (top) and 18S assay (bottom). (B) Dot plot of the melting temperatures determined by 94 positive patient samples and 235 negative patient samples tested by RT-LAMP assay. Data are pooled from six separate experiments performed on 3 separate RT-PCR machines. The average melting temperature (Tm) is depicted by a dotted line in (A) and below the graph in (B) with the standard of deviation.

    • -

      Figure S2. Inclusion of a melt curve stage complements the ability of RT-LAMP to accurately distinguish true SARS-CoV-2 positive samples. (A) Amplification curves (top) and dissociation curves (bottom) are depicted from a range of positive samples for each assay along with a negative sample for reference. TP indicates a true positive call based on amplification above background plus consistent melting curve and Tm consistent with genuine positives. (B) Amplification curve (left) and dissociation curve (right) are depicted of positive samples and NTCs. TP indicates the same criteria outlined in (A), whereas FP denotes a false positive, which despite amplifying above background, does not have a melting curve and temperature consistent with genuine positives.

    • -

      Figure S3. RT-LAMP yields identical results across different RT-PCR instruments and 96 versus 384-well plate formats. Dot plot comparing results from RT-LAMP assays (N gene, top; 18S, bottom) on the same clinical positive and negative sample serially diluted 2-fold using an ABI 7500 Fast or QuantStudio 5 machine (96-well vs. 384-well plate format).

    • -

      Supplementary Method 1: RT-LAMP Mastermix Preparation

    • -

      Supplementary Method 2: RT-LAMP RNA Loading and Running RT-PCR Machine

    • -

      Supplementary Method 3: RT-LAMP Results Reporting

    Data are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).

    Articles from Wellcome Open Research are provided here courtesy of The Wellcome Trust

    RESOURCES