Skip to main content
PLOS ONE logoLink to PLOS ONE
. 2022 Jun 30;17(6):e0270715. doi: 10.1371/journal.pone.0270715

A prospective diagnostic evaluation of accuracy of self-taken and healthcare worker-taken swabs for rapid COVID-19 testing

Helen R Savage 1, Lorna Finch 2, Richard Body 3, Rachel L Watkins 2; LSTM Diagnostics group2,; CONDOR steering group3,, Gail Hayward 4, Eloïse Cook 3, Ana I Cubas-Atienzar 2, Luis E Cuevas 1, Peter MacPherson 1,5,6, Emily R Adams 2,*
Editor: Sylvia Maria Bruisten7
PMCID: PMC9246218  PMID: 35771760

Abstract

Background

Rapid diagnostic tests (RDTs) developed for point of care detection of SARS-CoV-2 antigen are recommended by WHO to use trained health care workers to collect samples. We hypothesised that self-taken samples are non-inferior for use with RDTs to diagnose COVID-19. We designed a prospective diagnostic evaluation comparing self-taken and healthcare worker (HCW)-taken throat/nasal swabs to perform RDTs for SARS-CoV-2, and how these compare to RT-PCR.

Methods

Eligible participants 18 years or older with symptoms of COVID-19. 250 participants recruited at the NHS Test and Trace drive-through community PCR testing site (Liverpool, UK); one withdrew before analysis. Self-administered throat/nasal swab for the Covios® RDT, a trained HCW taken throat/nasal sample for PCR and HCW comparison throat/nasal swab for RDT were collected. RDT results were compared to RT-PCR, as the reference standard, to calculate sensitivity and specificity.

Findings

Seventy-five participants (75/249, 30.1%) were positive by RT-PCR. RDTs with self-taken swabs had a sensitivity of 90.5% (67/74, 95% CI: 83.9–97.2), compared to 78.4% (58/74, 95% CI: 69.0–87.8) for HCW-taken swabs (absolute difference 12.2%, 95% CI: 4.7–19.6, p = 0.003). Specificity for self-taken swabs was 99.4% (173/174, 95% CI: 98.3–100.0), versus 98.9% (172/174, 95% CI: 97.3–100.0) for HCW-taken swabs (absolute difference 0.6%, 95% CI: 0.5–1.7, p = 0.317). The PPV of self-taken RDTs (98.5%, 67/68, 95% CI: 95.7–100.0) and HCW-taken RDTs (96.7%, 58/60, 95% CI 92.1–100.0) were not significantly different (p = 0.262). However, the NPV of self-taken swab RDTs was significantly higher (96.1%, 173/180, 95% CI: 93.2–98.9) than HCW-taken RDTs (91.5%, 172/188, 95% CI 87.5–95.5, p = 0.003).

Interpretation

In conclusion, self-taken swabs for COVID-19 testing offer an accurate alternative to healthcare worker taken swabs for use with RDTs. Our results demonstrate that, with no training, self-taken throat/nasal samples can be used by lay individuals as part of rapid testing programmes for symptomatic adults. This is especially important where the lack of trained healthcare workers restricts access to testing.

Introduction

The Severe Acute Coronavirus 2 (SARS-CoV-2) is a novel pathogen causing Coronavirus Disease-19 (COVID-19) that emerged in December 2019 and spread quickly around the globe before being declared a pandemic on 20th March 2020. Confirmation of SARS-CoV-2 infection is recommended by real-time polymerase chain reaction (RT-PCR) testing, however this requires well-resourced laboratory facilities, which are not available in many settings [1]. Given the need to rapidly upscale testing, rapid diagnostic tests (RDTs) were developed to detect SARS-CoV-2 antigen(s) (Ag), which can be used at point of care without a laboratory infrastructure. Guidance from the World Health Organization (WHO) recommends using RDTs in settings with trained health workers to facilitate collecting samples and processing tests [1]. Currently, large-scale self-testing for SARS-CoV-2 is conducted in schools, workplaces, and homes in the U.K.; however concerns have been raised over the accuracy of these tests and the risk of missing infected individuals [2]. Many tests designed and regulated for “Professional Use Only” have been implemented for self-testing use, but little accuracy information exists for self-swabbing and interpretation.

Previous work has suggested that RDTs achieve higher sensitivity when performed by laboratory scientists (sensitivity of the Innova lateral flow test 79%, 95% CI 72–84%) than by healthcare workers (sensitivity 70%, 95% CI 63–76%) [3]. Sensitivity can be substantially affected by the quality of the sample and swabbing technique [4]. A small number of studies have compared self-taken to healthcare worker (HCW) taken swabs for RT-PCR, with a high degree of concordance [5, 6]. To the best of our knowledge, no studies have compared self-taken and HCW-taken samples with identical swab types and identical RDTs rather than comparing alternative sampling strategies.

We therefore set out to compare the sensitivity and specificity of self-taken and HCW-taken throat/nasal swabs to perform a RDT for SARS-CoV-2, and how these sampling approaches perform compared to the RT-PCR. We hypothesised that, if self-taken samples are accurate for use with RDTs in clinical and research settings, this could have substantial individual and public health benefit.

Methods

We conducted a prospective diagnostic accuracy evaluation to compare self-taken and HCW-taken throat/nasal swabs RDTs with a standard HCW-taken throat/nasal swab tested using RT-PCR. Participants were recruited as part of the ‘Facilitating Accelerated Clinical Evaluation of Novel Diagnostic Tests for COVID -19 (FALCON), workstream C (undifferentiated community testing)’ [7], which aims to evaluate the diagnostic accuracy of commercially supplied in-vitro diagnostic (IVD) tests for SARS-CoV-2 infection.

Participants were recruited consecutively when presenting at the Liverpool John Lennon Airport (Liverpool, UK) drive through community PCR testing site, a National Health Service (NHS) Test and Trace site for the general population with symptoms of COVID-19, defined as a high temperature, continuous cough or change in sense of taste or smell. People presenting for testing were assessed for eligibility in their vehicle and received a patient information sheet prior to offer of NHS test (which was taken after the study). Eligible participants were 18 years or older who verbally confirmed they currently had symptoms of COVID-19. If multiple occupants were in the vehicle, they were each assessed for eligibility. Informed verbal consent was taken and recorded by a researcher on site. Demographic and self-reported symptom data were recorded electronically; a list of symptoms from the FIND alliance and participants could also list additional symptoms they felt were relevant. We excluded anyone under 18 years, or who did not state they currently had COVID-19 symptoms, or if they did not consent to participate. We did not record people who declined to participate. As people presented to the testing centre in vehicles, we were unable to distinguish between drivers or relatives, and people presenting for testing services.

A HCW in personal protective equipment (PPE, in line with local guidelines) passed a self-collection kit for each participant into the vehicle. Each kit contained a short instruction sheet taken from the manufacturer protocol (see S1 File), a tissue, a swab, and collection tube. The participant self-administered their throat/nasal swab without further advice, observation, or supervision. Once complete the participant signaled the research team and a trained healthcare worker took a throat/nasal sample using a COPAN mini UTM (universal transport medium) kit 1ml for PCR from one side and a comparison swab for RDT from the other. All swabs taken for RDT were randomly numbered so that laboratory staff performing the RDTs were blinded to sample collection method and could not identify paired samples.

Samples were transported to the Liverpool School of Tropical Medicine (LSTM) by trained research staff (in accordance with the requirements for Category B substance UN3373 [8]) where they were processed and tested in a category 3 laboratory within 3 hours of sampling. UTM samples for PCR were aliquoted and frozen at -80°C. The Covios® COVID-19 Antigen Rapid Diagnostic test, which detects the SARS-CoV-2 nucleoprotein, was used for testing all RDT samples [9]. This test is CE marked and manufactured in the UK by Global Access Diagnostics (legal manufacturer Mologic) (patients 1–100 LOT: CALFD-102-1, patient 101–250 LOT: CALFD-130-1). RDTs were run according to the manufacturer’s instructions, this includes grading the result line from 0–10 on the RDT using a visual reference card. Each test was read by two trained researchers, if there was a disagreement between the two readers, a third reader was requested.

RNA was extracted from batched UTM samples using the QiAamp96 Virus Qiacube HT kit and RT-PCRs were run following manufacturer’s instruction using TaqPath COVID-19 RT-PCR on QuantStudio 5 (ThermoFisher). RT-PCR reactions volumes were made in 20 μl. Reverse transcription step was performed at 53°C for 10 minutes and this was followed by an activation step of 2 minutes at 95°C, then PCR was carried for 40 amplification cycles at 95°C for 3 minutes and 60°C for 30 seconds. Fluoresce was recorded in the FAM, VIC, ABY and JUN channels for the ORF1ab, N, S and MS2 targets respectively. RT-PCR was used as the reference standard test in this study for comparison of the RDT results.

Sample size and statistical analysis

Sample size was calculated using an alpha of 0.05, anticipated prevalence of 20% (based on the positivity rate of PCR tests of individuals presenting for testing within Liverpool in the week commencing 21st January 2021, calculated by Public Health England [10]), minimum test sensitivity of 80%, specificity 99%, and precision interval of 10% [11]. These gave a planned sample size of 308. We described participant characteristics using summary statistics and compared self-swab sampling RDT results with RT-PCR results. RDTs were graded 0–10, with 0 representing a negative result, and 1–10 positive results based on a visual scale using the manufacturer’s reference card. For each RT-PCR result with three target genes a mean of the three CT values was taken to give a single RT-PCR CT result for each sample; RT-PCR results with mean cycle threshold (CT) values <40 were considered positive and CT values ≥40 were graded as negative. We calculated sensitivity and specificity, and positive and negative predictive values (PPV and NPV), all with binomial exact 95% confidence intervals (CI) in R v4.1.1 (R Foundation for Statistical Computing, Vienna). Paired results were compared between self-taken and HCW-taken samples using McNemar’s test. Indeterminate RDT results were recorded but excluded from further analysis. Indeterminate RT-PCR results were repeated twice, and the repeat test result was used for analysis. RT-PCR data were classified according to the mean RT-PCR CT threshold values (<20, 20–24.9, 25–29.9, 30–34.9 and ≥35).

Recruitment started on 31st March 2021 and 100 participants had been recruited until the 21st May 2021, when local COVID-19 prevalence declined (positivity testing rate 0.4%) giving very small numbers of positive cases. Recruitment was temporarily halted until July 2021 when prevalence increased (positivity testing rate 12.9%) and a further 150 participants were recruited. Recruitment ended on 9th August 2021.

Ethical approval

Ethical approval was obtained from the National Research Ethics Service (reference 20/WA/0169) and the Health Research Authority (IRAS ID:28422, clinical trial ID: NCT04408170).

Role of the funding source

The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Results

Two hundred and fifty participants were recruited between the 31st March 2021 and the 9th August 2021. One participant withdrew after recruitment and did not wish data or samples to be included, leaving 249 participants for the analysis. The mean age of participants was 40 years (range 18–82, interquartile range [IQR] 30.0–50.0 years), 104 (41.7%) were male and 216 white British (86.7%) (Table 1). One hundred and eighty (72.3%) had received at least one vaccine dose against SARS-CoV-2 and of these 113 had received a second dose. The time interval since vaccination and the vaccine brand were not available. The most common self-reported symptoms by participants were cough (174, 69.9%), fever (78, 31.3%), sore throat (71, 28.5%) and headache (53, 21.3%); all participants were symptomatic. The median duration since symptom onset was 2 days (IQR 1–3 days). A full data table of all participant characteristics and results is available in the supplementary data.

Table 1. Characteristics of participants.

All (N, %) RT-PCR positive (N, %)
All 249 75
Age in years, mean, (range, IQR) 40 (18–82, 30.0–50.0) 37.6 (18–70, 24.5–50.0)
Male 104 (41.7) 42 (56.0)
Median symptom duration (days), range, IQR 2.0 (0–33, 1–3) 2.0 (0–32, 1–3)
Shortness of breath 7 (2.8) 2 (2.7)
Cough 174 (69.9) 53 (70.7)
Fever 78 (31.3) 31 (41.3)
Chest pain 11 (4.4) 7 (9.3)
Sore throat 71 (28.5) 19 (25.3)
Confusion 0 (0) 0 (0)
Rash 1 (0.4) 1 (1.3)
Loss of smell 27 (10.8) 17 (22.7)
Loss of taste 25 (10.0) 16 (21.3)
Abdominal pain 6 (2.4) 1 (1.3)
Vomiting 7 (2.8) 3 (4.0)
Diarrhoea 11 (4.4) 3 (4.0)
Headache 53 (21.3) 22 (29.3)
Tiredness/Fatigue 10 (4.0) 5 (6.7)
Tight chest 1 (0.4) 0 (0)
Other 70 (28.1) 31 (41.3)
White British 216 (86.7) 67 (89.3)
Irish 10 (4.0) 2 (2.7)
Other white 7 (2.8) 1 (1.3)
Indian 2 (0.8) 0 (0.0)
Mixed ethnic group 9 (3.6) 3 (4.0)
Other ethnic group 5 (2.0) 2 (2.7)
Vaccinated 1st dose 180 (72.3) 52 (69.3)
Vaccinated 2nd dose 113 (45.4) 32 (42.7)

Seventy-five participants (75/249, 30.1%) tested positive by RT-PCR. The mean age of RT-PCR positive participants was 37.6 years (range 18–70, IQR 24.5–50.0), 42 (56.0%) were male and 67 were white British (89.3%) (Table 1). Fifty-two (69.3%) of the 75 RT-PCR-positive participants had received a first COVID-19 vaccine dose and 32 (42.7%) a second dose. Since symptom onset, the median duration in days was 2 (IQR: 1–3) and the most commonly reported symptoms were cough (53, 70.3%), fever (31, 41.3%), headache (22, 29.3%), sore throat (19, 25.3%), loss of smell (17, 22.7%) and loss of taste (16, 21.3%).

Overall, self-taken throat/nasal RDTs were positive in 68/249 (27.3%, 95% CI: 21.9–33.3) participants, one (0.4%) was indeterminate and 180 (72.3%) negative. HCW-taken throat/nasal RTDs were positive in 61/249 (24.5%, 95% CI: 19.3–30.3) participants, none was indeterminate and 188 (75.5%) were negative. The participant with the indeterminate RDT was excluded from further analysis.

RDT kits using a self-taken swab had a sensitivity of 90.5% (67/74, 95% CI: 83.9–97.2) and specificity of 99.4% (173/174, 95% CI: 98.3–100.0) when compared to the reference standard (Table 2). HCW-taken RDTs had a sensitivity of 78.4% (58/74, 95% CI 69.0–87.8) and specificity of 98.9% (172/174, 95% CI: 97.3–100.0) compared to the reference standard. The difference in sensitivity was 12.2% (95% CI: 4.7–19.6, p = 0.003), the difference in specificity was 0.6% (95% CI: 0.5–1.7, p = 0.317). Of the self-HCW RDT pairs, 238/248 (96.0%) agreed, and 10/248 (4.0%) were discordant (Fig 1). Of the discordant pairs, on nine occasions the self-taken swab RDT was read as positive, while the HCW-taken swab was read negative (Table 3). Nine of these pairs were RT-PCR positive and one negative. The discordant pair that was RT-PCR negative, the HCW-taken swab RDT was positive, while the self-taken RDT was negative. The PPV of self-taken RDTs (98.5%, 67/68, 95% CI: 95.7–100.0) and HCW-taken RDTs (96.7%, 58/60, 95% CI 92.1–100.0) were not significantly different (p = 0.262). However, the NPV of self-taken swab RDTs was significantly higher (96.1%, 173/180, 95% CI: 93.2–98.9) than HCW-taken RDTs (91.5%, 172/188, 95% CI 87.5–95.5, p = 0.003).

Table 2. Sensitivity and specificity of self- and healthcare worker-taken swabs for COVID-19 rapid diagnostic testing.

Comparison to RT-PCR
Sensitivity (%) 95% CI Specificity (%) 95% CI PPV (%) 95% CI NPV (%) 95% CI
Self-taken RDT 90.5 83.9–97.2 99.4 98.3–100.0 98.5 95.7–100.0 96.1 93.3–98.9
HCW RDT 78.4 69.0–87.8 98.9 97.3–100.0 96.7 92.1–100.0 91.5 87.5–95.5

* CI = Confidence intervals, PPV = Positive predictive value, NPV = Negative predictive value

Fig 1. Correlation between self and healthcare worker graded RDT result by PCR cycle threshold.

Fig 1

Table 3. Table showing full results for participants with discrepant RDT results.

Participant RT-PCR result (mean Ct value) Self-taken RDT result (Reader 1/Reader 2) HCW-taken RDT result (Reader 1/Reader 2)
26 Positive (29.03) Positive (4/4) Negative (0/0)
38 Negative (>40) Negative (0/0) Positive (2/2)
106 Positive (21.68) Positive (3/3) Negative (0/0)
131 Positive (27.99) Positive (3/3) Negative (0/0)
140 Positive (25.44) Positive (3/3) Negative (0/0)
169 Positive (28.09) Positive (2/2) Negative (0/0)
189 Positive (20.35) Positive (8/8) Negative (0/0)
195 Positive (22.96) Positive (3/3) Negative (0/0)
231 Positive (27.33) Positive (6/6) Negative (0/0)
249 Positive (27.86) Positive (5/5) Negative (0/0)

Sensitivity of the RDTs varied by mean CT values (Table 4 and Fig 2), a full table of mean and CT values from each gene tested is available in the supplementary data. Self-taken and HCW-taken samples with CT values <20 had 100% (32/32, CI: 89.1–100.0) sensitivity; samples with CT values 20–24.9 had 91.7% (22/24, 95% CI: 73.0–99.0) for self-taken and 83.3% (20/24, 95% CI: 62.6–95.3) for HCW-taken RDTs. At CT values between 25 and 29.9, RDTs sensitivity was 80.0% (12/15, 95% CI: 51.9–95.7) for self-taken and 40.0% (6/15, 95% CI: 16.3–67.7) for HCW-taken swabs, while at CT values 30–34.9 both self-taken and HCW-taken swabs had sensitivity of 33.0% (1/3, 95% CI: 0.8–90.6). Sensitivity for samples with CT values ≥35 was 0% (0/1, 95% CI: 0.0–97.5).

Table 4. Sensitivity of self- and healthcare worker-taken swab for rapid diagnostic testing by RT-PCR CT ranges.

RT-PCR CT range
<20 20–24.9 25–29.9 30–34.9 ≥35
Self-taken RDT
Positive 32 22 12 1 0
Negative 0 1 3 2 1
Indeterminate 0 1 0 0 0
Sensitivity 100.0% 91.7% 80.0% 33.3% 0%
95% CI 89.1–100.0% 73.0–99.0% 51.9–95.7% 0.8–90.6% 0.0–97.5%
Cumulative sensitivity 100.0% 96.4% 93.0% 90.5% 89.3%
95% CI 89.1–100.0% 87.7–99.6% 84.3–97.7% 81.5–96.1% 80.1–95.3%
HCW RDT
Positive 32 20 6 1 0
Negative 0 4 9 2 1
Indeterminate 0 0 0 0 0
Sensitivity 100% 83.3% 40.0% 33.3% 0.0%
95% CI 89.1–100.0% 62.6–95.3% 16.3–67.7% 0.8–90.6% 0.0–97.5%
Cumulative sensitivity 100.0% 92.9% 81.7% 79.7% 78.7%
95% CI 89.1–100.0% 82.7–98.0% 70.7–89.9% 68.8–88.2% 67.6–87.3%

Fig 2. A graph to show the sensitivity of the Covios®SARS-CoV-2 RDT by mean CT range and cumulative sensitivities.

Fig 2

Discussion

This study found that the sensitivity of self-taken swabs for the detection of SARS-CoV-2 antigen was higher (90.5%) than using HCW-taken swabs (78.3%), with similar specificity. No RT-PCR-positive results from HCW- taken swabs were missed by self-taken swabs and the PPV and NPV for both methods were over 90%.

Current WHO guidance for implementing RDTs indicates swabbing to collect samples should be conducted by trained professionals [1]. Having HCW take swabs requires training, PPE to be available, regular donning and doffing, and close contact with potentially infectious individuals. All of which pose expenditure and added risks to those performing sampling, especially in countries where the majority of HCW remain unvaccinated. Using self-sampling for testing could reduce the workload of health workers and increase the ability of services to test patients in both clinical and research settings where trained workers are not available. These results show self-taken throat-nasal samples with only written and pictorial instructions can be used by the general public for RDTs and is not likely to reduce the sensitivity of testing, which could widen access.

Within this study, self-taken swabs had higher sensitivity than HCW swabs for RDTs in a general population setting in the UK. The self-swabbing technique was not monitored for quality, no participants failed to take the swabs and no assistance was given so the results could be extrapolated to other non-supervised settings. The high concordance of self- and HCW-taken results has been reported from studies comparing self-and HCW-taken swabs for PCR testing and also within studies looking at alternative swab types (nasopharyngeal, nasal only) for RDTs [6, 1214].

The limitations of this study are that the sampling order was not randomized as the HCW swab for RDT was taken after the swab for the RT-PCR, and only from one nostril; this may lower sensitivity, or participants may experience sampling fatigue. However, previous studies have shown that repeated sampling from one nostril using nasal mid-turbinate specimens does not impact RT-PCR sensitivity or CT values so, although we used throat/nasal sampling, this repeated sampling may not be the reason for lower sensitivity [15]. Participation was voluntary so people who were less confident to take their own sample may not have agreed to take part; it is also likely participants may have done previous COVID-19 tests, and so have experience of self-sampling.

The majority of participants with discrepant results between self-taken and HCW taken RDT results had a mean CT of between 25–30 on PCR (six of ten). Although there was only a small number of participants with a CT value in this range, these values are influential in estimates of the sensitivity of the HCW taken RDT tests. Because of the small numbers, we did not undertake further formal statistical analysis; however when participants with CT values within this range were excluded, our main conclusion that self-taken swabs were non-inferior to healthcare worker taken swabs did not change (sensitivity with CT 25–29.9 excluded: self taken RDT 91.7% [55/60]; HCW taken 88.3% [53/55]). Similar results were noted in the trial by García-Fiñana et al. [4] who also saw a decrease in sensitivity of RDTs compared to PCR in samples with a CT of greater than 24.4. Future research could compare self-taken and healthcare worker-taken samples by PCR including housekeeper genes to see if the quality of sampling varies and may affect CT values, particularly for samples with CT values over 25, as lower sampling quality could result in lower CT values and therefore a perceived poorer performance in this range.

By focusing on differences in the sampling process (thus removing issues of running and interpretation of RDTs), we have shown that the ability of individuals to take their own samples is unlikely to explain the differences in test accuracy, and that if individuals self-take samples for RDTs, the results can be as accurate as professionally taken swabs. Previously in the UK RDT positive tests were confirmed with follow-on PCR tests which, given the high prevalence of circulating infection, and the high workloads led to laboratory errors. Recent test discrepancies in South-West England, which reported large numbers of RDT-positive but PCR-negative results, were due to incorrect PCR results, with correct RDTs results [16]. Our study suggest that the public and healthcare professionals should trust RDT-positive tests from self-taken samples in symptomatic individuals. This is especially important in global settings where confirmatory laboratory testing is expensive and unlikely to be readily available.

In conclusion, self-taken swabs for COVID-19 testing offer an accurate alternative to healthcare worker taken swabs for use with RDTs. Our results demonstrate that, with no training, self-taken throat/nasal samples can be used by lay individuals as part of rapid testing programmes for symptomatic adults. Self-testing has the potential to widen access to early diagnosis for COVID-19 in clinical services and outreach settings; allowing access to therapies in the early stage of the illness. This is especially important where the lack of trained healthcare workers restricts access to testing.

Supporting information

S1 Appendix

(DOCX)

S1 Table. A table of all PCR positive participants and their characteristics.

(DOCX)

S1 File. Covios® leaflet from study.

(DOCX)

Acknowledgments

We acknowledge the support of the National Institute for Health Research (NIHR) Clinical Research Network, which supports delivery of the FALCON study. The views expressed in this article are those of the authors and not necessarily those of the NIHR, or the Department of Health and Social Care.

LSTM diagnostics group: Kate Buist, Karina Clerkin, Dr Thomas Edwards, Dr Susan Gould, Caitlin Greenland-Bews, Konstantina Kontogianni, Laryssa Mashenko, Caitlin R Thompson, Jahanara Wardale, Christopher T Williams and Dominic Wooding.

Lead author LSTM Diagnostics Group: Thomas Edwards, Thomas.edwards@lstmed.ac.uk

Condor steering group: Dr A. Joy Allen, Dr Julian Braybrook, Professor Peter Buckle, Professor Paul Dark, Dr Kerrie Davis, Professor Adam Gordon, Ms Anna Halstead, Dr Charlotte Harden, Dr Colette Inkson, Ms Naoko Jones, Dr William Jones, Professor Dan Lasserson, Dr Joseph Lee, Dr Clare Lendrem, Dr Andrew Lewington, Mx Mary Logan, Dr Massimo Micocci, Dr Brian Nicholson, Professor Rafael Perera-Salazar, Mr Graham Prestwich, Dr D. Ashley Price, Dr Charles Reynard, Dr Beverley Riley, Professor AJ Simpson (Professor Simpson is an NIHR Senior Investigator), Dr Valerie Tate, Dr Philip Turner, Professor Mark Wilcox, Dr Melody Zhifang.

Lead author CONDOR steering group: Richard Body, Richard.body@manchester.ac.uk

Data Availability

All relevant data are within the paper and its Supporting Information files.

Funding Statement

All authors have completed the ICMJE uniform disclosure form at http://www.icmje.org/disclosure-of-interest/ and declare: This study received funding from the UK Research Council through a PhD scholarship from the MRC Doctoral Training Partnership to HRS. PM is funded by Wellcome (200901/Z/16/Z), Wellcome Trust award. ‘Detecting and Excluding Coronavirus disease 2019 (COVID-19) at the Point of Need’ (220764/Z/20/Z), FALCON C-19 study was funded by National Institute for Health Research (NIHR), Asthma UK and the British Lung Foundation; and is supported by the global alliance for diagnostics (FIND). ERA contributed to this study design and analysis in her role as PhD supervisor to HRS at LSTM, she is also Director of Epidemics and NTDs at Mologic Ltd a UK diagnostics company who provided the RDTs for this study under joint Wellcome funding. This does not alter our adherence to PLOS ONE policies on sharing data and materials. For the purpose of open access, the author has applied a CC BY public copyright licence to any Author Accepted Manuscript version arising from this submission. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

References

  • 1.World Health Organization (WHO). SARS-CoV-2 antigen-detecting rapid diagnostic tests—An implementation guide. Geneva, 2020 http://apps.who.int.
  • 2.Torjesen I. Covid-19: How the UK is using lateral flow tests in the pandemic. BMJ 2021; 372: 1–3. doi: 10.1136/bmj.n287 [DOI] [PubMed] [Google Scholar]
  • 3.Peto T, Affron D, Afrough B, et al. COVID-19: Rapid antigen detection for SARS-CoV-2 by lateral flow assay: A national systematic evaluation of sensitivity and specificity for mass-testing. EClinicalMedicine 2021; 36: 0–6. doi: 10.1016/j.eclinm.2021.100924 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Garciá-Fiñana M, Hughes DM, Cheyne CP, et al. Performance of the Innova SARS-CoV-2 antigen rapid lateral flow test in the Liverpool asymptomatic testing pilot: Population based cohort study. BMJ 2021; 374. doi: 10.1136/bmj.n1637 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Altamirano J, Govindarajan P, Blomkalns AL, et al. Assessment of Sensitivity and Specificity of Patient-Collected Lower Nasal Specimens for Sudden Acute Respiratory Syndrome Coronavirus 2 Testing. JAMA Netw Open 2020; 3: 10–3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Tu Y-P, Jennings R, Hart B, et al. Swabs Collected by Patients or Health Care Workers for SARS-CoV-2 Testing. N Engl J Med 2020; 383: 494–6. doi: 10.1056/NEJMc2016321 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.NIH U.S. National Library of Medicine. Facilitating AcceLerated Clinical Validation Of Novel Diagnostics for COVID-19 (FALCON-C19) (FALCON-C19). 2021. https://clinicaltrials.gov/ct2/show/NCT04408170 (accessed Nov 17, 2021).
  • 8.Department for Transport. Packaging and transport requirements for patient samples–UN3373. United Kingdom. 2020. https://www.gov.uk/government/publications/packaging-and-transport-requirements-for-patient-samples-un3373/packaging-and-transport-requirements-for-patient-samples-un3373 (accessed Nov 16, 2021).
  • 9.Cubas-Atienzar AI, Bell F, Byrne RL, et al. Accuracy of the Mologic COVID-19 rapid antigen test: a prospective multi-centre analytical and clinical evaluation. Wellcome Open Res 2021; 6: 132. [Google Scholar]
  • 10.Liverpool City Council Corporate Intellignce team. Liverpool COVID-19 report: Produced on 21 January 2021. Liverpool, 2021.
  • 11.Fenn Buderer NM. Statistical methodology: I. Incorporating the prevalence of disease into the sample size calculation for sensitivity and specificity. Acad Emerg Med 1996; 3: 895–900. doi: 10.1111/j.1553-2712.1996.tb03538.x [DOI] [PubMed] [Google Scholar]
  • 12.Nikolai O, Rohardt C, Tobian F, et al. Anterior nasal versus nasal mid-turbinate sampling for a SARS-CoV-2 antigen-detecting rapid test: does localisation or professional collection matter? medRxiv 2021; pre-print: 2021.02.09.21251274–2021.02.09.21251274. doi: 10.1080/23744235.2021.1969426 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Lindner AK, Nikolai O, Rohardt C, et al. Head-to-head comparison of SARS-CoV-2 antigen-detecting rapid test with professional-collected nasal versus nasopharyngeal swab. Eur Respir J 2021; 57: 3–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Lindner AK, Nikolai O, Rohardt C, et al. Diagnostic accuracy and feasibility of patient self-testing with a SARS-CoV-2 antigen-detecting rapid test. 2020. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Levy JM, Frediani JK, Tyburski EA, et al. Impact of repeated nasal sampling on detection and quantification of SARS-CoV-2. Sci Rep 2021; 11: 1–5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Torjesen I. Covid-19: PCR testing is suspended at private laboratory after high rate of false negatives. BMJ. 2021; 375. 10.1136/bmj.n2535. [DOI] [PubMed] [Google Scholar]

Decision Letter 0

Sylvia Maria Bruisten

4 Apr 2022

PONE-D-22-05402A prospective diagnostic evaluation of accuracy of self-taken and healthcare worker-taken swabs for rapid COVID-19 testingPLOS ONE

Dear Dr. Savage,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

This is an interesting study which provides data that are informative for practical use. As two of the reviewers indicate, there should be more details provided on the data to really appreciate this work well.

Please submit your revised manuscript by May 19 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

  • A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

  • A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

  • An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Sylvia Maria Bruisten, Ph.D

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at 

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf.

2. Thank you for stating the following in the Competing Interests section: 

[ERA contributed to this study design and analysis in her role as PhD supervisor to HRS at LSTM, she is also Director of Epidemics and NTDs at Mologic Ltd a UK diagnostics company who provided the RDTs for this study under joint Wellcome funding.] 

Please confirm that this does not alter your adherence to all PLOS ONE policies on sharing data and materials, by including the following statement: "This does not alter our adherence to  PLOS ONE policies on sharing data and materials.” (as detailed online in our guide for authors http://journals.plos.org/plosone/s/competing-interests).  If there are restrictions on sharing of data and/or materials, please state these. Please note that we cannot proceed with consideration of your article until this information has been declared. 

Please include your updated Competing Interests statement in your cover letter; we will change the online submission form on your behalf.

3. One of the noted authors is a group or consortium [LSTM Diagnostics group, CONDOR steering group]. In addition to naming the author group, please list the individual authors and affiliations within this group in the acknowledgments section of your manuscript. Please also indicate clearly a lead author for this group along with a contact email address.

Additional Editor Comments:

This is an interesting study which provides data that are informative for practical use. As two of the reviewers indicate there should be more details provided to really appreciate this work well.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Partly

Reviewer #3: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: No

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: No

Reviewer #3: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The paper describes a well conducted study comparing self-taken and HCW taken throat/nasal swabs to perform RDT for SARS-CoV-2. The authors show that self-taken swabs had a higher sensitivity, when compared to RT-PCR and that the NPV of self-taken swabs RDTs was significantly higher than the NPV of HCW-taken RDTs.

Some minor issues:

RT-PCR is used as gold standard, which should be mentioned in the abstract and methods.

Interpretation:

Statements are made on “convenience” and “reduced risk of transmitting infections”, both of which may be true, but are not subject of this study.

As limitation of the study the non-randomized sampling order is mentioned. Reference 15 is cited in line 246 to illustrate that repeated sampling from one nostril does not impact RT-PCR sensitivity or CT values. However this paper shows that this holds for nasal mid-turbinate specimens, while it seems that anterior nasal specimens are used in this study, I think this should be mentioned in the discussion.

Reviewer #2: The manuscripts describes the comparison of self-taken and healthcare worker -taken throat swabs in rapid diagnostic tests (RTDs). The authors show the unexpected result of a higher sensitivity in self taken swab RTDs compared to healthcare taken swabs. The final conclusion of the paper is that RTDs offer substantial individual benefits.

Major comments

- In the methods (line 128) there is a multiplex SARS PCR described, but no data is shown on theses different genes, are there any discrepancies shown? I assume that all analyses are done on the spike protein, but this is not clear in the methods.

- Why is there not chosen to include an extra IC for a household gene (like beta-actin/ beta-globin), this might give some additional data on the quality of the swabs.

- Table 4. The difference in sensitivity is highly influenced by the group of samples with a PCR range 25-29.9, this is not mentioned or discussed. This is probably due to the small sample size in combination with Ct value, and possibly not to the swab method/quality. Are there any control experiments done to support that the turning point of the test indeed is in this range of Ct value.

Minor comment

- Abstract: in my opinion the abstract does not really invite to read the rest of the paper, background is only the method and hypothesis, and the findings is almost only numbers.

- Table 1: Ethnicity, vaccination status and the different symptoms are nicely reported, but there is no further link to the data.

- Table 3/Line 120: I am not sure what is meant by 'read graded'? In the resuts there is also no discrepancies shown in table 3., so I am wondering what the addiotional value is to the test and to the paper.

Reviewer #3: This is an interesting study. In order to give more clarity about the obtained results more details are required.

1. I in the Methods section the data are described. It seems none of the participants has been examined by a medical doctor to identify the health state. Instead, self-assessment as been used. Is this correct?

Specifically, each participant confirmed to have high temperature, contineous cough or change in smell of taste etc. Has the temperature been measures to confirm high temperature? Has any other examination or measurement been made?

Please adjust/modify the provided information correspondingly to reflect the actual situation.

2. If no medical examination has been performed this has to be clearly stated. This is not ideal because the symptoms mentioned in the paper are not unique to Covid-19. This should be also stated.

If all this is the case it seems two tests have been compared without precise information about the underlying health state of the patient.

3. The authors provide an estimate for the sample size assuming a prevelance of 20%. I assume this value is used for the estimate which results in 308 samples? The problem is a prevalence of 20% refers to the population of the UK. In constrast, the authors test only very specific sub-population which self-evaluates as suffering from covid. Hence, the prevalance is different. Please comment and revise correspondingly.

4. On line 157 it is mentioned that 250 participants have been used while the sample size estimate gave 308. Does this mean the study is by design underpowered?

5. The most crucial point of the study relates to the CT value of the RT-PCR test because it is well known that this has a severe impact (add citation).

The authors report different CT values (which is good) but the results therefor are unclear.

Specifically, the results in Tab 2 seem to contradict the literature showing that the higher CT the fewer the positive results. Please clarify this counter intuitive result.

6. The results about the different CT values seem to be the most interesting ones of the study. The discussion should be extended and the finding are highlighted.

7. I did not find information how the data could be obtained.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Mirjam Hermans

Reviewer #2: No

Reviewer #3: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PLoS One. 2022 Jun 30;17(6):e0270715. doi: 10.1371/journal.pone.0270715.r002

Author response to Decision Letter 0


9 May 2022

Dear Editor,

We would like to thank the referees for their constructive comments and recommendations made to this manuscript. We have made the revisions suggested and have prepared an itemized reply to all issues raised on the following pages as requested.

We hope these replies are satisfactory to the reviewers and your office. However, please contact us again if you feel there are issues that were not addressed appropriately.

Please include your updated Competing Interests statement in your cover letter; we will change the online submission form on your behalf.

Below is the updated Financial disclosure and competing interests statement:

Financial Disclosure and competing interests

All authors have completed the ICMJE uniform disclosure form at http://www.icmje.org/disclosure-of-interest/ and declare: This study received funding from the UK Research Council through a PhD scholarship from the MRC Doctoral Training Partnership to HRS. PM is funded by Wellcome (200901/Z/16/Z), Wellcome Trust award. ‘Detecting and Excluding Coronavirus disease 2019 (COVID-19) at the Point of Need’ (220764/Z/20/Z), FALCON C-19 study was funded by National Institute for Health Research (NIHR), Asthma UK and the British Lung Foundation; and is supported by the global alliance for diagnostics (FIND). ERA contributed to this study design and analysis in her role as PhD supervisor to HRS at LSTM, she is also Director of Epidemics and NTDs at Mologic Ltd a UK diagnostics company who provided the RDTs for this study under joint Wellcome funding. This does not alter our adherence to PLOS ONE policies on sharing data and materials. For the purpose of open access, the author has applied a CC BY public copyright licence to any Author Accepted Manuscript version arising from this submission. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

3. One of the noted authors is a group or consortium [LSTM Diagnostics group, CONDOR steering group]. In addition to naming the author group, please list the individual authors and affiliations within this group in the acknowledgments section of your manuscript. Please also indicate clearly a lead author for this group along with a contact email address.

The individual authors alongside their affiliations have been listed in the Acknowledgements section from line 294.

Reviewer #1: The paper describes a well conducted study comparing self-taken and HCW taken throat/nasal swabs to perform RDT for SARS-CoV-2. The authors show that self-taken swabs had a higher sensitivity, when compared to RT-PCR and that the NPV of self-taken swabs RDTs was significantly higher than the NPV of HCW-taken RDTs.

Some minor issues:

RT-PCR is used as gold standard, which should be mentioned in the abstract and methods.

We have updated the abstract and the main methodology to make sure this is clear:

Line 41 in abstract

RDT results were compared to RT-PCR as the reference standard to calculate sensitivity and specificity.

Line 136 in the Methods

RT-PCR was used as the reference standard test in this study for comparison of the RDT results.

Interpretation:

Statements are made on “convenience” and “reduced risk of transmitting infections”, both of which may be true, but are not subject of this study.

The abstract interpretation has been rewritten to focus on the study outcomes, line 54:

In conclusion, self-taken swabs for COVID-19 testing offer an accurate alternative to healthcare worker taken swabs for use with RDTs.. Our results demonstrate that, with no training, self-taken throat/nasal samples can be used by lay individuals as part of rapid testing programmes for symptomatic adults. This is especially important where the lack of trained healthcare workers restricts access to testing.

In the discussion line 243 the language has been changed to reflect this:

Using self-sampling for testing could reduce the workload of health workers and increase the ability of services to test patients in both clinical and research settings where trained workers are not available.

The final paragraph has been rewritten to focus on the study outcomes line 282:

In conclusion, self-taken swabs for COVID-19 testing offer an accurate alternative to healthcare worker taken swabs for use with RDTs. Our results demonstrate that, with no training, self-taken throat/nasal samples can be used by lay individuals as part of rapid testing programmes for symptomatic adults. Self-testing has the potential to widen access to early diagnosis for COVID-19 in clinical services and outreach settings; allowing access to therapies in the early stage of the illness. This is especially important where the lack of trained healthcare workers restricts access to testing.

As limitation of the study the non-randomized sampling order is mentioned. Reference 15 is cited in line 246 to illustrate that repeated sampling from one nostril does not impact RT-PCR sensitivity or CT values. However this paper shows that this holds for nasal mid-turbinate specimens, while it seems that anterior nasal specimens are used in this study, I think this should be mentioned in the discussion.

Paragraph commencing line 256 rewritten to reflect this:

The limitations of this study are that the sampling order was not randomised as the HCW swab for RDT was taken after the swab for the RT-PCR, and only from one nostril; this may lower sensitivity, or participants may experience sampling fatigue. However, previous studies have shown that repeated sampling from one nostril using nasal mid-turbinate specimens does not impact RT-PCR sensitivity or CT values so, although we used throat/nasal sampling, this repeated sampling may not be the reason for lower sensitivity. 15 Participation was voluntary so people who were less confident to take their own sample may not have agreed to take part; it is also likely participants may have done previous COVID-19 tests, and so have experience of self-sampling.

Reviewer #2: The manuscripts describes the comparison of self-taken and healthcare worker -taken throat swabs in rapid diagnostic tests (RTDs). The authors show the unexpected result of a higher sensitivity in self taken swab RTDs compared to healthcare taken swabs. The final conclusion of the paper is that RTDs offer substantial individual benefits.

Major comments

- In the methods (line 128) there is a multiplex SARS PCR described, but no data is shown on theses different genes, are there any discrepancies shown? I assume that all analyses are done on the spike protein, but this is not clear in the methods.

A mean CT value was calculated from the genes tested to give a single CT value for analysis. This has been updated in the sample size and statistical analysis section and also stated in the results with reference to the full data set provided as a supplementary table:

Line 145

For each RT-PCR result with three target genes, the mean of the three CT values was taken to give a single RT-PCR CT result for each sample; RT-PCR results with mean cycle threshold (CT) values <40 were considered positive and CT values ≥40 were graded as negative.

Line 218

Sensitivity of the RDTs varied by mean CT values (Table 4 and Figure 2); a full table of mean and CT values from each gene tested is available in the supplementary data.

- Why is there not chosen to include an extra IC for a household gene (like beta-actin/ beta-globin), this might give some additional data on the quality of the swabs.

The PCR used was a WHO approved PCR and therefore we were not able to make alterations to it. We believe this is a really interesting idea and we believe would make a complementary piece of follow-up using an in-house PCR however was outside the scope of this study.

- Table 4. The difference in sensitivity is highly influenced by the group of samples with a PCR range 25-29.9, this is not mentioned or discussed. This is probably due to the small sample size in combination with Ct value, and possibly not to the swab method/quality. Are there any control experiments done to support that the turning point of the test indeed is in this range of Ct value.

Line 264 an additional limitation was added to discuss the influence this small group of results has had on the overall sensitivity:

The majority of participants with discrepant results between self-taken and HCW taken RDT results had a mean CT of between 25-30 on PCR (six of ten). There was only a small number of participants with a CT value within this range however this group makes a large difference to the sensitivity of the HCW taken RDT tests. This group was too small to analyse further statistically; it is noted similar results were noted in the trial by García-Fiñana et al 4 in the group with a CT greater than 24.4. In future studies evaluating RDTs for SARS-CoV-2 this may be an important subset to review as this may influence test performance.

Minor comment

- Abstract: in my opinion the abstract does not really invite to read the rest of the paper, background is only the method and hypothesis, and the findings is almost only numbers.

Thank you for this feedback, we have revised the abstract at the start of the work in line with the comments of all the reviewers.

- Table 1: Ethnicity, vaccination status and the different symptoms are nicely reported, but there is no further link to the data.

On line 177 a link to the complete dataset within the supplementary files is provided.

A full data table of all participant characteristics and results is available in the supplementary data.

- Table 3/Line 120: I am not sure what is meant by 'read graded'? In the results there is also no discrepancies shown in table 3., so I am wondering what the additional value is to the test and to the paper.

The Covios test used has a visual grading card for the results line to give a numerical result. We reported the results as per the manufacturers guidance and for completion included these results in the manuscript. We have updated the description of the reading of the tests, line 125:

RDTs were run according to the manufacturer’s instructions; this includes grading the result line from 0-10 on the RDT using a visual reference card. Each test was read by two trained researchers, if there was a disagreement between the two readers, a third reader was requested.

Table 3 has been updated to focus on the positive/negative results with the visual graded result in brackets as for the CT value.

Participant RT-PCR result (mean Ct value) Self-taken RDT result (Reader 1/Reader 2) HCW-taken RDT result (Reader 1/Reader 2)

26 Positive (29.03) Positive (4/4) Negative (0/0)

38 Negative (>40) Negative (0/0) Positive (2/2)

106 Positive (21.68) Positive (3/3) Negative (0/0)

131 Positive (27.99) Positive (3/3) Negative (0/0)

140 Positive (25.44) Positive (3/3) Negative (0/0)

169 Positive (28.09) Positive (2/2) Negative (0/0)

189 Positive (20.35) Positive (8/8) Negative (0/0)

195 Positive (22.96) Positive (3/3) Negative (0/0)

231 Positive (27.33) Positive (6/6) Negative (0/0)

249 Positive (27.86) Positive (5/5) Negative (0/0)

Reviewer #3: This is an interesting study. In order to give more clarity about the obtained results more details are required.

1. I in the Methods section the data are described. It seems none of the participants has been examined by a medical doctor to identify the health state. Instead, self-assessment as been used. Is this correct?

Specifically, each participant confirmed to have high temperature, continuous cough or change in smell of taste etc. Has the temperature been measures to confirm high temperature? Has any other examination or measurement been made?

Please adjust/modify the provided information correspondingly to reflect the actual situation.

Line 102 in the Methods section has been clarified:

Demographic and self-reported symptom data were recorded electronically; a list of symptoms from the FIND alliance and participants could also list additional symptoms they felt were relevant.

Line 174 has also been clarified in the Results section:

The most common self-reported symptoms by participants were…

2. If no medical examination has been performed this has to be clearly stated. This is not ideal because the symptoms mentioned in the paper are not unique to Covid-19. This should be also stated.

We have clarified the self-reporting of symptoms as above in response to the first comments above.

If all this is the case it seems two tests have been compared without precise information about the underlying health state of the patient.

Using a prospective evaluation we aimed to account for the fact that the majority of symptomatic participants were unlikely to have current SARS-CoV-2 infection therefore we calculated a sample size based on 20% prevalence of SARS-CoV-2 in the tested population in this setting in the UK. Using RT-PCR positivity as the gold standard to assess for presence or absence of SARS-CoV-2 infection allowed us to compare the two RDTs taken concurrently.

3. The authors provide an estimate for the sample size assuming a prevalence of 20%. I assume this value is used for the estimate which results in 308 samples? The problem is a prevalence of 20% refers to the population of the UK. In contrast, the authors test only very specific sub-population which self-evaluates as suffering from covid. Hence, the prevalence is different. Please comment and revise correspondingly.

This prevalence was taken from the positivity testing rate for SARS-CoV-2 in Liverpool where the study was to take place in the week commencing 21st January 2021 when the study size was calculated. This was the prevalence in people self-identifying COVID-19 and presenting for testing in Liverpool U.K. We conducted the study within a PCR testing site, so this was the prevalence in the population we were studying.

We have updated the text at line 138 to clarify this point:

Sample size was calculated using an alpha of 0.05, anticipated prevalence of 20% (based on the positivity rate of PCR tests of individuals presenting for testing within Liverpool in the week commencing 21st January 2021, calculated by Public Health England 10), minimum test sensitivity of 80%, specificity 99%, and precision interval of 10%.

4. On line 157 it is mentioned that 250 participants have been used while the sample size estimate gave 308. Does this mean the study is by design underpowered?

Recruitment was halted early as high number of participants tested positive by RDT meaning prevalence was higher than in the calculated sample size and it was felt no further information would be gained by recruiting further participants.

5. The most crucial point of the study relates to the CT value of the RT-PCR test because it is well known that this has a severe impact (add citation).

The authors report different CT values (which is good) but the results therefor are unclear.

Specifically, the results in Tab 2 seem to contradict the literature showing that the higher CT the fewer the positive results. Please clarify this counter intuitive result.

For PCR methodology the more amplification cycles needed to identify the virus mean that less virus is present initially, therefore this is consistent with a lower positivity rate of RDT tests at higher CT values (as more cycles were needed to get a positive result). Within this test therefore this is a result that is consistent and found in many studies, including within the population in Liverpool.

6. The results about the different CT values seem to be the most interesting ones of the study. The discussion should be extended and the finding are highlighted.

We have extended the discussion around CT values and positivity rates as described above in response to the comments above from reviewer 2 and also this feedback and the comments above:

Line 264 - The majority of participants with discrepant results between self-taken and HCW taken RDT results had a mean CT of between 25-30 on PCR (six of ten). There was only a small number of participants with a CT value within this range however this group makes a large difference to the sensitivity of the HCW taken RDT tests. This group was too small to analyse further statistically; it is noted similar results were noted in the trial by García-Fiñana et al 4 in the group with a CT greater than 24.4. In future studies evaluating RDTs for SARS-CoV-2 this may be an important subset to review as this may influence perceived test performance.

7. I did not find information how the data could be obtained.

Clarification added to Line 177 that a full data table is available in the supplementary material:

A full data table of all participant characteristics and results is available in the supplementary data.

Attachment

Submitted filename: Response to reviewers.docx

Decision Letter 1

Sylvia Maria Bruisten

7 Jun 2022

PONE-D-22-05402R1A prospective diagnostic evaluation of accuracy of self-taken and healthcare worker-taken swabs for rapid COVID-19 testingPLOS ONE

Dear Dr. Savage,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

 Since only one of the previous reviwers was available to look at the rebuttal I have now also looked at it as reviewer number 4. There is still a comment on the lack of (internal) controls and the small group of samples with relatively high Ct values, which have a large influence on the results. Please clarify if this has an impact on the main message of your paper.

Please submit your revised manuscript by Jul 22 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

  • A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

  • A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

  • An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Sylvia Maria Bruisten, Ph.D

Academic Editor

PLOS ONE

Journal Requirements:

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

Additional Editor Comments:

Since there was only one of the previous reviewers available to look at the rebuttal I have now also looked at it as reviewer number 4.

Reviewer 2 still has a problem with the comment that there is a large influence of the group of samples with Ct values ranging between 25 and 30 and that this is a small group. If possible you should further discuss why you think that the data are still sound. Please make more clear that this will not influence the message of the paper.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #2: (No Response)

Reviewer #4: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #2: Partly

Reviewer #4: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #2: I Don't Know

Reviewer #4: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #2: Yes

Reviewer #4: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #2: Yes

Reviewer #4: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #2: In my opinion the comments are adressed only minor, and despite the conlusions are discussed in a better way, the paper still lacks the proper controls/control experiments supporting the differences shown driven by the group with PCR positivity 25-29.

Reviewer #4: The authors did a good job to answer all points made by the reviewers. It is an interesting study that deserves to be published.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #2: No

Reviewer #4: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PLoS One. 2022 Jun 30;17(6):e0270715. doi: 10.1371/journal.pone.0270715.r004

Author response to Decision Letter 1


10 Jun 2022

Dear Editor,

We would like to thank the editor for their constructive comments and recommendations made to this manuscript. We have reviewed the feedback from yourself and Reviewer 2 and have made the revisions suggested and have prepared an itemized reply to all issues raised on the following pages as requested.

We have especially focused on the comments and recommendations from Reviewer 2 and have reviewed our data to address their concerns. We have ensured that the subset of participants does not change the outcome or message of the paper and made this much clearer in the limitations section of our work. We hope that the addition of this information will be satisfactory and have also included their suggestions for further work which we are sadly unable to carry out as we do not have the scope or resources to undertake additional experiments at this time.

We hope these replies are satisfactory to the reviewers and your office.

We look forward to your decision.

Best regards,

Helen Savage

On behalf of all authors.

PONE-D-22-05402R1

A prospective diagnostic evaluation of accuracy of self-taken and healthcare worker-taken swabs for rapid COVID-19 testing

Response to comments from Editor

Since only one of the previous reviewers was available to look at the rebuttal I have now also looked at it as reviewer number 4. There is still a comment on the lack of (internal) controls and the small group of samples with relatively high Ct values, which have a large influence on the results. Please clarify if this has an impact on the main message of your paper.

Reviewer 2 still has a problem with the comment that there is a large influence of the group of samples with Ct values ranging between 25 and 30 and that this is a small group. If possible you should further discuss why you think that the data are still sound. Please make more clear that this will not influence the message of the paper.

Thank you for these helpful comments, and we will respond to the comments from the Editor and from Reviewer to together here.

We have carefully rereviewed the data in the manuscript and revised the wording of the text in the Discussion section to make clear that the subset of data would not change the overall conclusion of the paper if excluded and further work that could be undertaken to characterise this group.

The revised text (Line 265 – 279) now reads:

The majority of participants with discrepant results between self-taken and HCW taken RDT results had a mean CT of between 25-30 on PCR (six of ten). Although there was only a small number of participants with a CT value in this range, these values are influential in estimates of the sensitivity of the HCW taken RDT tests. Because of the small numbers, we did not undertake further formal statistical analysis; however when participants with CT values within this range were excluded, our main conclusion that self-taken swabs were non-inferior to healthcare worker taken swabs did not change (sensitivity with CT 25-29.9 excluded: self taken RDT 91.7% [55/60]; HCW taken 88.3% [53/55]). Similar results were noted in the trial by García-Fiñana et al4 who also saw a decrease in sensitivity of RDTs compared to PCR in samples with a CT of greater than 24.4. Future research could compare self-taken and healthcare worker-taken samples by PCR including housekeeper genes to see if the quality of sampling varies and may affect CT values, particularly for samples with CT values over 25, as lower sampling quality could result in lower CT values and therefore a perceived poorer performance in this range.

Response to Comments from Reviewer 2

In my opinion the comments are addressed only minor, and despite the conclusions are discussed in a better way, the paper still lacks the proper controls/control experiments supporting the differences shown driven by the group with PCR positivity 25-29.

Thank you for the suggestion. Please additionally see the Response to the Editor’s comments above where we have aimed to make it clear that this small group does not affect the conclusions of the paper. The PCR used was a WHO approved PCR and therefore we were not able to make alterations to it but we also agree this is an interesting idea and we believe would make a complementary piece of follow-up work using an in-house PCR and have therefore included this within the limitations as a piece of additional experimentation that would be beneficial as future research.

Attachment

Submitted filename: Response to reviewers.docx

Decision Letter 2

Sylvia Maria Bruisten

16 Jun 2022

A prospective diagnostic evaluation of accuracy of self-taken and healthcare worker-taken swabs for rapid COVID-19 testing

PONE-D-22-05402R2

Dear Dr. Savage,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Sylvia Maria Bruisten, Ph.D

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

All points were addressed, also the last one, specifying that the the small group of samples with elevated Ct values did not alter the conclusions.

Reviewers' comments:

Acceptance letter

Sylvia Maria Bruisten

22 Jun 2022

PONE-D-22-05402R2

A prospective diagnostic evaluation of accuracy of self-taken and healthcare worker-taken swabs for rapid COVID-19 testing

Dear Dr. Savage:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Sylvia Maria Bruisten

Academic Editor

PLOS ONE

Associated Data

    This section collects any data citations, data availability statements, or supplementary materials included in this article.

    Supplementary Materials

    S1 Appendix

    (DOCX)

    S1 Table. A table of all PCR positive participants and their characteristics.

    (DOCX)

    S1 File. Covios® leaflet from study.

    (DOCX)

    Attachment

    Submitted filename: Response to reviewers.docx

    Attachment

    Submitted filename: Response to reviewers.docx

    Data Availability Statement

    All relevant data are within the paper and its Supporting Information files.


    Articles from PLoS ONE are provided here courtesy of PLOS

    RESOURCES