Skip to main content
PMC home page
PLOS ONE logoLink to PLOS ONE
. 2020 Dec 16;15(12):e0244417. doi: 10.1371/journal.pone.0244417

The accuracy of healthcare worker versus self collected (2-in-1) Oropharyngeal and Bilateral Mid-Turbinate (OPMT) swabs and saliva samples for SARS-CoV-2

Seow Yen Tan 1,*,#, Hong Liang Tey 2,#, Ernest Tian Hong Lim 3,#, Song Tar Toh 4,#, Yiong Huak Chan 5, Pei Ting Tan 6, Sing Ai Lee 7, Cheryl Xiaotong Tan 8, Gerald Choon Huat Koh 9,‡, Thean Yen Tan 10,‡, Chuin Siau 11,‡
Editor: Dong-Yan Jin12
PMCID: PMC7744114  PMID: 33326503

Abstract

Background

Self-sampling for SARS-CoV-2 would significantly raise testing capacity and reduce healthcare worker (HCW) exposure to infectious droplets personal, and protective equipment (PPE) use.

Methods

We conducted a diagnostic accuracy study where subjects with a confirmed diagnosis of COVID-19 (n = 401) and healthy volunteers (n = 100) were asked to self-swab from their oropharynx and mid-turbinate (OPMT), and self-collect saliva. The results of these samples were compared to an OPMT performed by a HCW in the same patient at the same session.

Results

In subjects confirmed to have COVID-19, the sensitivities of the HCW-swab, self-swab, saliva, and combined self-swab plus saliva samples were 82.8%, 75.1%, 74.3% and 86.5% respectively. All samples obtained from healthy volunteers were tested negative. Compared to HCW-swab, the sensitivities of a self-swab sample and saliva sample were inferior by 8.7% (95%CI: 2.4% to 15.0%, p = 0.006) and 9.5% (95%CI: 3.1% to 15.8%, p = 0.003) respectively. The combined detection rate of self-swab and saliva had a sensitivity of 2.7% (95%CI: -2.6% to 8.0%, p = 0.321). The sensitivity of both the self-collection methods are higher when the Ct value of the HCW swab is less than 30. The specificity of both the self-swab and saliva testing was 100% (95% CI 96.4% to 100%).

Conclusion

Our study provides evidence that sensitivities of self-collected OPMT swab and saliva samples were inferior to a HCW swab, but they could still be useful testing tools in the appropriate clinical settings.

Introduction

The current “gold standard” for testing for SARS-CoV-2 requires health care workers to collect a nasopharyngeal (NP) sample from a patient. NP sampling is very uncomfortable for the patient and requires deployment of trained personnel and use of personal protective equipment (PPE) which are in limited supply.

A prior study has shown that a combination of oropharyngeal and anterior nares swabs is equivalent in sensitivity to an NP swab in 190 ambulatory symptomatic patients [1]. In another study on 236 ambulatory subjects, the performance of self-collected nasal and throat swabs is at least equivalent to that of health worker collected swabs for the detection of SARS-CoV-2 and other respiratory viruses [2].

The international community is actively searching for an even less invasive means of sample collection: saliva. In a recent study by Yale University on 29 subjects [3], it was suggested that a large volume sample of saliva collected from COVID-19 inpatients can be more sensitive than NP swabs for SARS-CoV-2 detection, and saliva samples had significantly higher COVID-19 viral titres than NP swabs (p = 0.001). Furthermore, the same study showed that sensitivity of COVID-19 in saliva was more consistent throughout extended hospitalization compared to NP swabs.

In addition, there are a number of studies done on saliva testing for COVID-19 which have shown promising results, reporting 91.7%, and 100% positivity in saliva samples of COVID-19 patients [4, 5]. Iwasaki et al found an overall concordance rate of 97.4% for COVID-19 detection with a strong concordance between NP swabs and saliva sampling (κ = 0.874) among 66 COVID-19 negative and 10 COVID-19 positive subjects [6]. Furthermore, a study done by To et al. showed that viral RNA could still be detected in saliva samples in a third of their twenty-three patients 20 days or longer after symptoms onset despite the development of COVID-19 antibodies [7]. A meta-analysis conducted on 26 saliva studies also showed a positive detection rate of 91%, comparable to the detection rate of 98% from nasopharyngeal swabs [8]. All these studies had small sample sizes (all <30 COVID-positive subjects) and only one study also sampled COVID-negative subjects.

It is still currently unknown whether a self-collected combined Oropharyngeal and Bilateral Mid-Turbinate (OPMT) sample, or a self-collected saliva sample is equivalent to a swab done by a health care worker (HCW). If the self-collection of samples is proven to be a reliable alternative to a HCW swab, it would reduce the reliance of trained personnel to collect samples and enable a rapid increase in testing capacity. It would also reduce greatly the biosafety risk that is posed to HCWs and help with PPE conservation efforts.

Materials and methods

Study design and trial oversight

This was a prospective study involving 401 subjects who were previously tested positive for COVID-19 by RT-PCR, and 100 healthy volunteers. This study was approved by the SingHealth Centralised Institutional Review Board. Written informed consent was obtained from the subjects.

Participants

The first group consisted of patients who were confirmed to have COVID-19, and who were cared for in either a hospital (Changi General Hospital), or a community care facility (Community Care Facility @ EXPO). Diagnosis of COVID-19 was confirmed via a positive RT-PCR from a nasopharyngeal swab. The subjects in this group were recruited within 3 days of admission to the study site and they were recruited from 31 May 2020 to 10 June 2020. The patients who were eligible were approached directly at the study site, and were invited to participate in the study, and the study procedures were carried out on the same day. Recruitment was carried out until the target sample size was achieved. At the time of the study, the majority of COVID-19 cases belong to the migrant worker population, which primarily consisted of healthy young male adults, mainly from Bangladesh and India. Hence, this group of subjects is not representative of the general population in Singapore.

Inclusion criteria applicable to this group include:

  • Male and female patients, ≥ 21 years-old

  • Tested positive for COVID-19

  • Admitted to study site within the previous 3 days

  • Ability to provide informed consent

  • Compliance with all aspects of study protocol, methods and provision of samples

  • Ability to read and understand English

Exclusion criteria applicable to this group include:

  • Nosebleeds in past 24 hours

  • Previous nasal surgery in past 4 weeks

  • Acute facial trauma within 8 weeks

  • Unable to demonstrate understanding of study and instructions

  • Experienced severe adverse reactions on prior nose and/or throat swabs

  • Not willing to have all 3 samples collected

The second group comprised 100 healthy volunteers who were asymptomatic and well on the day of the study, with no recent COVID-19 exposure. This was done on 18 and 19 July 2020. The study subjects were recruited via an open advertisement.

Inclusion Criteria for this group include:

  • Males and females, ≥ 21 years-old

  • Ability to provide informed consent

  • Capable of understanding and complying with the requirements of the study

  • Ability to read and understand English

Exclusion Criteria applicable to this group were:

  • Displaying symptoms of an acute respiratory infection

  • Known close contact with an individual diagnosed with COVID-19 within the last 3 months

  • Previously diagnosed with COVID-19

  • Nosebleeds in past 24 hours

  • Previous nasal surgery in past 4 weeks

  • Acute facial trauma within 8 weeks

  • Unable to demonstrate understanding of study and instructions

  • Experienced severe adverse reactions on prior nose and/or throat swabs

  • Not willing to have all 3 samples collected

Test procedures

Study subjects underwent three sequential test sample collection procedures within one study visit in the following order:

  1. Each subject self-collected a sample combining OP and bilateral MT swabs using a single swab stick;

  2. A trained healthcare worker then collected a combined OP and bilateral MT swab using another single swab stick;

  3. The subject then self-collected a saliva sample.

Study subjects were shown instructional videos for both the OPMT self-swab and saliva collection prior to commencing the test procedures. Study team members were present on site to observe and supervise the self-collection process. Posterior oropharyngeal saliva, commonly described as deep throat saliva was collected for this study. Synthetic fibre swabs were used for collection of the OP and MT samples by both subject and healthcare worker, and immediately placed in universal transport medium (UTM), while saliva samples were collected using the SAFER-Sample™ (by Lucence Diagnostics). All samples were double bagged and stored at air-conditioned room temperature in a chiller bag and transported to assigned laboratory on the same day. Upon arrival in the laboratory, they were stored at 2°C to 8°C. All samples were processed with 24 hours of sample collection.

Nucleic acid extraction was performed using PerkinElmer Nucleic Acid Extraction Kits (KN0212) on the Pre-Nat II Automated Workstation (PerkinElmer®, United States), Extraction of swab samples followed the indicated protocol for oropharyngeal swabs, while extraction of saliva samples followed a protocol consisting of pre-liquefaction with dithiothreitol (protocol attached in S1 File). Reverse transcription polymerase chain reaction (RT-PCR) was performed on the QuantstudioTM 5 Real Time PCR system (Thermo Fisher, United Kingdom) using the PerkinElmer® SARS-CoV-2 Real-time RT-PCR Assay. The targets were the ‘N’ gene and ‘ORF1ab’ gene. There is an internal control target that is present in every RT-PCR reaction. The cycle threshold (Ct) values of the ‘N’ gene were used in the analysis involving Ct values.

Outcomes

The primary objective of the study was to evaluate the accuracy of self-collected (2-in-1) OPMT swabs and self-collected saliva samples for SARS-CoV-2 versus that of HCW-collected (2-in-1) MT and OP swabs. The secondary objective was to evaluate the correlation of PCR Ct values of self-collected saliva samples and swabs with comparator healthcare worker-collected swabs.

Sample size

Firstly, we postulated that OPMT self-swabbing was as accurate as HCW-obtained swabs. Postulating a 100% accuracy, 400 subjects will be required to achieve a lower 95% confidence interval 99.1% (which gives a less than 1% error rate). With the computed sample size of 400 subjects, a non-inferiority could be achieved with at most a 7% difference for OPMT self-swabbing compared to the HCW-obtained swabs. If the study included only subjects who were diagnosed with COVID-19, all positive results would be regarded as true positives. Hence, to address that gap in the form of specificity of self-swabs and saliva testing in the diagnosis of COVID-19, a further study on 100 healthy subjects was conducted. The hypothesis was that with 100% accuracy, the error rate for a false negative was 3.6%.

Statistical analysis

All analyses were performed using SPSS 25.0 with statistical significance set at p < 0.05.

The estimates for the positivity results of the 3 methods were presented as numbers and percentages. The differences with 95% confidence interval (CI) between self-collection methods and HCW-obtained swabs to assess for non-inferiority was calculated. Sensitivity and specificity of the two self-collection methods were compared with HCW-obtained swabs and results were stratified by Ct values. Spearman’s test was used to assess the correlation of the PCR Ct values across the 3 groups.

Results

A total of 401 COVID-19 positive and 100 COVID-19 negative subjects were recruited. Of the 401 COVID-19 positive subjects, 23 were recruited from Changi General Hospital, and 378 were recruited from the community care facility @ Expo. The symptomatic COVID-19 positive subjects that were recruited were well patients, whose clinical presentation was that of an upper respiratory tract infection. None of the subjects required oxygen supplementation.

Only the demographic data of subjects from Changi General Hospital was known. The full demographic data of the subjects that were admitted to the community care facility could not be made available to us due to prevailing regulations of the study site during the period when the study was conducted, hence we do not have the data of the age of the subjects that were admitted to the community care facility. However, we were able to surmise that the age range of patients admitted to the community care facility was 21 to 45, due to the admission criteria to the facility, and the inclusion criteria for the study. A summary of the profile of recruited subjects are listed in Tables 1 and 2 below.

Table 1. Profile of COVID-19 positive subjects (N = 401).

N (%)
Age, years
Min–max 21–54
Mean (SD)* 37.26 (6.4)
Male 401 (100.0)
Presence of symptoms on study day
No 359 (89.5)
Yes 42 (10.5)
Duration between illness onset to study day, days
Range 1–25
Mean (SD) 5.65 (2.1)
Days from first positive swab to study day, days
Range 1–20
Mean (SD) 5.48 (1.8)
Open in a new tab

* Calculation based on the known age of 24 subjects.

Table 2. Profile of COVID-19 negative subjects (N = 100).

Gender N (%)
Female 51 (51.0)
Male 49 (49.0)
Age (years)
Mean (SD) 38.24 (10.16)
Range 22–70
Open in a new tab

All subjects went through the test procedures—500 participants (400 COVID-19 positive, 100 COVID-19 negative) were able to provide all 3 samples, and one subject was unable to provide a saliva sample despite a prolonged attempt. All participants tolerated the test procedures well and did not experience any adverse events.

In the group of subjects who were COVID-19 positive, twenty-seven (6.7%) patients were tested negative across all 3 samples. This may be explained by the fact that they are recovering and viral shedding may have ceased at point of testing. Forty-two (10.5%) subjects reported ≥1 symptom of acute respiratory infection (ARI) (e.g. fever, cough, rhinorrhoea, sore throat, malaise) on the day of study recruitment while 371 (92.5%) subjects reported being within 7 days from onset of COVID-19 illness.

The detection rates of the HCW swab, self-swab, saliva, and combined self-swab plus saliva samples were 82.8%, 75.1%, 74.3% and 86.5% respectively (Table 3). Compared to HCW-swabs, the detection rate was lower for self-swab by 8.7% (95% confidence interval, CI = 2.4% to 15.0%, p = 0.006) and for saliva samples by 9.5% (95%CI = 3.1% to 15.8%, p = 0.003). When the results of both the self-swab and saliva testing were combined, the detection rate was higher by 2.7% (95%CI = -2.6% to 8.0%, p = 0.321) but this was not statistically significant.

Table 3. Detection rates of various modalities in all subjects.

HCW Swab Self-Swab Saliva Self-Swab + Saliva
Count 336 301 297 347
Percentage 83.8% 75.1% 74.3% 86.5%
95% CI 79.8% - 87.3% 70.1% - 79.2% 69.7% - 78.5% 82.8% - 89.7%
Open in a new tab

The sensitivities of the self-swab, saliva and combined self-swab plus saliva testing, when compared to the HCW swab were 83.6%, 80.6% and 92.3% respectively. Table 4 shows the contingency tables comparing the HCW swab vs self-swab; HCW swab vs saliva, and HCW vs combined self-swab plus saliva respectively.

Table 4. Comparison between HCW swabs and the self-swab/saliva.

HCW Swab
Not detected Detected p value*
Self-swab
Not detected 45 55 <0.001
Detected 20 281 (83.6)
Saliva
Not detected 37 65 <0.001
Detected 27 270
Self-swab plus saliva
Not detected 27 26 0.207
Detected 37 310
Open in a new tab

* p value was obtained from McNemar test

Using the Ct values (‘N’ gene) of HCW swabs as reference, 3 categories of Ct values (i.e. <25, 25–30 and >30) were studied. It was observed that the sensitivity of self-swab (Table 5) and saliva testing (Table 6) performed better at the lower Ct values, suggesting that the sensitivity of self-collection methods approaches to that of HCW swab, when the viral load was higher.

Table 5. Sensitivity of self-swab, stratified by Ct values of HCW swab.

HCW Swab Ct Number of subjects Sensitivity
<25 60 100% (94.0–100)
25–30 81 96.3% (89.6–99.2)
>30 195 73.3% (66.5–79.4)
Open in a new tab

Table 6. Sensitivity of saliva, stratified by Ct values of HCW swab.

HCW Swab Ct Number of subjects Sensitivity
<25 60 96.7% (88.5–99.6)
25–30 81 92.6% (84.6–97.2)
>30 194 70.6% (63.7–76.9)
Open in a new tab

There was a good correlation of PCR Ct values between self-swab and HCW swab (r = 0.825, p<0.001) but moderate correlation between saliva samples and HCW swab (r = 0.528, p<0.001). The self-swab has a better agreement with the HCW swab. Using Wilcoxon Signed Rank Test, the difference in CT values between self-swab and HCW swab is statistically significant, where p = 0.026. Similarly for the saliva and HCW swab, where p<0.001. Figs 1 and 2 show the scatterplot of the correlation between the Ct values of the HCW swab and the self-swab as well as the saliva respectively. Table 7 shows the distribution of the Ct values of the 3 tests.

Fig 1. Correlation of Ct values of HCW swab and self-swab.

Fig 1

Open in a new tab

Fig 2. Correlation of Ct values of HCW swab and saliva.

Fig 2

Open in a new tab

Table 7. Distribution of Ct values of the HCW swab, self-swab and saliva.

Test Median (IQR*) of Ct value
HCW Swab 31.59 (26.77, 35.62)
Self-swab 31.65 (26.65, 35.94)
Saliva 33.10 (28.25, 36.23)
Open in a new tab

* IQR = Interquartile Range

One hundred healthy volunteers were recruited, and all of them were able to provide the 3 required samples. All the samples obtained from the healthy volunteers were tested negative for SARS-CoV2. This implies that the specificity of the self-swab and saliva sampling was 100% (95% CI 96.4% to 100%) with an error rate of 3.6% for having a false negative.

Discussion

This study shows that the sensitivity of a self-swab or saliva sample on its own is lower than HCW swab. However, the sensitivity of a combined self-swab and saliva collection is equivalent to that of a HCW swab. Another significant finding is that the self-swab and saliva samples have a higher sensitivity when the viral load is higher, and this generally occurs during the early stages of COVID-19. The sensitivity of both self-swab and saliva testing drops significantly when the Ct values of the HCW swab is more than 30. A study from Singapore [9] reported that viral cultures were negative from samples with Ct values > = 30 (i.e. when viral load is low), and the SARS-CoV-2 virus often cannot be isolated or cultured after day 11 of illness [10]. Thus, the results of this study support the use of self-testing methods as a replacement for a HCW swab in the early phase of COVID-19 illness when viral loads are high, and the sensitivities of the self-swab and saliva are similar to that of the HCW swab.

The strength of our study is the large number of subjects confirmed to have COVID-19. Besides that, the study also included a high proportion of asymptomatic individuals who were picked up because of Singapore’s proactive mass screening policy. The combination of self-swab and saliva sampling performed well in these asymptomatic subjects, implying that the strategy of combined self-testing, has the ability diagnose COVID-19 in asymptomatic individuals with a sensitivity equivalent to that of a swab by a HCW. The study results from the healthy volunteers indicate a low false positive rate with self-collection methods.

These findings, indicate that self-collection methods may be a useful tool for COVID-19 surveillance in the asymptomatic individuals, and in situations where testing capacity needs to be scaled up rapidly, without a need for large increase of manpower, and without increased infectious exposure to the swabbing staff. Testing strategies can be tailored based on the target population and the intended use of the various tests on its own or in combination.

The way the study findings were presented are unlike most studies involving saliva testing for COVID-19. This is probably due to the fact that our study is carried out on subjects who are already known to have COVID-19, unlike most studies which are done in testing centres where the potential subjects’ results are still unknown. This also meant that the sampling was done later in the subjects’ trajectory of illness, as they were first tested positive for COVID-19, then enrolled into the study. The later sampling possibly had a negative impact on the sensitivity of the saliva [11].

Another key study limitation, is that the demographics of the COVID-19-positive population was skewed, consisting solely of male migrant workers, the worst affected group of the pandemic in Singapore, at the time this study was conducted. Hence the results from this study might not be applicable to the general population, without the inclusion of paediatric and elderly population segments. The migrant worker population in this study, which consist of generally young and healthy males, is also not representative of the demographics of Singapore.

The addition of the stabilising solution to the deep throat saliva sample, could have also decreased the yield of the saliva testing. Studies utilising saliva test kits that do not require the addition of stabilising fluid generally report equivalent sensitivities of the saliva test when compared to a HCW swab [3, 12]. Hence the use of stabilising solution is a key consideration in future design of saliva test kits.

The study team members observed that, despite clear instructions, many subjects still needed guidance with the self-collection methods. For the self-swab, the most commonly encountered scenario was that, the subjects needed guidance in breaking the swab stick. The saliva collection presented a greater challenge to the subjects. The flow of saliva from the funnel into the collection container was not smooth, and the additional step of adding the stabilising fluid required prompting. These necessitated the presence of a trained staff to troubleshoot and ensure that the correct steps are carried out. We believe that these observations are useful in the re-design of collection containers to enhance results and end users’ acceptability. Both the self-swab and saliva collection require dexterity and this would limit its applicability in segments of the population who are not able to do so.

We caution against widespread, unsupervised implementation of self-collection methods. The reliability and effectiveness of self-collection methods may also be dependent on social and economic drivers, hence potentially influencing the test performance. For example, individuals who face a potential loss of income or unemployment if tested positive or travellers having a test done at immigration clearance may deliberately do a suboptimal self-test to influence the test outcome.

Hence, it is important to have designated personnel to supervise the self-collection process, ensuring that the correct test procedures are carried out. These personnel need not be a HCW and the supervision process will have a lower exposure risk (supervisor can be >1m away from subject), compared to the HCW-swabbing process where a HCW is <1m away and face-to-face with the subject.

Conclusion

This study demonstrates that while self-collection methods have a sensitivity of approximately 75%, it is inferior to the rate obtained by the health care worker administered swab (83.8%). The sensitivity of the self-collection methods is, however, higher and correlates better when Ct values of the HCW swabs are less than 30. The combined results of the saliva and self-swab test achieve a sensitivity equivalent to that of a health care worker administered swab. The specificity of the self-collection methods is 100%. Together with high specificity, we postulate that self-collection methods have their roles in diagnosis in early disease, where the viral load, and infectivity is high.

Supporting information

S1 File. Provisional protocol for saliva sample collected in Lucence SAFER kit.

(PDF)

Click here for additional data file. (252.6KB, pdf)
S2 File. Study protocol.

(PDF)

Click here for additional data file. (755KB, pdf)
S3 File. Table with Ct values of N gene.

(XLSX)

Click here for additional data file. (21.8KB, xlsx)

Acknowledgments

We thank all clinical, nursing and allied health staff who provided care for the patients at Changi General Hospital, and Community Care Facility @ EXPO; staff in the Changi General Hospital Clinical Trials & Research Unit for coordinating patient recruitment, logistics management and assistance.

Data Availability

All relevant data are within the manuscript and its Supporting Information files

Funding Statement

This study was funded by Sheares Healthcare Group Pte Ltd. The funder provided support in the form of salaries for authors SAL and CXT, but did not have any additional role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript. The specific roles of these authors are articulated in the ‘author contributions’ section. Besides that, author CXT is employed by Temasek International Pte Ltd, and was acting on behalf of Sheares Healthcare Group Pte Ltd for the study. Temasek International Pte Ltd did not have any additional role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript.

References

  • 1.LeBlanc JJ, Heinstein C, MacDonald J, Pettipas J, Hatchette TF, Patriquin G. A combined oropharyngeal/nares swab is a suitable alternative to nasopharyngeal swabs for the detection of SARS-CoV-2. J Clin Virol. 2020;128:104442 10.1016/j.jcv.2020.104442 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Wehrhahn MC, Robson J, Brown S, Bursle E, Byrne S, New D, et al. Self-collection: An appropriate alternative during the SARS-CoV-2 pandemic. J Clin Virol. 2020;128:104417 10.1016/j.jcv.2020.104417 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Wyllie AL, Fourmier J, Casanovas-Massana A, Campbell M, Tokuyama M, Vijayakumar P, et al. Saliva or Nasopharyngeal Swab Specimens for Detection of SARS-CoV-2. N Engl J Med 2020; 383:1283–1286. 10.1056/NEJMc2016359 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.To KK, Tsang OT, Yip CC, Chan K, Wu T, Chan JM, et al. Consistent detection of 2019 novel coronavirus in saliva. Clinical Infectious Diseases 2020; 71(15):841–3. 10.1093/cid/ciaa149 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Azzi L, Carcano G, Gianfagna F, Grossi P, Gasperina DD, Genoni A, et al. Saliva is a reliable tool to detect SARS-CoV-2. J Infect. 2020;81(1):e45–e50. 10.1016/j.jinf.2020.04.005 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Iwasaki S, Fujisawa S, Nakakubo S, Kamada K, Yamashita Y, Fukumoto T, et al. Comparison of SARS-CoV-2 detection in nasopharyngeal swab and saliva [published online ahead of print, 2020 Jun 4]. J Infect. 2020;81(2):e145–e147. 10.1016/j.jinf.2020.05.071 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.To KK, Tsang OT, Leung WS, Tam AR, Wu T, Lung DC, et al. Temporal profiles of viral load in posterior oropharyngeal saliva samples and serum antibody responses during infection by SARS-CoV-2: an observational cohort study. Lancet Infect Dis. 2020;20(5):565–574. 10.1016/S1473-3099(20)30196-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Czumbel LM, Kiss S, Farkas N, Mandel I, Hegyi A, Nagy Á, et al. Saliva as a Candidate for COVID-19 Diagnostic Testing: A Meta-Analysis. Front Med (Lausanne). 2020. August 4;7:465 10.3389/fmed.2020.00465 ; PMCID: PMC7438940. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Young BE, Ong SWX, Ng LFP, Anderson DE, Chia WN, Chia PY et al. Immunological and Viral Correlates of COVID-19 Disease Severity: A Prospective Cohort Study of the First 100 Patients in Singapore (4/15/2020). Available at SSRN: https://ssrn.com/abstract=3576846 or 10.2139/ssrn.3576846 [DOI]
  • 10.Position Statement from the National Centre for Infectious Diseases and the Chapter of Infectious Disease Physicians, Academy of Medicine, Singapore– 23 May 2020
  • 11.Jamal AJ, Mozafarihashjin M, Coomes E, Powis J, Li AX, Paterson A, et al. Sensitivity of nasopharyngeal swabs and saliva for the detection of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). Clin Infect Dis. 2020. June 25:ciaa848. 10.1093/cid/ciaa848 Epub ahead of print. ; PMCID: PMC7337630. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Rao M, Rashid FA, Sabri FSAH, Jamil NN, Zain R, Hashim et al. Comparing nasopharyngeal swab and early morning saliva for the identification of SARS-CoV-2. Clin Infect Dis. 2020. August 6:ciaa1156. 10.1093/cid/ciaa1156 Epub ahead of print. [DOI] [PMC free article] [PubMed] [Google Scholar]

Decision Letter 0

Dong-Yan Jin

14 Oct 2020

PONE-D-20-28601

The Accuracy of Healthcare Worker versus Self Collected (2-in-1) Oropharyngeal and Bilateral Mid-Turbinate (OPMT) Swabs and Saliva Samples for SARS-CoV-2

PLOS ONE

Dear Dr. Tan,

Thank you for submitting your manuscript to PLOS ONE.

Your manuscript has been reviewed by five experts in the field and their comments follow. All five reviewers like your work, but they also raised concerns and made specific suggestions. Addressing them in a revised paper will significantly improve the presentation and technical quality of your work. 

Please submit your revised manuscript by Nov 28 2020 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

  • A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

  • A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

  • An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

We look forward to receiving your revised manuscript.

Kind regards,

Dong-Yan Jin

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. In your Methods section, please provide additional information about the participant recruitment method and the demographic details of your participants. Please ensure you have provided sufficient details to replicate the analyses such as:

a) the recruitment date range (month and year),

b) a description of any inclusion/exclusion criteria that were applied to participant recruitment,

c) a table of relevant demographic details,

d) a statement as to whether your sample can be considered representative of a larger population, and

e) a description of how participants were recruited.

3. We note that you reference supplementary materials in your manuscript but there are no supplementary files attached. Please upload your supplementary files.

4. Thank you for providing the following Funding Statement: 

'This study was funded by Sheares Healthcare Group Pte Ltd. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.'

a. We note that one or more of the authors is affiliated with the funding organization, indicating the funder may have had some role in the design, data collection, analysis or preparation of your manuscript for publication; in other words, the funder played an indirect role through the participation of the co-authors.

If the funding organization did not play a role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript and only provided financial support in the form of authors' salaries and/or research materials, please review your statements relating to the author contributions, and ensure you have specifically and accurately indicated the role(s) that these authors had in your study in the Author Contributions section of the online submission form. Please make any necessary amendments directly within this section of the online submission form.

If the funding organization did have an additional role, please state and explain that role within your Funding Statement.

b. We also note that one or more of the authors are employed by another commercial company: Temasek Holdings

Please also declare this commercial affiliation, in your amended Funding Statement as well as a statement regarding the Role of Funders in your study. If the funding organization did not play a role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript and only provided financial support in the form of authors' salaries and/or research materials, please review your statements relating to the author contributions, and ensure you have specifically and accurately indicated the role(s) that these authors had in your study. You can update author roles in the Author Contributions section of the online submission form.

Please also include the following statement within your amended Funding Statement.

“The funder provided support in the form of salaries for authors [insert relevant initials], but did not have any additional role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript. The specific roles of these authors are articulated in the ‘author contributions’ section.”

If your commercial affiliation did play a role in your study, please state and explain this role within your updated Funding Statement.

c. Please also provide an updated Competing Interests Statement declaring these commercial affiliations along with any other relevant declarations relating to employment, consultancy, patents, products in development, or marketed products, etc.  

Within your Competing Interests Statement, please confirm that this commercial affiliation does not alter your adherence to all PLOS ONE policies on sharing data and materials by including the following statement: "This does not alter our adherence to  PLOS ONE policies on sharing data and materials.” (as detailed online in our guide for authors http://journals.plos.org/plosone/s/competing-interests). If this adherence statement is not accurate and  there are restrictions on sharing of data and/or materials, please state these.

Please note that we cannot proceed with consideration of your article until this information has been declared.

Please know it is PLOS ONE policy for corresponding authors to declare, on behalf of all authors, all potential competing interests for the purposes of transparency. PLOS defines a competing interest as anything that interferes with, or could reasonably be perceived as interfering with, the full and objective presentation, peer review, editorial decision-making, or publication of research or non-research articles submitted to one of the journals. Competing interests can be financial or non-financial, professional, or personal. Competing interests can arise in relationship to an organization or another person. Please follow this link to our website for more details on competing interests: http://journals.plos.org/plosone/s/competing-interests

Additional Editor Comments:

Please make your best effort to address the reviewers' concerns.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Partly

Reviewer #3: Yes

Reviewer #4: Yes

Reviewer #5: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: I Don't Know

Reviewer #3: Yes

Reviewer #4: Yes

Reviewer #5: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: Yes

Reviewer #3: Yes

Reviewer #4: No

Reviewer #5: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: No

Reviewer #3: Yes

Reviewer #4: Yes

Reviewer #5: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: In this study, Tan and colleagues present a comparison of self-collected swabs/ saliva vs. healthcare worker collected swabs. This is a controversial topic and worthy of exploration. Importantly, they find that self-collected options may be less sensitive than HCW collected swabs. The study is nicely performed and presented. It is particularly nice to see sample size analysis, which adds confidence in the conclusions. However, I do have a few recommendations to the authors to extract as much as possible from their data. I hope this helps them improve their manuscript further.

1. Please expand the abbreviation ‘OPMT’ when it first appears in the introduction.

2. Line 89: add a note on how patients in group 1 were confirmed to have COVID-19.

3. Can I confirm that the patient swabbed both mid-turbinates using a single swab stick and then put the same swab stick into their throat?

4. It is not clear what is meant by the ‘spitting’ method? Is this posterior oropharyngeal throat saliva collection? Collecting oral fluid, spitting out saliva, and collecting posterior oropharyngeal throat saliva are all likely to have different sensitivities for SARS-CoV-2 detection, so we need to define what exactly is being collected. Would actually be nice if the instructional videos could be uploaded as supplementary material.

5. Was there any particular timing of saliva collection? There is a tendency for early morning saliva to have higher viral loads (Hung DL et al, Open Forum Infect Dis, 2020).

6. Line 130 – 131: to clarify, the sample size was calculated based on a type I error rate of 1%?

7. Line 150: redundant ‘who’

8. Table 2 is not particularly useful and duplicates text in line 167 - 168. Could consider replacing with a contingency table of HCW swab vs saliva and HCW swab vs self-swab. This way, we can also check the % agreement and how many samples were detected by saliva/ self-swab, but not HCW swab.

9. Could add a McNemar test to compare sensitivities of saliva, self-swab and self-swab + saliva against the HCW-swab ‘gold standard’.

10. Table 3, 4: why is the total number of samples 336 (table 3), 335 (table 4)? The number of samples should be 401 – 27 (no. of negative samples) = 374? Are there missing data points?

11. Consider showing a scatterplot of the correlation of Ct values between self-swab and HCW swab and saliva and HCW swab.

12. Could include a column scatter plot comparing the RT-PCR Ct values of HCW swab, self-swab and saliva and statistically compare (? median) Cts of self-collected sample types to the HCW swab.

Reviewer #2: It is a great topic to compare the performance of self collected and HCW collected samples. But, the authors did not mention what clinical samples were used as gold standard for the diagnosis of COVID-19 in the 401 subjects. If HCW OP and MT swabs by HCW were used as gold standard, like procedure 2, listed under "test procedures", why the authors need to have a 2 stage design and include those previously tested positive for COVID-19 as subject and the second group of healthy volunteers as control? Why not include all the people when they were first tested for COVID-19 and test them with the three samples types at the very beginning?

The authors used the term 'detection rates' and 'negative correctness', are they referring to sensitivity and specificity which are more professional terms?

Reviewer #3: Tan SY and colleagues performed a cross-sectional study to investigate healthcare worker vs. self-collected OPMT swabs and saliva samples for the detection of SARS-CoV-2 among persons with a confirmed diagnosis of COVID-19 and healthy volunteers.

I have some concerns as follow:

Major comments:

1. Characteristics, e.g. age, the onset of symptoms, severity of the disease, of the study populations should be included to provide readers to understand the clinical setting of the study better. The different settings may associate with different sensitivity of each specimen. One study demonstrated the lower value of saliva for the diagnosis of COVID-19 in children (Chong CY, et al. Clin Infect Dis 2020; article in press). Many studies showed more testing agreement of saliva and nasopharyngeal swab at the earlier onset of the disease (Jamal AJ, et al. Clin Infect Dis 2020; article in press, Iwasaki S, et al. J Infect 2020; article in press).

2. The gene that was RT-PCR test should be described. Was a housekeeping gene included in the RT-PCR reaction? The presence of the housekeeping gene in the RT-PCR test could help to determine the adequacy of a specimen collection.

3. In the first paragraph of the discussion, the author stated: “Our findings corroborate with existing epidemiologic data which indicates that while viral RNA detection may persist in some patients, such persistent RNA detection likely represents non-viable virus and hence, such patients are noninfectious.” The author should demonstrate the results in their study that suggest the conclusion of this statement.

Minor comments:

1. The laboratory processing method that resulted in a lower yield of the saliva should be discussed. In this work, saliva samples were collected using the SAFER-Sample, which a saliva solubilizing solution was added to the samples. A preprint by Griesemer SB and coworkers demonstrated lower sensitivity to detect the virus when saliva stabilizing solution was added. Some studies showed a higher sensitivity of saliva when compare to nasopharyngeal swab. In these studies, the authors did not put either saliva stabilizing solution or viral transport media in the saliva specimen (Wylle AL, et al. N Engl J Med 2020; article in press; Rao M, et al. Clin Infect Dis 2020; article in press).

2. Please update the references. The preprints were accepted for publication.

3. The writing can be improved by reorganizing the content to increase the continuity of the idea and content in the manuscript.

Reviewer #4: The manuscript by Tan et al , compared the accuracy of healthcare worker versus self collected OPMT swabs and saliva samples for SARS-CoV-2. This is a comprehensive and important study during this COVID-19 pandemic.

I have the following minor comments:

- the authors may consider adding a flowchart for the workflow and sample size of the study

- the authors should include supplementary file for all single Ct values of different samples matching each patients

Reviewer #5: In the manuscript entitled “The Accuracy of Healthcare Worker versus Self Collected (2-in-1) Oropharyngeal and Bilateral Mid-Turbinate (OPMT) Swabs and Saliva Samples for SARS-CoV-2”, Seow Yen Tan et al compares the detection rate of SARS-C0V-2 by RT-PCR in self- collected samples and samples collected by HCW. In addition, they used saliva, OP swabs and both methods combined, and compared the sensitivity of each approach in 400 samples from patients diagnosed with COVID-19 and 100 negative subjects. They concluded that saliva and sel-collected samples are inferior to OP and HCW-collected ones, respectively. However, combining both self-collected samples provided a higher detection rate.

Some points to discuss:

1. How long it took to test saliva samples after collecting them? Although swab samples were preserved in transportation medium, saliva could be affected by time until testing;

2. Collecting saliva samples as the last procedure could affect results? Self-collected swab, followed by HCW collected swabs could interfere in the quantity and quality of saliva samples;

3. Line 167 “self-saliva” means “self-swab”?? Please, correct it;

4. In several parts of text authors refer to testing in “early phase” would provide better results: how early??In addition, there is no description on details of methodology, like time to test, preservation of samples, and characteristics of patients with positive and negative results. What about the time since diagnosis or duration of symptoms? Or severity of disease? These are important information to understand how samples were selected, and how such information could help explaining the results obtained;

5. Authors state at conclusion that self-collection would be preferable for use in low prevalence population? Why??

6. How author explain their results are so distinct of other reports, regarding sensitivity of saliva testing? It should be addressed in discussion. In limitations of study they cite the use of a very specific population (male migrants). How could this fact impact the results?

Minor comments: there are several typos and gramatical mistakes. A deeper English revision is warranted.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Siddharth Sridhar

Reviewer #2: No

Reviewer #3: No

Reviewer #4: No

Reviewer #5: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PLoS One. 2020 Dec 16;15(12):e0244417. doi: 10.1371/journal.pone.0244417.r002

Author response to Decision Letter 0

26 Nov 2020

Dear Editor,

Thank you for consideration of our manuscript. We have made considerable changes and will address your comments as well as the reviewers’ comments (in bold), point by point below.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

Have looked through the pages and the files seem to be fulfilling the requirement. Hope they are acceptable.

2. In your Methods section, please provide additional information about the participant recruitment method and the demographic details of your participants. Please ensure you have provided sufficient details to replicate the analyses such as:

a) the recruitment date range (month and year),

b) a description of any inclusion/exclusion criteria that were applied to participant recruitment,

c) a table of relevant demographic details,

d) a statement as to whether your sample can be considered representative of a larger population, and

e) a description of how participants were recruited.

The requested information have been included in the revised manuscript

3. We note that you reference supplementary materials in your manuscript but there are no supplementary files attached. Please upload your supplementary files.

Supplementary files have been uploaded. Apologies for missing them out in the initial submission.

4. Comment regarding affiliation of authors SAL and CXT to commercial entities.

I am unable to find the column to enter the amended funding statement in the entire revision process, hence the amended funding statement has been indicated in the cover letter as instructed by the video guide. The amended funding statement reads:

"This study was funded by Sheares Healthcare Group Pte Ltd.

The funder provided support in the form of salaries for authors SAL and CXT, but did not have any additional role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript. The specific roles of these authors are articulated in the ‘author contributions’ section.

This does not alter our adherence to PLOS ONE policies on sharing data and materials.

Besides that, author CXT is employed by Temasek Holdings, and was acting on behalf of Sheares Healthcare Group Pte Ltd for the study. Temasek Holdings did not have any additional role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript. This commercial affiliation does not alter our adherence to PLOS ONE policies on sharing data and materials."

Reviewer #1: In this study, Tan and colleagues present a comparison of self-collected swabs/ saliva vs. healthcare worker collected swabs. This is a controversial topic and worthy of exploration. Importantly, they find that self-collected options may be less sensitive than HCW collected swabs. The study is nicely performed and presented. It is particularly nice to see sample size analysis, which adds confidence in the conclusions. However, I do have a few recommendations to the authors to extract as much as possible from their data. I hope this helps them improve their manuscript further.

1. Please expand the abbreviation ‘OPMT’ when it first appears in the introduction.

- Abbreviation expanded as per recommendation

2. Line 89: add a note on how patients in group 1 were confirmed to have COVID-19.

- The sentence ‘Diagnosis of COVID-19 was confirmed via a positive RT-PCR from a nasopharyngeal swab’ was added.

3. Can I confirm that the patient swabbed both mid-turbinates using a single swab stick and then put the same swab stick into their throat?

- Correct, the same stick is used, but first by swabbing the throat, then the bilateral mid-turbinates.

4. It is not clear what is meant by the ‘spitting’ method? Is this posterior oropharyngeal throat saliva collection? Collecting oral fluid, spitting out saliva, and collecting posterior oropharyngeal throat saliva are all likely to have different sensitivities for SARS-CoV-2 detection, so we need to define what exactly is being collected. Would actually be nice if the instructional videos could be uploaded as supplementary material.

- We collected posterior oropharyngeal throat saliva, which we are now calling it deep throat saliva, which we hope would clarify the nature of the saliva that is collected. We are currently unable to upload the instructional video as the producer has not given the permission to upload it as a supplementary material.

5. Was there any particular timing of saliva collection? There is a tendency for early morning saliva to have higher viral loads (Hung DL et al, Open Forum Infect Dis, 2020).

- There was no particular timing for saliva collection. The only restriction we placed was that the subject should not have any food/drink 30 minutes prior to the saliva collection. We decided not to limit to early morning saliva, as when the test is being used on a large scale, it is likely that patients will be tested at any time of the day. Hence the study design to simulate a real life situation.

6. Line 130 – 131: to clarify, the sample size was calculated based on a type I error rate of 1%?

For clarity, we have changed ' An error rate of less than 1% was determined to be of clinical relevance so a sample size of at least 400 subjects was calculated' to ‘postulating a 100% accuracy , 400 subjects will be required to achieve a lower 95% Confidence interval 99.1% (which gives a less than 1% error rate)’

7. Line 150: redundant ‘who’

The word “who” deleted

8. Table 2 is not particularly useful and duplicates text in line 167 - 168. Could consider replacing with a contingency table of HCW swab vs saliva and HCW swab vs self-swab. This way, we can also check the % agreement and how many samples were detected by saliva/ self-swab, but not HCW swab.

Added table to address the comments in both Q8 and Q9

9. Could add a McNemar test to compare sensitivities of saliva, self-swab and self-swab + saliva against the HCW-swab ‘gold standard’.

Added table to address the comments in both Q8 and Q9

10. Table 3, 4: why is the total number of samples 336 (table 3), 335 (table 4)? The number of samples should be 401 – 27 (no. of negative samples) = 374? Are there missing data points?

The original Table 3 and 4 essentially captures all those that were positive of both tests. Table 3 contains the 336 subjects where both the self-swab and HCW swab were positive, and Table 4 contains the 335 that were positive on both the saliva and HCW swab.

11. Consider showing a scatterplot of the correlation of Ct values between self-swab and HCW swab and saliva and HCW swab.

Have added a scatterplot to show the correlation of the Ct values of the HCW swab and self-swab, and HCW swab and saliva.

12. Could include a column scatter plot comparing the RT-PCR Ct values of HCW swab, self-swab and saliva and statistically compare (? median) Cts of self-collected sample types to the HCW swab.

A table showing the distribution of the Ct values in the 3 tests were included as Table 7. Using Wilcoxon Signed Rank Test, the difference in CT values between self-swab and HCW swab is statistically significance, p=0.026 similarly for the saliva and HCW swab, p<0.001. The self-swab correlates better with the HCW swab.

Reviewer #2:

It is a great topic to compare the performance of self collected and HCW collected samples. But, the authors did not mention what clinical samples were used as gold standard for the diagnosis of COVID-19 in the 401 subjects. If HCW OP and MT swabs by HCW were used as gold standard, like procedure 2, listed under "test procedures", why the authors need to have a 2 stage design and include those previously tested positive for COVID-19 as subject and the second group of healthy volunteers as control? Why not include all the people when they were first tested for COVID-19 and test them with the three samples types at the very beginning?

The diagnosis of COVID-19 in this population was made based on a positive RT-PCR on a nasopharyngeal swab. During the study period in Singapore, all patients with COVID-19 are kept isolated for at least 21 days, before returning to the community. At that point in time, where the prevalence was not high in the nation, to achieve the desired sample size of 400, we would need to sample a much larger number. Hence we elected to recruit the patients that are admitted to the community care facility and in one of the public hospital. Recognizing that with this population, we would not be able to assess the specificity of the self-swab and saliva testing, hence the second group of healthy volunteers. This approach has allowed us to be assess both the sensitivity and specificity adequately, without too high a sampling burden.

The authors used the term 'detection rates' and 'negative correctness', are they referring to sensitivity and specificity which are more professional terms?

Duly noted on the comment. Have changed ‘detection rates’ and ‘negative correctness’ to ‘sensitivity’ and ‘specificity’ respectively.

Reviewer #3: Tan SY and colleagues performed a cross-sectional study to investigate healthcare worker vs. self-collected OPMT swabs and saliva samples for the detection of SARS-CoV-2 among persons with a confirmed diagnosis of COVID-19 and healthy volunteers.

I have some concerns as follow:

Major comments:

1. Characteristics, e.g. age, the onset of symptoms, severity of the disease, of the study populations should be included to provide readers to understand the clinical setting of the study better. The different settings may associate with different sensitivity of each specimen. One study demonstrated the lower value of saliva for the diagnosis of COVID-19 in children (Chong CY, et al. Clin Infect Dis 2020; article in press). Many studies showed more testing agreement of saliva and nasopharyngeal swab at the earlier onset of the disease (Jamal AJ, et al. Clin Infect Dis 2020; article in press, Iwasaki S, et al. J Infect 2020; article in press).

We have added a table on the demographics of the study subjects. However, the data that we have is not complete, as some data such as date of birth could not be given to the study due to regulations from the study site. The subjects generally had mild disease, with those who are symptomatic had only upper respiratory tract symptoms, without any need for supplemental oxygen. Most subjects were within 7 days of illness onset or first positive swab.

2. The gene that was RT-PCR test should be described. Was a housekeeping gene included in the RT-PCR reaction? The presence of the housekeeping gene in the RT-PCR test could help to determine the adequacy of a specimen collection.

The gene that was tested was the ‘N’ gene and the ‘ORF1ab’ gene. The Ct value from the ‘N’ gene was used in the statistical analysis involving Ct values. There is an internal control that is being run is each test.

3. In the first paragraph of the discussion, the author stated: “Our findings corroborate with existing epidemiologic data which indicates that while viral RNA detection may persist in some patients, such persistent RNA detection likely represents non-viable virus and hence, such patients are noninfectious.” The author should demonstrate the results in their study that suggest the conclusion of this statement.

Viral cultures were not done for this study. This statement was made based on the position paper from the Academy of Medicine, Singapore as well as data from a local study (References number 9 and 10) that above the Ct values of 30, SARS CoV 2 could not be cultured. Nonetheless, as out study did not involve viral cultures, we have removed this statement from the manuscript.

Minor comments:

1. The laboratory processing method that resulted in a lower yield of the saliva should be discussed. In this work, saliva samples were collected using the SAFER-Sample, which a saliva solubilizing solution was added to the samples. A preprint by Griesemer SB and coworkers demonstrated lower sensitivity to detect the virus when saliva stabilizing solution was added. Some studies showed a higher sensitivity of saliva when compare to nasopharyngeal swab. In these studies, the authors did not put either saliva stabilizing solution or viral transport media in the saliva specimen (Wylle AL, et al. N Engl J Med 2020; article in press; Rao M, et al. Clin Infect Dis 2020; article in press).

The evidence that have emerged since our study was conducted have indicated that neat samples have given a better yield, as indicated by your good self. The initial concern was that of biosafety, and SAFER Sample was one of the first saliva collection kits to be evaluated locally when the study was being planned. We have added the discussion point that the stabilising solution might have had a negative impact on the yield of saliva testing.

2. Please update the references. The preprints were accepted for publication.

References number 3 and 8 updated.

3. The writing can be improved by reorganizing the content to increase the continuity of the idea and content in the manuscript.

e

Revised as suggested

Reviewer #4: The manuscript by Tan et al , compared the accuracy of healthcare worker versus self collected OPMT swabs and saliva samples for SARS-CoV-2. This is a comprehensive and important study during this COVID-19 pandemic.

I have the following minor comments:

- the authors may consider adding a flowchart for the workflow and sample size of the study

The workflow of the study process was added to the manuscript. The sample size was not included in the flowchart as there was no dropout from the study per se, but there was 1 subject that could not provide a saliva sample, hence the additional recruitment of 1 subject, giving a total number of 401 COVID-19 positive subjects.

- the authors should include supplementary file for all single Ct values of different samples matching each patients

Added this into the supplementary material as suggested.

Reviewer #5:

In the manuscript entitled “The Accuracy of Healthcare Worker versus Self Collected (2-in-1) Oropharyngeal and Bilateral Mid-Turbinate (OPMT) Swabs and Saliva Samples for SARS-CoV-2”, Seow Yen Tan et al compares the detection rate of SARS-C0V-2 by RT-PCR in self- collected samples and samples collected by HCW. In addition, they used saliva, OP swabs and both methods combined, and compared the sensitivity of each approach in 400 samples from patients diagnosed with COVID-19 and 100 negative subjects. They concluded that saliva and sel-collected samples are inferior to OP and HCW-collected ones, respectively. However, combining both self-collected samples provided a higher detection rate.

Some points to discuss:

1. How long it took to test saliva samples after collecting them? Although swab samples were preserved in transportation medium, saliva could be affected by time until testing;

All samples (swabs and saliva) were processed within 24 hours of collection. The samples were stored at 2-8°C. Based on the recommendations provided by the manufacturers of SAFER-sample, the samples are stable for 72 hours.

2. Collecting saliva samples as the last procedure could affect results? Self-collected swab, followed by HCW collected swabs could interfere in the quantity and quality of saliva samples;

We determined the sequence of testing as such, thinking that doing the swabs first, will affect the saliva yield to a lesser degree, when compared to doing saliva collection first then the swabs. This is due to the theoretical possibility that by expectorating deep throat saliva, it would potentially reduce the presence of viral material.

3. Line 167 “self-saliva” means “self-swab”?? Please, correct it;

Typographical error corrected

4. In several parts of text authors refer to testing in “early phase” would provide better results: how early??In addition, there is no description on details of methodology, like time to test, preservation of samples, and characteristics of patients with positive and negative results. What about the time since diagnosis or duration of symptoms? Or severity of disease? These are important information to understand how samples were selected, and how such information could help explaining the results obtained;

The above information were added in the revised manuscript.

5. Authors state at conclusion that self-collection would be preferable for use in low prevalence population? Why??

We have modified the phrasing of the statement. The thought was that in the situations where asymptomatic infection is occurring, and mass surveillance is being undertaken, an enormous pool of trained swabbers is needed to carry this out. Hence self-collection methods would be useful in situations when number of trained swabbers need to be scaled up rapidly, without a corresponding jump in infectious exposure to the staff.

6. How author explain their results are so distinct of other reports, regarding sensitivity of saliva testing? It should be addressed in discussion. In limitations of study they cite the use of a very specific population (male migrants). How could this fact impact the results?

This could be due to the fact that the sampling process is unique that the study design was that the subjects were already known to be positive for COVID-19. Hence the way the analysis is conducted appears unique. Most other studies would be recruiting from a testing centres where the status of the subjects are unknown and the swab done by a healthcare worker is deemed to be the gold standard. Have added this into the discussion section.

The study population is certainly not generalizable to the local population. Nonetheless, the self-collection methods is meant to be as simple as possible so that the vast majority in the population is able to follow the instructions and provide a good sample.

Minor comments: there are several typos and gramatical mistakes. A deeper English revision is warranted.

Revision done as advised.

Attachment

Submitted filename: Response to Reviewers.docx

Click here for additional data file. (23.6KB, docx)

Decision Letter 1

Dong-Yan Jin

7 Dec 2020

PONE-D-20-28601R1

The Accuracy of Healthcare Worker versus Self Collected (2-in-1) Oropharyngeal and Bilateral Mid-Turbinate (OPMT) Swabs and Saliva Samples for SARS-CoV-2

PLOS ONE

Dear Dr. Tan,

Thank you for submitting your revised manuscript to PLOS ONE.

We have now received comments from the original reviewers. Two of them carefully picked up some typos and errors in your paper that must be corrected in the final version. 

After careful consideration, we feel that your paper has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Jan 21 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

  • A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

  • A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

  • An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

We look forward to receiving your revised manuscript.

Kind regards,

Dong-Yan Jin

Academic Editor

PLOS ONE

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #3: All comments have been addressed

Reviewer #4: All comments have been addressed

Reviewer #5: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #3: Partly

Reviewer #4: Yes

Reviewer #5: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #3: Yes

Reviewer #4: Yes

Reviewer #5: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #3: Yes

Reviewer #4: Yes

Reviewer #5: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #3: (No Response)

Reviewer #4: Yes

Reviewer #5: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Just a few minor points:

1. Correct grammar for this sentence: “Another study on 236 ambulatory, literate, mostly adult subjects the performance of self-collected nasal and throat swabs was at least equivalent to that of health worker collected swabs…”

2. Correct grammar for this sentence: “Self-collection of samples would reduce very significantly on the reliance of trained personnel to collect samples and ramp up testing capacity.”

3. Clarify in the methods that subjects were supervised during self-sample collection. This is hinted at in the discussion.

4. Line 155: inappropriately italicized ‘R’ in reverse-transcription.

5. Line 163: SARS-CoV-2, not SARS-Cov-2.

6. Line 265: Please elaborate on this statement. I can understand why the study findings would not be applicable to pediatric and very elderly populations, but the study findings should be broadly applicable to other sections of Singapore’s population?

Reviewer #3: My comments have been addressed.

However, some typos are still noted, e.g., Lines 246 and 247: the legends of figures 1 and 2 describe the same thing. The manuscript might be benifit from language editing.

In the first paragraph of the discussion, the authors stated the study on patients who are already known to be COVID-19 as the strength of the study. I think this is more likely to be the limitation, as the mean duration of the first positive swab to the study day and the mean duration between illness onset to study day were quite long. It is well known that Ct value correlated with days from symptom onset. Sensitivity of the virus detection is decreasing in saliva collected from later time of illness onset.

Ref: Jamal AJ, et al. Clin Infect Dis. 2020;ciaa848. Williams E, et al. J Clin Micriobiol. 2020;58(8):e00776-20.

Reviewer #4: The authors have addressed all my concerns, i have no further questions. The manuscript should be ready to published.

Reviewer #5: (No Response)

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Siddharth Sridhar

Reviewer #3: No

Reviewer #4: No

Reviewer #5: Yes: Carlos Brites

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PLoS One. 2020 Dec 16;15(12):e0244417. doi: 10.1371/journal.pone.0244417.r004

Author response to Decision Letter 1

9 Dec 2020

Dear Reviewers, thank you for the kind comments and feedback. Kindly see the responses below to your comments.

Reviewer #1: Just a few minor points:

1. Correct grammar for this sentence: “Another study on 236 ambulatory, literate, mostly adult subjects the performance of self-collected nasal and throat swabs was at least equivalent to that of health worker collected swabs…”

This sentence is corrected to: “In another study on 236 ambulatory subjects, the performance of self-collected nasal and throat swabs is at least equivalent to that of health worker collected swabs for the detection of SARS-CoV-2 and other respiratory viruses.”

2. Correct grammar for this sentence: “Self-collection of samples would reduce very significantly on the reliance of trained personnel to collect samples and ramp up testing capacity.”

This sentence is corrected to “If the self-collection of samples is proven to be a reliable alternative to a HCW swab, it would reduce the reliance of trained personnel to collect samples and enable a rapid increase in testing capacity. It would also reduce greatly the biosafety risk that is posed to HCWs and help with PPE conservation efforts.”

3. Clarify in the methods that subjects were supervised during self-sample collection. This is hinted at in the discussion.

Added in Line 145: “Study team members were present on site to observe and supervise the self-collection process”

4. Line 155: inappropriately italicized ‘R’ in reverse-transcription.

Thanks for pointing this out, this has been changed.

5. Line 163: SARS-CoV-2, not SARS-Cov-2.

Thanks for pointing this out, this has been changed.

6. Line 265: Please elaborate on this statement. I can understand why the study findings would not be applicable to pediatric and very elderly populations, but the study findings should be broadly applicable to other sections of Singapore’s population?

We made this statement as the demographic profile of the COVID-19 group (all males, generally young and healthy) is not representative of the demographics of the Singapore.

I have edited it to read: “Hence the results from this study might not be applicable to the general population, as the paediatric and very elderly were not included in the study. The migrant worker population in this study, which consist of generally young and healthy males, is also not representative of the demographics of Singapore.”

Reviewer #3: My comments have been addressed.

However, some typos are still noted, e.g., Lines 246 and 247: the legends of figures 1 and 2 describe the same thing. The manuscript might be benifit from language editing.

My apologies for the error, thank you for pointing this out. The legend is corrected, to reflect:

Fig 1: Correlation of Ct values of HCW swab and self-swab

Fig 2: Correlation of Ct values of HCW swab and saliva

The other typographical errors have been corrected to the best of our ability.

In the first paragraph of the discussion, the authors stated the study on patients who are already known to be COVID-19 as the strength of the study. I think this is more likely to be the limitation, as the mean duration of the first positive swab to the study day and the mean duration between illness onset to study day were quite long. It is well known that Ct value correlated with days from symptom onset. Sensitivity of the virus detection is decreasing in saliva collected from later time of illness onset.

Ref: Jamal AJ, et al. Clin Infect Dis. 2020;ciaa848. Williams E, et al. J Clin Micriobiol. 2020;58(8):e00776-20.

Agree with your point that the mean duration of the first positive swab to the study day and the mean duration between illness onset to study day were quite long hence the study design itself might have led to the results that we are seeing, where the yield of the saliva is not as good as the HCW swab. Our thought was that there was a large number of subjects who were known to be COVID-19 positive in our study, hence the perception of this being a strength.

I have restructured the discussion to improve the flow of the manuscript. The section pertaining to your comments above would read:

“The strength of our study is the large number of subjects confirmed to have COVID-19. Besides that, the study also included a high proportion of asymptomatic individuals who were picked up because of Singapore’s proactive mass screening policy. The combination of self-swab and saliva sampling performed well in these asymptomatic subjects, implying that the strategy of combined self-testing, has the ability diagnose COVID-19 in asymptomatic individuals with a sensitivity equivalent to that of a swab by a HCW. The study results from the healthy volunteers indicate a low false positive rate with self-collection methods.

These findings, indicate that self-collection methods may be a useful tool for COVID-19 surveillance in the asymptomatic individuals, and in situations where testing capacity needs to be scaled up rapidly, without a need for large increase of manpower, and without increased infectious exposure to the swabbing staff. Testing strategies can be tailored based on the target population and the intended use of the various tests on its own or in combination.

The way the study findings were presented are unlike most studies involving saliva testing for COVID-19. This is probably due to the fact that our study is carried out on subjects who are already known to have COVID-19, unlike most studies which are done in testing centres where the potential subjects’ results are still unknown. This also meant that the sampling was done later in the subjects’ trajectory of illness, as they were first tested positive for COVID-19, then enrolled into the study. The later sampling possibly had a negative impact on the sensitivity of the saliva [11].”

Reviewer #4: The authors have addressed all my concerns, i have no further questions. The manuscript should be ready to published.

Reviewer #5: (No Response)

Attachment

Submitted filename: Response to Reviewers 8 Dec.docx

Click here for additional data file. (15.7KB, docx)

Decision Letter 2

Dong-Yan Jin

10 Dec 2020

The Accuracy of Healthcare Worker versus Self Collected (2-in-1) Oropharyngeal and Bilateral Mid-Turbinate (OPMT) Swabs and Saliva Samples for SARS-CoV-2

PONE-D-20-28601R2

Dear Dr. Tan,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Dong-Yan Jin

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Acceptance letter

Dong-Yan Jin

14 Dec 2020

PONE-D-20-28601R2

The Accuracy of Healthcare Worker versus Self Collected (2-in-1) Oropharyngeal and Bilateral Mid-Turbinate (OPMT) Swabs and Saliva Samples for SARS-CoV-2

Dear Dr. Tan:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Professor Dong-Yan Jin

Academic Editor

PLOS ONE

Associated Data

    This section collects any data citations, data availability statements, or supplementary materials included in this article.

    Supplementary Materials

    S1 File. Provisional protocol for saliva sample collected in Lucence SAFER kit.

    (PDF)

    Click here for additional data file. (252.6KB, pdf)
    S2 File. Study protocol.

    (PDF)

    Click here for additional data file. (755KB, pdf)
    S3 File. Table with Ct values of N gene.

    (XLSX)

    Click here for additional data file. (21.8KB, xlsx)
    Attachment

    Submitted filename: Response to Reviewers.docx

    Click here for additional data file. (23.6KB, docx)
    Attachment

    Submitted filename: Response to Reviewers 8 Dec.docx

    Click here for additional data file. (15.7KB, docx)

    Data Availability Statement

    All relevant data are within the manuscript and its Supporting Information files

    Articles from PLoS ONE are provided here courtesy of PLOS

    RESOURCES