The sensitivity of respiratory tract specimens for the detection of SARS-CoV-2: A protocol for a living systematic review and meta-analysis

Abstract

Background

Highly sensitive, non-invasive, and easily accessible diagnostics for Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) are essential for the control of the Coronavirus Disease 2019 (COVID-19) pandemic. There is a clear need to establish a gold standard diagnostic for SARS-CoV-2 infection in humans using respiratory tract specimens.

Methods

Searches will be conducted in the bibliographic databases Medline, Embase, bioRxiv, medRxiv, F1000, ChemRxiv, PeerJ Preprints, Preprints.org, Beilstein Archive, and Research Square. Relevant government documents and grey literature will be sought on the FDA’s Emergency Use Authorizations website, the ECDC’s website, and the website of the Foundation for Innovative New Diagnostics. Finally, papers categorized as diagnosis papers by the EPPI Centre’s COVID-19 living systematic map will be added to our screening process; those papers are tagged with the diagnosis topic based on human review, rather than database searches, and thus this set of papers might include ones that have not been captured by our search strategy.

Introduction

Highly sensitive, non-invasive, and easily accessible diagnostics for Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) are essential for the control of the Coronavirus Disease 2019 (COVID-19) pandemic. Upon discovering that the pneumonia outbreak in the City of Wuhan in Hubei Province, China was caused by a novel betacoronavirus, medical professionals had to rely on diagnostic methods already in place for the detection for an emerging infectious disease with no established gold standard^1–3. While various polymerase chain reaction (PCR) methods are currently being used to test for the presence of SARS-CoV-2 viral RNA^4,5, there is still no universally agreed upon gold standard for which diagnostic specimen should be collected, nor how it should be collected.

Nasopharyngeal (NP) and oropharyngeal (OP) swabs are considered to be the gold standard for specimen collection for the majority of upper respiratory tract infections^6,7. The United States Center for Disease Control and Prevention had originally recommended NP swabs be used to collect a specimen for SARS-CoV-2 testing; however that recommendation has now been updated to include OP swabs, nasal mid-turbinate (NMT) swabs, anterior nasal (AN) swabs, NP wash/aspirate, and nasal wash/aspirate⁸. Incidentally, NP and nasal swabs have demonstrated insufficient sensitivity for detecting SARS-CoV-2 in various studies, particularly amongst asymptomatic patients and those with low viral loads^9–13. Saliva is perhaps the only clinical upper respiratory sample type still not formally recommended by the US CDC, despite showing promise for early viral RNA detection while circumventing the supply chain bottlenecks all swabs are subject to and having received Emergency Use Authorization (EU) by the US Food and Drug Administration (FDA) in at least once testing facility at Rutgers University^8,12,14. As such, clinicians are left with too many choices and not enough guidance on which specimens should be collected and how collection should be carried out.

There is a clear need to establish a gold standard diagnostic for SARS-CoV-2 infection in humans using respiratory tract specimens. We are aware of other, currently on-going and completed reviews of a similar nature, but none are comparing all of the specimen collection methods we are analyzing (NP, NMT, OP, AN swabs, NP and nasal washes, sputum, and saliva) nor do they implement the extensive search we show in this project^15–19. We are implementing a comprehensive and rigorous search and analysis strategy, with the goal of providing clear guidance to clinicians on which specimen type and collection method is preferred for accurate detection of SARS-CoV-2. The initial results will be published in a manuscript, following the PRISMA reporting guidelines²⁰. Monthly updates will occur for the duration of 2020 and be made available on the web. After the final web update, the published manuscript will be updated to reflect the data collected and analyzed since publication.

Primary Questions

1. Which respiratory tract sample collection method is the most sensitive and specific for the detection of SARS-CoV-2 in humans?

2. How do the sensitivity and specificity of detection for each specimen type compare to one another?

Domain of study

Studies that report the sensitivity of SARS-CoV-2 upper respiratory specimen diagnostics and specimen collection methods.

Searches

The search strategy will be designed by a medical librarian (KN) and peer reviewed by an independent medical librarian. The search has two elements: search terms related to sampling methods and search terms related to COVID-19. These two concepts will be operationalized with controlled vocabulary and keywords, drawing on existing search strategies, including the Ovid Expert Search for coronavirus. Potential expansions will be investigated: adding OR statements for diagnostic test performance and adding OR statements for screening. The searches will be updated shortly before the manuscript is submitted and again once per month for the duration of 2020.

Searches will be conducted in the bibliographic databases Medline and Embase (both on the Ovid platform). For the final update, a search may be conducted in PubMed, to take advantage of PubMed’s earlier receipt of article metadata from publishers.

Relevant preprints will be sought. Those with NIH funding may be retrieved through our MEDLINE search due to PubMed’s recent inclusion of NIH-funded preprints, but more will be sought through EuropePMC, which indexes preprints from bioRxiv, medRxiv, F1000, ChemRxiv, PeerJ Preprints, Preprints.org, Beilstein Archive, and Research Square.

Relevant government documents and grey literature will be sought on the FDA’s Emergency Use Authorizations website, the ECDC’s website, and the website of the Foundation for Innovative New Diagnostics.

In addition to database searching, the references in included papers and more recent publications citing included papers will be reviewed for additional relevant documents (backwards and forward citation chaining). Similarly, the references in similar reviews will be reviewed.

Finally, the papers categorized as diagnosis papers by the EPPI Centre’s COVID-19 living systematic map will be added to our screening process; those papers are tagged with the diagnosis topic based on human review, rather than database searches, and thus this set of papers might include ones that have not been captured by our search strategy²¹.

Records that meet the following criteria will be move into the full-text screening:

1. Published in either English, Spanish, French, Italian, Portuguese, or Chinese.

2. Published in a peer-reviewed journal or available on the web from a preprint server, governmental sources, or non-profit organization.

3. Published in 2020.

4. Focused on human subjects who are, or are suspected of being, infected with SARS-CoV-2.

5. Provide a direct indication or otherwise imply PCR-based diagnostic testing was performed on the patients

Records that meet the following criteria will not be included in the full-text screening:

1. Published in a language other than English, Spanish, French, Italian, Portuguese, or Chinese.

2. A commentary/opinion piece.

3. Focused on non-human patients.

4. Explicitly states they only use non-PCR based diagnostics.

5. Only uses non-respiratory samples for diagnostics.

In order to be included in this systematic review, all records must meet the following criteria:

1. Published in either English, Spanish, French, Italian, Portuguese, or Chinese.

2. Published in a peer-reviewed journal or available on the web from a preprint server, governmental sources, or non-profit organization.

3. Published in 2020.

4. Focused on human patients who are, or are suspected of being, infected with SARS-CoV-2.

5. Research methodologies pertaining to specimens collected, the method of collection, tools used for collection, laboratory tests run on specimen(s), analysis of test results, and reporting are provided and clearly described in order to undergo quality assessment.

6. Include a comparison of PCR-based diagnostics performed on at least two different respiratory specimens.

Records will be excluded from the review if they meet the following criteria:

1. Studies that do not include paired comparisons between two PCR-based diagnostics on respiratory tract specimens.

2. Records that do not provide detail on the specimens tested and/or how they were collected.

Draft search strategy (showing potential expansions)
Ovid MEDLINE(R) ALL <1946 to June 25, 2020>
1	exp Coronavirus/	18064
2	exp Coronavirus Infections/	17627
3	(coronavirus* or corona virus* or OC43 or NL63 or 229E or HKU1 or HCoV* or ncov* or covid* or sars-cov* or sarscov* or Sars-coronavirus* or Severe Acute Respiratory Syndrome Coronavirus*).mp.	42030
4	(or/1–3) and ((20191* or 202*).dp. or 20190101:20301231.(ep).) [this set is the sensitive/broad part of the search]	29010
5	4 not (SARS or SARS-CoV or MERS or MERS-CoV or Middle East respiratory syndrome or camel* or dromedar* or equine or coronary or coronal or covidence* or covidien or influenza virus or HIV or bovine or calves or TGEV or feline or porcine or BCoV or PED or PEDV or PDCoV or FIPV or FCoV or SADS-CoV or canine or CCov or zoonotic or avian influenza or H1N1 or H5N1 or H5N6 or IBV or murine corona*).mp. [line 5 removes noise in the search results]	19068
6	((pneumonia or covid* or coronavirus* or corona virus* or ncov* or 2019-ncov or sars*).mp. or exp pneumonia/) and Wuhan.mp.	1860
7	(2019-ncov or ncov19 or ncov-19 or 2019-novel CoV or sars-cov2 or sars-cov-2 or sarscov2 or sarscov-2 or Sars-coronavirus2 or Sars-coronavirus-2 or SARS-like coronavirus* or coronavirus-19 or covid19 or covid-19 or covid 2019 or ((novel or new or nouveau) adj2 (CoV on nCoV or covid or coronavirus* or corona virus or Pandemi*2)) or ((covid or covid19 or covid-19) and pandemic*2) or (coronavirus* and pneumonia)).mp.	29081
8	COVID-19.rx,px,ox. or severe acute respiratory syndrome coronavirus 2.os.	8075
9	(“32240632” or “32236488” or “32268021” or “32267941” or “32169616” or “32267649” or “32267499” or “32267344” or “32248853” or “32246156” or “32243118” or “32240583” or “32237674” or “32234725” or “32173381” or “32227595” or “32185863” or “32221979” or “32213260” or “32205350” or “32202721” or “32197097” or “32196032” or “32188729” or “32176889” or “32088947” or “32277065” or “32273472” or “32273444” or “32145185” or “31917786” or “32267384” or “32265186” or “32253187” or “32265567” or “32231286” or “32105468” or “32179788” or “32152361” or “32152148” or “32140676” or “32053580” or “32029604” or “32127714” or “32047315” or “32020111” or “32267950” or “32249952” or “32172715”).ui. [Articles not captured by this search when created in April 2020, pending further indexing by NLM]	50
10	or/6–9 [Lines 6 to 9 are specific to Covid-19]	29152
11	5 or 10	30228
12	11 and 20191201:20301231.(dt).	28113
13	(covid* or ncov* or 2019-novel CoV or SARS-CoV2 or SARS-CoV-2 or SARSCoV2 or SARSCov-2 or “severe acute respiratory syndrome coronavirus 2”).mp.	27537
14	(coronavirus* or corona virus*).mp. and 2020*.dp.	16570
15	((novel or new or “2019” or “19” or pandemic or crisis or outbreak or Wuhan or China) adj3 (coronavirus* or corona virus*)).mp.	9668
16	Covid-19.rx.	7963
17	coronavirus infections/ and 2020*.dp.	8100
18	Pneumonia, Viral/ and 2020*.dp.	7981
19	or/13–18	29822
20	12 or 19	29897
21	(specimen* or swab* or sample or samples or sampling or wash or aspirate or aspirates).mp.	2271408
22	(nasal or nasopharyngeal or tongue or saliva or transoral or transnasal or oropharyngeal or pharyngeal or nasopharynx or oropharynx or nares or rhinopharyngeal or rhinopharynx or turbinate).mp.	326472
23	(NP or NPS or OP or OPS or MTS or NMT).mp.	100147
24	(sputum or oral or buccal or mouth or nose or drool or gingival or gargle).mp.	915864
25	coronavirus infections/di or severe acute respiratory syndrome/di or COVID-19 diagnostic testing.px.	2766
26	saliva/ or sputum/ or exp nose/ or exp pharynx/	194669
27	25 and 26	138
28	specimen handling/	28054
29	21 or 22 or 23 or 24 or 27 or 28	3357925
30	20 and 29 and ((20191* or 202*).dp. or 20190101:20301231.(ep).) [start screening this set]	2373
31	[potential expansions 1 and 2: sensitivity terms, with or without PCR terms]	0
32	(sensitiv* or specific* or perform* or accura* or precis* or compar* or predictive value or ROC curve or index test* or LoD or limit* of detection or utility).mp.	11634892
33	exp sensitivity/ and specificity/	346232
34	exp Polymerase Chain Reaction/st	3775
35	specimen handling/st	2055
36	reference standards/	42192
37	exp reproducibility of results/	399804
38	or/32–37	11725122
39	exp polymerase Chain Reaction/ or molecular diagnostic techniques/	454902
40	(PCR or polymerase chain reaction or molecular).mp.	3026943
41	39 or 40	3029592
42	38 and 41 [sensitivity terms AND PCR terms]	1657426
43	20 and (29 or 42) and ((20191* or 202*).dp. or 20190101:20301231.(ep).) [COVID-19 + sample collection or sensitivity, but only if sensitivity comes along with PCR]	2971
44	20 and (29 or 41) and ((20191* or 202*).dp. or 20190101:20301231.(ep).) [COVID-19 + sample collection or sensitivity]	3652
45	[potential expansion 3: diagnosis terms]	0
46	(diagnosis or diagnostic or screening or testing or screen* or test* or surveillance).mp.	8250322
47	coronavirus infections/di or severe acute respiratory syndrome/di or COVID-19 diagnostic testing.px.	2766
48	20 and (29 or 41 or 46 or 47) and ((20191* or 202*).dp. or 20190101:20301231.(ep).)	8104

Types of study to be included

This review will consider records that describe methods of respiratory tract specimen collection for the purpose of SARS-CoV-2 viral RNA identification by PCR. Peer-reviewed and preprint case studies, case-control studies, cohort studies, observational studies, assay validation studies, and US and European government documents in English, Spanish, Italian, French, Portuguese, and Chinese will be included.

Participants/population

Individuals who have been tested for SARS-CoV-2 infection.

Comparator(s)/controls

Respiratory tract specimens collected for SARS-CoV-2 viral RNA detection.

Main outcome(s)

Sensitivity and reproducibility of specimen collection methods in the general population with SARS-CoV-2 infection.

• Measure of effect

• Sensitivity, specificity

Data extraction (selection and coding)

We will be following the Joanna Briggs Institute (JBI) Reviewer’s Manual²². Primary data selection and extraction will be performed in Covidence. In both the title-abstract and full-text screening stages, each record will be screened by two co-authors independently (except for records in Spanish, Italian, French, Portuguese, which will be screened by AC-M and records in Chinese by XS).

After search results are uploaded to Covidence, co-authors AJM, MIN, CCK, and ALW will screen the title and abstract of all retrieved English articles and documents to identify those that might match the aforementioned inclusion criteria. Records that clearly do not meet the criteria will be eliminated at this stage. Those that remain will proceed to full-text review. Any conflicts in the title/abstract screening will be resolved by consensus or the involvement of a fifth co-author if necessary.

Records that appear to potentially meet the inclusion criteria at the title/abstract screening phase will then be subject to full-text review by co-authors AJM, MIN, and CCK to determine if they meet the inclusion criteria for this review. At the full-text screen stage, records that do not meet the inclusion criteria will be eliminated and the reason for exclusion will be logged. Conflicts on records that should be ex/included will be resolved with consensus between AJM, MIN, and CCK. We will perform a dual extraction of details on the study type, specimen type, method of specimen collection, true positive N, false positive N, false negative N, true negative N, total number of patients included, patient age and sex, primer/probe sets used for the PCR, and positive/negative cut off values. We will also extract the first author’s last name, the year of publication, country of publication, and title of the record. In cases where a preprint and a published article describe the same research, data extraction and risk of bias assessment will be performed on the published article. We will be following the reporting guidelines outlined by PRISMA(source). After the initial manuscript is published, the subsequent monthly searches and screenings will only be done on English language records.

Risk of bias (quality) assessment

Each record that has been determined to meet the inclusion criteria upon full-text screening will be critically evaluated for risk of bias and reporting quality by at least two co-authors (AJM, MIN, CCK), following the risk of bias (critical appraisal) tools provided by JBI and the Standard for Reporting of Diagnostic Accuracy (STARD) checklist^22,23.

Strategy for data synthesis

Hierarchical modeling is the gold standard of meta-analysis of diagnostic test accuracy studies²⁴. x² tests will be performed to assess the heterogeneity of sensitivities and specificities of tests in the sample. The sensitivity and specificity of all studies will then be meta-analyzed using the hierarchical summary receiver operating characteristic (HSROC) model. Analysis will be conducted with the mada package in R Studio^25,26. Pooled sensitivity and specificity estimates, as well as 95% confidence intervals, will be calculated.

Analysis of subgroups or subsets

If heterogeneity is found or suspected, a subgroup analysis will be performed as needed. Potential sources of heterogeneity include anatomical subsites such as mid-turbinate and anterior nasal areas, and demographic factors. If statistical pooling is not possible for these subgroups, the results will be presented in a narrative format.