Sensitivity of SARS-CoV-2 RNA polymerase chain reaction using a clinical and radiological reference standard

Abstract

Objectives

Diagnostic tests for SARS-CoV-2 are important for epidemiology, clinical management, and infection control. Limitations of oro-nasopharyngeal real-time PCR sensitivity have been described based on comparisons of single tests with repeated sampling. We assessed SARS-CoV-2 PCR clinical sensitivity using a clinical and radiological reference standard.

Methods

Between March-May 2020, 2060 patients underwent thoracic imaging and SARS-CoV-2 PCR testing. Imaging was independently double- or triple-reported (if discordance) by blinded radiologists according to radiological criteria for COVID-19. We excluded asymptomatic patients and those with alternative diagnoses that could explain imaging findings. Associations with PCR-positivity were assessed with binomial logistic regression.

Results

901 patients had possible/probable imaging features and clinical symptoms of COVID-19 and 429 patients met the clinical and radiological reference case definition. SARS-CoV-2 PCR sensitivity was 68% (95% confidence interval 64–73), was highest 7-8 days after symptom onset (78% (68–88)) and was lower among current smokers (adjusted odds ratio 0.23 (0.12–0.42) p < 0.001).

Conclusions

In patients with clinical and imaging features of COVID-19, PCR test sensitivity was 68%, and was lower among smokers; a finding that could explain observations of lower disease incidence and that warrants further validation. PCR tests should be interpreted considering imaging, symptom duration and smoking status.

Introduction

Diagnostic tests for severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2) are important for clinical management and infection control. Detection of SARS-CoV-2 viral ribonucleic acid (RNA), using real time polymerase chain reaction (PCR) from naso- and oropharyngeal samples, is the principal method for diagnosing current infection.¹ Limitations in the sensitivity of PCR assays have been described based on the comparison of a single test with serial repeated sampling.² A pooled sensitivity of 88% from a single test relative to multiple sampling was described in a recent meta-analysis of 16 studies.³ This finding has implications for clinical care and infection control procedures in hospital settings and for epidemiological estimates that rely upon PCR testing.² In view of this, in addition to repeated sampling, thoracic imaging has been proposed as a complementary diagnostic strategy for individuals presenting with compatible symptoms in a pandemic setting.^{4 , 5} Characteristic radiological features have been described, including multifocal ground glass opacification, consolidation with a peripheral, subpleural distribution and progression to confluent consolidation with an acute respiratory distress syndrome- (ARDS) like pattern.^{4 , 6} Computed tomography (CT) has a higher sensitivity relative to plain chest X-ray (CXR) for elicitation of these features.⁷ Evaluation of the diagnostic performance of PCR assays is required to inform infection prevention and disease control, clinical management and epidemiological estimates which rely on PCR-confirmed cases.

We aimed to estimate the clinical sensitivity of rtPCR testing from naso-oropharyngeal samples using a clinical and radiological case definition among hospitalised patients presenting to an acute tertiary hospital in England during the first pandemic wave of coronavirus disease of 2019 (COVID-19). Our secondary objective was to describe the patient characteristics associated with SARS-CoV-2 PCR positivity among patients meeting a clinical and radiological reference standard.

Materials and methods

Inclusion and exclusion criteria

All patients attending our hospital and who underwent testing for SARS-CoV-2 between 13^th March 2020 to the 18^th May 2020 were considered for this analysis. The start was chosen as the date that Public Health authorities in the United Kingdom recognised a generalised pandemic and criteria for PCR testing no longer required a history of travel from an epidemic area or contact with a confirmed case. Patients underwent testing for SARS-CoV-2 according to criteria defined by Public Health England (Appendix 1). In brief, any patient admitted to the hospital with respiratory symptoms (cough, hoarseness, nasal discharge or congestion, shortness of breath, sore throat, wheezing or sneezing), fever (≥37.8 °C) or radiological evidence of pneumonia or ARDS was eligible for SARS-CoV-2 testing. CXR was routinely performed for all patients reporting respiratory symptoms. The decision to perform additional thoracic imaging was at the discretion of the treating clinician. We included patients in the analysis of the diagnostic sensitivity of PCR testing if SARS-CoV-2 testing was performed within seven days of the first CXR or CT meeting study radiological criteria for COVID-19 (Appendix 2). Patients were eligible for inclusion if they reported symptoms compatible with COVID-19, including respiratory, gastrointestinal or systemic symptoms (Appendix 3)⁸. We excluded patients with a non-COVID-19 clinical diagnosis which could represent an alternative cause of radiological findings, and patients who were asymptomatic, for whom imaging findings were incidental.

We used a scoring system for COVID-19 for thoracic computed tomography (CT)^{6 ,} and modified British Society for Thoracic Imaging (BSTI) definitions for chest X-ray (CXR)⁹ as radiological criteria for COVID-19 (Appendix 2). Lung ultrasound findings were not considered in this analysis. All thoracic imaging and radiological reports were retrieved from the hospital picture archiving and communications system. The primary clinical reports assessed the likelihood of COVID-19 using BSTI criteria as standard departmental practice. Patients with thoracic imaging reported as normal or with no imaging features compatible with COVID-19 in their first report were excluded from further analysis. We randomly selected a sample of 20% from this group for repeat reporting to evaluate the potential for misclassification. All patients with intermediate/possible or high probability of COVID-19 on the first radiology reporting underwent second reporting by a study consultant radiologist. Participants whose first and second reports were discordant underwent third reporting by a third consultant radiologist sub-specialising in infectious disease or thoracic imaging to adjudicate discordance. Study radiologists were blinded to clinical details and SARS-CoV-2 PCR test results and reported imaging on the basis that patients had suspected COVID-19.

CXR inclusion criteria, based on BSTI criteria (version 2.0), were one of:

• imulti-focal, peripheral ground glass opacification or consolidation, with or without reticular opacification or linear atelectasis;
• iisingle, unilateral peripheral ground glass opacity;
• iiiwidespread bilateral airspace consolidation in a pattern consistent with acute respiratory distress syndrome (ARDS)

Exclusion criteria included single lobar or segmental consolidation.

CT inclusion criteria, based on COVID-19 Reporting and Data System (CO-RADS) 4 (high level of suspicion) or 5 (very high level of suspicion), are described in Appendix 2.⁴ We excluded patients where CT images demonstrated lobar pneumonia, cavitating infection or tree in bud changes.

We collected anonymised data on the age, sex, presenting complaint, past medical history and outcomes of patients who met radiological criteria from hospital electronic medical records. Symptom onset was defined as the patient-reported date of the first recorded symptom. The study was conducted according to Standards for Reporting Diagnostic Accuracy Studies 2015 and QUADAS-2 criteria.^{10 , 11}

Laboratory investigations

Real time PCR of nose and throat swabs was performed using a commercial CE-marked in vitro diagnostic assay (Viasure SARS-CoV-2, Biotec, Spain), which detects two conserved target sequences in the SARS-CoV-2 open reading frame (ORF)-1ab and N genes. Extraction of nucleic acid was performed using the automated QIAsymphony platform (Qiagen, Germany). An internal control and negative controls were included on each plate. Detection of PCR amplification at <38 cycle thresholds in either target with a satisfactory amplification curve using a Lightcycler 480 (Roche Diagnostics, Switzerland) was considered a positive test, subject to adequate performance of controls, in accordance with manufacturer instructions.

Statistical analysis

Agreement between radiological reports was calculated using Cohen's kappa, based on blinded independent radiologists agreeing on the patient meeting study radiological criteria (ie. grouped BSTI criteria for CXR or CO-RADS 4 or 5 for CT images). To compare PCR positive and negative patients we used Wilcoxon rank-sum test for comparison of ordinal and continuous data and Pearson Chi-squared or Fisher's exact tests for categorical data as appropriate. We calculated adjusted p values for multiple hypothesis testing of symptoms and comorbidities using a univariable logistic regression model with Benjamini and Hochberg false discovery correction as implemented in the Stata package qqvalue.¹² We calculated the sensitivity of nasopharyngeal rtPCR by comparing with the reference definition. We used multivariable logistic regression to examine factors associated with a positive PCR result among patients meeting the clinical and radiological case definition. Variables were selected for model inclusion based on a priori selection for relevance, comprising the interval between symptom onset and PCR testing and participant age, or based on evidence of possible significance among symptoms and comorbidities (p < 0.20 after false discovery correction) aiming to minimise model Akaike and Bayesian information criterion. We tested models using a restricted cubic spline variable with four knots for duration of symptoms. The effect of PCR positivity on survival was assessed using a multivariable Cox proportional hazards model, right-censoring outcomes at 28 days, using the Effron method for ties. Statistical analysis was performed in Stata v16.1 (College Station, TX, USA).

Ethical approval

We analysed anonymised, routinely collected clinical data for this study. The study was approved as a quality improvement project by the hospital audit governance committee. In accordance with guidance from the National Health Service Health Research Authority in effect during this study, a requirement for individual patient consent was waived.

Results

Between 13^th March and 18^th May 2020, 2060 patients had thoracic imaging and PCR testing for SARS-CoV-2 (Fig. 1 ). We excluded 1090 patients who had no imaging features compatible with COVID-19 on their primary radiological report, and 69 patients who had an interval between imaging and PCR testing exceeding 7 days (n = 34), were asymptomatic²⁰ or who had an alternative diagnosis to explain the observed imaging features¹⁵, principally cardiac failure⁷, aspiration pneumonia⁵, or other diagnoses as listed in Fig. 1. A study radiologist (EJ) reviewed a randomly selected sample of imaging from 20% of patients excluded based on the first radiology report, and no cases compatible with COVID-19 were observed. A total of 2123 thoracic imaging studies were reported a second time by study radiologists, comprising 1823 CXRs and 300 CT studies among the 901 remaining patients with a median of 2 images (interquartile range (IQR) 1, 3) per patient. Agreement between the first and second independent reports occurred for 75.4% of patients; the kappa statistic was 0.51, indicating moderate agreement. Among patients with discordant first and second reports, (222/901 (24.6%)), a third report from an independent study radiologist with infectious diseases or thoracic imaging subspecialty expertise adjudicated the imaging met the study radiological criteria for COVID-19 in 76/222 (34.2%). Overall, 429 patients met our radiological and clinical criteria for COVID-19 and were included in the study (Fig. 1).

Fig. 1.

Flowchart of included patients in study of SARS-CoV-2 PCR sensitivity^aClinical reasons for exclusion, representing alternative aetiologies for imaging findings reported as high probability of COVID-19 (n=15), comprised: pulmonary oedema due to cardiac failure (n=7), aspiration pneumonia (n=5), massive transfusion following haemorrhage (n=1), interstitial lung disease (n=1), anti-glomerular basement membrane lung disease (n=1).

Among 429 included patients, 293 were SARS-CoV-2 RNA PCR positive. Median age was 67 (IQR 55, 78) and 172 (40.1%) were female. Characteristics of included patients are shown in Table 1 , stratified by SARS-CoV-2 PCR result. Age and gender distribution were similar among patients regardless of PCR result (Table 1). No difference was observed in duration of symptoms at admission or at PCR testing, interval between PCR testing and imaging, the number of PCR tests conducted, or whether symptom timing indicated community or hospital onset (Table 2 ) between patients based on PCR result. PCR-negative patients were more likely to have had thoracic CT imaging (37.5% vs 23.2% for PCR positive).

Table 1.

Characteristics of demographic data and investigations of included patients, stratified by SARS-CoV-2 PCR result.

Characteristics	All included patients n=429		Positive SARS-CoV-2 PCR test n=293		Negative SARS-CoV-2 PCR test n=136		P value
Age (median), years	67	(55, 78)	69	(57, 79)	63	(51, 76)	0.005
<40	33	(7.7)	17	(5.8)	16	(11.8)	0.03
40-59	116	(27.0)	74	(25.3)	42	(30.9)
60-79	195	(45.5)	141	(48.1)	54	(39.7)
≥80	85	(19.8)	61	(20.8)	24	(17.7)
Sex, female	172	(40.2)	115	(39.3)	57	(41.9)	0.60
Duration of symptoms at admission, days	7	(2,12)	7	(2,11)	7	(2,14)	0.83
Community onset	387	(90.2)	266	(90.8)	121	(89.0)	0.22
Indeterminate/probable hospital acquired	24	(5.6)	13	(4.4)	11	(8.1)
Definite hospital acquired	18	(4.2)	14	(4.8)	4	(2.9)
Duration of symptoms at time of PCR test, daysa	7	(3,13)	7	(3,12)	7	(3,14)	0.77
<3 days	83	(19.4)	55	(18.8)	28	(20.6)	0.27
4-7 days	96	(22.4)	65	(22.2)	31	(22.8)
8-14 days	146	(34.0)	108	(36.9)	38	(27.9)
>14 days	104	(24.2)	65	(22.2)	39	(28.7)
SARS-CoV-2 PCR testing
Interval between imaging and PCR testa, hours	3.2	(0.9, 15.1)	3.3	(1.0, 18.8)	3.2	(0.9, 14.5)	0.62
Number of PCR tests performed	1	(1,2)	1	(1,2)	1	(1,2)	0.72
1 test	238	(55.5)	169	(57.7)	69	(50.7)	0.80
2 tests	101	(23.5)	54	(18.4)	47	(34.6)
3 or more tests	90	(21.0)	70	(23.9)	20	(14.7)
Radiological imaging
CXR only	310	(72.3)	225	(76.8)	85	(62.5)	0.002
CT imaging	119	(27.7)	68	(23.2)	51	(37.5)
Number of CXR images	2	(1,3)	2	(1,3)	1	(1,2)	<0.001
Number of CT imagesb	1	(1,1)	1	(1,1)	1	(1,1)	0.64
Abnormal imaging findings
Unilateral	54	(12.6)	32	(10.9)	22	(16.2)	0.13
Bilateral	375	(87.4)	261	(89.1)	114	(83.8)
CT CORADS scorec
4: high probability	38	(34.6)	16	(27.1)	22	(43.1)	0.08
5: very high probability	72	(65.5)	43	(72.9)	29	(56.9)

^aIf multiple PCR tests were obtained, we used the PCR test closest to the first imaging that was reported as “highly probable” for COVID-19 for calculating intervals. ^b Among 124 individuals who underwent CT thoracic imaging. ^cCO-RADS score are shown for 110/119 individuals with CT imaging who met CT criteria for inclusion; the remaining 9 patients met inclusion criteria based on subsequent CXR findings

Table 2.

Symptoms at presentation, comorbidities and outcomes stratified by SARS-CoV-2 PCR result among patients with a high probability of COVID-19 based on clinical and radiological criteria

Characteristics n (%)	Positive SARS-CoV-2 PCR test n=293		Negative SARS-CoV-2 PCR test n=136		P value	Benjamini-Hochberg adjusted P valuea
Symptoms
Systemic symptoms	218	(74.4)	91	(66.9)	0.11
Fever	190	(64.9)	76	(55.9)	0.08	0.22
Rigor	15	(5.1)	6	(4.4)	0.75	0.83
Myalgia	48	(16.4)	16	(11.8)	0.21	0.45
Headache	22	(7.5)	14	(10.3)	0.33	0.50
Lethargy	79	(27.0)	40	(29.4)	0.60	0.77
Respiratory symptoms	260	(88.7)	119	(87.5)	0.71
Shortness of breath	196	(66.9)	94	(69.1)	0.65	0.78
Sore throat	22	(7.5)	6	(4.4)	0.23	0.45
Cough	215	(73.4)	93	(68.4)	0.29	0.47
Chest pain	33	(11.3)	26	(19.1)	0.03	0.11
Gastrointestinal symptoms	94	(32.1)	44	(32.4)	0.96
Abdominal pain	16	(5.5)	16	(11.8)	0.02	0.11
Anorexia	38	(13.0)	4	(2.9)	0.001	0.05
Nausea	22	(7.5)	11	(8.1)	0.83	0.83
Diarrhoea	47	(16.0)	28	(20.6)	0.25	0.45
Vomiting	32	(10.9)	23	(16.9)	0.08	0.22
Anosmia	9	(3.1)	6	(4.4)	0.48	0.67
Falls	22	(7.5)	2	(1.5)	0.01	0.11
Collapse	9	(3.1)	11	(8.1)	0.02	0.11
Acute confusion	46	(15.7)	20	(14.7)	0.79	0.79
Number of symptoms reported, median (IQR)	4	(3,5)	3	(3,5)	0.63
Usual residence
Own home	205	(71.4)	114	(83.8)	0.04
Own home with social care	39	(13.6)	13	(9.6)
Residential home	9	(3.1)	2	(1.5)
Nursing home	34	(10.5)	7	(5.2)
Smoking status
Never smoked	153	(56.5)	61	(46.6)	<0.001
Ex-smoker	97	(35.8)	34	(26.0)
Current smoker	21	(7.8)	36	(27.5)
Comorbidities
Number of comorbidities, median (IQR)	2	(1,3)	2	(1,3)	0.01
Chronic heart disease	91	(31.1)	19	(14.0)	<0.001	0.004
Peripheral vascular disease	14	(4.8)	2	(1.5)	0.09	0.28
COPD	44	(15.0)	24	(17.6)	0.49	0.61
Asthma	37	(12.6)	14	(10.3)	0.49	0.61
Other respiratory disease	16	(5.5)	6	(4.4)	0.65	0.69
Diabetes mellitus type 1	3	(1.0)	0	(0)	0.56
Diabetes mellitus type 2	82	(28.0)	30	(22.1)	0.19	0.32
Hypertension	125	(42.7)	42	(30.9)	0.02	0.09
Obesity	58	(19.8)	30	(22.1)	0.59	0.68
Dementia	34	(11.6)	10	(7.4)	0.18	0.32
Chronic neurological disease	47	(16.0)	10	(7.4)	0.01	0.09
Active malignancy	25	(8.5)	18	(13.2)	0.13	0.29
Chronic liver disease	12	(4.1)	8	(5.9)	0.41	0.61
Chronic renal disease	40	(13.7)	10	(7.4)	0.06	0.19
Haemodialysis	7	(2.4)	0	(0)	0.07
Immunocompromised	19	(6.5)	18	(13.2)	0.02	0.09
Rheumatological disease	15	(5.1)	6	(4.4)	0.75	0.75
Pregnancy	2	(0.7)	0	(0)	0.33
Clinical Frailty Score	3	(2,6)	3	(2,5)	0.25
Do not resuscitate order implemented	165	(56.3)	67	(49.3)	0.17
Outcomes
Received oxygen	253	(86.4)	95	(70.0)	<0.001
Maximum oxygen concentration (%)b	40	(32, 80)	34	(21, 60)	0.002
Critical care admission	53	(18.1)	8	(5.9)	0.001
Non-invasive ventilation	21	(7.2)	11	(8.1)	0.74
Invasive ventilation	52	(17.8)	3	(2.2)	<0.001
Cumulative incidence of mortality at 28 days  (% (95% CI)c	41.6	(36.2-47.5)	27.4	(20.7-35.7)	0.009

^aAdjusted P value for multiple hypothesis testing of 18 symptoms and 15 co-morbidities respectively using univariate logistic regression with Benjamini-Hochberg false discovery corrected P values. ^bMaximum oxygen concentration received among patients treated with supplemental oxygen therapy. ^cCumulative mortality incidence from Kaplan-Meier analysis, p-value is log-rank test.

Among patients meeting our clinical and radiological criteria for COVID-19, PCR sensitivity was 68.3% (95 CI 63.9–72.7). Subgroup analysis showed that sensitivity was related to duration from onset of symptoms, increasing from 66.3% (95% CI 55.9–76.7) at <3 days, to peak at 77.9% (95% CI 67.8–88.1) at 7-8 days after symptoms onset, before falling again in the second week (Fig. 2, Fig. 3 ), though with overlapping confidence intervals. Younger patients had a lower PCR sensitivity (51.5% among <40 years vs 71.8% among those ≥80 years) and sensitivity appeared higher among patients with bilateral imaging findings, although this was not a statistically significant finding. Current smokers were observed to have lower sensitivity relative to never or former smokers (36.8% vs 73.1%, p < 0.001) (Fig. 2). Sensitivity correlated with calendar date and the frequency of COVID-19 cases meeting our study definition and PCR positive cases (Supplementary appendix 4).

Fig. 2.

Clinical sensitivity of SARS-CoV-2 PCR relative to clinical and radiological reference standard, stratified by subgroups.

Fig. 3.

: Predicted sensitivity of PCR test according to duration of symptoms^a

^aPrediction plot of PCR positivity using four-knot restricted cubic spline for duration of symptoms at time of PCR testing. Knots were introduced at 0, 5, 10 and 22 days. Shaded area represents 95% confidence intervals.

No difference in the pattern of reported symptoms was observed between PCR positive and PCR negative individuals, after false discovery rate (FDR) correction for multiple hypothesis testing (Table 2). PCR-positive patients were more likely to have come from a nursing or residential home relative to PCR-negative patients and were less likely to be a current smoker. PCR-positive patients were more likely to have chronic heart disease, chronic neurological disease and to be immunocompromised; only an association with heart disease persisted after FDR correction. No difference in clinical frailty score or rates of do not resuscitate orders were observed. PCR-positive patients had increased disease severity, with a greater proportion receiving oxygen (86.4 vs 70.0%, p<0.001), being admitted to critical care (18.1% vs 5.9%, p = 0.001) and receiving invasive ventilation (17.8% vs 2.2%, p < 0.001). The Kaplan Meier cumulative incidence of mortality at 28 days was increased among PCR-positive patients (41.6% vs 27.4, p = 0.009 (Log-rank test). On univariable Cox proportional hazards analysis, mortality at 28 days was increased among PCR-positive patients (hazard ratio 1.62 (95% 1.12–2.34, p = 0.01), but following adjustment for age, smoking status and comorbidities, no difference in mortality rate was observed among PCR-positive, relative to PCR-negative patients (Table 3 ).

Table 3.

Cox-proportional hazards model: associations between PCR positivity and 28-day survival.

Model	Analysis	Hazard ratio (95% CI)	P value
1	Univariable: PCR positive, vs PCR negative patients	1.62 (1.12 – 2.34)	0.01
2	Model 1, adjusted for age	1.34 (0.93 – 1.94)	0.12
3	Model 1, adjusted for age and smoking status	1.30 (0.89 – 1.89)	0.17
4	Model 1, adjusted for age, smoking status and number of comorbidities	1.20 (0.82 – 1.76)	0.34

By multivariable logistic regression, after adjustment for age and the duration of symptoms, an increased odds of PCR positivity was associated with an increasing number of comorbidities and those requiring oxygen therapy, whereas lower odds of PCR positivity was independently associated with current smoking (odds ratio 0.23 (95% CI 0.12–0.42), p < 0.001) (Table 4 ). The association between current smoking and PCR result persisted in a second model after adjusting for potential confounders including chronic obstructive pulmonary disease (COPD), chronic heart disease and other respiratory disease (aOR 0.25 (95% CI 0.13 – 0.50, p < 0.001) (Model 2, Table 4) and no interaction between smoking and PCR-positive rate or calendar date was observed (Supplementary appendix 4).

Table 4.

Multivariable binomial logistic regression model for a positive SARS-CoV-2 PCR result among patients meeting a clinical and radiological reference standard for COVID-19^a

	Model 1		Model 2
Characteristic	Odds ratio (95% CI)	P value	Odds ratio (95% CI)	P value
Age, per year	1.00 (0.99 – 1.02)	0.54	1.00 (0.99 – 1.02)	0.59
Duration of symptoms at time of PCR testingb		0.21		0.20
3 days	Reference		Reference
5 days	1.09 (0.89 – 1.35)		1.14 (0.92 – 1.41)
7 days	1.13 (0.88 – 1.45)		1.22 (0.93 – 1.62)
9 days	1.08 (0.46 – 2.55)		1.25 (0.86 – 1.80)
14 days	0.76 (0.01 – 56.8)		1.14 (0.63 – 2.06)
Number of comorbidities, per additional comorbidity	1.22 (1.04 – 1.43)	0.02
Current, vs never or ex-smoker	0.23 (0.12 – 0.42)	<0.001	0.25 (0.13 – 0.50)	<0.001
Received oxygen therapy	1.90 (1.10 – 3.27)	0.02	1.89 (1.09 – 3.29)	0.02
COPD			0.95 (0.48 – 1.88)	0.89
Other respiratory disease			0.77 (0.27 – 2.17)	0.62
Chronic heart disease			2.87 (1.51 – 5.44)	0.001

^aModel 1 adjusted for age, duration of symptoms, number of comorbidities, smoking and receipt of oxygen therapy (as a marker of disease severity). Model 2 adjusted for additional confounders associated with smoking comprising COPD, chronic heart disease and respiratory disease. ^b A cubic spline for duration of symptoms at time of testing with four knots was included in the model. P value presented for this variable is the Wald likelihood test comparing a model including the cubic spline variable with one without it.

Conclusions

During the first peak of the COVID-19 pandemic, among hospitalised patients meeting a combined clinical and radiological definition of COVID-19, with imaging features of COVID-19 pneumonia, an oro-nasopharyngeal SARS-CoV-2 PCR test had an overall clinical sensitivity of 68%. The study was conducted in the setting of a high pre-test probability of disease during the first wave of the pandemic when COVID-19 dominated the overall hospital population. Our findings have implications for clinical management and for infection control. Clinicians should carefully consider PCR test results in the context of the limitations of test sensitivity and in view of the local epidemiology, the clinical syndrome, duration of symptoms and radiological findings before deciding on isolation and clinical management, and should avoid diagnostic anchoring based on PCR test results alone.

We observed a relationship between PCR sensitivity in the case population and time from symptom onset, with sensitivity highest between 7-8 days following symptom onset and declining in the second week of symptoms, although confidence intervals overlapped. Our findings are in keeping with a meta-analysis of studies of serial repeated PCR testing where the estimated median false-negative rate (equivalent to 100-sensitivity) was 38% (95% CI 18-65) on the day of symptom onset, decreasing to 20% (95% CI 12–30) by day 8 and increasing again to 66% (95% CI 54–77) at day 21.¹³ By contrast an individual patient data meta-analysis of 32 studies which reported serial repeated testing, estimated that the highest PCR sensitivity was found at 0–4 days post-symptom onset (89% 95% CI 83–93), declining to 81% at 0–4 days post-hospital admission, and was 54% (73–87) at 10-14 days after onset.¹⁴ As with other studies reporting routinely collected data, the relationship we observed between test sensitivity and the timing of sample collection relative to symptom onset is subject to ascertainment bias, since testing schedules were clinically driven and not standardised; measurement of early test sensitivity was limited to the subgroup of patients presenting soon after symptom onset, whose characteristics might differ from those presenting at a later stage. The overall evidence shows that PCR sensitivity declines beyond the second week after symptom onset and this is an important consideration for test interpretation. We did not observe important differences in the timing or quantity of PCR testing between PCR positive and negative groups, suggesting that heterogeneity in PCR sensitivity in our study was not driven by differences in testing schedules.

Our use of a combined clinical and radiological reference standard may offer some advantages over studies using serial repeated testing to understand test sensitivity, since patients may remain repeatedly PCR negative from the point of presentation to hospital, and such studies are likely to underestimate the true rate of false negatives(2). We carefully excluded patients with subtle early radiological features, thus our findings represent only the subgroup of patients with the most classical findings of COVID-19. We found that PCR sensitivity was particularly low among smokers, a finding robust to adjustment for the presence of potential associated diseases that could represent confounders including COPD, other respiratory disease and heart disease. Multiple studies have highlighted an apparent protective effect of current smoking on susceptibility to COVID-19 disease including large population studies of general practitioner records from the UK.^{15 , 16} In a meta-analysis including 55 studies, the relative risk for COVID-19 disease among smokers relative to never-smokers was 0.74 (95% credible interval 0.58-0.93).15, 16, 17, 18, 19 By contrast, among hospitalised COVID-19 patients, more severe disease has been described among smokers.^{20 , 21} Our finding of reduced PCR sensitivity could represent a compelling explanation for the surprisingly lower rates of disease observed among smokers. The mechanism for such a finding is unclear, but could relate to disruption of ciliary function, upregulation of ACE2 receptors among smokers, which are the entry receptor for SARS-CoV-2, and dysregulation of cytokine responses and neutrophil trafficking, potentially reducing viral load in the upper airway.^{22 , 23} The effect size of smoking is striking, and unexpected, and it is possible that bias or confounding is contributing to our estimate. Smokers, for example, may be more likely to have lung disease that is misclassified as COVID-19. We addressed this hypothesis via adjustment for potential confounders and by seeking a time-dependent effect of smoking on PCR sensitivity, which could be expected if there was significant misclassification of non-COVID-19 X-rays as COVID-19 incidence changed though the course of the pandemic. The strong association of smoking with reduced PCR sensitivity persisted in both analyses. Nevertheless our findings should be interpreted with caution and warrant further validation in other centres but do suggest that smoking status should be taken into consideration when interpreting PCR test results. We observed a higher rate of PCR sensitivity among the sickest patients, with higher rates of oxygen therapy, critical care admission, invasive ventilation and death among PCR positive, relative to PCR negative patients. This is in keeping with previous data showing a correlation between SARS-CoV-2 viral load and increased markers of inflammatory response and disease severity.^{24 , 25}

Real time PCR is a highly sensitive diagnostic modality, capable of detecting small quantities of viral genetic material in vitro. In a recent external quality assessment of 68 laboratories in 35 countries laboratories, the test kit we employed in this study had equivalent sensitivity to a range of other commercial assays.²⁶ Putative mechanisms for the heterogeneity in clinical sensitivity that we observed include biological processes, including the influence of comorbidities and immunological factors on viral shedding, and residual confounding by variation in sampling technique.³

The lack of an established reference test for COVID-19 necessitates caution interpreting studies that compare diagnostic modalities.³ Studies assessing the diagnostic performance of thoracic imaging have often used PCR testing as a reference standard, resulting in suggestions that CT or plain radiography lack specificity, whereas limitations in the PCR standard could cause misleading interpretations about the specificity of imaging findings.²⁷ Inter-observer kappa in our study was 0.51, with agreement in 75.4% of patients. Our findings are consistent with studies of radiologists reviewing chest CT imaging.⁴ The finding of moderate agreement formed the basis for our decision to include a third tier of specialist radiologists to evaluate discordant cases. In a recent analysis of the BSTI criteria for COVID-19 reporting, among a sample of CXRs from patients with COVID-19 pneumonia and historical images from patients with symptoms consistent with COVID prior to the emergence of the virus, test specificity was 100%, and sensitivity 44%, providing evidence that these radiological criteria can identify COVID-19 disease with high specificity.²⁸

Our study had several strengths. We considered the real-world clinical sensitivity of PCR testing, carefully considered clinical exclusion criteria, and applied rigorous radiological reporting, with review of acquired thoracic imaging by up to three independent expert radiologists. The main limitation of this study, in common with all diagnostic evaluation studies for COVID-19, was the lack of an authoritative reference standard. We created a combined clinical and radiological standard in a pandemic context that represented the combination of a clinical presentation consistent with COVID-19 with radiological features that were highly suggestive of COVID-19 disease. It is thus notable that the estimated sensitivity of PCR testing in this study applies only to this highly selected population with unambiguous radiological features of COVID-19 and does not apply to the wider population with COVID-19 disease including pauci-symptomatic patients or those with more subtle radiological changes. We attempted to mitigate information bias by blinding study radiologists to the clinical presenting features and PCR results, and had two, or in the event of discordance, three, independent radiologists reviewing thoracic imaging. It is possible that our case definition resulted in misclassification, leading to impaired specificity of the radiological case definition and reporting a falsely low PCR test sensitivity. We aimed to minimise misclassification wherever possible by, independently from study radiologists and imaging findings, reviewing the clinical presentation of patients, excluding patients who by the time of discharge or death had an alternative diagnosis that might represent the cause of imaging findings or were asymptomatic. It is notable that the commonest clinical presentation showing features that radiologists originally interpreted as COVID-19 were pulmonary oedema from cardiac failure and aspiration pneumonia, followed by interstitial lung disease or pulmonary haemorrhage, which represent important differential diagnoses.²⁹ At the time of this study, CT was often used in situations where the clinical team had a high clinical suspicion of COVID-19, yet PCR testing was negative. As such, lower PCR sensitivity was observed among people who underwent CT imaging; this was a pre-selected population who in most cases had already had an absence of classical findings on chest X-ray and a negative PCR result, leading to further diagnostic pursuit. It is likely that the true sensitivity of SARS-CoV-2 PCR testing lies between the result observed in this study and that observed in studies which compare a single test with repeated sampling. Future studies should consider the use of latent class analysis to triangulate serology, PCR sampling and imaging, ideally systematically employing CT imaging to better understand the true performance characteristics of diagnostic assays by synthesis of multiple imperfect tests.³⁰

In conclusion, using a combined clinical and radiological reference test in the setting of a high probability of COVID-19 during the first peak of the pandemic, we observed SARS-CoV-2 PCR clinical sensitivity of 68%. Clinicians should carefully consider the limitations of this diagnostic test, particularly in a setting where the pre-test probability is high and should avoid premature diagnostic closure on the basis of PCR testing alone. Interpretation of the test result should take into account the latest locally relevant epidemiological data, the nature and extent of radiological findings, smoking status, and the duration from symptom onset at the time of testing.

Declaration of Competing Interest

None.