Diagnostic accuracy and feasibility of patient self-testing with a SARS-CoV-2 antigen-detecting rapid test

Abstract

Background

Considering the possibility of nasal self-sampling and the ease of use in performing SARS-CoV-2 antigen-detecting rapid diagnostic tests (Ag-RDTs), self-testing is a feasible option.

Objective

The goal of this study was a head-to-head comparison of diagnostic accuracy of patient self-testing with professional testing using a SARS-CoV-2 Ag-RDT.

Study design

We performed a manufacturer-independent, prospective diagnostic accuracy study of nasal mid-turbinate self-sampling and self-testing with symptomatic adults using a WHO-listed SARS-CoV-2 Ag-RDT. Procedures were observed without intervention. For comparison, Ag-RDTs with nasopharyngeal sampling were professionally performed. Estimates of agreement, sensitivity, and specificity relative to RT-PCR on a combined oro-/nasopharyngeal sample were calculated. Feasibility was evaluated by observer and participant questionnaires.

Results

Among 146 symptomatic adults, 40 (27.4%) were RT-PCR-positive for SARS-CoV-2. Sensitivity with self-testing was 82.5% (33/40; 95% CI 68.1–91.3), and 85.0% (34/40; 95% CI 70.9–92.9) with professional testing. At high viral load (≥7.0 log₁₀ SARS-CoV-2 RNA copies/ml), sensitivity was 96.6% (28/29; 95% CI 82.8–99.8) for both self- and professional testing. Deviations in sampling and testing were observed in 25 out of the 40 PCR-positives. Most participants (80.9%) considered the Ag-RDT as easy to perform.

Conclusion

Laypersons suspected for SARS-CoV-2 infection were able to reliably perform the Ag-RDT and test themselves. Procedural errors might be reduced by refinement of the instructions for use or the product design/procedures. Self-testing allows more wide-spread and frequent testing. Paired with the appropriate information of the public about the benefits and risks, self-testing may have significant impact on the pandemic.

1. Background

Antigen-detecting rapid diagnostic tests (Ag-RDTs) for SARS-CoV-2 have been widely recommended as a complement to RT-PCR [1]. Recent studies have shown the equivalence of anterior nasal, nasal mid-turbinate (NMT), and nasopharyngeal (NP) sampling, and established the feasibility of self-sampling for a WHO-listed SARS-CoV-2 Ag-RDT [2], [3], [4]. Considering the ease-of-use in performing the test, self-testing is a feasible option. In Germany, first SARS-CoV-2 Ag-RDTs received special approvals for self-administration by laypersons on 24 February 2021 [5].

2. Methods

We conducted a manufacturer-independent prospective study of diagnostic accuracy, user acceptability and feasibility of an Ag-RDT when performed by patients themselves, with using a self-collected NMT sample. For comparison, professional NP-sampling with testing on Ag-RDT was performed on the same participant as well as combined oropharyngeal (OP)/NP-sampling for RT-PCR as a reference [2].

The study took place at the ambulatory SARS-CoV-2 testing facility of Charité - Universitätsmedizin Berlin, from 30 November to 11 December 2020. Participants eligible for inclusion were adults with high suspicion of SARS-CoV-2 infection. Participants had to be able to understand the written instructions in German or English, defined as a minimum Common European Framework of Reference for Languages (CEFR) level of B2 (upper intermediate). Participants were enrolled according to laboratory capacity in a consecutive series.

The Ag-RDT evaluated was the STANDARD Q COVID-19 Ag Test (SD Biosensor, Korea), which is also distributed by Roche [6]. The test was chosen in consultation with the Foundation of Innovative New Diagnostics based on data on accuracy and ease-of-use. The NP-sampling and nasal sampling kits were provided with differing flocked swabs, whereby the nasal swab used for self-sampling is less flexible with a larger sampling surface [4]. The participants received written and illustrated instructions in German or English, adapted from the manufacturer's instructions for use. Participants performed the procedures in a separate room without time restrictions. The procedures were observed by a study physician, without answering questions or providing corrections. The ease of use was evaluated with participant and observer questionnaires.

NMT self-sampling (both sides) was followed by professional NP-sampling (through one nostril) for Ag-RDTs and combined OP/NP-sampling (through the other nostril) for RT-PCR. The Ag-RDTs were performed directly after sampling at point-of-care by participants and trained study physicians.

All Ag-RDT results were interpreted by two blinded study physicians with a semi-quantitative assessment, in addition to the participant's interpretation of the self-test [4]. A minimum of 30 positive NP or NMT samples according to Ag-RDT was set, which is recommended by the WHO Emergency Use Listing Procedure to demonstrate sample type equivalency [7].

3. Results

After exclusion of 4 participants (n = 3 not fulfilling the CEFR minimum language criterion, n = 1 lost PCR specimen), 146 adults were included in the analysis (see Supplementary Figure S1). Of these, 40 participants (27.4%) tested positive by RT-PCR. Mean age was 35 years (Standard Deviation [SD] 11.5), and 51.4% were female. All participants were symptomatic at the time of presentation, with a mean duration of 3.4 days (SD 2.0) post symptom onset. Previous use of any RDTs and/or laboratory and/or home-test experience was reported by 29 participants (20.4%). A higher education degree was present in 84 (59.6%) participants. Thirty-eight participants (26.6%) were not native German or English speakers (see Supplementary Table S2).

Self-testing (including self-read-out) yielded a sensitivity of 82.5% (33/40 RT-PCR positives detected; 95% CI 68.1–91.3) and a specificity of 100% (104/104; CI 96.5–100) compared to RT-PCR (Table 1 ). The sensitivity with professional Ag-RDT testing was 85.0% (34/40; CI 70.9–92.9) and specificity was 99.1% (105/106; CI 94.8–99.5). In patients with high viral load (≥7.0 log₁₀ SARS-CoV-2 RNA copies/ml) the sensitivity was 96.6% (28/29; CI 82.8–99.8) for both self-testing and professional testing. The positive percent agreement between self-testing and professional testing on Ag-RDT was 91.4% (32/35; CI 77.6–97.0); the negative percent agreement was 99.1% (108/109; CI 95.0–100).

Table 1.

Antigen-RDT self-testing (NMT sample) with self-readout and professional readout of the result, as well as professional testing (NP sample) in RT-PCR positive outpatients from combined OP/NP swab. Ct-values and viral loads of the paired RT-PCR samples are shown as well as the duration of symptoms. Relevant protocol deviations in the self-test are noted.

No.	Ag-RDT self-use (NMT)		Ag-RDT prof.-use (NP)	RT-PCR (OP/NP)		Days of Symptoms	Deviations in Ag-RDT self-use
	self-readout	prof-readout		Ct value	Viral load1
1	pos.	pos. (+++)	pos. (+++)	14.822	9.57	2	sampling4
2	pos.	pos. (+++)	pos. (+++)	18.293	9.30	2	extraction5
3	pos.	pos. (+++)	pos. (+++)	16.202	9.16	2	extraction5
4	pos.	pos. (+++)	pos. (+++)	16.342	9.12	3	extraction5
5	pos.	pos. (+++)	pos. (+++)	16.892	8.96	1	sampling4, extraction5
6	pos.	pos. (+++)	pos. (+++)	19.513	8.94	2	extraction5, volume6
7	pos.	pos. (+++)	pos. (+++)	17.432	8.80	2
8	pos.	pos. (+++)	pos. (+++)	20.303	8.71	4
9	pos.	pos. (++)	pos. (+++)	17.742	8.70	3
10	pos.	pos. (+++)	pos. (+++)	20.353	8.69	4
11	pos.	pos. (+++)	pos. (+++)	17.942	8.64	3	extraction5, volume6
12	pos.	pos. (+++)	pos. (+++)	20.723	8.58	1
13	pos.	pos. (++)	neg.	18.502	8.48	1	sampling4, extraction5
14	pos.	pos. (+++)	pos. (+++)	21.343	8.40	3
15	pos.	pos. (+++)	pos. (+++)	18.962	8.34	7	extraction5, volume6
16	pos.	pos. (+++)	pos. (+++)	19.262	8.25	2	sampling4, extraction5
17	pos.	pos. (+++)	pos. (+++)	19.292	8.24	3
18	pos.	pos. (+++)	pos. (+++)	21.943	8.22	1	extraction5
19	pos.	pos. (+++)	pos. (+++)	19.482	8.19	4
20	pos.	pos. (+++)	pos. (+++)	19.492	8.18	4	sampling4
21	pos.	pos. (+++)	pos. (+++)	19.522	8.17	4	sampling4, extract.5, vol.6
22	pos.	pos. (+++)	pos. (+++)	19.742	8.11	2	sampling4, extraction5
23	pos.	pos. (+++)	pos. (+++)	19.952	8.05	3
24	pos.	pos. (++)	pos. (+++)	20.672	7.83	3	extraction5
25	pos.	pos. (+++)	pos. (+++)	23.713	7.70	6	sampling4
26	pos.	pos. (++)	pos. (+++)	23.993	7.62	3	extraction5, volume6
27	pos.	pos. (+++)	pos. (+++)	21.982	7.44	7
28	pos.	pos. (++)	pos. (+++)	24.893	7.35	8	sampling4, extraction5
29	neg.	pos. (++)	pos. (++)	22.672	7.24	3	read-out7
30	pos.	pos. (++)	pos. (++)	24.862	6.59	7	sampling4
31	neg.	neg.	pos. (+)	25.072	6.52	5
32	pos.	pos. (+++)	pos. (++)	28.583	6.26	8
33	neg.	neg.	neg.	27.052	5.94	1
34	pos.	pos. (++)	pos. (+)	29.712	5.15	5	extraction5, volume6
35	pos.	pos. (++)	pos. (+)	30.032	5.05	7	extraction5
36	pos.	pos. (+)	pos. (++)	30.282	4.98	7	sampling4, extraction5
37	neg.	neg.	neg.	35.693	4.15	7
38	neg.	neg.	neg.	33.292	3.91	14	sampling4, extract.5, vol.6
39	neg.	neg.	neg.	34.542	3.71	8
40	neg.	neg.	neg.	34.992	3.41	14	extraction5
	Sensitivity 33/40 (82.5%)	Sensitivity 34/40 (85.0%)	Sensitivity 34/40 (85.0%)

¹log10 SARS-CoV-2 RNA copies/ml;

²TIB Molbiol assay, E-gene target;

³roche cobas SARS-CoV-2 assay (E-gene, T2 target);

⁴sampling of possible reduced quality (unilateral, short, different position);

⁵extraction of possible reduced quality related to stirring of the swab or squeezing of the buffer tube;

⁶vol = volume: incorrect number of drops applied on the test device;

⁷read-out: misinterpretation of the test-result.

One patient with a positive self-test had falsely interpreted his result as negative. Regarding divergent results, there was one participant yielding a false-negative result with self-testing who performed the test according to instructions, and one false-negative result with professional testing despite a high viral load. In the latter patient, NP-sampling was not optimal due to poor tolerance, which occurs rather frequently in clinical practice.

Inter-rater reliability for the double read-out of the self-test by the participant and the study physician was very high (kappa 0.98) and close to that of two study physicians interpreting the self-test (kappa 1) or the professionally performed test (kappa 0.98).

Deviations of self-sampling included a more vertically-directed angle for sampling (23%), incorrect depth of insertion of the swab (4% too superficial, 10% too deep), reduced intensity of swabbing (as to duration 24%, rotations 10% and rubbing 42%) and unilateral NMT-sampling (6%). One intervention by the study physician was necessary because of a possible risk of injury when the patient tried to insert the swab upside down into the nose.

Deviations of self-testing were observed for the specimen extraction (less stirring of the swab 4%, inadequate tube squeezing while stirring 35% and while removing the swab 33%). Furthermore, it proved difficult to apply exactly 4 drops to the sample well of the test device (27% applying more, 4% less). Several drops coming out at once was the main problem (see Supplementary Table S3).

Participants pointed out that their test performance may have been impaired by nervousness (3%), fever or dizziness, poor concentration, feeling cold, aversion to self-sampling, language barrier and limited fine motor skills (1% each). On a scale from 1 (easy) to 5 (difficult), 114 (81%) participants stated that the test was easy to perform (scale 1/2); 23 (16%) found it medium easy/difficult (scale 3), and 4 (3%) difficult (scale 4).

4. Discussion

The data presented here demonstrates the feasibility and accuracy of self-testing in an unselected population and lay the ground for potential broader use. Self-testing with Ag-RDTs could not only alleviate overstretched RT-PCR capacity and medical personnel, but also may result in increased access to frequent testing and significant impact on the pandemic [8,9]. If implemented for society at large, recent modeling data suggest that repeated screening for SARS-CoV-2, combined with immediate reporting, isolation, and quarantine, can greatly reduce viral transmission. In this model, test sensitivity is of minor importance [8].

However, self-testing should be accompanied by widespread public campaigns informing about limited sensitivity, the importance of complementary hygiene measures, e.g., mask use, physical distancing, and the necessity of self-quarantine in case of a positive test. Reporting requirements in positive cases should be clarified. As long as Ag-RDTs test capacities are limited, deployment must be prioritized, also with a global perspective. Individually, Ag-RDT self-testing could contribute to resumption of a certain degree of normal life, e.g., self-testing could be done before visiting nursing homes.

The study is limited as it was performed in a single center. Patients were rather young and educated. The generalizability of the findings and applicability to settings with different prevailing patient characteristics needs to be confirmed.

In conclusion, we demonstrate that symptomatic laypersons can reliably perform a SARS-CoV-2 lateral-flow Ag-RDT and test themselves. Procedural errors might be reduced by refinement of the Ag-RDTs for self-testing, such as modified instructions for use or product design/procedures. Paired with the appropriate information of the public about the benefits and risks, self-testing may have significant impact on the pandemic.

CRediT authorship contribution statement

Andreas K. Lindner: Conceptualization, Methodology, Writing - original draft, Writing - review & editing. Olga Nikolai: Methodology, Investigation, Formal analysis, Writing - review & editing. Chiara Rohardt: Methodology, Investigation, Writing - review & editing. Franka Kausch: Investigation, Writing - review & editing. Mia Wintel: Investigation, Writing - review & editing. Maximilian Gertler: Project administration, Writing - review & editing. Susen Burock: Investigation, Writing - review & editing. Merle Hörig: Investigation, Writing - review & editing. Julian Bernhard: Investigation, Writing - review & editing. Frank Tobian: Formal analysis, Writing - review & editing. Mary Gaeddert: Formal analysis, Writing - review & editing. Federica Lainati: Supervision, Writing - review & editing. Victor M. Corman: Investigation, Formal analysis, Writing - review & editing. Terry C. Jones: Investigation, Formal analysis, Writing - review & editing. Jilian A. Sacks: Supervision, Writing - review & editing. Joachim Seybold: Supervision, Project administration. Claudia M. Denkinger: Conceptualization, Methodology, Supervision, Writing - review & editing. Frank P. Mockenhaupt: Conceptualization, Methodology, Supervision, Writing - original draft, Writing - review & editing.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.