Performance Assessment of a Rapid Molecular Respiratory Syncytial Virus Point-of-Care Test: A Prospective Community Study in Older Adults

Abstract

Background

Respiratory syncytial virus (RSV) causes a substantial burden in older adults. Viral load in RSV-infected adults is generally lower compared to young children, which could result in suboptimal sensitivity of RSV diagnostics. Although the Xpert® Xpress Flu/RSV assay has been used in routine clinical care, its sensitivity to diagnose RSV infection in older adults is largely unknown. We aimed to compare the performance of the Xpert® Xpress Flu/RSV assay with real-time reverse-transcription polymerase chain reaction (RT-PCR) in home-dwelling older adults (≥60 years of age).

Methods

Nasopharyngeal swabs were tested with Xpert® Xpress Flu/RSV and compared to RSV RT-PCR in older adults with acute respiratory tract infections with different levels of disease severity.

Results

We studied 758 respiratory samples from 561 older adults from 2 consecutive RSV seasons. Thirty-five (4.6%) samples tested positive for RSV by at least 1 of the assays, of which 2 samples were negative by Xpert® Xpress Flu/RSV and 3 samples by real-time RT-PCR. The positive percentage agreement (PPA) was 90.9% (95% confidence interval [CI], 76.4%–96.8%) and negative percentage agreement was 99.7% (95% CI, 99.0%–99.9%). Viral loads were low (≤103 copies/mL or cycle threshold value ≥34) in all cases with discordant results for the 2 assays.

Conclusions

The PPA of Xpert® Xpress Flu/RSV compared to routine RT-PCR is high for RSV detection in home-dwelling older adults. The assay is fast and easy to use at the point of care.

Clinical Trials Registration

NCT03621930.

Lower respiratory tract infections are estimated to be the fifth-leading cause of mortality worldwide [1]. Respiratory syncytial virus (RSV) is a major cause of respiratory infections in older adults (≥60 years) with a substantial disease burden [2–5]. The annual incidence rate of RSV infection in community-dwelling older adults is estimated at 3%–7.2% [6, 7]. It was estimated that approximately 14 000 (range, 5000–50 000) in-hospital deaths due to acute respiratory tract infections (ARTIs) in older adults were related to RSV in 2015 [3].

Currently, the gold standard for RSV diagnosis is laboratory-based real-time reverse-transcription polymerase chain reaction (RT-PCR). RT-PCR has the disadvantage that it requires technical skills and has a long turnaround time. Therefore, reliable rapid diagnostic tests are needed to improve patient management, to enable cohorting and isolation of hospitalized patients, to prevent unnecessary use of antibiotics [8], and for the use of RSV antivirals for treatment in the near future [9].

In recent years, several point-of-care tests (POCTs), among other rapid antigen diagnostic tests (RADTs) and molecular assays, have been developed to detect RSV. In general, RADTs are less sensitive compared to molecular POCTs. PCR-based molecular POCT assays are available and used in clinical practice because they are fast, easy to use by nonlaboratory personnel, and could be less expensive compared to routine RT-PCR; however, they are less suitable for high throughput. The turnaround time of most molecular POCTs is <1 hour. The use of molecular POCTs is associated with a significant reduction in hospital length of stay, testing costs, and isolation time [10, 11]. The Xpert^® Xpress Flu/RSV assay (Cepheid, Sunnyvale, California) is one of the commercially available molecular POCTs [12]. It is a real-time RT-PCR assay using a single disposable cartridge. Previous studies reported a sensitivity and specificity for RSV ranging from 90.5% to 100% and 99.6% to 100%, respectively [13–17]. However, these studies bear the risk of overestimating test accuracy as they were performed in medically attended or hospitalized patients [13–16], used remnant specimens [13, 15], were partially performed in children with predictable high viral loads [14, 15], were mostly sponsored by the manufacturer [14–16], and were performed in relatively small numbers of patients [13, 15, 17]. In an earlier report, we showed an unexpectedly low sensitivity for RADT BinaxNOW RSV in infants with different levels of care, thereby demonstrating the importance of validating these POCTs in different populations [18].

The aim of the current study was to evaluate the performance of the Xpert^® Xpress Flu/RSV assay [19] to diagnose RSV infection in home-dwelling older adults (≥60 years) with ARTI in different clinical settings as part of a large international prospective cohort study.

MATERIALS AND METHODS

Study Population

The study population consisted of older adults (≥60 years of age) with ARTI who were participating in the REspiratory Syncytial Virus Consortium in EUrope (RESCEU) [7] older adult cohort study during 2 consecutive RSV seasons, 2017–2018 and 2018–2019. RESCEU is a European Union–funded consortium aiming to determine RSV burden of disease in Europe. The study was performed in Belgium (Antwerp), the Netherlands (Utrecht), and the United Kingdom (Oxford). Participants were recruited from 17 general practices before the start of each RSV season. A total of 1040 community-dwelling older adults participated in the study, of whom approximately 50% were >75 years of age. Participants were followed during 1 RSV season; between 1 October and 30 April, nasopharyngeal swabs were collected for RSV testing each time a participant experienced an ARTI. Participants were contacted weekly by email or telephone during the RSV season to ask for symptoms of ARTI, which was defined as the presence of 1 or more of the following symptoms for at least 1 day: cough, nasal congestion or discharge, wheezing, or shortness of breath. Samples were taken by a trained member of the study team at home. Details of the study design and procedures have been previously described [7].

Data on age, sex, comorbidities, duration of symptoms of ARTI, and level of medical care needed were obtained by completing questionnaires and case report forms. We defined 3 levels of medical care: (1) participants with ARTI who were hospitalized; (2) participants with medically attended (MA) ARTI, defined as participants who were seen at the emergency department or general practice but were not admitted to the hospital; and (3) participants with non-MA ARTI who did not see any clinician during the entire ARTI episode.

Study Procedures

Two nasopharyngeal minitip flocked swabs (FLOQSwab, Copan Diagnostics) were collected by a member of the study team and directly stored in universal transport medium (UTM) (Copan Diagnostics, 3 mL) and MicroTest M4RT (Remel, 3 mL), respectively. Samples were transported at room temperature. Three hundred microliters of UTM was used for point-of-care (POC) analysis by the Xpert^® Xpress Flu/RSV assay (Cepheid) [19]. POC testing was performed within 24 hours. The remaining UTM sample was discarded. The MicroTest M4RT sample was stored in aliquots at –80°C for later analysis by RT-PCR assay. The staff was trained on how to sample patients and how to use the POCT before the start of the study.

Virology

Both assays reported information on viral load (cycle threshold [Ct] value for Xpert^® Xpress and copies/mL for RT-PCR). The Xpert^® Xpress POCT was performed according to the manufacturer’s instructions. In short, 300 μL of the viral transport medium mixed with the swab was aspirated with the included transfer pipette. The cartridge was opened and the entire content of the filled pipette was slowly expelled into the cartridge. Subsequently, the cartridge was inserted into the GeneXpert System. After approximately 30 minutes, test results were available on the screen. The assay targeted the RSV N gene, encoding the RSV nucleocapsid, using 3 RSV-A and 2 RSV-B strains [20]. A test was positive if the threshold was reached before completion of the full 40 PCR cycles. In case of a positive test, RSV viral load was reported as a Ct value.

For RT-PCR, an in-house–developed kit was used. RSV-A and -B were detected and quantified by duplex RT-PCR using specific amplification primers and fluorescent probes designed to detect the RSV N gene. The process involves extraction of nucleic acids, conversion of RNA to complementary DNA by reverse transcription, and detection by real-time PCR reaction using a calibration curve (absolute quantitation). Two hundred microliters of M4RT from nasal swab samples was used for the nucleic acid extraction (KingFisher, MagMax Core kit). Nucleic acids were eluted in a volume of 80 µL, and 2.5 µL of the elution was used per RT-PCR amplification. Limits of detection (LODs) were determined via probit approach, as recommended in the Clinical and Laboratory Standards Institute EP17-A2 guidance. Several dilutions of surrogate samples (M4RT transport medium spiked with different concentrations of RSV-A and RSV-B strains) were used for their determinations. The RSV-A RT-PCR has an LOD of 304 copies/mL, whereas the LOD for the RSV-B RT-PCR is 475 copies/mL. Clinical samples were considered positive when the load was higher than the respective LODs. RT-PCR of all samples was done at the same moment and location.

Statistical Analysis

Only samples tested with both assays were included in the analysis. Test results of the Xpert^® Xpress assay were compared to routine real-time RT-PCR as reference standard using positive percentage agreement (PPA), negative percentage agreement (NPA), and overall rate of agreement (ORA). Using percentage agreement rather than accuracy and sensitivity is recommended by the US Food and Drug Administration (FDA) when comparing results of a new test with an imperfect reference test, as RT-PCR is not 100% accurate and comparable to molecular POCT tests [21]. Confidence intervals (CIs) were calculated using the Wilson score test. Patient characteristics were compared between the 4 outcome categories using χ² or Fisher exact test for categorical data and Mann–Whitney U test for continuous data. P values < .05 were considered statistically significant. Multivariate logistic regression analysis was used to determine whether PPA of the tests was associated with age, duration of symptoms, or level of care. In these models PPA was used as binary outcome, defined as results positive for both assays and positive RT-PCR results combined with negative POCT result. Statistical analyses were conducted using R version 3.6.1 within RStudio version 1.2.5.

Ethical Considerations

This study was approved by the ethical review authorities in Belgium (reference number B300201732907), the Netherlands (reference number NL60910.041.17), and the United Kingdom (ethics reference 17/LO/1210, IRAS reference 224156). Participants provided informed consent before taking part in this study. The study was conducted in accordance with the Declaration of Helsinki, as revised in 2013.

RESULTS

Acute Respiratory Tract Infections

In total, 758 samples from 561 participants with symptoms of ARTI were tested with Xpert^® Xpress Flu/RSV and RT-PCR (Figure 1). Eighty-six ARTI episodes were excluded because 1 or both tests were not performed. Characteristics of excluded episodes did not differ from the included episodes except for country and severity, showing that significantly more hospitalizations and MA ARTI episodes did not have both tests performed (Supplementary Table 1). The median age of participants at the time of ARTI was 75 years (interquartile range [IQR], 67–80 years). Comorbidity was present in 291 (38.4%) participants, including cardiac disease, pulmonary disease, and diabetes (Supplementary Table 2). Three hundred ninety-six participants were tested once, 136 were tested twice, and 29 were tested 3 times or more (maximum 4 times) during separate ARTI episodes. Sample collection and participant characteristics of the 4 outcome categories are displayed in Table 1 and showed no significant differences between categories. Swabs were taken after a median duration of symptoms of 4 days (IQR, 2–6 days). Most respiratory episodes were mild with only 4 (0.5%) hospitalizations and 170 (22.4%) MA ARTI episodes (Table 1).

Figure 1.

Flowchart of participants of the REspiratory Syncytial Virus Consortium in EUrope (RESCEU) older adult cohort study with at least 1 acute respiratory tract infection (ARTI) during follow-up. Respiratory syncytial virus (RSV)–negative cases were negative by both reverse-transcription polymerase chain reaction (RT-PCR) and Xpert^® Xpress Flu/RSV (point-of-care test [POCT]).

Table 1.

Sample Characteristics, Stratified by Test Assay Result

Characteristic	POCT–/RT-PCR–	POCT+/RT-PCR+	POCT–/RT-PCR+	POCT+/RT-PCR–
No. of episodes	723	30	2	3
Country
Belgium	222 (30.7)	11 (36.7)	0 (0.0)	1 (33.3)
Netherlands	283 (39.1)	13 (43.3)	2 (100.0)	0 (0.0)
United Kingdom	218 (30.2)	6 (20.0)	0 (0.0)	2 (66.7)
Duration of symptoms at moment of sample collection, d, median (IQR)	4 (2–6)	3 (2–5)	4.5 (3–6)	6 (4.5–6)
Sex, female	391 (54.1)	15 (50.0)	2 (100.0)	2 (66.7)
Comorbidity	279 (38.6)	9 (30.0)	2 (100.0)	1 (33.3)
Age at ARTI episode, y, median (IQR)	75 (67–80)	75 (70–79.5)	69 (66.5–71.5)	79 (78.5–79)
Level of care needed
Hospitalized	4 (0.6)	0 (0.0)	0 (0.0)	0 (0.0)
MA ARTI	160 (22.1)	9 (30.0)	0 (0.0)	1 (33.3)
Non–MA ARTI	551 (76.2)	21 (70.0)	2 (100.0)	2 (66.7)
Not known	8 (1.1)	0 (0.0)	0 (0.0)	0 (0.0)

Data are presented as No. (%) unless otherwise indicated. Categories are based on Xpert^® Xpress Flu/RSV (POCT) and RT-PCR test results.

Abbreviations: –, negative; +, positive; ARTI, acute respiratory tract infection; IQR, interquartile range; MA, medically attended; POCT, point-of-care test; RT-PCR, reverse-transcription polymerase chain reaction.

RSV Acute Respiratory Tract Infection

RSV was detected in 35 samples (4.6%) by at least 1 of the assays (33 by Xpert^® Xpress, and 32 by RT-PCR). We found a PPA of 90.9% (95% CI, 76.4%–96.8%), and an NPA of 99.7% (95% CI, 99.0%–99.9%) for Xpert^® Xpress Flu/RSV compared to RT-PCR (Table 2). The ORA between both tests was 99.3% (95% CI, 98.5%–99.7%). Five samples showed discordant test results (Table 3). All discordant samples had a low viral load (≤10³ copies/mL or Ct value ≥34). Two of the 3 samples tested positive by Xpert^® Xpress and negative by RT-PCR and showed a low number of RSV viral copies with RT-PCR, but did not meet the threshold of viral copies to be considered positive. We found a moderately strong correlation between the Xpert^® Xpress Flu/RSV and RT-PCR for viral load (Pearson’s r = –0.70, P < .001; Figure 2). We found no significant effects of age, gender, duration of symptoms, comorbidity, and level of care on PPA using multivariate logistic regression tests.

Table 2.

Primary Outcome: Performance of Cepheid Xpert^® Xpress Flu/RSV Compared With Reverse-Transcription Polymerase Chain Reaction

Test	PPA, %	NPA, %	ORA, %
Xpert® Xpress Flu/RSV (n = 758 ARTI episodes)	90.9 (30/33)	99.7 (723/725)	99.3 (753/758)
95% CI, %	76.4–96.8	99.0–99.9	98.5–99.7

Abbreviations: ARTI, acute respiratory tract infection; CI, confidence interval; NPA, negative percentage agreement; ORA, overall rate of agreement; PPA, positive percentage agreement.

Table 3.

Characteristics of Patients With Discordant Test Results Between Xpert^® Xpress Flu/RSV (Point-of-Care Test) and Reverse-Transcription Polymerase Chain Reaction

Test Result	RSV Subtype	Ct Value POCT	RT-PCR, Viral Copies/mL (log10)	Duration of Symptoms at Moment of Sample Collection, d	Level of Care	Age, y	Sex
POCT+/RT-PCR–	RSV-B	36.0	399a (2.6)	3	Non-MA ARTI	78	Female
POCT+/RT-PCR–	RSV-B	34.8	290a (2.5)	6	MA ARTI	79	Female
POCT+/RT-PCR–	NA	36.1	0	6	Non-MA ARTI	79	Male
POCT–/RT-PCR+	RSV-A	NA	532 (2.7)	7	Non-MA ARTI	64	Female
POCT–/RT-PCR+	RSV-B	NA	1080 (3.0)	2	Non-MA ARTI	74	Female

RSV genotype was determined by RT-PCR.

Abbreviations: –, negative; +, positive; ARTI, acute respiratory tract infection; Ct, cycle threshold; MA, medically attended; NA, not available; POCT, point-of-care test; RSV, respiratory syncytial virus; RT-PCR, reverse-transcription polymerase chain reaction.

Below limit of detection for RSV-B (475 copies/mL).

Figure 2.

Scatterplot of viral copies/mL (log₁₀) by reverse-transcription polymerase chain reaction (RT-PCR) and cycle threshold (Ct) value by Xpert^® Xpress Flu/RSV (point-of-care test). Black diamond dots are tested positive with both assays. Crosses indicate discordant test results. Dots below the horizontal or left of the vertical lines are undetectable by 1 of the assays. The blue line shows the regression line with confidence interval (gray shadow).

DISCUSSION

This is the first community-based study of the performance of Xpert^® Xpress Flu/RSV in older adults. We found a high PPA and ORA between Xpert^® Xpress and RT-PCR (90.9% and 99.3%, respectively) for RSV detection. Test failure for either test was exclusively observed in patients with low viral load, around or below LOD of both tests.

Results of our study are comparable with previous studies that were performed in a clinical setting or in hospitalized patients showing a sensitivity of 90.5% to 100% [13–17]. The sample size of these studies varied between 172 and 2553 participants with a mixed age spectrum (infants to older adults), with RSV positivity varying from 3.5% to 55%. All studies used a nasopharyngeal swab or nasopharyngeal aspirate as the sampling method and used RT-PCR as the reference. The low viral load in all cases of discordant test results in our study was in line with earlier reports [16].

A strength of our study is that it is part of a large prospective clinical study with a well-defined, community-based population and performed in different countries across Europe. Our study design was based on clinical endpoints rather than virological, ensuring a low risk of bias. Therefore, we were able to evaluate the performance of Xpert^® Xpress Flu/RSV in a community setting that included mild disease, whereas other studies only evaluated the performance of Xpert^® Xpress Flu/RSV in a clinical setting or in hospitalized patients. Second, rather than using sensitivity to present concordance between both tests, we used PPA to describe the performance of Xpert^® Xpress Flu/RSV compared to RT-PCR. While real-time RT-PCR is widely used as the gold standard for virus detection, there is no assay with 100% accuracy. Most molecular POCT assays are using the same nucleic amplification method as RT-PCR and are known for their high sensitivity and specificity, similar to RT-PCR [12]. This way of displaying results is recommended when comparing results of a new test with an imperfect reference test [21].

There are several limitations to our study. First, we used UTM viral transport medium for analysis with Xpert^® Xpress according to the manufacturer’s instructions, and M4RT for RT-PCR analysis, both with different nasopharyngeal swabs and analyzed at different time points. This could have had an effect on viral load of the specimens. Both nasopharyngeal swabs were taken at the same moment by the same research personnel to minimize any possible effects on viral load. However, with low viral loads this could lead to a difference in test results. To our knowledge, there is no literature on viral transport media affecting viral load. For both tests we used the recommended viral transport medium. The M4RT samples were stored at –80°C until testing. This temperature allows long-term sample storage without significant effects on quality of samples. Second, as this is a community-based cohort study, the number of RSV-positive samples was relatively low (n = 35 [4.6%]). However, we are confident that our results are reliable, based on the high concordance between both tests in a representative range of viral loads. Third, 10.2% of ARTI episodes could not be used for this study because none or only 1 assay was performed. Although significantly more hospitalizations and MA ARTI episodes were missed, we do not believe this had an impact on our results because other studies have previously shown a high sensitivity in these populations [13–17]. Last, although Xpert^® Xpress Flu/RSV also reported influenza results of tested samples, influenza virus was not tested by RT-PCR for practical reasons. Influenza results of Xpert^® Xpress Flu/RSV in our cohort have been described previously [7].

RSV is a significant cause of moderate to severe respiratory tract infection in older adults [22]. Early detection of the virus can improve patient management and outcomes [23]. In addition, rapid testing can be important as companion diagnostics for use of future RSV antivirals at an early stage [24]. Molecular POC assays are highly sensitive and easy to use. The Xpert^® Xpress Flu/RSV assay is among the 4 low-complex Clinical Laboratory Improvement Amendments–waived molecular assays approved by the FDA [12]. It is performed on the Cepheid GeneXpert System, which can also be used for multiple other pathogens and is suitable for testing up to 16 samples at the same time. Hands-on time is estimated to be 1–2 minutes, and turnaround time is about 30 minutes [12, 13]. As the availability of molecular POCTs is increasing, these assays might also be introduced into outpatient settings. Our study added valuable information about the PPA in patients who needed different levels of care to existing literature, which is important to know before implementing molecular POCTs in these settings.

In conclusion, we have performed the first international prospective community-based study to compare the performance of a rapid molecular detection test for RSV infection with RT-PCR in home-dwelling older adults. We demonstrated that the PPA and ORA between Xpert^® Xpress Flu/RSV and routine RSV RT-PCR for RSV detection in home-dwelling older adults is high. The assay is fast and easy to use and therefore has the ability to improve patient management and outcomes.

Notes

Study group members. The RESCEU investigators are as follows: Roy Zuurbier, Koos Korsten, Theo Verheij, Marlies van Houten, Louis Bont, Joanne Wildenbeest (University Medical Centre Utrecht); Niels Adriaenssens, Samuel Coenen (University of Antwerp); Christopher Butler, Andrew Pollard (University of Oxford); Valerie Vantomme, Olivier Gruselle, Amanda Leach (GlaxoSmithKline); Harish Nair, Harry Campbell (University of Edinburgh); Philippe Beutels (University of Antwerp); Peter Openshaw (Imperial College London); Federico Martinon-Torres (Servicio Galego de Saude); Terho Heikkinen (University of Turku and Turku University Hospital); Adam Meijer (National Institute for Public Health and the Environment); Thea Kølsen Fischer (Statens Serum Institut); Maarten van den Berge (Academisch Ziekenhuis Groningen); Carlo Giaquinto (Fondazione PENTA for the Treatment and Care of Children with HIV-ONLUS); Michael Abram (AstraZeneca); Kena Swanson (Pfizer); Jeroen Aerssens (Janssen); Clarisse Demont, Scott Gallichan (Sanofi Pasteur); Brian Rosen (Novavax); Eva Molero (Team-It Research).

Acknowledgments. We thank all the participants, and all the members of the research teams. We thank Cepheid for unconditionally providing us with the point-of-care test used in this study.

Disclaimer. This manuscript reflects only the views of the authors. The European Commission is not responsible for any use that may be made of the information it contains.

Financial support. RESCEU has received funding from the Innovative Medicines Initiative 2 Joint Undertaking under grant agreement number 116019. This Joint Undertaking receives support from the European Union’s Horizon 2020 Research and Innovation Programme and European Federation of Pharmaceutical Industries and Associations (EFPIA).

Supplement sponsorship. This article appears as part of the supplement “Results from RESCEU study: Evidence for Policy,” sponsored by RESCEU.