Comparison of the Allplex^TM Respiratory Panel Assays and the automated Fast Track Diagnostics Respiratory pathogens 21 assay for the diagnosis of pediatric respiratory viral infections

Abstract

Acute respiratory tract infections frequently occur in children and represent one of the leading causes of morbidity and mortality worldwide. Quick and accurate pathogen detection can lead to a more appropriate use of antimicrobial treatment as well as timely implementation of isolation precautions. In the last decade, several commercial assays have been developed for the simultaneous diagnosis of respiratory pathogens, which substantially vary in formulation and performance characteristics. The aim of this study was to compare the performance of the “Allplex^TM Respiratory Panel Assays” (Seegene) with that of the automated “Fast Track Diagnostics Respiratory pathogens 21” assay (Siemens) for the diagnosis of pediatric respiratory viral infections. One hundred forty-five nasopharyngeal wash samples, collected at the Bambino Gesù Pediatric Hospital in Rome during the fall-winter 2017-2018 season, were processed and analyzed with both workflows. Our results suggest a high concordance between the two methods for positive and negative samples. Sensitivity and specificity were calculated with both tests as a reference method. For the Allplex^TM Respiratory Panel Assays, they were 98% and 100%, respectively, and for the Fast Track Diagnostics Respiratory pathogens 21 assay, they were both 100%. This comparative study allowed us to highlight the characteristics of the two assays to evaluate the best solution, on the basis of diagnostic routine and laboratory workflows, keeping in mind local epidemiology.

Introduction

Acute respiratory infections (ARIs) are the main cause of morbidity in patients under 18 years of age. Data collected from the World Health Organization indicate that nearly 2 million children died from ARIs in the year 2000, most in Africa and South East Asia [1, 2]. In children under 5 years old, 50% of these deaths were due to pneumonia [3].

In the western hemisphere, respiratory infections are more frequent during the cold months. Risk factors for acute respiratory infection include age, family history of frequent upper respiratory tract infections, asthma, allergies, heart disease or other lung problems, poor personal hygiene, active and passive smoke, and failure to comply with the vaccination program. [4]. However, anyone whose immune system might be weakened by another disease is at risk.

Preschool children may have up to 6-10 viral colds a year [5] due to the fact that humoral and phagocytic immunity reach their highest efficacy during the fifth or sixth years of age [6, 7], and also due to their behaviour and close contact with other children who might be virus carriers.

ARIs can be distinguished as upper respiratory tract infections (URIs), which mainly affect the nose and throat, and lower respiratory tract infections (LRIs), which affect the trachea, lower airways and lungs. Viral infections of the upper respiratory tract can occur at any age and include the common cold and flu; those of lower respiratory tract are more common among children and include croup, bronchitis and pneumonia [8].

In 2016, ARIs were the second most frequent cause of death, after prematurity, among children aged 1-5 years [9]. Data collected from the Global Burden of Disease Study 2016 showed that, immediately after pneumococcal pneumonia, respiratory syncytial virus (RSV) was the second leading cause of lower respiratory infection morbidity and mortality globally. Fifty-four percent of lower respiratory infection deaths attributable to RSV occurred in children younger than 5 years, similar to those attributable to Haemophilus influenzae type B [10].

Among viruses, RSV is also the leading cause of hospitalization in children under 15 years old, followed by adenovirus (ADV), rhinovirus (RV) and influenza A virus [11]. Since ARI symptoms such as cold, sore throat, and fever are common to different pathogens and opportunistic microorganisms (including fungi, viruses and bacteria), it is important to make use of a tool that allows the detection of many pathogens at the same time.

Guidelines for the management of community-acquired pneumonia in children indicate the need for molecular methodologies that allow a rapid and accurate diagnosis and a quick distinction between bacterial and viral respiratory infections, avoiding the prescription of inappropriate antibiotic therapy.

In the last decade, several commercial assays for the simultaneous detection of respiratory pathogens have been developed. These assays substantially vary in formulation and performance characteristics.

The aim of this study was to compare the performance of the Seegene workflow, with the Allplex^TM Respiratory Panel Assays (CE/IVD marked), and Siemens workflow, with Fast Track Diagnostics Respiratory pathogens 21 (CE/IVD marked) for diagnosis of pediatric respiratory viral infections.

Materials and methods

During the fall-winter 2017-2018 season, nasopharyngeal washes from pediatric patients admitted to the Bambino Gesù Children’s Hospital in Rome for suspected viral respiratory infection were collected and immediately processed for molecular virus detection using Allplex^TM Respiratory Panel Assays on an All-in-One Platform (Seegene, Korea). Residual washes were stored at -80° C. For the purpose of this comparative study, 145 stored samples were retrospectively selected and analyzed by Fast Track Diagnostics Respiratory pathogens 21 (FTD21) assay on a Versant kPCR Molecular System (Siemens Healthineers GmbH, Germany). One hundred samples with negative results for influenza virus and 45 samples from influenza-virus-positive washes were chosen randomly. The FTD21 assay was performed at the University Hospital of Campus Bio-Medico of Rome within one month of collection. Both processing techniques were carried out according to the manufacturer’s instructions. An internal control was included in each sample to determine the extraction efficiency and the degree of PCR inhibition; a negative control was included in every run to monitor carry-over contamination in both workflows.

Seegene workflow

Nucleic acid was extracted using a STARMag Universal Cartridge Kit (Seegene) on an automated Nimbus IV platform, which can process 30 samples per run. As recommended by the manufacturer, a total of 200 µl of each sample was extracted and eluted with 100 µl of elution buffer.

Real-time PCR was performed on a CFX96 Instrument (Bio-Rad Laboratories) with Allplex^TM Respiratory Panel Assays. The panel is made up of three mixes that allow the identification of 16 different viruses: influenza A virus (H1 and H3), distinguishing 3 subtypes (H1N1, H1N1-pdm09 and H3N2), influenza B virus, respiratory syncytial viruses A and B (RSV A/B), adenovirus (ADV), enterovirus (EV), parainfluenza viruses 1-4 (PIV1-4), metapneumovirus (MPV), bocavirus (BocV), rhinovirus (RV) and three coronaviruses (CoV NL63/229E/OC43). For each reaction, 8 µl of the extracted DNA/RNA, in a final volume of 25 µl, was used. The results were analyzed automatically using Seegene software (Seegene Viewer V2.0), and the whole process took 210 minutes.

According to the manufacturer, the analytical sensitivity is 100 copies/reaction (limit of detection) for the majority of the viruses. A lower sensitivity was observed for MPV (1000 copies/reaction). The analytical specificity was between 99 and 100% for all pathogens.

Following the datasheet indications for interpretation of results, samples with a cycle threshold (Ct) ≤ 42 were considered positive, and samples with no Ct or a Ct > 42 were considered negative.

Siemens workflow

Nucleic acid was extracted using the VERSANT Sample Preparation 1.0 on Molecular Diagnostics SP module Siemens automatic workflow (compatible with the validation system NucliSENS easyMAG - bioMérieux), which can process 17 samples per run. Five hundred µl of sample was extracted, and nucleic acid was eluted with 100 µl of elution buffer. Real-time PCR was performed using 10 µl of DNA/RNA in a final volume of 25 µl. The reaction was carried out on a QuantStudio 5DX Real-Time PCR System (Applied Biosystems) using the FTD21 assay. The panel is composed of five multiplex PCR tubes for the detection of 18 viruses: influenza A virus (H1 and H3), distinguishing only H1 subtype (H1N1 and H1N1-pdm09), influenza B virus, RSV A/B viruses, ADV, EV, parechovirus (PeV), PIV1-4, MPV, BocV, RV and four coronaviruses: CoVNL63, CoV229E, CoVOC43 and CoVHKU1. The whole process took 180 minutes.

According to the manufacturer, the analytical sensitivity is about 10⁴ genome copies/mL (limit of detection from probit analysis) for the majority of the viruses. Lower sensitivity was observed for EV and ADV (10⁵ genome copies/mL), while higher sensitivity was observed for influenza B virus (10² genome copies/mL), RSV and PIV1 (10³ genome copies/mL). The analytical specificity was between 99 and 100% for all pathogens.

Following the datasheet indications for interpretation of results, samples with a Ct ≤ 35 were considered positive, samples with a Ct between 35 and 37 were considered weakly positive only in the presence of a clear amplification curve, and samples with no Ct or a Ct > 37 were considered negative.

Confirmatory test

Discordant results for both monomicrobic (only one pathogen) and polymicrobic (more than one pathogen) infections were further analyzed by single Real-Time PCR CE/IVD marked (Rhino&EV/Cc R-GENE® and Influenza A/B R-GENE® - ARGENE bioMérieux).

Coronaviruses were subjected to sequencing of the spike gene region as described by Soonnarong et al. [12] to confirm subtyping when discordant results were obtained.

Statistical analysis

Cohen’s kappa coefficient and the confidence interval (CI) were calculated to evaluate the agreement of the two assays.

Definitions

Coinfection is defined as the presence of ≥ 2 viruses in the same sample.

Main pathogen, in case of coinfection, is defined as the pathogen present with the higher DNA/RNA load, as indicated by a lower cycle threshold.

Results

Patient populations and distribution of respiratory infection

We analysed 145 samples from 145 patients, screened at the admission to the Bambino Gesù Children’s Hospital by Seegene workflow. Seventeen out of 145 samples were negative for all tested pathogens, and seven of these (41%) were from immunocompromised patients, while ten (59%) were from immunocompetent patients. Respiratory pathogens were detected in 128 out of 145 samples. Seventy-three of these (57%) were from immunocompetent patients, while 55 (43%) were from immunocompromised patients. Concerning immunocompetent subjects, 24 out of 73 (33%) had been hospitalized for bronchiolitis, 32 out of 73 (44%) for acute respiratory syndrome different from bronchiolitis (other ARIs), and 17 out of 73 for non-respiratory pathology (non-ARIs), with a median length of stay of 5 (2-40) days.

Immunocompromised individuals included onco-hematologic, hematopoietic-stem-cell- or solid-organ-transplanted, and chronic lung disease patients. In these patients, the reasons for hospitalization were non-ARIs, respiratory infections, and intercurrent diseases (Table 1).

Table 1.

Distribution of 145 patients according to the presence of respiratory pathogens and immunological status

Patient group	Respiratory pathogen		No.
Immunocompetent	Positive	Bronchiolitis	24
		Oher ARIs	32
		Non-ARIs	17
	Negative		10
Immunocompromised	Positive		55
	Negative		7

Immunocompetent and immunocompromised patients with bronchiolitis were predominantly affected by monomicrobic infections (16/24, 67%; 42/55, 76%).

Monomicrobic infections were observed as frequently as polymicrobic ones among patients with other ARIs or non-ARIs.

Comparison of Seegene workflow and Siemens workflow results

The analysis of concordance between the two methods showed that 126 out of 145 (87%) samples tested for respiratory viruses proved to be concordant: 109 tested positive and 17 negative. Among the positive concordant samples, 81 were monomicrobic (100% concordance) and 28 polymicrobic, with all pathogens involved in coinfection, defined as the presence of ≥ 2 viruses in the same sample, correctly identified by both methods. Of the 19 (13%) discordant samples, 15 (79%) were polymicrobic and in each case concordant for the main pathogens. In these 15 samples, a total of 39 pathogens were found: 21 of these were detected by both systems (Ct < 37) and 18 were detected by only one assay. In particular, in the Siemens workflow, 14 out of 18 of these co-pathogens (5 of which were MPV) were detected with a Ct range of 35-37, while the Seegene workflow detected 4 out of 18 co-pathogens with Ct values between 37 and 40. Of the other four discordant samples, two tested negative by Seegene workflow but positive by Siemens workflow: one was positive for influenza A virus (H1N1-pdm09), and the other for RV, both with low viral load. The latter results were confirmed using a specific real-time PCR assay. Notably, the clinical history of the patients indicated a previous respiratory infection that could explain the presence of residual virus. Finally, 2 out of 19 samples were discordant for the CoV type. Specifically, the Seegene workflow identified CoVOC43, while the Siemens workflow identified CoVHKU1. Sequencing confirmed the presence of the CoVHKU1 subtype. Moreover, the Siemens workflow identified six PeVs, a target not available in the Seegene workflow.

In conclusion, the Siemens workflow correctly identified 128 out of 128 positive samples versus 126 out of 128 identified by the Seegene workflow.

Both methods revealed a high concordance for positive and negative samples. K agreement was between 0.89 and 1 for all targets except for MPV and PIV (Table 2).

Table 2.

Percentage of concordance and Cohen’s kappa value of viral targets

Target	Siemens/Seegene workflow	% concordance	Cohen’s kappa value
Adenovirus	7/7	100	1
Bocavirus	10/9	99	0.94
Coronavirus	9/7	99	0.87
Enterovirus	3/3	100	1
Influenza A virus (H1N1-pdm09)	34/33	99	0.98
Influenza B virus	19/18	99	0.97
Metapneumovirus	17/13	97	0.85
Rhinovirus	42/41	99	0.97
Respiratory syncytial virus	39/39	100	1
Parainfluenza virus	3/2	99	0.80
Negative	19/17	99	0.97

Sensitivity and specificity were calculated with both tests as a reference method. For Allplex^TM Respiratory Panel Assays, they were 98% and 100%, respectively, and for Fast Track Diagnostics Respiratory pathogens 21, they were both 100%.

Discussion

ARIs are a significant public health problem because they are widespread and are associated with direct and indirect costs. It is well known that viruses are the main cause of ARIs in young children [13–15]. A genetic predisposition or the exposure to environmental factors may promote the entry of respiratory viruses and bacteria, allowing the establishment of a serious infection of the airways. Moreover, recent studies have shown that respiratory multi-viral infections are associated with longer paediatric intensive care unit stays and negative clinical outcomes [16].

Since the symptoms associated with ARIs are common in viral, bacterial and fungal infections, multi-target tests are particularly useful for a differential diagnosis. Rapid etiological identification supports the correct management and treatment of patients, allowing, where necessary, prompt isolation. Multiplex molecular assays respond to this need by identifying more than 80% of lower respiratory infections in hospitalized children [17]. For these reasons, the Food and Drug Administration has approved several rapid multiplex panels, which show high sensitivity and specificity [18, 19]. In our study, we compared two of them: the Allplex^TM Respiratory Panel Assays on the Seegene All-In-One Platform and FTD21, recently automated on the Siemens Versant kPCR Molecular System, for the detection of viral respiratory pathogens.

Our results clearly confirmed that both the Seegene and Siemens workflows performed well for routine detection of most respiratory viruses with high sensitivity and specificity. High concordance between the two assays was also observed. Discordance between platforms was mainly detected with samples containing multiple pathogens, some of which were present in low amounts. This was primarily associated with the detection of metapneumovirus, as previously reported by other authors [20]. Though the main pathogens were always correctly identified by both assays, the Siemens workflow performed better in detecting low viral loads than the Seegene workflow. However, the low levels of respiratory viruses detected, especially in the presence of pathogens with a low Ct value (high DNA/RNA load), are likely to be related to underlying or residual infection. Thus the clinical meaning of these results is still open to discussion.

For CoV detection, the Seegene system does not include CoVHKU1 as a target, but as it has been demonstrated that CovHKU1 and OC43 cluster in the same branch (betacoronavirus subgroup A) of the CoV phylogenetic tree [21], it is possible that the OC43 primers bind to gene region that is conserved in both types, thereby causing a spurious pairing that recognizes HKU1 as OC43.

Our data contrast with those reported by Barratt et al. [22], who reported high specificity of the Seegene assay for CoVOC43.

As both workflows performed well for routine detection of the most important respiratory viruses in paediatric patients, other factors should be considered when choosing the assay to be used for the diagnostic routine. The Seegene workflow is able to subtype RSV A/B and influenza A virus (H1N1-pdm09/H1N1/H3) due to the simultaneous detection of more gene targets that are used in the Siemens procedure, which subtypes only influenza A virus (H1N1). On the other hand, the latter is the only test able to detect PeV and subtype CoVHKU1. Both assays have a bacterial panel that can be integrated; in this study, we compared only viral panels, according to clinical request (suspicion of respiratory virus infections).

Another aspect to consider when choosing a diagnostic multiplex platform is the multiplexing capability per run. Seegene can process up to 30 samples in each run, so it is also suitable for heavy routines, while Siemens achieves 17 samples/run.

Finally, both platforms utilize software to perform automatic data recording and interpretation, leading to the same hands-on time and simplified analysis of results if performed by expert technicians.

Currently, several single-use cartridge assays are available for diagnosis of multiple respiratory infections. These systems are completely automated and provide faster results (about 1 h), but they are difficult to implement in a standard routine because they are very expensive.

In conclusion, the use of syndromic panels, such as those produced by Seegene and Siemens, both CE/IVD, is to be preferred in standard laboratory settings, particularly in paediatric ones. The choice of the assay used should be based on the diagnostic routine and laboratory workflow, keeping in mind the epidemiology of the target population.

Compliance with ethical standards

Ethical statement

This study does not require ethical committee approval because the data were properly anonymised and de-identified prior to analysis and informed consent was obtained at the time of collection of the original data and samples.