Sensitivity and specificity of in vitro diagnostic device used for influenza rapid test in Taiwan

Abstract

The pandemic influenza A/H1N1 outbreak resulted in 18,449 deaths in over 214 countries. In Taiwan, the influenza rapid test, an in vitro diagnostic device (Flu-IVD), only requires documented reviews for market approval by the Taiwan Food and Drug Administration. The purpose of this study was to investigate the analytical sensitivity and specificity of Flu-IVDs used in Taiwan. Analytical sensitivity and specificity tests were performed for influenza antigens A/California/7/2009 (H1N1) virus, A/Victoria/210/2009 (H3N2) virus, B/ Brisbane/60/08 virus, and human coronavirus OC43. A total of seven domestic and 31 imported Flu-IVD samples were collected, of which, 20 samples had inadequate labeling, including those with removed package inserts or incorrect insert information. The analytical sensitivity of Flu-IVDs for A/H1N1, A/H3N2, and Flu B was 500–1000 ng/mL, 1000 ng/mL, and 1000 ng/mL, respectively. For the 50% cell culture infective dose (CCID₅₀) label, the average A/H1N1 and A/H3N2 sensitivity for Flu-IVDs was log₁₀ 5.8 ± 0.5 and log₁₀ 6.6 ± 0.5 CCID₅₀/mL, respectively. As to the specificity test, no product cross-reacted with human coronavirus OC43. This study provides important information on the Flu-IVD regulation status and can thus help the government formulate policies for the regulation of in vitro diagnostic devices in Taiwan.

1. Introduction

In 1998, a triple-reassortant swine influenza virus from human, swine, and avian genomes was identified in the USA [1]. Ten years later, humans were found to be infected with these influenza viruses [2]. A novel human influenza A (H1N1) virus caused a worldwide respiratory disease outbreak in April 2009. The major transmission route for influenza virus is via respiratory droplets released while coughing [3–5]. Influenza virus replicates in the epithelial cells of the upper respiratory tract, which destroys the host cells, further spreading to infect other cells. The clinical manifestations of influenza are fever, headache, myalgia, fatigue, rhinorrhea, sore throat, and cough. The impact of influenza outbreaks is much higher in children than in adults. Most healthy adults will recover after 3–7 days [6–8]. Influenza A/H3N2 was the predominant serotype during the 2010–2011 season, whereas the A/H1N1 2009 virus still cocirculated with the A/H3N2 and influenza B strains [9]. The pandemic influenza A/H1N1 outbreak resulted in over 18,449 deaths in over 214 countries until 2012 [10,11].

Conventional diagnostic testing of influenza virus included virus incubation and nucleic acid amplification test. Virus incubation is the first method used to diagnose virus infections, while it takes about 7 days to amplify the virus in the host cells. A nucleic acid amplification test is the most sensitive method to detect virus infection, but pretreatment of the sample, including virus isolation and nucleic extraction, still requires about 1 day. The above methods are not suitable for the rapid screening of influenza virus [12]. In vitro diagnostic devices (IVDs) play crucial roles in disease diagnosis because of their rapid and convenient properties. Diagnostic accuracy of IVDs is dependent on their sensitivity and specificity. A previous study evaluated the 3M rapid detection test for the respiratory syncytial virus, and found the sensitivity and specificity of the device to be 74% and 100%, respectively [13]. In April 2011, a study conducted by Iregbu et al [14] on the performance of Dual Path Platform testing kits revealed a sensitivity of 100% and specificity of 99.5%. By contrast, a study by Shrivastava et al [15] reported that the commercial Dengue NS1 antigen kit had a sensitivity of 26% against 91 clinical samples. Few studies have reported the sensitivity of influenza IVDs (Flu-IVDs) in detecting influenza A. These previous studies used four commercialized rapid test kits [16–20].

Flu-IVD rapid test kits are based on immunochromato-graphic tests. This type of IVD is designed for easy and fast operation, which enables physicians to obtain results at the primary influenza screening sites. The Flu-IVD rapid test kits have been classified into a low-risk class I device according to the medical device regulation and risk assessment in Taiwan. They are applied and approved by the Taiwan Food and Drug Administration (FDA) for pre-market approvals via the fast tract pathway, that is, they are exempt from the substantial performance review.

In this study, we collected Taiwan-FDA-approved Flu-IVDs from the Taiwanese market and evaluated their performance. The standard antigens prepared from viral cultures and purified hemagglutinins (HAs) were used to evaluate the analytical sensitivity and specificity of the Flu-IVDs. The results represent the actual responses of these rapid test kits and provided references for future amendments in regulations.

2. Methods

2.1. Analytical standard

The international standards for HA and influenza virus were used as the analytical standards for the Flu-IVDs sensitivity test. For HA standards, the Influenza Antigen A/California/7/ 2009 (H1N1, NYMCX-179A, Egg derived, NIBSC code: 09/146), A/ Victoria/210/2009 (H3N2, NYMCX-187, NIBSC code: 10/102), and Influenza B/Brisbane/60/08 Antigen Reagent (NIBSC code: 08/352) were obtained from the National Institute for Biological Standards and Control (Potters Bar, Herts, UK). The influenza A/Taiwan/9042/2008 (H1N1) virus with 108.23 50% cell culture infective doses (CCID₅₀)/mL and A/Taiwan/439/ 2009 (H3N2) virus with 106.3 CCID₅₀/mL from Chang Geng University were used as influenza viral particle standards. The human coronavirus OC43 (HCoV-OC43, ATCC VR-759) was obtained from American Type Culture Collection (Manassas, VA, USA).

2.2. Specimen collection

Flu-IVDs were purchased directly from the manufacturers and local distributors in 1–3 different lots. The product name, lot number, and manufacturing country of each Flu-IVD are listed in Tables 1 and 2.

Table 1.

List of domestic influenza in vitro diagnostic devices collected in Taiwan.

Product name	Lot no.	Manufacturing country
Firstep Influenza A&B Test Card	10072609	Taiwan
Formosa One Sure Flu A/B Rapid Test Kit	MS19A21	Taiwan
	MS19A71
	MS19A81
Long TERM Influenza A&B	990510-06	Taiwan
	990629-01
	990928-07

Table 2.

List of imported influenza in vitro diagnostic devices collected in Taiwan.

Product name	Lot no.	Manufacturing country
DIAQUICK influenza Ag Dipstick	1301/069039-U	Austria
BD Directigen EZ Flu A + B	9341551	China
Feng Chi Kaibilia Influenza A + Influenza B nucleoprotein antigen Rapid Test Kit	AB091102	China
	AB100501
	AB100502
On-call Influenza A&B Rapid Test Strip	FLU00880002	China
Actim Influenza A&B Test Kit	0021825	Finland
	0021888
	0081929
“Jolex” BioTracer Influenza A&B	11321910	Korea
	11321911
	11321912
Bioland NanoSign Influenza A/B Antigen	INF101109	Korea
“SD” Influenza Ag	069041	Korea
	069042
	069043
EZ-TRUST Influenza A&B Rapid Screen Test	W7100401	Singapore
	W7100402
	W7100403
Linear Influenza A + B rapid test Kit	Z-031	Spain
	Z-036
	Z-037
VEGAL Influenza A + B	Z031	Spain
Binax NOW Influenza A&B	43842	USA
Genzyme Influenza A&B Test	091511A	USA
Meridian TRU FLU Test	731230.075	USA
	731230.082
Applied Biotech Influenza A + B test	911067	USA
	912150
	1008156
Unipath Clearview Exact Influenza A&B	FLU9090087	UK

2.3. Appearances and label survey

The appearance of the products and the labels on the Flu-IVDs were compared with information on their licenses, including Chinese and English names, license numbers, name of manufacturer, address of manufacturer, and expiration dates.

2.4. Sensitivity analysis

Sensitivity was assayed as described previously, with modifications [21]. The sensitivity of the collected Flu-IVDs was specified as HA concentrations or CCID₅₀. The HA standards consisting of Influenza Antigens A/California/7/2009 (H1N1), A/Victoria/210/2009 (H3N2) and B/Brisbane/60/08 were diluted individually to 0.25–50,000 ng/mL, 0.25–13,000 ng/mL, and 0.25–13,500 ng/mL, respectively, with the dilution buffer provided in the Flu-IVDs. Additionally, the influenza A/ Taiwan/9042/2008 (H1N1) and A/Taiwan/439/2009 (H3N2) viruses were prepared from infected rhabdomyosarcoma (RD) cells and diluted to concentrations of 101.3–105.3 CCID₅₀/mL and 101.23–107.23 CCID₅₀/mL, respectively, with minimum essential medium (Gibco, Grand Island, NY, USA). The sensitivity assay of each Flu-IVD, including the positive and negative controls, was carried out by following the instructions in the operation manuals. Two independent and well-trained technicians performed the experiments.

2.5. Specificity analysis

HCoV-OC43 was prepared from infected MRC-5 cells. The extract containing 1 × 10⁷ CCID₅₀/mL of virus was used to evaluate the specificity of Flu-IVDs. The experimental protocol was carried out in accordance with the operation manuals of each Flu-IVDs. Two independent and well-training researchers carried out the experiments.

3. Results and discussion

Flu-IVDs are classified as class 1 medical devices, indicating that the license could be easily obtained through document reviews. Therefore, the quality of Flu-IVDs needs to be investigated and monitored. In Taiwan, suspected influenza patients are immediately tested with Flu-IVDs at the primary screening site in epidemics areas. The therapeutic strategies are divided into two parts based on the test results. If the test result is positive and the patient exhibits influenza-like symptoms, the patient is immediately administered antiviral drugs such as osletamivir (Tamiflu) and continuously monitored with Flu PCR test kits. However, negative test results indicate that the patient only has a common cold, and no antiviral drug treatment is given. The use of Flu-IVDs plays a crucial role in controlling influenza epidemics. Hence, false-positive outcomes may cause inappropriate administration of osletamivir, which could lead to future challenges in the prevention of influenza. Prior to 2009, several commercial Flu-IVD rapid test kits were approved in Taiwan. However, after the influenza A/H1N1 pandemics in 2009, the Taiwan FDA issued many licenses for similar products. We thus aimed to examine the analytical sensitivity and specificity of Flu A-IVDs used in Taiwan.

3.1. Specimen collection

A total of 38 products with 19 Flu-IVD licenses were collected, of which seven were domestic and 31 were imported Flu-IVDs. Product labels of 20 samples did not comply with the regulations in Taiwan FDA. For example, those with unsatisfactory product labels lacked a product name and/or license number on the packaging or buffer containers. Some products had no package insert in them, and some package inserts did not indicate the sensitivity of the product. The percentage of unsatisfactory product labels in domestic and imported kits was 43% (3/7) and 55% (17/31), respectively.

3.2. Analytical sensitivity

Based on the reaction principles, the detecting targets of the collected samples were grouped as HA and viral particles. The Flu-IVDs belonging to the HA group were evaluated for their analytical sensitivity with the World Health Organization (WHO) international standard Influenza Antigens A/Califor-nia/7/2009 (H1N1), A/Victoria/210/2009 (H3N2), and B/Bris-bane/60/08. As shown in Table 3, seven products of three Flu-IVD licenses (A–C) that were manufactured in the USA and Spain used HA concentrations as their sensitivity label. The analytical sensitivity of these Flu-IVDs against A/H1N1, A/ H3N2, and Flu B was 500–1000 ng/mL, 1000 ng/mL, and 1000 ng/mL, respectively. For A/H1N1 detection, products A and B displayed higher analytical sensitivity than product C. The sensitivity results for A/H3N2 and Flu B were identical in these samples. We obtained seven products from three domestic Flu-IVDs (D–F) and 17 products from eight Flu-IVDs (G–N) imported from China, the UK, Finland, Singapore, USA, and Korea with CCID₅₀ labels. In the past 5 years, type A/ Taiwan/439/2009 (H3N2) and type A/Taiwan/9042/2008 (H1N1) were the two main strains of influenza outbreak in Taiwan. Hence, these two virus strains were used as influenza viral particle standards for CCID₅₀ test [22]. The averages of A/H1N1 and A/H3N2 sensitivity were log₁₀ 5.8 ± 0.5 CCID₅₀/mL and log₁₀ 6.6 ± 0.5 CCID₅₀/mL, respectively. In Fig. 1, the unbroken, broken, and dotted lines represent the mean, mean ± 1 standard deviation (SD) and mean ±2 SD in the sensitivity tests, respectively. For A/H1N1, the sensitivity of all products was within mean ± 1 SD (Fig. 1A). As to A/H3N2 sensitivity, however, the products E, I, L, and M fell into the mean ± 2 SD range (Fig. 1B). Seven products from five imported Flu-IVDs did not have test sensitivity labeling. Therefore, HA concentrations were used to determine their sensitivities. In Table 4, the sensitivities of these seven samples against H1N1, H3N2, and Flu B were 500–1000 ng/mL, 500–3250 ng/mL, and 1000–6750 ng/mL, respectively.

Table 3.

Evaluation of sensitivity of Flu-IVDs for hemagglutinin concentrations.

Source	Standard type	Product codea	Sensitivity results (ng/mL)
			H1N1		H3N2
Import	Flu A	A	500		1000
		B	500		1000
		C	1000		1000
	Flu B	A		1000
		B		1000
		C		1000

^aSeven products of three Flu-IVD licenses (code A–C) were analyzed.

Fig. 1.

Evaluation of the sensitivity of Flu-IVDs for H1N1 (A) and H3N2 (B) in 50% cell culture infective dose. Unbroken (–), broken (——), and dotted lines (......) indicate the mean, mean ± SD values, and mean ± 2 SD values, respectively. Product codes are designated by a letter from D to N. Seven products of three domestic Flu-IVD licenses (code D–F) and 17 products of eight imported Flu-IVD licenses (from code G–N) were analyzed. Flu-IVD = influenza in vitro diagnostic device; SD = standard deviation.

Table 4.

Unlabeled sensitivity of Flu-IVDs.

Source	Standard type	Product codea	Results (ng/mL)
			H1N1		H3N2
Import	Flu A	O	500		500
		P	1000		6500
		Q	500		3250
		R	500		500
		S	500		500
	Flu B	O		1000
		P		1000
		Q		ND
		R		1000
		S		6750

ND = not detected.

^aSeven products of five Flu-IVD licenses (code O–S) were analyzed.

Previous researchers have used clinical specimens to monitor the clinical performance of Flu-IVDs. A study by Drexler et al [17] found that BinaxNow Influenza A&B Rapid Test Kits had a poor clinical sensitivity; the sensitivity was 11.1% for the pandemic (H1N1) 2009 virus (16 positive cases against 144 polymerase-chain-reaction-confirmed cases). A study by Uyeki et al [20] also showed that the QuickVue Influenza A + B Test had a low sensitivity (27%) for the detection of both influenza A and B viruses. The Center for Disease Control and Prevention screened 45 specimens with positive results for novel influenza A (H1N1) using Directigen EZ Flu A + B and obtained a sensitivity of 49% [23].

We assumed that one of the reasons for the discrepancies was that the human epidemic strains of influenza viruses used in the Flu-IVDs did not match the current influenza virus outbreak strains. Pandemic influenza virus strains varied from year to year and from country to country. For example, the influenza A/Texas/1/77 strain was used as the master influenza virus strain to develop the monoclonal antibodies in the BinaxNow Influenza A&B Rapid Test; however, the major virus strain that caused the influenza outbreak in Taiwan was the type A/Taiwan/439/2009 (H3N2) and type A/Taiwan/9042/ 2008 (H1N1). Other Flu-IVDs failed to mention the master influenza virus strain used in their preparations. Therefore, inappropriate master influenza virus strains used in the Flu-IVD preparations may contribute to the low detection sensitivity. Moreover, studies have shown that analytical sensitivity did not directly reflect the clinical sensitivity on patient specimens. The analytical sensitivity of influenza H5N1 detection and seasonal influenza was the same, although their clinical performances were poor [23,24]. Manufacturers should follow WHO recommendations for the annually pandemic influenza virus strains to be used in their preparation of the Flu-IVDs in order to improve the quality. We also suggest that the class level for Flu-IVDs in Taiwan for government regulations be elevated so that the government can monitor the quality of IVDs using a lot release system. Furthermore, Taiwan FDA should establish a reference or proficiency panel to evaluate in vitro Flu-IVDs in the near future.

3.3. Analytical specificity

Patients suffering from respiratory illness such as cough, asthma, and emphysema are usually diagnosed with acute respiratory tract infections. Viruses such as influenza virus, coronavirus, and respiratory syncytial virus commonly cause respiratory tract infections [25]. Several studies have conducted a specificity test by real-time polymerase chain reaction analysis for different respiratory-tract-infection-related viruses [26,27]. The specificity of Flu-IVDs played an important role in the selective interaction of influenza A and B viruses. According to the epidemiological studies, HCoV caused >15% of common colds in adults, of which OC43 was the most common [28]. Hence, we used the common HCoV-OC43 to determine the specificity of Flu-IVDs in this study and the result showed that not all Flu-IVDs cross-reacted with HCoV-OC43, suggesting good specificity (Table 5).

Table 5.

Specificity results of all influenza in vitro diagnostic devices.

Test virus	Cross-reactivity	Specificity rate
Human coronavirs-OC43	0/38	100%

In conclusion, seven domestic and 31 imported Flu-IVDs were collected to analyze the sensitivity and specificity. Analytical sensitivity of Flu-IVDs for A/H1N1, A/H3N2, and Flu B was 500–1000 ng/mL, 1000 ng/mL, and 1000 ng/mL, respectively. Average sensitivity of A/H1N1 and A/H3N2 for Flu-IVDs was log₁₀ 5.8 × 0.5 CCID₅₀/mL and log₁₀ 6.6 ± 0.5 CCID₅₀/mL, respectively. No products cross-reacted with HCoV-OC43. Our results could provide more information for future policy-making strategies and preparedness against seasonal and pandemic influenza.

Funding Statement

This work was supported by grants from the Department of Health, Executive Yuan, Taiwan (grant no. DOH99-FDA-72020).