Abstract
Three armored RNAs (virus-like particles [VLPs]) containing target sequences from enterovirus 71 (EV71) and coxsackievirus A16 (CA16) and a pan-enterovirus (pan-EV) sequence were constructed and used in an external quality assessment (EQA) to determine the performance of laboratories in the detection of EV71 and CA16. The EQA panel, which consisted of 20 samples, including 14 positive samples with different concentrations of EV and either EV71 or CA16 armored RNAs, 2 samples with all 3 armored RNAs, and 4 negative-control samples (NaN3-preserved minimal essential medium [MEM] without VLPs), was distributed to 54 laboratories that perform molecular diagnosis of hand, foot, and mouth disease (HFMD) virus infections. A total of 41 data sets from 41 participants were returned; 5 (12.2%) were generated using conventional in-house reverse transcription-PCR (RT-PCR) assays, and 36 (87.8%) were generated using commercial real-time RT-PCR assays. Performance assessments of laboratories differed; 12 (29.3%) showed a need for improvement. Surprisingly, 4 laboratories were unable to detect EV71 RNA in any samples, even those containing the highest concentration of 107 IU/ml. Furthermore, the detection sensitivity for EV71 among all laboratories (82.1%) was substantially lower than that for EV (97.4%) or CA16 (95.1%). Overall, the results of the present study indicate that EQA should be performed periodically to help laboratories monitor their ability to detect HFMD viruses and to improve the comparability of results from different laboratories.
INTRODUCTION
Hand, foot, and mouth disease (HFMD) is an infectious disease commonly diagnosed in young children and characterized by mucocutaneous papulovesicular rashes on the hands, feet, mouth, and buttocks. Human enterovirus 71 (EV71) and coxsackievirus A16 (CA16) are the principal causative agents of HFMD. EV71 is of special concern, because it is more often associated with major outbreaks and causes complications of greater severity and higher rates of mortality than other enteroviruses. In contrast, CA16-associated HFMD has a milder outcome and is accompanied by a much lower incidence of severe complications (7, 16, 21).
HFMD is an important public health concern worldwide (20). Since 1997, several large epidemics of HFMD have been reported in the Asia-Pacific region, especially in Southeast Asia, including Malaysia (2), Taiwan (24), and Singapore (5). In March 2008, an EV71-associated HFMD outbreak occurred in Fuyang City, Anhui Province, China, and subsequently spread rapidly and extensively across the entire country; a total of 488,955 HFMD cases, 126 of which were fatal, were reported nationwide in 2008 (28).
As no vaccine or antiviral drug is currently available, early and rapid detection is critical for HFMD prevention and control (30). At present, the causative agents of HFMD can be effectively diagnosed by the detection of infectious virus, viral antigens, viral genomic RNA, or antiviral antibodies (11, 17, 25, 27). Due to their speed, sensitivity, and specificity, reverse transcription-PCR (RT-PCR)-based molecular diagnostic assays are increasingly used to detect EV71 and CA16 RNA for HFMD diagnosis (8, 14, 15). Various commercial and in-house assays are currently available. Due to the limited availability of well-characterized reference reagents, most HFMD detection assays lack critical standardization of performance, which causes difficulties in comparing results between laboratories and consequently complicates the clinical interpretation of laboratory results for HFMD.
To date, many biological materials, including live virus, inactivated virus, and naked RNA, have been used as controls for the molecular diagnosis of RNA viruses (1, 19). However, each of these materials has some inherent disadvantages, such as the biosecurity risk of live virus, the risk of residual activity of inactivated virus, and the inability of clinicians to evaluate sample processing and RNA extraction and the relative instability of naked RNA. A potential alternative is armored RNA, a noninfectious and quantifiable synthetic substitute for live or inactivated RNA virus that can be spiked into clinical specimens without risking degradation, thus enabling simultaneous monitoring of nucleic acid extraction efficiency and the amplification process of the detection assay (3, 12, 18).
In this study, we constructed 3 types of armored RNAs carrying genomic sequence fragments from EV71 and CA16: one RNA specific for EV71, one RNA specific for CA16, and one RNA designed to detect all EV serotypes. Moreover, a pioneering national external quality assessment (EQA) study was organized within the nationwide HFMD reference laboratory network to evaluate the accuracy of EV71 and CA16 PCR assays used in laboratories in China.
MATERIALS AND METHODS
Preparation of armored RNAs.
Three gene fragments were amplified by RT-PCR using EV71 and CA16 RNA as templates. Both EV71 RNA and CA16 RNA were kindly provided by the Beijing Center for Disease Prevention and Control (CDC). The EV71 5′ untranslated region (5′UTR) packaged into virus-like particles (EV-VLPs) is a pan-enterovirus sequence that is conserved among all enteroviruses. The primers used in this study are described in Table S1 in the supplemental material. As described in previous studies (13, 26), inclusion of a hepatitis C virus (HCV) 5′UTR in the chimeric armored RNA sequence allows the ready assignment of an international unit value to the chimeric armored RNA for quantitative detection. The HCV 5′UTR sequence was amplified from a pNCCL-HCV plasmid (constructed by our laboratory) and ligated to the EV 5′UTR by overlapping-extension PCR. Therefore, 3 armored RNAs, AR-HCV-EV(UTR)-994b (EV-VLP), AR-EV71(VP)-2557b (EV71-VLP), and AR-CA16(VP)-2601b (CA16-VLP), were generated by the conventional armored RNA technique (18) and further optimized as previously described in a report from our laboratory (29). The three armored RNAs were confirmed to contain the target sites for all PCR assays evaluated in this study.
EV-VLP armored RNA was calibrated using the WHO HCV RNA International Standard (version 3.0 [http://www.nibsc.ac.uk/documents/ifu/06-100.pdf]), and the other 2 armored RNAs (EV71-VLP and CA16-VLP) were quantified using calibrated EV-VLP as a standard. Here, the armored RNA (EV-VLP) content data are expressed in international units per milliliter based on the WHO HCV RNA International Standard (22).
Temperature and time stability assessments were performed for the three armored RNAs prepared in the study. The quantified armored RNAs were diluted with NaN3-preserved minimal essential medium (MEM) to yield 103 and 105 IU/ml dilutions. For each dilution, a single batch was separated into 50 aliquots in individual time point samples of 0.5 ml. Two samples were stored at −80°C as controls. The remaining 48 samples were divided into four groups (12 samples/group) and incubated at 37°C, 25°C, 4°C, and −20°C, respectively. At each of the time points of 1 week, 2 weeks, 3 weeks, 4 weeks, 8 weeks, and 6 months, two samples of each group were removed and stored at −80°C until the completion of the experiment. All of the 50 samples were quantified in a single procedure using an enterovirus nucleic acid detection kit (real-time RT-PCR assay; Guangzhou Daan Gene Co., Ltd., Guangzhou, China).
In addition, two aliquots of the 103 IU/ml and 105 IU/ml dilutions were subjected to 5 freeze-thaw cycles and quantified using the enterovirus nucleic acid detection kit.
Performance verification of VLP in two different matrices.
Two of the VLP mixtures (EV-EV71-VLP and EV-CA16-VLP [EV-VLP mixed with EV71-VLP and with CA16-VLP, respectively, at a copy number ratio of 2:1:1]) were spiked in serial 10-fold dilutions into MEM and pooled negative throat swab fluid, respectively, at dilutions from 103 IU/ml to 106 IU/ml. Two panels that included 7 positive samples and 2 negative controls were prepared. After nucleic acid extraction using a QIAmp viral RNA minikit (Qiagen, Hilden, Germany), the two panel samples were tested for each of the aforementioned three targets in duplicate experiments in a single run by using the enterovirus nucleic acid detection kit. Finally, the cycle threshold (CT) values determined for each target in the two panels were compared by a paired t test (see Table S2 in the supplemental material).
Participants.
The laboratories that perform HFMD molecular diagnosis in China were invited to participate in the EQA study. The invitees are all members of the Chinese Laboratory Network of HFMD Diagnosis, which is composed of municipal and provincial laboratories. These laboratories receive training and evaluation from the national Chinese Centers for Disease Control (CDC) laboratory annually, take responsibility for testing and surveillance of HFMD in their respective districts, and submit the surveillance data and 10 positive strains to the national laboratory each month during periods of peak HFMD prevalence.
Proficiency panel.
The panel samples were designed and coded as indicated in Table 1. Tenfold serial dilutions of the 3 armored RNAs were made using NaN3-preserved MEM. The final panel consisted of a set of 16 positive samples and four NaN3-preserved MEM samples containing no VLPs for use as negative controls. Aliquots of 500 μl were assigned code numbers and stored at −20°C until distribution. Before distribution to the participants, the panel samples were tested using an in-house RT-PCR assay recommended by the Chinese CDC (http://www.chinacdc.cn/n272442/n272530/n3479265/n3479308/appendix/fujian1%20shouzukoubiaobencaijijijiancejishufangan.pdf) to confirm the positivity or negativity of the sample results.
Table 1.
EQA panel description and composite laboratory score for each panel member
| Sample code | Sample content (VLP)a | Concn (IU/ml)b | No. of correct results reported/total no. of results (%) |
|---|---|---|---|
| HFMD 02 | EV-EV71 | 107 | 37/41 (90.2) |
| HFMD 10 | EV-EV71-CA16 | 106 | 37/41 (90.2) |
| HFMD 14 | EV-EV71 | 105 | 36/41 (87.8) |
| HFMD 15 | EV-EV71 | 105 | 36/41 (87.8) |
| HFMD 06 | EV-EV71 | 104 | 35/41 (85.4) |
| HFMD 17 | EV-EV71 | 104 | 36/41 (87.8) |
| HFMD 08 | EV-EV71 | 103 | 28/41 (68.3) |
| HFMD 19 | EV-EV71 | 103 | 19/41 (46.3) |
| HFMD 12 | EV-CA16 | 107 | 41/41 (100) |
| HFMD 20 | EV-EV71-CA16 | 106 | 37/41 (90.2) |
| HFMD 04 | EV-CA16 | 105 | 41/41 (100) |
| HFMD 05 | EV-CA16 | 105 | 41/41 (100) |
| HFMD 07 | EV-CA16 | 104 | 40/41 (97.6) |
| HFMD 16 | EV-CA16 | 104 | 37/41 (90.2) |
| HFMD 09 | EV-CA16 | 103 | 36/41 (87.8) |
| HFMD 18 | EV-CA16 | 103 | 31/41 (75.6) |
| HFMD 01 | None | 38/41 (92.6) | |
| HFMD 03 | None | 36/41 (87.8) | |
| HFMD 11 | None | 39/41 (95.1) | |
| HFMD 13 | None | 39/41 (95.1) |
Equal copy numbers of EV-VLP were mixed with EV71-VLP or CA16-VLP as indicated, except for EV-EV71-VLP and EV-CA16-VLP, which were mixed with EV71-VLP and CA16-VLP, respectively, at a copy number ratio of 2:1:1. The EV-EV71 and EV-CA16 samples were designed as specificity controls for each other.
The target concentrations of the panel samples were selected to range from 103 to 107 IU/ml, closely matching the concentration ranges that were detected in clinical samples as reported by Chang et al. (6).
Since the armored RNA was stable at higher temperatures, samples and detailed instructions were sent by express shipping at ambient temperature (approximately 10°C to 20°C) to 54 laboratories, and it took less than 1 week for them to reach those destinations. The recipients were requested to submit the results and other information on the assay details (RT-PCR method and RNA extraction procedure) within 6 weeks of receiving the panel.
Evaluation of results.
Results were scored as optimal (20/20 [100%] correct responses), acceptable (at least 18 [≥90%] correct responses), or improvable (fewer than 18 [<90%] correct responses). Scored results were released to participants in an anonymous manner.
Statistical analysis.
Data collected were entered into a spreadsheet in Microsoft Excel (Microsoft Corp., Bellingham, WA) and analyzed using SPSS 13.0 for Windows. The rates of correct responses for EV-, EV71-, and CA16-positive samples were compared using Pearson's chi-square test, which, together with Fisher's exact test, was also used to compare the sensitivity and specificity of participating laboratory PCR assays versus those of the Chinese CDC RT-PCR assay performed in the laboratory of one of the coauthors (J.I.) of the present study. Paired t tests were performed to compare the cycle threshold values of VLPs determined using two different specimen matrices. A P value of <0.05 was considered statistically significant.
RESULTS
Construction of armored RNAs.
Armored RNAs for EV, EV71, and CA16 were constructed successfully and were confirmed by sequencing to contain the corresponding full-length target sequence of all available assays for HFMD. Quantification using the WHO HCV international RNA standard yielded a concentration of 1.5 × 1012 IU/ml. As expected, all 3 types of armored RNA exhibited strong resistance to both RNase and DNase treatment.
At a concentration of 105 IU/ml, the three armored RNAs were stable in NaN3-preserved MEM for 2 weeks at 37°C, 4 weeks at 25°C, 2 months at 4°C, and more than 6 months at −20°C. At a concentration of 103 IU/ml, reduced stabilities were observed (1 week at 37°C, 2 weeks at 25°C, 4 weeks at 4°C, and more than 2 months at −20°C). Stability was defined as a decrease of no more than 0.5 log10 IU/ml compared to the control tube concentrations stored at −80°C for the entire duration. The current stability findings are consistent with those of prior reports (3, 12, 18, 26, 29).
Performance comparison between armored RNAs in MEM and clinical sample matrices.
There were no significant differences between the two panels in the cycle threshold (CT) values determined for each of three targets (P > 0.05) (see Table S2 in the supplemental material), consistent with previous studies (4, 12).
Laboratory PCR methods and EQA performance.
A total of 54 laboratories included in the national HFMD detection laboratory network were invited to participate in this EQA study. Among them, 41 laboratories replied with their results as requested, with 1 data set from each participant. The response rate was 75.9%.
As shown in Table 2, laboratories used various methods, including five monoplex commercial real-time RT-PCR kits (PCR assays A to E), one triplex commercial real-time RT-PCR assay (PCR assay F), and one in-house conventional RT-PCR assay recommended by the Chinese CDC (http://www.chinacdc.cn/n272442/n272530/n3479265/n3479308/appendix/fujian1%20shouzukoubiaobencaijijijiancejishufangan.pdf), to detect EV71 and CA16. Of the 41 data sets, 5 (12.2%) were generated using conventional in-house assays and 36 (87.8%) were produced by commercial real-time RT-PCR assays. The performance results differed substantially among all participating laboratories and among the laboratories using the same PCR assay (Table 2). Generally, laboratories using commercial assay B and the in-house PCR performed better than laboratories using assay A or C. Laboratories using assay D performed very poorly. Four laboratories, 2 using assay A (of the total of 17 laboratories using assay A) and both of the laboratories using assay D, failed to detect EV71 in any samples. Evaluations of the cumulative data according to comparisons of assay results (Table 3) support these findings, showing that the sensitivity of assay A for detection of EV71 was 74.5% (P < 0.05) and that assay D failed to detect any samples positive for EV71. Additionally, assay D showed decreased sensitivity for CA16. Only 31.7% (13/41) of the laboratories met criteria for optimal performance; 16 (39%) of the 41 participants showed acceptable performance, and the results from 12 (29.3%) of 41 laboratories indicated a need for improvement in HFMD diagnosis.
Table 2.
Performance of 41 laboratories participating in the EQA study
| Laboratory no. | Extraction methoda | RT-PCR assayb | No. of correct samples/total no. of samples (%) | Classification |
|---|---|---|---|---|
| 32 | QIAamp | A | 0 (100) | Optimal |
| 17 | RNeasy | A | 1 (95) | Acceptable |
| 20e | QIAamp | A | 1 (95) | Acceptable |
| 21 | QIAamp | A | 1 (95) | Acceptable |
| 24d | QIAamp | A | 1 (95) | Acceptable |
| 37d | QIAamp | A | 1 (95) | Acceptable |
| 9 | QIAamp | A | 2 (90) | Acceptable |
| 13 | QIAamp | A | 2 (90) | Acceptable |
| 26d | QIAamp | A | 2 (90) | Acceptable |
| 29d | QIAamp | A | 2 (90) | Acceptable |
| 10 | TRIzol | A | 3 (85) | Improvable |
| 11e | QIAamp | A | 3 (85) | Improvable |
| 30d | QIAamp | A | 3 (85) | Improvable |
| 31d | QIAamp | A | 3 (85) | Improvable |
| 7 | QIAamp | A | 9 (55) | Improvable |
| 8c | QIAamp | A | 9 (55) | Improvable |
| 14c,d | QIAamp | A | 10 (50) | Improvable |
| 2 | QIAamp | B | 0 (100) | Optimal |
| 4 | QIAamp | B | 0 (100) | Optimal |
| 25 | QIAamp | B | 0 (100) | Optimal |
| 27 | QIAamp | B | 0 (100) | Optimal |
| 33 | QIAamp | B | 0 (100) | Optimal |
| 34 | QIAamp | B | 0 (100) | Optimal |
| 35 | QIAamp | B | 0 (100) | Optimal |
| 38 | QIAamp | B | 0 (100) | Optimal |
| 41 | QIAamp | B | 0 (100) | Optimal |
| 12 | QIAamp | B | 1 (95) | Acceptable |
| 40d | QIAamp | B | 1 (95) | Acceptable |
| 18 | QIAamp | B | 3 (85) | Improvable |
| 28 | QIAamp | C | 0 (100) | Optimal |
| 39 | QIAamp | C | 2 (90) | Acceptable |
| 3e | QIAamp | C | 5 (75) | Improvable |
| 15c | QIAamp | D | 9 (55) | Improvable |
| 6c,e | QIAamp | D | 14 (30) | Improvable |
| 5 | QIAamp | E | 2 (90) | Acceptable |
| 1 | QIAamp | F | 1 (95) | Acceptable |
| 16 | RNeasy | In-house | 0 (100) | Optimal |
| 36 | QIAamp | In-house | 0 (100) | Optimal |
| 19d | TRIzol | In-house | 1 (95) | Acceptable |
| 23e | QIAamp | In-house | 2 (90) | Acceptable |
| 22 | TRIzol | In-house | 3 (85) | Improvable |
QIAamp, QIAamp viral RNA minikit (Qiagen) (n = 36 [87.8%]); RNeasy, RNeasy minikit (Qiagen) (n = 2 [4.9%]); TRIzol (Invitrogen) (n = 3 [7.3%]).
A, B, C, D, E, and F represent 6 commercial TaqMan real-time RT-PCR kits for the detection of hand, foot, and mouth disease. A, Kinghawk Pharmaceutical, Beijing, China; B, Guangzhou Daan Gene Co., Ltd., Guangzhou, China; C, Promega Corporation, Fitchburg, WI; D, Bioperfects Technologies, Shanghai, China; E, Beijing Ipe-bio Technologies Co., Ltd., Beijing, China; F, Beijing Sunbiostar Gene Technologies Co., Ltd., Beijing, China. A to E are monoplex assays, and F is a triplex assay. In-house, in-house conventional RT-PCR assay for HFMD recommended by the Chinese CDC.
The laboratory (n = 4) failed to detect EV71 in any of the samples.
The laboratory (n = 14) returned one false-positive result.
The laboratory (n = 5) returned two false-positive results.
Table 3.
Comparison of sensitivity and specificity data between different commercial and in-house assays
| Assaya | No. of laboratories using the indicated assay | EV detection |
EV71 detection |
CA16 detection |
|||
|---|---|---|---|---|---|---|---|
| No. of correct positive results/total no. of positive results (% sensitivity) | No. of correct negative results/total no. of negative results (% specificity) | No. of correct postivie results/total no. of postive results (% sensitivity) | No. of correct negative results/total no. of negative results (% specificity) | No. of correct positive results/total no. of postive results (% sensitivity) | No. of correct negative results/total no. of negative results (% specificity) | ||
| A | 17 | 265/272 (97.4) | 65/68 (95.6) | 114/153 (74.5)b | 181/187 (96.8) | 146/153 (95.4) | 185/187 (98.9) |
| B | 12 | 191/192 (99.5) | 48/48 (100) | 106/108 (98.1) | 131/132 (99.2) | 106/108 (98.1) | 132/132 (100) |
| C | 3 | 44/48 (91.7) | 10/12 (83.3) | 24/27 (88.9) | 33/33 (100) | 25/27 (92.6) | 33/33 (100) |
| D | 2 | 28/32 (87.5)b | 7/8 (87.5) | 0/18 (0)b | 22/22 (100) | 14/18 (77.8) | 21/22 (95.5) |
| E | 1 | 15/16 (93.8) | 4/4 (100) | 7/9 (77.8) | 11/11 (100) | 9/9 (100) | 11/11 (100) |
| F | 1 | 16/16 (100) | 4/4 (100) | 8/9 (88.9) | 11/11 (100) | 9/9 (100) | 11/11 (100) |
| In-house | 5 | 80/80 (100) | 19/20 (95) | 44/45 (97.8) | 53/55 (96.4) | 42/45 (93.3) | 55/55 (100) |
| Total | 41 | 639/656 (97.4) | 157/164 (95.7) | 303/369 (82.1) | 442/451 (98) | 351/369 (95.1) | 448/451 (99.3) |
We next assessed the sensitivities of detection for the 3 targets (EV, EV71, and CA16) (Table 4). The sensitivity for EV71 (82.1%) was much lower than that for EV (97.4%) or CA16 (95.1%) (P < 0.05), but no obvious differences between the sensitivities for EV and CA16 were observed (P > 0.05). As expected, assay accuracy declined with decreasing concentration, and most of the detection failures occurred at the 1 × 103 IU/ml target concentration. At that target concentration, the rates of detection were 90.2% for EV, 59.8% for EV71, and 84.1% for CA16.
Table 4.
Summary of results for the 4 types of target specimens in the EQA
| Samplea | Concn (IU/ml) | No. of replicates per laboratoryb | No. of positive samples/no. of samples tested (%) | Total no. of positive samples/total no. of samples tested (%) |
|---|---|---|---|---|
| EV | 107 | 2 | 82/82 (100) | 639/656 (97.4) |
| 106 | 2 | 82/82 (100) | ||
| 105 | 4 | 164/164 (100) | ||
| 104 | 4 | 163/164 (99.4) | ||
| 103 | 4 | 148/164 (90.2) | ||
| EV71 | 107 | 1 | 37/41 (90.2) | 303/369 (82.1) |
| 106 | 2 | 74/82 (90.2) | ||
| 105 | 2 | 72/82 (87.8) | ||
| 104 | 2 | 71/82 (86.6) | ||
| 103 | 2 | 49/82 (59.8) | ||
| CA16 | 107 | 1 | 41/41 (100) | 351/369 (95.1) |
| 106 | 2 | 82/82 (100) | ||
| 105 | 2 | 82/82 (100) | ||
| 104 | 2 | 77/82 (93.9) | ||
| 103 | 2 | 69/82 (84.1) | ||
| Negative | 0 | 4 | 12b/164 (7.9) | 12/164 (7.3) |
EV71 and CA16 were used as specificity controls for each other. False-positive results for EV (n = 7) and EV71 (n = 6) were present in 12 negative samples. Negative sample 11 was simultaneously contaminated by EV and EV71 in laboratory 20. No false-positive CA16 results were reported for negative samples.
Because positive samples contained two or three targets, the total number of replicates exceeds the number of samples tested (n = 20).
All of the false-negative results were reported by 23 laboratories, and the overall proportion of false-negative results was 101/1,394 (7.3%). Considering that over 40% (17/41) of laboratories in the EQA program used assay A, the lower sensitivity of that assay for EV71 may contribute to false-negative results during routine testing of suspected HFMD cases. The performance of assay D was unacceptable during this EQA, as the two laboratories that used that assay had scores of 30 and 55 (Table 2) and the assay failed to detect any samples containing EV71 and also had lower sensitivity for CA16 (Table 3).
A total of 19 (1.8%) false-positive results (of the total of 1,066 results) were reported in the study. The individual specificities of detection for EV, EV71, and CA16 were 157/164 (95.7%), 442/451 (98%), and 448/451 (99.3%), respectively, with no significant differences found among the 3 targets. There were also no statistical differences observed in specificity comparisons of each commercial assay versus the in-house assay (Table 3).
DISCUSSION
This study was the first national EQA program designed to evaluate molecular diagnosis of EV71 and CA16 by the use of armored RNAs as virus surrogates. Our data demonstrate that armored RNA serves as a robust and stable alternative to infectious or inactivated virus in proficiency programs.
The 41 laboratories that participated in this EQA used six different TaqMan RT-PCR assays and one conventional RT-PCR assay. Our results in this study were not completely in agreement with those of previous EQA studies (9, 10), which reported that commercial real-time PCR technologies exhibited better sensitivity than conventional PCR. The less-than-perfect scores observed in this EQA study were likely due to a combination of poor assay performance (in particular, commercial assays A and D) and poor laboratory proficiency (contamination observed in 14 laboratories). However, several commercial assays performed well and represent a standardized method that would facilitate interpretation and comparability of results among different laboratories.
In this EQA, a triplex commercial real-time RT-PCR assay (kit F) was used by only one participant. Our communications with all participants revealed that most laboratories prefer single assays over multiplex methods, as they indicated that single assays customarily produce more sensitive and reliable results. The inclusion of multiple templates in a single reaction mixture has the potential to reduce the sensitivity of multiplex RT-PCR assays due to competition of reagents. Recently, accurate multiplex RT-PCR assays that simultaneously distinguish among EV71, CVA16, and other enteroviruses have been described (8, 23).
Laboratory use of commercial RT-PCR assay A in this EQA resulted in a wide range of scores, from 50 to 100%. Poor sensitivity for EV71 was the primary factor in less-than-perfect scores (Table 3). Further investigation of assay A revealed that higher rates of false-negative results were associated with specific lots of the EV71-specific reagents. While assay D was used by only two participants in this study, its false-negative rate for the EV71 samples reached a startling 100%. More studies are required to determine whether there are some subgenotypes of EV71 virus that are out of the detection scope of this assay. The lower sensitivity of some commercial assays for EV71 indicated an urgent need to improve kit performance characteristics, internal quality control by the manufacturers, and external quality assurance in laboratories. In addition, false-positive results in some of the laboratories indicated the need for improved laboratory practices and quality management. The present report emphasizes that EQA is a very important tool for assessing the quality of diagnostic laboratory tests.
In summary, 3 armored RNAs for HFMD viruses were successfully constructed and used in a nationwide EQA in China. The results of this first nationwide EQA not only verified the feasibility of the use of those armored RNAs to serve as control samples but also highlighted a series of problems regarding HFMD diagnosis, such as the low sensitivity exhibited by some commercial assays and the overall poor performance of some participating laboratories. An EQA should be performed periodically to help laboratories monitor their ability to detect HFMD viruses and to improve the concordance of results from different laboratories. In addition, given the adaptability of RNA viruses, more HFMD viruses with a wider range of genotypes should be included in future EQA.
Supplementary Material
ACKNOWLEDGMENTS
This study was supported by the National Natural Science Foundation of China (30371365, 30571776, and 30972601).
We gratefully acknowledge the contributions of all of the partners and participant laboratories.
All authors declare that we have no financial conflicts of interest.
Footnotes
Supplemental material for this article may be found at http://jcm.asm.org/.
Published ahead of print on 24 August 2011.
REFERENCES
- 1. Aarthi D., Ananda Rao K., Robinson R., Srinivasan V. A. 2004. Validation of binary ethyleneimine (BEI) used as an inactivant for foot and mouth disease tissue culture vaccine. Biologicals 32:153–156 [DOI] [PubMed] [Google Scholar]
- 2. AbuBakar S., et al. 1999. Identification of enterovirus 71 isolates from an outbreak of hand, foot and mouth disease (HFMD) with fatal cases of encephalomyelitis in Malaysia. Virus Res. 61:1–9 [DOI] [PubMed] [Google Scholar]
- 3. Beld M., et al. 2004. Highly sensitive assay for detection of enterovirus in clinical specimens by reverse transcription-PCR with an armored RNA internal control. J. Clin. Microbiol. 42:3059–3064 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Bressler A. M., Nolte F. S. 2004. Preclinical evaluation of two real-time, reverse transcription-PCR assays for detection of the severe acute respiratory syndrome coronavirus. J. Clin. Microbiol. 42:987–991 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Chan K. P., et al. 2003. Epidemic hand, foot and mouth disease caused by human enterovirus 71, Singapore. Emerg. Infect. Dis. 9:78–85 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Chang L. Y., et al. 2004. Transmission and clinical features of enterovirus 71 infections in household contacts in Taiwan. JAMA 291:222–227 [DOI] [PubMed] [Google Scholar]
- 7. Chang L. Y., et al. 1999. Comparison of enterovirus 71 and coxsackie-virus A16 clinical illnesses during the Taiwan enterovirus epidemic, 1998. Pediatr. Infect. Dis. J. 18:1092–1096 [DOI] [PubMed] [Google Scholar]
- 8. Chen T. C., et al. 2006. Combining multiplex reverse transcription-PCR and a diagnostic microarray to detect and differentiate enterovirus 71 and coxsackievirus A16. J. Clin. Microbiol. 44:2212–2219 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. de Pagter P. J., Schuurman R., de Vos N. M., Mackay W., van Loon A. M. 2010. Multicenter external quality assessment of molecular methods for detection of human herpesvirus 6. J. Clin. Microbiol. 48:2536–2540 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Domingo C., et al. 2010. 2nd international external quality control assessment for the molecular diagnosis of dengue infections. PLoS Negl. Trop. Dis. 4:e833. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Foo D. G., et al. 2008. Identification of immunodominant VP1 linear epitope of enterovirus 71 (EV71) using synthetic peptides for detecting human anti-EV71 IgG antibodies in Western blots. Clin. Microbiol. Infect. 14:286–288 [DOI] [PubMed] [Google Scholar]
- 12. Hietala S. K., Crossley B. M. 2006. Armored RNA as virus surrogate in a real-time reverse transcriptase PCR assay proficiency panel. J. Clin. Microbiol. 44:67–70 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Huang Q. Y., Cheng Y. J., Guo Q. W., Li Q. G. 2006. Preparation of a chimeric armored RNA as a versatile calibrator for multiple virus assays. Clin. Chem. 52:1446–1448 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Jiang T., et al. 2011. Development and evaluation of a reverse transcription-loop-mediated isothermal amplification assay for rapid detection of enterovirus 71. J. Clin. Microbiol. 49:870–874 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Kessler H. H., et al. 1997. Rapid diagnosis of enterovirus infection by a new one-step reverse transcription-PCR assay. J. Clin. Microbiol. 35:976–977 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Li L., et al. 2005. Genetic characteristics of human enterovirus 71 and coxsackievirus A16 circulating from 1999 to 2004 in Shenzhen, People's Republic of China. J. Clin. Microbiol. 43:3835–3839 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Lipson S. M., Walderman R., Costello P., Szabo K. 1988. Sensitivity of rhabdomyosarcoma and guinea pig embryo cell cultures to field isolates of difficult-to-cultivate group A coxsackieviruses. J. Clin. Microbiol. 26:1298–1303 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Pasloske B. L., Walkerpeach C. R., Obermoeller R. D., Winkler M., DuBois D. B. 1998. Armored RNA technology for production of ribonuclease-resistant viral RNA controls and standards. J. Clin. Microbiol. 36:3590–3594 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Pisani G., et al. 2008. External quality assessment for the detection of HCV RNA, HIV RNA and HBV DNA in plasma by nucleic acid amplification technology: a novel approach. Vox Sang. 95:8–12 [DOI] [PubMed] [Google Scholar]
- 20. Qiu J. 2008. Enterovirus 71 infection: a new threat to global public health? Lancet Neurol. 7:868–869 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Ryu W. S., et al. 2010. Clinical and etiological characteristics of enterovirus 71-related diseases during a recent 2-year period in Korea. J. Clin. Microbiol. 48:2490–2494 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Saldanha J., Lelie N., Heath A., and the WHO Collaborative Study Group 1999. Establishment of the first international standard for nucleic acid amplification technology (NAT) assays for HCV RNA. Vox Sang. 76:149–158 [DOI] [PubMed] [Google Scholar]
- 22a. Simes R. 1986. An improved Bonferroni procedure for multiple tests of significance. Biometrika 73:751–754 [Google Scholar]
- 23. Thao N. T., et al. 2010. Development of a multiplex polymerase chain reaction assay for simultaneous identification of human enterovirus 71 and coxsackievirus A16. J. Virol. Methods 170:134–139 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24. Wang J. R., et al. 2002. Change of major genotype of enterovirus 71 in outbreaks of hand-foot-and-mouth disease in Taiwan between 1998 and 2000. J. Clin. Microbiol. 40:10–15 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25. Wang S. Y., Lin T. L., Chen H. Y., Lin T. S. 2004. Early and rapid detection of enterovirus 71 infection by an IgM-capture ELISA. J. Virol. Methods 119:37–43 [DOI] [PubMed] [Google Scholar]
- 26. Wei Y. X., et al. 2008. RNase-resistant virus-like particles containing long chimeric RNA sequences produced by two-plasmid coexpression system. J. Clin. Microbiol. 46:1734–1740 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27. Xu F., et al. 2010. Performance of detecting IgM antibodies against enterovirus 71 for early diagnosis. PLoS One 5:e11388. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28. Yang F., et al. 2009. Enterovirus 71 outbreak in the People's Republic of China in 2008. J. Clin. Microbiol. 47:2351–2352 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29. Zhan S. E., et al. 2009. Armored long RNA controls or standards for branched DNA assay for detection of human immunodeficiency virus type 1. J. Clin. Microbiol. 47:2571–2576 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30. Zhang D., Lu J., Lu J. 2010. Enterovirus 71 vaccine: close but still far. Int. J. Infect. Dis. 14:e739–e743 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.