External quality assurance of serological diagnosis of dengue, chikungunya and Japanese encephalitis virus infection

Highlights

• •External quality assurance of serological diagnosis of dengue virus (DENV), chikungunya virus (CHIKV) and Japanese encephalitis virus (JEV) was undertaken at 124 virus research and diagnostic laboratories (VRDLs) over 2 years.
• •The average concordance of 124 VRDLs for both 2018–19 and 2019–20 was 98%.
• •In 2018–19, 78.33% of VRDLs reported 100% concordance for DENV, CHIKV and JEV IgM panels.
• •In 2019–20, 79.68% of VRDLs showed 100% concordance for DENV, CHIKV and JEV IgM panels.
• •This study clearly indicates good proficiency in serodiagnosis of DENV, CHIKV and JEV infections in VRDLs.

Abstract

Introduction

Dengue, chikungunya and Japanese encephalitis are the most common arthropod-borne viral diseases in India. Due to overlapping clinical symptoms, accurate, high-quality and timely laboratory-based differential diagnosis is essential for control and containment of outbreaks. This is most commonly done by detection of IgM antibodies in serum using enzyme-linked immunosorbent assays. The Resource Centre for Virus Research and Diagnostic Laboratories (VRDLs) in Pune, India organized an external quality assurance (EQA) study to check the accuracy of serological diagnostics in the VRDL network.

Methods

Three panels, one each for anti-dengue virus, anti-chikungunya virus and anti-Japanese encephalitis virus IgM antibodies, comprising six human serum samples (two positive and four negative) were distributed to test the sensitivity, specificity and reproducibility of serological testing in 124 VRDLs across India in 2018–19 and 2019–20.

Results

Among the 124 VRDLs, the average concordance for both 2018–19 and 2019–20 was 98%. In 2018–19, 78.33%, 13.33% and 6.66% of VRDLs reported 100% concordance, 91–99% concordance and 81–90% concordance with the reference results, respectively, and 1.66% of VRDLs had concordance <80%. In 2019–20, 79.68%, 14.06% and 4.68% of VRDLs reported 100% concordance, 91–99% concordance and 81–90% concordance with the reference results, respectively, and 1.56% of VRDLs had concordance <80%.

Conclusion

The EQA programme was beneficial for assessing and understanding the performance of the VRDLs. The study data indicate good proficiency in serological diagnosis of dengue, chikungunya and Japanese encephalitis in the VRDL network laboratories. Further expansion of the EQA programme to cover other viruses of public health importance will increase confidence among the VRDL network, and generate evidence of high-quality testing.

Introduction

Emerging infectious diseases continue to endanger public health, assisted by commercial globalization, travel, human relocation and environmental disruption [1,2]. The consequences of these factors were witnessed during the coronavirus disease 2019 pandemic. Many pandemic threats can be attributed to viruses from either zoonotic or vector-borne sources [3].

The geographical distribution of disease vectors increases the occurrence of outbreaks of emerging and re-emerging infectious diseases [2]. Aetiologies of arboviral origin are sufficiently common to cause significant morbidity and mortality worldwide [2]. The most common arboviral infections found in tropical and subtropical regions are dengue, chikungunya and Japanese encephalitis.

Dengue virus (DENV) from the Flaviviridae family is transmitted by the vector Aedes aegypti, and has spread rapidly in tropical and subtropical regions in recent years [4,5]. While the annual incidence of DENV infection is reported to be >390 million cases worldwide, India reported an average of 130,000 cases between 2015 and 2020 ( [5]). Up to June 2022, 10,172 confirmed cases and three deaths of DENV infection have been reported in India [6]. A. aegypti is also the transmission vector for chikungunya virus (CHIKV), an alpha virus of the Togaviridae family, which presents similar symptoms as dengue. Since 2004, CHIKV has spread rapidly and has been recorded in more than 60 countries [7]. India reported an average of 65,942 confirmed cases between 2015 and 2020, and 22,724 confirmed cases of chikungunya have been reported in India up to June 2022 ( [8]).

Japanese encephalitis virus (JEV) is a flavivirus transmitted via Culex spp. Japanese encephalitis is another viral infectious disease which has a wide distribution in many countries (Centers for Disease Control and Prevention, n.d.). Globally, approximately 68,000 clinical cases of Japanese encephalitis are reported every year. Cases of Japanese encephalitis have been reported regularly from states in northern and north-eastern India, but naive non-endemic regions of the country have seen a spread in viral activity recently [9]. India reported an average of 1618 cases of Japanese encephalitis from 2015 to 2020, and 62 cases have been reported up to June 2022 ( [10]).

The overlapping symptoms, common vectors and similar geographical distribution of these three arboviral infections raises concern, and demands good clinical and laboratory diagnosis.

Laboratory test results are an essential component of diagnostic decision-making, and surveillance and control of diseases of public health importance. The implementation of quality assurance (QA) is an important activity in managing a diagnostic laboratory [11,12]. QA should ensure that quality improvement processes are in place and are integrated within the QA programme of the organization; good practice ideas are applied; poor clinical performance is recognized and dealt with promptly; and the quality of data collected is of a high standard.

External quality assurance (EQA) is used to ensure the analytical quality of a laboratory. Participating laboratories are expected to process survey specimens using the same methods that they use routinely with patient specimens. Hence, EQA is considered as an indirect assessment of laboratory performance with clinical samples. The aim of an EQA programme is to improve laboratory performance through scientific recommendations and standardization, while considering quality specifications [13].

Understanding challenges due to emerging/re-emerging viral infections and limited capacity for timely detection of viruses in India, the Department of Health Research/Indian Council of Medical Research (ICMR) initiated a programme to establish a network of virus research and diagnostic laboratories (VRDLs) to enhance the country's capacity for early identification and diagnosis of all viral infections of public health importance. To date, 130 VRDLs have been established, and two resource centres for VRDLs (RCVRDLs) have been identified for the network in order to build the capacity of the VRDLs. ICMR-National Institute of Virology (NIV), Pune acts as the resource centre for laboratory-related capacity building, training, and establishing QA and quality control programmes; and ICMR-National Institute of Epidemiology, Chennai acts as the resource centre for data management in the VRDL network. Both the RCVRDLs have played a significant role in competence building of the VRDLs to a great extent. VRDLs have the capacity to diagnose a wide ambit of different categories of viral diseases, such as viruses transmitted by the respiratory route or the faeco-oral route, vector-borne viruses, sexually transmitted viruses, neurotropic viruses and cancer-causing viruses. Routine diagnosis and outbreak reports are captured in a specified case report format, which captures epidemiological data, syndromic data, geographical data and laboratory results.

The VRDLs help to cover the entire country for timely diagnosis/identification of viruses, and during outbreaks can generate data on viral diseases to facilitate quick deployment of adequate resources and measures to save human lives. Quality diagnostic testing standards are maintained by the implementation of interlaboratory quality control and EQA programmes by RCVRDL, ICMR-NIV, Pune.

As part of the laboratory QA programme, an EQA scheme was launched by RCVRDL, ICMR-NIV, Pune for proficiency testing of serological tests for DENV, CHIKV and JEV from 2018 for VRDL network laboratories. This study summarizes the performance of VRDLs observed in two rounds of EQA for serodiagnosis using IgM capture enzyme-linked immunosorbent assay (ELISA) for DENV, CHIKV and JEV infections between 2018 and 2020.

Methods

Ethical statement

All samples were collected with informed consent from the patients, and the study was approved by ICMR-NIV Institutional Ethics Committee (IHEC No. NIV/IHEC/2016/D-316). The samples used in this study for preparation of the EQA panels did not include personal identifiers or patient data.

Participating laboratories

In total, 124 VRDLs participated in this EQA programme for the detection of IgM for DENV, CHIKV and JEV by ELISA. All samples used in the preparation of the EQA panel were serum samples. Evaluation results were made available to the participating laboratory and ICMR alone.

Panel preparation and distribution

A 06 sample panel was prepared for distribution to the participating laboratories. Two of the six samples were positive and four were negative. The EQA sample panel was prepared from the samples available in sufficient volume in the sample repository of the study laboratory, and were heat inactivated at 56°C for 1 h before further preparation. For preparation of the panel, DENV, CHIKV and JEV IgM-positive serum samples with optical densities (OD) >1.00 at 450 nm were pooled to make an average volume of 5 mL. For the DENV-positive panel, samples of all four DENV serotypes were pooled. Likewise, samples from different CHIKV serotypes were pooled to prepare the positive panel. Negative samples for DENV, CHIKV and JEV with OD <0.09 were pooled to prepare a pool of 5 mL. The negative panel included two samples that were prepared by pooling samples presenting cross-reactivity for the respective testing viruses. The pooled samples were aliquoted (20 µL) in each vial, coded and tested in duplicate by different operators (interoperator testing) on different days. EQA panel samples were assayed in duplicate by three operators to validate stability and homogeneity. Five sample aliquots were selected at random and assayed in triplicate to assess homogeneity, and for stability, samples were stored at -80 °C, -20 °C, 4 °C and room temperature for 24 h and 72 h. Three panels from each storage condition were selected at random and assayed three times. Blinded EQA samples were prepared and distributed on dry ice to 124 VRDLs.

Serological testing methods

All VRDLs used kits from the same manufacturer (ICMR-NIV, Pune, India) for the EQA programme. NIV Dengue IgM capture ELISA kit was used for the detection of DENV IgM, NIV Chikungunya IgM capture ELISA kit was used for the detection of CHIKV IgM, and NIV Japanese Encephalitis IgM capture ELISA kit was used for the detection of JEV IgM. All tests were performed in accordance with the manufacturer's instructions.

Evaluating the EQA results and statistical analysis

The EQA results were scored based on qualitative results; every correct positive or negative test result was compared with the expected result documented earlier at RCVRDL, ICMR-NIV Pune. The percentage of concordance for DENV, CHIKV and JEV was calculated separately for each VRDL. Laboratories that had a concordance rate ≥90% compared with the expected results were regarded as satisfactory and considered to pass the EQA challenge. VRDLs that had a concordance rate <90% compared with the expected results were asked to perform repeat testing in duplicate for specific parameters, and contacted for root cause analysis. All test kits included the manufacturer's instructions for threshold cut-off settings and interpretation. For evaluation of semi-quantitative data, the mean, median and interquartile range (IQR) of OD values were converted to box-and-whisker plots. Analyses were performed using GraphPad Prism 9.2.

Results

Participating laboratories and panel preparation

In total, 124 VRDLs across India participated in this EQA programme: 60 VRDLs participated in 2018–19 and 64 participated in 2019–20. The participating VRDLs included functional laboratories under the VRDL project initiated by the Department of Health Research, Government of India. The three tier national laboratories – namely medical college level, state level and regional level laboratories – participated in both 2018–19 and 2019–20.

During preparation of the panel, the stability of the panel samples was tested by storing them at -80 °C, -20 °C, 4 °C and room temperature for 24 h and 72 h. No significant changes in the quality and activity of the samples was identified post storage.

Overall performance of the laboratories

The overall performance of the participating VRDLs for the EQA panel is shown in Figure 1. For 2018–19, 47 of the 60 (78.33%) participating VRDLs reported 100% concordance with the reference results for DENV, CHIKV and JEV IgM panels, eight of 60 (13.33%) VRDLs had concordance of 91–99%, four of 60 (6.66%) VRDLs had concordance of 81–90%, and one VRDL (1.66%) had concordance <80%. In 2019–20, 51 of the 64 (79.68%) participating VRDLs showed 100% concordance with the reference results, nine of the 64 (14.06%) VRDLs had concordance of 91–99%, three of 64 (4.68%) VRDLs had concordance of 81–90%, and one (1.56%) VRDL had concordance <80%. The average concordance for both 2018–19 and 2019–20 was 98% (Table S1, see online supplementary material).

Figure 1.

Overall performance of participating virus research and diagnostic laboratories (VRDLs) for external quality assurance (EQA) panel of dengue virus (DENV), chikungunya virus (CHIKV) and Japanese encephalitis virus (JEV) IgM capture enzyme-linked immunosorbent assay. Average concordance (DENV, CHIKV and JEV) of 60 VRDLs in 2018–19 (A) and 64 VRDLs in 2019–20 (B) was 98%.

Individual performance of the laboratories

Dengue

In 2018–19, 59 of the 60 participating VRDLs tested all test samples correctly (Figure 2A), and one VRDL reported correct results for five of six samples. Of the 64 VRDLs that participated in 2019–20, 57 reported all samples correctly and seven reported five of six samples correctly (Figure 2D).

Figure 2.

External quality assurance (EQA) for dengue virus (DENV), chikungunya virus (CHIKV) and Japanese encephalitis virus (JEV) serological testing. EQA performance of individual virus research and diagnostic laboratories (VDRLs). Green bars above the baseline indicate correctly tested samples; purple bars below the baseline indicate incorrectly tested samples. (A–F) VRDLs are sorted by the parameter tested and the year of participation in EQA.

Chikungunya

In 2018–19, 49 of the 60 participating VRDLs reported correct results for all the samples, 10 VRDLs reported incorrect results for one of six samples, one VRDL reported incorrect results for two of six samples, and one VRDL reported incorrect results for four of six samples (Figure 2B). In 2019–20, 60 of the 64 participating VRDLs reported correct results for all samples, three VRDLs reported correct results for five of six samples, and one VRDL reported incorrect results for three of six samples (Figure 2E).

Japanese encephalitis

In 2018–19, 57 of the 60 participating VRDLs reported correct results for all the samples, and three VRDLs reported incorrect results for three of six samples (Figure 2C). In 2019–20, 57 of the 64 participating VRDLs reported correct results for all the samples, and seven VRDLs reported correct results for five of six samples (Figure 2F).

Analysis of results of individual samples

Of the six samples sent for each parameter, four samples were negative and two were positive. The OD values received from all the participating VRDLs were compared with the reference OD. Less variation was observed in the ODs of negative samples compared with the ODs of positive samples. The descriptive statistics (mean, standard deviation and 95% confidence interval of mean) of the OD values obtained from the participating laboratories were also analysed. Positive samples had high standard deviations, and negative samples had low standard deviations (Table S2, see online supplementary material).

The average IQRs of the positive samples in 2018–19 were 0.994 for DENV, 1.07 for CHIKV and 0.85 for JEV, and the IQRs for negative samples were 0.060 for DENV, 0.082 for CHIKV and 0.076 for JEV (Figure 3A,C,E). Two of the sample results discordant with the reference results were reported to be false negatives, and the remaining samples were equivocal/intermediate.

Figure 3.

Performance of virus research and diagnostic laboratories (VDRLs) in external quality assurance programme 2018–20. Reported optical density (OD) values of samples were compared with reference OD values from the Resource Centre of VDRLs (RCVDRL). Median OD values are indicated by bars, quartiles are indicated by boxes, and interquartile ranges are indicated by whiskers. DEN, dengue virus; CHIK, chikungunya virus; JE, Japanese encephalitis virus.

The average IQRs of the negative samples in 2019–20 were 0.060 for DENV, 0.089 for CHIKV and 0.055 for JEV, and the IQRs for positive samples were 1.00 for DENV, 0.809 for CHIKV and 0.518 for JEV (Figure 3B,D,F). Six of the discordant sample results were false positives/false negatives, and seven samples were equivocal/intermediate.

Interlaboratory comparison: Youden two-sample plots of positive samples

Figure 4 shows Youden two-sample plots for the positive samples of the DENV, CHIKV and JEV EQA panels. Each dot in these panels represents one pair of positive sample from one participating laboratory. The median of each sample is used as the estimate of distribution location as it is not dependent on extreme values, as is the mean. Two median lines were drawn, parallel to the X and Y axes, respectively. The point where these lines meet is called the ‘Manhattan median’. After this, a 45^o reference line was drawn through the Manhattan median. The ellipse/circle in each panel was constructed to enclose data pairs that were consistent with the consensus medians at approximately 95% of the VRDLs.

Figure 4.

Youden plots for positive samples of dengue virus (DENV), chikungunya virus (CHIKV) and Japanese encephalitis virus (JEV) external quality assurance panel 2018–20. Each plot represents the distribution of virus research and diagnostic laboratories as a point of optical density values of two positive samples plotted on the X and Y axes.

These plots were used to measure bias in the measurements. Points near the 45^o reference line but far from the Manhattan median indicate large systematic error. Points that lie far from the 45^o line indicate large random error. Points outside the circle indicate large total error. Points scattered close to the 45^o reference line and the Manhattan median represent similarity in measurements. Points lying close to the 45^o reference line suggest careful following of the test procedure. Points that lie well away from the line indicate sample-specific interferences.

The Youden plots in Figure 4 suggest that all of the VRDLs showed good precision in measurement and proper following of the test procedure, except for the few laboratories whose results lie outside the 95% circle.

Discussion

The EQA programme for detection of DENV, CHIKV and JEV IgM has been conducted every year since 2018 to understand the quality of serological diagnosis of DENV, CHIKV and JEV infection, and to assess the overall performance of VRDLs. The EQA programme provides a platform for achieving high standards and synchronization in diagnostic procedures [13]. In the current situation, it is imperative to have these goals as diagnostic results not only influence the patient management and healthcare system, but also affect socio-economic and political decisions in designing the strategies for disease containment and control of disease in new geographic areas.

Contrary to internal quality control (IQC), which ensures and assesses the correctness and reproducibility of a test in real time, EQA assesses the performance of a laboratory against other laboratories, and also evaluates long-term performance of laboratories [14]. Additionally, IQC involves analysing control material and assessing the performance with predefined values and limits. EQA, on the other hand, helps a laboratory to assess and compare the results of each analyte with those obtained from other laboratories [15]. Although EQA cannot provide a real-time analysis, it tests the robustness of the testing methods and the accuracy of the participating laboratories [16].

In total, 124 VRDLs across India participated in this 2-year EQA programme. Each year, the EQA panel was distributed to some new laboratories and some from the last distribution. The EQA panel was implemented for serodiagnosis of IgM antibodies for the three most common arboviral infections in India. This EQA programme focused on the detection of IgM as this is a front-line diagnostic assay for clinical diagnosis of these viral infections.

All the participating VRDLs used kits from the same manufacturer for testing the EQA panel, which eased the analysis and comparison of the results received from the VRDLs. The overall diagnostic performance for the detection of DENV, CHIKV and JEV IgM was good, with 98% overall concordance for the participating VRDLs for both years. The VRDLs that participated in both 2018–19 and 2019–20 showed gradual improvement or consistent performance, with the exception of a few VRDLs.

DENV IgM testing had fewer discordant samples each year compared with CHIKV and JEV IgM testing, and JEV IgM testing had fewer discordant results compared with CHIKV IgM testing. The greatest number of discordant samples was seen for CHIKV IgM testing. This may be due to changes in trained contractual project-based staff.

Although the overall performance of DENV, CHIKV and JEV IgM testing was acceptable, there were differences in the OD values of the panel samples. The larger IQRs of positive samples indicate significant variation in the OD values of the positive samples compared with the reference results. Some variation was also observed in the OD values of the negative samples. The fact that the majority of samples lay close to the Manhattan median suggests good analytical accuracy of the participating VRDLs. The Youden plots also suggests errors in sample measurement. Points lying in the upper left and lower right quadrants suggest that the results are good for one sample and not accurate for the other. These discrepancies may have arised due to errors in handling of samples or improper implementation of the test procedure. All VRDLs are required to comply with good clinical laboratory practice guidelines in order to ensure precision and accuracy, and minimize errors in diagnosis.

A major limitation of this EQA was the limited number of samples. An EQA with a larger set of samples in the panel would facilitate precise assessment of the VRDLs. Another limitation was the number of VRDLs participating in the EQA programme. Due to the lack of availability of reference samples and logistical constraints, a large number of VRDLs cannot be involved in the EQA programme at the same time. A critical point in management of the EQA programme is the shipment of samples. Of note, the EQA facilitated participants to test characterized samples of different origin, and assess their own performance. In this way, they may identify inadequacies in their protocols which would otherwise remain undetected.

Conclusion

In conclusion, the EQA programme was beneficial for assessing and understanding the performance of the VRDLs. QA has an important role in the clinical management of patients and policy making. The results suggest that the overall quality of VRDLs has improved since initiation of the EQA programme, but more VRDLs should be involved in this programme to ensure continuous and accurate laboratory diagnosis.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.