Performance comparison of two malaria rapid diagnostic test with real time polymerase chain reaction and gold standard of microscopy detection method

Abstract

Background

The diagnostic test for malaria is mostly based on Rapid Diagnostic Test (RDT) and detection by microscopy. Polymerase Chain Reaction (PCR) is also a sensitive detection method that can be considered as a diagnostic tool. The outcome of malaria microscopy detection depends on the examiner’s ability and experience. Some RDT has been distributed in Indonesia, which needs to be evaluated for their results.

Objective

This study aimed to compare the performance of RightSign RDT and ScreenPlus RDT for detection of Plasmodium in human blood. We used specific real-time polymerase chain reaction abTESTMMalaria qPCRII) and gold standard of microscopy detection method to measure diagnostic efficiency.

Methods

Blood specimens were evaluated using RightSign RDT, ScreenPlus RDT, Microscopy detection, and RT-PCR as the protocol described. The differences on specificity (Sp), sensitivity (Sn), positive predictive value (PPV), and negative predictive value (NPV) were analyzed using McNemar and Kruskal Wallis analysis.

Results

A total of 105 subjects were recruited. Based on microscopy test, RightSign RDT had sensitivity, Specificity, PPV, NPV, 100%, 98%, 98.2%, 100%, respectively. ScreenPlus showed 100% sensitivity, 98% specificity, 98.2% PPV, 100% NPV. The sensitivity of both RDTs became lower (75%) and the specificity higher (100 %) when using real-time PCR. Both RDTs showed a 100% agreement. RTPCR detected higher mix infection when compared to microscopy and RDTs.

Conclusion

RightSign and ScreenPlus RDT have excellent performance when using microscopy detection as a gold standard. Real-time PCR method can be considered as a confirmation tool for malaria diagnosis.

Introduction

Indonesia is a malaria endemic area. The province of Papua has Indonesia’s highest malaria burden. All Plasmodium species are present in Papua, including the Plasmodium knowlesi (P. knowlesi) originally discovered on Kalimantan Island. The most common types of Plasmodium infection in Papua are Plasmodium falciparum (P. falciparum) and Plasmodium vivax (P. vivax). Plasmodium ovale (P. ovale) and Plasmodium malariae (P. malariae) also can be found in Papua. The highest cause of morbidity and mortality in Papua is P. falciparum.^1,2

Several methods for diagnosing malaria have been established since WHO has confirmed how important new tests are to diagnose Plasmodium spp rapidly, reliably, accurately and cheaply, to address numerous shortcomings in microscopic examination as the WHO’s gold standard for malaria research.^3,4

Microscopic examination has advantages, namely allowing to identify species definitively, determine the condition of parasitemia, monitor malaria treatment response, easy execution and inexpensive^3,5. However, the weakness of microscopy examination is the difficulties in detecting extrimely low parasitemia and mix infections, and it is time consuming. Although microscopy examination is gold standard in malaria diagnosis, it is better to be accompanied by RDTs or other methods.^3,5

Other testing methods have been developed to diagnose malaria are the based on proteins produced by Plasmodium, such as Histidine-Rich Protein 2 (HRP-2) or enzymes such as Pan-Plasmodium Lactate Dehydrogenase (pLDH) and Pan-Specific Aldolase. This method has been widely used and is considered an alternative to malaria testing. This examination is also very prominent, especially in areas where microscopic examination is not available, or it is available but there is no laboratory staff experienced in the examination of blood or some other disadvantage of microscopy examination. Commercially enzyme-based examination tools generally use serological techniques to detect antigens, such as Rapid Diagnostic Tests (RDTs) or Immunochromatography Test (ICT) and Enzyme Linked Immunosorbent Assay (ELISA)^4,6,7.

The RDTs or ICT detect malaria antigens when the blood of the patient passes through a membrane containing Plasmodium-specific antibodies. The benefits of RDT are reliability in testing, quick processing times and tests, and cheap review fees, which is why it is very useful in endemic areas. The most commonly used RDTs are methods of detection based on HRP2 and pLDH. The advantages of RDTs are their parasitic detection limit of 50-100 parasites/μL. The disadvantage of RDT are reduced effectiveness and efficiency in examining large samples, because time consuming and a high level of concentration and accuracy are required.^{3, 8}

Most RDTs examine two proteins at once, such as a combination of a specific protein P. falciparum (HRP-2) with pLDH, a protein that is shared by all species of Plasmodium (non-specific) pLDH) or HRP- 2 combination comnined with a species specific protein species of Plasmodium vivax (Pv-pLDH). Three proteins also can be detected at once, such as HRP-2, nonspecific pLDH and Pv-pLDH that can distinguish between falciparum malaria and non-falciparum malaria or mixed malaria. ⁹

RightSign and ScreenPlus are available comercially in Indonesia. RightSign detects P. falciparum (HRP-2) and Pv-pLDH. ScreenPlus detecs P. falciparum (HRP-2) and Pan LDH. Both RDTs have not been evaluated in field study in Indonesia.

Molecular-based Polymerase Chain Reaction (PCR) techniques have used for years in detecting Plasmodium spp. A study on Polymerase Chain Reaction (PCR) even said that the sensitivity and specificity of PCR are higher than microscopic examination. PCR has a high sensitivity and specificity, but has several disadvantages: it is a fairly complex examination method, it is expensive, and requires expertise to be performed.^3,10,11

This study aimed to compare the performance of RightSign RDT and ScreenPlus RDT for detection Plasmodium in human blood. The difference performance of RDTs malaria compared to microscopy detection, and Real-Time Polymerase Chain Reaction (Real Time (RT)-PCR abTES^TMMalaria qPCRII).

Materials and Methods

Ethical approval

Ethical approval number 169/EC/KEPK/FKUA/2019 was obtained from the Health Research Ethics Committee of the Faculty of Medicine, Universitas Airlangga.

Sample collection

This study was a cross-sectional analytic study conducted at Merauke Hospital, Papua and Clinical Pathology Laboratory, Faculty of Medicine, Universitas Airlangga /Dr. Soetomo General Academic Hospital Surabaya during November 2018-June 2019. A total of 105 whole blood samples were collected in tubes containing Ethylene Diamine Tetraacetic Acid (EDTA).

Species identification and determination of parasite density

Identification of Plasmodium species and calculation of the parasitemia index (PI) were performed microscopically on Giemsa-stained thick and thin blood smears. PI was calculated using WHO guidelines Simultaneously the samples were using RightSign RDT and ScreenPlus RDT.¹²

RightSign RDT and ScreenPlus RDT

The positivity and antigen detection of the Plasmodium species in RightSign RDT were obtained through lines or bands that arise in the line test. RightSign uses the immunochromatography test (ICT) method, with nitrocellulose membranes detects P. falciparum-specific anti-Histidine Rich Protein II (HRP II) on the Pf test line and non-specific anti-Plasmodium Lactate Dehydrogenase (pLDH) from Plasmodium spp (P. falciparum, P. vivax, P. malariae, P. ovale) on the Pan test line. ScreenPlus detected P. vivax specific anti-Plasmodium Lactate Dehydrogenase (pLDH) antibody on the Pv test line and P. falciparum-specific anti-Histidine Rich Protein II (HRP II) antibodies. The differences on specificity (Sp), sensitivity (Sn), positive predictive value (PPV), and negative predictive value (NPV) were analyzed using McNemar and Kruskal Wallis analysis.

Results confirmation

Results confirmation of thick and thin blood smears examination was carried out in the Clinical Pathology Laboratory Faculty of Medicine, Universitas Airlangga and Dr. Soetomo general academic hospital, Surabaya. The inclusion criteria for sample acceptance were patients of all ages, males and females, clinical examinations showed symptoms of fever (specific or non-specific malaria) and who were willing to take part in the study by signing an informed consent form. The exclusion criteria were patients who have received malaria treatment and patients with fever but with negative results of malaria microscopy examination. The interpretation of microscopy and RDTs results was carried out by two blind and independent observers. Based on the McNemar test between the two observers’ results, the results were not significantly different with P<0.05.

Real Time PCR

DNA examination of Plasmodium malaria using the Rotor-Gene® Q PCR from Qiagen, Tokyo, Japan with the abTESTMMalaria qPCR III reagent kit. AbTESTMMalaria qPCR III (AITbiotech Pte Ltd, Singapore) can identify four species of Plasmodium spp. (P. falciparum, P. vivax, P. ovale, P. malariae, P.knowlesi). There are two main steps: DNA extraction from blood samples and amplification of DNA extracts using primers pairs and probes that hydrolyze double-dye (doubledye hydrolysis probes) which are very specific. Double-dye hydrolysis probes are fluorescent substances that are able to emit light, then the light will be captured by optical detectors on the Rotor-Gene® Q device, which results in an increase in signal in the wave graph. This probe attaches to the primers of each Plasmodium species, which are compatible with the Rotor-Gene® Q optical detector channel, each consisting of FAM (compatible with Green channels), HEX (compatible with Yellow channels), ROX (compatible with Orange channels ) and Cy5 (compatible with Red channel), Quasar 705 (compatible with Crimson channel), VIC, TAMRA, TEXAS RED.

Results

Acquisition of samples

As many as 105 samples were obtained, which consisted of 67/105 (63.8%) were male and 38/105 (36.2%) were female. The average of age of patients was 31.19 years old. Median and long IQR fever obtained from 28 malaria patients who had a history of fever were five days (2-60) Microscopy examination resulted in 54 (51.42%) out of 105 samples were positive of Plasmodium, and 51 (48.58%) were negative. Examination using RightSign and ScreenPlus resulted in 51.42% were positive (Table 1).

Parasitemia index

Out of 54 positive samples, 34 (63%) were P. falciparum, 17(31.5%) were P. vivax and 3 (5.5%) were mix infections of both species. Based on parasite density, the highest parasitemia index was P. vivax which in range of 1000-10,000/μL, followed by P. falciparum of 1000-10,000/μL and 10,001-200,000/μL, while the mixed infection was evenly distributed as <1000, 1000-10,000 and 10,001-200,000/μL.

In this study, a high parasitemia index was obtained with each dominance of 1000-10,000 parasites/μL was 41.2% in P.falciparum and 52.9% in P.vivax, while the parasitemia index<1000 parasites/μL was 5.9% in P. falciparum and P. vivax (Table 1).

The negative samples

Fifty negative Plasmodium based on RDT examination, consisted of 25 (50%) samples with Dengue fever, 4 (8%) samples with urinary tract infections, 3 (6%) samples with pneumonia, and 4 (8%) samples with local infection, 3 (6%) samples with sepsis and 11 (22%) samples with hepatitis B.

Comparison of microscopy, two RDTs and RT-PCR

The diagnostic value of Right Sign and ScreenPlus against microscopy consisted Sn = 100% and Sp = 98%, PPV = 98, 2%, NPV = 100% (Table 2). The results of the sensitivity and specificity of both tests on gold standard microscopy in this study obtained high, namely 100% and 98%. The results showed a good compatibility between RightSign, ScreenPlus and microscopy. The diagnostic value of RightSign and ScreenPlus against Real- Time PCR Sn = 75.3 % and Sp = 100 %, PPV = 100%, NPV = 64% (Table 2).

McNemar test showed that no significance different between RightSign with microscopic examination with P=1. There was a significant difference between RightSign and RT-PCR Results (Tables 3 and 4). Table 5 shows the variety of Plasmodium species detected by RightSign vs. microscopic and Table 6 showed the distribution of Plasmodium species against RTPCR. RT-PCR detected more mix infection compared to microscopic, 24.8% vs. 2.9 %.

Tables 7 and 8 shows the comparison of the species of Plasmodium detected by ScreenPlus, microscopic and RT-PCR. The percentage was the same as the results of RightSign. McNemar statistical analysis on the performance differences between RightSign and ScreenPlus showed that there was no significant difference with P>0.05 (Tables 9 and 10). Mann Whitney analysis revealed no significant difference on parasitemia index between ScreenPlus and RightSign (P>0.05).

Discussion

The location for the sampling population in this study was Merauke, one of the districts in the Papua Province. Riskesdas in 2013 stated that Papua Province was one of the provinces with the highest malaria burden in Indonesia with Annual Parasites Incidence (API) of 45.85% in 2016. High malaria burden areas will affect the background of malaria exposure in the population, thus providing higher parasitic densities compared to non-endemic regional populations. ⁴ Parasitic density will produce a high antigenemia protein resulting in high positivity in the detection of RDT protein antigens, as well as the positivity of Plasmodium detection by microscopy.

Table 1.

Basic characteristics of subjects and study samples.

Variable	(n)	Total %
Total malaria	105	52.4
Positive microscopy test	54/105	51.42
Negative microscopy test	51/105	48.58
Positive rightsign test	54/105	51.42
Positive screenplus test	54/105	51.42
Parasitemia index P. vivax per μL blood*	34/54	63
<1000	2/34	5.9
1000-10,000	18/34	52.9
10,001-200,000	14/34	41.2
>200,000	0/34	0
Parasitemia index P. falciparum per μL blood*	17/54	31.5
<1000	1/17	5.9
1000-10,000	7/17	41.2
10,001-200,000	7/17	41.2
>200,000	2/17	11.7
Parasitemia index mixed (P. falciparum & P. vivax) per μL blood*	3/54	5.5
<1000
1000-10,000	1/3	33.3
10,001-200,000	1/3	33.3
>200,000	1/3	33,3
Age (Mean±SD) years	0/3	0
Gender	31.19 ± 17.97	-
Male	67/105	63.8
Female	38/105	36.2
Day of fever (Median (IQR)) days (n=28)	5 (2-60)	-

Table 2.

Diagnostic performance of RightSign and ScreenPlus compared to microscopic and real-time PCR Examination.

Diagnostic performance	RightSign		ScreenPlus
	Microscopic	RT-PCR	Microscopic	RT-PCR
Sn (%) (CI 95%)	100	75.3	100	75.3
Sp (%) (CI 95%)	98	100	98	100
PPV (%) (CI 95%)	98.2	100	98.2	100
NPV (%) (CI 95%)	100	64	100	64

Table 3.

Comparison of RightSign vs. microscopic examination.

Right Sign	Microscopic results		Total	P-value*
	Positive	Negative
Positive	54 (98.2%)	1 (1.8%)	55 (100%)	1.000
Negative	0 (0%)	50 (100%)	50 (100%)
Total	54 (51.4%)	51	(48.6%)	105 (100%)

^*Statistical analysis using McNemar test. N =105, significance at P<0.05.

Table 4.

Comparison of RightSign vs. PCR results.

Right Sign	PCR Results		Total	P-value*
	Positive	Negative
Positive	55 (100%)	0 (0%)	55 (100%)	< 0,001
Negative	18 (36%)	32 (64%)	50 (100%)
Total	73 (69.5%)	32 (30.5%)	105 (100%)

^*Statistical analysis using McNemar test. N =105, significance at P<0.05 .

RDT sensitivity was influenced by various factors which included location and population sampling, antigenemia protein in the sample itself and optimal temperature stability of the reagent kit.^{13, 14}

Optimal temperature stability influences Rapid diagnostic test (RDT) sensitivity (4-30ºC). If the optimal temperature is stable, during delivery transportation or during examination, the RDT performance will run well and provide valid results.¹⁴ In this study, the temperature stability of the reagent kit was maintained at 4-30^oC with the storage of reagent kits carried out in the refrigerator before being used. RDT sensitivity was also influenced by the suitability between observers in reading RDT results. Two observer concordance was analyzed by the McNemmar test with P-value>0.05 and no significant difference was found in the results. Arum I et al. showed similar results of RDT with this study. The diagnostic value of RDT against microscopy with Sn 100%, Sp 96.99%, PPV 83.2%, and NPV 100%.¹⁵

One false positive result of P. falciparum in RightSign and ScreenPlus in this study could be attributed to protein antigenemia HRP-2 malaria parasite that could still be identified in the blood of the patient for up to 30 days after antimalarial therapy due to slow clearance of HRP-2 in the blood. Another reason is the presence of gametocytes in the blood following antimalarial therapy. Gametocytes in the blood continue to produce all three antigenemia proteins (HRP-2, p-LDH and aldolase). Antigenemia p-LDH and aldolase proteins have faster clearance time from the blood after antimalarial therapy.^16,17 Rheumatoid factor and heterophilic antibodies in the patient’s blood are other cause of false positivity in the RDT.¹⁸

The results of the comparison between RDTs and gold standard microscopy examination were not statistically significantly different. The examiners involved in the microscopy examination may have influenced the results, however, the trained and certified malaria microscopy examiner should produce a true and reliable results.^10,12 Conformity between the results of RDTs and microscopy was also inseparable from the density of parasitemia. The population of malaria species in this study was dominated by P. vivax and then followed by P. falciparum, so that although there was a high parasitemia, but because of the dominance of malaria species occupied by P. vivax, the chance to produce false positives in pLDH was small. Patients with high P. falciparum parasitemia could give false positive results in pLDH and end with high P. vivax findings on RDT examination.¹⁹ The sensitivity of RightSign and ScreenPlus to RT-PCR was lower than the sensitivity to microscopy as the gold standard for malaria research, but the specificity is higher. A research in Flores reported that real-time PCR found that RT PCR was able to detect almost 8 times more cases of Plasmodium infection compared to microscopic examination as the gold standard for malaria testing. This is due to the ability of RT PCR to detect submicroscopic infections, with or without clinical malaria, that are not detected microscopically.^10,20 This condition may be due to the ability of RT PCR to detect DNA fragments of malaria parasites when microscopically detecting the existence of Plasmodium malaria.²¹

Table 5.

Comparison of species detection between RightSign and microscopic.

Right Sign		Microscopic			Total
	Negative	P. falciparum	P. vivax	Mix
Negative	50 (100%)	0 (0%)	0 (0%)	0 (0%)	50 (100%)
P. falciparum	1 (5.6%)	16 (88.9%)	16 (88.9%)	1 (5.6%)	18 (100%)
Pan	0 (0%)	1 (2.7%)	1 (2.7%)	2 (5.4%)	37 (100%)
Total	51 (48.6%)	17 (16.2%)	17 (16.2%)	3 (2.9%)	105 (100%)

Table 6.

Comparison of species detection between RightSign and RT-PCR.

Right Sign		RT-PCR			Total
	Negative	P. falciparum	P. vivax	Mix
Negative	32 (64%)	3 (6%)	10 (20%)	5 (10%)	50 (100%)
P. falciparum	0 (0%)	5 (27.8%)	0 (0%)	13 (72.2%)	18 (100%)
Pan	0 (0%)	0 (0%)	29 (78.4%)	8 (21.6%)	37 (100%)
Total	32 (30.5%)	8 (7.6%)	39 (37.1%)	26 (24.8%)	105 (100%)

Table 7.

Comparison of species detection between ScreenPlus and microscopic results.

ScreenPlus		Microscopic examination			Total
	Negative	P. falciparum	P. vivax	Mix
Negative	50 (100%)	0 (0%)	0 (0%)	0 (0%)	50 (100%)
P. falciparum	1 (5.9%)	17 (94.4%)	0 (0%)	0 (0%)	18 (100%)
P. vivax	0 (0%)	1 (2.7%)	34 (91.9%)	2 (5.4%)	37 (100%)
Total	51 (48.6%)	18 (16.2%)	34 (32.4%)	3 (2.9%)	105 (100%)

Table 8.

Comparison of species detection between ScreenPlus and RT-PCR.

ScreenPlus		PCR Results			Total
	Negative	P. falciparum	P. vivax	Mix
Negative	32 (64%)	3 (6%)	10 (20%)	5 (10%)	50 (100%)
P. falciparum	0 (0%)	5 (29.4%)	0 (0%)	13 (72.2%)	18 (100%)
P. vivax	0 (0%)	0 (0%)	29 (78.4%)	8 (21.6%)	37 (100%)
Total	32 (30.5%)	8 (7.6%)	39 (37.1%)	26 (24.8%)	105 (100%)

Table 9.

Comparison of ScreenPlus and RightSign.

RightSign results	Screen plus results		Total	P-value*
	Positive	Negative
Positive	55 (100%)	0 (0%)	55 (100%)	1,000
Negative	0 (0%)	50 (100%)	50 (100%)
Total	55 (52,4%)	50 (47,6%)	105 (100%)

^*Statistical analysis using McNemar test. N =105, significance at P<0.05.

Table 10.

Comparison of species detection between ScreenPlus and RightSign

RightSign Results	ScreenPlus Results			Total
	Negative	P. falciparum	P. vivax
Negative	50 (100%)	0 (0%)	0 (0%)	50 (100%)
P. falciparum	0 (0%)	18 (94.4%)	0 (0%)	18 (100%)
Pan	0 (0%)	0 (0%)	37 (100%)	37 (100%)
Total	50 (47.6%)	17 (16.2%)	37 (35.2%)	105 (100%)

Molecular methods are universally accepted to be more sensitive than microscopy, however PCR (multiplex PCR, real time PCR, or conventional) requires more sophisticated laboratories, trained personnel, longer times, and higher costs because PCR examination is not to be a routine examination of malaria. Malaria examination results can be more accurate if PCR is used as a reference standard for malaria testing because PCR method is able to detect parasitemia below the microscopic limit.

Conclusions

RightSign and ScreenPlus RDT have a very good performance. The analysis of Plasmodium antigen detection by RightSign, ScreenPlus, and microscopy research was not significantly different. Further research is needed to find out the diagnostic value of non-falciparum and non-vivax Plasmodium in RightSign and ScreenPlus. Real time-PCR detected higher mix infection when compared to microscopy and RDTs, therefore real-time PCR method can be considered as an effective confirmation tool for malaria diagnosis.

Funding Statement

Funding: The work was supported by Simlitabmas Programme by Kementretian Riset Teknologi dan Pendidikan Tinggi Republik Indonesia (KEMENRISTEKDIKTI RI), contract number: 544/UN3.14/LT/2019.