External quality assessment of Giemsa-stained blood film microscopy for the diagnosis of malaria and sleeping sickness in the Democratic Republic of the Congo

Abstract

Objective

To report the findings of a second external quality assessment of Giemsa-stained blood film microscopy in the Democratic Republic of the Congo, performed one year after the first.

Methods

A panel of four slides was delivered to diagnostic laboratories in all provinces of the country. The slides contained: (i) Plasmodium falciparum gametocytes; (ii) P. falciparum trophozoites (reference density: 113 530 per µl); (iii) Trypanosoma brucei subspecies; and (iv) no parasites.

Findings

Of 356 laboratories contacted, 277 (77.8%) responded. Overall, 35.0% of the laboratories reported all four slides correctly but 14.1% reported correct results for 1 or 0 slides. Major errors included not diagnosing trypanosomiasis (50.4%), not recognizing P. falciparum gametocytes (17.5%) and diagnosing malaria from the slide with no parasites (19.0%). The frequency of serious errors in assessing parasite density and in reporting false-positive results was lower than in the previous external quality assessment: 17.2% and 52.3%, respectively, (P < 0.001) for parasite density and 19.0% and 33.3%, respectively, (P < 0.001) for false-positive results. Laboratories that participated in the previous quality assessment performed better than first-time participants and laboratories in provinces with a high number of sleeping sickness cases recognized trypanosomes more frequently (57.0% versus 31.2%, P < 0.001). Malaria rapid diagnostic tests were used by 44.3% of laboratories, almost double the proportion observed in the previous quality assessment.

Conclusion

The overall quality of blood film microscopy was poor but was improved by participation in external quality assessments. The failure to recognize trypanosomes in a country where sleeping sickness is endemic is a concern.

Introduction

With 97% of its population living in areas of stable malaria transmission, the Democratic Republic of the Congo is among the countries with the highest malaria burden in Africa.^1,2 For diagnosing malaria, the microscopic detection of Plasmodium in thick blood films remains the gold standard, although reliable rapid diagnostic tests are increasingly available.³ In addition, microscopic examination of stained blood films also enables the detection of other pathogens, including Trypanosoma brucei ssp., some of which cause human African trypanosomiasis. Half of all people at risk of African trypanosomiasis live in the Democratic Republic of the Congo, where almost 80% of all reported cases have occurred.⁴

The microscopic detection of Plasmodium and Trypanosoma requires technical skill, thorough training, maintenance of expertise and regular handling of samples containing the parasites. The quality of microscopy, however, often remains inadequate because of poor-quality equipment, insufficient training or a lack of quality assurance.⁵ In 2002, the World Health Organization (WHO) launched an external quality assessment programme to improve the proficiency of microbiological testing for diseases prone to cause epidemics in the WHO African Region, including malaria.^6,7

In 2010, the first, national, large-scale external quality assessment in the Democratic Republic of the Congo was carried out in four of the country’s 11 provinces.⁸ It revealed the poor quality of malaria microscopy. In this paper, we present the results of the second, nationwide external quality assessment of stained blood film microscopy. Since trypanosomiasis is widespread in the Democratic Republic of the Congo and accurate diagnosis is important, the assessment included a slide with trypanosomes. In addition, given the increasing use of rapid diagnostic tests for malaria,² participants in the quality assessment were questioned about use of these tests.

The data obtained during the second external quality assessment of stained blood film microscopy in the Democratic Republic of the Congo demonstrate the benefits of such assessments, show they are feasible when carried out in collaboration with vertical, disease-oriented programmes and highlight the difficulty of diagnosing human African trypanosomiasis.

Methods

The second external quality assessment was organized by the Institut National de Recherche Biomédicale in Kinshasa, which is the reference laboratory for the National Malaria Control Programme (Programme national de lutte contre le paludisme) in the Democratic Republic of the Congo. Between July and October 2011, 356 clinical laboratories in the country’s 11 provinces were contacted. The laboratories had either subscribed to and participated in the previous external quality assessment, which was also organized by the Institut National de Recherche Biomédicale,⁸ or were part of the network of the National Malaria Control Programme or the National Tuberculosis Control Programme (Programme national de lutte contre la tuberculose). The number of human African trypanosomiasis cases in each province in 2011 is shown in Fig. 1.⁹ No data were available for Sud-Kivu because of the ongoing war and the number of cases was assumed to be under 100.

Fig. 1.

Laboratories participating in the external quality assessment^a and human African trypanosomiasis cases, by province, Democratic Republic of the Congo, 2011

^a The number of participating laboratories in each province is shown.

Source of shapefile: Institute of Tropical Medicine, Antwerp, Belgium.

The samples used in the external quality assessment comprised four Giemsa-stained, thick and thin blood films (Table 1), prepared as previously described.⁸ All blood samples were prepared using ethylenediaminetetraacetic acid (EDTA) and were left over from samples drawn for routine clinical care from patients presenting at the Institut National de Recherche Biomédicale. Slides containing trypanosomes were prepared from rodent blood infected with Trypanosoma brucei brucei and diluted in human EDTA blood. Thick and thin blood films were placed on a single slide (Menzel-Gläzer, Braunschweig, Germany). After fixation of the thin blood films with methanol, the films were treated with Giemsa stain (Merck, Darmstadt, Germany). For conservation, the blood films were covered with Entellan (Merck, Darmstadt, Germany) and a coverslip. Parasite density was determined at 1000-times magnification by six expert microscopists from the Institut National de Recherche Biomédicale and the Institute of Tropical Medicine in Antwerp, Belgium, and the reference value was taken to be the mean density ± 2 standard deviations (SDs). For all slides, the presence or absence of Plasmodium species and the species present were evaluated using real-time polymerase chain reaction methods.^10,11 The slides were sent to coordinators in each province who transported them to participating laboratories along with a teaching sheet explaining the procedure that should be used to determine parasite density. On detecting a malaria infection, participants were to report the Plasmodium species present, the developmental stage of the parasite and parasite density, expressed as trophozoites per µl. Other findings were to be reported as comments.

Table 1. Sample analysis, external quality assessment of laboratories, Democratic Republic of the Congo, 2011.

Samplea	Information accompanying sample	No. of laboratories reporting	No. of laboratory reports that contained:
			No error	Minor error	Major error
Slide 1: P. falciparum gametocytes	14-year-old patient with fever, abdominal pain, asthenia and diarrhoea	268	120	97 (Plasmodium infection not reported)	47 (Plasmodium infection reported but the developmental stage was not reported or incorrect)
				4 (gametocytes reported but no or wrong species identified)
Slide 2: no parasites	39-year-old patient, control visit	269	218	–	51 (Plasmodium infection reported)
Slide 3: Trypanosoma brucei ssp.	Sample received at Institut National de Recherche Biomédicale for scientific research	270	120	14 (both Plasmodium and Trypanosoma infection reported)	136 (Trypanosoma infection not reported)
Slide 4: P. falciparum trophozoites (density: 113 530 per µl)	44-year-old patient with general weakness, asthenia and headache	271	37	23 (reported P. falciparum infection but not developmental stage)	5 (Plasmodium infection not reported)
				103 (trophozoite density moderately below reference value)b	30 (wrong or no species reported)
				14 (trophozoite density moderately above reference range)b	6 (P. falciparum gametocytes reported)
				21 (no trophozoite density reported)	30 (trophozoite density well below reference range)b
					2 (trophozoite density well above reference range)b

P. falciparum, Plasmodium falciparum; SD, standard deviation; ssp., subspecies.

^a Giemsa-stained thick and thin blood films.

^b A reported trophozoite density within the reference range of 63 000 to 164 000 per μl (e.g. within 2 SDs of the reference value of 113 530 trophozoites per µl) was considered correct; a density between 11 353 and 62 999 per µl was moderately below the reference range; a density between 164 001 and 1 135 3-00 per µl was moderately above; a density of < 11 353 per µl was well below; and a density of > 1 135 300 per µl was well above.

Participants were also asked to complete a questionnaire on: (i) their use of microscopy and rapid diagnostic tests for malaria diagnosis; (ii) the number of tests performed each month; (iii) the proportion of test results that were positive; and (iv) the number of staff performing diagnostic tests and their training. In addition, information was requested about the type and brand of the rapid diagnostic tests used and occurrence of shortages.

The results of the microscopic analysis were categorized as: without errors; with minor errors; or with major errors (Table 1). The distinction between a minor and a major error was based on the potential effect the error could have on the patient's diagnosis and clinical management. A parasite density less than 2 SDs from the mean (reference) density, as determined by the expert microscopists, was considered to be without error. A density that lay outside this range was considered a major error if it was more than 10 times or less than one 10th the reference value; all other densities outside the range were considered minor errors. A slide analysis was considered correct if no error or a minor error was made and incorrect if a major error was made (Table 1). The target standard for the performance of the slide analysis was a correct result for each of the four slides. Data were entered into Excel spreadsheets (Microsoft, Redmond, United States of America) and analysed using Stata version 10.0 (StataCorp. LP, College Station, USA). Differences between proportions were tested for significance using the χ² or Fisher exact test and differences between two groups of continuous variables were tested for significance using the Mann–Whitney rank–sum test. Trends in proportions were assessed using the χ² test for trends.

Results

Responses were received from 277 of the 356 (77.8%) laboratories contacted. Of the 277 responders (Fig. 1), 11 (4.0%) were provincial reference laboratories, 109 (39.4%) were general hospitals, 143 were health centres (51.6%) and 14 (5.1%) were private laboratories. Nationally, coverage was 91.7% for provincial reference laboratories (11 of 12), 27.7% for hospitals (109 of 393) and 1.7% for health centres (143 of 8266). Coverage for private laboratories was not known. The median time taken for the quality assessment samples to arrive at participating laboratories was 27 days (range: 1–109). Reports were returned with a median delay of 7 days (range: 1–83).

Sample analysis

Table 1 summarizes the results of the analysis of the four external quality assessment samples at participating laboratories. Slide 1, which contained Plasmodium falciparum gametocytes only, was read without errors by 44.8% of participants, whereas 37.7% made a minor error and 17.5% made a major error. Slide 2, which contained no parasites, was scored as negative by 81.0% of participants but the remaining 19.0% made the major error of reporting the presence of Plasmodium. Slide 3, which contained trypanosomes, was reported correctly by 44.4% of participants. In addition, 5.2% reported the presence of trypanosomes but also malaria infection, which was considered a minor error. However, 50.4% missed the diagnosis of trypanosomiasis. Of the 136 participating laboratories that did not report trypanosomes, 51.5% made no comment, 44.9% reported the presence of Plasmodium, 1.8% could not identify the parasite or reported the presence of an unusual parasite form, 0.6% suggested that Loa loa microfilaria were present and 0.6% reported fibres on the slide. Slide 4 contained P. falciparum trophozoites at a reference density of 113 530 per µl. Only 13.7% of participating laboratories recorded a parasite density less than 2 SDs from the mean (i.e. 63 000 to 164 000 per μl), which was considered without error; minor errors were made by 59.4% and major errors, by 26.9%. The trophozoite densities reported ranged between 32 and 24 935 064 per µl (Fig. 2). The most common error, made by 49.1%, was to report a parasite density below 63 000 per µl. Finally, 11.1% of participants estimated the density to be less than one 10th the reference value (i.e. < 11 353 per µl), whereas 0.7% estimated it to be more than 10 times the reference value (i.e. > 1 135 300 per µl).

Fig. 2.

Laboratory measurement of Plasmodium falciparum trophozoite density on test slides, external quality assessment, Democratic Republic of the Congo, 2011

RV, reference value; SD, standard deviation.

^a The RV is 113 533 trophozoites per µl.

^b One tenth of the RV is 11 350 trophozoites per µl.

^c –2 SD from the RV is 63 000 trophozoites per µl; +2 SD from the RV is 164 000 trophozoites per µl.

Note: The RV and SD for the trophozoite density were determined from measurements by a group of expert microscopists.

Results for all four slides were received from 263 participating laboratories; 92 (35.0%) reported a correct result (i.e. no error or a minor error) on all slides (Fig. 3). However, 3.0% reported no correct results and 11.0% reported only one correct result. Significantly fewer laboratories reported a correct result for slide 3, which contained trypanosomes: 50.2% reported a correct result compared with 73.4 to 82.1% for the other slides (P = 0.02).

Fig. 3.

Venn diagram of laboratories^a reporting correct results for individual test slides and their combination, external quality assessment, Democratic Republic of the Congo, 2011

P. falciparum, Plasmodium falciparum; ssp., subspecies.

^a The results illustrated are for the 263 participating laboratories that returned all four test slides.

Note: The eight laboratories in the green zone had no correct results.

Questionnaire results

Of the 271 laboratories that responded, 257 (94.8%) reported that they usually diagnosed malaria by thick blood film microscopy. The median number of slides processed was 159 per month (interquartile range, IQR: 86–352; maximum: 2532), by a median of 2 (IQR: 1–3) laboratory technicians. The median proportion that tested positive was 51.9% (IQR: 31.0–70.5). Only 100 of the 271 (36.9%) laboratories had taken part in training on malaria microscopy; the median time since training was 3 years (IQR: 1–5).

Rapid diagnostic tests for malaria were used by 44.3% (120/271) of the participating laboratories. The median number of tests performed was 160 per month (IQR: 36–188; maximum: 1283), by a median of 4 (IQR: 2–6) people. The median proportion that tested positive was 40.0% (IQR: 24.2–58.0). Overall, 64 of the 120 (53.3%) laboratories had taken part in training on rapid diagnostic tests, mostly in 2010, and the median length of experience was 8 months (IQR: 6–12). In 13.3% of the laboratories, rapid tests had replaced microscopy for malaria diagnosis. Stock disruption was reported by 30.8%. In addition, 29.2% of the laboratories used rapid diagnostic tests during night shifts, 7.5% used them in emergencies and 2.5% used them for ensuring transfusion safety, whereas 35.0% did not specify the circumstances in which they used the tests. The most common test, used by 75% of the laboratories, was an SD Bioline malaria rapid test (Standard Diagnostics Inc., Hagal-Dong, Republic of Korea), followed by the Paracheck Pf (Orchid Biomedical Systems, Goa, India), used by 14.2%.

Factors affecting analytical performance

Whether a laboratory regularly used microscopy or rapid diagnostic tests for malaria diagnosis had no significant effect on the proportion that reported a correct result for all four slides: 85 of 240 laboratories that used microscopy reported four correct results compared with 2 of 11 laboratories that used only rapid tests (P = 0.3). Moreover, laboratories that had taken part in training did not have better results than those that had not (P = 0.09). There was no significant difference between laboratories that had four correct results and other laboratories in the median number of slides processed per month: 166 per month (IQR: 88–438) versus 152 per month (IQR: 83–286), respectively (P = 0.2). In particular, laboratories that processed more than 300 slides per month did not perform significantly better than those that processed fewer (P = 0.052). Nor did higher-level referral centres perform significantly better: 54.5% of provincial reference laboratories correctly read all four slides, compared with 42.3% of hospitals and 30.6% of health centres (P = 0.08 for trend). However, laboratories that were participating in an external quality assessment for the second time did perform significantly better: 64 of 152 (42.1%) laboratories participating for the second time reported four correct results compared with 28 of 111 that were participating for the first time (25.2%; P = 0.005). Finally, 110 of 193 (57.0%) laboratories in provinces with more than 100 African trypanosomiasis cases annually (Fig. 1) correctly reported trypanosomes on slide 3, compared with 24 of 77 (31.2%) laboratories in provinces with fewer cases (P < 0.001).

There were significantly fewer serious errors in the second external quality assessment than in the first.⁸ In the present quality assessment, 51 of 269 (19.0%) laboratories reported false-positive results for the slide with no parasites, compared with 58 of 174 (33.3%) in the first assessment (P < 0.001). Major errors in parasite density estimates were also less frequent in this assessment: 32 of 186 (17.2%) laboratories compared with 23 of 44 (52.3%) in the first external quality assessment (P < 0.001).

Discussion

Overall, the quality of Giemsa-stained blood film microscopy observed in the present external quality assessment was poor. The principle serious errors were: not diagnosing trypanosomiasis; not recognizing P. falciparum gametocytes; diagnosing malaria from a slide with no parasites; and substantial quantitative errors in parasite density estimates.

The present assessment had several limitations. The diagnostic competence of the participating laboratories may have been overestimated and might not be representative of all laboratory staff involved because the most experienced staff may have read the slides for the external quality assessment.¹² In addition, the test slides were stained before receipt and staining quality is known to be poor in routine laboratory settings.⁸ Another limitation stems from the estimated distribution of human African trypanosomiasis cases in different provinces. The disease is highly focal and, in particular, the risk varies greatly across Orientale, Équateur, Katanga and Maniema provinces, which contain large areas where there is no risk.¹³ Consequently, a laboratory may serve a population that lives either completely inside or outside an area where there is a disease risk and, therefore, its experience with the disease may not be typical of the province in which it is located.

A major strength of this external quality assessment was the synergistic collaboration with existing government networks and vertical programmes, in particular with the national tuberculosis control programme.¹⁴ We found that the quality assessment could be carried out without the need for new structures or organizations. Working with existing networks and programmes is less expensive and more sustainable. Moreover, over the long term, it is probably more feasible to test diagnostic proficiency by external quality assessment than by using a quality control system to cross-check routine blood slides, which can result in a heavy workload.¹⁵ Despite the logistic challenges of organizing external quality assessments in poorly accessible areas of countries lacking in infrastructure, such as the Democratic Republic of the Congo, the response time was not much longer than that reported by WHO for quality assessments of national public health laboratories in other parts of Africa.⁷

Another advantage of external quality assessments is that they are usually conceived as didactic programmes: they often include a feed-back report that explains the background to and pitfalls of testing and refreshes knowledge of state-of-the-art techniques. The present quality assessment involved the distribution, at little cost, of a report on the previous external quality assessment and a teaching sheet, both of which helped encourage participation. However, external quality assessment programmes do not replace training. Since 2011, the National Malaria Control Programme in the Democratic Republic of the Congo has updated technical guidance on malaria diagnosis. The education of trainers has started at the provincial level and, to date, training has been carried out at over 100 health-care facilities in four provinces. Unfortunately, the lack of infrastructure in the country is a limitation. Nevertheless, our findings show that performance can be improved: (i) laboratories that participated in an external quality assessment for a second time performed better; (ii) there were fewer false-positive results than in the previous assessment; and (iii) the number of major quantitative errors in estimating parasite density was lower than previously. These observations confirm findings in the United Kingdom.¹²

External quality assessments also provide the opportunity to introduce laboratories to new or rarely-seen parasites.¹² As observed in WHO external quality assessments, correctly identifying blood parasites other than P. falciparum can be a problem.⁷ That half of participating laboratories in our assessment did not recognize the presence of trypanosomes is a concern, especially since the diagnosis of African trypanosomiasis in the Democratic Republic of the Congo will increasingly shift from active case-finding by mobile teams to integration into the horizontal health-care system.⁴ In contrast, ability to identify trypanosomes was relatively good in the slides included in the external quality assessments performed by WHO⁷ and in the United Kingdom¹². However, most participants in WHO’s assessments were national reference laboratories, where training and reference documentation may be better.

In WHO’s assessments, an estimated parasite density in the target range of 0.5 to 1.5 times the reference value was regarded as acceptable. This range corresponds approximately to our definition of no error. Overall, 13.7% of laboratories in our assessment made no error in their parasite density estimate compared with 51% that reported an acceptable value in the WHO assessments.⁷ In both the present and previous external quality assessment in the Democratic Republic of the Congo,⁸ parasite density tended to be underestimated. One reason may be that density has been expressed in parasites per µl in the country only since 2010. The distribution of teaching sheets during the current external quality assessment may explain why there were fewer errors in parasite density estimates than in the previous assessment.

In addition, the proportion of false-positive results for Plasmodium in the current assessment (i.e. 19%) was lower than previously but higher than the 8 to 15% and the 2% reported, respectively, by two external quality assessments carried out in settings where malaria is not endemic.^12,16 Although the laboratories in our assessment should have encountered Plasmodium more often, they were probably less familiar with external quality assessments and expected the slides in a malaria microscopy assessment to contain parasites. On the other hand, the proportion of false-positive results was much lower than that observed in a study of the sensitivity and specificity of rapid diagnostic tests and malaria microscopy carried out in 2011 in the Democratic Republic of the Congo, in which the false-positive rate for routine microscopy was 70.6%.¹⁷ The difference may be due to the poor quality of Giemsa staining and of the reagents used under routine conditions^8,18 or to the possibility that better laboratories are more likely to participate in external quality assessments.

In contrast to previous and other external quality assessments, we found no clear association between analytical performance and how regularly a laboratory handled infected samples. In areas where malaria is endemic, laboratory staff may acquire the necessary expertise even if they only assess 40 slides per month. The difference in performance between the current and previous external quality assessments in the Democratic Republic of the Congo could be due to differences in participants or to an overall improvement.

The results of the questionnaire were similar in current and previous assessments. The lack of training remains of concern. Notably, the use of rapid diagnostic tests has almost doubled since 2010, when they were used by only 24.7% of participants. However, there seems to be no clear strategy for deciding when to use them. The upward trend in the Democratic Republic of the Congo appears to reflect that observed throughout Africa.²

In conclusion, the overall quality of stained blood film microscopy in the Democratic Republic of the Congo was poor. In particular, the failure to recognize trypanosomes in a country where sleeping sickness is endemic is concerning. Since the use of rapid diagnostic tests has risen, relevant external quality assessments are increasingly important. Participating in an external quality assessment appeared to improve a laboratory’s performance but this needs to be confirmed by longer follow up. Although organizing quality assessments and training is challenging in vast, resource-poor countries, it is feasible when carried out in collaboration with existing government networks and vertical programmes. Moreover, external quality assessments can provide opportunities for educational activities, such as distributing teaching sheets, and the involvement of major stakeholders can increase awareness of the importance of these assessments, ensure they are sustainable and create opportunities for continued funding.