Performance of Clinical Laboratories in South African Parasitology Proficiency Testing Surveys between 2004 and 2010

Abstract

Performance in proficiency testing (PT) schemes is an objective measure of a laboratory's best performance. We examined the performance of participants in two parasitology PT schemes in South Africa from 2004 through 2010. The average rates of acceptable scores over the period were 58% and 66% for the stool and blood parasite schemes, respectively. In our setting, participation in PT alone is insufficient to improve performance; a policy that provides additional resources and training seems necessary.

TEXT

Proficiency testing (PT) schemes are a vital component of quality assurance systems in medical laboratories, providing an external and objective evaluation of their best performance. Satisfactory participation in PT schemes is required for ISO15189 accreditation of diagnostic laboratories (5). Consistently good PT performance provides assurance to clients (patients, doctors, and funders) that routine test results are accurate, reliable and of high quality (10). PT schemes provide blinded challenges for processing by routine methods. Microbiology PT schemes are complex, time consuming, and labor intensive to coordinate; therefore, they need to be fruitful undertakings (11). The Parasitology Reference Laboratory of the National Institute for Communicable Diseases in South Africa currently coordinates several national and international PT schemes.

Two national PT schemes are stool and blood parasitology for South African medical laboratories (public and private), as well as laboratories in Namibia, Swaziland, and other African countries. PT surveys are provided three times per year, comprising between two and five blinded challenges. Stool parasite challenges include stool and urine suspensions for microscopy and fixed stool smears for staining and microscopy. Blood parasite challenges include Giemsa-stained blood films for microscopy and unstained blood films for staining and microscopy, as well as quantitation of Plasmodium falciparum parasites. Occasionally, whole-blood aliquots are included to assess performance of rapid malaria antigen tests. Participants are periodically requested to return self-stained slides for assessment of staining quality.

Approximately 10 referee laboratories are selected annually among participants that perform consistently well, to provide a benchmark for the level of assessment and to monitor the quality of challenges. Using the referees' results as a standard, some challenges may be assessed less strictly or not at all if there is insufficient referee consensus (<70%). Challenges are evaluated according to an established scoring protocol (12). Participants receive individual reports with the expected results and their assessment, as well as a detailed teaching commentary and a corrective action form to record reasons for poor performance. South African public laboratories are expected to submit corrective action forms to their regional managers.

For this report, we analyzed the results of participating laboratories in the two national parasitology PT schemes from 2004 through 2010 to determine performance trends. The range of challenges is shown in Tables 1 and 2. The minimum acceptable response for a survey (75%) was the classification criterion. For all 21 surveys of both PT schemes, the proportions of participants with acceptable responses were analyzed for trends (Fig. 1 and 2) by using Chi-square tests for linear trend (EpiInfo 6; CDC, Atlanta, GA).

Table 1.

Challenges included in the stool parasite PT scheme, South Africa, 2004 to 2010

Challengesa	No. of times challenge was sent	Total no. of responses scored	No. of acceptable responses	% of acceptable responses
Ascaris lumbricoides ova	2	368	291	79
Blastocystis hominis cysts	3	580	393	68
Cryptosporidium species oocysts (Ziehl-Neelsen)	11	2,014	1,103	55
Cyclospora cayetanensis oocysts (Ziehl-Neelsen)	1	158	73	46
Cystoisospora belli oocysts (Ziehl-Neelsen)	8	1,324	894	68
Entamoeba coli cysts	5	997	635	64
Entamoeba histolytica/E. dispar cysts	2	423	221	52
Enterobius vermicularis ova	1	207	179	86
Giardia lamblia cysts	6	1,190	761	64
Hymenolepis nana ova	1	217	143	66
Cystoisospora belli oocysts	1	132	81	61
Schistosoma hematobium ova	3	548	426	78
Schistosoma mansoni ova	1	135	99	73
Strongyloides stercoralis larvae	3	661	431	65
Taenia species ova	6	1,107	783	71
Mixed infection: helminth + protozoan	5	934	636	68
Mixed infection: 2 coccidians	3	648	470	73
Mixed infection: 2 helminths	4	798	639	80
Mixed infection: 2 protozoa	1	157	72	46
No parasite present	9	1,764	1,127	64
No parasite present (Ziehl-Neelsen)	8	1,253	735	59
Ziehl-Neelsen stain assessment	3	630	295	47

^aIncludes paper challenges and excludes challenges that were not assessed.

Table 2.

Challenges included in the blood parasite PT scheme, South Africa, 2004 to 2010

Challengea	No. of times challenge was sent	Total no. of responses scored	No. of acceptable responses	% of acceptable responses
No parasite present	13	2,432	1,648	68
Plasmodium falciparum	23	4,669	3,240	69
P. falciparum thin-film parasite count	12	2,263	1,327	59
P. falciparum thick-film parasite count	1	198	72	36
Plasmodium malariae	3	586	335	57
Plasmodium ovale	4	845	592	70
Plasmodium vivax	1	213	130	61
Relapsing malaria species	1	211	127	60
Mixed infection (P. falciparum + P. malariae)	1	158	86	54
Trypanosoma brucei species	8	1,518	1,102	73
Microfilariae (Wuchereria bancrofti, Loa loa, and Mansonella perstans)	3	570	293	51
Relapsing fever, Borrelia spp.	2	357	181	51
Malaria rapid antigen test	3	475	300	63
Stain assessment	5	981	367	37

^aIncludes paper challenges and excludes challenges that were not assessed.

Fig 1.

Percentage of participants with acceptable results (≥75%) in the stool parasite PT scheme, South Africa, 2004 to 2010. Dotted line represents linear trend.

Fig 2.

Percentage of participants with acceptable results (≥75%) in the blood parasite PT scheme, South Africa, 2004 to 2010. Dotted line represents linear trend.

The trend in the proportion of acceptable results over the period was slightly but significantly positive for the stool parasite scheme (P = 0.02) but neutral for the blood parasite scheme (P = 0.62). For the stool parasite scheme, there was a period of relative stability in 2007 to 2009, but no other steady tendencies were observed (Fig. 1). The average rate of acceptable results was 58%, excluding laboratories that did not submit results (24% of the total enrolled). Similarly, participant performance for the blood parasite scheme fluctuated markedly (Fig. 2); the average rate of acceptable results was 66%, excluding nonresponding laboratories (28% of the total). There was a marked increase in participants over the 7-year period, from 135 to 236 and from 134 to 242 for the stool and blood parasite schemes, respectively.

The average rates of acceptable results were disappointingly low for the two schemes. In general, collectively poor performance was due to difficult challenges, while collectively good performance resulted from easily recognizable parasites and lenient or nonassessment of challenges due to lack of referee consensus.

In the stool parasite scheme, poor performance was often associated with misidentification of protozoal cysts and misidentification of yeasts on Ziehl-Neelsen-stained stool smears as oocysts of Cryptosporidium species or Cyclospora cayetanensis. For the blood parasite scheme, low rates of acceptable results were associated with parasite counting and with staining challenges. Many laboratories, lacking required knowledge, experience, and resources, struggled with parasite counts. About 55% to 80% of participants failed to return their slides for stain assessment, despite the 4-week deadline.

PT challenges are intended to be treated in the same manner as routine specimens, but generally they are given special attention (8), and the results are likely to reflect laboratories' best proficiency (4). Poor PT performance and nonresponders suggest a high probability of poor routine work performance. Consequently, the suboptimal PT performance of laboratories is alarming; especially as PT challenges are generally straightforward, with few unusual or atypical specimens. However, we did not analyze individual laboratory performance over time, precluding identification of individual consistent improvement or deterioration.

The lack of marked collective improvement of participants in the parasitology PT schemes is probably due to a number of factors. Diagnostic laboratories in sub-Saharan Africa face many difficulties (9). The shortage of qualified laboratory staff and high staff turnover hamper the accumulation of knowledge and skills (1). In addition, clinical parasitology training during tertiary education and employment is very basic (7). There is high variability between laboratories in the type and quality of specimens analyzed (4). Another major factor is the frequent lack of essential resources, such as good microscopes with measuring eyepieces, textbooks, and atlases.

Two similar parasitology PT schemes in the United Kingdom showed that participation in their schemes improved the standard of diagnostic parasitology (6). UKNEQAS assessed individual participant results of challenges that were repeated over time, whereas we looked at overall survey results. It is possible that their participants have greater access to training and resources than ours. As PT provider and the national reference laboratory, we have attempted to improve the overall quality of parasitology diagnosis through additional parasitology training courses across the region.

As part of a quality management system (3), PT schemes are useful tools, but participation in them alone is inadequate to improve laboratory performance in South Africa. Proper resources and training in parasitology are needed, as well as low staff turnover and a culture of quality that is supported by management. Poor performance is an indicator for urgent managerial intervention, and we recommend that such laboratories suspend parasitology testing until they can demonstrate acceptable performance in the PT schemes.