External Quality Assessment in the Evaluation of Laboratory Performance of Faecal Culture

Abstract

In Finland, all laboratories carrying out diagnostics of infectious diseases in humans are approved by the Regional State Administrative Agencies and are obligated to participate in External Quality Assurance rounds. Performance in these rounds is thought to reflect the quality of laboratory work. In the 6-year study period, 17 Finnish laboratories received 48 simulated faecal specimens for the culturing of diarrhoeal pathogens, yielding altogether 586 faecal culture External Quality Control specimens and 581 reports. The results were correct in 92% of all reports and in 67% of all specimens. False-negative Salmonella results were given for 2 of the 18 specimens, one with biochemically atypical Salmonella strain and the other with a low count of Salmonella cells. False-negative Shigella report was given for 6 of the 7 specimens in some participating laboratory. Detection of all common faecal pathogens is especially relevant to patient safety, public health, and epidemiological surveillance.

Introduction

Reliable laboratory diagnostics is the basis of good patient care and safety. When diagnostics concern infectious diseases such as foodborne infections or diarrhoeal diseases, it has major epidemiological implications, including a role in recognising disease outbreaks and assessing prevention and control effectiveness.

The Finnish Communicable Disease Act¹ stipulates that investigations needed for diagnosing communicable diseases are to be conducted at the National Institute for Health and Welfare (THL) and other laboratories approved for this purpose by the Regional State Administrative Agencies (RSAAs). The detailed procedures for implementing the legal regulations, generally called the licensing system for clinical microbiology laboratories, were created in 1993. The main purpose was to validate the reliability of laboratory diagnostics of infectious diseases independent of the laboratory performing the diagnostics.

Currently, more than 750 Finnish clinical laboratories are approved for performing diagnostics for infectious diseases. Most of them are small health care centres, performing only a few tests, mostly so-called point-of-care tests. Twenty-one laboratories are specialised clinical microbiology laboratories with an approval to conduct a vast array of investigations, including, among others, the cultivation of faecal specimens.

One valid tool to monitor the quality of laboratory work is to follow the laboratories’ success in External Quality Assessment (EQA), which is generally said to reflect the daily routine performance of a laboratory. The aim of this study was to evaluate the quality performance of laboratories cultivating human faecal specimens for Salmonella, Shigella, Campylobacter, Yersinia species, and enterohaemorrhagic Escherichia coli (EHEC). To this end, the results of the External Quality Control (EQC) specimens in EQA rounds during the years 2009-2014 were analysed.

Materials and Methods

Approval of clinical laboratories

The clinical microbiological laboratories are approved by the Regional State Administrative Agencies (RSAAs). The RSAA requests an expert statement on laboratory preconditions from THL before the approval of a laboratory. The preconditions for approval of a laboratory include that its quality control is organised appropriately, ie, the laboratory must participate in at least 4 EQA rounds annually for each clinical microbiology investigation they offer for sale. By request, the clinical microbiology laboratories must give RSAAs and THL all relevant information relating to their microbiological activities, including all data on their EQC results.¹ The approval is generally valid for 3 years² and it is given only for investigations listed on the approval. The list of the names of all laboratory investigations (the Nomenclature of Laboratory Investigations) is maintained at the Finnish Local and Regional Authorities.³ The Nomenclature defines the microbes that should at least be included in the faecal culture panel in all Finnish clinical laboratories.

EQA schemes for faecal bacterial pathogens

In Finland, EQA schemes are commonly bought from Labquality Ltd (www.labquality.fi), which is a Finnish company specialised in producing a wide range of EQA services since 1971. The Faecal Culture Scheme, started in 2001, consists of 2 EQC specimens that are sent 4 times a year to every laboratory that has ordered this scheme. These EQC specimens are lyophilised simulated faecal samples. They are designed by a Finnish clinical microbiology expert and manufactured according to the quality standards ISO 9001 and ISO 17043. Before the specimens are sent to the laboratories, the viability of the bacteria in the specimen is checked by the national reference laboratory at THL. Depending on the EQA round, the specimens may include common diarrhoea-causing bacteria Salmonella, Shigella, Campylobacter, and/or Yersinia. This is also the minimum content of the faecal culture panel in Finland.³ The same EQC specimens may also contain other faecal bacterial pathogens such as EHEC or Vibrio species, which should be suspected based on the clinical background information given for these EQC specimens. In 2009- 2014, a total of 48 EQC stool specimens were sent from Labquality Ltd to each participating laboratory. Forty-three of these specimens contained 1 or 2 bacterial faecal pathogens, and the remaining 5 specimens contained normal faecal flora (Table 3).

Table 3.

Success of Finnish and European laboratories in EQA rounds of faecal culture (including Salmonella, Shigella, Campylobacter, Yersinia, EHEC) during the study period of 2009–2014.

Expected pathogen Yearly occurrence in EQC specimens		No. of specimens (n = 48)	Finnish laboratories (n = 17)	Other European laboratories (n = 17-90)a
Salmonella serotype		18
1b/2012, 1/2013	S. Abony	2	100%	89%–90%
1/2014	S. Agona	1	100%	98%
1/2010, 1/2011, 1/2012, 1/2014	S. Enteritidis	4	100%	90%–100%
1/2010	S. Give	1	100%	97%
1/2009	S. Infantis	1	67%	82%
1/2009	S. Poona	1	100%	95%
3/2010, 1/2011, 1/2013, 1/2014	S. Typhimurium	6	33%–100%	47%–98%
1/2011, 1/2012	S. Virchow	2	100%	92%–99%
Shigella spp.		7
1/2009, 1/2014	S. sonnei	2	67%–75%	63%–76%
1/2011, 1/2012, 2/2013, 1/2014	S. flexneri	5	83%–100%	60%–88%
Campylobacter sp.		5
1/2009, 1/2010, 1/2012, 1/2013, 1/2014	C. jejuni	5	93%–100%	11%–55%
Yersinia sp.		6
1/2009, 1/2011, 1/2012, 1/2013, 1/2014	Y. enterocolitica	5	92%–100%	45%–77%
1/2014	Y. pseudotuberculosis	1	92%	78%
EHEC		1
1/2010	E. coli O157	1	93%	45%
Two pathogens		6
1/2009	S. Typhimurium, C. coli	1	100%	15%
1/2009	S. flexneri, Aeromonas hydrophila	1	25%	9%
1/2011	S. Typhimurium, S. boydii	1	50%	43%
1/2012	S. Enteritidis, C. jejuni	1	100%	15%
2/2013	S. Enteritidis, S. Agona	2	100%	96%
Negative (normal faecal flora)		5
1/2009, 1/2010, 2/2011, 1/2012		5	92%–100%	89%–96%

Abbreviations: EHEC, Enterohaemorrhagic Escherichia coli; EQC, External Quality Control.

Vibrio strains were not included in the study period. Results are as a percentage of correct results of total number of reports received.

^aThe number of European laboratories varied considerably between rounds even within the same year.

^bThe number of particular pathogen(s) in the particular year.

Participants

Of the 21 Finnish specialised clinical microbiology laboratories, 17 participated during the study period in the faecal EQAs organised by Labquality Ltd. These laboratories included 4 university or university hospital laboratories, 10 central hospital laboratories, and 3 private laboratories. Eight of the laboratories participated in all rounds. The same EQC specimens were also sent to between 17 and 90 laboratories in 11 to 13 other European countries. In Finland, the specialised clinical microbiology laboratories will culture faecal specimens routinely for all the 4 bacteria mentioned above. In other European countries, Campylobacter and/or Yersinia species are not always searched for in routine faecal specimens and, thus, not in faecal EQC specimens either. For cultivation and investigation of the EQC specimens, each laboratory was asked to use the methodology they normally use in their everyday work with human specimens received as part of health care.

Evaluation of EQC results

The EQC results sent by the laboratories to Labquality Ltd are confidential. Therefore, access to the Labquality Ltd database by the clinical microbiology expert at THL was possible only after receiving permission from each participating Finnish laboratory. Further information needed for the study was collected from THL’s registers. The evaluation of the EQC results of other European laboratories was collected from the summary reports of Labquality Ltd.

The EQC result was considered correct when the expected pathogen (or pathogens) was found. The identification of the Salmonella to the genus level was acceptable based on the definition policy of the National Task Force on Bacterial EQA, whereas other pathogens were identified to the species level. The quality performance of the Finnish laboratories was evaluated based on the number and proportion of correct and false results reported to Labquality Ltd. In addition, the performance of Finnish and other European laboratories participating in the same EQA schemes of Labquality Ltd were compared. Only the final result, correct or incorrect, was meaningful to this study; the specific methodology of the laboratories used to obtain the result was not evaluated.

Statistical methods

The Fisher exact test and χ² test were used for statistical analyses. A P value below 0.05 was considered to indicate statistical significance.

Results

EQC results of Finnish laboratories

Ten central hospital laboratories, 4 university or university hospital laboratories, and 3 private laboratories took part in this faecal culture scheme. Over a period of 6 years, the total number of laboratories was 17. In the years 2009, 2011, and 2014, the number of participants was 13, and in the years 2010, 2012, and 2013, the number was 12. The number of faecal EQC specimens was 48, and the total number of these specimens sent by Labquality Ltd to the participating laboratories was 586. Of these, 344 specimens were sent to central hospital laboratories, 154 to university laboratories, and 88 to private laboratories. The university and private laboratories reported their results to Labquality Ltd in 100% of the specimens they received, whereas the corresponding number in central laboratories varied annually from 93% to 100%. Of the reported 581 results, the proportion of the correct results was statistically similar in all laboratories: 91% in central hospital and 94% both in university and private laboratories. The overall percentage of the correct results was 92%. During the years 2009-2014, it varied from 80% to 96% in central hospital laboratories, 79% to 100% in university laboratories, and 81% to 100% in private laboratories (Table 1).

Table 1.

Number of faecal External Quality Control specimens in 2009–2014 submitted by Labquality to participating laboratories (8 specimens/laboratory/year), their reports received, and correct results in those reports.

Year	Laboratory type	Central hospital (n = 10)	University/university hospital (n = 4)	Private (n = 3)	Total (n = 17)
2009	Reports received	56 (100%)	24 (100%)	16 (100%)	96 (100%)
	Correct results	45 (80%)	21 (88%)	13 (81%)	79 (82%)
2010	Reports received	52 (93%)	24 (100%)	24 (100%)	100 (96%)
	Correct results	49 (94%)	19 (79%)	22 (92%)	90 (90%)
2011	Reports received	56 (100%)	24 (100%)	16 (100%)	96 (100%)
	Correct results	51 (91%)	23 (96%)	16 (100%)	90 (94%)
2012	Reports received	56 (100%)	26 (100%)	12 (100%)	94 (100%)
	Correct results	54 (96%)	26 (100%)	12 (100%)	92 (98%)
2013	Reports received	48(100%)	32 (100%)	12 (100%)	92 (100%)
	Correct results	45 (94%)	32 (100%)	12 (100%)	89 (97%)
2014	Reports received	71 (99%)	24 (100%)	8 (100%)	103 (99%)
	Correct results	66 (92%)	23 (96%)	8 (100%)	97 (93%)
Total	Reports received/specimens sent	339/344 (99%)	154/154 (100%)	88/88 (100%)	581/586 (99%)
	Correct results	310 (91%)	144 (94%)	83 (94%)	537 (92%)

All the participants correctly reported 32 (67%) of the 48 specimens. Discrepant results were given for 16 specimens (Table 2). Two false reports were caused by mixing the specimens (Yersinia enterocolitica and Campylobacter jejuni). All laboratories found Campylobacter in the 5 specimens where it was present. Enterohaemorrhagic Escherichia coli is not investigated in all participating laboratories, but based on the brief clinical background information given in the accompanying instruction letter, the expected result was either a correct culture result or ‘not tested, sample will be sent to another laboratory for further analysis’. Only 1 laboratory reported it incorrectly. Salmonella was present in 18 specimens, and 16 (89%) of them were reported correctly by all participants. Two of them (Salmonella Infantis and Salmonella Typhimurium) were assessed to be negative in 12 laboratories. Shigella was present in 7 specimens, and only in 1 of them it was correctly reported by all participants. Six of the 7 Shigella specimens were reported as false-negatives by at least one of the participants. Six specimens included 2 pathogenic bacterial species, and 2 of these specimens were correctly reported by all participants. Only 3 (25%) laboratories reported both Shigella and Aeromonas in 2009 and 6 (50%) both Shigella and Salmonella in 2011. The 2 specimens with 2 Salmonella serogroups (Enteritidis and Agona) in 2013 were considered correct when Salmonella was found, and this was reported correctly by all participants.

Table 2.

Expected faecal pathogens and number of laboratories and number of specimens with false culture result in faecal culture EQA rounds during the period 2009–2014.

Expected faecal pathogens	No. of EQC specimens with expected pathogens	No. of laboratories with false resulta	No. of EQC specimens with false result
Single pathogens
Salmonella serotype	18	12	2
Shigella spp.	7	13	6
Campylobacter sp.	5	1	1
Yersinia sp.	6	3	3
EHEC	1	2	2
Two pathogens
Shigella flexneri+ Aeromonas hydrophila	1	9	1
Salmonella Typhimurium + Shigella boydii	1	6	1
Total	48	14	16

Abbreviations: EHEC, Enterohaemorrhagic Escherichia coli; EQC, External Quality Control.

^aThe total number of laboratories was 17.

Total number of false faecal culture results was 46 of all 581 reports. The evaluation of the results of the individual clinical microbiology laboratories showed a total of 12 false-negative Salmonella results and 13 Shigella results. Fourteen (82%) of the 17 laboratories gave at least 1 false result with the 48 specimens. Among the 48 specimens, 2 (4%) to 3 (6%) false results were common, although 1 university laboratory reported 5 (10%) and 1 central hospital laboratory reported 8 (17%) false results. At least 1 false-negative Salmonella result was reported by 10 (59%) and 1 false-negative Shigella result in 9 (53%) of the 17 laboratories (Figure 1).

Figure 1.

The number of faecal External Quality Control results reported and false results in these reports during the period of 2009-2014. C indicates central hospital laboratory; PR, private laboratory; U, university laboratory.

Comparison of Finnish and other European laboratories

The number of other European laboratories that participated in the same EQA rounds of the Faecal Culture Scheme as the Finnish laboratories varied between rounds even within the same year, from 17 to 90 per a round (Table 3). The percentage of correct results varied from 9% to 99% among the other European laboratories and from 25% to 100% among the Finnish laboratories. Some of the specimens (especially one with S. Infantis and one with S. Typhimurium) were challenging for all participants.

Discussion

The EQC specimens sent to the participating laboratories were lyophilised mixtures of bacteria and the focus was on the analytical process. The EQA rounds analysed in this study showed that the detection of common pathogens, such as typical serotypes of Salmonella or strains of Campylobacter and Yersinia species, was quite effective in all Finnish clinical microbiology laboratories. However, the results showed that 2 to 3 false faecal culture reports were common among the laboratories. Of the 48 faecal culture specimens, all of the laboratories were able to give correct reports for 32 (67%) of the specimens. Especially challenging was the detection of Shigella strains. Shigella was present as a single bacterial pathogen in 7 specimens, and only in 1 specimen it was found by all participants. In contrast, as a single bacterial pathogen, Salmonella was found in 16 of the 18 specimens by all participants. These false reports were due to difficulties in detecting the biochemically atypical S Infantis strain and detecting pathogen in the sample containing low count of S Typhimurium cells. In addition, EQA specimens that contained more than 1 pathogen were difficult for the participants. Depending on the efforts made in the laboratory to analyse the EQC specimens, the success may even reflect the maximum quality of the laboratory. Laboratories have been shown to succeed better with specimens known to be EQC specimens than they operate without this knowledge.⁴ This raises the real possibility that in normal daily routines, the quality performance of certain laboratories may be even lower than shown by these EQA rounds. Vibrio species were not present in the EQC specimens during the study period; thus, their detection could not be evaluated.

A reliable detection and diagnosis system for all faecal bacterial pathogens has an important role in both individual patient care and public health.

In Finland, per year about 70 000 to 75 000 faecal samples are analysed to detect faecal diarrhoeal bacteria.⁵

Annually, in 2009 to 2014, there were 89 to 160 cases of Shigella, 1600 to 2400 cases of Salmonella, 4000 to 5000 cases of Campylobacter, 500 to 600 cases of Yersinia, and 20 to 98 cases of EHEC reported to the Finnish National Infectious Diseases Register.^6-10 The corresponding figures reported to European Centre for Disease Prevention and Control (ECDC) were the following: 6000 to 7000 cases of Shigella, 90 000 to 110 000 cases of Salmonella, 200 000 to 240 000 cases of Campylobacter, 6500 to 7700 cases of Yersinia, and 3700 to 9500 cases of EHEC.¹¹ All reported cases are laboratory confirmed. When prioritising communicable diseases according to their public health relevance, Shigella, Salmonella, Campylobacter, Yersinia, and EHEC are considered to be in the high or highest priority group.^12,13

One multicenter study in 6 Asian countries found that half of the patients with culture-negative faecal samples were positive by polymerase chain reaction for Shigella.¹⁴ Transportation of faecal specimens is difficult because of the fastidious nature of some of the enteric pathogens. Delay may hamper the recovery of enteric pathogens. For example, most Shigella strains are sensitive to the pH change that occurs when the temperature of the stool decreases and thus might get missed by culture methods. Although arriving alive at the laboratory, after cultivation on standard selective and differential culture media, isolation and final identification depend on the experience of the investigator.

In the culture of faecal specimens, the consequences of a false-negative report may be serious. The results of EQC should raise awareness in each of the participating laboratories about the strengths and weaknesses of the laboratory and therefore give an opportunity to find a way to improve their quality.¹⁵ These EQC results showed clearly that improvements need to be made especially in the detection of colonies of Shigella species and atypical Salmonella strain, and also in the enrichment procedure of Salmonella, as well as searching for a second or even third bacterial pathogen despite already having found one.

The high overall reporting rate, 96% to 100%, for the EQC results in 2009-2014 showed good commitment to EQA and demonstrates that EQA has become routine in the laboratories. The proportions of all correct reports in the laboratory groups were similar (91% and 94%), showing that the laboratories performance was not dependent on the type of laboratory.

Each laboratory is responsible for using the methods that produce correct results regardless of the specimens, ie, daily routine specimens or EQC specimens. Based on the information collected from the EQC reports sent to Labquality and the questionnaire sent by THL to all Finnish clinical microbiological laboratories, these laboratories use similar, widely accepted diagnostic methods and culture media^16-18 when searching for pathogens in clinical specimens. Selenite or selenite-cysteine broth is used to enrich Salmonella from stool specimens. After enrichment, the subculture is inoculated on xylose lysine deoxycholate agar.^19,20 New chromogenic culture media have come into use in some laboratories and may offer improvements in the detection of Salmonella strains.^21,22 In the case of rare findings like Shigella in Finland, the laboratories should make more effective use of the educational role of EQC samples. Difficulties with Shigella, and especially in combination with other faecal pathogens, have been evident already in previous Finnish studies that analysed the UK Neqas EQC results for the period 1995–1997.^23,24 The fact that there has been no improvement in these results in nearly 20 years suggests that the opportunities to exploit EQA rounds to improve the daily laboratory diagnostics of infectious diseases have not been fully exploited by some Finnish laboratories.

To meet the needs for faster and more extensive diagnostics of diarrhoea-causing pathogens, new molecular methods are being developed and taken into use.^25–27 Everyday diagnostics of faecal pathogens are still mainly based on culturing and recognition of typical colonies on agar plates. Isolation of the bacteria also enables their further analysis and antimicrobial susceptibility testing. Functional culture methods of faecal pathogens become even more important at this juncture, given that antimicrobial resistance among these bacteria is increasing.^28–31

Several Finnish clinical microbiology laboratories have been accredited by the Finnish Accreditation Service. It is a matter of discussion as to whether the more accurate microbiological diagnostics in Finland compared with some other European countries are the result of voluntary accreditation, the legislation-based approval system, or compulsory involvement in EQA. In some other countries, the benefits of accreditation have been modest and likely to be contributed by the increased awareness of quality-related issues that are part of the quality assurance process in laboratory accreditation.³²

Compared with the other European laboratories that took part in the EQA rounds of Labquality in 2009-2014, Finnish laboratories in general succeeded well. However, 7 of the EQC specimens contained Campylobacter, Yersinia, and Aeromonas or EHEC, which do not belong to the test panel of faecal culture in all European laboratories. This may explain some of the low percentages of correct results in the other European laboratories. The use of different test panels by some European laboratories may cause confusion for possible customers from other European countries because the specimens found to be culture negative may be positive for microbes that are not being routinely tested in these laboratories. This emphasises the importance and need for standardised test panels in European laboratories. In Finland, the test panels for various microbiology specimens such as faecal culture have been defined for years in an official tome called ‘the Nomenclature of Laboratory Investigations in Finland’. Campylobacter is the most common and Yersinia a relatively common faecal pathogen in several European countries, thus justifying their place in the faecal culture test panel and, therefore, their inclusion in these EQA specimens. Enterohaemorrhagic Escherichia coli is of public health concern, given its potential for disease outbreaks and the risk of serious complications. Therefore, all clinical microbiology laboratories should also be aware of it. Also, a good communication between the clinician and the laboratory is needed in the search for a rare pathogen.