Multicenter Assessment of Gram Stain Error Rates

Abstract

Gram stains remain the cornerstone of diagnostic testing in the microbiology laboratory for the guidance of empirical treatment prior to availability of culture results. Incorrectly interpreted Gram stains may adversely impact patient care, and yet there are no comprehensive studies that have evaluated the reliability of the technique and there are no established standards for performance. In this study, clinical microbiology laboratories at four major tertiary medical care centers evaluated Gram stain error rates across all nonblood specimen types by using standardized criteria. The study focused on several factors that primarily contribute to errors in the process, including poor specimen quality, smear preparation, and interpretation of the smears. The number of specimens during the evaluation period ranged from 976 to 1,864 specimens per site, and there were a total of 6,115 specimens. Gram stain results were discrepant from culture for 5% of all specimens. Fifty-eight percent of discrepant results were specimens with no organisms reported on Gram stain but significant growth on culture, while 42% of discrepant results had reported organisms on Gram stain that were not recovered in culture. Upon review of available slides, 24% (63/263) of discrepant results were due to reader error, which varied significantly based on site (9% to 45%). The Gram stain error rate also varied between sites, ranging from 0.4% to 2.7%. The data demonstrate a significant variability between laboratories in Gram stain performance and affirm the need for ongoing quality assessment by laboratories. Standardized monitoring of Gram stains is an essential quality control tool for laboratories and is necessary for the establishment of a quality benchmark across laboratories.

INTRODUCTION

Clinical microbiology laboratories have undergone dramatic changes with the implementation of novel technologies, yet traditional techniques, such as the Gram stain, still play key roles throughout the diagnostic process (1–3). Gram stains are initially used as a preanalytical indicator of specimen quality and acceptability for culture. They also give the clinician preliminary information regarding the nature of potential pathogens present in the patient specimen and thus serve to guide empirical therapy. Although the Gram stain has been the staple of clinical microbiology laboratories for over a century, it is still considered a high-complexity procedure by the Clinical Laboratory Improvement Amendments (CLIA) program. The manual nature of the staining process and the subjectivity of Gram stain interpretation contribute to the incidence of errors (4–7). Inappropriate specimen sampling, specimen processing, smear preparation, and prior antibiotic therapy are all factors that can have an adverse impact on Gram stain result. The inherent nature of some organisms may also produce misleading results; for example, Acinetobacter spp. may stain Gram positive, while Bacillus spp. species may appear Gram negative. In addition, staining practices, such as use of cytospin and fixation methods (heat versus methanol), may vary from one laboratory to another and even within laboratories, with a potentially significant impact on the quality of the result (8).

Clinical chemists have made concerted efforts to characterize the nature and incidence of errors in chemistry laboratories (9, 10). They have also begun to generate multicenter data on the incidence of errors in preanalytic, analytic, and postanalytic testing, with the long-term goal of developing quality indicators specifically for the reduction of errors in clinical laboratories (11, 12). In fields such as surgical pathology, CLIA '88 mandates correlation of results for cases in which both cytologic and surgical specimens are collected from the same site (13). Such mandates have incentivized pathologists to attempt to standardize, categorize, measure, and determine the impacts of errors in their field (13–18). These studies have identified challenges that are not dissimilar from those faced by clinical microbiologists. The frequency of errors in cancer diagnosis was found to be 11.8% in cytologic-histologic specimen comparisons (13). Attempts to assess root causes of these errors have been hampered by significant interobserver variability and lack of consensus (19). In spite of these challenges, these efforts have led to interventions that reduced the incidence of errors in surgical pathology (15).

Discrepant Gram stain results can occur for a number of reasons. Positive Gram stains with negative culture results could be due to fastidious or nonviable organisms and failure to order appropriate testing, such as anaerobe cultures. False-negative Gram stains could occur due to inadequate specimen or smear preparation or failure to examine an adequate number of fields. In addition, training and maintenance of proficiency for Gram staining remain challenging (5, 20). The consolidation of clinical microbiology laboratories has left many community hospitals with limited laboratory capacity. Gram stains are often read by generalists with limited training and proficiency in microbiology, which has an adverse impact on the quality of results (5, 7). The lack of reproducibility of Gram stain results, particularly with respiratory specimens, has also led some to question its utility (2, 4, 21). Studies have shown that errors occur and can have significant adverse clinical impacts (3, 6, 22). The College of American Pathology (CAP) does recommend that laboratories correlate Gram stain results with culture results, but they have not specified how this should happen. Despite its importance in the clinical decision-making process, there are no comprehensive data available regarding the incidence of errors in the performance and interpretation of Gram stains. Smaller studies have only examined specific specimen types within single institutions, but laboratories seeking to benchmark their performance against peer institutions will struggle to find appropriate measures. We have evaluated Gram stain performance for a subset of specimens submitted as part of routine patient care across four different clinical microbiology laboratories at large tertiary care centers, using standardized criteria in order to generate data that could potentially be used to set a benchmark for acceptable standards for Gram stain performance and to identify key areas for improvement.

MATERIALS AND METHODS

The study included four full-service microbiology laboratories at tertiary care institutions and state medical centers with diverse patient populations. The annual bacterial culture volume of the participating laboratories ranged from 90,000 to 300,000. During the course of the study, participant laboratories prospectively screened smear/culture results for all nonblood specimens that had a Gram stain performed as part of routine culture. This included respiratory, fluid, biopsy tissue, and wound sample cultures. Specimens not included were urine cultures that did not receive routine Gram stains and fluid cultures that were inoculated directly to blood culture bottles without culture plates to allow for quantitation of growth. Screening was performed either by manual review or the use of automated laboratory information system (LIS) reports. While the study took place in 2014, the actual duration of study and time of performance varied between sites due to limitations of staffing and varied test volumes between sites.

Selection criteria.

Data from review of ∼42,000 culture results at one of the participating sites (R. Cavagnolo, personal communication) indicated a 94% correlation between culture and Gram stain results for nonblood specimens with positive cultures showing moderate/many (3+/4+) colonies in routine aerobic cultures. The relatively high correlation between Gram stain and culture results in this subset made it ideal for screening for discrepant results. In contrast, data from the same site showed that specimens with few (2+) colonies on culture only had 76% correlation with Gram stain results, while specimens with rare (1+) colonies on culture had 29% correlation with Gram stain results.

Results were considered discrepant and flagged for review if specimens had Gram stain and culture results that fit the following criteria: (i) culture demonstrated moderate/many colonies of a particular organism but Gram stain was negative for an organism with corresponding morphology/stain characteristics, or (ii) Gram stain showed moderate/many bacteria but culture was negative for growth of a corresponding organism.

Gram stain was performed as per laboratory protocols at individual sites. No major differences were noted in Gram stain protocols between sites, with the exception of smear fixation methods (heat versus methanol), with one site (C) using methanol fixation while the remaining sites used heat fixation. Site A utilized an automated Gram stainer (Midas 3; EMD Millipore, Darmstadt, Germany). Use of cytospin for concentration of fluid specimens was performed at all sites as per individual laboratory protocols for sterile fluids. Specimens which had results that fit the screening criteria were classified as “discrepant,” and review of the original Gram stain was performed by designated laboratory staff (senior technologist/laboratory director), who first examined the slide at low power (100×) and then examined at least 40 oil immersion (1,000×) fields/slide. Discrepant smears initially reported as positive for bacteria were considered errors if a corresponding organism was not observed after smear review or if obvious artifacts were noted. Discrepant smears initially reported as negative were considered errors if the presence of an organism was noted in 3 or more fields upon smear review. Reviewers did not correlate presence/absence or quantification of white blood cells (WBC) or epithelial cells except to determine acceptability of respiratory specimens for culture as per site-specific criteria. In addition, for specimens that were Gram stain positive but culture negative, laboratories determined whether anaerobe cultures were performed.

Data analysis.

The number of slides that were unable to be retrieved for review was documented. Error rates were calculated based on the number of slides available for review. Projected error rates for all slides were calculated as the percentage of the total number of specimens included in the study and as the number of errors per 1,000 specimens. Statistical analysis was performed using Microsoft Excel to compare the proportion of errors per site to the total for all sites, using Z-scores. The Z-score for each site indicated the standard deviation from the mean.

RESULTS

During the course of the study, results from a total of 6,115 specimens (976 to 1,864 specimens/site) were reviewed for discrepant smear/culture results. The incidence of discrepant specimens was relatively consistent across all sites (4 to 6%), with a total of 303/6,115 (5%) results considered discrepant as per the study criteria (Table 1). The logistical difficulties of retrieving slides for secondary review in high-volume laboratories meant that only 263/303 (87%) of slides were available for review. This suggests that laboratories should develop reliable slide retention practices. More than half of the discrepant results were smear negative/culture positive (58%), although this number varied between sites (15% to 85%). Sites A and D had a relatively higher proportion of smear-positive/culture-negative specimens (61% to 85%) than did sites B and C, which had a higher proportion (79% to 85%) of smear-negative/culture-positive discrepants (Table 1). Among the smear-positive/culture-negative discrepant results, 87/111 (78%) were specimens that were considered acceptable for anaerobic culture. Anaerobe cultures were not ordered for 49/87 (56%) of these specimens, with the proportions of specimens missing anaerobe culture ranging from 37% to 100% between sites (Table 1).

TABLE 1.

Incidence of discrepant results

Site	Total no. of specimens	No. (%) of discrepant results	No. (%) of slides reviewed	Gram stain/culture result (no. [%])		No. (%) of Pos/Neg specimens without anaerobe culture
				Neg/Pos	Pos/Neg
A	1,864	73 (4)	67 (92)	10/67 (15)	57/67 (85)	30/57 (53)
B	1,631	83 (5)	78 (94)	62/78 (79)	16/78 (20)	10/16 (63)
C	1,644	91 (6)	74 (81)	63/74 (85)	11/74 (15)	6/6 (100)
D	976	57 (6)	44 (77)	17/44 (39)	27/44 (61)	3/8 (37)a
Total	6,115	303 (5)	263/303 (87)	153/263 (58)	111/263 (42)	49/87 (56)

^aAt site C, 5/11 of specimens and at site D 19/27 specimens were not acceptable for anaerobe culture.

Discrepant smear/culture results as per the study criteria accounted for 5% (303/6,115) of total smears reviewed. Among discrepant slides, 263/303 (87%) were reviewed, with 62/263 (24%) found to be erroneously reported (Table 2). While the percentage of total smears that were discrepant as per study criteria was relatively similar across sites, the actual error rate varied from 9% to 45% of discrepant slides reviewed. The projected percentage of total smears that were incorrectly read was 1.2%, although this ranged significantly, from 0.4% to 2.7% between sites (Table 2; Z-scores of 3.6 to −3.1). The average projected error rate (calculated to include unreviewed smears) per 1,000 specimens was 12.3 (3.8 to 26.6) (Table 2). The incidence rates for errors by specimen type across all sites were as follows: fluids (13%), wound swabs (33%), respiratory (38%), tissue (8%), and abscess (8%).

TABLE 2.

Analysis of discrepant results and Gram stain error rate

Site	No. (%) of reader errors	Projected no. (%) of total errors	Projected errors/1,000 smears	Z-score
A	6/67 (9)	7 (0.4)	3.8	−3.1042
B	24/78 (31)	25 (1.5)	15.3	1.0884
C	12/74 (16)	15 (0.9)	9.1	−0.9565
D	20/44 (45)	26 (2.7)	26.6	3.6351
Total	62/263 (24)	73 (1.2)	6.9

The majority of incorrectly read smears (50/62 [81%]) were false-negative results where one or more organisms were not reported on initial examination (Table 3). In 8/50 false-negative smears, subsequent review found more than one type of organism morphology that had been missed, for a total of 60 morphotypes not reported (Table 3). The most frequently missed organism morphologies were Gram-positive cocci (28/60 [47%]) and Gram-negative bacilli/coccobacilli (19/60 [32%]) (Table 3). Less frequently missed organisms included yeast (13%) and Gram-positive bacilli (8%). False-positive smears represented only 19% (12/62) of incorrectly read smears. The majority of these (9/12) were respiratory specimens that should have been rejected as per laboratory criteria due to contamination with oral flora. Such results were classified as false positives, because the Gram stain result, which should have been suppressed as per laboratory criteria, indicated the presence of a predominant pathogen that was not found or reported on subsequent culture (Table 3).

TABLE 3.

Summary of type of Gram stain errors

Type (no.) and nature of Gram stain error	No. of samples
False-negative smear (50)
No. of organisms	60
Gram-positive cocci	28
Gram-negative bacilli/coccobacilli	19
Gram-positive bacilli	5
Yeast	8
False-positive smear (12)
No. of organisms	12
Gram-positive bacilli	1
Gram-negative bacilli/coccobacilli	2
Other	9a

^aNine sputum specimens that should have been rejected as per the laboratory criteria.

DISCUSSION

It has been reported that preventable medical errors are responsible for the death of 400,000 Americans annually, the third highest cause of mortality after heart disease and cancer (23). If 60 to 70% of medical decisions are based on laboratory results, it stands to reason that erroneous laboratory results play some role in avoidable patient mortality (24). Anatomic pathologists have made significant efforts to track, categorize, and determine the impacts of errors in slide review in multicenter studies, but they have had difficulties in developing consensus on interpretation of results and root causes (15, 18, 19). Clinical microbiologists face similar obstacles in attempts to characterize the incidence and nature of errors in their laboratories. Many of the tests performed, such as the Gram stain, are prone to variability and differences in interpretation between technologists. Studies that have examined Gram stain error rates have generally focused on practices at a single institution and/or with limited specimen types (2–6, 21, 25, 26). The absence of multicenter benchmark data on error rates in clinical microbiology laboratories limits the ability to effectively measure their performance against their peers. Since errors in clinical microbiology laboratories can have significant clinical impacts (22), it is imperative that greater efforts be made to generate data on their frequency of occurrence in clinical microbiology laboratories. This study represents the first such attempt to generate multicenter data on error rates during the performance of Gram stains.

The incidence of discrepant results for specimens was relatively similar between the four sites (4 to 6%), but the incidence of errors varied significantly (0.4% to 2.7%) (Table 2; Z-score of 3.6 to −3.1). For sites B and C, the majority of discrepant results (79% and 85%, respectively) were Gram stain negative/culture positive, while for sites A and D, discrepant results were primarily Gram stain positive/culture negative (85% and 61%, respectively) (Table 1). While all sites had quality control processes in place for monitoring of Gram stain/culture correlations, the extent of the programs varied based on location. Site A had a rigorous automated daily management system for monitoring discrepant Gram stain/culture results and had the lowest error rate (0.4%). Site A also had the lowest proportion of smear-negative/culture-positive specimens (15%), but conversely, this site had the highest proportion (85%) of smear-positive/culture-negative specimens. Site C conducted a daily manual secondary review of all culture reports and routinely performed secondary review on smears from sterile fluids that showed the presence of neutrophils but with no organisms noted on initial review, and this laboratory had the second lowest error rate (0.9%). Site B relied on manual detection of discrepant smear/culture results by technical staff but captured corrected reports daily by using an automated report from the LIS, with immediate review with staff involved. Site D used manual processes during routine workflow for the detection and follow-up of discrepant Gram stain/culture results. The lack of consistency in the nature of discrepancies between sites indicates that a variety of factors are involved, and addressing the issue of Gram stain consistency will be challenging.

The majority of discrepant results (58%) were Gram stain negative/culture positive. As discussed previously, standardized determinations of root causes in similar studies across multiple sites have been problematic (19). While this study did not track root causes of discrepant results, participating laboratories made a number of observations during the course of the study. Discrepant findings involving smear-negative/culture-positive results often had very little cellular material noted upon secondary review. This suggested that there was inadequate material on the slide, possibly due to use of suboptimal specimens, such as swabs for culture/smear preparation rather than actual tissue/fluid specimens. Alternatively, lack of material on the slide could also reflect failure to perform adequate fixation prior to staining. Mangels et al. noted that use of methanol fixation held significant advantages over heat fixation, including reduced distortion of cellular morphology, reduced background debris, and increased likelihood for detection of microorganisms on smear (8). However, in our study the incidence rate of Gram stain errors in the laboratory did not appear to correlate specifically with method of smear fixation utilized. Site C, which used methanol fixation for smear preparation, had an error rate of 0.9%, while the remaining sites, which used heat fixation, had error rates of 0.4% to 2.7%. This could simply indicate that the method of smear fixation was not the sole factor that contributed to the incidence of errors. Other problems noted across all sites included inappropriate use of cytospin for concentration of specimens. The use of cytospin for smear preparation has been shown to improve sensitivity of Gram stains (27, 28). However, concentration of sterile fluids with high cellular content can lead to overly thick smears, which can increase the risk of false-negative results due to difficulty in detecting the organism among cellular material/debris. Inappropriate use of eSwabs (Copan Diagnostics Inc., Murrieta, CA) contrary to the manufacturer's instructions was also noted to contribute to discrepant results. Other factors contributing to these discrepancies included failure to order anaerobic culture, which was the case for 49/87 (56%) of appropriate specimens across all sites. This suggested that physician education may be necessary to ensure that appropriate laboratory testing is ordered. Laboratories may want to consider including comments regarding the presence of anaerobic bacteria in specimens with positive Gram stains but negative cultures where anaerobic cultures were not requested. For the 38/87 (44%) discrepants with Gram stain-positive/culture-negative results that did not yield growth of any organism on anaerobic culture, it is possible that when confirmed by secondary review, these findings represented the presence of nonviable or fastidious organisms (Table 1). It is also possible that these results indicate problems with anaerobe specimen collection, transport, or culture process that lead to loss of viability of the organism.

Secondary review of discrepant smears determined that 24% of discrepant results across all sites were due to errors in technologist interpretation of Gram stains (Table 2). The percentage of total smears that were discrepant with culture results was relatively similar between sites (4% to 6%), but the proportion of discrepant results that were errors ranged from 9% to 45% (Table 2). The actual incidence of errors/1,000 smears varied significantly between sites, with at least 2/4 sites showing Z-scores of >3 standard deviations from the mean (Table 2). The relatively high incidence of errors among discrepant results suggests that this subset of smears may serve as an ideal target for quality assurance monitoring efforts. The majority of erroneous smear results were false negatives (50/62 [81%]), with 8/50 involving multiple organisms, for a total of 60 unreported morphotypes (Table 3). These primarily included either Gram-positive cocci (47%) or Gram-negative bacilli/coccobacilli (32%) (Table 3). Participants at each site observed that factors contributing to false-negative smears included failure to examine an adequate number of fields, to visually demarcate the area of smear on the slide, and to distinguish actual organisms from stain debris or background. In particular, the inability to distinguish organisms from stain debris was most prominently noted with Gram-positive cocci, which represented nearly half (28/60) of all organisms missed on false-negative smears (Table 3). The majority (9/12) of false-positive smears were due to failure to reject respiratory specimens that were contaminated with oral flora (Table 3). These smears were reported to have predominant Gram-negative bacilli, while culture growth demonstrated mixed flora with no predominant organism (data not shown). Only 3/12 false positives were smears where the presence of bacteria was reported but no bacteria were detected on secondary review, indicating that this was not a major source of Gram stain errors. The data indicated that the majority of errors occurs within the subset of smear-negative/culture-positive discrepant results, with 50/153 (33%) discrepants found to be errors, compared to 12/111 (11%) errors in the smear-positive/culture-negative subset.

While the data did not indicate specific factors as the cause of Gram stain errors, based on observations made during this study, participating laboratories are in the process of implementing process improvements to address potential areas of concern. These include education of laboratory staff on key aspects of smear preparation, including the following: demarcation of the smear on the slide, use of methanol fixation, and appropriate use of cytospin. Overly thick smears can be prevented by avoiding the use of cytospin for fluids with high cellular content. Smears should not be so thick that newsprint is not visible through the slide. Staff were cautioned to ensure that smears that appeared to have inadequate material were repeated and that an adequate number of fields were examined. At site C, double review of smears was routinely performed only for a subset of specimens that met specific criteria. While this approach may help to reduce error rates, the logistics may be challenging for some laboratories. There are no data available on the impact of double review of smears in clinical microbiology, but similar approaches in surgical pathology and cytology have met with mixed success (15, 29). Meier et al. noted that double review of slides from breast and prostate cases significantly reduced the incidence of misinterpretations (15). However, Raab and colleagues found that double slide viewing did not lower the frequency of errors detected during cytologic-histologic correlation (29). Additional steps include physician education and feedback on the need to order anaerobic testing when necessary, or a systems-based approach of reflexive ordering of anaerobic culture for specific specimen types. Avoiding the use of inadequate specimens, such as swabs for culture, could reduce the incidence of discrepant results. Anatomic pathologists as well as data from one of the study sites have shown that tracking and categorization of laboratory errors by type of error and individual can reveal patterns that, when addressed in real time, can reduce the incidence of errors (15, 18, 30). Further studies are required to determine the impacts of these measures.

This study has a number of limitations. The screen for discrepant results was limited to nonblood specimens which met specific criteria, i.e., either many/moderate organisms on Gram stain or many/moderate organisms on culture. The error rates obtained using this approach only apply to this subset of specimens, and the true error rate for Gram stains in clinical microbiology laboratories may vary from the numbers obtained here. Smears were not evaluated for accuracy of reporting of the presence and quantitation of white blood cells or epithelial cells. Positive blood culture smears were excluded due to the low rate of Gram stain errors with this specimen type (6, 26). Only site A utilized an automated Gram stainer, and therefore it is difficult to infer whether this may have had any impact on the quality of Gram stains. Sites used different approaches to capture discrepant results: site A used an automated program to capture discrepant smear/culture results, but sites B, C, and D relied on manual review of smear/culture results for discrepancies. It is possible that the manual nature of data collection practices could have impacted study results. Determination of whether a discrepant result was actually an error relied on secondary review by a senior technologist/laboratory director. Consistent interpretation of Gram stain results can be challenging within a single laboratory and even more so across multiple institutions and could potentially have biased study data.

The data presented here are the first step toward establishing a benchmark for the incidence of errors during the performance of Gram stains. The average error rate for all sites was 1.2%; however, significant variation was noted in the incidence of errors between sites (Z-scores ranging from 3.6 to −3.1). The results suggest that additional data may be required to establish acceptable ranges for Gram stain performance. While this finding is not surprising given the simultaneously technical and subjective nature of Gram stains, it highlights the need to address the diverse issues that impact Gram stain error rates and standardize performance between laboratories. As health care moves toward a greater transparency and focus on the reduction of the potential for harm due to medical errors, laboratories cannot begin to improve if we do not first know where we stand. Monitoring of Gram stain error rates is an essential first step in this process.