Mechanisms To Assess Gram Stain Interpretation Proficiency of Technologists at Satellite Laboratories

Abstract

To address Gram stain interpretation proficiency in a satellite/centralized microbiology laboratory paradigm, two programs were devised. In quality assurance program 1, nonmicrobiology technologists at satellite laboratories were required to interpret standardized Gram-stained specimens of clinical material prepared by an experienced microbiologist at a central laboratory. In quality assurance program 2, clinical Gram stains prepared and read by the satellite laboratorians were reviewed by experienced microbiologists at the central laboratory. Satisfactory performance (94%) was achieved in quality assurance program 1. In contrast, quality assurance program 2 had a significantly lower overall performance (89%; P < 0.0001) due to poorer identification of host cells (93%) and bacteria (84%). A variety of intervention mechanisms, including continuous monitoring, resulted in overall performance improvement (P ≤ 0.006). While a technologist challenge has educational merit, having a microbiologist review previously read slides is a better indicator of the technologist's Gram stain interpretation proficiency.

Gram staining of primary clinical specimens has tremendous clinical utility. Studies have shown that Gram staining of sputum can be predictive for several etiologies of lower respiratory tract disease (1, 10, 11, 15, 16, 17, 19, 23). Gram stain data also influence the treatment of clinically significant bloodstream infections (8, 13). In addition, the proper performance of the primary Gram stain procedure can guide the processing of expectorated sputum specimens (14) and aid in the interpretation of cultured skin and soft tissue specimens (3, 26).

Bartlett (2) cited the virtual elimination of house staff laboratories (as an indirect result of the Clinical Laboratory Improvement Amendments of 1988) and the outsourcing of primary specimens as two factors contributing to the decline in clinical microbiology studies for patients with lower respiratory tract disease. Others have raised concerns over the clinical diagnostic value of the Gram stain in terms of protocol standardization (5, 7, 18). As a result, many have discouraged the utilization of Gram-stained smears as predictors of certain clinical infections, particularly those involving the lower respiratory tract (10, 20, 21).

Technologist competency has become of even greater importance with the centralization of microbiology testing and the creation of satellite processing (rapid-response) laboratories staffed by nontraditional microbiologists (6). Moreover, accrediting agencies mandate the proficiency documentation of all technologists who execute clinical Gram staining protocols. In this report, we describe a mechanism for objectively assessing the Gram stain interpretation competency of nonmicrobiology laboratorians and present a quality assurance program for monitoring and increasing the proficiency of Gram staining.

Full-spectrum clinical laboratories were present at three Milwaukee, Wisconsin, inpatient facilities until 1988, when two hospitals (B and C) consolidated their clinical microbiology laboratories into a freestanding building (a central laboratory). Subsequently, the central laboratory assimilated the clinical microbiology services of a third hospital (A) in 1999. Algorithms were devised in which the primary clinical specimens for routine bacteriology were processed onto appropriate media (22) and smeared for Gram staining and interpretation (25) by trained nonmicrobiology technologists at the satellite laboratories. Within 24 h, Gram-stained smears were forwarded to the central laboratory for confirmation of results.

Three primary clinical specimens from a variety of sources were smeared and Gram stained quarterly by an experienced microbiologist at the central laboratory (quality assurance program 1). The number of cells (viewed at ×100 magnification for eukaryotic cells and ×1,000 magnification for prokaryotic cells) was identified as rare (<1 per field), few (1 to 5 per field), moderate (5 to 20 per field), or abundant (>20 per field). Included within these primary clinical specimens were sputa to be evaluated for culture processing acceptability, based on the criteria of Murray and Washington (14). Seventy satellite laboratory technologists were made aware of the specimen source and were asked to provide Gram stain interpretation and semiquantitation of the eukaryotic and prokaryotic cells from the challenge slides. The results were forwarded to the central laboratory microbiologist. Furthermore, the central laboratory microbiologists (on a rotating basis) daily selected a number of the satellite-laboratory-prepared smears that was commensurate with the relative microbiology volume generated by that hospital (quality assurance program 2). A descriptive Gram stain report was documented by the microbiologist and forwarded to the microbiology supervisor.

Reviews of the Gram-stained preparations in quality assurance programs 1 and 2 were documented using the following system: 1 point was given for each correct Gram stain reaction, 1 point for each bacterial morphology correctly identified, 1 point for each host cell (squamous epithelial or inflammatory) morphology correctly identified, and 1 point for the quantity of each host or bacterial cell type correctly identified. The acceptable range for quantitation was ±1 gradation of the expected value. Credit for quantity, Gram reaction, or cell type involved was not granted if a sputum specimen was rejected for culture; the technologist received a slide value of 0. In addition, points were subtracted from a score when additional cell types or bacteria were erroneously reported. Finally, point values for host cells (quantity plus presence/absence) and bacteria (quantity plus Gram stain result and morphology) were summed per preparation and expressed as a percentage of the expected value. The standard error of the mean was calculated for all measures of Gram stain interpretation proficiency testing. Comparisons of interlaboratory performances, cell types, and the two measures of Gram stain interpretation proficiency testing were facilitated by the t test for independent samples. Assessment of the temporal change (improvement) in laboratory performance was facilitated by one-way analysis of variance (24). The alpha level was set at 0.05 before investigations commenced, and all P values were two-tailed.

The overall Gram stain interpretation proficiency rate for 70 satellite laboratory technologists, as determined by quality assurance program 1, was 94% (Fig. 1). However, host cells were interpreted with a higher success rate than bacteria (98% versus 89%; P < 0.0001). When data from individual hospitals were examined (Fig. 1), personnel at hospital C showed the lowest proficiency for host cell recognition and identification of bacteria. Upon the review of 2,517 randomly chosen Gram-stained preparations by quality assurance program 2, no significant differences in proficiency for the detection of host cells or bacteria were found among the laboratorians of the three satellite laboratories. Similar to the results for quality assurance program 1, greater proficiency in interpretation of host cells than in bacterial observations was demonstrated by satellite laboratorians (P < 0.001).

FIG. 1.

Gram stain interpretation proficiencies of three satellite laboratories, determined by quality assurance program 1 and delineated by interpretation of bacteria (black bars) and host cells (white bars) and the sum of the two categories (shaded bars).

When the results of the two quality assurance programs were compared, the proficiency in interpretation of Gram stain results of quality assurance program 2 was 5.4% lower than that of quality assurance program 1 (P < 0.0001; Fig. 2). Analogous differences were detected between the programs for identification of bacteria and host cells (P ≤ 0.03). Upon delineation of individual hospital data by cell type, the rates of proficiency noted for quality assurance program 1 were generally higher than those for quality assurance program 2 (P ≤ 0.04).

FIG. 2.

Comparison of quality assurance programs 1 and 2 for measure of overall Gram stain interpretation proficiency.

The implementation of quality assurance programs 1 and 2 caused a significant increase in Gram stain interpretation proficiency in the fourth quarter of 2006 (P ≤ 0.006; Fig. 3). Improvements occurred in host cell and bacterial identification among the participants of quality assurance program 1 (P ≤ 0.01) and in host cell identification among the participants of quality assurance program 2 (P = 0.03). Among the hospitals, the greatest improvement occurred with hospital C (P ≤ 0.005; Fig. 4A and B). The nonmicrobiology laboratorians at hospital C performed consistently less skillfully than laboratorians at hospitals A and B prior to the implementation of the quality assurance programs.

FIG. 3.

Gram stain interpretation proficiency of nonmicrobiology laboratorians as determined by quality assurance programs 1 (squares) and 2 (diamonds) for four quarters of 2006.

FIG. 4.

Gram stain interpretation proficiency of hospital C nonmicrobiology laboratorians, measured at the onset and later stages of quality assurance programs 1 (A) and 2 (B).

The diagnostic value of the Gram stain has been questioned from a variety of perspectives. In a meta-analysis, Reed et al. (21) reported Gram stain sensitivity of 15% to 100% for the diagnosis of pneumococcal pneumonia, with specificity ranging between 11% and 100%. In a regional proficiency testing program, Church et al. (5) reported a significant lack of standardization in reporting quantities of host cells and bacteria and raised concerns about how this could impact patients over a continuum of care. Cooper et al. (7) documented low intertechnologist agreement rates for cell quantitation from Gram-stained lower respiratory tract specimens. In a broad sense, following a regional microbiology laboratory restructuring, a number of subsequent proficiency challenges revealed that error rates increased in a number of laboratories, such as rural facilities (noted to possess neither a technologist devoted to microbiology duties nor on-site pathologist or medical microbiologist oversight) that took on additional microbiology responsibilities (6). Combined with the clinical and bench-level utility of the Gram stain procedure, the preceding factors provide an impetus for evaluating and maintaining the proficiency of microbiologists and technologists who perform this process.

The central laboratory microbiologists who determined the expected values for slides throughout these quality assurance programs were subjected to in-house Gram stain interpretation proficiency testing. On a quarterly basis, all microbiologists reported findings for a randomly selected slide set. Data were pooled, and the expected value for each slide was derived from the majority of responses for each parameter. Unlike quality assurance program 1, which granted credit for results within ±1 gradation of an expected value for quantitation, in-house proficiency testing used higher stringency (i.e., only one acceptable value for quantitation). The microbiologist who prepared the proficiency material and graded the responses for quality assurance program 1 was one of two microbiologists who performed at 100% of the expected value for in-house proficiency testing during the time frame of this investigation. The central laboratory microbiologists demonstrated a mean in-house proficiency rate of 95%. No significant differences existed between the average microbiologist proficiency score and the mean demonstrated by the microbiology laboratory as a whole (P ≥ 0.06). These data confirmed the validity of the expected values.

Data from both quality assurance programs demonstrated the competency of satellite laboratorians in the interpretation of Gram-stained material. However, greater proficiency was found in host cell analysis (≥93%) than in bacterial analysis (≥84%). The rate of successful interpretation of host cells was markedly higher than that observed in an intralaboratory study (7). Cooper et al. (7) limited their study to lower respiratory tract smears and exercised more stringency in their grading of quantitative measures. However, the satellite technologists in our study may have had more breadth of knowledge in hematology and sterile body fluid analysis.

Quality assurance program 1 yielded higher rates of proficiency than did quality assurance program 2. This was true for each component of the monitoring and was realized at all three satellite laboratories. While intralaboratory technologist collaboration may or may not have played a role in the increased frequency of correct responses in quality assurance program 1, one aspect that could have played a role was the nature of the Gram-stained smear itself. To ensure consistency in the survey, specimens were preselected and smeared onto slides, with contents stained, coverslipped, and previewed by the microbiologist at the central laboratory before presentation to the three satellite laboratories. In contrast, Gram-stained smears encountered in quality assurance program 2 were prepared during routine operations of the satellite laboratory. As such, there may have been greater opportunity for microbiologists to review overdecolorized gram-positive cocci and degenerated leukocytes—cell types that may have been incorrectly reported by satellite technologists.

A key component of any continuous quality assurance monitoring program is ad libitum intervention (9). Throughout this study, four major means of correction were implemented. First, Gram-stained smears from either quality assurance program 1 or 2 that yielded an egregiously incorrect response from a satellite laboratorian were returned to a designated point person from that laboratory. This individual personally reviewed the slide with the technologist and discussed any discrepancies. Second, one-on-one interactions at a dual-headed microscope took place between a central laboratory microbiologist and the satellite laboratorian. Third, personnel from the satellite laboratories and the central laboratory formed a system-wide microbiology subcommittee, which provided a conduit for the exchange of data, policies, and concerns. Finally, digital images of interesting findings from quality assurance program 2 were disseminated via interlaboratory electronic mail in the format of a case presentation. Comprising this document were the chart review, organism epidemiology, and discussion of the relevance of the Gram-stained smear in the specific case. These mailings also provided a forum for discussion of aberrant findings in Gram-stained preparations, such as organisms partially treated with antimicrobial agents (4, 12).

Analysis of variance data for the four quarters of quality assurance program 1 (Fig. 3) revealed an increase in Gram stain interpretation proficiency. Analogous improvement was seen via quality assurance program 2. However, neither hospital A nor hospital B demonstrated any improvement in quality assurance program 1 or 2 throughout the course of this monitoring period. It is likely that these two laboratories had the most experience with satellite microbiology processing. In further support of this statement, hospitals A and B possessed the two highest proficiency rates (Fig. 1). In contrast, hospital C showed significant improvement in total Gram stain interpretation proficiency (Fig. 4) and the interpretation of bacteria in both quality assurance programs.

In conclusion, we describe a quantitative measure of Gram stain interpretation proficiency that can be applied to challenges of satellite laboratorians and an off-site review of technologist performance. These monitors can assess proficiency to the level of the laboratory, technologist, cell type, and quantitative measure. Technologist challenge is best utilized as an educational tool. However, an off-site review program is likely a better indicator of overall day-to-day Gram stain interpretation proficiency. When combined, these measures enable a multisite microbiology service to generate valid data both for patient care and for the practices of the clinical microbiology laboratory.