Performance of Various Testing Methodologies for Detection of Heteroresistant Vancomycin-Intermediate Staphylococcus aureus in Bloodstream Isolates

Abstract

The best screening method for detecting heteroresistant vancomycin-intermediate Staphylococcus aureus (hVISA) remains unclear. Using population analysis profiling utilizing the area under the concentration-time curve (PAP-AUC) as the gold standard, we screened 458 consecutive methicillin-resistant S. aureus (MRSA) bloodstream isolates to determine the most accurate and cost-effective testing strategy to detect the presence of heteroresistance. All isolates were also tested using the macromethod Etest (MET) and glycopeptide resistance detection (GRD) Etest. The MIC was determined by several methods, including standard vancomycin Etest, vancomycin broth microdilution (BMD), and Vitek2 testing. Fifty-five (12%) hVISA and 4 (1%) VISA isolates were detected by PAP-AUC. Compared to PAP-AUC, the sensitivities and specificities of MET, GRD Etest, BMD (using a MIC cutoff of ≥2 mg/liter), and standard vancomycin Etest (using a MIC cutoff of ≥2 mg/liter) were 89 and 55%, 71 and 94%, 82 and 97%, and 71 and 94%, respectively. Combination testing increased the overall testing accuracy by reducing the number of false-positive results. Cost was determined predominately by the number of PAP-AUC runs required following a screening assay. The most cost-effective strategy was BMD (using a MIC cutoff of ≥2 μg/ml) as a standalone assay or in combination with PAP-AUC, provided that BMD testing was batched. GRD Etest remained an alternative, with 71% of hVISA isolates detected. Prevalence influenced both cost and test accuracy, with results remaining unchanged for hVISA prevalences of up to 25%. Implementation of any testing strategy would therefore be dependent on balancing cost with accuracy in a given population and clinical context.

INTRODUCTION

Heterogenous vancomycin-intermediate (heteroresistant) Staphylococcus aureus (hVISA) is characterized by the presence of a resistant subpopulation, usually at a frequency of 1 in 10⁶, in an otherwise fully susceptible population (26). Increased cell wall thickness is a consistent feature of these isolates (11). In vitro, it has been demonstrated that these isolates emerge from vancomycin-sensitive Staphylococcus aureus (VSSA) isolates under appropriate selection pressure (6).

Subsequent to its first description in Japan in 1997 (8), hVISA has been reported worldwide, with the first Australian isolates reported in 2001 (24). The prevalence of hVISA varies worldwide, with rates dependent on multiple factors. These include testing methodologies employed, patient populations tested, and the source of the original Staphylococcus aureus isolates selected (e.g., infection versus colonization isolates) (20). In Australia, hVISA represents approximately 9 to 13% of all methicillin-resistant S. aureus (MRSA) bloodstream isolates (3, 9).

The clinical significance of these isolates remains uncertain. Vancomycin treatment failures have been associated with hVISA infections (2, 3, 24). In addition, these isolates have been associated with high bacterial loads and persistent bacteremia (7, 13, 15). Despite these associations, no study has detected worse outcomes with hVISA infections than with infections with susceptible isolates.

The optimal laboratory detection of hVISA remains unclear (10). Many of the traditional susceptibility platforms prove unreliable because the inocula used are below the required threshold to detect resistant subpopulations (10, 26). A modified population analysis profile method using the area under the concentration-time curve (PAP-AUC) is considered the gold standard and most reliable method (25). However, PAP-AUC is very labor-intensive and time-consuming, which limits its use. As a consequence, several screening assays, including the macromethod Etest (MET) and glycopeptide resistance detection (GRD) Etest, have been developed (10, 22, 23). Controversy remains, however, regarding which testing algorithm to use for the accurate detection of hVISA. A recent review attempted to address this question and suggested using either a laboratory-directed approach, utilizing the vancomycin broth microdilution (BMD) MIC, MET, GRD Etest, and/or PAP, or a clinically directed approach based on treatment failure, defined as persistent bacteremia despite adequate vancomycin trough levels (10).

We therefore undertook this study to determine the utility of various testing strategies, the accuracy of testing algorithms, and the most cost-effective approach to detecting hVISA in patients with MRSA bacteremia, using PAP-AUC as the gold standard.

(This work was presented in part at the 50th Interscience Conference on Antimicrobial Agents and Chemotherapy, Boston, MA, 12 to 15 September 2010 [21].)

MATERIALS AND METHODS

Isolate selection.

All stored (−80°C) consecutive nonduplicate MRSA bloodstream infection (BSI) isolates from Liverpool Hospital, Sydney, Australia, during the period 1997 to 2008 were included in the study. Retrieved isolates were subcultured twice for 48 h on horse blood agar (HBA) prior to testing. For each patient episode, the initial isolate was examined. Multiple isolates from the same patient were included if these isolates represented (i) a new BSI episode, defined as an episode occurring more than 14 days after the initial isolate; and/or (ii) a persistent BSI episode, defined as a positive blood culture isolate occurring >3 days and <14 days after the initial isolate.

All runs included the following appropriate negative and positive controls: Mu3 (ATCC 700698; hVISA), Mu50 (ATCC 700699; VISA), and a vancomycin-sensitive Staphylococcus aureus strain (ATCC 29213). All testing results were read by 2 independent scientists. Discordant results between scientists were resolved by a third scientist, with 2 similar readings accepted as the true result.

Laboratory methods. (i) PAP-AUC ratio.

PAP-AUC was performed on all isolates as previously described (23). Briefly, following an overnight culture in brain heart infusion (BHI) broth (BBL, Becton Dickinson, MD), serial dilutions of the isolate were inoculated onto BHI agar (BBL, Becton Dickinson, MD) plates containing increasing vancomycin concentrations (0 to 8 μg/ml) and incubated for 48 h (in air at 35°C). All subsequent colonies were counted and plotted against the vancomycin concentration. Isolates were classified as hVISA or VISA if the ratio of the AUC of the test strain to that of Mu3 (ATCC 700698) was 0.9 to 1.3 or >1.3, respectively (23, 25). PAP-AUC runs with results close to the cutoff (AUCs of 0.8 to 1) were repeated.

(ii) Vancomycin BMD.

Vancomycin BMD MICs were determined according to CLSI criteria (4). In brief, isolate suspensions prepared in Mueller-Hinton II broth (MH II; cation adjusted) (BBL, Becton Dickinson, MD) were serial diluted from a starting concentration of 1 × 10⁶ CFU/ml with increasing concentrations of vancomycin (0.25 to 32 μg/ml). Following 24 h of incubation at 35°C in air, the MIC was recorded as the concentration of the first well with complete inhibition of growth by the naked eye. Since BMD is not a routine test in our laboratory, the test was repeated for a subset of isolates (n = 137, including all hVISA isolates [according to PAP-AUC results]) with high MICs (>1 μg/ml) to determine the reproducibility of our results.

(iii) MET.

Modified Etests were performed according to the manufacturer's instructions (AB Biodisk, Solna, Sweden). Briefly, 200 μl of a suspension at a 2 McFarland standard (prepared in BHI broth) was inoculated onto a 90-mm BHI agar plate and swabbed evenly. Vancomycin and teicoplanin Etest strips were applied to the dry agar surface and read after an incubation of 48 h at 35°C in air. The test was considered positive if (i) the teicoplanin MIC was ≥12 mg/liter or (ii) the teicoplanin MIC was ≥8 mg/liter and the vancomycin MIC was ≥8 mg/liter. METs with discordant results compared to the gold standard were repeated.

(iv) GRD Etest.

The GRD Etest (AB Biodisk, Solna, Sweden) was performed according to the manufacturer's instructions. Briefly, a saline suspension at a 0.5 McFarland standard was inoculated onto MH plus 5% blood agar plates (BBL, Becton Dickinson, MD) and incubated for 48 h at 35°C in air. Isolates were considered GRD positive if the MIC of vancomycin or teicoplanin was ≥8 μg/ml. GRD Etests with discordant results compared to the gold standard were repeated.

(v) Vancomycin Etest.

The vancomycin Etest MIC was determined according to the manufacturer's instructions (AB Biodisk, Solna, Sweden). Briefly, a saline suspension at a 0.5 McFarland standard was inoculated onto MHA (BBL, Becton Dickinson, MD) and incubated for 24 h at 35°C in air. The MIC was determined as the point at which the growth inhibition ellipse intersected the Etest strip.

(vi) Vitek2 testing.

Determination of the MIC by Vitek2 testing (AST-P579 Gram-positive susceptibility card; bioMérieux, Inc., Durham, NC) was performed for all isolates according to the manufacturer's instructions.

Statistical analysis, multiple testing strategy, cost-benefit analysis, and impact of disease prevalence.

The chi-square test was used for categorical data, using SAS, version 9.2, for Windows (SAS Institute, Cary, NC). All testing was compared to PAP-AUC as the gold standard. Testing accuracy was determined by the following equation: accuracy = (number of true negative results + number of true positive results)/number of tested isolates. Cost-benefit analyses and multiple testing strategies were compared using an algorithm adapted from the work of Crobach et al. (5) (Fig. 1), with test accuracy derived by the following equation for a given prevalence: accuracy = {(sensitivity of test 1 × sensitivity of test 2 × prevalence) + (1 − prevalence) × [specificity of test 1 + (1 − specificity of test 1) × (1 − specificity of test 2)]} × 100. The impact of disease prevalence on single-test accuracy was calculated using the following equation: accuracy = [sensitivity × prevalence + (1 − prevalence) × specificity] × 100.

Fig. 1.

Application of a two-step algorithm to test 10,000 isolates with a population hVISA prevalence of 10%. This resulted in a testing accuracy of 97.8% [(828 + 8,955)/10,000] based on the following equation: accuracy = (total number of true positive samples + total number of true negative samples)/total number of samples tested. (Adapted from reference 5 with permission of the publisher.)

RESULTS

Four hundred seventeen MRSA BSI cases were identified over the 12-year study period (Table 1). One hundred fifty-two cases had a subsequent blood culture after the initial blood culture (>3 and <14 days after the initial blood culture bottle), of which 41 (27%) were positive for MRSA. Population analysis profiling of all 458 isolates identified 55 (12%) hVISA and 4 (1%) VISA isolates. The 4 VISA isolates were excluded from the remaining analysis. The rates of hVISA prevalence varied widely throughout the 12 years, with a biphasic pattern (Table 1).

Table 1.

Numbers of MRSA BSI episodes and hVISA isolates and hVISA frequencies over a 12-year span

Period	No. of episodes			Total no. of isolates	No. of hVISA isolates	% hVISA isolates
	MRSA BSIa	Episodes with repeat blood culturesb	Episodes with subsequent positive blood culture
1997–1998	19	5	0	19	0	0
1999–2000	52	17	2	54	3	5.5
2001–2002	80	26	2	82	28	34.1
2003–2004	79	20	3	82	6	7.3
2005–2006	83	39	17	100	15	15
2007–2008	104	45	17	121	3	2.4
Total	417	152	41	458	55	12

^aMultiple isolates from the same patient were included if these isolates represented a new bacteremic episode, defined as occurring >14 days after the initial isolate.

^bRepeat cultures were taken >3 days and <14 days after the initial isolate.

hVISA isolates were detected in the initial blood culture in 93% (51/55 isolates) of cases, with the remaining 4 hVISA isolates confirmed with the subsequent blood culture only. The hVISA phenotype was stable, with no reversion to VSSA detected for hVISA infection episodes with repeat blood cultures.

Persistent bacteremia occurred in 39% (7/18 episodes) of hVISA infection episodes, compared to 25% (34/134 episodes) of VSSA infection episodes (the P value for the difference was not significant). Although limiting hVISA testing to these isolates would increase the prevalence to 17% (7/41 isolates), the overall effect would be small and would underestimate the total hVISA burden.

Vancomycin MICs determined by broth microdilution, standard Etest, and Vitek2 testing were determined for all isolates. A correlation between heteroresistance and MIC existed, irrespective of the method used. The likelihood of detecting heteroresistance increased with higher MICs (Fig. 2) (P < 0.001).

Fig. 2.

Proportions of hVISA isolates detected by different methods of vancomycin MIC determination. BMD, broth microdilution; Etest, standard Etest.

The sensitivity, specificity, negative and positive predictive values, and testing accuracy relative to PAP-AUC as the gold standard were determined for the various methods, including using various MIC cutoffs (Table 2). Vancomycin broth microdilution using a MIC cutoff of ≥2 μg/ml resulted in the highest specificity (97%) and was the most accurate test, giving an hVISA prevalence of 12%. Decreasing the MIC cutoff increased the testing sensitivity, with a corresponding decline in specificity and test accuracy. The GRD Etest performed similarly to a standard Etest using a MIC cutoff of ≥2 μg/ml, with an accuracy of 91.2%. Although the macromethod Etest was one of the more sensitive tests (89%), it had a low specificity and was thus the least accurate assay in determining the presence of heteroresistance. The low sensitivity and accuracy of the Vitek method precluded using this method for hVISA detection, and no further cost or performance analysis was done for this method.

Table 2.

Comparison of hVISA detection methods to PAP-AUC at hVISA prevalence of 12%^c

Method (MIC cutoff [μg/ml])	No. of isolatesa		Sensitivity (%)	Specificity (%)	NPV (%)	PPV (%)	% Accuracyb
	hVISA (TP = 55)	VSSA (TN = 399)
Etest (≥1.5)	50	263	91	66	98	27	69
BMD (≥1.5)	49	335	89	84	98	43	85
MET	49	218	89	55	97	21	59
BMD (≥2)	45	388	82	97	97	80	95
Etest (≥2)	39	375	71	94	96	62	91
GRD Etest	39	375	71	94	96	62	91
Vitek2 (≥2)	14	382	25	96	90	45	87

^aA total of 454 isolates were examined (4 VISA isolates were excluded).

^bAccuracy = (TP + TN)/total number of isolates tested (454 isolates).

^cTP, number of true positive results; TN, number of true negative results; NPV, negative predictive value; PPV, positive predictive value; BMD, vancomycin broth microdilution; GRD Etest, glycopeptide resistance detection Etest; MET, macromethod Etest.

The cost estimate and labor time required for each of the different testing methods are shown in Table 3. For PAP-AUC and BMD, costs of consumables declined with increased numbers of samples tested, with an optimal run size of 7. Since costs and labor times were prohibitive for single PAP-AUC and BMD tests, the remaining calculations are for batched testing using the optimal run size (i.e., 7).

Table 3.

Labor times and costs for hVISA detection by various testing methods

Test	No. of isolates per run	Cost (AU$) of reagents per run	Total labor time (h) per runa	Total cost (AU$) per isolateb
PAP-AUC	1–7	14–40	7–9	70–320
MET	1	9	0.5	29
GRD Etest	1	2	0.25	12
BMD	1–7	3–8	0.75	8–38
Etest	1	5	0.25	15
Vitek2	1	8	0.25	18

^aLabor time includes setup, result reading, and interpretation.

^bTotal cost per isolate = [(reagent cost + labor cost [at AU$40/h]) × time]/number of isolates per run. Cost ranges are representative of 1 to 7 isolates per run. Costs are quoted in Australian (AU) dollars.

To increase overall accuracy and to determine the most cost-effective testing strategy, multiple testing algorithms were evaluated (Fig. 1). Only testing algorithms that resulted in an overall accuracy of ≥90% and detected more than 70% of hVISA isolates are presented in Table 4. Regardless of the number of tests included, overall sensitivity (i.e., the percentage of hVISA isolates detected) was determined predominantly by the sensitivity of the initial test. In contrast, overall testing accuracy was determined predominantly by the specificity of the final assay. Since PAP-AUC was the most expensive test, the addition of PAP-AUC to any other test increased the cost and labor time required. However, this corresponded with an increase in overall accuracy by effectively differentiating true positive results from false-positive results. Of all the testing strategies analyzed, BMD using a MIC cutoff of ≥2 μg/ml was the most cost-effective single test (11% of PAP-AUC cost), with 82% of hVISA isolates detected. The addition of PAP-AUC following a positive BMD run (using a MIC cutoff of ≥2 μg/ml) was the most effective combination. For laboratories that do not have access to BMD or PAP-AUC, testing using the GRD or standard Etest (with a MIC cutoff of ≥2 μg/ml) will give the highest predictive value for a reasonable cost (Table 4).

Table 4.

Performance costs and labor times required for various testing strategies to test 454 MRSA isolates for the presence of hVISA at a heteroresistance prevalence of 12%

Test 1 (MIC cutoff [μg/ml])	Test 2 (MIC cutoff [μg/ml])	Test 3	% Accuracya	% of hVISA isolates detected	Total cost (AU$)b (% of PAP-AUC cost)	Total labor time (h) (% of PAP-AUC time)
PAP-AUC			100	100	32,200 (100)	460 (100)
Etest (≥1.5)	PAP-AUC		98.9	91	19,789 (61)	352 (77)
BMD (≥1.5)	PAP-AUC		98.7	89	11,501 (36)	193 (42)
MET	PAP-AUC		98.7	89	29,145 (91)	520 (113)
BMD (≥2)	PAP-AUC		97.8	82	7,598 (24)	121 (26)
BMD (≥1.5)	Etest (≥1.5)	PAP-AUC	97.7	81	11,360 (35)	188 (41)
BMD (≥1.5)	MET	PAP-AUC	97.5	79	12,374 (38)	206 (45)
BMD (≥2)	Etest (≥1.5)	PAP-AUC	96.9	75	8,615 (27)	138 (35)
BMD (≥2)	MET	PAP-AUC	96.7	73	9,329 (29)	151 (33)
Etest (≥2)	PAP-AUC		96.5	71	11,196 (35)	194 (42)
GRD Etest	PAP-AUC		96.5	71	9,834 (31)	194 (42)
BMD (≥2)			95.4	82	3,632 (11)	49 (11)
BMD (≥2)	MET		93.9	73	5,623 (17)	83 (18)
BMD (≥2)	Etest (≥1.5)		93.5	75	4,662 (14)	66 (14)
Etest (≥2)			91.2	71	6,810 (21)	114 (25)
GRD Etest			91.2	71	5,448 (17)	114 (25)

^aAccuracy = (number of true positive samples + number of true negative samples)/total number of samples.

^bBMD and PAP-AUC costs are based on 7 samples per run. For multitest strategies, only positive samples from the prior test were tested (Fig. 1).

Both accuracy and cost were codependent on hVISA prevalence. Thus, we undertook a sensitivity analysis of testing accuracy to determine whether our findings changed with differing prevalence rates (Fig. 3). Tests with higher sensitivities were more accurate with increasing prevalences, while tests with higher specificities became less accurate. Although thresholds existed at which certain tests were more accurate than others, our findings remained unchanged for hVISA prevalences between 5 and 25%.

Fig. 3.

Sensitivity analysis of testing accuracies sorted by hVISA prevalence. BMD, broth microdilution; Etest, standard Etest. The GRD Etest was equivalent in performance to the standard Etest using a MIC cutoff of ≥1.5 μg/ml.

One hundred thirty-seven (30%) isolates had repeated BMD testing, with 99% (135/137 isolates) reproducibility. Similarly, 6% (29/454 tests) of all PAP-AUC tests were repeated and represented PAP-AUC results of between 0.8 and 1.1. Although the AUC varied by a mean of ±0.1, 90% (26/29 classifications) of the original PAP-AUC classifications were reproducible. Repeat testing of samples with discordant GRD Etest and MET results was performed, with overall agreement with the original discordant result in 99% of cases for both methods.

DISCUSSION

Screening for heteroresistance remains difficult, as no optimal laboratory approach has been determined. Our study, containing the largest number of consecutive MRSA bloodstream isolates categorized by the PAP-AUC gold standard, provides useful insights into the utility of the various proposed testing algorithms (10). Furthermore, the overall prevalence of hVISA (12%) detected was sufficiently high to make assessment of the various screening assays possible.

A clinically directed approach limiting testing to high-risk patients would underestimate the overall hVISA burden. An alternate strategy is therefore required if documentation of the total burden is necessary. For patient management, this may not be essential, as hVISA isolates in less severe infections are not associated with significantly worse clinical outcomes (9, 20).

A laboratory-directed approach, in contrast, significantly improves the ability to determine the hVISA burden. However, no single test was accurate enough compared to PAP-AUC. The GRD Etest sensitivity (74%) was lower than those in other published reports (89 to 93%) (12, 27). Similarly, the specificity of MET (54%) in our study was significantly lower than those in previous reports (88 to 96%) (12, 22, 27). Reasons for these differences are unclear and may reflect specific MRSA subtype characteristics. However, recent MET and GRD Etest assessments performed on consecutive clinical isolates showed significantly lower sensitivities (44% to 69%) and specificities (48% to 98%) (1, 17, 18) than those in previous laboratory-characterized evaluations. This suggests that some of these differences may reflect hVISA phenotype instability (22).

The proportion of hVISA isolates detected, similar to the case in other studies, increased with higher MICs, irrespective of the testing method (14, 16, 19), and thus MIC cutoffs were evaluated by method to determine appropriate screening assays. An Etest using a MIC cutoff of ≥1.5 μg/ml had the highest sensitivity (91%) but was associated with a test accuracy of 69%. In contrast, vancomycin broth microdilution using a MIC cutoff of 2 μg/ml was the most accurate test (95%) but had a corresponding decline in sensitivity. Including cost as part of the analysis, BMD was the most cost-effective strategy. However, this method requires batch testing, which may be infrequent in settings with a low MRSA prevalence and may prompt the use of the GRD Etest as a suitable alternative test.

An alternative strategy is to employ a 2- or 3-step combination testing algorithm. Although combinations are able to increase the overall accuracy of testing by reducing the number of false-positive results, the percentage of hVISA isolates detected does not increase and is determined by the initial test. In assessing cost, the highest cost and most labor time are associated with PAP-AUC; thus, decreasing or eliminating the need for PAP-AUC significantly impacts the overall cost and labor required. Among the testing strategies examined, a MET screening assay followed by PAP-AUC was similar in cost to PAP-AUC (91% of PAP-AUC cost) and thus would negate MET as a viable screening assay.

Both accuracy and cost are dependent on the prevalence of hVISA, with our findings remaining unchanged for hVISA prevalences of up to 25%. Application of our results should be instituted only with knowledge of local hVISA epidemiology. The optimal choice of testing strategy for a given institution is thus dependent on balancing the prevalence and required rate of hVISA detection against the cost and labor required. In our opinion, the combination of BMD (using a MIC cutoff of ≥2 μg/ml) followed by PAP-AUC is the most cost- and time-effective strategy. Alternatively, if detection of hVISA is critical, despite increasing the cost, BMD using a MIC cutoff of ≥1.5 μg/ml followed by PAP-AUC is the testing strategy to use. However, batch testing is required for these cost savings, so this strategy may not be practical in instructing patient management.

Unfortunately, no ideal strategy was found that would reliably allow for hVISA testing to be introduced into routine laboratory testing, and thus further research is needed in this area.

This study has several limitations. Since no genetic marker for heteroresistance exists, PAP-AUC is assumed to be the gold standard test for detecting the hVISA phenotype (10). The variability and problems with reproducibility of laboratory testing with respect to heteroresistance are well documented (22). Repeat testing of all isolates may have altered our results. However, this was not detected in the subset of isolates retested. hVISA phenotypes are known to be unstable, with the ability to revert to VSSA under various conditions, including isolate passage on vancomycin-free media (22). It is unclear whether storage and subculture of the isolates impacted our results. However, all hVISA MICs were retested, with no phenotype instability detected. VISA isolates were excluded from the analysis. However, our results did not change if these were considered hVISA isolates (data not shown). Finally, intraobserver variability was not corrected for with multiple operators over the 12-year study period.

In conclusion, we believe that this study contributes to an increasing understanding of detection of hVISA in the current literature. Previously accepted screening methods (MET and GRD Etest) did not perform as well as expected in our study, raising the possibility that they may not be as reliable as previously thought. Based on our data, laboratory implementation of optimal hVISA testing strategies should take into account clinical settings, with the aim of balancing maximum detection and cost. Ongoing research is required to determine better detection methods that are less costly and time-consuming than those currently available, allowing for more widespread laboratory implementation.