Criteria for Reducing Unnecessary Testing for Herpes Simplex Virus, Varicella-Zoster Virus, Cytomegalovirus, and Enterovirus in Cerebrospinal Fluid Samples from Adults

Abstract

Excessive utilization of laboratory diagnostic testing leads to increased health care costs. We evaluated criteria to reduce unnecessary nucleic acid amplification testing (NAAT) for viral pathogens in cerebrospinal fluid (CSF) samples from adults. This is a single-center split retrospective observational study with a screening cohort from 2008 to 2012 and a validation cohort from 2013. Adults with available results for herpes simplex virus 1/2 (HSV-1/2), varicella-zoster virus (VZV), cytomegalovirus (CMV), or enterovirus (EV) NAAT with CSF samples between 2008 and 2013 were included (n = 10,917). During this study, 1.3% (n = 140) of viral NAAT studies yielded positive results. The acceptance criteria of >10 nucleated cells/μl in the CSF of immunocompetent subjects would have reduced HSV-1/2, VZV, CMV, and EV testing by 63%, 50%, 44%, and 51%, respectively, from 2008 to 2012. When these criteria were applied to the 2013 validation data set, 54% of HSV-1/2, 57% of VZV, 35% of CMV, and 56% of EV tests would have been cancelled. No clinically significant positive tests would have been cancelled in 2013 with this approach. The introduction of a computerized order entry set was associated with increased test requests, suggesting that computerized order sets may contribute to unnecessary testing. Acceptance criteria of >10 nucleated cells/μl in the CSF of immunocompetent adults for viral CSF NAAT assays would increase clinical specificity and preserve sensitivity, resulting in significant cost savings. Implementation of these acceptance criteria led to a 46% reduction in testing during a limited follow-up period.

INTRODUCTION

Nucleic acid amplification testing (NAAT) for viral etiologies of central nervous system (CNS) infections provides rapid diagnosis for optimization of therapy (1, 2). However, this testing can be costly and may not always be used judiciously (3). In this era of cost containment, with a growing emphasis on the optimal use of diagnostic tests (4–8), we assessed ordering practices for viral NAAT with cerebrospinal fluid (CSF) specimens. Our objective was to develop criteria to reduce unnecessary testing.

Herpes simplex virus 1 (HSV-1) is the most common treatable cause of viral encephalitis (9), while three of the most commonly identified causes of aseptic meningitis are herpes simplex virus 2 (HSV-2), varicella-zoster virus (VZV), and enteroviruses (EV) (10, 11). In addition, cytomegalovirus (CMV) infections of the CNS may be of clinical concern among immunocompromised individuals (12). NAAT has become the gold standard for diagnosis of most viral CNS infections (1, 2).

Viral CSF NAAT has several important benefits. First, early diagnosis enables adequate administration of antiviral treatment for HSV-1/2, VZV, and CMV, if necessary. Second, while no targeted EV therapies currently exist, utilization of NAAT for EV diagnosis has been correlated with reductions in length of stay, ancillary testing, antibiotic utilization, and costs in pediatric populations (13, 14). However, utilization of viral NAAT assays in cases with low pretest probability may unnecessarily increase costs while decreasing the positive predictive value of the testing (13, 15). Several previous studies evaluated the implementation of laboratory screening criteria to reduce superfluous CSF NAAT for HSV-1/2 (16, 17). However, those studies were restricted to HSV-1/2 and had limited sample sizes.

Herein we describe a retrospective study of nearly 11,000 CSF NAAT assays and we propose screening criteria for HSV-1/2, VZV, CMV, and EV testing. Using data from 2008 to 2012, we identified CSF characteristics with very high negative predictive values for HSV-1/2, VZV, CMV, and EV. We then retrospectively applied those screening criteria to CSF requests for a 1-year period (2013), to assess the cost savings and impact of the proposed algorithm. Finally, we describe the impact of implementation of the acceptance criteria described herein.

(This work was presented in part at the 49th Annual Meeting of the Academy of Clinical Laboratory Physicians and Scientists, San Francisco, CA, May 30, 2014.)

MATERIALS AND METHODS

Study overview.

Barnes-Jewish Hospital (BJH) is an urban, adult, tertiary care hospital in St. Louis, MO. Data on all HSV-1/2, VZV, CMV, and EV NAAT studies ordered at BJH between January 1, 2008, and December 31, 2013, were obtained from the laboratory information system. Data on laboratory CSF characteristics and immune status were collected for all subjects with positive NAAT results and either twice as many randomly selected subjects with negative results (2008-2012 cohort) or 10% of all subjects with negative results (2013 cohort). Subjects with negative NAAT results were selected with a random number generator. Data on protein levels, glucose levels, cell counts, and cell differential counts for the CSF specimens were collected from the electronic medical record (EMR). Postmortem samples and samples from individuals less than 18 years of age were excluded. Individuals lacking CSF protein and/or cell count data were excluded from analyses. CMV test results from May to October 2009 were not available, as those tests were performed elsewhere; those results were excluded. For the purposes of analysis, positive NAAT results from subjects who met the various acceptance criteria were considered true-positive results. This study was approved by the Washington University in St. Louis Institutional Review Board/Human Research Protection Office.

Criteria for defining immunocompromised subjects.

Subjects were classified as immunocompromised if they met any of the following criteria (defined a priori) at the time of test ordering: (i) HIV infection (independent of CD4⁺ cell count), (ii) oral or intravenous immunosuppressant therapy, (iii) use of cytotoxic medications, including chemotherapy, (iv) history of solid organ or bone marrow transplant, (v) aplastic anemia, or (vi) primary immunodeficiency. To assess the risk of bias, the immune status of all subjects with positive NAAT results and an equal number of subjects with negative results was assessed by a second coder (n = 244). The kappa statistic for interrater reliability was 0.93.

Diagnostic testing.

All viral NAAT assays were qualitative assays performed at St. Louis Children's Hospital virology laboratory (St. Louis, MO). Prior to August 2008, HSV-1/2 was detected with a laboratory-developed NAAT gel-based assay; after that time, it was detected with the MultiCode-RTx HSV 1&2 kit (EraGen), which was validated internally for use with CSF samples. VZV was detected with a laboratory-developed NAAT assay with gel-based detection of the amplification product. Prior to August 2009, CMV testing was performed with a laboratory-developed NAAT gel assay, as described previously (18); after August 2009, CMV was detected with a TaqMan real-time PCR assay (19). EV was detected after July 2009 with the Xpert EV assay (GeneXpert; Cepheid) and prior to that time by using an analyte-specific reagent assay on the SmartCycler system (Cepheid). CSF cell counts were determined by automated flow cell digital imaging (iQ200; Iris Diagnostics). White blood cell (WBC) differential counts were manually determined by counting up to 100 cells prepared by cytocentrifugation. The reference ranges reported for CSF WBC counts and protein concentrations were 0 to 5 cells/μl and 15 to 45 ng/ml, respectively. The reference ranges reported for CSF neutrophil and lymphocyte frequencies were 40 to 80% and 0 to 6%, respectively. Reference ranges for CSF glucose levels and red blood cell (RBC) counts were not reported.

Computerized physician order entry analysis.

Computerized physician order entry (CPOE) implementation occurred for inpatient services, excluding the emergency department, between July and October 2009. Due to limitations of our EMR, it was not possible to accurately determine patient locations at the time of sample collection. Therefore, samples from the emergency department were included in the analysis, with the potential limitation of underestimating the effects of CPOE implementation on ordering practices. Since CPOE was implemented in different divisions of the hospital at different times during the 4-month implementation period, this period was excluded from analysis.

Cost analysis.

Annual cost savings were determined by multiplying the test price by the frequency of cancelled negative tests and the number of negative tests ordered per year. The 2014 Centers for Medicare and Medicaid Services (CMS) midpoint price was $64.70 for each NAAT assay evaluated here (http://www.cms.gov/Medicare/Medicare-Fee-for-Service-Payment/ClinicalLabFeeSched/clinlab.html). As the viral NAAT samples were sent out to an external laboratory, the variable cost of each NAAT assay is proprietary information, although similar to the CMS midpoint prices.

Statistical analysis.

Two-tailed Mann-Whitney tests were used to compare data. P values of <0.05 were considered significant. Statistical analysis was performed using Prism GraphPad 6.0.

RESULTS

Subject demographics.

During the study period (2008 to 2013), the most commonly ordered CSF viral NAAT assays at BJH were for HSV-1/2, VZV, CMV, and EV. We used data from 2008 to 2012 to evaluate the acceptance criteria and data from 2013 to validate the optimal criteria. In 2008 to 2012, 8,668 tests were performed, of which only 1.4% were reported as positive (Table 1). Of 4,799 HSV-1/2 tests from 4,297 subjects, 56 tests (1.2%) were positive; 13 (0.27%) were positive for HSV-1 and 43 (0.91%) were positive for HSV-2. Of 1,588 VZV tests from 1,456 subjects, 27 (1.7%) were positive. Of 1,428 CMV tests from 1,284 subjects, 20 (1.4%) were initially reported as positive. Of 956 EV tests from 935 subjects, 20 (2.1%) were positive.

TABLE 1.

Frequency of testing and rates of positive viral NAAT results with adult CSF samples (2008–2012)

Test and result	No. (%) of samples	Age (median [IQR])a (yr)
HSV-1/2
Negative	4,743	52 (38–65)
Positive	56b (1.1)	50 (37–63)
VZV
Negative	1,561	53 (40–65)
Positive	27 (1.7)	45 (36–57)
CMV
Negative	1,408	52 (39–64)
Positive	20 (1.4)	46 (39–63)
EV
Negative	956	52 (36–64)
Positive	20 (2.0)	28 (25–36)
Total
Negative	8,668	52 (38–65)
Positive	123 (1.4)	51 (36–63)

^aIQR, interquartile range.

^bThirteen samples were positive for HSV-1 and 43 were positive for HSV-2.

For all four tests combined, subjects with negative results were of similar ages (median, 52 years [interquartile range, 38 to 65 years]), compared with those with positive results (median, 51 years [interquartile range, 36 to 63 years]). However, in contrast to HSV-1/2, VZV, and CMV results, EV-positive subjects were significantly younger (median, 28 years [interquartile range, 25 to 36 years]) than EV-negative subjects (median, 52 years [interquartile range, 36 to 64 years]; P < 0.001), consistent with the known epidemiology of EV (20).

Ordering practices.

The low prevalence of positive results and the variation in age between EV-positive and EV-negative subjects prompted an evaluation of ordering practices. CPOE for inpatient units was implemented throughout the hospital between July and October in 2009. After CPOE implementation, clinicians could request NAAT assays individually or from an order set that included laboratory tests for pathogens of the central nervous system. The order set included 14 options for infectious disease testing, including HSV-1/2, VZV, CMV, and EV NAAT (order set A). However, the order set was modified 22 months later and HSV-1/2 was removed (for unclear reasons), while VZV, CMV, and EV remained (order set B). This required clinicians to initiate a separate HSV-1/2 NAAT order. HSV-1/2 NAAT orders increased 5% with order set A and then decreased back to pre-CPOE levels when HSV-1/2 was removed from the order set (order set B). In contrast, orders for VZV, CMV, and EV (all included in both order sets A and B) increased 62%, 62%, and 54%, respectively, over the same time period (Fig. 1A). Interestingly, this increase in VZV, CMV, and EV test ordering was associated with an 11% relative decrease in positive NAAT results per month, suggesting that increased ordering did not improve detection of viral CNS infections.

FIG 1.

Correlates of unnecessary viral NAAT with CSF samples. (A) Computerized physician order entry (CPOE) for non-emergency department inpatient orders and an accompanying CSF order set were implemented in the autumn in 2009. VZV, CMV, and EV NAAT assays were in the order set, while the HSV-1/2 assay was not. Orders increased disproportionately (shown above the bars) for tests in the order set. (B) Despite the seasonality of EV meningitis, EV orders were not seasonal, suggesting unnecessary testing.

Given the discrepancy in age between EV-positive and EV-negative subjects, we evaluated ordering practices for EV tests. EV meningitis is seasonal, with predominance in the summer and early autumn (20). Of the 20 EV-positive samples, 19 were collected between April and October (Fig. 1B). One EV-positive test result was noted in December, from an immunocompromised subject with three previous EV-positive results in that calendar year. In contrast to the seasonal prevalence of EV meningitis (20), EV test ordering practices did not exhibit seasonal variation.

Laboratory characteristics of NAAT-positive and NAAT-negative CSF samples.

We sought to assess laboratory characteristics of CSF samples that correlated with positive NAAT results for HSV-1/2, VZV, CMV, and EV, with the goal of establishing laboratory acceptance criteria for viral NAAT. We retrospectively assessed cases from 2008 to 2012 to screen the predictive values of CSF protein levels, glucose levels, RBC counts, nucleated cell counts, lymphocyte percentages, and neutrophil percentages for viral NAAT results.

Viral CNS infections are typically associated with normal or slightly decreased CSF glucose concentrations, elevated protein contents, and lymphocyte-predominant pleocytosis (21). The median protein concentration was greater for HSV-1/2-positive subjects than for HSV-1/2-negative subjects (101 versus 43 mg/dl; P = 0.002) (Fig. 2A). Similarly, VZV-positive subjects had increased protein concentrations, compared to VZV-negative subjects (94.5 versus 45.0 mg/dl; P = 0.002). However, there were no significant differences in protein levels for subjects with positive versus negative CMV (61.5 versus 62.0 mg/dl; P = 0.99) and EV (48.0 versus 60.0 mg/dl; P = 0.61) results, suggesting that these parameters would not be broadly applicable viral NAAT inclusion criteria.

FIG 2.

Cytological and biochemical correlates of viral NAAT results. (A) CSF protein levels were higher in HSV-1/2- and VZV-positive samples but not in CMV- or EV-positive samples. (B) CSF glucose concentrations were lower in HSV-1/2-positive samples than in negative samples. (C) CSF RBC counts were higher in HSV-1/2-positive samples than in negative samples. (D) CSF nucleated cell counts were higher in NAAT HSV-1/2-, VZV-, CMV-, and EV-positive samples. (E) CMV- and EV-positive samples had reduced CSF lymphocyte counts, relative to negative samples, while HSV-1/2-positive samples had relative lymphocytosis. (F) EV-positive samples had increased CSF neutrophil counts, relative to EV-negative samples. Dashed lines, upper and lower limits (0 if not shown) of the reference ranges. No reference range was established for CSF glucose levels. Solid line in panel D, proposed nucleated cell count acceptance criterion. Whiskers, 2.5% and 97.5% limits. Dots, outliers.

Median CSF glucose concentrations were significantly lower in HSV-1/2-positive subjects than in HSV-1/2-negative subjects (56.5 mg/dl versus 69.5 mg/dl; P < 0.001). There were no significant differences in glucose concentrations for subjects with positive versus negative VZV, CMV, and EV results (Fig. 2B). However, median RBC counts were significantly higher in HSV-1/2-positive subjects than in HSV-1/2-negative subjects (25 versus 8.5 cells/μl; P = 0.029) (Fig. 2C), while RBC counts were similar for subjects with positive versus negative VZV, CMV, and EV results. Median CSF nucleated cell counts were significantly higher for HSV-1/2-positive (250 versus 1.5 cells/μl; P < 0.001), VZV-positive (87 versus 3.5 cells/μl; P < 0.001), CMV-positive (11.5 versus 1.0 cells/μl; P = 0.019), and EV-positive (113 versus 6.0 cells/μl; P < 0.001) subjects than for the corresponding negative subjects (Fig. 2D).

The CSF specimens from HSV-1/2-positive subjects were relatively lymphophilic, compared to those from HSV-1/2-negative subjects (84.0% versus 67.5%; P = 0.001) (Fig. 2E), while there was no difference in neutrophil frequencies (Fig. 2F). The frequencies of lymphocytes and neutrophils did not significantly differentiate VZV-positive and VZV-negative subjects (Fig. 2E and F). The CSF specimens from CMV-positive subjects had relative lymphopenia, compared to those from CMV-negative subjects (48.0% versus 78.0%; P < 0.001). The CSF specimens from EV-positive subjects had relative lymphopenia (60.0% versus 80.0%; P = 0.026) and neutrophilia (14.5% versus 3.0%; P = 0.004), compared to those from EV-negative subjects.

Screening of laboratory acceptance criteria.

Based on our findings and those of others (3, 16, 17, 22), we evaluated three different laboratory acceptance criteria for HSV-1/2, VZV, CMV, and EV, including (i) WBC counts of >5 cells/μl or protein concentrations of >50 mg/dl, (ii) CSF WBC counts of >5 cells/μl, and (iii) WBC counts of >10 cells/μl (Table 2). All three laboratory cutoff criteria were evaluated with or without automatic acceptance of immunosuppressed subjects; therefore, six different acceptance criteria were assessed.

TABLE 2.

Assessment of screening criteria and cost savings according to viral etiology^a

Criteria used	HSV-1/2		VZV		CMV		EV		EV (April to October)		Total
	Positive tests accepted	Negative tests cancelled	Positive tests accepted	Negative tests cancelled	Positive tests accepted	Negative tests cancelled	Positive tests accepted	Negative tests cancelled	Positive tests accepted	Negative tests cancelled	Positive tests accepted	Negative tests cancelled
Total no.	56	112	27	54	20	40	20	40			123	246
No. with cell counts	55	112	27	54	20	39	20	39			122	244
No. with protein levels	56	112	26	54	19	40	20	39			121	245
WBC count of >5 cells/μl or protein level of >50 mg/dl (% [no./total no.])	91 (50/55)	55 (62/112)	88 (23/26)	41 (22/54)	68 (13/19)	21 (8/39)	100 (20/20)	33 (13/39)	95 (19/20)	80 (32/40)	88 (106/120)	50 (123/244)
WBC count of >5 cells/μl (% [no./total no.])	85 (47/55)	77 (86/112)	78 (21/27)	63 (34/54)	25 (5/20)	51 (20/39)	100 (20/20)	49 (19/39)	95 (19/20)	65 (26/40)	76 (93/122)	59 (144/244)
WBC count of >10 cells/μl (% [no./total no.])	85 (47/55)	83 (93/112)	67 (18/27)	69 (37/54)	25 (5/20)	74 (29/39)	95 (19/20)	64 (25/39)	90 (18/20)	75 (30/40)	73 (89/122)	66 (160/244)
WBC count of >5 cells/μl or protein level of >50 mg/dl or immunosuppressed (16) (% [no./total no.])	95 (52/55)	46 (52/112)	96 (25/26)	33 (18/54)	84 (16/19)	26 (10/39)	100 (20/20)	36 (14/39)	95 (19/20)	58 (23/40)	94 (113/120)	39 (94/244)
WBC count of >5 cells/μl or immunosuppressed (% [no./total no.])	95 (52/55)	58 (65/112)	96 (26/27)	48 (26/54)	80 (16/20)	28 (11/39)	100 (20/20)	38 (15/39)	95 (19/20)	60 (24/40)	93 (114/122)	48 (117/244)
WBC count of >10 cells/μl or immunosuppressed (17) (% [no./total no.])	95 (52/55)	63 (71/112)	96 (26/27)	50 (27/54)	80 (16/20)	44 (17/39)	100 (20/20)	51 (20/39)	95 (19/20)	68 (26/40)	93 (114/122)	55 (135/244)
Optimal algorithm (% [no./total no.])											93 (113/122)	58 (141/244)

^aAnnual savings were calculated as price per test × % cancelled × total negative tests/5-year time period. The calculated savings were as follows: HSV-1/2, $38,907; VZV, $10,100; CMV, $7,942; EV (April to October), $8,041; total, $64,989.

The least stringent acceptance criteria assessed required either a WBC count of >5 cells/μl or a protein concentration of >50 mg/dl. This would have resulted in acceptance of 88% of positive samples (106/120 samples) and cancellation of 50% of negative samples (123/244 samples) (Table 2). Allowing for automatic acceptance of immunosuppressed subjects increased the number of positive samples accepted to 94% (113/120 samples), while it decreased the number of cancelled specimens to 39% (94/244 samples).

Acceptance based solely on the finding of >5 nucleated cells/μl adversely affected the number of positive samples accepted at 76% (93/122 samples) when immune status was not considered; however, a WBC count threshold of >5 cells/μl and automatic acceptance of immunosuppressed subjects increased the acceptance rate for positive samples to 93% (114/122 samples), with a cancellation rate of 48% for negative specimens (117/244 samples). Increasing the WBC count acceptance criterion from >5 cells/μl to >10 cells/μl with automatic acceptance of immunosuppressed subjects had no effect on the number of positive specimens accepted (93% [114/133 samples]), while it further increased the proportion of negative specimens cancelled from 48% to 55% (135/244 samples).

Therefore, acceptance criteria based on CSF WBC counts of >10 cells/μl with automatic acceptance of immunosuppressed subjects maximized acceptance of positive specimens and cancellation of negative specimens. These criteria would have resulted in the cancellation of 7% of positive specimens (8/122 samples) among 8,791 tests. These eight cancelled positive tests represent three of 55 HSV-1/2-positive NAAT assays, one of 27 VZV-positive NAAT assays, and four of 20 CMV-positive NAAT assays. Seven of these eight positive NAAT assays were deemed clinically insignificant by the treating clinicians. No EV-positive results would have been missed; however, if EV testing was restricted to the EV season (April to October), then one positive specimen would have been cancelled, while the proportion of negative tests cancelled would have increased from 51% (20/39 tests) to 68% (26/40 tests). The one missed EV-positive sample was from an immunosuppressed subject with three previous EV-positive test results in that calendar year, all of which occurred during the EV season and were associated with WBC counts of >10 cells/μl. The optimal acceptance criteria included testing specimens from all immunosuppressed subjects and samples from immunocompetent subjects with >10 cells/μl, in addition to cancellation of EV testing outside the EV season. Using a CMS midpoint price of $64.70 per test (http://www.cms.gov/Medicare/Medicare-Fee-for-Service-Payment/ClinicalLabFeeSched/clinlab.html), applying these acceptance criteria would have resulted in laboratory savings of $64,989 per year from 2008 to 2012 (Table 2).

Validation of laboratory acceptance criteria.

Next, we retrospectively applied the laboratory acceptance criteria determined with the 2008-2012 screening cohort to a 2013 validation cohort. In 2013, 17 (0.80%) of 2,126 tests were positive for HSV-1/2 (n = 9), VZV (n = 5), CMV (n = 2), or EV (n = 1) (Table 3). Applying the acceptance criteria of WBC counts of >10 cells/μl with automatic acceptance of immunosuppressed subjects and EV testing only from April to October would have resulted in the acceptance of 88% of positive tests (15/17 tests). Of the two tests not automatically accepted, one subject presented with acute appendicitis and developed postoperative altered mental status, aphasia, and right upper extremity weakness. He was ultimately diagnosed as experiencing a cerebrovascular accident, and the positive HSV-2 result was deemed clinically insignificant after an infectious disease consultation. The second subject presented with headache and vertigo and had a positive HSV-1 test result, which was deemed clinically insignificant after an infectious disease consultation. The patient was discharged with a diagnosis of migraine.

TABLE 3.

Validation of acceptance criteria (2013)

Test	No.				% (no./total no.)		Cost savings ($)
	Total	Negative	Positive	Indeterminate	Negative tests cancelled	Positive tests accepted
HSV-1/2	974	965	9	0	54 (51/95)	78 (7/9)a	33,518
VZV	485	480	5	0	57 (27/47)	100 (5/5)	17,841
CMV	429	427	2	0	35 (15/43)	100 (2/2)	9,637
EV	238	236	1	1	56 (18/32)	100 (1/1)	8,589
Total	2,126	2,108	17	1	51 (111/217)	88 (15/17)a	69,585

^aIn each case, both excluded tests were clinically insignificant, according to an infectious disease consultation.

In addition to accepting all clinically significant true-positive specimens in 2013, these acceptance criteria would have excluded two clinically insignificant positive tests and 51% of negative tests (111/217 tests). This would have saved $69,585, assuming the CMS midpoint prices, and improved the positive predictive value of NAAT.

Implementation of laboratory acceptance criteria.

We implemented the aforementioned CSF NAAT acceptance criteria in July 2014. Prior to implementation, we educated clinicians in the departments of neurology, internal medicine, and emergency medicine regarding the rationale and evidence for the acceptance criteria. At the time of manuscript preparation, the acceptance criteria had been in place for 137 days. We observed 16% fewer NAAT orders than during the same period in 2013, suggesting that our educational efforts might have been effective (Table 4). Thirty-six percent of requested tests were cancelled without a clinician override. The overall reductions in the various NAAT assays were 53%, 41%, 41%, and 38% for HSV, VZV, CMV, and EV, respectively, compared to the same time period in 2013. This corresponded to a 46% overall reduction in tests performed, with a savings of $24,004 (based on CMS midpoint pricing), which annualizes to $63,992. This is similar to the 51% of NAAT results that did not meet the acceptance criteria in 2013 and the predicted annual savings of $69,585.

TABLE 4.

Effects of implementation of acceptance criteria

Parameter	HSV-1/2	VZV	CMV	EV	Total
No. of NAAT assays requested/performed in 2013	365	190	164	90	809
No. of NAAT assays requested in 2014 (% change from 2013)	294 (−20)	166 (−13)	139 (−15)	83 (−8)	682 (−16)
No. of NAAT assays performed in 2014 (% change from 2013)	173 (−53)	112 (−41)	97 (−41)	56 (−38)	438 (−46)
No. of NAAT assays rejected (% of requested)	121 (41)	54 (33)	42 (30)	27 (33)	244 (36)
Realized savings ($)	12,422	5,047	4,335	2,200	24,004
Annualized savings ($)	33,117	13,454	11,556	5,864	63,992

DISCUSSION

Establishing laboratory acceptance criteria for HSV-1/2, VZV, CMV, and EV NAAT with CSF specimens may increase the positive predictive value of such tests (by increasing the pretest probability), improve patient care, and reduce health care costs. We have demonstrated that acceptance criteria of >10 WBCs/μl in the CSF of immunocompetent patients would have reduced viral NAAT assays by 51% in our hospital in 2013, which is similar to the 46% reduction we prospectively observed in a brief postimplementation period. These acceptance criteria would have prevented nine clinically insignificant or false-positive results during a 6-year period, including one HSV-1, three HSV-2, one VZV, and four CMV results.

An important question regarding these data is as follows: why are so many viral CSF NAAT assays ordered for patients without pleocytosis? There are several potential explanations. First, the costs of laboratory tests and the potential harm of false-positive results may not be prioritized as highly as the potential harm and medicolegal costs of failing to diagnose a treatable viral infection of the CNS. Second, it may be more convenient for clinicians to order NAAT assays prior to the availability of CSF cell count, protein level, and glucose level results. Third, order sets with comprehensive lists of potential etiologies may contribute to nonselective ordering practices. This hypothesis is supported by the fact that we determined that the inclusion of VZV, CMV, and EV tests in an order set correlated with an approximate 60% increase in testing. Fourth, there may be a misperception of the clinical utility and/or cost-effectiveness of viral NAAT, particularly for EV.

As no specific antiviral therapy for EV meningitis exists, the rationale for testing among adults is extrapolated from pediatric studies, in which EV-positive results were associated with reduced antibiotic use, length of stay, and admission costs (14, 23). A subsequent study suggested that EV testing was cost-effective for children only with an EV prevalence of greater than 5.9%, in contrast to the 1.7% described for our patient population (13). There are several limitations in extrapolating the pediatric data to adults (23). First, the prevalence of CNS EV infections among adults is less than that among children (20). Second, using positive EV NAAT results to exclude concurrent alternative infectious etiologies, as is done in the pediatric population, may not be appropriate for adults. Additional studies are required to determine whether the rationale used in pediatric care can be extrapolated to adult populations.

Implementation of the described acceptance criteria would have reduced NAAT results considered clinically insignificant by the treating providers. Of the three HSV-1/2-positive subjects whose testing would have been cancelled based on the algorithm described, one subject, who tested positive for HSV-2 in the CSF, presented with 1 month of altered mental status and recent genital herpes. A second subject had NAAT results positive for HSV-2 but was deemed to have experienced a cerebrovascular accident. The HSV-2 results for both subjects were clinically considered to be false-positive results at the time, after infectious disease consultation. A third subject presented with fever and aphasia and subsequently was treated for HSV-1 encephalitis after positive NAAT results were obtained. This sample would have been cancelled under all six acceptance criteria evaluated; however, clinical suspicion was high enough to warrant treatment (see discussion below). The one missed VZV-positive subject presented with an acute unilateral headache. Two days after lumbar puncture, a pathognomonic vesicular rash emerged in the V1 distribution, and then NAAT of the preexisting CSF specimen was requested.

Of the four CMV-positive specimens that would have been cancelled, results for two subjects were determined to be clinically insignificant after infectious disease consultation. One CMV-positive subject was discharged without regard for the positive result, and antiviral therapy was not initiated. The fourth CMV specimen that would have been excluded was from a previously healthy young female patient who presented with lower extremity weakness. She was discharged with a diagnosis of CMV transverse myelitis. She returned 10 months later with an exacerbation, despite having received adequate therapy for CMV, and ultimately was diagnosed with multiple sclerosis.

While the acceptance criteria we describe may effectively reduce costs and unnecessary testing, the importance of appropriate clinical judgment cannot be overstated. Among the 10,981 tests performed, the criteria described herein may have resulted in cancellation of one test from a case of potential HSV-1 encephalitis. This subject had other signs of HSV-1 infection, including a prodrome of fever, headache, and malaise followed by aphasia, apraxia, and memory impairment. Furthermore, magnetic resonance imaging of the brain demonstrated findings concerning for HSV-1 encephalitis, with the left insula and anterior temporal cortex demonstrating increased T2 signal, diffusion restriction, and parenchymal contrast enhancement. However, there was no evidence of hemorrhage or leptomeningeal enhancement. The CSF specimen exhibited a protein concentration of 41 mg/dl, a glucose concentration of 84 mg/dl (with a blood glucose concentration of 140 mg/dl), 5 nucleated cells/μl, with 85% lymphocytes, 12% monocytes, and 3% neutrophils, and 453 RBCs/μl. While this case only meets the criteria proposed by Bhaskaran et al. for a possible true-positive result (15), the severity of HSV-1 encephalitis warrants antiviral treatment even without a definitive laboratory-based diagnosis. It should be noted that delayed administration of acyclovir may worsen patient outcomes and thus mitigate or negate cost savings. Additionally, it is important to note that a negative NAAT result does not rule out HSV-1/2 encephalitis, as NAAT results can be negative early in the course of illness (24). These data illustrate the importance of a streamlined process for “physician override” of algorithmic testing approaches for patients with clinical signs and symptoms consistent with a specific diagnosis, to facilitate testing when it is clinically warranted.

To fully capture the cost savings of the described acceptance criteria and to reduce the erroneous cancellation of testing for specimens meeting the acceptance criteria, efficient and accurate implementation is essential. While the automatic assessment of CSF nucleated cell counts can be easily integrated by laboratory information systems, the assessment of immune status is more challenging for laboratory personnel and/or building into a laboratory information system algorithm. Currently in our hospital, specimens received from locations where all patients are immunosuppressed are automatically accepted. For the remaining specimens, the microbiology resident or fellow on call assesses each specimen for the acceptance criteria. This is done once daily 7 days per week, immediately prior to the batched sending out of NAAT specimens, and generally requires only 1 to 2 min per sample. For samples that do not meet the acceptance criteria, a comment appears in the EMR with the reason for cancellation and a laboratory contact telephone number for automatic override if the clinical suspicion remains strong. All cancelled specimens are held for 30 days for additional testing if needed. A more automated implementation approach would include enabling clinicians to communicate the patient's immune status at the time of order entry, either through a check box or with separate CSF NAAT order sets for immunocompetent and immunosuppressed patients.

Our limited prospective follow-up monitoring after implementation of the NAAT acceptance criteria was associated with a 46% reduction in NAAT assays. While this was predominantly due to the laboratory rejecting testing of specimens not meeting the set criteria, there was a 16% reduction in provider orders for these tests. This may be due to a combination of our educational efforts and increased awareness of ordering practices. Future observation is warranted, to determine whether this change in ordering frequency is sustainable, especially in the setting of an academic medical center.

The strengths of this study include the comparison of various acceptance criteria and the inclusion of four commonly ordered NAAT assays, namely, those for HSV-1/2, VZV, CMV, and EV. For HSV-1/2, we assessed more positive samples than the three prior studies evaluating this clinical issue combined (16, 17, 22). Previous acceptance criteria for HSV-1/2 based on CSF WBC counts alone or in combination with CSF protein concentrations have been reported (16, 17). Our data suggest that the CSF protein concentration provides little, if any, additive prognostic value when the CSF WBC count is known (data not shown). In our patient population, overemphasis on elevated CSF protein concentrations may contribute to unnecessary test ordering. We also demonstrate the importance of considering a patient's immune status in applying the acceptance criteria, as 20% of the positive NAAT tests (25/122 tests) were from immunocompromised subjects who lacked pleocytosis.

This study has several limitations. First, we report data from a single urban tertiary care center. While our patient population is diverse, our findings may not be able to be extrapolated to other medical centers or other patient populations, such as children. Second, institution-specific ordering practices may affect our low prevalence of positive results. Third, this is a retrospective study that may inherently introduce selection and misclassification biases, although the degree of such biases is likely minimal, given the data collection methods and high interrater correlation. Fourth, despite our large sample size, the number of positive tests is low overall. Finally, no CSF acceptance criteria maintained sufficient sensitivity in the absence of accounting for patient immune status. While acceptance criteria based on WBC counts can be automated in laboratories, assessment of immunocompetence is difficult to automate; therefore, immune status may best be indicated by the ordering provider on the test requisition or assessed by skilled laboratory personnel. New informatics and communication methods that engage ordering providers in algorithmic testing may increase the efficiency of applying NAAT acceptance criteria.

In conclusion, there are critical demands for health care cost containment and quality improvement (4–8). Establishing laboratory acceptance criteria for viral NAAT of CSF samples is one example of an approach to both reducing costs and improving patient care by reducing unnecessary testing and its associated consequences.