Decision Analysis: Point-of-care Chlamydia Testing vs. Laboratory-based Methods

Abstract

OBJECTIVE

To evaluate and compare the performance of several different methods available for detection of Chlamydia trachomatis (Ct) infection, and to explore possible testing and treatment strategies incorporating point-of-care testing versus laboratory-based tests.

DESIGN

Prospective trial and decision analysis.

SETTING

Large, urban, publicly funded sexually transmitted disease clinic.

PARTICIPANTS

1,384 female patients.

METHODS

Each subject was tested for Ct infection by direct fluorescent antibody (DFA, Sanofi/Kallestad, Chaska, MN), optical immunoassay (OIA, Thermo Electron, Point of Care and Rapid Diagnostics, Louisville CO), McCoy cell culture (in-house method), and polymerase chain reaction (microwell PCR, microwell assay, Roche, Branchburg NJ).

RESULTS

Performing a rapid in-clinic test on women who did not meet empiric treatment criteria would have increased the overall proportion of infected persons receiving same-day treatment from 48.6% to 79.1% using DFA or 78.4% using OIA.

CONCLUSIONS

Use of empiric treatment criteria and same-day point-of-care testing for patients not meeting the empiric treatment threshold appears to be an appropriate, useful, and cost-effective strategy for increasing same-day treatment of Ct infections in this population.

INTRODUCTION

Chlamydia trachomatis (Ct) infection is the most common sexually transmitted disease (STD) in the United States.¹ It is an important cause of urethritis in men; in women it can cause cervicitis, salpingitis and pelvic inflammatory disease (PID) and may lead to ectopic pregnancy and infertility.² In many patients, signs and symptoms are absent. Historical reports of clinical and behavioral variables are not always useful in predicting risk for Ct infection.^2,3 According to the Centers for Disease Control and Prevention (CDC), “Laboratory screening represents an important facet of efforts to control Chlamydia”.³ A variety of laboratory tests are now available for the diagnosis of Ct infection including isolation in cell culture, direct florescent antibody (DFA) assays, enzyme immunoassays (EIA), nucleic acid probe (NAP), and nucleic acid amplification tests (NAAT) such as polymerase chain reaction (PCR) and others.⁴

Relatively simple and rapid immunoassays have also been developed for Ct detection.⁴ Advantages include providing results while a patient is available for counseling and treatment and eliminating the need for transport of specimen to the reference laboratory. However, some reports indicate that simple and rapid immunoassays produce less than acceptable results.^5–7 DFA, although often thought of as a reference method, can also be configured for use as a point-of-care method in certain settings.

Given the relative dearth of comparative data in the literature, we felt that it would be important to evaluate the use of rapid laboratory methods for Ct detection in an STD clinic setting, in the hopes of obtaining data that could guide providers in similar settings as they strive to optimize the early detection and treatment of cervical Ct infection. In addition to a laboratory method comparison of a rapid optical immunoassay (OIA, Thermo Electron, Point of Care and Rapid Diagnostics, Louisville, CO) with DFA (Pathfinder, Sanofi/Kallestad, Chaska, MN), cell culture, and PCR (microwell assay, Roche, Branchburg, NJ), we also aimed to assess the use of risk indicators and presumptive treatment criteria and to provide a decision analysis of possible testing and treatment strategies.

Previous studies on STD clinic populations have shown that up to 20% of patients with positive chlamydia or gonorrhea (GC) tests fail to return for treatment within 30 days, and an additional 30% fail to return two weeks after test results.⁸ These people are likely to remain infectious and contribute substantially to the spread of these diseases in the community. They may also be at increased risk of inflammatory complications such as PID.⁹ In fact, as many as 30% of untreated or uncured Ct infections progress to PID, and each case of PID averages over $4,000 in future medical costs.¹⁰ Rapid testing holds some promise in addressing this problem by reducing the number of lost-to-follow-up and thus untreated persons with Ct.⁶

METHODS

After obtaining IRB approval, consecutive female clients being evaluated for STDs at the City of Milwaukee Health Department (MHD) Sexually Transmitted Disease Clinic were offered participation in the study. Informed consent was obtained from 1,384 (79.5%) of the 1,741 women who attended the clinic during the study period. Most of the remaining 357 women were ineligible for the study because they were presenting to the clinic for reasons other than an initial STD evaluation.

Demographic and behavioral data were collected on each study subject during the routine interview portion of the visit. During the physical exam portion of the visit, four cervical specimens were collected from each of the 1,384 study subjects. The first swab was for Neisseria gonorrhoeae culture (and for Gram stain if there was a cervical discharge present). Swabs 2 and 3 were tested for Ct using the DFA and OIA methods. The collection order of swabs 2 and 3 were alternated weekly. Swab 4 was collected last and placed in Micro Test™ M-4 medium (Remel, Lenexa, KS) to be used for both Ct cell culture and Ct PCR.

The OIA tests were done at the MHD STD Clinic. The majority of the OIA tests were batched rather than performed while the patient waited. Gonorrhea and chlamydia cultures were done per standard procedures at the nearby Milwaukee Health Department Laboratory (MHD lab) using modified Thayer-Martin plates for gonorrhea culture and McCoy cells and iodine stain for chlamydia. MHD lab staff removed an aliquot of the M-4 medium for Ct culture testing and then forwarded the remaining M-4 medium and the DFA slide by daily courier to the Wisconsin State Laboratory of Hygiene (WSLH) in Madison. DFA slides were stained and read daily Monday through Friday at WSLH by experienced technologists using 3 elementary bodies as a cutoff for positivity; PCR was done twice weekly (Tuesdays and Fridays) according to the manufacturer's package insert. Residual M-4 was stored at -70°C for referee testing at WSLH by ligase chain reaction [(LCR) Abbott Laboratories, Abbott Park, IL] at the conclusion of the study. All results (except the referee testing) were reported to the clinic. Patients qualified for treatment of chlamydia if they met the clinic's presumptive treatment criteria or if any of the 4 chlamydia tests were positive.

The performance characteristics of each laboratory test for Ct were calculated using an expanded gold standard. Specifically, any concordant result from all four methods was considered a true result. All discordant results in which the cell culture was positive were considered to be true positives. All specimens yielding discordant results in which the culture was negative were resolved by testing the frozen M-4 remnant using LCR; any of these remnants testing positive by LCR and at least one other method were considered true positives. The sensitivity, specificity and predictive values of each individual test were then calculated in comparison to the expanded gold standard. Using these performance characteristics, various potential diagnostic and treatment strategies were evaluated using decision analysis.

RESULTS

The demographic characteristics of the study population, detailed in table 1, very closely mirrored the clinic population during the study period (data not shown). As detailed in table 2, a total of 148 of the 1,384 women were classified as infected with Ct for a prevalence of 10.7%. Of the 148 true positives, the OIA method detected 95 (64.2%) and DFA testing identified 109 (73.6%). Sensitivity, specificity, and predictive values for all methods are shown in table 3.

Table 1.

Characteristics of study subjects.

	# of Subjects	% of Total		# of Subjects	% of Total
Age			Race
<15	12	0.9%	American Indian	7	0.5%
15–19	332	24.2%	African American / Black	1,022	74.5%
20–24	347	25.3%	Asian/Pacific	12	0.9%
25–29	238	17.3%	Hispanic	48	3.5%
30–34	185	13.5%	White	192	14.0%
≥ 35	258	18.8%	Other / Unknown	91	6.6%
History of prior STD			Regular condom use
No	560	40.8%	Yes	368	26.8%
Yes	812	59.2%	No	1,004	73.2%

Table 2.

Chlamydia assay results and interpretations.

Final Interpretation	Assay					Number of Specimens
	Culture	OIA	DFA	PCR	Referee LCR
Positive
	+	+	+	+	*ND	73
	+	+	−	+	+	3
	+	−	+	+	+	3
	+	−	−	+	+	2
	+	−	−	−	−	2
	−	−	−	+	+	28
	−	−	+	+	+	16
	−	+	+	+	+	14
	−	+	−	+	+	2
	−	+	−	−	+	2
	−	+	+	−	+	1
	−	−	+	−	+	2
	Total “true” positives					148
Negative
	−	+	−	−	−	11
	−	−	−	+	−	2
	−	−	+	−	−	1
	−	−	−	−	*ND	1,222
	Total “true” negatives					1,236
Total						1,384

*ND = Not Done

Table 3.

Sensitivity, specificity, and predictive values (using expanded gold standard).

Test	Sensitivity [95% C.I.]	Specificity [95% C.I.]	Predictive Value of Positive Test (in this population) [95% C.I.]	Predictive Value of Negative Test (in this population) [95% C.I.]
Cell Culture	56.1%	100%*	100%	95.0%
	[47.7 – 64.1]	[99.6 – 100]	[94.5 – 100]	[93.6 – 96.1]
OIA	64.2%	99.1%	89.6%	95.9%
	[55.9 – 71.8]	[98.4 – 99.5]	[81.8 – 94.5]	[94.6 – 96.8]
DFA	73.6%	99.9%	99.1%	96.9%
	[65.7 – 80.4]	[99.5 – 100]	[94.3 – 100]	[95.8 – 97.8]
PCR	95.3%	99.8%	98.6%	99.4%
	[90.1 – 97.9]	[99.3 – 100]	[94.5 – 99.8]	[98.8 – 99.8]

*Cell-culture positivity was one defining criterion for true positive.

The relative risk of infection in women meeting the clinic's standard empiric treatment criteria as shown in table 4, was 3.7 (95% C.I. = 2.4 - 5.7) as compared to women not meeting the criteria.

Table 4.

Presumptive treatment criteria for the MHD STD Clinic during the study period.

1.	Having known positive untreated chlamydia, gonorrhea or syphilis (i.e., a previous positive test from any provider in the past 60 days with no history of treatment).
2.	Having mucopurulent cervicitis (this was a criterion during the study; subsequent revisions of CDC recommendations no longer recommend empiric treatment for this clinical entity).
3.	Reporting a sex partner with chlamydia, gonorrhea, syphilis, epididymitis or non-gonococcal urethritis.
4.	Having symptoms of PID.

Decision analysis (table 5) was performed to determine if use of DFA or OIA as a point-of-care test, with or without use of empiric treatment criteria, would have increased the proportion of infections treated.

Table 5.

Possible Rapid Testing / Treatment Strategies (prevalence = 10.7%; sensitivities and specificities as noted in table 3).

	Clients (N = 1384)		Infections (n = 148)
Strategy	% (#) Tested	% (#) Treated same day**	% (#) Treated (theoretic maximum)***	% (#) Treated on same day**	% (#) Likely to be left untreated****
Presumptive Treatment with no testing	0% (0)	22.5% (312)	48.6% (72)	48.6% (72)	51.4% (76)
Presumptive Treatment plus Universal Culture (standard in clinic before study)	100% (1384)	22.5% (312)	75.7% (112)	48.6% (72)	29.7% (44)
Presumptive Treatment with selective PCR testing*	77.5% (1072)	22.5% (312)	96.6% (143)	48.6% (72)	12.8% (19)
Universal PCR; no presumptive treatment	100% (1384)	0% (0)	95.3% (141)	0% (0)	23.6% (35)
Universal point-of-care OIA testing; treat if OIA positive	100% (1384)	7.7% (106)	64.2% (95)	64.2% (95)	35.8% (53)
Universal point-of-care DFA testing; treat if DFA positive	100% (1384)	7.9% (110)	73.6% (109)	73.6% (109)	26.4% (39)
Presumptive Treatment plus selective point-of-care OIA testing*	77.5% (1072)	25.7% (356)	78.4% (116)	78.4% (116)	21.6% (32)
Presumptive Treatment plus selective point-of-care DFA testing*	77.5% (1072)	25.8% (357)	79.1% (117)	79.1% (117)	20.9% (31)

*for those not meeting presumptive treatment criteria.

**assuming 100% compliance with waiting for positive results and accepting same-day treatment.

***assuming 0% failure to return rate, and 100% compliance waiting for positive results and accepting same-day treatment.

****assuming 20% failure to return rate if not treated same-day in clinic.

Decision analysis revealed that performing a rapid in-clinic test on all study subjects without using other empiric treatment criteria would have resulted in 64.2% or 73.6% of infected persons receiving treatment using the OIA method and DFA method, respectively. Using empiric treatment criteria plus performing a rapid in-clinic test on those not meeting the treatment criteria would have increased the overall proportion of infected persons receiving same-day treatment from 48.6% with empiric treatment only, to 79.1% using DFA testing or 78.4% using the OIA method, while only requiring testing on 77.5% of the entire population (table 5).

Of the 1,384 study subjects, 1,372 (99.1%) had gonorrhea results available by chart abstraction. In this group of 1,372, the total of those positive for gonorrhea was 119 (8.7%) and 146 (10.6%) were positive for chlamydia. Of those with chlamydia, 25.3% (37 of 146) had gonorrhea, and of those with gonorrhea, 31.1% (37 of 119) had concomitant chlamydia. The relative risk of having chlamydia in a patient with gonorrhea was 3.6 (95% CI: 2.6 to 4.9) as compared to women without gonorrhea. The relative risk for gonorrhea in women who had chlamydia was 3.8 (95% CI: 2.7 to 5.4) as compared to women who did not have chlamydia.

DISCUSSION

Many studies have evaluated laboratory approaches to chlamydia screening, including amplified molecular methods, DFA, OIA, non-amplified molecular methods, and other rapid solid phase antigen detection methods.^6,11–15 However, we are aware of only one study that has compared a rapid testing modality such as OIA directly with DFA, culture, and NAAT on a head-to-head basis,¹² and we are not aware of any study that performed decision analysis on such information.

Miller¹⁶ postulates a model for testing and treatment decision-making based on the probability of infection. In this model, persons whose probability of infection is above that of the threshold for performing a lab test but below that for providing empiric treatment should receive a lab test prior to treatment, but persons whose probability of infection exceeds the threshold for providing empiric treatment should be treated without testing. Persons whose probability of infection is below the threshold probability for testing and below the threshold probability for treatment should receive no testing nor treatment. To our knowledge there have not been any studies that have evaluated this theoretical model in an actual practice-based setting.

Our results agree with the general consensus that the sensitivities of rapid methods are too low to be recommended for universal use in screening programs.¹⁷ However, rapid testing appears to be potentially useful when incorporated into a scheme such as that described by Miller, particularly in settings where the potential for losing patients to follow-up is increased. Specifically, if reasonable criteria can be developed to establish an appropriate threshold for empiric treatment, then testing and the costs associated with testing can be avoided for clients whose risk of infection exceeds that established clinical treatment threshold. This also minimizes the problem of failing to treat infections with false negative test results. In our study, the presumptive treatment criteria defined a group whose relative risk for Ct was 3.7 times the tested-but-not-presumptively-treated group, indicating that the criteria effectively identified persons whose risk of infection was elevated enough to warrant treatment.

In STD clinics, it may be reasonable to assume that all patients presenting for services have a probability of infection that exceeds the threshold for testing (i.e., everyone should receive either testing and/or treatment). In such a case, any client who fails to meet the criteria for empiric treatment should be tested. Furthermore, given concerns about losing clients to follow-up, it would be advantageous to test such persons on a while-you-wait basis. Our data show that performing while-you-wait DFA or OIA testing on such patients would have substantially increased the number of infections detected and treated on the same day. This becomes particularly significant in light of the previously mentioned finding that as many as 20% of patients testing positive for chlamydia or gonorrhea do not return for treatment within 1 month.⁸

In settings other than STD clinics, such as family planning clinics or private family physicians' offices, there may well be a large cohort of patients for whom the probability of infection is low enough to be below the threshold for empiric treatment but high enough to warrant testing. This is supported by estimates of untreated chlamydial prevalence in the general (“non-high risk”) population of as much as 3% (adults aged 18 to 35) to 6% (males 18 to 19 years of age).^18,19 It is also supported by the high rate of chlamydia infections which are not associated with symptoms; up to 85% of uncomplicated chlamydia infections in women and up to 40% in men are asymptomatic, and as many as 60% of cases of chlamydial PID are without symptoms.^10,16 We suspect that many physicians do not have an appropriately high index of suspicion for testing for chlamydia.

Setting an appropriate threshold for empiric treatment is important for many reasons. Treating persons without laboratory confirmation of infection could result in a loss of potential prevention opportunities. For example, effective partner tracing and contact interventions are less likely in the absence of laboratory-confirmed disease. Overuse of antibiotics is also an important concern. These concerns must be weighed against the likelihood that without same-day empiric treatment, (a) individuals will continue to transmit disease and risk developing complications until the laboratory result is returned and they return to the point-of-care for treatment, and (b) depending on the clinical setting, a relatively smaller or larger proportion of infected individuals may not return for treatment for several weeks or longer, and some will fail to return at all.

Any approach to STD diagnosis and treatment comes with tradeoffs, including the approach described by Miller that we have subjected to decision analysis here. If one sets the thresholds too low, over-testing, over-diagnosis and over-treatment result. On the other hand, more infections are treated and more secondary infections and complications are prevented. Conversely, as the thresholds are set higher, costs of diagnosis and treatment go down. Fewer uninfected individuals are treated, but more infections are missed and thus more secondary infections and complications will result.²⁰ Similarly (to a greater or lesser extent depending on the presence of effective presumptive treatment strategies), use of diagnostic tests with relatively lower sensitivities can also result in more untreated disease and increased secondary spread of infection.

As shown in table 5, our use of decision analysis demonstrates these tradeoffs clearly. While these findings are probably generalizable to similar settings, differing prevalence rates in other clinical practices might lead to the use of different criteria to determine appropriate testing and empiric treatment thresholds. In any case, we agree with Steen²⁰ that “socio-demographic and behavioral data (risk assessment)” make sense in helping to “identify patients with a higher prevalence (prior probability) of infection.” What we have done is to show via decision analysis the pros and cons of a variety of testing and treatment strategies, using risk assessment findings in combination with the individuals' signs and symptoms to determine whether or not a patient's risk of GC or Ct infection is above or below a given testing or treatment threshold.

In our population, 25.3% of our chlamydia-positive women also had gonorrhea, and 31.1% of our gonorrhea-positive women had concomitant chlamydia. The relative risk for one of these infections given the presence of the other was roughly the same in either case and was substantial (RR = 3.6 - 3.8.). These data support an approach of empiric treatment for chlamydia in patients who have gonorrhea as well as empiric treatment for gonorrhea in those patients who have chlamydia. Such an approach would be particularly important in situations where the prevalence of N. gonorrhoeae is relatively high. This is in contradistinction to current CDC recommendations for empiric treatment against chlamydia in persons diagnosed with gonorrhea, but not vice versa.²¹

This study has some important limitations. The study included only women, so the results may or may not be generalizable to men. The chlamydia culture method used by the MHD lab utilized iodine stain, which is both less expensive and less sensitive than fluorescent antibody techniques. Because NAAT testing is much more sensitive than the OIA, DFA, or culture methods, we chose to always take the NAAT swab last; it is theoretically possible that more complete randomization of all swabs collected may have better ensured an unbiased comparison of performance methods.

In this study, the OIA and DFA tests were not actually done while patients waited for results. Instead, the OIA tests were run in small batches and the DFA slides were sent to the state lab to be read by experienced technologists. Use of DFA as a point-of-care test is unproven and may not be feasible in many settings. In addition, some women might not actually wait in clinic for a rapid test result, and return rates might vary by age or other factors. Therefore, the performance of these tests in this study may not reflect their performance under actual “while-you-wait” conditions.

Use of rapid tests with relatively low sensitivities for screening creates the risk of patients with false-negative results continuing to spread disease. Our data would indicate that this risk can be mitigated by use of presumptive treatment criteria. Performing NAAT testing on all DFA or OIA negatives would also address this issue, but at the cost of dual testing on a substantial proportion of patients.

Our data indicate that the study population well-represented the female population of the clinic. Since we believe this clinic and its population are fairly typical of urban STD clinics, our findings can probably be generalized to other STD clinics. However, the STD clinic setting is a specialized one, so it may not be possible to generalize all of our findings to other settings such as Family Planning Clinics or private physicians' offices.

CONCLUSION

Use of specific criteria to define a threshold for empiric treatment without testing, combined with same-day point-of-care testing for patients not meeting the empiric treatment threshold, can be an appropriate and useful strategy for increasing same-day treatment of chlamydia infections in women presenting to STD clinics. Either DFA testing (assuming appropriate training and logistics for use as a point-of-care rather than a reference test) or the Thermo Electron OIA test appears to be potentially applicable in this regard. Such an approach would likely decrease the transmission of chlamydia by untreated infectious individuals in the community. Similar strategies may be applicable for men presenting to STD clinics, as well as to patients of either gender in other settings.