Comparison of Algorithms to Triage Patients to Express Care in a Sexually Transmitted Disease Clinic

Abstract

Background

The ideal approach to triaging STD clinic patients between testing-only express visits and standard visits with clinician evaluation is uncertain.

Methods

In this cross-sectional study, we used classification and regression tree analysis to develop and validate the optimal algorithm for predicting which patients need a standard visit with clinician assessment (i.e., to maximize correct triage). Using electronic medical record data, we defined patients as needing a standard visit if they reported STD symptoms, received any empiric treatment, or were diagnosed with an infection or syndrome at the same visit. We considered 11 potential predictors for requiring medical evaluation collected via computer-assisted self-interview (CASI) when constructing the optimized algorithm. We compared test characteristics of the optimized algorithm, the Public Health–Seattle and King County STD Clinic’s current 13-component algorithm, and a simple two-component algorithm including only presence of symptoms and contact to STD.

Results

During 10/2010–06/2015, 18,653 unique patients completed a CASI. In the validation samples, the optimized, current, and simple algorithms appropriately triaged 90%, 85%, and 89% of patients, respectively. The optimized algorithm had lower sensitivity for identifying patients needing standard visits (men: 94%; women: 93%) than the current algorithm (men: 95%; women: 98%), as did the simple algorithm (men: 91%; women: 93%). The optimized, current, and simple algorithms triaged 31%, 23%, and 33% of patients to express visits, respectively.

Conclusions

The overall performance of the statistically optimized algorithm did not differ meaningfully from a simple two-component algorithm. In contrast, the current algorithm had the highest sensitivity but lowest overall performance.

Introduction

Many sexually transmitted disease (STD) clinics have implemented express care models, which include a testing-only visit option without a clinician evaluation, to meet the high demand for services in the context of low resources¹ and increasing rates of many STDs.² Express care may increase clinic efficiency and reduce long-term costs, allowing limited clinician time and resources to be focused on patients in need of same-day treatment and facilitating rapid access to STD screening.³ However, there is also the potential to miss health conditions of public health importance that could have been identified during a clinician evaluation.⁴

STD clinics have implemented a variety of express care models, each with its own method of patient history collection, triage process, and distinct set of criteria for determining patients’ visit types (Table 1).^5–13 Triage criteria may be disease-focused (e.g., patient has genital symptoms) or health service-focused (e.g., female patient who wants to discuss contraception with a clinician), depending on the patient population and local public health priorities and resources. Although not reported in the literature, we have observed that some STD clinics use a simple triage algorithm including only whether the patient has any symptoms or had known contact to a sex partner with an STD or HIV (hereafter referred to as “contact to STD”). To our knowledge, only one study has attempted to formally construct triage algorithms that reliably predict who will receive specific STD diagnoses that could be missed in testing-only visits.¹⁴ Using medical record data from three STD clinics, Xu and colleagues found that age and/or specific STD-related symptoms were associated with each of four types of diagnoses that could be missed in express visits: genital herpes or warts, trichomoniasis, symptomatic vulvovaginal candidiasis or bacterial vaginosis, and nongonococcal nonchlamydial urethritis. However, age and/or specific symptoms had low discriminatory ability for predicting those diagnoses, even when used in combination. Other studies have evaluated clinics’ respective express care models in different ways, but no published studies to date have compared the relative discriminatory ability of multiple triage algorithms for predicting who needs to see a clinician within the same population.

TABLE 1.

Triage Criteria Used to Identify Patients Needing a Clinician Evaluation in Previously Published STD Clinic Express Care Models

STD Clinic Location	Triage Criteria Indicating Need for a Standard Visit (any of the following)	Reference
Amsterdam, The Netherlands	(1) STD symptoms, (2) known contact to a sex partner with an STD, (3) STD in the past 6 months, (4) has been paid for sex, (5) MSM, (6) female reporting anal sex in the past 6 months	Heijman TL et al. 2007
Chicago, United States	9 questions regarding symptoms and behavioral risk (not specified)	Wong W et al. 2006
Denver, United States	(1) STD symptoms, (2) MSM, (3) known contact to a sex partner with an STD in the past 4 months, (4) injection drug use in the past 4 months, (5) transactional sex in the past 4 months, (6) other high-risk behavior such as multiple sex partners	Shamos SJ et al. 2008
Edmonton, Canada	(1) ≤16 years old, (2) symptoms, (3) known contact to a sex partner with an STD, (4) receptive anal sex since last testing visit, (5) sexual assault in the past 2 weeks, (6) transactional sex in the past 6 months	Gatrix J et al. 2015
Phoenix, United States	(1) STD symptoms, (2) MSM, (3) known contact to a sex partner with an STD or HIV, (4) injection drug use, (5) pregnant, (6) referred by a community provider or public health investigator	Rukh S et al. 2014
Melbourne, Australia	[Hypothetical] (1) Known contact to a sex partner with an STD, (2) MSM, (3) ever used injection drugs, (4) symptoms, (5) <16 years old, (6) sex worker; (7) sex partner from a country with >1% HIV prevalence	Yeung A et al. 2010
New York City, United States	(1) Symptoms, (2) known contact to a sex partner with an STD, (3) requests examination	Borrelli JM et al. 2008
Seattle, United States	(1) Symptoms, (2) positive STD result from another provider and needs treatment, (3) known contact to a sex partner with an STD or HIV, (4) sex partner with symptoms, (5) has HIV and has not had an HIV care visit in the past 9 months, (6) has hepatitis C and wants a referral for care, (7) has genital herpes and wants preventive or suppressive medication, (8) heterosexually active female who wants to discuss contraception, (9) female requesting emergency contraception, (10) female ≥21 years old who has not had a Pap test in the past year, (11) ≤18 years old, (12) ever had a transgender sex partner, (13) is transgender	Chambers LC et al. 2017
Sydney, Australia	(1) STD symptoms, (2) from a priority population (MSM, Aboriginal people, sex workers, intravenous drug users, HIV-positive people, and youth <25 years old)	Knight V et al. 2013

Abbreviations: HIV, human immunodeficiency virus; MSM, man who has sex with men; STD, sexually transmitted disease.

We recently reported an evaluation of the express care triage algorithm currently used in the Public Health – Seattle & King County (PHSKC) STD Clinic in Seattle, Washington.¹² This algorithm accurately identifies patients with an infection or syndrome that can be diagnosed and treated at that visit, but it triages <25% of patients to express care which limits efficiency. The ideal algorithm would triage a substantial percentage of patients to express care without inappropriately triaging any patients to express care.

The goal of the present study was to construct and evaluate alternative triage algorithms to determine which approach most reliably identifies clinically relevant cases in our clinic while maximizing triage to express care. To achieve this goal, we (1) constructed and validated an optimized algorithm that maximizes correct triage of patients between express visits and standard visits with a clinician evaluation, and (2) compared the performance of the optimized algorithm, the PHSKC STD Clinic’s current algorithm, and a “simple” algorithm that considers only presence of symptoms and contact to STD.

Methods

Study Design and Sample

The PHSKC STD Clinic implemented systematic computer-assisted self-interview (CASI) assessments for collecting patient history and developed a computerized algorithm to automate triage in 2010.¹⁵ However, due to the staffing model in place, almost all patients continued to receive a clinician evaluation irrespective of triage status.¹² Therefore, the clinic database contains both CASI responses and outcomes of a clinician evaluation for each patient over a 5-year period.

This cross-sectional study included patients attending the PHSKC STD Clinic who completed a CASI at a new problem visit between October 1, 2010 and June 30, 2015. The 36% of new problem visits without a CASI were excluded.¹² Our analysis excluded transgender persons because their patient history was collected in-person rather than through CASI during the analysis period. For individuals with multiple new problem visits with a CASI during this period, we randomly selected one visit for inclusion in the analysis to maintain independence of observations.

Outcome Definition

The outcome for this study, “need for a standard visit,” was defined using electronic medical record data. The outcome definition was identical to that used in our prior study reporting performance of the current triage algorithm.¹² We coded each visit to indicate whether the patient needed a standard visit or could have received an express visit based on the clinician assessment. Our definition of needing a standard visit was disease-focused and intended to identify patients with an infection of public health importance that could be diagnosed and treated at that visit. We classified patients as needing a standard visit if they reported key symptoms (anogenital symptoms, abdominal pain [for women], body rash, and symptoms concerning to the patient for acute HIV infection) to the CASI or clinician, received any empiric treatment, or were diagnosed with a key infection or syndrome at the same visit. To ascertain symptoms concerning to the patient for acute HIV, our CASI asked: Did you come to the clinic to test for HIV because of symptoms you think may be a sign that you were recently infected with HIV? Key infections or syndromes were possible to diagnose with clinical examination and point-of-care testing alone and of public health importance due to the potential for ongoing transmission or health consequences of delayed treatment. These included primary or secondary syphilis; urethral, cervical, or rectal Neisseria gonorrhoeae detected on Gram stain; nongonococcal urethritis; epididymitis; proctitis; mucopurulent cervicitis; pelvic inflammatory disease; bacterial vaginosis; vaginal candidiasis; trichomoniasis; urinary tract infection; genital ulcer of unknown etiology; skin and soft tissue infection (i.e. cellulitis); and herpes simplex virus. Patients who did not meet any of these criteria were classified as eligible for express care. Importantly, HIV diagnosed by point-of-care test was not a criterion for a standard visit because a patient with a positive HIV test during an express visit can be routed to see a clinician at that time. Patients with symptomatic acute HIV were defined as needing standard visits due to their symptoms.

Potential Predictors

To develop the optimized algorithm, we selected 11 potential predictors of need for a standard visit. All are self-reported measures collected by CASI that have been highly associated with the key infections or syndromes that we determined would require a standard visit. These included age (years),^2,14,16,17 key symptoms,^5,14,18 sore throat, positive STD result from another provider and needs treatment, contact to STD, sex partner with symptoms,¹⁸ same-sex partner in the past year,^2,17,18 number of sex partners in the past two months,^17,18 popper (for men who have sex with men [MSM] only)¹⁹ or methamphetamine use in the past year,^20,21 crack cocaine use in the past year, and ever had transactional sex.²²

Other Measures

After optimizing the triage algorithm, we compared its performance to the PHSKC STD Clinic’s current algorithm and a simple algorithm. The current algorithm triages patients to a standard visit for 13 disease- and health service-focused reasons (Table 1). The simple algorithm triages patients to a standard visit for two disease-focused reasons: report of any symptoms (including sore throat) or contact to STD.

We classified patients who were mistriaged by an algorithm into two categories. We defined patients who needed a standard visit but were triaged to express visits by the algorithm as “underserved” and patients who did not need a standard visit based on the study definition but were triaged to standard visits by the algorithm as “overserved.” This terminology reflects the disease-focused outcome for this analysis.

Statistical Analysis

Construction and Validation of the Optimized Algorithm

We used independent development and validation samples to construct and validate the optimized algorithm. Separately for men and women, the new problem visits was randomly divided in half, resulting in sex-specific development and validation samples (Supplement). We assumed that all patients with key symptoms should be evaluated by a clinician. We did not make this assumption for contact to STD because, in practice, we have noticed variability in what patients report as contact to STD, some of which do not require evaluation (e.g., contact from >1 year ago, contact for which the patient already received treatment, contact to an STD for which treatment is not indicated). Therefore, we created sex-specific development and validation “sub-samples” that excluded individuals with key symptoms, which were utilized to develop and validate the initial sex-specific optimized algorithms.

We subjected each development sub-sample to classification and regression tree (CART) analysis, considering the 11 potential predictors. CART analysis is a non-parametric, empiric statistical method for developing prediction models.²³ It yields a sequence of nested risk stratification trees (i.e., decision trees) composed of progressive binary splits that divide patients into increasingly homogenous groups in terms of their outcome value. At each split, CART considers each predictor and utilizes that which best separates patients with and without the outcome. For continuous predictors, CART considers the cut-off value that best predicts the outcome. Predictors may be utilized more than once within the tree or not at all. We utilized CART analysis because it does not require assumptions about the underlying distributions of the predictors and is well-suited to considering complex interactions between predictors that are difficult to model with logistic regression.

We considered the optimal tree to be that with the lowest misclassification error rate (mistriaged visits / total visits) when the trees were applied to the validation sub-sample. Subsequently, we programmed the final optimized algorithm into Stata 13 (StataCorp, College Station, Texas), including the decision rules contained in each of the sex-specific optimal trees and the key symptoms criterion. This final optimized algorithm reflects what would be used in clinic operations.

Comparison of Triage Scenarios

We compared three triage scenarios in the full validation samples for men and women, each considering a separate triage algorithm: the final optimized algorithm, the PHSKC STD Clinic’s current algorithm, and the simple algorithm. For each scenario, we estimated the sensitivity, specificity, and area under the receiver operating curve (AUC) for appropriately triaging patients between express visits and standard visits. The gold standard was need for a standard visit based on the clinician evaluation. We also calculated the percent express visits to approximate the efficiency of each algorithm. Finally, we characterized the patients mistriaged by the optimized and simple algorithms. The characteristics of patients mistriaged by the current algorithm were published previously.¹²

This study was approved by the University of Washington Human Subjects Division. Analyses were conducted in Salford Predictive Modeler 7 (Salford Systems, San Diego, California) and Stata 13.

Results

Between October 1, 2010 and June 30, 2015, 18,653 patients completed a CASI at 32,102 new problem visits. After randomly selecting one visit for patients with multiple visits (n=13,449 visits excluded), there were 18,653 unique patients with a CASI, including 13,968 men (75%) and 4,685 women (25%) (Table 2). Just under half of men (45%) reported sex with men in the past year. The majority of patients were seeking care for key symptoms (49% of men, 60% of women).

TABLE 2.

Characteristics of Patients Attending the Public Health – Seattle and King County STD Clinic for New Problem Visits with a Completed CASI, Overall and by Sex; Seattle, Washington, October 2010 through June 2015^*

Characteristic	Total (N = 18,653), n (%)	Men (n = 13,968), n (%)	Women (n = 4,685), n (%)
Age, y†
≤18	269 (1)	151 (1)	118 (3)
19–24	3,336 (18)	2,118 (15)	1,218 (26)
25–29	4,179 (22)	3,005 (22)	1,174 (25)
≥30	10,866 (58)	8,691 (62)	2,175 (46)
Race
White	11,366 (61)	8,898 (64)	2,468 (53)
Black	3,507 (19)	2,423 (17)	1,084 (23)
Native American/Alaskan Native	215 (1)	130 (1)	85 (2)
Asian/Pacific Islander	1,494 (8)	1,035 (7)	459 (10)
Multiple races	561 (3)	308 (2)	253 (5)
Other or unknown	1,510 (8)	1,174 (8)	336 (7)
Hispanic ethnicity†	1,418 (8)	1,100 (8)	318 (7)
Sex/Sexual Orientation
Men who have sex with men	6,319 (34)	6,319 (45)	–
Men who have sex with women only	7,649 (41)	7,649 (55)	–
Women	4,685 (25)	–	4,685 (100)
Reason for visit‡
Positive STD result	869 (5)	692 (5)	177 (4)
Key symptoms§	9,618 (52)	6,814 (49)	2,804 (60)
Contact to STD/HIV	1,917 (10)	1,669 (12)	248 (5)
Other	6,249 (34)	4,793 (34)	1,456 (31)

Abbreviations: CASI, computer-assisted self-interview; HIV, human immunodeficiency virus; STD, sexually transmitted disease.

^*Transgender individuals were excluded because their patient history was collected in-person rather than with CASI during the analysis time-period. For patients with multiple visits, we randomly selected one visit for inclusion in the analysis.

^†n = 3 missing age, n = 1,008 missing Hispanic ethnicity.

^‡Mutually exclusive and hierarchical categories.

^§Key symptoms include anogenital symptoms, abdominal pain (for women), body rash, or symptoms concerning to the patient for acute HIV infection.

For men, the full development and validation samples each contained 6,984 patients; the development and validation sub-samples, which excluded men with key symptoms, contained 3,443 and 3,344 patients, respectively. For women, the full development and validation samples contained 2,343 and 2,342 patients, respectively; the development and validation sub-samples contained 873 and 857 patients, respectively. The optimal risk stratification trees for predicting need for a standard visit among men and women without key symptoms are presented in Figure 1. Among men, the optimal tree included three predictors: contact to STD, positive STD result from another provider, and sex partner with symptoms. Among women, the optimal tree included five predictors: contact to STD, positive STD result from another provider, sex partner with symptoms, age, and number of sex partners in past two months. The optimal tree for women included interactions among some predictors, so not all women meeting an individual criterion would be triaged to standard visits. The criteria in the final optimized algorithm, current PHSKC STD Clinic algorithm, and simple algorithm are summarized in Table 3.

FIGURE 1.

The Optimal Risk Stratification Tree for Predicting Need for a Standard Visit Among Men Without Key Symptoms (A) and Women Without Key Symptoms (B), Developed and Validated Using Classification and Regression Tree Analysis*^†

Abbreviations: EV, express visit; SV, standard visit.

*Key symptoms: anogenital symptoms, abdominal pain (for women), body rash, or symptoms concerning to the patient for acute HIV infection.

^†n (%) reflect the distribution of men and women in the respective validation sub-samples (excluding patients with key symptoms).

TABLE 3.

Triage Criteria for Identifying Patients Needing a Clinician Evaluation in the Optimized Algorithm, the Public Health–Seattle and King County STD Clinic Current Algorithm as of October 1, 2015, and a Simple Algorithm

Triage Criteria Indicating Need for a Standard Visit	Optimized Algorithm – Men	Optimized Algorithm – Women*	Current Algorithm	Simple Algorithm
Key symptoms†	✓	✓
Any symptoms			✓	✓
Contact to STD/HIV	✓	✓	✓	✓
Positive STD result from another provider and needs treatment	✓	✓	✓
Sex partner with symptoms	✓	✓	✓
≥4 sex partners in the past 2 months		✓
<40 years old		✓
≤18 years old			✓
Has HIV and has not had an HIV care visit in the past 9 months			✓
Has hepatitis C and wants a referral for care			✓
Has genital herpes and wants preventive or suppressive medication			✓
Heterosexually active female who wants to discuss contraception			✓
Female requesting emergency contraception			✓
Female ≥21 years old who has not had a Pap test in the past year			✓
Ever had a transgender sex partner			✓
Is transgender			✓

Abbreviations: HIV, human immunodeficiency virus; STD, sexually transmitted disease.

^*Algorithm includes interactions among some of these criteria, so not all patients meeting an individual criterion would be triaged to standard visits.

^†Key symptoms include anogenital symptoms, abdominal pain (for women), body rash, or symptoms concerning to the patient for acute HIV infection.

When we considered the three triage scenarios in the full validation samples for men and women, the optimized, current, and simple algorithms appropriately triaged 8,418 (90%), 7,943 (85%), and 8,273 (89%) patients, respectively (Table 4). The optimized and simple algorithms had similar overall performance (AUC) for men (optimized: 0.88, simple: 0.87) and women (optimized: 0.90, simple: 0.89) (Table 5). The AUC for the current algorithm was lower (men: 0.83, women: 0.65) due to lower specificity. However, the current algorithm had the highest sensitivity for men (95%) and women (98%). The optimized and simple algorithms triaged a similar percentage of patients to express visits (31% and 33%, respectively), indicating similar efficiency. The current algorithm triaged a lower percentage of patients to express visits (23%), primarily due to the health service-focused criteria for needing a standard visit.

Table 4.

Performance of Triage Algorithms for Identifying Patients Needing a Clinician Evaluation by Sex; Seattle, Washington, October 2010 through June 2015

	Needed a Standard Visit	Could have had an Express Visit
Optimized Algorithm
Men (n = 6,984), n (%)
Triaged to standard visit	4,355 (62)	397 (6)
Triaged to express visit	305 (4)	1,927 (28)
Women (n = 2,342), n (%)
Triaged to standard visit	1,554 (66)	92 (4)
Triaged to express visit	114 (5)	582 (25)
Current Algorithm*
Men (n = 6,984), n (%)
Triaged to standard visit	4,420 (63)	653 (9)
Triaged to express visit	240 (3)	1,671 (24)
Women (n = 2,342), n (%)
Triaged to standard visit	1,630 (70)	452 (19)
Triaged to express visit	38 (2)	222 (9)
Simple Algorithm
Men (n = 6,984), n (%)
Triaged to standard visit	4,230 (61)	404 (6)
Triaged to express visit	430 (6)	1,920 (27)
Women (n = 2,342), n (%)
Triaged to standard visit	1,550 (66)	101 (4)
Triaged to express visit	118 (5)	573 (24)

^*Public Health – Seattle and King County STD Clinic current algorithm as of October 1, 2015.

Table 5.

Test Characteristics of Triage Algorithms for Identifying Patients Needing a Clinician Evaluation by Sex; Seattle, Washington, October 2010 through June 2015

	Optimized Algorithm	Current Algorithm*	Simple Algorithm
Men (n = 6,984)
Sensitivity, %	93.5	94.9	90.8
Specificity, %	82.9	71.9	82.6
AUC (95% CI)	0.88 (0.87–0.89)	0.83 (0.82–0.84)	0.87 (0.86–0.88)
Express visits, n (%)	2,232 (32)	1,911 (27)	2,350 (34)
Women (n = 2,342)
Sensitivity, %	93.2	97.7	92.9
Specificity, %	86.4	32.9	85.0
AUC (95% CI)	0.90 (0.88–0.91)	0.65 (0.64–0.67)	0.89 (0.87–0.90)
Express visits, n (%)	696 (30)	260 (11)	691 (30)
Men and Women (n = 9,326)
Express visits, n (%)	2,928 (31)	2,171 (23)	3,041 (33)

Abbreviations: AUC, area under the receiver operating curve; CI, confidence interval.

^*Public Health – Seattle and King County STD Clinic current algorithm as of October 1, 2015.

The primary reasons that men who would have been underserved by the optimized and simple algorithms needed a standard visit were report of key symptoms to the clinician that were not disclosed to the CASI (47% and 34% of underserved men, respectively) and receipt of empiric therapy for something other than contact to STD despite being asymptomatic (27% and 26%, respectively). The primary reason that women who would have been underserved needed a standard visit was diagnosis with a key infection or syndrome despite being asymptomatic (45% for both algorithms). Compared to the current algorithm, use of the simple algorithm during this 5-year period would have delayed diagnosis and treatment of 3 cases of early syphilis and 55 cases of gonorrhea. For all 3 early syphilis cases and 49 gonorrhea cases (89%), the patient reported having a positive STD result from another provider.

The primary reason that men and women would have been overserved by the optimized and simple algorithms was report of contact to STD to the CASI when they did not subsequently receive empiric therapy during the clinician evaluation (70–84% of overserved patients). In addition, a substantial percentage of overserved men (16%) and overserved women (27%) were triaged to standard visits by the optimized algorithm because they reported having a sex partner with symptoms to the CASI but did not subsequently receive empiric therapy or a key diagnosis during the visit.

Discussion

We found that the performance of a simple two-component algorithm for triaging STD clinic patients was comparable to a fully optimized algorithm developed with CART. The current and more complex PHSKC algorithm had slightly higher sensitivity for the disease-focused outcome but triaged substantially fewer patients to express visits than each of the other algorithms. In addition to the factors in the simple algorithm (symptoms and contact to STD), the optimized algorithm included having a positive STD result from another provider, having a sex partner with symptoms, and age and number of sex partners in the past two months (for women). In most settings, the benefit of having a simple triage algorithm would outweigh the minute gain in sensitivity with the optimized algorithm.

Our finding that STD-related symptoms and contact to STD are the most useful factors for excluding patients from express care provides evidence to support a common clinical practice. It is generally consistent with other studies that have evaluated the predictive ability of various criteria for identifying specific STD diagnoses¹⁴ and STD test positivity.⁵ Our CART analysis to develop an optimized algorithm did not indicate much benefit from adding other potential predictors, providing confidence that the simple algorithm is appropriate for identifying patients in need of same-day treatment.

However, the appropriate triage criteria for a clinic will depend on the patient population, local public health priorities and resources, motivation for implementing an express care model, and relative value clinic leadership place on resources saved and other benefits associated with more extensive use of express care versus the cost of missed opportunities to diagnose and treat patients when they initially present for care. For example, in our clinic, while use of the simple algorithm during the analysis period would have increased clinic efficiency and access to STD screening, treatment of some cases of early syphilis (<1 case/year) and gonorrhea (~12 cases/year) would have been delayed. We would need to add a third criterion, positive STD result from another provider, to the simple algorithm to identify most of these cases, although this issue may not be applicable in some settings. Moreover, our study outcome focused solely on infections and syndromes that could be detected and treated at that visit. This is a central purpose of STD clinics, and, in the context of limited resources, triaging patients based on key symptoms and recent contact to STD may be sufficient to identify patients needing a clinician evaluation while facilitating rapid STD screening for many patients (>33%) without clinician assessment. However, the mission of many STD clinics includes providing health services to vulnerable populations. In this context, an algorithm that includes providing clinician visits to patients needing contraceptive services, linkage to HIV and hepatitis C treatment, or other priorities may be more appropriate. Notably, during most of the study period, the PHSKC STD Clinic was not yet systematically identifying candidates for HIV pre-exposure prophylaxis.²⁴ This is now a priority and requires ascertainment of risk factors different from those of clinical triage.

This study had important limitations. First, our analysis was restricted to visits with a CASI at a single STD clinic that serves a patient population which is currently and increasingly composed of predominantly MSM. Hispanic patients were less likely to complete a CASI than non-Hispanic patients.¹² While we utilized independent development and validation samples to construct the optimized algorithm, our findings may not be generalizable to other patient populations. Second, there was likely some misclassification of our outcome variable if some patients truly received express visits. Finally, our disease-focused outcome did not incorporate all aspects of high-quality STD services (e.g., patient preferences).

In conclusion, we evaluated three triage algorithms that can be implemented in STD clinics to achieve goals of improved clinic efficiency or improved disease detection. We found that simple triage criteria of symptoms and contact to STD can have high sensitivity for identifying patients with infections or syndromes that can be diagnosed at the visit, while also maximizing clinic efficiency. Additional health service-focused criteria can be layered on top of these simple criteria, depending on local resources and public health priorities, while maintaining moderately high efficiency.