Which Patients in the Emergency Department Should Receive Preexposure Prophylaxis? Implementation of a Predictive Analytics Approach

Abstract

Emergency Departments (EDs) have the potential to play a crucial role in HIV prevention by identifying and linking high-risk HIV-negative clients to preexposure prophylaxis (PrEP) care, but it is difficult to perform HIV risk assessment for all ED patients. We aimed to develop and implement an electronic risk score to identify ED patients who are potential candidates for PrEP. Using electronic medical record (EMR) data, we used logistic regression to model the outcome of PrEP eligibility. We converted the model into an electronic risk score and incorporated it into the EMR. The risk score is automatically calculated at triage. For patients whose risk score is above a given threshold, an automated electronic alert is sent to an HIV prevention counselor who performs real time HIV prevention counseling, risk assessment, and PrEP linkage as appropriate. The electronic risk score includes the following EMR variables: age, gender, gender of sexual partner, chief complaint, and positive test for sexually transmitted infection in the prior 6 months. A risk score ≥21 has specificity of 80.6% and sensitivity of 50%. In the first 5.5 months of implementation, the alert fired for 180 patients, 34.4% (62/180) of whom were women. Of the 51 patients who completed risk assessment, 68.6% (35/51) were interested in PrEP, 17.6% (9/51) scheduled a PrEP appointment, and 7.8% (4/51) successfully initiated PrEP. The measured number of successful PrEP initiations is likely an underestimate, as it does include patients who initiated PrEP with outside providers or referred acquaintances for PrEP care.

Introduction

An estimated 1.2 million people in the United States are at substantial risk for HIV infection and would benefit from preexposure prophylaxis (prEP) for HIV prevention.¹ Many at risk individuals, however, are unaware of the existence of PrEP for HIV prevention and even fewer are regularly taking PrEP.^2,3 Given the large discrepancy in the number of people who would benefit from PrEP compared with those actively taking PrEP, there is clearly a need to find additional venues for engaging at risk individuals in PrEP care.

The Emergency Department (ED) is one venue where many individuals who are at substantial risk for HIV infection access care. Indeed, EDs have become an important setting for routine HIV testing because ED patients are often disproportionately affected by HIV and may have limited access to primary care.^4–7 With routine opt-out HIV testing, EDs have uncovered thousands of new HIV diagnoses and have engaged patients at each stage of the HIV care continuum.^8–13 EDs, however, often offer little in regards to HIV prevention services for those who test negative for HIV. Many of these HIV-negative individuals remain at substantial risk for future HIV infection and could benefit from HIV prevention counseling and PrEP. One recent study found that among 785 patients newly diagnosed with HIV in South Carolina, more than half (54%) had at least one recent prior ED visit, representing missed opportunities for PrEP initiation before HIV seroconversion.¹⁴

In a high-volume urban ED that practices routine HIV testing, it is difficult to perform HIV prevention counseling and risk assessment for all patients who test negative for HIV. Therefore, we developed an electronic risk score utilizing electronic medical record (EMR) data present at the time of the ED encounter to identify at risk HIV-negative patients who would benefit from HIV prevention services, including prevention counseling and PrEP. We further sought to incorporate the risk score into the EMR and utilize it to guide HIV prevention counseling and PrEP initiation for at risk individuals receiving care in the ED.

Methods

The implementation of our predictive analytics approach to identify potential candidates for HIV prevention services occurred in three phases. In Phase I, we developed the electronic risk score. In Phase II, we incorporated the risk score into the EMR. Finally, in Phase III, we implemented the risk score to guide HIV prevention counseling and PrEP linkage.

Phase I: development of electronic risk score

To develop the electronic risk score, we performed post-test counseling and HIV risk assessment for patients who tested negative for HIV in the University of Chicago Medicine's (UCM) Adult ED from August 1, 2015 through November 30, 2016. Staff contacted patients telephonically after their ED visit to notify patients of their negative HIV test result, perform a behavioral risk assessment, and provide education regarding HIV prevention including PrEP. Based on patients' self-reported risk behaviors, we determined whether they were at substantial risk for future HIV infection using criteria adapted from the Centers for Disease Control and Prevention's (CDC's) PrEP guidelines.¹⁵ Patients were classified as potential candidates for HIV prevention services if they met one or more of the following criteria based on self-report: (1) man who reported condomless sex with another man in the past 6 months, (2) man or woman who reported sex with a HIV-positive partner in the prior 6 months, (3) man or woman who reported sex with multiple partners in the past 30 days, regardless of partner's gender, (4) bacterial sexually transmitted infection (STI) in the past 6 months, or (5) injection drug use with needle sharing in the past 6 months.

We created a logistic regression model using only EMR-based variables available at the time of ED triage to model the outcome of “HIV prevention services candidate,” as defined above. Potential EMR predictors included age, race, ethnicity, sex at birth, gender of sexual partners, substance use, STI testing patterns and results, ICD9/10 codes, and chief complaints related to STI symptoms. Chief complaints considered to be potentially related to an STI included the following: “STD Check Male,” ”STD Check Female,” “Penile Discharge,” “Vaginal Complaint,” “Rectal problem,” and “Rectal bleeding.” We evaluated potential predictors for missing data and excluded variables with >5% missing data. We created bivariate and multivariable logistic regression models to examine the association between the EMR variables and the outcome. Variables that were significant at p ≤ 0.1 in the bivariate model were evaluated in the multivariable model. Using stepwise regression, variables that remained significant in the multivariable model with p ≤ 0.05 were included in the final model.

To convert the model into an electronic risk score, the beta coefficients from the multivariable logistic regression model were linearly transformed to numeric risk scores. Beta coefficient values were multiplied by 10 and rounded to the nearest integer to determine the risk score point value for each risk factor.¹⁶ We then used these points to calculate a risk score for each patient.¹⁷ Additional EMR variables that are highly associated with CDC guidelines for PrEP use (e.g., man who have sex with men [MSM], positive STI test in the prior 6 months) were also included in the electronic risk score, even if there were missing data or the variables were not significant in the multivariable model. Youden index was computed to help determine the optimal risk score cutoff that balanced sensitivity and specificity, and the C statistic was calculated to assess the discrimination of the risk model.^18,19 For internal validation, the final model was refit 1000 times using the bootstrapping technique.^16,20,21 Statistical analysis was performed using Stata version 13 (StataCorp, College Station, TX). This study was reviewed by the University of Chicago Institutional Review Board and determined to be exempt.

Phase II: incorporation of risk score into EMR

Following the development and validation of the risk score, the electronic risk score was incorporated into the EMR. Our institution utilizes Epic (Epic Systems Corp., Verona, WI) as an EMR platform. At the time of ED triage, the electronic risk score is automatically calculated for all patients in the ED, utilizing the EMR variables in the scoring system.

For patients whose risk score is above a given threshold, an automated electronic alert is generated in the form of both (1) a message sent to a pager held by an HIV prevention counselor, and (2) a best practice alert delivered to an HIV prevention electronic in-basket.

Phase III: implementation of electronic risk score

From June 13, 2017 through November 30, 2017, we pilot tested the use of the electronic risk score to guide HIV prevention counseling and PrEP initiation. After receiving a page indicating that a patient who was a potential candidate for HIV prevention services had checked into the adult ED, an HIV prevention counselor would conduct a chart review of the patient's record to determine whether it would be appropriate to approach the patient for HIV prevention counseling. Patients in the ED for a high acuity problem (e.g., suspected myocardial infarction, sickle cell pain crisis, psychiatric emergency), pregnant patients, and patients already known to be HIV positive were not approached for HIV prevention counseling. All other patients were approached for HIV risk assessment and prevention counseling during the hours of 8 a.m.–5 p.m., Monday through Friday. For evenings and weekends, the HIV prevention counseling pager was not monitored in real time. Instead, during normal business hours the HIV prevention counselor reviewed the in-basket for alerts that had been generated overnight or on the weekend, and contacted patients via telephone for HIV risk assessment and prevention counseling. If patients were discharged from the ED before the HIV prevention counselor could meet with them, they were also contacted by phone after the ED visit. Three attempts to reach the patients by phone were made.

The HIV prevention counseling session included assessment of HIV knowledge, behavioral risk, self-perception of risk, education regarding PrEP, and referral for PrEP if appropriate. Patients undergoing in-person prevention counseling were also offered HIV testing. Patients who met CDC guidelines for PrEP initiation and were amenable to initiating PrEP were scheduled for an initial PrEP care appointment and had baseline labs drawn. Patients who did not meet CDC guidelines for PrEP initiation, but were interested in learning more about PrEP were provided with the phone number for the Chicago PrEPline, and information regarding drop-in PrEP clinic hours at a local community health center.

Results

Phase I: risk score development

In the risk score development phase of the project, 478 patients completed post-test counseling and HIV risk assessments. Of these, 34.5% (164/478) were considered potential candidates for HIV prevention services. EMR predictors that were significant in the model included gender, age, and chief complaint. The electronic risk score included these variables, as well as gender of sexual partner, and positive STI test in the prior 6 months. Table 1 shows beta-coefficients in the model and subsequent risk score values utilized in the electronic risk score. Table 2 shows sensitivity and specificity at various thresholds of the risk score. For the initial pilot phase of utilizing the electronic risk score in the ED, we chose a threshold of 21 to trigger an electronic alert for HIV prevention counseling. While a score of 21 has a lower sensitivity of 50%, it has a relatively high specificity of 80.6%. During the initial implementation phase, our goal was to pilot the use of the risk score for HIV prevention counseling in the ED, which is why we chose to maximize specificity at the expense of sensitivity.

Table 1.

Electronic Medical Record-Based Risk Score for Potential Candidates for HIV Prevention Services

Predictor	Coefficient	Estimate × 10	Risk score value
Male	0.69	7	7
Chief complaint related to STI-associated symptoms	0.58	6	6
Age ≤20	1.26	13	13
Age 21–24	0.83	8	8
Positive STI test in the prior 6 months	N/A	—	21
MSM (reported in chart)	N/A	—	21

MSM, man who have sex with men; N/A, not applicable; STI, sexually transmitted infection.

Table 2.

Cutoff Values, Sensitivity, and Specificity

Risk score	Sensitivity (%)	Specificity (%)	Correctly classified (%)	LR+	LR−
≥8	98.2	8.6	39.3	1.07	0.21
≥15	76.8	41.7	53.8	1.32	0.56
≥16	72.6	54.1	60.5	1.58	0.51
≥21	50.0	80.6	70.1	3.02	0.71
≥38	14.0	100.0	70.5		0.86

LR, likelihood ratio.

Phase II: incorporation of risk score into EMR

Figure 1 shows an example of the EMR ED patient roster with the risk score column (X). Risk scores ranged from 0 to 63. Figure 2 demonstrates an example calculation of the risk score for a sample patient. Given that the risk score was dependent on the information available in the EMR, we reviewed the charts of the 478 patients in the risk score development phase, and found that 71% of electronic charts were missing gender of sexual partner in the social history section.

FIG. 1.

EMR electronic risk score. Example of emergency department roster with the risk score column (X) in the EMR. EMR, electronic medical record. (Color image can be found at www.liebertonline.com/apc).

FIG. 2.

Sample HIV prevention risk score calculation in EMR. (Color image can be found at www.liebertonline.com/apc).

Phase III: implementation of electronic risk score

During the first five and a half months of implementation, the electronic alert was generated for 180 individuals in the ED. Table 3 details the characteristics of patients who received an electronic alert, including demographics and ED chief complaint. After initial chart reviews, 18 patients were deemed not appropriate to approach for HIV prevention counseling due to high acuity of medical problems. Of the remaining 163 patients, 26 were approached for in-person HIV prevention counseling and risk assessment during their ED stay, and 35 were successfully contacted via telephone after their ED visit. The remaining patients were not able to be reached for post-test counseling or risk assessment due to incorrect contact information, no answer, patient refusal, or unavailability of HIV prevention counselor. Patients who were successfully contacted for HIV prevention counseling did not differ significantly in age, sex, race, or chief complaint from those who were not reached for counseling (data not shown).

Table 3.

Characteristics of Emergency Department Patients Who Received an Electronic Alert for HIV Prevention Services

Patient characteristics	N (%)
Sex
Male	118 (65.6%)
Female	61 (33.9%)
Transgender female	1 (0.6%)
Age (years, mean ± SD)	25.4 ± 9.8
Race/ethnicity
African American	162 (90.0%)
Caucasian	8 (4.4%)
Latinx	8 (4.4%)
Mixed race/other	2 (1.1%)
Chief complaint
STI related complaints	75 (41.7%)
Abdominal pain	24 (13.3%)
Vaginal complaint	18 (10.0%)
Urinary problem	11 (6.1%)
Rectal complaint	5 (2.8%)
Drug or alcohol intoxication	3 (1.7%)
Mental illness	2 (1.1%)
Other	42 (23.3%)

SD, standard deviation.

Patients who were contacted during their ED visit were more likely to complete HIV prevention counseling and risk assessment than those who were reached by phone after their visit (96.2%, 25/26 vs. 74.3%, 26/35, p = 0.02). Among those who completed risk assessment and counseling, 68.6% (35/51) were interested in PrEP. Nine of these patients were scheduled for an initial PrEP care appointment. The rest of the patients were given the phone number to the Chicago PrEP line, an information warmline and PrEP linkage service for PrEP initiation at the major PrEP providers in the city. Of those scheduled for a PrEP appointment, one-third (3/9) successfully initiated PrEP. Of those who were provided drop-in hours for PrEP care, 3% (1/35) successfully initiated PrEP (Fig. 3).

FIG. 3.

Flow diagram of outcomes among patients receiving electronic alert. PrEP, preexposure prophylaxis. (Color image can be found at www.liebertonline.com/apc).

Discussion

In a large urban ED setting, we created an EMR-based risk score to identify potential candidates for HIV prevention counseling and PrEP. While ideally all patients would receive counseling regarding HIV prevention, in a busy urban ED setting, providing in-depth counseling and behavioral risk assessment for everyone tested for HIV is not feasible. In a limited resource environment, our electronic risk score can be used to identify the patients who are at highest risk for future HIV infection and who would most benefit from counseling, risk assessment, and PrEP initiation. The electronic risk score incorporates age, sex, chief complaint, and STI test results, all simple fields easily accessible in the EMR. Thus, it could be easily adopted by other healthcare providers that utilize a basic EMR. Moreover, because the risk score is automated, it provides a real time alert to prompt HIV prevention efforts. In clinical care, there is often a discrepancy between providers' intention to prescribe PrEP and actual PrEP prescription.²² An automated alert may improve adherence to best practices in HIV prevention.^9,23

We chose to utilize an on-call HIV prevention counselor to perform HIV risk assessment and PrEP linkage. Depending on staffing resources, another model would be to train ED providers to provide these services. We worked closely with the ED staff to ensure that the HIV prevention counseling was not impeding ED workflow or delaying ED care for patients by providing counseling for patients in a private area while they were waiting to be evaluated and treated by ED staff.

We found that in-person prevention counseling at the time of ED visit was more successful than telephone counseling after the visit. Patients whom we reached in person were more likely to agree to HIV prevention counseling and were more likely to complete HIV risk assessment. Many potential candidates for HIV prevention services accessed care in the ED in the evenings and on weekends when the HIV prevention counselor was not available. Attempts to reach these patients by phone after their ED visit were often unsuccessful. The UCM ED serves a high-poverty community, and some patients may have inconsistent telephone access or may not have cell phone plans with unlimited voice service. Expanding the hours when in-person HIV prevention counseling is available would likely improve the ability to provide HIV prevention counseling and PrEP referral for at risk patients.

While a substantial number of patients were interested in PrEP, only 11% (4/35) of those interested successfully initiated PrEP. This number is likely an underestimate of the actual number of patients who began PrEP as a result of this program. Some patients reported that they planned to initiate PrEP with their primary care provider or at another clinic, and we did not follow up with patients to confirm whether or not they did so. Other patients mentioned that while they themselves did not feel they needed PrEP, they planned to refer friends or family members for PrEP initiation. A substantial proportion of patients who were scheduled for PrEP care did not attend their PrEP appointments (67%, 6/9). This is similar to other PrEP linkage to care rates.²⁴

While many PrEP outreach strategies primarily focus on MSM, women, particularly African American women, account for a substantial percentage of new HIV infections in the United States.²⁵ Thirty-four of individuals who were identified by the electronic alert were cis-gender women. Of the women who completed HIV risk assessment and prevention counseling, the majority (61%) were interested in PrEP. This suggests that the ED is a potential venue to reach at risk women for HIV prevention counseling and PrEP referral. Many at risk women sought care in the ED for sexual health concerns (i.e., concern for STI), and were open to learning about and possibly initiating PrEP.

This study had some notable limitations. The electronic risk score was developed in an urban ED on the south side of Chicago, serving a predominantly African American, high-poverty community residing in high-HIV prevalence neighborhoods. Therefore, our results may not be generalizable to other healthcare settings, but other EDs in large urban areas may serve similar populations. In addition, the risk score is dependent on electronic data available in the EMR. If gender of sexual partner is missing in the EMR documentation, then the electronic alert may miss potentially at risk individuals. In our EMR, a substantial number of charts (71%) were missing data regarding gender of sexual partner. In addition, if a patient had an STI diagnosed at another healthcare facility, it would not be detected by the electronic algorithm.

For our pilot implementation, we chose a relatively high threshold score of 21 that had high specificity (>80%), but lower sensitivity (50%). Therefore, we likely missed providing HIV prevention counseling for some patients who might have been PrEP candidates. In the future, we plan to lower the threshold score for the electronic alert to improve sensitivity and reach more at risk patients. Future efforts could also investigate the feasibility of rapid PrEP initiation during the actual ED visit after obtaining a negative HIV test result and baseline labs.

In conclusion, individuals who seek care in the ED are often at disproportionate risk for HIV. EDs have the potential to play a crucial role in HIV prevention by identifying and linking high-risk HIV-negative clients to PrEP care. We have developed and implemented an automated electronic risk score to allow EDs to quickly and efficiently identify at risk individuals for PrEP and link them to care.

Acknowledgments

This work was funded by Gilead Sciences, grant #801266112. We thank the University of Chicago Information Technology and Emergency Department staff for their assistance with implementing the EMR-based risk score in the ED.

Author Disclosure Statement

J.P.R. and J.A.S. report receiving grant money from Gilead Sciences.