Primary Care Providers’ Perspectives on Using Automated HIV Risk Prediction Models to Identify Potential Candidates for Pre-exposure Prophylaxis

Abstract

Identifying patients at increased risk for HIV acquisition can be challenging. Primary care providers (PCPs) may benefit from tools that help them identify appropriate candidates for HIV pre-exposure prophylaxis (PrEP). We and others have previously developed and validated HIV risk prediction models to identify PrEP candidates using electronic health records data. In the current study, we convened focus groups with PCPs to elicit their perspectives on using prediction models to identify PrEP candidates in clinical practice. PCPs were receptive to using prediction models to identify PrEP candidates. PCPs believed that models could facilitate patient-provider communication about HIV risk, destigmatize and standardize HIV risk assessments, help patients accurately perceive their risk, and identify PrEP candidates who might otherwise be missed. However, PCPs had concerns about patients’ reactions to having their medical records searched, harms from potential breaches in confidentiality, and the accuracy of model predictions. Interest in clinical decision-support for PrEP was greatest among PrEP-inexperienced providers. Successful implementation of prediction models will require tailoring them to providers’ preferences and addressing concerns about their use.

INTRODUCTION

There continue to be approximately 40,000 new HIV infections in the US annually.[1] HIV pre-exposure prophylaxis (PrEP) can decrease the risk of HIV acquisition by over 90% and is effective when prescribed in routine care settings.[2-7] The US Centers for Disease Control and Prevention (CDC) recommends PrEP as an HIV-prevention option for persons at substantial risk for HIV infection,[8] and the United States Preventive Services Task Force (USPSTF) issued a Grade A recommendation for PrEP for at risk individuals.[9] However, PrEP uptake has been limited thus far.[10,11] The CDC estimates that 1.2 million Americans have indications for PrEP,[12] but only approximately 200,000 people were prescribed PrEP in 2018.[13]

Primary care providers (PCPs) are likely to encounter patients who are candidates for PrEP, however, few PCPs have prescribed PrEP, in part due to the difficulty of identifying suitable candidates.[14-16] The USPSTF cites a need for validated tools to help providers better assess HIV risk and identify PrEP candidates. To address this need, we and others have developed and validated HIV risk prediction models using electronic health records (EHR) data from integrated health care systems in Massachusetts[17] and California.[18] These models use machine learning, in which computer algorithms detect patterns in data that predict outcomes of interest,[19] including risk for HIV acquisition. The models use demographics (e.g., race, primary language), diagnoses (e.g., prior syphilis or HIV counseling), laboratory tests (e.g., positive gonorrhea tests), and prescriptions (e.g., treatments for opioid addiction or syphilis) to predict HIV risk. The models have high predictive performance and can be used to alert clinicians about patients who merit clinical evaluations for PrEP.

In preparation for implementation in clinical settings, we conducted a qualitative study with PCPs to learn their perspectives on using HIV risk prediction models to identify PrEP candidates in primary care. We explored PCPs’ thoughts on implementation of a machine learning algorithm driven by EHR data, how they would prefer to receive clinical alerts generated by HIV risk prediction models, and the supports providers would need to act upon alerts.

METHODS

Study Setting and Population

PCPs were recruited from two healthcare organizations in Massachusetts: Atrius Health, a multispecialty practice that provides primary and specialty health care to approximately 800,000 patients at 32 sites, and Fenway Health, a federally-qualified community health center in Boston that provides primary care to about 35,000 patients, approximately half of whom identify as sexual and gender minorities. Fenway Health is the largest PrEP provider in New England.

Between October 2016 and April 2017, we used email invitations to recruit diverse PCPs (physicians, physician assistants, or nurse practitioners) for three focus groups at Atrius Health (n=34 PCPs in total) and one at Fenway Health (n=8 PCPs).

Focus Group Procedures

Sixty-minute focus group discussions were moderated by a physician who was board certified in infectious diseases using a semi-structured discussion guide. Participating PCPs provided verbal consent and completed surveys assessing their demographics and their experiences and confidence with PrEP prescribing using a 4-point unipolar scale (Not at all, A little, Somewhat, Very). The moderator first described the prediction models and then explored the following topics: PCPs’ opinions about using prediction models to identify PrEP candidates; how and when PCPs would prefer to receive alerts from prediction models; what PCPs would do with the information in alerts; any additional supports they would need to take action after receiving alerts; and how HIV prediction models might change their clinical practices.

We audio recorded, transcribed, and analyzed discussions using deductive and inductive content analysis.[20] Three study team members reviewed transcripts independently and generated lists of discrete concepts relating to topics in our discussion guide and emergent topics. Concepts were organized into a codebook and applied to the transcripts independently by two study team members using Atlas.ti version 7 (Berlin, Germany). Coding discrepancies were discussed until consensus was achieved. Coded transcripts were reviewed for recurring patterns and used to generate themes relating to our research objectives. Transcripts and codes were reviewed iteratively and themes were further refined.

Study procedures were approved by the Institutional Review Boards at Harvard Pilgrim Health Care and Fenway Health.

RESULTS

Participant characteristics

Of 42 PCPs, 13 (31%) were male, 28 (67%) were female, and one (2%) was genderqueer. Participants’ median age was 46 years (range 25-75), 71% were physicians, and 64% had practiced for 10 or more years. Atrius providers in general were PrEP-inexperienced: approximately one-third (32%) had never discussed PrEP with a patient and over half (52%) had never prescribed PrEP; 52% were somewhat or very confident they could identify PrEP candidates and 35% were somewhat or very confident they could safely prescribe PrEP. In contrast, Fenway Health providers were PrEP-experienced: all had prescribed PrEP to more than 10 patients and 50% of providers had prescribed to more than 50 patients; 88% were very confident they could identify PrEP candidates and all were very confident they could prescribe PrEP safely.

Theme 1: Providers perceived that HIV risk prediction models would be valuable tools for improving their preventive healthcare practices and PrEP prescribing.

PCPs generally believed that EHR-based prediction models would be valuable for identifying PrEP candidates who might otherwise be missed by busy clinicians. They thought that prediction models could integrate multiple HIV risk predictors to produce more accurate risk assessments than providers could generate using their own qualitative review of EHR data.

“I think I should be probably offering PrEP to more people than I am offering it to. And I don’t always think of it when I should. So in that way, it would improve my practice….” (Physician, Atrius)

Additional perceived benefits included promoting patient-provider dialogue about HIV and PrEP and possibly helping patients to more accurately perceive their personal risk for HIV acquisition.

“I wonder if that would actually prompt more discussion with people who are higher risk…say you know, listen, this isn’t to point you out, but I just want to get some more information. This is an option, this is a way to keep you safe…” (Physician, Atrius)

For patients at low risk for HIV who nonetheless perceived their risk to be high, providers anticipated using prediction models to reassure patients and reinforce their current protective practices.

“I would use it as being more helpful in convincing people they don’t need PrEP. Because we get a lot of people who…are using condoms 100% of the time who only have one partner and they’re like, ‘I have to be on PrEP.’” (Physician, Fenway)

Fenway Health providers believed that prediction models might help de-stigmatize and standardize HIV risk assessment for PCPs in general primary care settings who might have less experience with sexual health care. Models might thus empower these PCPs to prescribe PrEP instead of referring to specialists.

“An algorithm sort of standardizes it, and then by default sort of normalizes it to a degree, in a context where [providers] may be uncomfortable.” (Physician, Fenway)

Providers also indicated they might spend more time discussing HIV prevention with patients assigned higher risk scores and less time with patients assigned lower risk scores, which could improve efficiency.

Theme 2: Theoretical and practical concerns about implementing HIV risk prediction models could influence providers’ adoption of these models.

Despite perceived benefits, PCPs would be skeptical of using the prediction models unless they understood how the models worked. For example, providers wanted to know the types of EHR data that were used to assess patients’ risks and how algorithms integrated data.

“It would be really important to us to know what goes in [the model].” (Physician, Atrius)

Some PCPs were uneasy about potential breaches of privacy with EHR-based models. Providers also worried that patients might react negatively if they discovered that computer algorithms were using their health data to detect HIV risk or if patients felt they were inappropriately categorized as high-risk. PCPs were concerned that this may jeopardize meaningful conversations about HIV risk.

“I can see a very awkward kind of silence when the patient looks at you and says, ‘Why are you asking me this?’ And we have to respond, ‘Well some big brother decided that you may be at risk for HIV. They looked at your charts’.” (Physician, Atrius)

Concerns were also raised about medicolegal implications of prediction models, including consequences of not addressing alerts about high HIV risk scores.

Providers were not confident that models could accurately ascertain HIV risk using only EHR data. Providers expressed concern that prediction models may not be accurate unless providers document patterns of care comprehensively, which they noted does not always happen. Models might miss patients who are at risk for HIV if patients’ health records lack parameters necessary to compute HIV risk, including behaviors not consistently recorded in EHRs, such as having sexual partners who are living with HIV.

“…someone might only have Atrius Health medical records for the last year and they might have a history of gonorrhea 3 years ago that we don’t have in our health records…it’s hard to imagine an algorithm that works and so it’s hard to trust it.” (Physician, Atrius)

Theme 3: Providers lacked consensus about preferred ways to receive information about patients with high HIV risk scores.

Some providers would prefer to receive alerts without disruption of existing workflows. Providers suggested the addition of a static “HIV PrEP” prompt to existing EHR dashboards containing health maintenance recommendations, so they could address HIV prevention at times of their choosing. Other providers preferred to receive alerts immediately before or during visits with patients. There was no consensus about optimal ways to receive alerts.

“I think we get pop-up fatigue when we have too many pop-ups popping-up at you.” (Physician, Atrius)

A few providers drew upon their experiences with commonly used cardiovascular risk calculators and suggested implementing HIV prediction models in a similar fashion.

“I would love it…You know how we have the [cardiovascular] risk score? It’s so easy. You just go da-da-da and it gives you the risk score? …You as a clinician could bring it up when you wanted.” (Physician, Atrius)

Providers also differed in their preferences around the type of HIV risk information that would be presented. Some providers wanted descriptive information about HIV risk (e.g. “Your patient has an estimated X% probability of undiagnosed or future HIV”), while others preferred prescriptive alerts (e.g. “Your patient is likely an appropriate candidate for PrEP”). However, consensus did not exist for how to define thresholds for high HIV risk.

“Since this is going to be presented as a probability percentage, is there an agreement about what is the cut-off to consider PrEP for a patient? 20%? 30%? 50%? Where do you draw the line?” (Physician, Atrius)

Despite differing opinions about risk thresholds, a perceived benefit of receiving quantitative information about HIV risk was the potential to develop graphic tools to illustrate various levels of risk that providers could share with patients.

“…in approaching a patient with it, it’s nice to be able to be like, ‘Look at this. Here’s the graphic and this is where you fall, and this is why I’m a little concerned. Let’s start this conversation.” (Physician, Atrius)

Providers anticipated sharing information with patients about their specific HIV risk factors, when appropriate, to prompt discussions about behavior modification, which they felt might improve clinical practice irrespective of PrEP use.

“What is the factor in your sexual history that is really making you high risk, and maybe we can adjust that.” (Physician, Atrius)

Theme 4: Providers would trust their clinical judgment over the results of prediction models if their HIV risk assessments differed.

Providers were asked how they would react if their clinical risk assessment and the model risk assessment differed. In general, providers would trust their clinical judgment based on patients’ health histories over model predictions.

“I think ultimately it would come down to a discussion with the patient.” (Physician, Fenway)

One provider stated that they would err on the side of offering PrEP to patients flagged by prediction models even if their clinical histories suggested low-risk behaviors, because the risk of withholding effective HIV preventive services would be too high.

Clinicians with more experience providing HIV preventive care were curious as to how prediction models would compare to their own expert assessments of patients’ risk, which could in turn influence their own risk assessment practices.

“I’m really curious to compare the algorithm to my own sort of practice and history [….] I think it’ll be interesting for all of us to kind of learn to see how that would compare and contrast.” (Physician, Fenway)

Theme 5: Interest in using HIV risk prediction models and clinical decision-support for PrEP were greatest among PrEP-inexperienced providers.

PrEP-inexperienced providers (i.e., those with little or no experience prescribing PrEP) were not confident they could identify patients who might benefit from PrEP and felt uncomfortable prescribing PrEP. They expressed a need for more information about PrEP and prescribing guidance, including information on appropriate clinical monitoring for PrEP patients.

“I think most of us don’t have the conversation because we don’t have familiarity with PrEP and how to use it, when to use it, what are the side effects that we need to discuss with patients in terms of the medication. Maybe if we had more information we could initiate the conversation more often.” (Physician, Atrius)

PrEP-inexperienced providers would find clinical decision-support to be particularly important for PrEP, as compared to other preventive health interventions, because of the sensitivity and infrequency of discussions about HIV in general primary care.

“We’re not uncomfortable telling people they have risks for heart disease, and cholesterol, and colon cancer. But we are uncomfortable…at least I am, with this.” (Physician, Atrius)

In addition to clinical decision-support, PrEP-inexperienced providers would also like the option of accessing PrEP-experienced practitioners for expert support or referrals.

“I think PrEP right now is probably still manageable within a primary care practice…I sort of favor having some sort of back up, not to be completely managed, but there are just questions that come up that probably are useful to be able to curbside or have somebody.” (Physician, Atrius)

PrEP-experienced providers anticipated minimal impact of clinical decision-support on their clinical practice because HIV risk assessments and PrEP prescribing were already normative for them.

DISCUSSION

In this study, primary care clinicians perceived that using a machine learning algorithm driven by EHR data to identify persons at increased risk for HIV acquisition could improve their preventive healthcare practices and potentially increase their prescribing of PrEP. Providers viewed HIV risk prediction models as tools with the potential to improve patient-provider dialogue about HIV prevention and PrEP, including counseling about individual HIV risk and behavioral risk-reduction. However, providers’ trust in HIV risk prediction tools would be contingent upon their understanding how prediction models worked. PCPs were also concerned about how they could use these models without creating awkward discussions with patients and how to integrate these tools into their workflow most efficiently. Additional concerns included maintaining the privacy of patients’ personal health information as related to HIV risk and possible medico-legal implications of not acting upon alerts generated by these models. In light of USPSTF’s recognition that providers need additional tools to help them identify candidates for PrEP, our study provides useful information about ways to implement HIV risk prediction models so they are acceptable, useful, and effective.

In our study, PCPs expressed concerns about potential loss of privacy of patients’ protected health information if health care professionals other than their clinical providers had access to their EHR data (e.g., to generate model predictions) or their HIV risk scores. Generating prediction models with strict adherence to data privacy, and devising protocols for limiting access to HIV risk scores to those clinicians who provide direct patient care, could facilitate acceptance and trust in the security of HIV risk prediction models. Some providers also worried that patients might react negatively upon learning that their EHR data was being used to predict HIV risk, which could engender mistrust among patients. Providers may benefit from trainings to discuss prediction models in a transparent manner with patients and reassure patients about measures used to protect the privacy of their personal health data, which could help providers and patients feel more comfortable with the use of these models. An additional concern raised by PCPs in the implementation of HIV risk prediction models included potential medico-legal consequences of not addressing alerts about HIV risk scores. Engaging legal counsel at health care organizations to provide clinicians with guidance about actual versus perceived medicolegal risks with the deployment of these models, and practical ways to mitigate risks, could also assuage providers’ concerns.

We found that providers lacked consensus about how they would prefer to receive alerts from risk prediction models. Some providers preferred alerts that would interrupt their workflow, to ensure they received this information in a timely manner, whereas others strongly opposed such interruptions given their desire to limit disruptive EHR alerts. Effective implementation of alerts from HIV risk prediction models will require input from frontline providers and informatics experts, so that they meet the diverse preferences of providers and promote behavior-change among providers. This could include individualizing the delivery of alerts according to end-user preferences, when technologically feasible, or engaging other members of the health care team (e.g. nurses) to receive alerts on behalf of providers to minimize workflow disruptions. An advantage of EHR-based decision-support tools is that their implementation can be tailored to meet the needs of individual practices or providers, as their skills and experience with PrEP are likely to vary.

Our findings suggest that clinical decision-support for PrEP is likely to have the greatest impact among PrEP-inexperienced providers. These tools could help catalyze discussions about PrEP that otherwise may not occur given these providers’ lack of confidence in identifying PrEP candidates. Despite their lack of confidence in HIV risk assessment, providers indicated that they would trust their clinical judgment about HIV risk over model assessments when there were discrepancies. This suggests that implementation of risk prediction models is unlikely to result in overprescribing of PrEP, which is a potential downside to the use of any prediction model. Overall, our findings suggest that automated HIV prediction models have the potential to increase the number of primary care providers that prescribe PrEP, which could help meet the public health imperative to increase PrEP prescribing.

Prior research has shown that providers generally believe that PrEP should be provided in multiple settings where persons at increased risk for HIV access care, including primary care and sexual health clinics.[21,22] However, providers have lacked consensus on whether PrEP provision should be primarily under the purview of primary care physicians or specialized physicians, such as infectious disease physicians.[23] By building generalists’ confidence, skills, and experience with HIV risk assessments and PrEP prescribing, comprehensive clinical decision-support for PrEP could help resolve this dilemma, and facilitate more readiness to prescribe PrEP among generalized and specialized providers.

This is one of the first qualitative studies to explore primary care providers’ perspectives on using automated HIV risk prediction models to identity PrEP candidates. Our finding that providers are overall optimistic about using these models in clinical practice corroborates prior studies in which providers indicated interest in using manual tools to help with HIV risk assessments.[24] PCPs have extensive experience with clinical decision-support tools in other areas of preventative medicine, such as cardiology (e.g., cardiovascular risk assessment and statin prescription). A prior qualitative study explored how providers understand and use cardiovascular risk scores and found that providers desired simple and clear guidance on how to use risk scores, including which of the many available risk models to use. These insights can inform the types of guidance that accompany HIV models.[25] Another qualitative study found that providers use a range of strategies when assessing cardiovascular risk, sometimes using an objective method of risk assessment instead of clinical judgment, and other times adjusting absolute risk based on individual patient factors or disregarding objective risk assessments in favor of clinical experience.[26] The 2013 American College of Cardiology and American Heart Association cholesterol guidelines stress that successful implementation of primary prevention is based on shared decision-making and effective risk communication between providers and patients.[27] Lessons from these and other prediction tools can inform the implementation of HIV risk prediction tools. It is important to acknowledge that providers and patients may be less comfortable having conversations about sexual health and HIV risk as compared to conversations about cardiovascular disease because of stigma or lack of training in sexual healthcare. Thus, the delivery of clinical decision-support tools for HIV prevention may need to be accompanied by provider training in effective sexual health care and communication in order to facilitate these discussions.

Our study design has limitations. The preferences and opinions of PCPs in this study represent those of two ambulatory care practices in Boston, Massachusetts, so results may not be generalizable to other settings. However, representativeness was not the goal of this qualitative study, and we were successful in purposive sampling of providers with varying experience with PrEP. Another limitation is that we framed our focus group discussions around pre-existing HIV risk prediction tools, so results might not be applicable to other tools.

CONCLUSION

Our qualitative findings suggest that PCPs are receptive to using EHR data and automated prediction models to help them identify PrEP candidates despite having some reservations about the potential accuracy of models and patients’ perceptions of these tools. The findings from this study can directly inform how PrEP clinical decision-support tools are tailored and delivered in clinical settings so that they are maximally acceptable and effective. In this way, health care organizations and clinicians can realize the full potential of these tools to improve PrEP use, which can thus decrease new HIV transmissions.