Implementing Electronic Substance Use Disorder and Depression and Anxiety Screening and Behavioral Interventions in Primary Care Clinics Serving People with HIV: Protocol for the Promoting Access to Care Engagement (PACE) Trial

Abstract

Background:

Substance use disorders (SUDs) and psychiatric disorders are common among people with HIV (PWH) and lead to poor outcomes. Yet these conditions often go unrecognized and untreated in primary care.

Methods:

The Promoting Access to Care Engagement (PACE) trial currently in process examines the impact of self-administered electronic screening for SUD risk, depression and anxiety in three large Kaiser Permanente Northern California primary care clinics serving over 5,000 PWH. Screening uses validated measures (Tobacco, Alcohol, Prescription medication, and other Substance use [TAPS]; and the Adult Outcomes Questionnaire [AOQ], which includes the Patient Health Questionnaire [PHQ-9] and Generalized Anxiety Disorder [GAD-2]) delivered via three modalities (secure messaging, tablets in waiting rooms, and desktop computers in exam rooms). Results are integrated automatically into the electronic health record. Based on screening results and physician referrals, behavioral health specialists embedded in primary care initiate motivational interviewing- and cognitive behavioral therapy-based brief treatment and link patients to addiction and psychiatry clinics as needed. Analyses examine implementation (screening and treatment rates) and effectiveness (SUD, depression and anxiety symptoms; HIV viral control) outcomes using a stepped-wedge design, with a 12-month intervention phase implemented sequentially in the clinics, and a 24-month usual care period prior to implementation in each clinic functioning as sequential observational phases for comparison. We also evaluate screening and treatment costs and implementation barriers and facilitators.

Discussion:

The study examines innovative, technology-facilitated strategies for improving assessment and treatment in primary care. Results may help to inform substance use, mental health, and HIV services.

1. Introduction

Substance use disorders, depression, and anxiety are common among people with HIV (PWH) and add significant complexity to clinical care, resulting in poor HIV outcomes and higher mortality [1, 2]. Evidence-based substance use disorder (SUD), depression and anxiety interventions have been developed yet not effectively implemented in primary care settings serving PWH and other patient populations. For example, significant barriers to effective implementation including under-identification, limited intervention expertise, poor treatment fidelity, and clinic time constraints have been well documented [3–5].

SUDs have serious consequences for PWH, including poor antiretroviral therapy (ART) adherence [6–8], limiting HIV RNA control [9–11], and contributing to increased HIV transmission risk. However, substance use is often underreported to providers [12, 13], and even when recognized, providers often fail to advise cutting back or provide other interventions [14–16]. Yet when patients reduce use, HIV and mental health outcomes improve [17–19].

PWH also have a high burden of psychiatric problems, particularly depression [20–22], with prevalence ranging from 36% in a national sample of HIV-positive adults [23] to 53% in a primary care sample [24]. Anxiety among PWH also is very common but has been under-investigated relative to depression [25, 26]. As with SUDs, psychiatric disorders contribute to numerous medical comorbidities, poor ART adherence, faster HIV disease progression, higher cost of care, and higher mortality [27–30].

SUD and mental health screening in primary care is a key challenge for health care systems, and this difficulty impacts many other patient populations apart from PWH. For example, interviewer-administered surveys often lose fidelity due to wording changes when administered by clinical staff [31, 32]. Other screening barriers include insufficient staff training, reluctance to discuss sensitive topics, and limited provider time in the context of busy primary care clinics [32, 33]. If properly implemented, computerized self-administered screening can address these challenges, is highly acceptable to patients [34–40] and is cost effective.

In addition to screening, primary care-based intervention delivery and/or referral to specialty addiction and mental health treatment is necessary to improve care. Behavioral health specialists (BHSs) are trained to identify, triage and manage patients with behavioral health problems, including depression and substance use problems, but are underutilized [41–43]. BHSs can effectively deliver behavioral interventions such as motivational interviewing (MI) for SUDs and brief cognitive behavioral therapy (CBT) for depression and anxiety, and link patients to specialty SUD and mental health treatment as needed. Thus, BHSs can enhance care teams and help overcome behavioral health care implementation challenges that many clinics face.

The Promoting Access to Care Engagement (PACE) trial currently in process examines a novel intervention that combines routine electronic screening for SUD and mental health delivered to PWH every six months in 3 HIV primary care clinics in Kaiser Permanente Northern California (KPNC). Results are viewed by clinic staff including a BHS to deliver behavioral treatment. The study examines both implementation outcomes (e.g., screening and treatment rates), and effectiveness (e.g., substance use, mental health symptoms, and HIV viral control), to inform the field regarding strategies for primary care-based behavioral health treatment integration.

2. Methods

2.1. Specific aims.

Aim 1.

To evaluate the implementation of computerized SUD, depression and anxiety screening and BHS-delivered intervention in HIV primary care.

Aim 2.

To examine the effectiveness of computerized screening and BHS-delivered intervention in HIV primary care among patients with moderate or high SUD risk, depression and anxiety severity.

Aim 3.

To determine implementation costs and cost-effectiveness of screening and BHS-delivered intervention.

Aim 4.

Perform key informant interviews to evaluate provider- and clinic-level implementation barriers and facilitators to computerized screening and BHS-delivered intervention.

2.2. Study design overview

This hybrid design Type 2 intervention study [44] evaluates the implementation, effectiveness and cost of electronic screening and treatment for SUDs, depression and anxiety among PWH. The study is based in KPNC primary care clinics in Oakland, Sacramento and San Francisco that have an HIV care focus, and collectively serve over 5,000 PWH. The study team tracks scheduled appointments of PWH in these three clinics using the KPNC HIV registry and asks patients to complete an electronic screening instrument prior to their routine visits via secure messaging through the patient portal (www.kp.org) into the EHR. If screening is not completed in advance, patients are asked to complete it on a tablet in the waiting room or on a desktop computer in the exam room. The results are directly incorporated into the EHR and viewable by clinical staff including physicians and BHSs. Physicians review screening results and engage BHS support as needed, with screening results also informing BHS treatment sessions. The study employs a stepped wedge design [45], with implementation and effectiveness outcomes in the 12-month intervention phase compared with outcomes in the 24-month observational phase (i.e., usual care). Rollout of the study intervention (computerized screening + BHS-delivered treatment) occurs sequentially at the 3 clinics, allowing for refinement of procedures and accommodating implementation challenges at each site. The observational periods are also sequential (Figure 1). Study procedures were approved by the University of California, San Francisco and KPNC Institutional Review Boards.

Figure 1.

Stepped-wedge design to evaluate outcomes in the PACE Trial.

Notes: *Each time period represents six months. Observational period cells are light gray. Intervention period cells are dark gray. Rollout of the study intervention (computerized screening + BHS-delivered treatment) occurs sequentially at the 3 HIV primary care clinics, starting with Oakland. The observational periods are also sequential.

2.3. Study setting

KPNC is an integrated healthcare delivery system serving >4.3 million members at 21 hospital-based Medical Centers and >200 medical offices. This system provides comprehensive outpatient and inpatient care, including pharmacy and laboratory services. The study sites were selected because of the size of their HIV-positive patient populations, complementary demographics, and willingness of clinic directors to participate (Table 1).

Table 1.

Characteristics of HIV patients in Kaiser Permanente Northern California clinics participating in the PACE Trial.

	Oakland	Sacramento	San Francisco
Patients, n	1,275	850	2,937
Men (%)	85	90	97
Mean age in years	51	50	53
Race/ethnicity (%)
White	36	57	57
African-American	35	15	9
Hispanic	14	13	16
Asian	6	5	8
Other	2	2	3
Unknown	7	8	7
HIV risk (%)
Heterosexual	20	16	4
Injection drug use	5	7	7
Men who have sex with men	67	67	76
Other	1	2	<1
Unknown	8	8	12
Prior AIDS diagnosis (%)	52	56	59
Receiving ART (%)	94	92	94
Mean CD4, cells/µl	701	735	682
HIV RNA<75 copies/ml, (%)	87	86	87

Notes: Data are based on electronic health records as of 12/31/18. PACE = Promoting Access to Care Engagement. ART = antiretroviral therapy medication.

2.4. Conceptual model for evaluating implementation

Prior studies indicate that key facilitators of successful primary care implementation include having a flexible treatment protocol, focusing on targeted problems and solutions that work within the time constraints of primary care, and being patient-centered [46, 47]. These features are consistent with the Practical, Robust Implementation and Sustainability Model (PRISM) [48], which holds that implementation of interventions in medical care settings is influenced by intervention, patient, and organizational characteristics. (Figure 2).

Figure 2.

Conceptual model of factors associated with successful intervention implementation in health care settings.

Notes: Conceptual model is based on the Practical, Robust Implementation and Sustainability Model (PRISM) [48]. BHS= behavioral health specialist; SUD = substance use disorder.

We anticipate that key intervention characteristics affecting intervention adoption are the mode of screening delivery and the accessibility and effectiveness of the intervention. Regarding patient characteristics, our work on automated mental health and substance use screening has found that older adults were less likely to complete a computerized intake system [49]. Among HIV patients in KPNC, patient use of the shared EHR was higher among those with medical need (i.e., recent CD4 decline or HIV RNA increase) [50]. Others have found that among PWH with psychiatric disorders, accessing psychiatric services was associated with older age, white race, and greater HIV severity [51]; and that among those with SUD, accessing treatment was associated with black race [51], lower CD4, not taking ART [52], and injection drug use [53].

Organizational characteristics also influence implementation [54, 55]. Measures of “resources for change” in primary care, including history of practice changes, structure of relationships with specialty care, and attitudes towards evidence-based treatment predict adoption of new primary care guidelines [56, 57]. We measure these factors in our model, including qualitative evaluation of competing priorities and quantitative measures of attitudes towards evidence-based practices and levels of behavioral health integration.

2.5. Computerized screening modalities

Three modalities are used to administer the screening questionnaire: 1) self-administration online via the electronic patient portal; 2) self-administration in clinic on a secure tablet provided by reception; 3) self-administration during the clinic visit via desktop computer. Our intent is to maximize online completion, with the second and third options serving as backup approaches for patients not completing the questionnaire prior to their visit. Electronic administration of questionnaires via waiting room tablets is a workflow widely used in KPNC primary care clinics. Study sites already utilizing tablets for other screening instruments are provided with additional tablets to ensure availability for PACE patients, and integrated administration of the screening instrument into existing workflows. Study sites not already using tablets are provided the necessary equipment.

Patients who have any upcoming in-person appointment with an HIV care provider are flagged to complete the questionnaire if they have not done so in the prior 6 months, and the questionnaire is attached to the appointment in the EHR. Patients who have activated their accounts in the KPNC electronic patient portal then receive a secure message via email up to two weeks prior to their visit, which includes a link to complete the questionnaire online. Patients who do not complete the questionnaire prior to their appointment have a flag in the EHR that alerts reception staff to provide them with a tablet at clinic registration. Patients unable to complete the questionnaire online or by tablet also have the opportunity to complete it via desktop computer in the exam room. Desktop administration uses EHR functionality that limits patient access to only the questionnaire. All three modalities automatically import patient responses directly into EHR and results are immediately available to clinicians (Figure 3).

Figure 3.

PACE intervention approach to computerized substance use, depression and anxiety screening and treatment in HIV primary care clinics.

Notes: TAPS/AOQ = Tobacco, Alcohol, Prescription Medication and other Substance Use/Adult Outcomes Questionnaire; PCP = primary care provider; BHS = behavioral health specialist; EHR = electronic health record. KP.org is Kaiser Permanente’s electronic patient portal.

2.6. Computerized patient self-administered questionnaire

SUD risk is measured using the Tobacco, Alcohol, Prescription medication, and other Substance use (TAPS) instrument [58, 59]. The Adult Outcomes Questionnaire (AOQ), initially developed by KPNC, includes the Patient Health Questionnaire (PHQ-9) [60] for depression and the Generalized Anxiety Disorder (GAD-2) [61] for anxiety. These measures are validated for electronic self-administration [62, 63]. In our study, these measures are combined as the TAPS/AOQ.

Prior to beginning the questionnaire, introductory text informs patients that TAPS/AOQ screening is part of research on ways to improve patient care, completion is optional, and their answers will be recorded in the EHR. Patients are directed to contact their primary care providers or KPNC Psychiatry if they experience any emotional distress when responding, and are provided a KPNC after-hours mental health phone number. Instructions indicate that in case of medical or psychiatric emergencies they should call 911 or go to their nearest hospital.

Automated TAPS/AOQ reporting is enabled via a questionnaire feature of the EHR. The generated reports, automatically displayed in a clinical progress note using a smart phrase in the EHR, include yes/no responses for self-reported use of tobacco, alcohol (4+ drinks/day for women, 5+ drinks/day for men), and 6 specific drug classes (including illicit and non-medical drug use) in the past year, follow-up questions on use in the prior 3 months, and a substance use risk level (low, medium and high) for each substance. Depression and anxiety scores are presented numerically. The intent of the report is to serve as a prompt in the EHR for medical providers to initiate a conversation and/or referral to the BHS, and a starting point for BHSs as they conduct further assessment and MI- and CBT-based treatment.

2.7. BHS-delivered intervention

BHSs are Masters-level licensed clinical social workers, marriage and family therapists or doctoral-level psychologists with experience in behavioral interventions. Currently BHSs provide care across medical centers. They often co-manage patients’ medical conditions through non-pharmacologic interventions, collaborate with primary care providers, and provide consultation in the areas of mental health, behavioral medicine, and health psychology.

BHS intervention includes patient engagement and patient-centered goal setting (e.g., noting screening results but also focusing on patients’ priorities and concerns). BHSs offer individual MI/CBT sessions (by phone and in person) according to clinical assessment and patient preference, in line with organizational capacity and department guidelines. MI and CBT can be combined to address co-occurring mental health and substance problems [64]. Intervention has no specific time limits, but MI and brief CBT can be effectively delivered in less than 6 sessions [65, 66], and we will examine BHS utilization and time demands in our analysis. BHSs use MI to enhance motivation to initiate specialty SUD or psychiatry treatment within KPNC for higher severity patients, which is available for members in facilities nearby.

2.8. Clinician collaboration and training

The study team is meeting with clinic primary care providers and staff throughout the implementation phase to assist with planning, based on our prior work on computerized screening, e.g., workflow and staff training [49]. Each site is responsible for identifying a PACE “champion” who serves as a primary liaison between the clinical site and the study team, advising on adaptations necessary for local implementation, providing introductions for study team to clinic management and staff, and functioning as the “face of the study” in the clinic. The champions participate in frequent study team meetings to give them a thorough understanding of the project. Orientation for physicians and other providers (e.g., nurse practitioners) is provided by the PACE champion for each site and consists of two one-hour sessions. These trainings include overview and rationale for the project, workflows for each screening modality, use of the new screening measure (TAPS/AOQ), and information regarding how to locate TAPS/AOQ scores in the EHR.

The team also provides BHSs training on MI and CBT-based interventions for depression, anxiety and substance use problems. Training includes 10 hours of instruction and supervised practice of brief, solution-focused, MI/CBT interventions, similar to prior trials [65, 67]. BHS training also includes new technology and workflow (e.g., accessing TAPS/AOQ screening results in the EHR), interpretation and utilization of TAPS/AOQ scores, guidelines for when to help patients initiate specialty SUD and psychiatry care (e.g., based on severity, complexity, and patient goals), and follow-up (monitoring patient engagement and following up as needed with patient and care team to ensure linkage to treatment). Monthly consultation with authors DDS and ASL helps to reinforce key clinical skills and troubleshoot system and workflow challenges.

2.9. Observational phase

The 24 months prior to implementation at each study clinic is considered the observational phase, consisting of usual care. Hazardous drinking screening has been implemented systematically using NIAAA guidelines [68], along with standardized tobacco screening; both screenings are administered verbally by certified and licensed medical assistants during the rooming process, as part of Kaiser Permanente’s essential preventive health measures. Screening for other substances is not standardized and relies on physicians’ clinical practices. A combination of appointment codes and appropriate International Classification of Diseases, (ICD) 9^th and 10^th edition indicate screening, e.g., a depression diagnosis code is indicative that screening took place. Drug use data are captured in a section of the medical record where providers can enter information about a patient’s social history. Treatment for SUDs includes advice from primary care providers to reduce use and services in specialty addiction treatment clinics [69]. Treatments provided for depression and anxiety are based on current best practices for medication management and psychotherapy [70–72]. Mild to moderate depression/anxiety are often addressed in primary care with medication, whereas patients with more severe symptoms are usually referred to psychiatry clinics. We track these aspects of patient care in the EHR.

2.10. Data Sources

Study data are obtained from the EHR, HIV Registry, and informant interviews. The EHR database links information using unique member identifiers from multiple data sources including demographics, membership, ambulatory, inpatient, laboratory, patient questionnaire, and prescription data. The HIV Registry maintains up-to-date lists of all HIV patients, HIV transmission risk factors, dates of known HIV infection, AIDS diagnoses, and complete HIV-related lab and pharmacy data. Based on the HIV registry, a total of 9,418 HIV patients were current KPNC members as of December 2018, including 5,062 (54% of all KPNC HIV patients) from the three study clinics (Table 1). Informant interviews by the study team include physicians, medical assistants, BHSs and patients from each facility, to be conducted following implementation.

2.11. Measures

We include relevant patient demographic factors in all analyses, including age, sex, race/ethnicity, and HIV risk group. Insurance coverage-related variables include deductible level and Medicaid versus other source of coverage [73]. Other measures for each Aim are described below. See also Table 2.

Table 2.

PACE study measures by aim.

Domain	Study Measure	Source	Aim 1*	Aim 2*	Aim 3*	Aim 4*
Demographics	HIV Risk Status	EHR	X	X	X
	Age		X	X
	Gender		X	X
	Race		X	X
	Neighborhood Deprivation Index		X	X
Comorbidities	Charlson Score	EHR	X	X	X
	VACS Index Score			X
HIV Viral Control	CD4 Value	HIV Registry	X	X	X
	Viral Load Value		X	X	X
	AIDS Status		X	X	X
	Length of Time with HIV		X	X
	HIV Care Retention			X	X
	AIDS-related Diagnoses			X	X
	ART Medication Adherence		X	X	X
Screening	SUD Screening (e.g., TAPS/AOQ, alcohol and drug screening by providers)	TAPS/AOQ and EHR	X	X	X
	Depression and Anxiety Screening (e.g., AOQ, psychiatric diagnoses)		X	X	X
Screening costs	Cost per patient	KPNC General ledger			X
Primary care treatment	Outpatient Care	EHR	X	X
	Utilization—Visits		X	X	X
	Brief Interventions		X	X	X
	BHS-Delivered Intervention
Specialty care treatment	Psychiatry Visits	EHR	X	X	X
	SUD Clinic Visits		X	X	X
Substance use and mental health symptoms	SUD Symptoms	EHR	X	X	X
	Depression Symptoms (e.g., Alcohol/dependence/abuse, ICD-09/10 codes)		X	X	X
	Anxiety Symptoms		X	X	X
Treatment	SUD/Depression/Anxiety/Smoking Treatment; Antidepressant Medications	EHR	X	X	X
Provider Factors	Evidence-Based Practice Attitude Scale	Interview				X
Clinic Factors	Integrated Practice Assessment Tool	Interview				X
Barriers & Facilitators	Qualitative Interview Data	Interview				X

Notes:

^*Aim 1: To evaluate the implementation of computerized SUD and depression screening and BHS-delivered intervention in HIV primary care.

^*Aim 2: To examine the effectiveness of computerized screening and BHS-delivered intervention in HIV primary care among patients with moderate or high SUD risk or depression severity.

^*Aim 3: To determine implementation costs and cost-effectiveness of screening and BHS-delivered intervention.

^*Aim 4: Perform key informant interviews to evaluate provider- and clinic-level implementation barriers and facilitators to computerized screening and BHS-delivered intervention.

PACE = Promoting Access to Care Engagement; EHR = electronic health record; SUD = substance use disorder; KPNC = Kaiser Permanente Northern California; ART = antiretroviral therapy; VACS = Veteran’s Aging Cohort Study; BHS = behavioral health specialist; AOQ = adult outcomes questionnaire.

2.11.1. SUD, depression and anxiety screening

During the observational phase, measures include EHR-based provider-documented screening for SUDs, depression and anxiety. Intervention phase screening includes patient self-administered screening using the TAPS/AOQ instrument offered to patients every six months and recorded in the EHR.

2.11.2. BHS-delivered treatment

During the observational and intervention phases, treatment is defined as one or more in-person or phone visits with a BHS. We also measure the number of phone and in-person visits.

2.11.3. SUD, depression and anxiety treatment initiation

We analyze SUD and psychiatry treatment initiation rates using electronic administrative data; i.e., whether the patient attended at least one in-person or phone visit with a BHS and/or in SUD or psychiatry treatment.

2.11.4. Psychotropic medication prescription rates

Prescription rates are extracted from the EHR and include standard antidepressant and anxiolytic medications based on best practices guidelines [74].

2.11.5. Antiretroviral (ART) medication adherence

ART adherence is defined using the Medication Possession Ratio (MPR) [75, 76] for each ART prescribed [77]. MPR is calculated using a numerator of days’ supply dispensed from first fill to end of the interval (i.e., 12 months), and a denominator of total days between first fill to end of the interval, with values ranging from 0% to 100%. Mean refill adherence is calculated across individual ARTs dispensed in the interval [76, 78].

2.11.6. HIV RNA response

HIV RNA control is defined as undetectable HIV RNA levels below 75 copies/ml.

2.11.7. Retention in care

We use the Institute of Medicine’s encounter-based definition of retention, defined as ≥2 HIV primary care visits within a 12-month period, ≥90 days apart [79].

2.11.8. VACS Index Score

The VACS score has been shown to predict mortality [80]. It incorporates 7 routinely collected clinical variables, including age, CD4, HIV RNA, hemoglobin, fibrosis index 4 (FIB-4), Hepatitis C, and estimated glomerular filtration rate. The FIB-4 index incorporates age, platelets, and two liver function test results, including aspartate aminotransferase and alanine aminotransferase. The VACS Index 2.0, adding albumin, WBC, and BMI to version 1.0 and using continuous variables, provides improved mortality risk discrimination [81].

2.11.9. Substance use, depression and anxiety

The TAPS assesses frequency (never, less than monthly, monthly, weekly, daily or almost daily) of past-year substance use for 4 broad categories of substances (tobacco, alcohol, prescription medications used other than as prescribed, and other drugs) and provides a risk score for each specific substance used in the past 3 months: 1) tobacco, 2) alcohol, 3) opioid pain medication misuse, 4) anxiety/sleep medication misuse, 5) stimulant medication misuse, 6) cannabis, 7) illicit stimulants (cocaine, crack and methamphetamine), and 8) heroin [58]. We added items about other drug use and injection drug use. We use the PHQ-9[60] for depression and GAD-2[61] for anxiety, measured continuously.

2.11.10. Screening costs

These costs include equipment cost of the tablet computers and the labor costs for programming each screening modality (patient portal, tablet, and desktop), as well as time spent on instrument programming and the development of tracking systems, derived from the KPNC general ledger. Self-administered computerized screening entails no additional labor cost.

2.11.11. BHS training costs

Training consists of approximately 10 hours of BHS staff time and includes MI/CBT training, BHS consultation with the research team.

2.11.12. Intervention Costs

We include costs associated with clinic staff meetings and training to orient providers to the project and BHS role. BHS-delivered intervention may be done over the phone or in person. Intervention time is based on appointment codes to determine the total cost of BHS interventions with HIV-positive patients during observation versus intervention phase. The cost will be estimated by the average time spent in delivering the intervention x wage per minute of staff performing the intervention.

2.11.13. The Evidence-Based Practice Attitude Scale (EBPAS)

This is a 36-item scale developed to measure provider attitudes in the context of mental health care [57], and has strong psychometric properties [82]. The EBPAS yields an overall score and 12 subscales related to the importance of using evidence-based interventions, e.g., MI and CBT, perceived administrative support, and potential barriers to intervention delivery.

2.11.14. The Integrated Practice Assessment Tool (IPAT)

This 8-item instrument is used to assess levels of integration of behavioral health in the context of primary care, based on 6 levels of care integration [83]. The IPAT will measure the degree to which BHSs are effectively integrated into the clinics.

2.11.15. Qualitative interviews

Interviews with 15 patients, 15 providers, as well as technology partners are conducted over the phone or in person. Patients and providers are asked to share their experiences with electronic screening, the degree to which providers incorporate results into patient visits, perceptions regarding working with BHSs, barriers and facilitators to screening and treatment, benefits and challenges of the three different screening modalities, and perceptions regarding acceptability and sustainability of computerized screening and BHS intervention. Technology partners are asked about their perceptions regarding work with the study team in developing systems related to TAPS/AOQ delivery and result reporting. Interviews are recorded and transcribed.

2.11.16. Data analysis

The study’s primary aims are to examine the implementation, effectiveness and cost of computerized screening and BHS-delivered intervention. HIV, SUD, depression and anxiety outcomes will be compared between the observational and intervention phase, including short-term (i.e., within 12-month intervention phase) and long-term (i.e., within 1–2 years after initial intervention phase) outcomes. As described below, we will also employ two complementary study designs: 1) a repeated cross-section design with a different mix of individuals in each cluster (clinic) for examining implementation outcomes (e.g., Aim 1: screening rate) and 2) a cohort design for examining effectiveness in outcomes over time within the same individuals (e.g., Aim 2: retention in HIV care). Data analysis for both the cross-sectional and cohort approaches will use the random effects modeling framework which accounts for correlation between individuals within clusters and over time and is generalizable to non-normal distributions [84]. These models may be estimated using SAS PROC MIXED or PROC NLMIXED for normal or non-normal distributions allowing for correlation between observations and variable cluster sizes. We will assess the incremental cost-effectiveness ratio (ICER) as the additional cost per unit of outcome (e.g., 1% increase in mean ART adherence) between the observation and intervention phases. A cost-effectiveness acceptability curve will be obtained from the distribution of the ICER which will be constructed using the bootstrap method based on the variances of estimates of costs and effectiveness [85]. ICER for screening and BHS intervention will be calculated separately.

We will integrate qualitative and quantitative findings using a triangulation approach to identify varying levels of agreement between study components [86], as in our prior work [73]. The qualitative analysis will note recurring implementation-related themes, comparing interviewees in different clinics on demographics, position and knowledge of and attitudes about integration of behavioral health into HIV primary care. Analysis also will note recurring patient experience-related themes (e.g., screening modalities, medical provider/BHS use of screening results, willingness to engage with a BHS).

Analyses will include all PWH who are members of the 3 clinics during the study time period (~5,000). We anticipate adequate statistical power. For example, assuming an overall screening rate of 85% of whom 25% screen positive and one third of whom are severe enough to initiate specialty SUD treatment, we will have a power of .97 to detect a 12% difference in the treatment initiation rate between the observation and intervention phases. Hypothesis tests involving closed cohort designs (i.e., repeated measures on the same individual as in Aim 2) will have greater power than cross-sectional designs with the same sample size. For example, we will have > .99 power to detect a difference of .15 standard deviations in ART adherence between initial and follow-up screenings.

3. Results

The study is currently in process, with outcomes anticipated to be available in 2021. Results below describe key factors affecting development of screening instruments, refinement of study procedures in conjunction with IRB modifications, and clinical concerns addressed in collaboration with clinic leaders prior to launching the intervention phase of the study.

3.1. Technological considerations

The study team worked closely with KPNC Information Technology groups to integrate screening into the EHR; ensure that the TAPS/AOQ could be effectively delivered across three modalities each with different technical demands; create and distribute job aids (i.e., training guides) to clinicians and reception staff describing how to administer the survey and access results in structured clinical flowsheets and notes; as well as provide ongoing support and troubleshooting.

We also worked directly with the KPNC Quality and Operations Support (QOS) group, which served as a liaison with EHR developers during questionnaire development. In order to have operational leadership support to pilot the questionnaire in the EHR, a business case was developed to justify the use of resources and document how functionality would be affected and improved. Once approved, the research team met weekly with QOS to develop and test the questionnaire itself and refine associated functions and tools. One challenge was identifying a SUD screening instrument that was thorough, brief enough for patients to complete routinely, and technologically feasible. For example, the team needed to select an instrument to accommodate Epic® limitations (e.g., branching capabilities), and then adapt it to be Epic®-compatible.

3.2. Microsoft Access® database

An important initial challenge was how to minimize the burden of TAPS/AOQ administration on clinic workflows. Study staff worked with clinical leads to identify criteria for appropriate patient appointments with which to link a questionnaire (e.g., excluding telephone/video visits, and visits with non-HIV providers) and timing the administration based on a combination of factors (upcoming appointment with participating provider, no questionnaire completed in past 6 months). Even within an integrated health care system such as KPNC, the three individual clinic sites have different appointment codes, adding to the complexity of administering automated screening, and necessitating development of an Access® database for the study that draws on EHR data. By using the tracking system to support to clinicians and clinic staff, the research team reduced the impact on clinic workflows.

3.3. Online HIV reporting tool

The KPNC Division of Research has actively maintained the KPNC HIV registry since 1988 to identify cases of HIV as well as provide data and reports to support clinical case management, quality of care assessments, and research studies. The HIV Registry is updated monthly with newly identified cases, and registry staff maintain a web-based reporting tool used by HIV case managers and clinicians to access up-to-date data on HIV patients at their respective KPNC medical center. The application (internally known as iHIV) includes >80 custom reports, such as lists of HIV patients sorted by provider with information on demographic factors (e.g., age, race, sex) or clinical factors (last CD4 or HIV RNA test results). The research team worked with KPNC HIV registry staff to develop a new report, updated daily, for TAPS/AOQ responses. The report displays summary level patient TAPS/AOQ responses that can be sorted and filtered to better identify patients who require clinical follow-up. These customized reports can then be exported to Excel or other software.

3.4. Institutional Review Board

The study team spent considerable time developing study materials and protocols that were both feasible within the constraints of the EHR functionality and were acceptable to the KPNC Institutional Review Board (IRB). Two issues of concern involved (1) adequate informed consent and HIPAA authorization given delivery of the survey using existing health system tools (i.e., patient portal and clinic tablets) and (2) provisions for any emotional distress that may arise from the screening tool including how to track patients who endorse suicidal ideation. A strong case was made to the IRB that mental health and substance use screening is a central part of HIV care, and that the study innovation was in the method and not the content of screening.

Based on these discussions with the IRB and the submission of additional modifications, a waiver of signed informed consent and signed HIPAA authorization was granted. IRB-approved language was provided both electronically in the patient secure messaging, and on a paper information sheet given to patients along with the tablets, to inform them of the purpose of screening, that completion is voluntary and part of a research study, that clinical staff and researchers would see the responses, and that mental health resources are available to them if they experienced distress.

3.5. Clinician concerns regarding use of screening results

Planning for use of a new instrument to assess SUDs, the TAPS, included provider education about the instrument’s interpretation and implications for clinical follow-up. Providers were instructed that any SUD risk score >0 merits discussion with the patient, consideration of patient presentation and history, and potentially referral to the BHS or specialty addiction care. Additional clinician concerns included limited knowledge about and access to personalized SUD specialty treatment referrals, even though KPNC is an integrated health system where most patients’ coverage includes access to such treatment. When necessary, the study team identified specialty department liaisons and facilitated personalized introductions for BHSs and PACE Champions.

Rollout of the computerized AOQ to the general HIV population also prompted clinician questions. Although the AOQ has been used widely in KPNC for several years, including for HIV patients with new diagnoses of major depression, provider familiarity with the instrument varied. Clinicians were provided guidance about existing KPNC protocols for interpreting AOQ scores and responding with appropriate clinical interventions. Additional physician concerns about expanded use of the AOQ included increase in clinician or staff time needed to review a greater volume of incoming patient responses, especially in cases where the questionnaire is completed up to 2 weeks prior to a visit.

The study team assisted clinicians in identifying self-reported suicidal ideation in the AOQ (any thoughts of self-harm in the prior two weeks). In addition to training on KPNC protocols for interpreting and responding to online AOQ scores, and information about existing ways that the region has addressed liability concerns, the study team offered additional assistance in identifying endorsement of thoughts of self-harm during the study. Specifically, the study team built into the tracking database an alert for suicidal ideation endorsed prior to the visit and a protocol whereby study staff alerted providers via EHR staff message once suicidal ideation was identified (which varies based on EHR and tracking system update schedules and staff hours).

4. Discussion

Previous studies have identified SUD and mental health as substantial problems among PWH that are not often effectively identified and treated in the context of primary care services. The research team developed a novel approach to improve care integration for this clinic population. This hybrid design intervention study is currently in process and evaluates both implementation and effectiveness. Findings will inform the field regarding: 1) how self-administered computerized SUD, depression and anxiety screening and behavioral interventions can be effectively implemented in HIV primary care; 2) which patient and organizational characteristics influence screening and intervention delivery; 3) to what extent implementation of screening and intervention impacts HIV care, SUD, depression and anxiety outcomes; and 4) financial costs of implementation. The study addresses high-priority HIV research topics since effective treatment of SUD and psychiatric disorders would improve retention in care, alleviate HIV treatment disparities, and reduce SUD- and mental health-related medical comorbidity [87].

The study addresses challenging aspects of integrating computerized patient questionnaires into health care settings. The three screening modalities we employ (secure messaging, tablet-based administration in clinic waiting rooms, and desktop computer administration in exam rooms) each have advantages and disadvantages. It often is advantageous to have patients respond to health surveys prior to their visit, e.g., via secure messaging, to give providers clinical information in advance and to maximize efficiency of clinic visits and front desk workflows. Yet these procedures necessitate additional tracking systems and procedures to deliver questionnaires and monitor responses, and patients without internet access may not be reached. The electronic screening tools developed by the research team aim to provide maximum flexibility for the clinics and to reach the largest number of patients either at or before routine visits.

Given the complexity of these challenges, the study team is supporting the clinical sites by using the KPNC HIV registry in combination with appointment databases to identify eligible patients, deliver questionnaires, and monitor patient responses. Although the study team has worked closely with the clinics in the mechanics of the screening process, the team is leaving details regarding utilization of screening results up to clinic staff including clinic leads, clinic champions, and BHS staff. It is not known whether clinics will be able to sustain these same procedures over time, given the staffing commitment that would be required (e.g., dedicated clinic staff that would track upcoming visits, attach questionnaires to visits in the EHR, and monitor responses). However, we anticipate that if the study shows that these systems can be well-integrated into patient care and have a measurable impact on patient outcomes at a manageable cost, this may help motivate a commitment to adapting and sustaining these processes after the study has ended.

4.1. Strengths and Limitations

Given the population of insured HIV patients, generalizability to public systems or resource-constrained environments that do not have access to an EHR or a BHS may be limited. In addition, the population mostly consists of white race/ethnicity and men who have sex with men, although this reflects the demographics of reported AIDS cases in California [88]. Nevertheless, our large cohort size, and selection of clinics with a higher prevalence of minorities and women helps maximize generalizability to groups less represented. Finally, with the ACA and expansion of Medicare, KPNC has seen significant increases in enrollment of PWH [89]. As California’s benchmark ACA Insurance Exchange plan, other health systems are emulating KPNC’s integrated care model, and KPNC’s impact on HIV care and prevention is large [79, 90].

We will compare baseline measures of participants with and without missing data to detect systematic pattern of missingness in TAPS/AOQ completion (including patient demographic characteristics and medical comorbidities, provider demographics and years of experience, and clinic location associated with missingness). We will also test the assumption of missing at random (MAR) using a pattern-mixture modeling approach [91]. To account for such missing data, we will use modern missing data techniques, with emphasis on multiple imputation methods [92–94] that provide a useful umbrella for handling different forms of missing data, including coarse-measured [95] or mis-measured [96] covariates. These will be used to adjust for some covariates in cross-sectional effectiveness outcome comparisons because participants are not randomized, and because some individuals may be part of one cohort but not the other (e.g., due to patients entering or leaving the health plan over the course of the study).

4.2. Conclusion

Identification and treatment of SUDs and psychiatric disorders is a significant challenge for primary care HIV clinics. The PACE study will examine the impact of self-administered electronic screening for SUD risk, depression and anxiety in three large KPNC primary care clinics serving over 5,000 patients with HIV. Based on screening results and physician referrals, behavioral health specialists embedded in primary care initiate MI- and CBT-based brief treatment, and link patients to addiction medicine and psychiatry clinics as needed. Analyses will examine implementation (e.g., screening and treatment rates) and effectiveness outcomes (e.g., SUD, depression and anxiety symptoms; and HIV viral control). We also will evaluate screening and treatment costs and implementation barriers and facilitators. Results will help to inform effective strategies for improving substance use, mental health, and HIV clinical outcomes, as well as broader integration of SUD screening and treatment in primary care settings.