The Brief Human Immunodeficiency Virus (HIV) Index: A Rapid 3-Item Scale to Measure Engagement in HIV Care

Abstract

We created a brief version of The Index, a validated patient-reported measure that has potential to quickly identify patients at risk for poor retention. We analyzed Index scores from 2406 patients from 2016 to 2017 in a national cohort of patients in human immunodeficiency virus (HIV) care. Index scores predicted poor retention 12 months after administered.

Operational definitions of retention in human immunodeficiency virus (HIV) care are designed to capture engagement with HIV care systems, often via patterns of appointment attendance or viral load (VL) testing frequency (eg, ≥2 VLs taken within 6 months) [1, 2]. These definitions can be a clinically useful indicator with moderate predictive validity for future HIV treatment success [2, 3]. However, these definitions require abstraction of appointment records and VL data and decision rules/algorithms for defining poor versus good retention. Thus, a brief screening tool that does not require medical record data and instead focuses on a person's sense of connection to HIV care could help identify patients at risk for poor retention. The tracking and promotion of retention remains a critical strategy to end the US HIV epidemic, which we argue is a product of engagement with HIV care [4].

Whereas retention data mark an interaction with HIV care, engagement is a distinct construct defined as “the ongoing interaction of patients, their providers, and care settings that is characterized by a patient's sense of connection to and active participation in care.” [5]. Measuring engagement in this way can be a useful strategy to identify patients more likely to miss visits and benefit from additional retention support.

The engagement construct has been supported by qualitative, quantitative and consensus-building studies [5, 6]. These studies resulted in a 10-item measure called The Index of HIV Care Engagement [5], which has robust psychometric properties and measurement invariance (ie, equivalence in its performance in Black, Latino, and non-Latino White patients) [5, 7]. It was also validated prospectively, predicting retention and viral suppression 12 months after administered in several US clinics [8].

This brief report describes the empirical development of a 3-item version of the Index. This report leverages the original Index data collected from a US-based, multi-site cohort of people with HIV (PWH) in care published in Clinical Infectious Diseases in 2021 [8].

METHODS

Study Setting and Population

Data were from the Center for AIDS Research Network of Integrated Clinical Systems (CNICS), an ongoing cohort study of adult PWH in HIV care in 7 cities located across the United States (Table 1). CNICS has a centralized data repository of longitudinal information from electronic medical records (EMRs). Patient-reported outcomes are collected every 4–6 months and submitted to the centralized data repository. The study was exempt from review by the University of California, San Francisco's Institutional Review Board.

Table 1.

Sample Characteristics

Characteristics
Age (N, %)
18–29	457 (17.1)
30–39	757 (28.3)
40–49	907 (33.8)
>50	559 (20.9)
Contributing sitesa (n, %)
San Diego	519 (19.4)
Birmingham	589 (22.0)
Seattle	511 (19.1)
Boston	322 (12.0)
Baltimore	568 (21.2)
San Francisco	171 (6.4)
Race/ethnicity (n, %)
White	1071 (39.9)
Black	1201 (44.8)
Latinx	268 (10.0)
Other	131 (4.8)
Not reported	9 (0.50)
Index (M, SD)b
10-item version	4.50 (0.60)
3-Item Version	4.44 (0.64)
Median HIV care visits a year after Index (IQR)	4 (2–6)

Abbreviations: HIV, human immunodeficiency virus; IQR, interquartile range; M, mean; SD, standard deviation.

^aHIV care sites are affiliated with the University of California, San Diego, University of Alabama Birmingham, University of Washington, Fenway Health, Johns Hopkins University, University of California, San Francisco.

The original data set was created with 2680 patients who reported retention data and responded to at least 75% of the 10 Index items. Our dominance analysis required responses to all 10 items and was reduced to 2406 patients.

Participants were: (1) living with HIV, (2) 18 years of age or older, and (3) had at least 2 prior HIV primary care visits in 1 year at a CNICS site [8]. Index data were collected in English in CNICS from April 2016 to March 2017 prior to appointments on a tablet/computer connected to an encrypted internet network (A Spanish version is now available—See Supplementary Appendix).

Predictor

The Index has 10 items (eg, How often do you leave your HIV care appointment feeling like you got really good care? See Supplementary Appendix) [5]. The response scale options range from 1 to 5 (eg, 1 = Not at all to 5 = A great deal). Total scores are the summed items divided by the number of items. divided by 10. The Index had a sensitivity of .55 (95% confidence interval [CI], .48–.62) and specificity of 0.57 (.50–.67) [8]. Positive and negative predictive values were .36 (.33–.37) and .75 (.75–.76) [8]. These results generated a mean cutoff score of 4.7 (from 1 to 5) as “positive” test result for sub-optimal engagement.

Outcomes

Retention was operationalized as continuity-based EMR measure: (0) not keeping or (1) keeping 75% of scheduled primary care appointments in the 12 months after Index administration [8].

Analyses

We ran a dominance analysis in STATA 17. Dominance analysis is a regression-based method for comparing and ranking a set of items (ie, predictors) based on their relative importance to an outcome [9, 10]. Effect sizes show the relative contribution of 1 item over another item (Item 1 vs Item 2, Item 1 vs Item 3, etc.) in the prediction of an outcome (retention), and in the context of every subset of items (eg, Item 1 vs Item 2 with Item 3 present in the model, and so on) [8]. A standardized dominance weight is reported for each item as the average standardized squared semi-partial correlation. Items are then ranked based on these weights. The analysis also yields patterns of dominance (complete, conditional, general). One item can have an average effect size larger than another item across all comparisons, but it can also always have a larger effect size in every comparison. Items are said to show a complete dominance pattern when they always dominate other items. Lastly, as a predicative validity test to ensure the Brief Index also associated with retention as did the 10-item Index [8], we ran a single unadjusted logistic regression.

RESULTS

The sample had 2608 patients with retention and Index data (at least 75% of Index items responded to) (Table 1) [8], which was reduced to 2406 with complete Index data. In total, 29.6% of patients were not retained the year after the Index was administered, whereas70.4% were (Table 1). The median number of visits the year following Index was 4 (interquartile range [IQR], 2–6).

Dominance analysis ran 1023 regressions. Dominance patterns showed that 3 items completely dominated the other 7 items: (1) How well do you follow through on your HIV care when things in your life get tough? (dominance weight = .64); (2) How open do you feel you can be with your HIV care provider? (dominance weight = .09); and (3) How often do you leave your HIV care appointment feeling like you got really good care? (dominance weight = .08). Cronbach's alpha was.61 (the 10-item version was.88). The mean 3-item mean was 4.44 (SD = .64, range 1–5), nearly identical to the 10-item mean (M = 4.5, SD = .60), and it correlated strongly with the 10-item mean (r = .89, P < .001). Lastly, higher 3-item Index scores were associated with an odds ratio of 1.78 for being retained in HIV care (95% CI: 1.56–2.03, P < .001).

DISCUSSION

A 3-item version of the previously validated Index was able to identify patients with subsequent non-retention in HIV care. We empirically reduced the number of items from 10 to 3 to expand its application without compromising its performance.

Interestingly, the face validity of the 3 most dominant items related to 3 different domains of engagement. How well do you follow through on your HIV care when things in your life get tough may be viewed as a patient resiliency domain, whereas how open do you feel you can be with your HIV care provider captures an element of the patient-provider relationship and how often do you leave your HIV care appointment feeling like you got really good care covers a general sense of the clinic experience. Indeed, the formative work targeted multiple dimensions of the engagement process and construct [5–8]. As expected, the predicative validity test showed that the 3-item Index mean associated with retention, which has been observed in prior research [5, 7, 8].

Despite intensive efforts in gathering electronic clinical and laboratory data, predicting retention with common appointment attendance and VL data has its challenges [10]. For example, VL data in low-income settings may underestimate retention due to economic constraints and laboratory capacity to collect VL data, whereas high-income settings with little barriers to collection of VL may overestimate retention [11]. Relatedly, there is heterogeneity in the conceptualization of engagement as it is a multi-dimensional construct, and because highly self-managed PWH have better adherence and viral suppression, the construct of engagement to predict retention has been underappreciated [12, 13].

The Brief Index requires little training and can be administered prior to HIV care appointments once per year. For clinical use, lower scores can be used to initiate conversations about engagement. In research, scores can be used as outcome of engagement or retention interventions. Finally, it may be generally applied in other settings where HIV care outcomes require patient connection and participation in their HIV care, but its use in international setting, and study of the engagement construct, warrants further research.

Limitations include a highly retained population in academic medical centers that may be unique from other clinic settings. Appointment attendance was the only clinical data used (no other laboratory, pharmacy, or mortality data was used), and all data were collected before coronavirus disease 2019 (COVID-19) and use of telehealth visits. A limitation of the 3- versus 10-item Index was a reduction the reliability estimate, but this is expected as Cronbach's alpha uses the number of items and total variance as inputs that make high reliability difficult to achieve with very few items. Finally, engagement is a dynamic process and may change over time. It is likely that engagement changes in patients who are just establishing a connection to HIV care. In sum, the Brief Index is a theoretically driven measure that captures engagement levels and has the potential to be used to allocate resources for faster retention support.

Supplementary Data

Supplementary materials are available at Clinical Infectious Diseases online. Consisting of data provided by the authors to benefit the reader, the posted materials are not copyedited and are the sole responsibility of the authors, so questions or comments should be addressed to the corresponding author.

Notes

Acknowledgment. The authors would like to thank the patients who participated in this study for their time and contributions.

Financial support. This work was supported by the following grants from the National Institutes of Health: R01MH102198 (to K. A. C. and M. O. J.), R24AI067039, P30AI027763, P30MH062246 (to M. O. J.), K24DA037034 (to M. O. J.), K24AI167805 (to K. A. C.) K01MH113475 (to J. A. S.), P30AI027767 (to M. J. M.), P30AI094189 (to R. D. M.), U01DA036935 (to R. D. M.), T32 MH19105-34 (Kirschstein National Research Service Award to N. L. B.).