Abstract
Adherence to antiretroviral therapy (ART) is an important determinant of clinical success assessed in many HIV studies. Harmonizing adherence data from studies that use different measures is difficult without a co-calibration equation to convert between validated instruments. Our purpose was to co-calibrate two commonly used adherence measures: the AIDS Clinical Trials Group (ACTG) questionnaire and the Visual Analog Scale (VAS).
We used robust linear regression to develop a co-calibration equation in a clinical care cohort. The outcome was the 30-day VAS percentage of ART taken and the predictors were ACTG questions. We evaluated the equation’s goodness-of-fit in five STTR (Seek, Test, Treat, Retain) consortium studies where individuals completed both measures: 2 criminal justice; 2 international; and 1 other high-risk vulnerable population.
We developed a three-phase decision rule to convert ACTG to VAS in 1045 participants. First, when the last missed dose on the ACTG was reported as >30 days ago the VAS was set to 100% (N=582). Second, if “doses missed” was zero for all items, VAS was 100% (N=104). Third, among remaining participants (N=359), VAS was estimated as 96.8% minus 2.9% times the number of missed doses (“doses per day” was non-significant). Correlation between predicted and reported VAS was r=0.80 in the criminal justice group (N=446), r=0.46 in the international group (N=311), r=0.32 in the other vulnerable population (N=63), and r=0.66 overall. When outliers due to inversion of the VAS scale were excluded (n=25), these correlations were 0.88, 0.78, 0.80, and 0.86, respectively.
We concluded that a simple decision rule and equation allowed us to co-calibrate between two widely used adherence measures thus combining data from studies with different instruments. This study highlighted issues with VAS inversions and its limitations as a single item. Combining studies using different instrument facilitates larger pooled data sets to address key research questions.
Keywords: adherence, HIV, calibration, visual analogue scale, antiretroviral therapy
Introduction
Detecting and addressing sub-optimal adherence to antiretroviral therapy (ART) is a crucial aspect of HIV clinical care. Identifying a widely accepted, easily implemented, standard approach to assessment of ART adherence in HIV care settings and clinical studies has remained challenging(Bangsberg, 2006), leading to the use of multiple instruments across studies.
Harmonizing self-reported ART adherence data from multiple studies that use different measures would allow data to be combined to address questions strengthened by having larger sample sizes. However, harmonizing studies that use different measures is difficult without a co-calibration formula to convert among validated scales. Co-calibration would also facilitate interpretation of findings across studies that used different measures by enhancing our understanding of the properties of different individual measures relative to each other.
Patient self-reported measures remain the most common approach to assessing adherence but vary in their question phrasing, recall periods, and response items(Stirratt et al., 2015). We conducted this study to co-calibrate data from two of the most accepted and widely used approaches to measuring ART adherence among persons living with HIV (PLWH); items from the AIDS Clinical Trials Group (ACTG) questionnaire and the Visual Analog Scale (VAS) adherence measure(Amico et al., 2006; Chesney et al., 2000; Simoni et al., 2006). The primary aim of this study was to co-calibrate ART adherence data among PLWH from multiple studies to the same metric to evaluate the validity of the conversion formula in disparate vulnerable populations.
Methods
Study settings
This study combines data from two sources. The first source is data from five studies within the Seek Test Treat and Retain (STTR) consortium funded by the National Institute of Drug Abuse(Chandler et al., 2015). The 5 studies included in these analyses were selected based on simultaneous collection of both adherence measures (described below) and were grouped into categories based on their study populations: criminal justice (CJ) (Carda-Auten et al., 2014; Gordon, Kinlock, McKenzie, Wilson, & Rich, 2013), international (Tomori et al., 2014; Wechsberg et al., 2014), or other vulnerable populations (VP) (Glenn et al., In press). The second data source is the University of Washington site of the Centers for AIDS Research (CFAR) Network of Integrated Clinical Systems (CNICS)(Kitahata et al., 2008) cohort, a longitudinal observational study of PLWH in primary care. Assessments for this study were completed between 7/2009 and 6/2013.
Adherence measures
We included responses from several Adult AIDS Clinical Trial Group (AACTG) items(Chesney et al., 2000; Simoni et al., 2006), including the last time any HIV medications were missed; whether any ART doses were missed during the previous weekend; and how many doses of ART were missed over the last 14 days. Data were also available from a 30-day VAS adherence item(Amico et al., 2006; Simoni et al., 2006). The VAS was scored as a percentage from 0 to 100%. ART medication data were used to identify the number of doses of ART per day. Study procedures for all sites were approved by local Institutional Review Boards.
Statistical analyses
To create a formula for co-calibration between the two instruments, decision rules were first created using CNICS data. We made decision rules based on the last missed dose item or the 14-day item versus the VAS. The first decision rule was that if the response on the last missed dose item was ≥1 months ago the predicted VAS was set to 100%. The second decision rule was to set the predicted VAS to 100% if the number of missed doses in the last 14 days was zero.
Robust linear regression, which assigns weights to each observation in order to de-emphasize outliers, was then used to fit a VAS prediction equation for the participants who did not fit into the decision rules. The predictors included doses per day and results from the weekend and the 14-day missed dose items. This equation to estimate VAS scores obtained after application of the decision rules is called the co-calibration formula.
The formula was used to predict a VAS score for each person based on their responses to ACTG items. Then the performance of the formula was evaluated by comparing the predicted VAS to the reported VAS. We calculated Pearson’s correlation both with and without outliers, which were defined as those with a predicted and reported VAS which were more than 50% apart.
Data heterogeneity issues make it difficult to combine all of the STTR studies into a single decision rule estimate and so we present results stratified by the context of the population (CJ, international, other VP).
Results
Among 1,045 CNICS participants the mean age was 45 years and 87% were male (see Table 1). Rates of 100% VAS adherence varied across groups categorized on the basis of the AACTG item assessing timing of the last missed dose (data not shown). We set predicted VAS to 100% for those who responded that their last missed dose was 1 or more months ago (N=582), or that the number of missed doses in the last 14 days was zero (N=104).
Table 1.
Description of demographic characteristics and adherence responses from participants in the CNICS and STTR groups
| CNICS | CJ | International | Other VP | |
|---|---|---|---|---|
| Number of participants | 1045 | 446 | 311 | 63 |
| N (%) | N (%) | N (%) | N (%) | |
| Age | ||||
| <30 | 77 (7%) | 46 (10%) | 49 (16%) | 1 (2%) |
| 30–39 | 222 (21%) | 112 (25%) | 194 (62%) | 4 (6%) |
| 40–49 | 408 (39%) | 179 (40%) | 62 (20%) | 29 (46%) |
| ≥50 | 338 (32%) | 109 (24%) | 6 (2%) | 29 (46%) |
| Sex | ||||
| Male | 905 (87%) | 341 (76%) | 232 (75%) | 34 (54%) |
| Female | 136 (13%) | 95 (21%) | 79 (25%) | 29 (46%) |
| Transgender | 4 (0%) | 9 (2%) | 0 (0%) | 0 (0%) |
| Unknown | 0 (0%) | 1 (0%) | 0 (0%) | 0 (0%) |
| Adherence % (VAS) | ||||
| <75 | 66 (6%) | 51 (11%) | 16 (5%) | 25 (40%) |
| 75–84 | 52 (5%) | 27 (6%) | 1 (0%) | 6 (10%) |
| 85–94 | 182 (17%) | 73 (16%) | 6 (2%) | 13 (21%) |
| ≥95 | 745 (71%) | 295 (66%) | 288 (93%) | 19 (30%) |
| Missed Doses (14 days) | ||||
| 0 | 667 (64%) | 278 (62%) | 283 (91%) | 30 (48%) |
| 1 to 5 | 341 (33%) | 86 (19%) | 22 (7%) | 12 (19%) |
| 6 to 10 | 23 (2%) | 38 (9%) | 1 (0%) | 7 (11%) |
| ≥11 | 14 (1%) | 44 (10%) | 4 (1%) | 14 (22%) |
| When ART last missed | ||||
| in the past week | 191 (18%) | 125 (28%) | 18 (6%) | 28 (44%) |
| 1–2 weeks ago | 154 (15%) | 40 (9%) | 6 (2%) | 13 (21%) |
| 2–4 weeks ago | 118 (11%) | 26 (6%) | 11 (4%) | 3 (5%) |
| 1–3 months ago | 140 (13%) | 58 (13%) | 13 (4%) | 6 (10%) |
| more than 3 months ago | 132 (13%) | 85 (19%) | 37 (12%) | 4 (6%) |
| never skip medications | 310 (30%) | 112 (25%) | 226 (73%) | 9 (14%) |
ART: antiretroviral therapy; CJ: criminal justice population; CNICS: Centers for AIDS Research Network of Integrated Clinical Systems; VAS: visual analogue scale; VP: vulnerable population
When we fit a robust linear regression line to predict VAS among the 359 remaining people, the coefficients for the missed dose last weekend item and doses/day were not statistically significant and so these were dropped as predictors. The robust linear regression using only the 14-day item had an r2 of 0.17, and estimated VAS as:
| ( Table 2). |
Table 2.
Robust linear regression of VAS by missed doses results from CNICS and STTR CJ cohorts.
| Missed doses/2 weeks | Intercept | |||||
|---|---|---|---|---|---|---|
| N | N (weight <0.1) | Coefficient | 95% CI | Coefficient | 95% CI | |
| CNICS | 359 | 35 | −2.86 | (−3.09, −2.64) | 96.83 | (96.02,97.64) |
| CJ | 167 | 14 | −2.07 | (−2.18, −1.95) | 99.27 | (97.26,101.28) |
CJ: criminal justice population; CNICS: Centers for AIDS Research Network of Integrated Clinical Systems; VAS: visual analogue scale
Finally, the predicted VAS was set to zero if the equation predicted a value of less than zero to preserve the correct range for the VAS.
Applying the co-calibration formula to all 1045 in the CNICS sample, the correlation between predicted and reported VAS was 0.61 (Table 3). There were 20 outliers (2%), and without these outliers the correlation was 0.73. Many of the outliers were people who apparently inverted the VAS scale and reported 0–1% on the VAS despite also reporting “never skipping” medications (data not shown).
Table 3.
Performance of the co-calibration equation in the CNICS and STTR cohorts as measured by correlation of the predicted and reported VAS with and without outliers.
| Correlation (All) | Number of Outliers (%) | Correlation (Without Outliers) | |
|---|---|---|---|
| CNICS | 0.61 | 20 (2%) | 0.73 |
| CJ | 0.80 | 8 (2%) | 0.88 |
| International | 0.46 | 5 (2%) | 0.78 |
| Other VP | 0.32 | 12 (19%) | 0.80 |
CJ: criminal justice population; CNICS: Centers for AIDS Research Network of Integrated Clinical Systems; VAS: visual analogue scale; VP: vulnerable population
The STTR validation sample included 446 people in the CJ group, 311 people in the international group, and 63 people in the other VP group. The age distribution is similar for CNICS and CJ, while the international group is younger and the other VP group is older (Table 1).
The correlation between predicted and reported VAS for STTR overall was 0.66, and was 0.81 for CJ, 0.46 for international, and 0.32 for other VP (Table 3). The percentage of outliers was 9% overall, 2% for CJ and international groups, and 19% for other VP. Removing the outliers increased the correlations overall and for CJ to 0.86 and 0.88 respectively, and more for international and other VP groups to 0.78 and 0.80 respectively.
For comparison, the robust linear regression model which predicts the reported VAS with the 14-day ACTG item was fit in the STTR CJ group (r2 of 0.27). The number of missed doses coefficient was 0.8% smaller than in CNICS at −2.1% per missed dose, while the intercept was 2.8% higher at 99.5% (Table 2).
Discussion
This study demonstrates the feasibility of generating an equation to co-calibrate ART adherence instruments across diverse patient populations, thereby allowing adherence data from multiple studies with different approaches to adherence measurement to be combined and compared to answer questions requiring larger sample sizes. This study also demonstrates the negative impact of outliers and particularly individuals who appear to have inverted the VAS scale with the greatest impact among the international and other VP groups.
There has been an increasing push to harmonize data sources to address important research questions. By co-calibrating just a small number of select instruments or items, domains for many STTR studies can be on the same metric facilitating stronger methodological approaches to combining data across studies.
This study is also an important example of some of the limitations and barriers to co-calibration. The similarity of data can be compromised even when the same or similar measures are used. When combining data sets from across the world, these differences can be magnified for reasons including differences in language, culture, and method of administration. We found similar co-calibration performance across the different groups but, as described above, this was the case only when the outliers were excluded. This impact raises questions as to the appropriateness of the VAS as a single item of adherence.
Strengths and Limitations
This study included PLWH from six diverse studies including those in routine clinical care, treated and untreated drug users, and individuals in prison, on parole, or on probation. However, the sample size for the international and other VP studies were smaller than ideal thus limiting the strength of the findings. In addition, studies and sites used different approaches to collect the adherence information.
This study calibrated two adherence measures that differed in timeframe (e.g. 14 vs. 30 day items) which may have contributed to discrepancies between the measures.
Conclusion
A simple decision rule and equation allowed us to co-calibrate between two widely used measures of ART adherence among groups of PLWH and allowed for these measures to be placed on the same scale. Effective co-calibration allows results across studies that used different instruments to be combined to facilitate the development of larger pooled data sets to address high-priority research questions.
Acknowledgments
Research presented in this paper is the result of secondary data analysis and was supported by 5U01DA037702 from the National Institute on Drug Abuse (NIDA) of the National Institutes of Health. Primary data collection for STTR studies was supported by grants:
5R01DA030768, 5R01DA030747, 5R01DA030781, 5R01DA030771, 5R01MH094090, 5R01DA030796, 5R01DA030778, 1R01DA030766, 7R01DA030770, 5R01DA030762, 5R01DA030776, 5R01DA030793, 1R01DA032059, 1R01DA032083, 1R01DA032106, 1R01DA032061, 1R01DA032110, 1R01DA032080, 1R01DA032082, 5R01DA032057, 1R01DA032098, 1R01DA032100.
This work was also supported by the University of Washington Center for AIDS Research NIAID Grant (AI-27757) and the Centers for AIDS Research Network of Integrated Clinical Systems (CNICS) grant (AI-067039).
The authors thank the other investigators, the staff, and particularly the participants of the individual STTR studies for their valuable contributions. A full list of participating STTR investigators and institutions can be found at http://www.sttr-hiv.org.
Footnotes
The authors have no relevant financial conflicts or benefits that arise from application of this research to disclose.
Findings have been presented in part at the 9th International Conference on HIV Treatment and Prevention Adherence
Contributor Information
Robin Nance, University of Washington, Harborview Medical Center, 325 9th Ave, Box 359931, Seattle, WA 98106, 206-616-5234.
J.A. Chris Delaney, University of Washington, Sandpoint Magnuson Park, 6200 NE 74th Street, Suite 210, Box 354922, Seattle, WA 98115, 206-897-1918.
Carol Golin, University of North Carolina at Chapel Hill, Sheps Office number: 919-966-7939, Fax: 919-966-5764.
Wendee Wechsberg, Director, RTI Global Gender Center, RTI International, 3040 Cornwallis Road, RTP, NC 27709-2194, 919-541-6422.
Chinazo Cunningham, Associate Chief, Division of General Internal Medicine Director, Albert Einstein College of Medicine/Montefiore Medical Center, 111 E. 210th Street Bronx, NY 10467, 718-920-5971, Fax: 718-561-5165.
Frederick Altice, Yale University School of Medicine, 135 College Street, Suite 323, New Haven, CT 06510-2283, 1-203-737-2883.
Katerina Christopoulos, University of California, San Francisco, 1001 Potrero Ave, SFGH 80, San Francisco CA 94110, 415-476-4082.
Kevin Knight, Associate Director for CJ Studies, Institute of Behavioral Research, Texas Christian University, Box 298740, Fort Worth, TX 76129.
Vu Quan, Johns Hopkins Bloomberg School of Public Health, Department of Epidemiology, 615 North Wolfe Street, W6508, Baltimore, MD 21205.
Michael Gordon, Friends Research Institute, Inc., 1040 Park Avenue, Suite 103, Baltimore, MD 21201, 410-837-3977 ext.251, Fax: 410-752-4218.
Sandra Springer, Yale School of Medicine, 135 College Street, Suite 323, New Haven, CT 06510, 203-500-4057.
Jeremy Young, University of Illinois at Chicago, 808 S Wood Street #888, Chicago, Illinois, 312-413-0579.
Paul K Crane, University of Washington, Harborview Medical Center, 325 9th Ave, Box 359680, Seattle, WA 98106, 206-744-1831.
Kenneth H Mayer, The Fenway Institute, 7 Haviland Street, Boston, MA, 02115, 617-267-0900.
Michael J Mugavero, University of Alabama, Birmingham, 908 20th St S Ste 250, Birmingham, AL 35205, 205-934-1917.
Carlos Del Rio, Hubert Professor and Chair at Emory University, Emory / Rollins School of Public Health, 1518 Clifton Rd, Atlanta, GA 30322, 404-727-1557.
Richard Kronmal, University of Washington, Sandpoint Magnuson Park, 6200 NE 74th Street, Suite 210, Box 354922, Seattle, WA 98115, 206-897-1918.
Heidi Crane, University of Washington, Harborview Medical Center, 325 9th Ave, Box 359931, Seattle, WA 98106, 206-744-6649.
References
- Amico KR, Fisher WA, Cornman DH, Shuper PA, Redding CG, Konkle-Parker DJ, … Fisher JD. Visual analog scale of ART adherence: association with 3-day self-report and adherence barriers. J Acquir Immune Defic Syndr. 2006;42(4):455–459. doi: 10.1097/01.qai.0000225020.73760.c2. Retrieved from http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation&list_uids=16810111. [DOI] [PubMed] [Google Scholar]
- Bangsberg DR. Monitoring adherence to HIV antiretroviral therapy in routine clinical practice: the past, the present, and the future. AIDS Behav. 2006;10(3):249–251. doi: 10.1007/s10461-006-9121-7. Retrieved from http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation&list_uids=16783537. [DOI] [PubMed] [Google Scholar]
- Carda-Auten J, Golin C, Groves J, Gould M, Flynn P, Knight K, Wohl D. Association between engagement in substance abuse and mental health services among recently incarcerated HIV-positive individuals and medication adherence: Early findings from the imPACT (Individuals Motivating to Participate in Adherence, Care, and Treatment) Trial. Paper presented at the 7th Academic and Health Policy Conference on Correctional Health; Houston, TX. 2014. [Google Scholar]
- Chandler RK, Kahana SY, Fletcher B, Jones D, Finger MS, Aklin WM, … Webb C. Data Collection and Harmonization in HIV Research: The Seek, Test, Treat, and Retain Initiative at the National Institute on Drug Abuse. Am J Public Health. 2015;105(12):2416–2422. doi: 10.2105/AJPH.2015.302788. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Chesney MA, Ickovics JR, Chambers DB, Gifford AL, Neidig J, Zwickl B, Wu AW. Self-reported adherence to antiretroviral medications among participants in HIV clinical trials: the AACTG adherence instruments. Patient Care Committee & Adherence Working Group of the Outcomes Committee of the Adult AIDS Clinical Trials Group (AACTG) AIDS Care. 2000;12(3):255–266. doi: 10.1080/09540120050042891. Retrieved from http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation&list_uids=10928201. [DOI] [PubMed] [Google Scholar]
- Glenn M, Sohler N, Starrels J, Maradiaga J, Jost J, Arnsten J, Cunningham C. Characteristics of methadone maintenance treatment patients prescribed opioid analgesics. Subst Abuse. doi: 10.1080/08897077.2015.1135225. In press. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Gordon MS, Kinlock TW, McKenzie M, Wilson ME, Rich JD. Rapid HIV testing for individuals on probation/parole: outcomes of an intervention trial. AIDS Behav. 2013;17(6):2022–2030. doi: 10.1007/s10461-013-0456-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kitahata MM, Rodriguez B, Haubrich R, Boswell S, Mathews WC, Lederman MM, … Saag MS. Cohort profile: the Centers for AIDS Research Network of Integrated Clinical Systems. Int J Epidemiol. 2008;37(5):948–955. doi: 10.1093/ije/dym231. Retrieved from http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation&list_uids=18263650. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Simoni JM, Kurth AE, Pearson CR, Pantalone DW, Merrill JO, Frick PA. Self-report measures of antiretroviral therapy adherence: a review with recommendations for HIV research and clinical management. AIDS Behav. 2006;10(3):227–245. doi: 10.1007/s10461-006-9078-6. Retrieved from http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation&list_uids=16783535. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Stirratt MJ, Dunbar-Jacob J, Crane HM, Simoni JM, Czajkowski S, Hilliard ME, … Nilsen WJ. Self-report measures of medication adherence behavior: recommendations on optimal use. Transl Behav Med. 2015;5(4):470–482. doi: 10.1007/s13142-015-0315-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Tomori C, Go VF, Tuan le N, Huong NM, Binh NT, Zelaya CE, … Quan VM. “In their perception we are addicts”: social vulnerabilities and sources of support for men released from drug treatment centers in Vietnam. Int J Drug Policy. 2014;25(5):897–904. doi: 10.1016/j.drugpo.2014.04.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wechsberg WM, Zule WA, Ndirangu J, Kline TL, Rodman NF, Doherty IA, van der Horst CM. The biobehavioral Women’s Health CoOp in Pretoria, South Africa: study protocol for a cluster-randomized design. BMC Public Health. 2014;14:1074. doi: 10.1186/1471-2458-14-1074. [DOI] [PMC free article] [PubMed] [Google Scholar]