Table 5.
Domain | Quantitative results | Qualitative results | Findings were concurrent, complementary, or discrepant, and comments |
---|---|---|---|
Feasibility | Components and assessments were feasible Having participants obtain their own lab reports from HIV care settings was feasible but challenging (e.g., because participants lacked smartphones and had lower levels of technical abilities). This was complicated by the COVID-19 pandemic. These challenges reduced the number of lab reports provided for analysis. | Compensation for study visits was the main reason participants initially enrolled in the study, but they continued in the study at high rates mainly because of their positive experiences in the study. Compensation overall led participants to feel respected and valued, and this, in turn, promoted retention and engagement. |
Qualitative and quantitative results were complementary. Compensation is likely necessary but not sufficient for retention and engagement. The quality of the participant experience drives feasibility. The ClinCard system we used to provide compensation virtually and quickly likely played a role in participants' positive study experiences. We recommend similar studies support participants in obtaining lab reports, and if possible, provide multiple ways for them to do so with minimum burden and hassle, along with cell phones. |
Acceptability overall | The study was acceptable to participants overall in a number of respects (e.g., information was helpful, privacy was respected, needs pf racial/ethnic group were understood). Overall acceptability dropped from 82 to 69% at the second FU. Almost all noted they would continue to take HIV medication after the study ended. | Project was acceptable overall. Qualitative results rarely highlight negative or unacceptable experiences in the study. |
Lower rates of acceptability at the second FU suggest some participants may not have gotten their needs met in the study. Qualitative results may over-estimate acceptability since those with less positive experiences may decline to be interviewed or to comment. |
Evidence of efficacy on viral load and suppression | Viral load levels decreased, and rates of viral suppression increased at FU1 and FU2. This suggests some or all of the components may be “active.” | Some participants discussed ways the project fostered the desire and ability to take HIV medication at higher levels and achieve HIV viral suppression, as well as barriers they experienced. Not all participants wished to achieve viral suppression at this time. | Findings were congruent, with qualitative results perhaps presenting a more favorable view of the effects of the components compared to quantitative findings. |
Financial rewards (overall) | NA (see below) | We found both fixed compensation and lottery prizes enhanced positive experiences related to study participation, were not seen as coercive or pressuring, and were seen as a form of encouragement to achieve viral suppression, but not necessarily a primary motivating factor or reason in and of themselves to achieve viral suppression. There were three themes: the financial reward was sufficient to motivate changes in HIV medication adherence behavior, the reward was appreciated but not necessarily a primary motivating factor or reason to achieve viral suppression, or it increased desire to achieve viral suppression but this was not sufficient to overcome serious structural and individual-level barriers to HIV medication adherence. Both levels of this component are acceptable, and feasible if participants provide lab reports and present for FU. |
Financial rewards require relatively less emotional and cognitive effort for participants compared to counseling components. Financial rewards require relatively less effort for staff compared to counseling components. Financial rewards for viral suppression hold promise but the type (fixed vs. lottery), amount, and timing warrant further study. |
Lottery prize | Feasibility was high since the component is not labor intensive to administrate. At FU1, 62% said the prize prompted them to achieve HIV undetectable viral load, dropping to 43% at FU2. Almost all intended to continue to take medication after the prize was received. Approximately 20% achieved HIV viral suppression at FU1 and 40% at FU2. CIs indicate the lottery prize was the most promising of the component levels with respect to reducing HIV viral load. | The lottery prize appeared more interesting, memorable, and engaging than fixed compensation, consistent with behavioral economic theory. See above re: themes related to the effects of financial rewards on behavior. |
Quantitative and qualitative findings are largely congruent: prizes can encourage or support behavior but may not be a primary motivator. It is possible the effects of prizes operate largely outside conscious awareness, consistent with behavioral economic theory. Qualitative results provide insights into the ways the prize was seen and its effects. Taken together, findings suggest the lottery prize may be more promising than fixed compensation for viral suppression, consistent with behavioral economic theory. |
Fixed compensation | Equivalent to findings for lottery prize with respect to acceptability and feasibility. Approximately 30% achieved HIV viral suppression at FU1 and 25% at FU2. | See above section on financial rewards. | See above |
TMQQ | Feasibility was high: 82% answered at least one QQ and on average responded to 11 of 21 questions. Acceptability was modest (< 70%). Approximately 25% achieved HIV viral suppression at FU1 and 30% at FU2, with no difference between those who received TMQQ or not. In addition to influencing viral suppression, this component was intended to foster engagement in the study. Quantitative results suggest this component had the smallest effect on reducing HIV viral load, but may be useful as a low-touch engagement tool. | TMs were seen as informational, but the information provided was quite basic. Some were frustrated by how easy the QQs were. But, participants reported it “felt good” to get the answers correct. Frequency of TMQQs was acceptable and more frequent TMQQs would be acceptable and feasible. TMs were not necessarily connected to motivation to take HIV medications. TMQQs became reminders to take HIV medication that day for some. The information about HIV was found to be interesting and useful by most. TMQQs were commonly experienced as a form of positive connection with the study. TMQQs were experienced as an aspect of the participant's overall, generally positive, relationship with the study. |
Qualitative findings add rich description and context to understanding this component, which was intended in part to foster engagement in the study in the period during which participants might seek to achieve viral suppression. Reminders of HIV status and HIV medication can induce negative feelings. The TMQQs did not appear to do so. Thus, providing short, easy, and private intervention content that does not directly refer to the need to take HIV medication that that yielded compensation may induce positive emotions, or at least no negative emotions. This may be valuable and may support study engagement over time. Findings suggest this component is promising and warrants further study, including regarding more challenging questions and/or individualized messages, and more frequent messages. The TMQQ component requires relatively less emotional and cognitive effort for participants compared to counseling components. Quantitative questions may need refinement to better assess perspectives on this component. |
MI counseling sessions | The component was feasible: 95% completed all three sessions, despite sessions being virtual and problems with phones being common. It was also highly acceptable and participants reported it influenced their decisions to take HIV medication (Table 3). Approximately 25% achieved HIV viral suppression at FU1 and 33% at FU2, with no large difference between those who received the component or not. | Highly acceptable and associated with insights and various types of behavior change, such as substance use challenges. It is not clear whether the habit-formation aspect of this component was useful. |
Few MI interventions have been carried out virtually (on the phone, not a Voice over Internet Protocol) and during COVID-19. Participants had very high levels of motivation at entry into the study, which may have reduced the need for this component. It is possible serious structural and individual-level barriers to HIV medication adherence including those related to COVID-19 impeded behavior change even when sessions were provided. |
Motivation (mediator of component effects) | Motivation for HIV viral suppression is high at study enrollment (~90/100). | Motivation was assessed indirectly in the qualitative results (e.g., goals for viral suppression). Some components increased motivation for viral suppression, as noted above, but in many cases, motivation was not sufficient to overcome serious structural and individual-level barriers to HIV medication adherence. | We cannot explain with precision why motivation for HIV viral suppression is very high at enrollment (although it may be related to COVID-19), but participants were not virally suppressed (despite taking HIV medication in some cases) and many did not achieve viral suppression during the study. There is a large literature on multi-level barriers to viral suppression, but we do not yet understand participants' perspectives on this phenomenon. |
Diverting (selling) HIV medication | Not assessed | Diverting HIV medications is common and a power structural impediment to achieving HIV viral suppression for some. | Pharmacies eliciting illegal medication diversion were a serious barrier to HIV management for some. |
Effects of COVID-19 | Not assessed | COVID-19 overall created impediments to HIV management but also increased motivation to manage HIV in some cases. | Understanding COVID-19 as a contextual factor was vital to interpreting study findings. |
How to improve procedures and components | Some measures may need refinement for more precise estimates of effects. | Various improvements to components and study procedures (regarding laboratory reports) were identified. | Having both quantitative and qualitative data and integrating results was useful. Quantitative data captured experiences of participants as a whole and qualitative data provided richness, detail, and context but seemed skewed toward more positive experiences with the study. Quantitative data required us to not over-estimate the positive aspects of participants' experiences. |