Evaluation of a Computer-Based HIV Education Program for Adults Living with HIV

Abstract

This study evaluated a computer-delivered HIV and antiretroviral treatment education program in adults (N=102) living with detectable HIV viral loads (>200 copies/mL). The self-paced program provided immediate feedback for responses and financial incentives for responding correctly. The program was divided into three courses and a test of content from all three courses was delivered before and after participants completed each course. Test scores on the content delivered in Courses 1, 2 and 3 improved only after participants completed training on the relevant course. Initial test scores were positively correlated with health literacy and academic achievement; were negatively correlated with viral load; and were lowest for participants living in poverty. Education, academic achievement, and health literacy were related to how much participants learned following each course. Computer-based education is a convenient, effective approach to promoting an understanding of HIV and its treatment.

INTRODUCTION

The Panel on Antiretroviral Guidelines for Adults and Adolescents (1) and the World Health Organization (2) recommend that all persons living with HIV initiate medications collectively known as antiretroviral therapy (ART). If people living with HIV maintain consistent use of ART, they can experience a reduction in the amount of HIV in their blood (i.e., viral load) to undetectable levels. Achieving viral suppression leads to increased survival (3,4), reduced health care costs (5), and reduced HIV transmission (6–8). Despite these benefits and efforts to expand access to ART, some people living with HIV are not reliably sustained on ART and do not maintain a suppressed viral load (9,10). In Baltimore, Maryland, where this research was conducted, only about 40% of people living with HIV take antiretroviral medications and achieve an undetectable level of viral suppression (11). Adherence to the prescribed ART regimen is required for viral suppression and associated benefits (1). Poor adherence is most commonly the reason for ART treatment failure.

Interventions to reduce pill burden (12), incentivize engaging in care (13), observe therapy (14), and prompt use of medications (15) have been designed to improve adherence to ART in people living with HIV. Thompson and colleagues reviewed the literature and made a strong recommendation that interventions to improve ART adherence include education and counseling (16). Education might be especially important because studies have found that people living with HIV have limited knowledge about the disease and its treatment. For example, an early study (17) measured health literacy and knowledge about HIV and ART and found that many people, regardless of health literacy did not understand the meaning of their HIV viral load test result or their CD4 cell count, a measure commonly used by clinicians to estimate immune function. More recent studies assessing HIV and ART-related knowledge in China (18) and India (19) found that many people living with HIV had misconceptions about ART dosing regimens and the need for daily adherence, and even lacked knowledge about how HIV is transmitted. Across the studies, this limited knowledge about HIV and ART was associated with poorer ART adherence. Teaching people with HIV about CD4, viral load, and how ART affects these measures may improve adherence, but it may also help them to communicate with healthcare providers, a factor related to patient retention in standard care (20).

Patient education is often delivered by healthcare providers. For example, the Ciel Bleu study (21) evaluated effects of healthcare provider-delivered education on HIV knowledge and ART adherence. Adults living with HIV were randomly assigned to an education group who received standard care, a planning card, and personalized education about HIV and its treatment; or a control group who received standard care and a planning card, but were placed on a waitlist for the education intervention. The education intervention involved three, one-hour-long sessions that were scheduled over one year. At the end of the year, the education group had greater improvement in their knowledge of HIV and its treatment and reported greater ART adherence on a questionnaire relative to the control group. Although the intervention was effective, three hours of education delivered by physicians and nurses may not be feasible in many settings. To promote implementation of HIV education, Brock and Smith (22) evaluated an interactive educational intervention that was displayed on a personal digital assistant (PDA) provided to patients at an outpatient care facility. The intervention led to improvements in patients’ knowledge about HIV and medications relative to baseline knowledge, patients reported that the intervention was valuable and easy to follow, and self-reported ART adherence increased following the intervention.

Harnessing technology to help people learn about HIV and ART is beneficial because programs can be taken at an individualized pace, disseminated with consistency across patients, and may be more cost-effective than therapist-delivered interventions (23). Patients might even learn more and rate computer-based education more favorably than therapist-delivered intervention (24). Principles of effective instruction (25) such as repeated presentation of critical information with opportunities for overt responses (26), frequent opportunities for feedback (27), and performance-based incentives (28) can be integrated into HIV education without the need to train instructors in these methods. Recently, Getty and colleagues (29) applied these learning principles to teach populations at risk for HIV about pre-exposure prophylaxis (PrEP). The education course consisted of information about HIV, PrEP, and types of risky behavior and was delivered via interactive computer modules. To evaluate the program, researchers asked participants to complete multiple-choice tests before and after three sections of the course. Scores on test items related to each course improved significantly after participants completed the relevant course and learning maintained over repeated testing. A similar course could improve knowledge about HIV and ART in people living with HIV.

Early studies found that teaching people about HIV and ART can improve HIV-related knowledge and self-reported adherence (21,22). Although more recent studies have incorporated education into adherence interventions (30) and used technology to deliver education (23), there is a need for updated research on HIV and ART knowledge and the effectiveness of HIV education at improving that knowledge. This study was an experimental evaluation of a computer-delivered training program designed to teach adults with HIV with detectable viral loads about HIV and ART. The objectives of the study were to evaluate characteristics related to HIV and ART knowledge, describe performance in the HIV education program, evaluate the effectiveness of the HIV education program, and determine the factors, if any, related to learning in the HIV education program.

METHODS

Participants and Setting

Participants in this study were HIV-positive adults who met eligibility criteria for a parent randomized control trial evaluating a novel incentive intervention to promote viral load suppression in people living with HIV. Participants completed the HIV education program described in this study prior to being randomized into groups in the parent clinical trial. Eligible participants had detectable HIV viral loads (>200 copies/mL), had an HIV diagnosis for at least 12 weeks, and were not currently receiving HIV medical care or had been in HIV medical care for at least 12 weeks. Research staff recruited participants in Baltimore, Maryland, USA between November 2015 and August 2017 from medical clinics that serve adults living with HIV, from programs that serve people living with HIV, through newspaper and radio advertisements, and through word-of-mouth compensated referrals. Participants were excluded from the study and seen by a clinician if they reported current suicidal or homicidal ideation or had a severe psychiatric disorder.

Interested individuals completed a brief phone interview and potenitally eligible individuals were invited for a full intake interview in which they completed assessments. Participant characteristics were obtained from the Addiction Severity Index (ASI; 31); reading, spelling, and math grade levels were obtained from the Wide Range Achievement Test (WRAT-3 or WRAT-4; 32); health literacy was measured by the TOFHLA (33); the Beck Depression Inventory (BDI-II; 34) screened for depression; and HIV exposure was measured using a checklist with HIV exposure categories listed by the CDC. To confirm detectable viral load, we obtained records of a recent blood test from the participant’s healthcare provider or the participant provided a blood sample onsite and we arranged for testing at a diagnostic facility. Opioid, cocaine, and cannabis use was measured by collecting and testing a urine sample for opioids, cocaine, and THC from all participants with the exception of two participants who were on dialysis and were unable to provide a urine sample.

A total of 116 participants met eligibility criteria based on viral load. Fourteen participants were excluded from this analysis because they did not complete the intake assessments (n=12) or did not return after starting the HIV education program (n=2), making the final study sample N=102. Informed consent was obtained from all individual participants included in this study. All participants who completed the HIV education program described in the current study were randomized into the parent trial, details of which will be reported elsewhere when complete.

HIV Education Program

Following intake assessments, participants were invited to complete a computer-based, self-paced HIV education program that was designed to teach them about HIV and ART. The program was created and delivered using our computer-based training authoring and course presentation system called ATTAIN. ATTAIN is a novel, customizable software that allows instructional designers to develop courses with multimedia stimuli without the need for computer programming skills. The software allows for assessment using multiple-choice and fill-in-the-blank questions, immediate feedback for responding to those questions, random and repeated presentation of questions, continued presentation of questions until the learner meets criteria based on speed and accuracy of responding, and seamless integration of a system for delivering financial incentives for performance designed to motivate learner engagement and progress through training. Many of these features have been used with considerable effectiveness to teach use of the computer keyboard and numeric keypad (28) and to teach people with substance use disorder about HIV and PrEP (29). Access to ATTAIN can be obtained by contacting the corresponding author.

The training materials were based on information provided by the CDC (http://www.cdc.gov/hiv/) and were divided into three courses that differed by broad topic area. Course 1 was an introduction to HIV and AIDS, which defined terms and provided information about the stages of HIV, signs and symptoms of those stages, and the duration of each stage. Course 2 provided practical information about medication adherence including a description of the need to take ART medication, benefits of sticking with a medication regimen, what to do if a dose is missed, tips for staying on a treatment plan, and how to read a calendar. Course 3 was a guide to viral load and CD4 cell count as measures of health. The course described undetectable and decreasing viral loads, how to differentiate between graphs of viral load and CD4 cell count, how to read and interpret increases and decreases in viral load and CD4 cell count, and how to determine viral load and CD4 cell count by looking at points on a graph.

The HIV education program consisted of 19 modules (7 in Course 1, 7 in Course 2, and 5 in Course 3) that participants took sequentially. The average Flesch-Kincaid Reading Level across the 19 modules was 7.3 (SD=2.2 grade levels). To increase accessibility of the information, any text that was presented was accompanied by audio which participants could replay by clicking a button. Each module was divided into Presentation and Mastery Units. Presentation Units were designed to introduce learners to new material, while Mastery Units allowed participants to practice answering questions about that material.

Within each Presentation Unit, material was presented in presentation screens that displayed information without requiring overt responses by the learner. To engage learners during the Presentation Units, question screens, each consisting of a single multiple-choice question, were interspersed between presentation screens. Participants had unlimited time to complete each Presentation Unit.

Each Mastery Unit consisted of a series of question screens with multiple-choice questions about the content in the previous Presentation Unit. Question screens were presented in random order in a one-minute practice trial (i.e., “timing”). For the first two participants (P1 and P2), we piloted fluency-based mastery criteria in which the participants were required to respond accurately and quickly to pass a Mastery Unit timing. That is, for each one-minute timing, P1 and P2 were required to respond correctly to a minimum of 10 questions with a maximum of 3 incorrect responses to move on to the next module. If P1 and P2 did not meet those mastery criteria, they were required to repeat the Mastery Unit timing. P1 repeated the Mastery Unit timings from Modules 1–4 a total of 20 times. Because we were concerned about attrition prior to randomization into the parent trial, we removed those fluency-based mastery criteria and simply required that participants complete the module with no minimum accuracy or speed. Thus, for modules 5–19 for P1 and P2 and for all modules for the other participants, there were no response criteria for passing Mastery Units other than completing a single one-minute timing. Once a participant completed a Mastery Unit timing, the participant progressed to the next module.

On question screens, participants were presented a single question centered on the screen above two to four answer choices centered within choice boxes. The choice boxes were of equal size and were presented horizontally across the screen. On a given question screen, one of the choices in the choice boxes was correct and the remaining choices were incorrect. The location of the correct choice (e.g., left, middle, or right on trials that included three choices) was presented randomly across trials. Participants responded by clicking on a single choice box with the mouse. After a participant selected an answer choice, they received feedback. Following a correct response, the outline of the selected choice turned green and the word “Correct” appeared on the screen for 1 second. Following an incorrect response, the outline of the selected choice box turned red, the word “Incorrect” appeared on the screen for 1 second, and the participant was required to answer the same question again.

Participants were provided with an information screen at the beginning of each module which displayed the minimum number of correct responses required, the maximum number of incorrect responses allowed to pass, and the amount in financial incentives awarded for each correct response. Participants could earn $0.05 for each correct response and $4 for completing each module. Participants were not penalised for incorrect responses. Upon completion of a Presentation or Mastery Unit, the total amount the particpant earned was displayed on the computer and was automatically added to the participant’s account. The more quickly a participant provided correct responses in the mastery modules, the more money they could earn. On average, participants earned $121.56 (range $97.25 to $161.91).

Testing

A 52-item test assessed performance prior to and following training in each of the three courses. The test consisted of multiple-choice questions with two to four response options (chance responding=44.2% correct). The Flesch-Kincaid Reading Level for the test was grade 7.5. Participants could earn $0.02 for each question they answered correctly on the test for a maximum total of $4.16 across the four administrations. Participants were provided with their test score (number of correct/incorrect responses and amount of incentives earned) after completing each administration of the test. Participants were not provided with feedback on their responses to each question in the test.

At each test administration, the 52 questions were presented once each and there were no criteria for passing the test. Some questions were taken directly from the education program modules (i.e., module questions) and other questions (i.e., novel questions) were original to the test. Novel questions tested the same content as module questions, but differed in their wording. When averaging across participants, the difference between scores on novel and module questions was only 1.5 percentage points. Thus, we concluded that performance generalized across novel and module questions and examined test scores collapsing across those question types.

Experimental Design

We used a multiple-probe design to evaluate learning in the education program (35). This repeated-measures design is typically used to evaluate both the maintenance of and gains in performance prior to and following a training program and has been used in previous research evaluating the effectiveness of a computer-based PrEP education program (29). In the current study, the HIV education program was divided into three courses consisting of several modules each. Each participant completed the 52-item test described above four times (Test 1, Test 2, Test 3, and Test 4). Participants completed the test prior to beginning the education program, and then again after the participant completed the modules within each of the three courses. Participants completed Test 1 prior to taking Course 1, Test 2 after taking Course 1 and prior to taking Course 2, Test 3 after taking Course 2 and prior to taking Course 3, and Test 4 after taking Course 3. Due to software and administrator errors, one participant did not complete Test 1, another participant did not complete Test 2, and a third participant did not complete Test 3. All other participants completed every test and none of the participants missed more than one test administration.

To evaluate the effectiveness of the HIV education program, the 52 items on the test were divided into three groups corresponding to the material covered in the three Courses. Test items 1–24 (i.e., Subtest 1) corresponded to Course 1, test items 25–44 (i.e., Subtest 2) corresponded to Course 2, and test items 45–52 (i.e., Subtest 3) corresponded to Course 3. The hypothesis was that performance on each subtest would increase in the test administration that followed completion of the relevant course. Further, we expected that performance on each subtest would maintain in the test administrations following the relevant course (for Course 1 and Course 2) and that performance on each subtest would not improve in the test administrations prior to the relevant course (for Course 2 and Course 3). For example, we expected that scores on Subtest 2 would be lower for Test 1 and Test 2 (the tests that were administered before Course 2) than for Test 3 and Test 4 (the tests that were administered after Course 2). Furthermore, scores on Subtest 2 would not improve from Test 1 to Test 2 and would maintain from Test 3 to Test 4.

Data Analysis

One goal of the study was to describe the relation between baseline knowledge of HIV and ART and other participant characteristics. Performance on the first administration of the 52-item test (Test 1, collapsing across subtests) represented baseline HIV and ART knowledge. Relations between performance on Test 1 and participant characteristics were described using independent-samples t-tests for dichotomous variables and Pearson correlations for continuous variables. For these and all other analyses, values of p<.05 were considered statistically significant.

Another goal of the study was to describe performance in the HIV education program. We characterized performance separately for each course. Performance measures included the time it took participants to complete Presentation Units, the time it took for participants to complete the Mastery Units, and the number of items correct per minute in Mastery Units (i.e., fluency) for each course.

The primary goal of this study was to evaluate learning in the HIV education program. The main dependent variable was performance (percentage of responses correct) on the subtest related to each course. Performance on Subtest 1, Subtest 2, and Subtest 3 were entered into three separate piecewise mixed-effects models (for Course 1, Course 2, and Course 3, respectively). The piecewise mixed-effects model was flexible in that it estimated separate parameters describing the increase in performance in the test immediately following the relevant course, whether performance maintained across subsequent tests, and whether performance maintained across the tests taken prior to the relevant course. For Subtest 1, the model estimated performance improvement after Course 1 (Test 1 to Test 2) and whether performance maintained following the course (Test 2 through Test 4). For Subtest 2, the model estimated the stability of performance prior to Course 2 (Test 1 and Test 2), improvement in performance following Course 2 (Test 2 to Test 3), and whether scores maintained after Course 2 (Test 3 and Test 4). For Subtest 3, the model estimated performance improvement after Course 3 (Test 3 to Test 4) and the stability of performance prior to Course 3 (Test 1 through Test 3).

The piecewise mixed-effects model permitted analyses of covariates and their interaction with test performance related to each course. The model was adjusted to include poverty and high-school education as dichotomous covariates and log-transformed viral load; standardized score in reading, spelling and math from the WRAT; and TOFHLA score as continuous covariates. For all covariates, we evaluated the interaction between those variables and performance improvement following courses. This interaction analysis allowed us to determine the factors, if any, that were related to learning in the HIV education program.

RESULTS

Participant Characteristics

Table 1 shows participant characteristics. Participants were mostly Black, unmarried, unemployed, and living in poverty. Although most participants reported having a high school diploma or General Educational Development (GED) certificate, average grade-levels in reading, spelling, and math were at the eighth grade or below and most participants had standard WRAT scores in the “Average”, “Low-Average”, or “Deficient” range. Most participants had health literacy in the “Adequate” range (i.e., scores of 75 or higher) on the TOFHLA. Sixty-eight percent of participants had BDI scores in the minimal range (i.e., scores of 13 or lower). Over half of the participants reported getting HIV through heterosexual contact and most of the other participants reported exposure through injection drug use or male-to-male sexual contact. Mean viral load was significantly higher than the lowest measurable value using current testing procedures (20 copies/mL). Sixty-seven participants had urine samples positive for opioids, cocaine, and/or THC (52/67 of those participants were positive for cocaine or opioids). Relation between Participant Characteristics and Baseline (Test 1) Performance

I.

Participant Characteristics at Study Intake

Characteristic	Percent, Median, or Mean
Malesa	% (no.)	53.9 (55)
Race/Ethnicity	% (no.)
Black		89.2 (91)
White		9.8 (10)
Other		1 (1)
Age (years)	Mean (SD)	47 (10)
Employment in Past 30 Days	% (no.)	18.6 (19)
Living in Povertyb	% (no.)	82.4 (84)
Marital Status	% (no.)
Never Married		70.6 (72)
Divorced or Separated		18.6 (19)
Widowed		7.9 (7)
Married		3.9 (4)
High School Diploma or GED	% (no.)	67.6 (69)
WRAT Grade Level	Mean (SD)
Reading		8 (3.8)
Spelling		7.6 (4.1)
Math		5.9 (2.8)
TOFHLA Score	Mean (SD)	79.8 (19.5)
BDI Score	Mean (SD)	9.9 (9.4)
HIV Exposure	% (no.)
Heterosexual Contact		55.9 (57)
Injection Drug Use		17.7 (18)
Men Who Have Sex with Men		14.7 (15)
Born to a Mother at Risk for HIV		4.9 (5)
Multiple Exposure		3.9 (4)
Other		2.9 (3)
Viral Load (copies/mL)c	Median (Q1, Q4)	12,413 (1,554, 39,709)
Time Since Diagnosis of HIV (years)	Mean (SD)	17.6 (9.2)
Positive Urine at Intake	% (no.)
THC/Cannabis		29 (29)
Cocaine		47 (47)
Opioids		17 (17)
THC/Cannabis, Cocaine, or Opioids		67 (67)

^aAll participants identified as either male or female, including 2 participants who also identified as transgender.

^bLiving in Poverty was calculated using income, age, and family size data from the ASI and 2017 Poverty Thresholds from the US Census Bureau.

^cThe distribution of viral load was skewed. The median viral load is shown with lower (Q1) and upper (Q4) quartiles.

The average score on Test 1 was 65.1% (range 32.7%–84.6%). A single-sample t-test confirmed that Test 1 scores were significantly higher than chance (44.2%), t(100)=22.5, p < .001, d=2.2. Thus, participants had some working knowledge about HIV and ART prior to taking any of the courses. Table 2 shows participant characteristics statistically significantly related to performance on Test 1. Participants who were living in poverty and who did not have a high-school diploma or GED scored lower than participants living above the poverty line with a high-school diploma or GED. Performance on Test 1 was also positively related to reading, spelling, and math scores; positively related to health literacy; and negatively related to viral load. No other participant characteristics included in Table 1 were statistically significantly related to Test 1 performance.

II.

Relation between Participant Characteristics and HIV and ART Knowledge in the First Test

Variable (n)	Test 1 Score (SD)	Statistica
Poverty		t(99)=−2.5, p<.05
At or below Poverty Line (84)	64.0 (9.4)
Above Poverty Line (18)	70.0 (7.4)
Education		t(99)=−3.1, p<.01
< 12 Grade (32)	61.0 (10.3)
≥ 12 Grade (69)	67.0 (8.2)
WRAT Reading Scoreb		r=0.44, n=100, p<.001
Deficient (17)	58.4 (10.2)
Borderline (28)	63.8 (7.3)
Low Average (19)	65.0 (9.2)
Average or High Average (36)	69.5 (8.5)
WRAT Spelling Scoreb		r=0.37, n=100, p<.001
Deficient (22)	58.7 (9.7)
Borderline (17)	64.8 (8.8)
Low Average (25)	66.8 (8.3)
Average or High Average (36)	68.1 (8.5)
WRAT Math Scoreb		r=0.37, n=100, p<.001
Deficient (20)	59.7(11.2)
Borderline (24)	65.6 (8.6)
Low Average (30)	64.0 (8.4)
Average (26)	70.2 (7.0)
TOFHLA Score		r=0.50, n=101, p<.001
Inadequate (13)	55.0 (10.4)
Marginal (13)	62.3 (5.5)
Adequate (75)	67.3 (8.4)
Viral Load (copies/mL)c		r=−0.24, n=101, p<.05
First Quartile (26)	67.9 (11.3)
Second Quartile (25)	65.3 (8.1)
Third Quartile (25)	64.8 (7.8)
Fourth Quartile (25)	62.3 (9.4)

^aStatistical tests were either independent-samples t-tests (t) or Pearson correlations (r).

^bStandard scores on the WRAT were categorized as deficient (<70), borderline (70–79), low average (80–89), average (90–109), or high average (110–119). Very few participants were in the high average range in reading (n=1) and spelling (n=3), so average was combined with high average.

^cViral load was log transformed prior to being entered into statistical tests.

Training

Table 3 shows the time participants spent, on average, in Presentation and Mastery Units for each course and in total. On average, the HIV education program took a little over an hour to complete. The participant who took the longest time to complete the HIV education program finished in 2 hours and 17 minutes. Table 3 also shows accuracy and fluency in Mastery Units. On average, participants answered questions with greater than 86% accuracy and answered over 15 questions correct per minute in Mastery Units.

III.

Duration, Accuracy, and Fluency (with Range) in Each Course of the HIV Education Program

	Course la	Course 2	Course 3	Total
Presentation Time in Minutes	6.8	10	6.3	23.1
	2.1–56.2	2.9–44.6	1.9–29.5	7.9–86.8
Mastery Time in Minutes	17.2	23.5	11.8	52.5
	13.2–26.9	13.1–26.9	7.0–15.1	34.6–60.3
Total Time in Minutesb	24	33.5	18.1	75.6
	15.9–71.8	17.7–69.3	9.6–43.0	45.9–136.7
Accuracy in Mastery Units (%)c	83.9	90.7	84.4	86.3
	33.3–100	59.3–99.7	45.0–100	47.2–99.8
Fluency in Mastery Unitsd	13.6	16.8	15.8	15.4
	3.3–32.8	3.9–41.2	5.8–43.6	4.8–34.9

^aCourse 1 measures exclude P1 and P2 who received repeated practice.

^bDuration, accuracy, and fluency were calculated for each participant and then averaged across participants.

^cAccuracy is the percentage of questions answered correctly averaged across Mastery Unit timings.

^dFluency is expressed as the number of questions correct per minute averaged across Mastery Unit timings.

Evaluation of HIV Education Program Effectiveness

In general, the percentage of participants answering correctly increased for each question from Test 1 to Test 4 (see Supplementary Data). Figure 1 shows percentage of items correct (i.e., scores) on each course-related subtest, averaged across participants, for the four test administrations. The top panel shows scores on Subtest 1 (items from Course 1), the middle panel shows scores on Subtest 2 (items from Course 2), and the bottom panel shows scores on Subtest 3 (items from Course 3). The vertical dashed line shows when participants completed the relevant course. On average, scores increased after participants completed each course. Table 4 shows the change in scores (i.e., expected slopes) on each course-related subtest before participants completed the relevant course (baseline), immediately after participants completed the relevant course (learning period), and following the learning period (maintenance) based on piecewise mixed-effects model analyses. Scores on Subtest 1 significantly improved following Course 1 and did not change in the subsequent tests. Scores on Subtest 2 significantly improved following Course 2 and did not change prior to the course or change in subsequent tests. Scores on Subtest 3 significantly improved following Course 3 and did not change prior to the course. Table 4 shows significant covariates for the model describing learning in each Course. Health literacy score on the TOFHLA was a significant covariate for the models describing Course 1 and Course 2. Poverty, viral load, and standardized reading scores from the WRAT were significant covariates for the model describing Course 3.

Figure 1.

Each graph shows the percentage correct on each subtest across each of the four administrations of the test. The top panel shows percentage correct on Subtest 1 (Course 1), the middle panel shows percentage correct on Subtest 2 (Course 2), and the bottom panel shows percentage correct on Subtest 3 (Course 3). Percentage correct is averaged across participants and error bars are standard deviation. The vertical dashed lines indicate when participants completed each Course within the HIV education program. Horizontal dotted lines represent percentage correct by chance.

IV.

Change in Scores during Baseline, Learning Period, and Maintenance and Covariates Calculated from the Adjusted Piecewise Mixed-Effects Model^a

	Course 1	Course 2	Course 3
Baselineb	-	2.03 (1.14), .08	0.77 (1.08), .48
Learning Periodc	18.78 (1.10), <.001	14.45 (1.14), <.001	15.31 (2.06), <.001
Maintenanced	−0.13 (0.58), .82	−1.24 (1.14), .28	-
Covariatese
Poverty	−2.90 (2.76), .29	−2.40 (2.23), .28	−10.88 (3.52), <.01
Education	3.05 (2.78), .27	3.33 (2.24), .14	−2.39 (3.53), .50
Viral Load	−1.94 (1.16), .10	−0.49 (0.94), .60	−3.24 (1.48), .03
Reading	0.53 (0.59), .37	−0.14 (0.47), .77	2.15 (0.75), <.01
Spelling	−0.11 (0.53), .83	0.08 (0.43), .85	−1.22 (0.68), .07
Math	0.63 (0.47), .18	0.28 (0.38), .47	0.65 (0.60), .28
Health Literacy	0.32 (0.09), <.001	0.16 (0.07), .03	0.20 (0.11), .08

^aThe change in scores was calculated from coefficients in the adjusted piecewise mixed-effects model and are shown above as the expected slope (SE), p-value.

^b”Baseline” represents the change in scores for the subtest prior to the relevant course.

^c”Learning Period” is the change in scores for the subtest immediately prior to the relevant course and immediately after the course.

^d”Maintenance” is the change in scores for the subtest after the relevant course.

^eCovariates for the overall model describing each course are listed below the main effects of training. Statistically significant slopes are in bold.

Factors Related to Learning in the HIV Education Program

Table 5 shows the relation beween covariates and how much participants learned in each course based on the piecewise mixed-effects model interaction analyses for the learning period. Below each covariate, we list the change in scores on course-related subtests (i.e., expected slopes) for categories of the covariate. The categories were levels of the dichotomous covariates and low-scoring and high-scoring groups for continuous covariates. For Course 1, participants with a high school diploma or GED learned more from the course than participants who did not have a diploma or GED. For Course 1 and Course 3, higher reading, spelling and math scores on the WRAT and higher health literacy scores on the TOFHLA were related to greater learning than lower scores. For Course 2, higher reading and spelling scores on the WRAT and higher health literacy scores on the TOFHLA were related to greater learning than lower scores. The expected slopes for categories of each covariate were all positive and the lowest slope represented an improvement of over 11 percentage points.

V.

The Relation between Covariates and Learning Calculated from the Piecewise Mixed-Effects Model Interaction

	Course 1 Learning	Course 2 Learning	Course 3 Learning
Poverty	−4.68 (2.87), .10a	−7.69 (5.21), .14	3.00 (7.21), .68
In Povertyb	17.94 (1.20)	14.05 (1.24)	15.68 (2.25)
Not in Poverty	22.63 (2.61)	17.56 (2.73)	13.03 (4.86)
Education	10.76 (2.24), <.001	3.34 (4.14), .42	10.13 (5.76), .08
<12 grade	11.39 (1.85)	12.05 (1.96)	7.32 (3.51)
>=12 grade	22.15 (1.26)	15.87 (1.35)	19.08 (2.43)
Viral Load	−2.01 (1.17), .08	−2.58 (2.10), .44	1.49 (2.96), .62
Lowc	18.74 (1.08)	14.63 (1.12)	15.26 (2.04)
High	16.86 (1.54)	12.58 (1.59)	15.29 (2.88)
Reading	0.33 (0.08), <.001	0.4 (0.14), .006	0.49 (0.20), .01
Low	18.77 (1.07)	14.39 (1.08)	15.39 (1.99)
High	23.13 (1.51)	18.64 (1.54)	22.90 (2.83)
Spelling	0.25 (0.07), <.001	0.37 (0.12), .002	0.41 (0.17), .02
Low	18.76 (1.07)	14.40 (1.09)	15.42 (1.97)
High	22.64 (1.52)	18.95 (1.54)	23.07 (2.82)
Math	0.45 (0.09), <.001	0.28 (0.16), .07	0.56 (0.22), .008
Low	18.83(1.05)	14.42 (1.11)	15.27 (1.99)
High	24.44 (1.50)	17.56 (1.57)	22.77 (2.82)
Health Literacy	0.36 (0.05), <.001	0.39 (0.10), <.001	0.35 (0.14), .01
Low	18.72 (1.01)	14.59 (1.06)	15.32 (1.96)
High	25.64 (1.43)	19.78 (1.50)	23.05 (2.79)

^aThe statistical test relating the covariate to learning is shown as coefficient (SE), p-value and is above the expected slopes (SE) for categories of the covariate.

^bExpected slopes are shown separately for each level of the dichotomous variables Poverty and Education.

^cFor continuous variables, expected slopes are shown for the “Low” group as the mean (i.e., an average value on the variable) and the “High” group as the mean plus one standard deviation (i.e., a high value on the variable).

Acceptability

Following Test 4, participants had the option to complete a survey about the HIV education program. Figure 2 shows the mean ratings (bar) and each individual participant’s rating (filled circle). For the first seven items, participants responded on a visual analog scale (VAS) ranging from 0 (not at all) to 100 (extremely) whether the HIV education program could be described according to adjectives. The slider was initially at the 0 anchor and if a participant did not use the mouse to drag the slider, the response was not recorded. On average, participants described the course as helpful (VAS score M=88, SD=28), challenging (VAS score M=66, SD=35), enjoyable (VAS score M=78, SD=25), and interesting (VAS score M=82, SD=29); few participants described the course as a waste of time (VAS score M=6, SD=16), boring (VAS score M=18, SD=25), or frustrating (VAS score M=28, SD=26). For the final two items on the survey, participants rated on a VAS whether the information they learned in the course would be helpful in managing their HIV treatment and on a Likert-type scale whether they would change how they managed their HIV in the future based on what they learned in the Course. Most participants responded that the information would help them manage their treatment (VAS score M=96, SD=13) and change their behavior (Likert-type rating M=4.6, SD=0.7).

Figure 2.

Acceptability ratings of the HIV education program collected after participants completed the final test. Each filled circle represents a single participant’s rating (circles are overlapping when many participants provided the same rating) and the bars represent the mean rating across all respondents. The number of respondents for each item is listed above each bar. The first eight items are visual analog scale (VAS) ratings for each adjective with anchors of “Not at All” (0) and “Extremely” (100) and correspond to the left y-axis. The rightmost item shows ratings on a 1–5 Likert-type scale with anchors of “Strongly Disagree” (1) and “Strongly Agree” (5) and corresponds to the right y-axis. Wording for the rightmost two items are: *”The information I learned in the HIV Education Course will be helpful in managing my HIV treatment.” **”I will change how I manage my HIV in the future based on what I learned in the HIV Education Course.”

DISCUSSION

Education programs can teach people living with HIV about the disease and its treatment (21,22) and can be combined with other interventions to improve adherence to ART (16). This study evaluated whether a computer-based HIV education program improved knowledge about HIV and ART in people living with HIV who had detectable viral loads. The HIV education program was effective in improving participants’ knowledge about HIV and ART as measured by a cumulative test. Performance on the subtests related to each course improved immediately following each relevant course. Furthermore, performance did not change in subtests taken prior to relevant courses and performance maintained at high levels in subtests taken after relevant courses.

The computer-based HIV education program was delivered in ATTAIN, which proved to be an effective and acceptable presentation system to teach participants about HIV and ART. We were able to present questions to participants repeatedly and consistently, provide - opportunities for participants to practice answering questions in random order, provide feedback on whether or not participants answered questions correctly, and issue financial incentives for correct responding. Previous research found similar efficacy using this methodology to teach adults at risk for HIV about PrEP (29). We extended their findings because we were able to observe learning and retention following each course without requiring that participants achieve a specified level of speed and accuracy. We also studied the effectiveness of this computer-based training program in people living with HIV who had detectable viral loads.

Participants in this study were recruited because they had detectable viral loads despite being diagnosed with HIV for at least 12 weeks and being linked to care. Thus, it is probably the case that these participants were not taking their medication every day. However, they still had some working knowledge of HIV and ART prior to completing the education program. On the first test, which represented baseline knowledge, over 90% of participants responded correctly that medication is necessary to control HIV and that medications should be taken every day (see Supplementary Data). Sociodemographic characteristics predicted some of the individual differences in overall accuracy on the first test. Accuracy was positively related to education; achievement in reading, spelling, and math; and health literacy; and was negatively related to viral load and poverty.

Sociodemographic factors were also related to learning immediately following each course. We did find that participants with less than a high-school education; those low achievement in reading, spelling, and math; and those with low health literacy learned about HIV and ART. That is, the model describing the increase in scores during the learning period had positive expected slopes for all these groups. However, these groups learned about HIV and ART to a lesser degree than participants with higher education, achievement, and health literacy. Because of limited training time, we did not require that participants meet criteria based on accuracy and speed to pass each course. Getty et al. included these criteria but did not have the statistical power to detect whether sociodemographic characteristics predicted knowledge gains (29). Requiring accurate and speedy performance has been successful in improving academic outcomes in populations with low educational achievement (36). These measures of behavioral fluency (47) may be an inexpensive and effective approach to equalizing educational gains across different populations.

Notably, we found that living in poverty did not predict learning following the HIV education program. The fact that initial knowledge about HIV and ART was related to poverty signals the need for targeted HIV education and treatment-as-prevention educational approaches in low-income populations. This need is especially urgent because research has suggested that there is a concentrated HIV epidemic among areas of urban poverty (38) and that poverty is a driving force in the current HIV epidemic (39). Our computer-based education program could be a useful tool to build health equity. The progam could be provided at low cost within community health programs.

Health literacy was a significant covariate for every course and a significant interaction term for most courses. We found that participants with higher health literacy performed with greater accuracy prior to the HIV education program and improved to a greater degree following each course than participants with lower health literacy. This finding was not surprising given related research indicating that health literacy plays a mediating role in the association between demographic characteristics and health knowledge (40). Several other studies have demonstrated that health literacy is related to adherence to ART (41,42), HIV viral load CD4 count (43), and HIV knowledge (17). The role of health literacy in adherence may not be clear cut (44,45), but a review of the literature suggests that improving health literacy can improve treatment adherence (46). Future research could evaluate whether modifying our HIV education program to require that participants continue training until they respond correctly and quickly (37) or an adaptive design (47) that meets patients at their level of mastery improves learning outcomes for people with low health literacy.

Some limitations to our study should be noted. In our assessment of acceptability, there was a low response rate to some of the VAS items (e.g., “A Waste of Time” was only rated by 27 participants). There were no explicit instructions that participants were required move the slider for a response to be registered. Participants may have thought they were providing “0” ratings but failed to move the slider so the computer program could register that response. Missing ratings were more likely to occur for the negatively valenced items (“Frustrating”, “Boring”, and “A Waste of Time”) than for the positively valenced items (“Interesting”, “Enjoyable”, and “Helpful”). If ratings of “0” were imputed for all missing values, then the pattern of ratings shown in Figure 2 would hold. Our study population may not be generalizable to all people living with HIV who are linked to care yet have unsuppressed viral loads. However, our computer-based education program can be easily disseminated and can be evaluated in the future with other populations (e.g., rural people living with HIV). Because our education program was delivered as a standard treatment for all participants in an ongoing clinical trial, our evaluation could not determine if increased knowledge of HIV and ART led to decreases in viral load or increases in CD4 counts. Future research can evaluate whether participants who receive the education course have improved adherence relative to participants who are waitlisted for the education course.

CONCLUSIONS

Our computer-based training program improved knowledge about HIV and ART in participants with unsuppresed HIV viral loads. Most participants rated the training program favorably. Programs delivered by health care providers (21) sometimes require extensive resources and multi-day training sessions to build competency in trainers. A benefit of our computerized method is that it can be integrated with mobile technology to disseminate HIV and ART education widely without the need to train staff. More research is needed on implementation, especially in modifying the program to promote educational equity.