Skip to main content
PMC home page
Archives of Clinical Neuropsychology logoLink to Archives of Clinical Neuropsychology
. 2016 Jan 6;31(2):176–185. doi: 10.1093/arclin/acv090

Health-Related Everyday Functioning in the Internet Age: HIV-Associated Neurocognitive Disorders Disrupt Online Pharmacy and Health Chart Navigation Skills

Steven Paul Woods 1,2,*, Jennifer E Iudicello 2, Erin E Morgan 2, Marizela V Cameron 2, Katie L Doyle 3, Tyler V Smith 2, Clint Cushman 2, The HIV Neurobehavioral Research Program (HNRP) Group
PMCID: PMC4758381  PMID: 26743327

Abstract

This study evaluated the effects of HIV-associated Neurocognitive Disorders (HAND) on 2 Internet-based tests of healthcare management. Study participants included 46 individuals with HIV infection, 19 of whom were diagnosed with HAND, and 21 seronegatives. Participants were administered Internet-based tests of online pharmacy and health records navigation skills in which they used mock credentials to log in to an experimenter-controlled website and independently perform a series of typical online health-related behaviors (e.g., refill a prescription, read and interpret an electronic chart note). HAND was associated with significantly lower accuracy on both the online pharmacy and health records navigation tasks. Among the HIV+ participants, poorer performance on the online healthcare navigation tasks was associated with fewer years of education, higher plasma viral load, less frequent Internet use, and lower health literacy. Findings indicate that individuals with HAND may have marked difficulties navigating the Internet to complete important health-related behaviors.

Keywords: HIV/AIDS, Health literacy, World Wide Web, Medication management, Neuropsychology, Electronic health records

Introduction

Health literacy can be defined as “the capacity to obtain, communicate, process, and understand basic health information and services to make appropriate health decisions” (The Patient Protection and Affordable Care Act, 2010). Low health literacy is evident in approximately 20–40% of individuals living with HIV infection (Kalichman & Rompa, 2000) and is associated with reduced engagement in health care (Jones et al., 2013), nonadherence to antiretroviral therapies (ART) (Jones et al., 2013; Kalichman, Ramachandran, & Catz, 1999), and poorer HIV disease outcomes (Kalichman & Rompa, 2000). The Internet plays an increasingly critical role in the health literacy and health care of persons living with HIV disease, approximately 50% of whom use the Internet for healthcare purposes (Dorner et al., 2014; Thomas & Shuter, 2010). HIV+ individuals' use of health-related Internet resources, such as navigating the World Wide Web to obtain essential health-related information (e.g., disease symptoms, and manage their health care [e.g., online pharmacy refills]), is associated with better general health status (Kalichman et al., 2005; Schnall et al., 2014). The Internet is also increasingly being used as a means of delivering HIV prevention and adherence interventions, including those that specifically target health literacy (Wawrzyniak, Ownby, McCoy, & Waldrop-Valverde, 2013).

However, limited Internet access and poor online navigation skills may represent a serious barrier to optimal HIV health outcomes. Approximately two-thirds of HIV-infected individuals require assistance in using the Internet (Mayben & Giordano, 2007). Indeed, there are many potential hazards of using the Internet for health-related activities, including obtaining erroneous information about symptoms and treatments (Kunst et al., 2002), misinterpreting information in electronic medical records, and mismanaging healthcare accounts and schedules. These challenges may be particularly relevant for persons from disadvantaged backgrounds, who are at high risk for low health literacy (Kalichman & Rompa, 2000) and may have difficulty in effectively navigating the Internet (Blackstock et al., 2014; Mayben & Giordano, 2007).

Neurocognitive impairment is another important clinical factor that could interfere with Internet navigation skills (Goverover et al., 2010). HIV-associated Neurocognitive Disorders (HAND) affect approximately half of HIV-infected persons in the era of combination ART (cART) (Heaton et al., 2010) and adversely impact a wide range of health outcomes, such as medication nonadherence (Hinkin et al., 2002). Individuals with HAND also evidence deficits in the fundamental (e.g., numeracy) and higher-order (i.e., comprehension and application) aspects of health literacy (Morgan et al., 2015). Accordingly, this pilot study sought to determine the effects of HAND on two novel Internet-based tasks of healthcare management that involve independent navigation of an online pharmacy and electronic health records. It was hypothesized that individuals with HAND would demonstrate poorer performance on the online health tasks when compared with HIV+ persons without HAND and seronegatives. Within the HIV+ cohort as a whole, it was expected that online health task performance would be associated with more severe HIV disease, self-reported difficulties in online health behaviors, and lower scores on well-validated measures of health literacy.

Methods

Sixty-seven participants were enrolled from ongoing studies at a neuroAIDS research center, which recruits from local HIV clinics and community-based organizations. Study exclusions included neuromedical conditions (e.g., seizure disorders, closed head injury, and stroke), severe psychiatric conditions (e.g., psychosis), positive Breathalyzer for alcohol or urine toxicology for illicit drugs, and the use of the Internet less than five times over the past 5 years. HIV serostatus was determined with Medmira rapid tests and HAND was classified according to Frascati criteria (Antinori et al., 2007) as derived from a comprehensive neuropsychological, psychiatric, and medical evaluation obtained from the parent study. There were three primary study groups of interest: HIV− (n = 21), HIV+ without HAND (HAND−; n = 27), and HIV+ with HAND (HAND+; n = 19).

Clinical Characterization

Study participants underwent comprehensive characterization of clinical functioning as part of a center-wide parent study. A neuromedical exam, history, and phlebotomy/labs provided information on estimated duration of HIV infection, current and nadir CD4 counts, plasma levels of HIV RNA, current cART regimen, and hepatitis C infection. Current (i.e., 30-day) and lifetime mood and substance use disorders were determined with the Composite International Diagnostic Interview (version 2.1) (World Health Organization, 1998). More recent mood (i.e., in the past week) was assessed using the Profile of Mood States (McNair, Lorr, & Droppleman, 1981). The neurocognitive batteries used to derive HAND classifications were constructed in accordance with Frascati research diagnostic criteria (Antinori et al., 2007) and details regarding these assessments are provided subsequently and can be found elsewhere (Cysique et al., 2006; Woods et al., 2004). Raw scores were converted to demographically adjusted T-scores, which were then used to generate a Global Deficit Score for which a cutpoint of ≥0.5 was used to designate persons with global neurocognitive impairment for determination of HAND (see Carey et al., 2004). Among HIV+ participants, 41% were classified with HAND, which included 5 persons with Asymptomatic Neurocognitive Impairment, 12 with Minor Neurocognitive Disorder, and 2 with HIV-associated Dementia.

Online Assessment of Health-Related Behaviors

Participants were assessed using two web-based tests developed in PHP and Javascript. Raw data on participant actions (i.e., page loads and what triggered them) were stored in a MySQL database and later analyzed via a scoring script.

Test of online pharmacy skills

All participants first completed a brief version of the Medication Management Test-Revised (MMT-R; see Scott et al., 2011). After completing the MMT-R, participants were instructed to use an experimenter-controlled website (i.e., test of online pharmacy skills [TOPS]) that was modeled after the current versions of various commercial institutions (CVS, Rite Aid, etc.) in order to manage their mock prescriptions (see Supplementary material online, Fig. S1). The TOPS website included five main sections (including login) and six sub-pages. Participants were asked to log in using a mock identity provided by the experimenter (contained in a wallet that included a mock driver's license, credit cards, and insurance information) at the start of the assessment and used across all components of the TOPS site. Once the participant successfully logged in, they saw five different tabs in the navigation bar (i.e., Welcome, Prescription Management, Resources, Drug Information Center, and Logout). Participants were instructed to complete the following specific tasks in any order on the pharmacy website: (i) Submit a request to refill their prescription for the medication, Celetra, which was part of the MMT-R administered just prior to TOPS; (ii) Make a request to fill a new prescription for a new medication, Parlenol. In doing so, they were asked to enter all necessary primary insurance information (from the insurance card in their mock wallet) in order to process the medication for in-store pick-up; (iii) Activate a reminder that would alert them via text message to a smartphone (an iPhone provided by the experimenter) when the new Parlenol prescription was available for pick-up; (iv) Enter the confirmation code received via text on the smartphone on the TOPS pharmacy website to complete the activation process; and (v) Check for possible drug interactions for their new Parlenol prescription. Participants were instructed to pay close attention to these possible drug interactions, because they might be asked to recall them after completion of the task. The task instructions were in front of participants at all times. The following variables of interest were generated: (i) total correct (i.e., refill, new prescription request, reminder request, activation confirmation, and drug interaction), (ii) total errors (e.g., log-in failures), and (iii) total time to complete the task.

Test of online health records navigation

The test of online health records navigation (TOHRN) is an experimenter-controlled website designed to simulate a real patient-oriented electronic healthcare management interface (see Supplementary material online, Fig. S1). The website contains information about a network of doctors, as well as different sections to which participants could navigate in order to find information on lab results, current prescriptions and diagnoses, and messages from various healthcare entities. Participants began by logging in to their electronic health record using a mock username and password (linked to the identity in the wallet provided earlier to the participant in the TOPS scenario). Once successfully logged in, the participants had access to six tabs in the navigation bar at the top of the home page: Home, My Account, Appointment, My Medical Records, Message Center, and Logout. Participants were provided with the following instructions, which were available to them at all times during the assessment: “1) There are several unread messages in your message center. Log onto the website and read your messages; 2) Be sure to look for any messages regarding test results; 3) Check your lab results. If you receive abnormal test results, follow the instructions your doctor left in his note to you.” To complete the task correctly, the participant must login correctly, locate the correct message in their message center (i.e., doctor’s message about test results) and open it, observe that the test results are abnormal, and note that the doctor provides instructions in his message to schedule a follow-up appointment in 30 days. In order to follow the doctor's instructions, the participant must select the appropriate appointment (with the correct doctor, on the correct day) from a set of available appointment slots displayed on a calendar in the Appointments section. The following variables of interest were generated: (i) total correct (i.e., message, lab result, doctor's note, and scheduled appointment), (ii) total errors, and (iii) total time to complete.

Neurocognitive Correlates

Domain-based raw neurocognitive test scores were generated for correlational analyses with TOPS and TOHRN. The seven domains were constructed as follows: (i) Motor (dominant and nondominant hands of the Grooved Pegboard Test; Kløve, 1963); (ii) Processing speed (WAIS-III Digit Symbol; Psychological Corporation, 1997), Trail Making Test Part A (Reitan & Wolfson, 1985), and detection (reaction time) and identification (reaction time) subtests of the CogState (www.cogstate.com); (iii) Attention/Working Memory (PASAT-50; Gronwall, 1977), the one-back (accuracy) and two-back (accuracy) subtests from the CogState; (iv) Learning (HVLT-R Total 1–3; Brandt & Benedict, 2001), BVMT-R Total 1–3 (Brandt & Benedict, 2001), and CogState one-card learning (accuracy); (v) Memory (HVLT-R Delayed Recall; Brandt & Benedict, 2001), BVMT-R Delayed Recall (Brandt & Benedict, 2001), and CogState continuous paired associates (accuracy); (vi) Verbal Fluency (Delis, Kaplan, & Kramer, 2001) (animals and letter/FAS); and (vii) Executive Functions (WCST-64; Heaton, 1993) perseverative responses, Trail Making Test Part B (Reitan & Wolfson, 1985), and the Iowa Gambling Task (Bechara, 1994) total.

Health Literacy Correlates

In addition to the MMT-R, participants also completed a brief battery of well-validated health literacy measures, including REALM (Davis, Crouch, & Long, 1991), HIV Knowledge 18 (HIV-K-18) (Carey & Schroder, 2002), Expanded Numeracy Scale (Lipkus, Samsa, & Rimer, 2001), Short Assessment of Health Literacy (Lee et al., 2010), TOFHLA Reading Comprehension (Parker et al., 1995), and Newest Vital Sign (Weiss et al., 2005).

Internet Use and Anxiety Correlates

Participants completed the eHealth Literacy Scale (eHEALS) (Norman & Skinner, 2006), which is a widely used measure of participants' knowledge and skills in using the Internet for health-related purposes. Participants completed a brief self-report questionnaire regarding their Internet use habits, including whether they own (or have regular access to) a personal computer, and how often and for what purposes they use the Internet (e.g., health care). Participants were asked about the frequency and severity of any difficulties they experience performing health-related tasks online (e.g., pharmacy and health records). In addition, participants completed items from the Computer Anxiety Scale, which asks about general anxiety related to using computers and the Internet.

Statistical Analyses

The online task scores were non-normally distributed (Shapiro–Wilk W tests, ps < .01). As such, our primary hypotheses regarding the effects of HAND group on the online tasks were tested using a series of Wilcoxon Rank Sums tests (NB: findings did not differ interpretively if a parametric approach was used for analysis). Effect sizes for the pair-wise comparisons to follow-up a significant omnibus test were estimated using Hedges’ g, for which values of 0.8, 0.5, and 0.2 are considered large, medium, and small, respectively. Although the HAND groups differed on a few key clinicodemographic factors (e.g., sex and depression), these variables were not associated with online task scores in the entire study sample (ps > .10) and were thus not considered for inclusion as covariates. Spearman's ρ or χ2 tests were used to determine the associations between the online tasks and various demographic factors, HIV disease variables, neurocognitive domain scores, and health literacy tasks within the HIV+ group. In an effort to limit Type I error, such analyses were only conducted for online task variables that showed significant HAND effects. Given the relatively small sample sizes and corresponding risk of Type II error, the critical α was set at 0.05. All analyses were conducted using JMP version 11.2.

Results

Table 1 shows that three study groups were broadly comparable on sociodemographic factors, with an exception of a larger proportion of men in the two HIV+ samples (ps < .05). The HIV+ participants were also more likely to have a history of Major Depressive Disorder (p < .05) and be engaged in health care (p < .05). None of these factors were related to the primary Internet task variables of interest (ps > .10) and were therefore not included as covariates. As can be seen in Table 2, participants with HAND were less likely to use a home computer, use the Internet daily, use the Internet for ADLS, and were more likely to report difficulties using the Internet (ps < .05). There were not differences in smartphone ownership, computer-related anxiety, or eHEALS (ps > .10).

Table 1.

Demographic and clinical characteristics of the study cohort

Variable HIV−1 (n = 21) HAND−2 (n = 27) HAND+3 (n = 19) Group effects*
Sociodemographics
 Age (years) 45.3 (14.5) 47.3 (9.2) 47.0 (9.2) —
 Education (years) 13.7 (2.1) 13.6 (2.9) 12.8 (2.0) —
 Sex (% women) 28.6 11.1 5.3 1 > 2
 Ethnicity (%) —
  Caucasian 47.6 55.6 52.6
  Hispanic 23.8 22.2 15.8
  Black 23.8 14.8 26.3
  Other 4.8 7.4 5.2
 Employed (%) 61.9 55.6 31.6 —
Psychiatric
 Major depression (%) 23.8 63.0 68.4 1 > 2, 3
 Generalized anxiety (%) 14.3 3.9 15.8 —
 POMS total (of 200) 39.1 (36.1) 43.2 (26.3) 54.1 (31.0) —
 Alcohol use disorder (%) 33.3 51.9 63.2 —
 Substance use disorder 4.8 25.9 15.8 —
Medical
 Healthcare provider (% engaged) 71.4 100 100 1 < 2, 3
 Hepatitis C (%) 6.7 11.5 5.3 —
 AIDS (%) — 60.0 61.1 —
 Duration of HIV (years) — 13.1 (9.3) 15.0 (8.8) —
 Nadir CD4 (cells/μL) — 198.3 (210.5) 224.8 (230.3) —
 Current CD4 (cells/μL) — 632.2 (311.1) 620.6 (307.2) —
 Plasma HIV RNA (% detectable) — 4.0 26.3 2 < 3
 ART (% prescribed) — 96.0 94.4 —
Open in a new tab

Note: POMS = Profile of Mood States; ART = antiretroviral therapies. *p<.05.

Table 2.

Computer and Internet use characteristics of the study groups

Variable HIV−1 (n = 21) HAND−2 (n = 27) HAND+3 (n = 19) p-Value Group effects
General computer use
 Use home computer (%) 95.2 88.9 73.7 .123 —
 Own smartphone (%) 70.0 88.9 63.2 .103 —
 Use WWW daily (%) 90.5 66.7 42.1 .005 1 > 3
 Use online pharmacy (%) 10.0 44.4 36.8 .037 1 > 2, 3
 Use online medical charts (%) 20.0 63.0 47.4 .014 1 < 2, 3
 Difficulties using computer (%) 23.8 11.1 26.3 .363 —
 Anxious using computer (%) 33.3 11.1 31.6 .130 —
eHEALS
 Internet literacy for health 32.1 (8.1) 31.7 (6.5) 31.3 (5.6) .920 —
 Internet important for health 8.4 (1.4) 7.5 (2.4) 8.1 (2.1) .272 —
 Internet use for health 4.0 (1.2) 3.7 (1.2) 3.8 (1.0) .760 —
Test of online pharmacy skillsa
 Total correct (of 5) 3.5 (1.0) 3.5 (1.5) 1.8 (1.4) .000 1, 2 > 3
 Completion time (min) 11.0 (4.2) 10.3 (3.4) 10.0 (4.7) .787 —
 Total errors (no.) 2.8 (1.3) 2.5 (2.2) 3.0 (1.7) .668 —
Test of online health records navb
 Total correct (of 4) 3.1 (0.8) 3.0 (0.8) 2.1 (1.0) .003 1,2 > 3
 Completion time (min) 5.3 (3.5) 5.2 (2.1) 4.34 (1.9) .437 —
 Total errors (no.) 1.9 (1.5) 1.7 (0.9) 2.1 (1.2) .353 —
Open in a new tab

Notes: HAND = HIV-associated Neurocognitive Disorders. Nav = navigation; WWW = World Wide Web.

aHIV−, n = 16; HAND−, n = 19; and HAND+, n = 17.

bHIV−, n = 20, HAND−, n = 24, and HAND+, n = 17.

Test of Online Pharmacy Skills

Table 2 shows the significant effects of HAND on TOPS total correct, with the HAND subjects performing significantly worse than HIV− (Hedges’ g = −1.4) and HIV+ (Hedges’ g = −1.1) subjects without HAND (ps < .05). Figure 1 illustrates that none of the HAND+ participants successfully completed all TOPS items (vs. 29% of the HAND−). There were no group effects for completion time or log-in errors (ps > .10). In the HIV+ group as a whole (n = 36), worse TOPS total scores were significantly related to lower levels of education (correlation = .49, p = .003), but not other demographic variables or psychiatric diagnoses displayed in Table 1 (ps > .10). With regard to HIV disease and medical variables, lower TOPS performance was associated with higher HIV RNA in plasma (correlation = −0.47, p = .006), but not to other factors (all ps > .10). Lower TOPS scores were significantly related to less frequent computer and Internet use, and not owning a smartphone (ps < .05), but not to eHEALS, online pharmacy use, general difficulties using computers, or anxiety related to computer use (all ps > .10). Figure 2 shows that TOPS was correlated in the expected direction with most of the neurocognitive domains (ps < .05), with the exception of attention/working memory (p > .10). Figure 3 shows that TOPS total correct was also significantly correlated with numeracy, TOHFLA reading comprehension, the MMT-R, and Newest Vital Sign.

Fig. 1.

Fig. 1.

Open in a new tab

Proportion of the study groups who successfully completed the individual items on the (a) test of online pharmacy skills and (b) test of online health records navigation.

Fig. 2.

Fig. 2.

Open in a new tab

Correlations between neurocognitive domains and online pharmacy (TOPS; n = 36) and health records (TOHRN; n = 41) total scores in the HIV+ group. *p < .05. TOPS = test of online pharmacy skills; TOHRN = test of online health records navigation.

Fig. 3.

Fig. 3.

Open in a new tab

Correlations between standard health literacy tasks and the online pharmacy (n = 36) and health records (n = 41) total scores in the HIV+ group. *p < .05.

Test of Online Health Records Navigation

As shown in Table 2, there was an omnibus effect of study group on the TOHRN total correct measure (p > .05), which was driven by poorer performance in the HAND group when compared with the HIV− (Hedges’ g = −1.1) and HIV+ (Hedges’ g = −1.0) participants without HAND (ps < .05). Figure 1b shows that only 5.8% of the HAND+ participants successfully completed all TOHRN items (vs. 36.8% of the HAND−). There were no group differences in terms of completion time or initial registration errors (ps > .10). In the HIV+ group as a whole (n = 40), worse TOHRN total scores were related to lower levels of education (correlation = .40, p = .01) and significantly related to higher HIV RNA in plasma (correlation = −.32, p = .045), but not any other demographics, medical, or psychiatric variables shown in Table 1 (ps > .10). Lower TOHRN total accuracy scores were associated with less frequent Internet use, general difficulties using computers, and anxiety related to computer use (ps < .05), but not to eHEALS variables or experience using online health managers (ps > .10). Figure 2 shows that TOHRN scores were not correlated with motor skills, processing speed, or verbal fluency (ps > .10), but showed medium-to-large associations with attention/working memory, learning, memory, and executive functions (ps < .05). Finally, TOHRN total correct was significantly correlated with several of our health literacy tests, including numeracy, TOHFLA reading comprehension, the MMT-R, and Newest Vital Sign (see Fig. 3).

Discussion

Results of this preliminary study suggest that individuals with HAND experience marked difficulties accurately completing routine online healthcare tasks. The effect sizes for the adverse impact of HAND on these online health-related behaviors were quite large, suggesting they may be of some clinical relevance. Importantly, the HAND effects were not better explained by demographic factors, histories of mood and substance use disorders, HIV disease severity, and co-infection with hepatitis C. These findings are consistent with prior research suggesting that individuals with HAND are at risk for low health literacy (Morgan et al., 2015) and difficulties with HIV health-related behaviors, such as medication adherence (Hinkin et al., 2002). Notably, less than 10% of the HAND group was successfully able to complete all of the prescribed Internet tasks. Indeed, failure rates on individual items, such as scheduling an appointment with the treating physician, were surprisingly high across all groups. This was especially since our team made considerable efforts during early piloting phases to ensure that the sites were straightforward, intuitive, and representative. These findings suggest that Internet-based health-related behaviors such as these may be highly vulnerable to disruption by neurocognitive impairment.

The HAND group was also less likely to use the Internet on a daily basis. Post hoc analyses within the HIV+ group showed that inclusion of Internet use frequency alongside HAND group status in a multiple regression predicting the TOPS and TOHRN total scores did not dampen the HAND effects (ps remained < .05). Thus, it may be that individuals with HAND are simply less likely to use the Internet regularly due to frustration with the navigation process. Not surprisingly, the HIV+ groups were more likely to have used online pharmacy and electronic health records than their seronegative counterparts. Despite that conservative bias, the effects of HAND persisted on these tasks. The online tasks were reliably associated with self-perceived difficulties using the Internet in daily life, thereby providing some direct evidence of their ecological relevance.

A series of correlational analyses with established measures of health literacy provided evidence of convergent validity for the tests of online health-related behaviors in the HIV+ sample. Poorer performance on the online pharmacy and health records navigation tasks was moderately associated with fewer years of education and lower scores on well-validated measures of health literacy, including the Expanded Numeracy Scale, TOFHLA Reading Comprehension, MMT-R, and the Newest Vital Sign. These correlational data suggest that online health-related behaviors are at least partly dependent on both basic (e.g., numeracy) and higher-order (e.g., understanding and appraisal of health-related information) aspects of health literacy. Moreover, difficulties accurately navigating the online health-related tasks were associated with higher plasma viral loads. Although correlational, this linkage suggests that online behaviors may influence HIV health outcomes. Post hoc analyses in the HIV+ group showed that the HAND effect on both Internet tasks persisted when HIV RNA was included in the models (ps < .05).

There was some variability in the specific cognitive domains that were associated with online pharmacy and health records navigation in the HIV+ group. Lower accuracy on both Internet-based tasks was associated with poorer learning (i.e., acquisition of new verbal and visual information), memory (i.e., delayed recall of newly learned verbal and visual information), and executive functions (i.e., cognitive flexibility and risky decision making). Such findings are consistent with prior research suggesting that HIV-associated deficits in episodic memory and executive functions are among the strongest and most reliable predictors of everyday functioning, including health-related behaviors (Hinkin et al., 2002; Woods et al., 2009). The online pharmacy task was additionally associated with measures of basic motor skills, psychomotor speed, and verbal fluency, perhaps suggesting that the rate of processing is important for this particular paradigm. Indeed, performance on the pharmacy task was associated with nearly all of the neurocognitive domains, akin to the findings of Morgan and colleagues (2015) in which a multidimensional assessment of higher-order health literacy was associated with multiple cognitive domains, most notably verbal fluency and attention/working memory. These speeded factors were not associated with performance on the health records task, which was instead correlated with measures of basic attention and working memory (Morgan et al., 2015). Future work may focus on relationships with other neurocognitive functions that have shown sensitivity to real-world problems in HIV disease, such as prospective memory (Woods et al., 2009).

The findings from this study should be interpreted with consideration of its methodological limitations. First and foremost, this study used a cross-sectional design with relatively small sample sizes that may have been underpowered to detect more modest between- and within-group effects. Second, despite our efforts to minimize Type I error (e.g., by limiting the online task variables examined), we cannot rule out the possibility that some of the correlational analyses may be false positives, especially given the number of statistical tests conducted on the clinicodemographic correlates of online task performance within the HIV+ group. However, the Type I error risk must also be interpreted in the context of the small sample size (and thus corresponding risk of Type II error) and the interesting pattern of correlational findings. Third, in terms of psychometrics, a few of the online task items (e.g., email message opening on TOHRN) had ceiling effects, which may have limited the sensitivity of the paradigms. Finally, the external validity of these findings to other aspects of online health behaviors (e.g., tablet operation) and persons with different sociodemographic characteristics (e.g., HIV+ women, as well as both younger and older HIV+ adults) remains to be determined.

Future studies may wish to examine the ecological relevance (e.g., incremental diagnostic accuracy) and cognitive mechanisms (e.g., multitasking) of a more comprehensive series of Internet tasks of everyday functioning that will also include other measures of online health care (e.g., provider and insurance searches), communication (e.g., email and social networking), transportation (e.g., planning a trip on a mass transit system), and household activities (e.g., shopping and banking). The relevance of these measures in predicting retention in HIV care and other applied health-related behaviors (e.g., adherence and appointment attendance) might also be worthwhile. Such research may directly influence the development of targeted cognitive neurorehabilitation efforts designed to improve Internet-based functional deficits (Kalichman et al., 2006) and health outcomes for persons living with HIV disease.

Supplementary Material

Supplementary material is available at Archives of Clinical Neuropsychology online.

Funding

This research was supported by National Institutes of Health grants R21-MH098607, R01-MH073419, T32-DA31098, L30-DA034362, L30-DA032120, K23-DA037793, P50-DA026306, U24-MH100928, and P30-MH62512.

Conflict of Interest

None declared.

Supplementary Material

Supplementary Data
supp_31_2_176__index.html (901B, html)

Acknowledgements

The San Diego HIV Neurobehavioral Research Program [HNRP] group is affiliated with the University of California, San Diego, the Naval Hospital, San Diego, and the Veterans Affairs San Diego Healthcare System, and includes Director: Igor Grant, M.D.; Co-Directors: J. Hampton Atkinson, M.D., Ronald J. Ellis, M.D., Ph.D., and J. Allen McCutchan, M.D.; Center Manager: Thomas D. Marcotte, Ph.D.; Jennifer Marquie-Beck, M.P.H.; Melanie Sherman; Neuromedical Component: Ronald J. Ellis, M.D., Ph.D. (P.I.), J. Allen McCutchan, M.D., Scott Letendre, M.D., Edmund Capparelli, Pharm.D., Rachel Schrier, Ph.D., Debra Rosario, M.P.H., Neurobehavioral Component: Robert K. Heaton, Ph.D. (P.I.), Mariana Cherner, Ph.D., Jennifer E. Iudicello, Ph.D., David J. Moore, Ph.D., Erin E. Morgan, Ph.D., Matthew Dawson; Neuroimaging Component: Terry Jernigan, Ph.D. (P.I.), Christine Fennema-Notestine, Ph.D., Sarah L. Archibald, M.A., John Hesselink, M.D., Jacopo Annese, Ph.D., Michael J. Taylor, Ph.D.; Neurobiology Component: Eliezer Masliah, M.D. (P.I.), Cristian Achim, M.D., Ph.D., Ian Everall, FRCPsych., FRCPath., Ph.D. (Consultant); Neurovirology Component: Douglas Richman, M.D. (P.I.), David M. Smith, M.D.; International Component: J. Allen McCutchan, M.D. (P.I.); Developmental Component: Cristian Achim, M.D., Ph.D. (P.I.), Stuart Lipton, M.D., Ph.D.; Participant Accrual and Retention Unit: J. Hampton Atkinson, M.D. (P.I.); Data Management Unit: Anthony C. Gamst, Ph.D. (P.I.), Clint Cushman (Data Systems Manager); Statistics Unit: Ian Abramson, Ph.D. (P.I.), Florin Vaida, Ph.D., Reena Deutsch, Ph.D., Anya Umlauf, M.S. The views expressed in this article are those of the authors and do not reflect the official policy or position of the Department of the Navy, Department of Defense, nor the United States Government. The authors thank John DeLuca and Yael Goverover for their guidance in the early conceptualization of the Internet tasks and Donald Franklin and Stephanie Corkran for their help with data processing.

Contributor Information

Collaborators: The HIV Neurobehavioral Research Program (HNRP) Group, Igor Grant, J. Hampton Atkinson, Ronald J. Ellis, J. Allen McCutchan, Thomas D. Marcotte, Jennifer Marquie-Beck, Melanie Sherman, Ronald J. Ellis, J. Allen McCutchan, Scott Letendre, Edmund Capparelli, Rachel Schrier, Debra Rosario, Robert K. Heaton, Mariana Cherner, Jennifer E. Iudicello, David J. Moore, Erin E. Morgan, Matthew Dawson, Terry Jernigan, Christine Fennema-Notestine, Sarah L. Archibald, John Hesselink, Jacopo Annese, Michael J. Taylor, Eliezer Masliah, Cristian Achim, Ian Everall, Douglas Richman, David M. Smith, J. Allen McCutchan, Cristian Achim, Stuart Lipton, J. Hampton Atkinson, Anthony C. Gamst, Clint Cushman, Ian Abramson, Florin Vaida, Reena Deutsch, Anya Umlauf, John DeLuca, Yael Goverover, Donald Franklin, and Stephanie Corkran

References

  1. Antinori A., Arendt G., Becker J. T., Brew B. J., Byrd D. A., Cherner M. et al. (2007). Updated research nosology for HIV-associated neurocognitive disorders. Neurology, 69, 1789–1799. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Bechara A. (1994). Iowa Gambling Task. Lutz, FL: Psychological Assessment Resources. [Google Scholar]
  3. Blackstock O. J., Cunningham C.O., Haughton L. J., Garner R. Y., Norwood C., Horvath KJ (2014). General and health-related Internet use among an urban, community-based sample of HIV-positive women: Implications for intervention development. AIDS Care, 20, 1–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Brandt J., Benedict R. H. B. (2001). Hopkins Verbal Learning Test-Revised (HVLT-R). Lutz, FL: Psychological Assessment Resources, Inc. [Google Scholar]
  5. Carey C.L., Woods S.P., Gonzalez R., Conover E., Marcotte T.D., Grant I. et al. (2004). Predictive validity of global deficit scores in detecting neuropsychological impairment in HIV infection. Journal of Clinical and Experimental Neuropsychology, 26, 307–319. [DOI] [PubMed] [Google Scholar]
  6. Carey M. P., Schroder K. E. E. (2002). Development and psychometric evaluation of the brief HIV knowledge questionnaire. AIDS Education and Prevention: Official Publication of the International Society for AIDS Education, 14, 172–182. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Cysique L.A., Maruff P., Darby D., Brew B.J. (2006). The assessment of cognitive function in advanced HIV-1 infection and AIDS dementia complex using a new computerised cognitive test battery. Archives of Clinical Neuropsychology: The Official Journal of the National Academy of Neuropsychologists, 21, 185–194. [DOI] [PubMed] [Google Scholar]
  8. Davis T. C., Crouch M. A., Long S. W. (1991). Rapid assessment of literacy levels of adult primary care patients. Family Medicine, 23, 433–435. [PubMed] [Google Scholar]
  9. Delis D. C., Kaplan E., Kramer J. (2001). Delis-Kaplan Executive Function System (D-KEFS). San Antonio, TX: Psychological Corporation. [Google Scholar]
  10. Dorner T. E., Schulte-Hermann K., Zanini M., Leichsenring B., Stefanek W (2014). Health literacy, source of information and impact on adherence to therapy in people living with HIV. Journal of the International AIDS Society, 17 (4 Suppl. 3), 195–199. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Goverover Y., O'Brien A.R., Moore N.B., DeLuca J. (2010). Actual reality: A new approach to functional assessment in persons with multiple sclerosis. Archives of Physical Medicine and Rehabilitation, 91, 252–260. [DOI] [PubMed] [Google Scholar]
  12. Gronwall D. M. A. (1977). Paced auditory serial addition task: A measure of recovery from concussion. Percept Motor Skill, 44, 367–373. [DOI] [PubMed] [Google Scholar]
  13. Heaton R. K. (1993). Wisconsin Card Sorting Test—Computer version 4. Lutz, FL: Psychological Assessment Resources. [Google Scholar]
  14. Heaton R. K., Clifford D., Franklin D. et al. (2010). HIV-associated neurocognitive disorders persist in the era of combination antiretroviral therapy: CHARTER Study. Neurology, 75, 2087–2096. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Hinkin C.H., Castellon S.A., Durvasula R.S., Hardy D.J., Lam M.N., Mason K.I. et al. (2002). Medication adherence among HIV+ adults: Effects of cognitive dysfunction and regimen complexity. Neurology, 59, 1944–1950. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Jones D., Cook R., Rodriguez A., Waldrop-Valverde D. (2013). Personal HIV knowledge, appointment adherence and HIV outcomes. AIDS and Behavior, 17, 242–249. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Kalichman S.C., Cain D., Cherry C., Pope H., Eaton L., Kalichman M.O. (2005). Internet use among people living with HIV/AIDS: Coping and health-related correlates. AIDS Patient Care and STDs, 19, 439–448. [DOI] [PubMed] [Google Scholar]
  18. Kalichman S.C., Cherry C., Cain D., Pope H., Kalichman M., Eaton L. et al. (2006). Internet-based health information consumer skills intervention for people living with HIV/AIDS. Journal of Consulting and Clinical Psychology, 74, 545–554. [DOI] [PubMed] [Google Scholar]
  19. Kalichman S. C., Ramachandran B., Catz S. (1999). Adherence to combination antiretroviral therapies in HIV patients of low health literacy. Journal of General Internal Medicine, 14, 267–273. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Kalichman S., Rompa D. (2000). Functional health literacy is associated with health status and health-related knowledge in people living with HIV-AIDS. Journal of Acquired Immune Deficiency Syndromes, 25, 337–344. [DOI] [PubMed] [Google Scholar]
  21. Kløve H. (1963). Grooved pegboard. Lafayette, IN: Lafayette Instruments. [Google Scholar]
  22. Kunst H., Groot D., Latthe P.M., Latthe M., Khan K.S. (2002). Accuracy of information on apparently credible websites: Survey of five common health topics. BMJ (Clinical Research ed.), 324, 581–582. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Lee S.Y., Stucky B.D., Lee J.Y., Rozier R.G, Bender D.E. (2010). Short assessment of health literacy-Spanish and English: A comparable test of health literacy for Spanish and English speakers. Health Services Research, 45, 1105–1120. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Lipkus I. M., Samsa G., Rimer B. K. (2001). General performance on a numeracy scale among highly educated samples. Medical Decision Making: An International Journal of the Society for Medical Decision Making, 21, 37–44. [DOI] [PubMed] [Google Scholar]
  25. Mayben J. K., Giordano T. P. (2007). Internet use among low-income persons recently diagnosed with HIV infection. AIDS Care, 19, 1182–1187. [DOI] [PubMed] [Google Scholar]
  26. McNair D. M., Lorr M., Droppleman L. F. (1981). Manual for the profile of mood states. San Diego, CA: Educational and Industrial Testing Services. [Google Scholar]
  27. Morgan E.E., Iudicello J.E., Cattie J.E., Blackstone K., Grant I., Woods S.P. et al. (2015). Neurocognitive impairment is associated with lower health literacy among persons living with HIV infection. AIDS and Behavior, 19, 166–177. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Norman C. D., Skinner H. A. (2006). eHEALS: The eHealth Literacy Scale. Journal of Medical Internet Research, 8, e27. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Parker R.M., Baker D.W., Williams M.V., Nurss J.R. (1995). The test of functional health literacy in adults: A new instrument for measuring patients’ literacy skills. Journal of General Internal Medicine, 10, 537–541. [DOI] [PubMed] [Google Scholar]
  30. Psychological Corporation. (1997). Wechsler Adult Intelligence Scale—3rd edition (WAIS-III). San Antonio, TX: Psychological Corporation. [Google Scholar]
  31. Reitan R. M., Wolfson D. (1985). The Halstead-Reitan Neuropsychological Test Battery: Theory and clinical interpretation. Tucson, AZ: Neuropsychology Press. [Google Scholar]
  32. Schnall R., Wantland D., Velez O., Cato K., Jia H. (2014). Feasibility testing of a web-based symptom self-management system for persons living with HIV . The Journal of the Association of Nurses in AIDS Care, 25, 364–371. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Scott J.C., Woods S.P., Vigil O., Heaton R.K., Schweinsburg B.C., Ellis R.J. et al. (2011). A neuropsychological investigation of multitasking in HIV infection: Implications for everyday functioning. Neuropsychology, 25, 511–519. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. The Patient Protection and Affordable Care Act (PPACA). (2010). Pub. L. No. 111-148, 124 Stat. 119.
  35. Thomas S., Shuter J. (2010). Internet access and usage in a sample of inner-city HIV-infected patients. The Journal of the Association of Nurses in AIDS Care, 21, 444–448. [DOI] [PubMed] [Google Scholar]
  36. Wawrzyniak A. J., Ownby R. L., McCoy K., Waldrop-Valverde D. (2013). Health literacy: Impact on the health of HIV-infected individuals. Current HIV/AIDS Reports, 10, 295–304. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Weiss B.D., Mays M.Z., Martz W., Castro K.M., DeWalt D.A., Pignone M.P. (2005). Quick assessment of literacy in primary care: The Newest Vital Sign. Annals of Family Medicine, 3, 514–522. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Woods S.P., Dawson M.S., Weber E., Gibson S., Grant I., Atkinson J.H. et al. (2009). Timing is everything: Antiretroviral non-adherence is associated with impairment in time-based prospective memory. Journal of the International Neuropsychological Society, 15, 42–52. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Woods S.P., Rippeth J.D., Frol A.B., Levy J.K., Ryan E., Soukup V.M. et al. (2004). Interrater reliability of clinical ratings and neurocognitive diagnoses in HIV. Journal of Clinical and Experimental Neuropsychology, 26, 759–778. [DOI] [PubMed] [Google Scholar]
  40. World Health Organization. (1998). Composite International Diagnostic Interview (CIDI, version 2.1). Geneva, Switzerland: World Health Organization. [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary Data
supp_31_2_176__index.html (901B, html)
supp_acv090_acv090supp.docx (1.6MB, docx)

Articles from Archives of Clinical Neuropsychology are provided here courtesy of Oxford University Press

RESOURCES