Abstract
Objective:
HIV infection and current substance use (SU) are linked to cognitive and functional deficits, yet findings on their combined effects are mixed. Neurocognitive intra-individual variability, measured as dispersion of scores across a neuropsychological battery, is associated with worse cognitive outcomes and functional deficits among HIV+ adults, but has not been studied in the context of HIV+ adults with current SU. We hypothesized that, among HIV+ adults, current SU would be associated with greater dispersion, and that greater dispersion would be associated with worse medication adherence and that this relationship would be worse among substance users.
Method:
Forty HIV+ adults completed neuropsychological, psychiatric/SU, and medical evaluations and an electronic medication adherence measure. General linear models evaluated the main effect of SU status on neurocognitive dispersion, and models stratified by SU status evaluated the effect of dispersion on medication adherence, adjusting for relevant covariates.
Results:
The SU+ group showed greater dispersion than the SU− group t(38)=2.74;p=.049, d=0.81 but this association did not survive multiple comparisons. Stratified analyses indicated a negative relationship between dispersion and medication adherence among the SU+ group but not in the SU− group, however this effect was reduced after accounting for depressive symptoms.
Conclusions:
We found preliminary evidence that current SU is associated with greater neurocognitive dispersion among HIV+ adults. SU and neurocognitive dispersion may have a synergistic effect on medication adherence, however this effect is largely accounted for by depressive symptoms. Future research should examine progression of dispersion in HIV and consequent neurocognitive and functional deficits in those with current SU.
Keywords: HIV, substance use, HAND, intra-individual variability, dispersion
Introduction
HIV infection and substance use co-occur at alarming rates, with estimates indicating that 80% of HIV-seropositive (HIV+) adults have used illicit substances in their lifetime (Meade, Conn, Skalski, & Safren, 2011; SAMHSA, 2010). Additionally, almost 50% of HIV+ adults meet diagnostic criteria for a comorbid substance use disorder (SUD; Bing et al., 2001; Galvan, Burnam, & Bing, 2003; Rabkin, McElhiney, Ferrando, Van Gorp, & Lin, 2004). These numbers are concerning given that both HIV and substance use can have a detrimental impact on the central nervous system (CNS; Hauser et al., 2007; Lundqvist, 2010; Turchan et al., 2001; Venkatesan, Nath, Ming, & Song, 2007). However, findings on the combined impact of HIV and substance use on cognitive functioning are equivocal, with some studies indicating a combined or synergistic effect (Meade et al., 2011; Rippeth et al., 2004) and while others do not (Basso & Bornstein, 2003; Byrd et al., 2011; Durvasula et al., 2000). Yet clinically, HIV+ adults with current substance use raise greater public health concerns compared to those without current substance use. For instance, substance use is associated with higher risk of HIV transmission, less utilization of HIV health services, and worse overall health-related quality of life compared to HIV+ adults without current substance use (Korthuis et al., 2008; Lucas, Cheever, Chaisson, & Moore, 2001).
While HIV and substance use may increase risk for detrimental health outcomes, their combined impact on cognition remains unclear. One reason for this may be due to the fact that the vast majority of prior research in this area has focused on comparing mean performance between groups on traditional outcome measures of cognition (i.e., test scores), as opposed to investigating the intra-individual variability (IIV) in performance within individuals. Generally, IIV can be defined as either variability across multiple test scores (dispersion) such as on performance across a neuropsychological battery (MacDonald, Nyberg, & Bäckman, 2006), or as variability in performance within a test (inconsistency) such as on a reaction time task. Studies indicate that IIV captures unique variance in cognitive abilities that is not reflected from measuring absolute or mean level of performance (Ram, Rabbitt, Stollery, & Nesselroade, 2005). In addition, IIV may be a sensitive behavioral indicator of compromised CNS functioning, particularly in aging (Bielak, Hultsch, Strauss, MacDonald, & Hunter, 2010; Bielak, Cherbuin, Bunce, & Anstey, 2014; Christensen et al., 1999) and can differentiate between neurodegenerative disorders (Burton, Strauss, Hultsch, Moll, & Hunter, 2006; Murtha, Cismaru, Waechter, & Chertkow, 2002).
In the context of HIV, cross-sectional studies have demonstrated that greater inconsistency is associated with worse global cognitive functioning, immunological dysfunction (i.e., nadir CD4 and viral load) and poorer medication adherence (Ettenhofer, Foley, Castellon, & Hinkin, 2010; Levine et al., 2008). Greater dispersion is associated with increased need for assistance in activities of daily living and suboptimal medication adherence (Morgan, Woods, Grant, & Group, 2012), and with older age (Morgan et al., 2011). Recent longitudinal studies found that increases in dispersion over time was associated with worse medication adherence (Thaler et al., 2015), white matter changes (Jones et al., 2017), and incident cognitive impairment and greater mortality risk (Anderson et al., 2018). Of note, the relationship between IIV and outcomes remained after controlling for mean performance. Thus, the available studies provide evidence that IIV may be a marker for cognitive and functional deficits among HIV+ adults, and may provide important information not fully captured from traditional indices (i.e., mean performance) of neurocognitive functioning.
Markers of IIV may provide a subtle measure of CNS dysfunction to reliably detect the neurocognitive effect of substance use among HIV+ adults. Most of the aforementioned studies of IIV among HIV+ cohorts either excluded or controlled for substance use. There is some indication that current substance use may exacerbate IIV. Greater IIV is observed among current and past methamphetamine users (Morgan et al., 2014; Fassbender, Lesh, Ursu, & Salo, 2015). However, it is unclear whether current substance use is associated with greater IIV among HIV+ adults. Similarly, while greater IIV is associated with worse medication adherence among HIV+ adults (Ettenhofer et al., 2010; Thaler et al., 2015), whether substance use strengthens this relationship remains unclear.
The aim of the current study was to evaluate the role of neurocognitive dispersion, current substance use, and medication adherence among a sample of HIV+ adults. First, we hypothesized that HIV+ adults with current substance use would have greater neurocognitive dispersion than those without current substance use. Second, we hypothesized that greater neurocognitive dispersion would be associated with worse medication adherence and that this relationship would be worse among substance users.
Methods
Participants
Our sample included 40 HIV+ men and women who were evaluated as part of their enrollment in the NIMH-funded Medication Adherence Study (MAS; Grant #: K23MH079718; PI: M. Rivera Mindt, Ph.D., ABPP), a study of HIV and cognitive/functional outcomes among Latinx. Participants were recruited primarily through community outreach and self-referral in New York City, particularly in East Harlem. In addition, participants were also recruited from other HIV-related studies at the Mount Sinai Medical Center.
Inclusion criteria for the study comprised stable antiretroviral therapy for at least 12 weeks prior to the date of evaluation and identification as either Latino/a or Non-Hispanic white. Exclusion criteria included a diagnosis of a psychiatric condition associated with neurocognitive compromise (i.e., schizophrenia, psychosis, bipolar disorder), particular neuromedical conditions (i.e., epilepsy, traumatic brain injury with a loss of consciousness >60 minutes, brain cancer or tumor, neurosurgery, Lupus, Multiple Sclerosis, Parkinson’s disease). The substance use (SU+) group included participants that met criteria for a DSM-IV diagnosis of current substance use disorder (abuse and/or dependence within the past 12 months). The SU− group included all participants who did not meet criteria for any current or past substance use disorder, nor did they have a positive urine toxicology on the day of the evaluation. Furthermore, SU− participants were matched, as best as possible, to SU+ participants who were selected from a pool of individuals (N=162). Factors considered in this matching process included: age, total years of education, gender, ethnicity, CD4+ T cell count, premorbid intellectual function (i.e., Wide Range Achievement Test-3 [WRAT3] Reading subtest test score; Wilkinson, 1993) and percentage detectable HIV viral load. This matching procedure has been proposed to be used among confounded cohorts of HIV+ adults in order to improve ability to isolate the cognitive and neurologic effects of HIV and substance use (Byrd et al., 2013). Table 1 summarizes the demographic and clinical characteristics of the sample.
Table 1.
Demographic, Clinical, and Functional Characteristics by Group (N=40)
| SU+ (n = 17) |
SU− (n = 23) |
t/X2 | |
|---|---|---|---|
| Demographic Characteristics | |||
| Age; M (SD) | 44.29 (9.85) | 47.74 (9.29) | −1.12 |
| Education; M (SD) | 12.06 (2.65) | 13.56 (3.05) | −1.66 |
| Male; % (n) | 82.35 (14) | 60.87 (14) | 2.15 |
| Race/Ethnicity; % (n) | .05 | ||
| Non-Hispanic White | 29.41 (5) | 26.09 (6) | |
| Hispanic/Latino | 70.59 (12) | 73.91 (17) | |
| WRAT-III Est. VIQ; M (SD) | 86.06 (19.73) | 95.83 (11.68) | −1.82 |
| Global NC; M (SD) | 39.00 (7.60) | 43.62 (7.14) | 2.14* |
| HIV Characteristics | |||
| CD4 Count; Median (IQR) | 467 (277-582) | 484.5 (273-790) | 0.45 |
| % AIDS | 5.88 (1) | 0 | 1.21 |
| Nadir CD4 | 195 (67.5-263) | 67 (18-170) | −1.78 |
| Plasma HIV viral load (log10); | 1.38 (1.38-1.88) | 1.38 (1.38-1.67) | 0.26 |
| Median (IQR) | |||
| Substance Use Characteristics | |||
| Positive Urine Toxicology % (n) | |||
| Marijuana | 29 (5) | - | |
| Cocaine | 12 (2) | - | |
| Opiates | 6 (1) | - | |
| Methamphetamine | 0 | - | |
| Barbiturates | 6 (1) | - | |
| Benzodiazepine | 0 | - | |
| Phencyclidine | 13 (2) | - | |
| Substance Use Disorder | |||
| Alcohol | 47 (8) | - | |
| Cocaine | 24 (4) | - | |
| Methamphetamine | 6 (1) | - | |
| Marijuana Opiates | 24 (4) | - | |
| Sedatives | 6 (1) | - | |
| Other | 0 | - | |
| Hallucinogen | 12 (2) | - | |
| Inhalants | 6 (1) | - | |
| 0 | - |
Note. NC = Neurocognitive; WRAT-III = Wide Range Achievement Test-Reading subtest, 3rd Edition.
= p < .05; Substance Use Disorder reflects current diagnosis.
Procedure and Measures
Once consent was obtained, participants underwent a comprehensive battery of neuromedical, neuropsychological, and psychiatric evaluations. Both Institutional Review Boards at the Mount Sinai Medical Center and Fordham University approved all procedures.
Psychiatric and Substance Use Evaluation
Participants completed the Beck Depression Inventory-Second Edition (BDI-II) to evaluate depressive symptoms over the past two weeks (Beck, Steer, & Brown, 1996). In order to minimize the impact of somatic complaints associated with medical symptoms of HIV, the BDI-II Fast Screen (BDI-FS) score (derived from 7 items of the BDI) was used (Beck, Steer, & Brown, 2000; Krefetz, Steer, Jermyn, & Condoluci, 2004).
Substance use disorder diagnoses were obtained with the Composite International Diagnostic Interview, 2.1. (CIDI 2.1; World Health Organization, 1997), which provides diagnoses based on criteria from the Diagnostic and Statistical Manual – Fourth Edition, Text Revision (DSM-IV-TR; (American Psychiatric Association, 2000). In addition, a urine toxicology screening was conducted on the day of their visit in order to screen recent illicit substance use. It screened for amphetamine, barbiturates, benzodiazepines, cannabinoids, cocaine, opiates, phencyclidine, methadone, and propoxyphene. Illicit status was determined by review of prescribed medications. If participants were visibly intoxicated on the day of the evaluation, either by alcohol or an illicit substance, their study visit was rescheduled.
Neurocognitive Functioning
Overall neurocognitive functioning was evaluated with a comprehensive neuropsychological battery designed to assess seven cognitive domains: executive functions, processing speed, attention/working memory, learning, memory, verbal fluency, and motor skills. Tests selected for the battery were chosen for their sensitivity to HIV related impairment (Heaton et al., 2011; Woods et al., 2004). Summary scores for each test were transformed to demographically-corrected (i.e., age, education, gender, race) T-scores. Table 2 lists the neurocognitive tests and their normative references. A global neurocognitive average T-score was calculated from the mean of the tests of the battery. In addition to the neurocognitive battery, participants completed the Wide Range Achievement Test-Reading subtest, 3rd Edition (WRAT-III; Wilkinson, 1993) as measure of estimated premorbid IQ.
Table 2.
Neuropsychological Test Battery and Normative Data Sources by Ability Domain
| Neuropsychological domain and tests | Normative data sources |
|---|---|
| Executive functioning | |
| Wisconsin Card Sorting Task-64 Item Version | Kongs, Thompson, Iverson, & Heaton (2000)1,2 |
| Trail Making Test (Part B) | Heaton, Miller, Taylor & Grant (2004)1,2,3,4 |
| Attention/Working memory | |
| WAIS-III Letter Number Sequencing | Heaton, Taylor & Manly (2003)1,2,3,4 |
| PASAT Total Correct | Diehr et al. (2003)1,2,4 |
| Learning | |
| Hopkins Verbal Learning Test-Revised (Total recall) | Benedict et al. (1998)1 |
| Brief Visuospatial Memory Test-Revised (Total recall) | Benedict (1997)1 |
| Memory | |
| Hopkins Verbal Learning Test-Revised (Delayed Recall Trial) | Benedict et al. (1998)1 |
| Brief Visuospatial Memory Test-Revised (Delayed Recall Trial) | Benedict (1997)1 |
| Speed of Information Processing | |
| WAIS-III Digit Symbol | Heaton, Taylor & Manly (2003)1,2,3,4 |
| WAIS-III Symbol Search | Heaton, Taylor & Manly (2003)1,2,3,4 |
| Trail Making Test (Part A) | Heaton, Miller, Taylor & Grant (2004)1,2,3,4 |
| Verbal Fluency | |
| Controlled Oral Word Association Test | Heaton, Miller, Taylor & Grant (2004)1,2,3,4 |
| Fine Motor Skills | |
| Grooved Pegboard Time (dominant hand) | Heaton, Miller, Taylor & Grant (2004)1,2,3,4 |
| Grooved Pegboard Time (nondominant hand) |
Note. Superscript number indicates which normative demographic corrections were made:
Age
Education
Gender
Race
Neurocognitive Intra-Individual Variability
The measure of neurocognitive IIV was operationalized as the dispersion of test scores across the neurocognitive battery. First, the individual standard deviation (ISD) was calculated for each participant across each of their demographically-adjusted T-scores for the individual tests in the battery. Following, the coefficient of variation (CoV) was calculated for each individual in order to adjust for mean neurocognitive performance. CoV was computed as their ISD/Global T Score. Therefore, the resulting CoV ratio score served as our index of neurocognitive dispersion. Higher neurocognitive dispersion scores indicates greater variability across neurocognitive abilities, while lower represents more consistency in performance across neurocognitive abilities. This method followed the same procedures as previous studies (Thaler et al., 2015; Tractenberg & Pietrzak, 2011).
Medication Adherence
Antiretroviral (ARV) medication adherence was measured through the Medication Event Monitoring System (MEMS, AARDEX Group). The MEMS consists of a medication bottle, with a chip in the bottle cap that records the time and date the bottle is opened. MEMS data is more strongly related to objective biological measures of medication adherence (i.e., viral load) compared to self-report or pill count measures (Liu et al., 2001; McNabb, Nicolau, Stoner, & Ross, 2003; Vriesendorp et al., 2007). The main outcome variable was the percentage of doses recorded by the MEMS cap relative to the total number of doses prescribed over the last 30 days. The mean number of days tracked was 36 days (SD = 7.6; range = 27 to 62).
Data Analysis
All neurocognitive variables (i.e., global T-score, the neurocognitive dispersion score) were normally distributed. Comparison analyses between SU+ and SU− groups used t-tests and the Wilcoxon rank sum test for continuous variables (i.e., age, CD4 counts, neurocognitive dispersion, etc.) when appropriate and Fisher’s exact test for binary variables (i.e., ethnicity, gender, etc.). Pearson’s correlations were calculated to evaluate the associations between dispersion and demographic, and HIV characteristics.
In order to identify potential covariates prior to conducting the above-mentioned analyses, a series of bivariate analyses were computed between dispersion and all demographic (age, education, gender, race/ethnicity), psychiatric (BDI-II FS Score), and HIV immunostatus (CD4 count, nadir CD4 count, viral load) variables. To be included as a covariate in the analyses, the variable must have been different between substance use groups (p <.05) and correlated with dispersion (p <.05).
For the first hypothesis, general linear models were conducted to examine the main effect of substance use on dispersion. For the second hypothesis, we ran general linear models stratified by substance use status to evaluate the effect of dispersion on the MEMS-derived continuous variable of percent medication adherence over the last 30 days (% adherence). False discovery rate (FDR) was applied to correct for multiple comparisons (Benjamini, Drai, Elmer, Kafkafi, & Golani, 2001). All models were initially run unadjusted and then adjusted for the depressive symptomatology. All calculations used the statistical package JMP version 13.1 (SAS, 2016).
Results
Overall, the sample was 70% male and 73% Latino/a with a mean age of 46.28 years (SD = 9.56) and a mean education of 12.93 years (SD = 2.96). Three-percent of the entire sample was immunosuppressed (i.e., CD4 count<200), and 29% of the sample had detectable HIV viral load levels (> 40 copies/mL). The sample demonstrated average premorbid intellectual functioning (M = 91.68; SD = 16.15), and low average global neurocognitive functioning (M = 41.40; SD = 7.58). In terms of medication adherence, the sample demonstrated an average adherence rate of 86% (SD = 19.42). Furthermore, in terms of the substance use characteristics of the SU+ group, the most prevalent current substance use diagnoses were for alcohol (47%), cocaine (24%), and cannabis (24%). Urine toxicology, positive for 50% of the SU+ group, revealed recent use of cannabis (29%), phencyclidine (13%), cocaine (12%), and illicit opiates (6%).
Table 1 shows that the SU+ (n = 17) and SU− (n = 23) groups were comparable on all demographic, and HIV immunostatus variables (all p’s>.05). The SU+ group had on average worse global neurocognitive functioning (t(38) = 2.14; p = .039, Cohen’s d = 0.65) compared to the SU− group. Participants differed on current depressive symptomatology. On average, the results of a Wilcoxon rank sum test revealed that the SU+ group endorsed greater symptoms of depression compared to the SU− participants (Z = 2.40; p = .017). Follow up correlational analyses were computed to examine the association between BDI-FS scores with neurocognitive dispersion, which revealed a positive relationship (r = .35; p = .029). Thus, the BDI-FS score was included as covariate in the subsequent analyses. Global neurocognitive functioning was not included as a covariate since it is already represented in the coefficient of variation dispersion score.
Substance Use and Neurocognitive Dispersion
Figure 1 shows the relationship of neurocognitive dispersion by SU group. After controlling for depressive symptomatology, the SU+ group had greater neurocognitive dispersion (M = 0.31, SD = 0.09) compared to the SU− group (M = 0.24, SD = 0.08) with a large effect size noted (Cohen’s d = 0.81; t(38) = 2.04; p = .049). This association did not survive multiple comparison correction. Although the groups differed in global neurocognitive functioning in the descriptive analyses, the difference in global neurocognitive functioning did not remain after controlling for depressive symptoms (t(38) = 1.84; p = .074).
Figure 1.
Relationship of substance use status and neurocognitive dispersion.
* Note: Bars represent 95% confidence intervals.
Neurocognitive Dispersion and Medication Adherence by Substance Use Status
Across the entire sample, there was no relationship between neurocognitive dispersion and medication adherence in both unadjusted (B[SE] = −48.42[34.95], 95% CI [−119.17,22.33], p = .174) and adjusted models (B[SE] = −44.23[30.99], 95% CI [−107.08, 18.631], p = .162). Figure 2 depicts the relationship of neurocognitive dispersion and medication adherence by substance use status. In unadjusted models, there was a negative relationship between neurocognitive dispersion and medication adherence (B[SE] = −75.55[32.07], 95% CI [−140.90,−4.19], p = .039; partial η2 = .25) among the SU+ group, but not within the SU− group (B[SE] = −70.00[64.47], 95% CI [−204.07,64.07], p = .290; partial η2 = .05). However, the effect in the SU+ group was reduced when adjusting for depressive symptomatology (B[SE] = −64.79[31.75], 95% CI [−132.89,3.30], p = .061; partial η2 = .23).
Figure 2.
Relationship between neurocognitive dispersion and medication adherence by substance use status.
* Note: Bars represent 95% confidence intervals.
Discussion
The overall aim of the current study was to investigate the association of current substance use, neurocognitive dispersion, and medication adherence within the context of HIV. Our results provide preliminary evidence that current substance use is associated with greater neurocognitive dispersion even after accounting for relevant covariates (depressive symptomatology). In addition, among substance users, greater dispersion was associated with suboptimal medication adherence although this effect was reduced when accounting for depressive symptoms. Although our main finding did not survive multiple comparison correction, this is likely attributable to reduced power due to our small sample size. Despite the loss of statistical significance, given the large effect size (Cohen’s d = 0.81), these results suggest that current substance use may increase neurocognitive dispersion which in turn may influence medication adherence. Further research with larger samples is needed to fully evaluate these hypotheses with adequate statistical power.
A preliminary finding of this study was that neurocognitive dispersion was the only measure of neurocognitive functioning to independently differentiate between HIV+ adults with and without current substance use disorders. While adults with current substance use disorders demonstrated worse global neurocognitive functioning, this difference was largely explained by depressive symptoms whereas neurocognitive dispersion differed between the groups even after accounting for depressive symptoms. Generally, studies have yielded equivocal results with regard to the additive or synergistic effects of substance use disorders and HIV on neurocognitive functioning (Basso & Bornstein, 2003; Byrd et al., 2011; Durvasula et al., 2000; Meade et al., 2011; Rippeth et al., 2004). However, these equivocal results may be due to a lack of measurement sensitivity due to reliance solely on measures of mean overall neurocognitive performance, and not in measures of intra-individual variability (IIV) such as dispersion of scores from a neuropsychological battery. In contrast, the current study suggests that measures of IIV, such as neurocognitive dispersion, may be more sensitive to detecting neurocognitive changes in HIV+ substance users.
Frontostriatal dysfunction has been implicated as a potential source for greater IIV (Bunce et al., 2010; Bunce et al., 2007; Lövdén et al., 2013; MacDonald, Karlsson, Rieckmann, Nyberg, & Backman, 2012). Specifically, frontal white-matter hyperintensities (Bunce et al., 2010; Bunce et al., 2007) and smaller prefrontal volumes (Lövdén et al., 2013) are associated with greater IIV. Given that frontostriatal dysfunction is well documented in HIV infection (Kumar et al., 2009; Woods, Moore, Weber, & Grant, 2009), this study provides evidence that IIV can provide valuable information about neurocognitive functioning among HIV+ adults. Another potential mechanism may be dysregulation of the dopaminergic system. Both HIV infection and substance use disorders are associated with dysfunction in the dopamine systems of the brain. HIV targets and damages dopamine rich areas of the brain such as the basal ganglia and related structures (e.g., substantia nigra, and caudate nucleus; Purohit, Rapaka, & Shurtleff, 2011). Substances of abuse are well known to increase and dysregulate brain dopamine levels (Purohit et al., 2011). Concomitant substance use in HIV infected adults may also increase dopaminergic dysfunction. Reduced dopamine transporters have been reported in the basal ganglia (i.e., putamen and caudate) of HIV+ adults with current cocaine use compared to those HIV+ non-substance users and HIV− controls (Chang et al., 2008). Furthermore, it is hypothesized that drug-induced increases of dopamine levels in the CNS may increase HIV infection in monocytes, macrophage, and T cells in the brain impacting neurocognitive functioning (Gaskill, Calderon, Coley, & Berman, 2013). Given the important relationship that the dopamine system has with HIV infection and substance use disorders, it is possible that dopamine dysregulation may be driving the increase in neurocognitive dispersion observed in HIV+ substance users. Larger studies of HIV+ adults with and without current substance use should investigate more directly the role of dopaminergic functioning and IIV.
Our hypothesis that greater neurocognitive dispersion would be associated with suboptimal medication adherence, particularly among substance users, was partially supported. The possible synergistic effect of substance use and dispersion on medication adherence was largely accounted for by current depressive symptoms. This is not surprising given that prior studies have found that depressive symptoms are more influential on medication adherence than global neurocognitive functioning and current substance use (Scott et al., 2018). However, it should be noted that the medium effect size of dispersion on medication adherence remain largely unchanged (partial η2 = .23-.25) after adjusting for the confounding effect of depressive symptoms. Our small sample size may have limited power to adequately detect statistical significance for a substance use by dispersion interaction on medication adherence.
While prior studies have associated greater IIV with worse medication adherence and we did not find an overall relationship of dispersion and adherence across the entire sample there are methodological differences that may explain the equivocal findings. First, prior research has differed on their operationalization of IIV and medication adherence. One study that found greater IIV was associated with worse medication adherence, used reaction time variability as their measure of IIV (Ettenhofer et al., 2010) while the Thaler et al. study (2015) used change in neurocognitive dispersion over time as their measure of IIV. Second, other studies of neurocognitive dispersion established their relationship with medication adherence through self-reported adherence (Morgan et al., 2012) while the current study used an objective measure of adherence (i.e., MEMS caps). These differences in defining IIV and medication adherence may be driving these inconsistent findings. It may be that the influence of dispersion on medication adherence is best observed in longitudinal changes of dispersion, rather than evaluating its relationship at any given time point. Future studies should evaluate longitudinal changes in dispersion are associated with longitudinal changes in medication adherence, and whether this relationship is moderated by substance use.
Moving forward, future studies should look at the relationship of neurocognitive dispersion in HIV+ adults with current substance use longitudinally to assess if this population is indeed at higher risk for functional and neurocognitive declines. For instance, Morgan and colleagues (2011) found higher rates of dispersion observed in older HIV+ individuals compared to older HIV− and younger HIV+ individuals. Given that dispersion is associated with worse functional and immunological outcomes as found in this and prior studies (Morgan et al., 2012; Thaler et al., 2015), substance-using HIV+ adults might be at higher risk for functional decline with age. Moreover, it is important to understand whether a certain type of IIV (i.e., reaction time variability, dispersion) has better sensitivity to neurocognitive and functional impairments. Thus, it is crucial to identify if one measure of IIV is preferred over the other. Similarly, it is unknown if greater variability within a specific cognitive domain differentially increases risk of worse cognitive and health outcomes among HIV+ adults. For instance, a prior study found greater IIV within memory tests among individuals with amnestic mild cognitive impairment compared to matched controls (Troyer, Vandermorris, & Murphy, 2016) and another study reported greater IIV in speed of information processing was associated with cognitive impairment among patients with multiple sclerosis compared to healthy controls (Bodling, Denney, & Lynch, 2012). Larger studies would be well suited to evaluate patterns of dispersion across a neurocognitive profile. Furthermore, although variability in neurocognitive performance is not uncommon (Schretlen, Munro, Anthony, & Pearlson, 2003), nor is it unexpected in aging populations (Bielak et al., 2014), the literature demonstrates that greater than normal increases in IIV is related to worse neurocognitive and functional outcomes. However, “normal” levels of IIV are yet to be defined. With this in mind, future studies should address what are normative levels of IIV in order to establish cutoffs to assist clinicians and medical practitioners in becoming aware of possible functional deficits, particularly when working with populations at risk for poor health outcomes such as HIV+ substance users.
There are some limitations associated with this study that need to also be addressed. First of all, the small sample size limited the power needed to capture stronger relationships between IIV and our outcomes, or the potential additive effect of substance use and suboptimal adherence on dispersion. Although the association between substance use and neurocognitive dispersion did not survive multiple comparison correction, we did detect a large effect size (Cohen’s d = 0.81) which may indicate that this relationship will reach statistically significance in well-powered study. These results require replication in larger cohorts. Second, the racial/ethnic diversity of the sample is both a strength and a limitation. Given that the design of the parent study focused on evaluating the cognitive and functional outcomes among Latinx compared to non-Hispanic whites, it is unknown if our findings are generalizable to Latinx outside of the New York City who are primarily of Caribbean ancestry or among individuals of other racial/ethnic backgrounds (i.e., African-American, Asian, multi-racial Americans). A third limitation was the high rate of polysubstance use within the SU+ group. The SU+ group met criteria for several substance use disorders, both past and currently, in addition to recent use as demonstrated in their urine toxicology, which limits our ability to understand which specific substances might be leading to greater dispersion. While some of the neurocognitive effects of substance use can be generalizable to all substances (i.e., aspects of executive functioning generally impacted regardless of substance), there is evidence of substance-specific effects on neurocognitive functioning (see for review Fernandez-Serrano et al. 2011). Even among polysubstance users, the self-reported drug of choice can be associated with specific cognitive deficits. For instance, psychostimulants are associated with worse executive functioning when compared to polysubstance users whose drug of choice are opioids (see for review Fernandez-Serrano et al. 2011). Thus, future studies should be designed to determine the specific effects of substances on neurocognitive dispersion. It should be noted that although this might be a limitation, these rates of substance use are consistent with U.S. prevalence rates for the HIV+ population (SAMSHA, 2010), which confers greater generalizability to the current findings. Finally, we only analyzed current and not past substance use or patterns of substance use. It is unclear whether our results would be generalizable to individuals with a history of substance use but that no longer meet criteria for a substance use disorder. Similarly, patterns of substance use can influence cognitive, HIV-specific health, and behavioral outcomes. For instance, heavy alcohol use was associated with both suboptimal cART adherence and worse HIV viral suppression, while binge drinking was only associated with suboptimal cART adherence (Cook et al., 2007). In the current study we were unable to evaluate the association of different patterns of substance use on neurocognitive IIV.
The current findings provide an important contribution to the literature on IIV in HIV+ adults. Prior research has shown that IIV, as measured through reaction time variability, is associated with higher viral load, worse medication adherence, and current substance use (Ettenhofer et al., 2010; Levine et al., 2008; Levine et al., 2006). This study extends prior work by demonstrating that neurocognitive dispersion is another valid measure of intra-individual variability that might provide unique information pertinent to current functional outcomes (i.e., current substance use, medication adherence). These findings, taken in concert with the growing evidence of the association of IIV with worse neurocognitive functioning and functional outcomes among HIV+ adults and other neurological populations (MacDonald, Li, & Backman, 2009; Thaler et al., 2015), may provide useful information for neuropsychologists working with HIV+ substance users in research and clinical practice.
Public Significance Statements.
HIV infection and substance use are associated with worse health outcomes but their joint impact on cognition is unclear. This study provides preliminary support that variability in cognitive abilities may be a more sensitive marker of the joint impact of HIV infection and substance use on the central nervous system and may lead to worse medication adherence among HIV+ adults.
Acknowledgements:
This research was supported by K23MH07971801 and an Early Career Development Award from the Northeast Consortium for Minority Faculty Development (to MRM); R24MH59724 and U01MH083501 (to SM); N01MH22005 (Igor Grant PI, subcontract to Susan Morgello); by the Clinical Research Center of the Mount Sinai School of Medicine (M01-RR00071); and by the National Science Foundation Graduate Research Fellowship under Grant No. 1144474 (to MAR).
The authors wish to thank Susan Morgello, M.D. for all her help and support throughout the development of this manuscript, the Harlem Community Academic Partnership (HCAP), the Manhattan HIV Care Network, and our participants for their contributions to our research.
References
- AARDEX Group. Medication Event Monitoring System: Sion, Switzerland. [Google Scholar]
- American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed., text rev. ed.): Washington, DC: Author. [Google Scholar]
- Anderson AE, Jones JD, Thaler NS, Kuhn TP, Singer EJ, & Hinkin CH (2018). Intraindividual variability in neuropsychological performance predicts cognitive decline and death in HIV. Neuropsychology, 32(8), 966. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Basso MR, & Bornstein RA (2003). Effects of past noninjection drug abuse upon executive function and working memory in HIV infection. J Clin Exp Neuropsychol, 25(7), 893–903. doi: 10.1076/jcen.25.7.893.16489 [DOI] [PubMed] [Google Scholar]
- Beck AT, Steer RA, & Brown G (2000). BDI-II Fast Screen for Medical Patients Manual. London: The Psychological Corporation. [Google Scholar]
- Beck AT, Steer RA, & Brown GK (1996). Beck depression inventory-II. San Antonio, 78(2), 490–498. [Google Scholar]
- Benjamini Y, Drai D, Elmer G, Kafkafi N, & Golani I (2001). Controlling the false discovery rate in behavior genetics research. Behavioural brain research, 125(1-2), 279–284. [DOI] [PubMed] [Google Scholar]
- Bielak A, Hultsch D, Strauss E, MacDonald S, & Hunter M (2010). Intraindividual variability is related to cognitive change in older adults: evidence for within-person coupling. Psychology and Aging, 25(3), 575. [DOI] [PubMed] [Google Scholar]
- Bielak AA, Cherbuin N, Bunce D, & Anstey KJ (2014). Intraindividual variability is a fundamental phenomenon of aging: evidence from an 8-year longitudinal study across young, middle, and older adulthood. Dev Psychol, 50(1), 143–151. doi: 10.1037/a0032650 [DOI] [PubMed] [Google Scholar]
- Bing EG, Burnam MA, Longshore D, Fleishman JA, Sherbourne CD, London AS, … Shapiro M (2001). Psychiatric disorders and drug use among human immunodeficiency virus-infected adults in the United States. Arch Gen Psychiatry, 58(8), 721–728. [DOI] [PubMed] [Google Scholar]
- Bodling AM, Denney DR, & Lynch SG (2012). Individual variability in speed of information processing: An index of cognitive impairment in multiple sclerosis. Neuropsychology, 26(3), 357. [DOI] [PubMed] [Google Scholar]
- Bunce D, Anstey KJ, Cherbuin N, Burns R, Christensen H, Wen W, & Sachdev PS (2010). Cognitive deficits are associated with frontal and temporal lobe white matter lesions in middle-aged adults living in the community. PLoS One, 5(10), e13567. doi: 10.1371/journal.pone.0013567 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bunce D, Anstey KJ, Christensen H, Dear K, Wen W, & Sachdev P (2007). White matter hyperintensities and within-person variability in community-dwelling adults aged 60-64 years. Neuropsychologia, 45(9), 2009–2015. doi: 10.1016/j.neuropsychologia.2007.02.006 [DOI] [PubMed] [Google Scholar]
- Burton CL, Strauss E, Hultsch DF, Moll A, & Hunter MA (2006). Intraindividual variability as a marker of neurological dysfunction: a comparison of Alzheimer's disease and Parkinson's disease. Journal of Clinical and Experimental Neuropsychology, 28(1), 67–83. [DOI] [PubMed] [Google Scholar]
- Byrd DA, Fellows RP, Morgello S, Franklin D, Heaton RK, Deutsch R, … Group, C. (2011). Neurocognitive impact of substance use in HIV infection. J Acquir Immune Defic Syndr, 58(2), 154–162. doi: 10.1097/QAI.0b013e318229ba41 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Byrd DA, Robinson-Papp J, Mindt MR, Mintz L, Elliott K, Lighty Q, & Morgello S (2013). Isolating cognitive and neurologic HIV effects in substance-dependent, confounded cohorts: a pilot study. Journal of the International Neuropsychological Society, 19(4), 463–473. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Chang L, Wang GJ, Volkow ND, Ernst T, Telang F, Logan J, & Fowler JS (2008). Decreased brain dopamine transporters are related to cognitive deficits in HIV patients with or without cocaine abuse. Neuroimage, 42(2), 869–878. doi: 10.1016/j.neuroimage.2008.05.011 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Christensen H, Mackinnon A, Korten A, Jorm A, Henderson A, & Jacomb P (1999). Dispersion in cognitive ability as a function of age: A longitudinal study of an elderly community sample. Aging, Neuropsychology, and Cognition, 6(3), 214–228. [Google Scholar]
- Cook R, Zhou Z, Kelso-Chichetto N, Janelle J, Morano J, Somboonwit C, … Cook C (2017). Alcohol consumption patterns and HIV viral suppression among persons receiving HIV care in Florida: an observational study. Addiction science & clinical practice, 12(1), 22. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Durvasula RS, Myers HF, Satz P, Miller EN, Morgenstern H, Richardson MA, … Forney D (2000). HIV-1, cocaine, and neuropsychological performance in African American men. J Int Neuropsychol Soc, 6(3), 322–335. [DOI] [PubMed] [Google Scholar]
- Ettenhofer ML, Foley J, Castellon SA, & Hinkin CH (2010). Reciprocal prediction of medication adherence and neurocognition in HIV/AIDS. Neurology, 74(15), 1217–1222. doi: 10.1212/WNL.0b013e3181d8c1ca [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fassbender C, Lesh TA, Ursu S, & Salo R (2015). Reaction time variability and related brain activity in methamphetamine psychosis. Biol Psychiatry, 77(5), 465–474. doi: 10.1016/j.biopsych.2014.07.028 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fernandez-Serrano MJ, Pérez-García M, & Verdejo-Garcia A (2011). What are the specific vs. generalized effects of drugs of abuse on neuropsychological performance? [DOI] [PubMed] [Google Scholar]
- Galvan FH, Burnam MA, & Bing EG (2003). Co-occurring psychiatric symptoms and drug dependence or heavy drinking among HIV-positive people. J Psychoactive Drugs, 35 Suppl 1, 153–160. doi: 10.1080/02791072.2003.10400510 [DOI] [PubMed] [Google Scholar]
- Gaskill PJ, Calderon TM, Coley JS, & Berman JW (2013). Drug induced increases in CNS dopamine alter monocyte, macrophage and T cell functions: implications for HAND. J Neuroimmune Pharmacol, 8(3), 621–642. doi: 10.1007/s11481-013-9443-y [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hauser KF, El-Hage N, Stiene-Martin A, Maragos WF, Nath A, Persidsky Y, … Knapp PE (2007). HIV-1 neuropathogenesis: glial mechanisms revealed through substance abuse. J Neurochem, 100(3), 567–586. doi: 10.1111/j.1471-4159.2006.04227.x [DOI] [PMC free article] [PubMed] [Google Scholar]
- Heaton RK, Franklin DR, Ellis RJ, McCutchan JA, Letendre SL, Leblanc S, … Group, H. (2011). HIV-associated neurocognitive disorders before and during the era of combination antiretroviral therapy: differences in rates, nature, and predictors. J Neurovirol, 17(1), 3–16. doi: 10.1007/s13365-010-0006-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Jones J, Kuhn T, Mahmood Z, Singer E, Hinkin C, & Thames A (2017). Longitudinal Intra-Individual Variability in Neuropsychological Performance Relates to White Matter Changes in HIV. Neuropsychology. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Korthuis PT, Zephyrin LC, Fleishman JA, Saha S, Josephs JS, McGrath MM, … Network, H. I. V. R. (2008). Health-related quality of life in HIV-infected patients: the role of substance use. AIDS Patient Care STDS, 22(11), 859–867. doi: 10.1089/apc.2008.0005 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Krefetz DG, Steer RA, Jermyn RT, & Condoluci DV J. J. o. C. P. i. M. S. (2004). Screening HIV-Infected Patients with Chronic Pain for Anxiety and Mood Disorders with the Beck Anxiety and Depression Inventory-Fast Screens for Medical Settings. 11(4), 283–289. doi: 10.1023/b:Jocs.0000045348.28440.82 [DOI] [Google Scholar]
- Kumar AM, Fernandez JB, Singer EJ, Commins D, Waldrop-Valverde D, Ownby RL, & Kumar M (2009). Human immunodeficiency virus type 1 in the central nervous system leads to decreased dopamine in different regions of postmortem human brains. J Neurovirol, 15(3), 257–274. doi: 10.1080/13550280902973952 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Levine AJ, Hardy DJ, Barclay TR, Reinhard MJ, Cole MM, & Hinkin CH (2008). Elements of attention in HIV-infected adults: evaluation of an existing model. J Clin Exp Neuropsychol, 30(1), 53–62. doi: 10.1080/13803390601186684 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Levine AJ, Hardy DJ, Miller E, Castellon SA, Longshore D, & Hinkin CH (2006). The effect of recent stimulant use on sustained attention in HIV-infected adults. J Clin Exp Neuropsychol, 28(1), 29–42. doi: 10.1080/13803390490918066 [DOI] [PubMed] [Google Scholar]
- Liu H, Golin CE, Miller LG, Hays RD, Beck CK, Sanandaji S, … Wenger NS (2001). A comparison study of multiple measures of adherence to HIV protease inhibitors. Ann Intern Med, 134(10), 968–977. [DOI] [PubMed] [Google Scholar]
- Lovden M, Schmiedek F, Kennedy KM, Rodrigue KM, Lindenberger U, & Raz N (2013). Does variability in cognitive performance correlate with frontal brain volume? Neuroimage, 64, 209–215. doi: 10.1016/j.neuroimage.2012.09.039 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lucas GM, Cheever LW, Chaisson RE, & Moore RD (2001). Detrimental effects of continued illicit drug use on the treatment of HIV-1 infection. J Acquir Immune Defic Syndr, 27(3), 251–259. [DOI] [PubMed] [Google Scholar]
- Lundqvist T (2010). Imaging cognitive deficits in drug abuse. Curr Top Behav Neurosci, 3, 247–275. doi: 10.1007/7854_2009_26 [DOI] [PubMed] [Google Scholar]
- MacDonald SW, Karlsson S, Rieckmann A, Nyberg L, & Backman L (2012). Aging-related increases in behavioral variability: relations to losses of dopamine D1 receptors. J Neurosci, 32(24), 8186–8191. doi: 10.1523/JNEUROSCI.5474-11.2012 [DOI] [PMC free article] [PubMed] [Google Scholar]
- MacDonald SW, Li SC, & Backman L (2009). Neural underpinnings of within-person variability in cognitive functioning. Psychol Aging, 24(4), 792–808. doi: 10.1037/a0017798 [DOI] [PubMed] [Google Scholar]
- MacDonald SW, Nyberg L, & Bäckman L (2006). Intra-individual variability in behavior: links to brain structure, neurotransmission and neuronal activity. Trends in neurosciences, 29(8), 474–480. [DOI] [PubMed] [Google Scholar]
- McNabb JJ, Nicolau DP, Stoner JA, & Ross J (2003). Patterns of adherence to antiretroviral medications: the value of electronic monitoring. AIDS, 17(12), 1763–1767. doi: 10.1097/01.aids.0000076307.76477.5c [DOI] [PubMed] [Google Scholar]
- Meade CS, Conn NA, Skalski LM, & Safren SA (2011). Neurocognitive impairment and medication adherence in HIV patients with and without cocaine dependence. J Behav Med, 34(2), 128–138. doi: 10.1007/s10865-010-9293-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Morgan EE, Doyle KL, Minassian A, Henry BL, Perry W, Marcotte TD, … Translational Methamphetamine, A. R. G. (2014). Elevated intraindividual variability in methamphetamine dependence is associated with poorer everyday functioning. Psychiatry Res, 220(1-2), 527–534. doi: 10.1016/j.psychres.2014.07.008 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Morgan EE, Woods SP, Delano-Wood L, Bondi MW, Grant I, & Group, H. I. V. N. R. P. (2011). Intraindividual variability in HIV infection: evidence for greater neurocognitive dispersion in older HIV seropositive adults. Neuropsychology, 25(5), 645–654. doi: 10.1037/a0023792 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Morgan EE, Woods SP, Grant I, & Group, HNRP. (2012). Intra-individual neurocognitive variability confers risk of dependence in activities of daily living among HIV-seropositive individuals without HIV-associated neurocognitive disorders. Arch Clin Neuropsychol, 27(3), 293–303.doi: 10.1093/arclin/acs003 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Murtha S, Cismaru R, Waechter R, & Chertkow H (2002). Increased variability accompanies frontal lobe damage in dementia. Journal of the International Neuropsychological Society, 8(03), 360–372. [DOI] [PubMed] [Google Scholar]
- Purohit V, Rapaka R, & Shurtleff D (2011). Drugs of abuse, dopamine, and HIV-associated neurocognitive disorders/HIV-associated dementia. Mol Neurobiol, 44(1), 102–110. doi: 10.1007/s12035-011-8195-z [DOI] [PubMed] [Google Scholar]
- Rabkin JG, McElhiney M, Ferrando SJ, Van Gorp W, & Lin SH (2004). Predictors of employment of men with HIV/AIDS: a longitudinal study. Psychosom Med, 66(1), 72–78. [DOI] [PubMed] [Google Scholar]
- Ram N, Rabbitt P, Stollery B, & Nesselroade JR (2005). Cognitive performance inconsistency: intraindividual change and variability. Psychology and Aging, 20(4), 623. [DOI] [PubMed] [Google Scholar]
- Rippeth JD, Heaton RK, Carey CL, Marcotte TD, Moore DJ, Gonzalez R, … Group, H. (2004). Methamphetamine dependence increases risk of neuropsychological impairment in HIV infected persons. J Int Neuropsychol Soc, 10(1), 1–14. doi: 10.1017/S1355617704101021 [DOI] [PubMed] [Google Scholar]
- SAMHSA. (2010). The NSDUH Report: HIV/AIDS and Substance Use. Retrieved from https://www.samhsa.gov/sites/default/files/hiv-aids-and-substance-use.pdf
- Schretlen DJ, Munro CA, Anthony JC, & Pearlson GD (2003). Examining the range of normal intraindividual variability in neuropsychological test performance. J Int Neuropsychol Soc, 9(6), 864–870. doi: 10.1017/S1355617703960061 [DOI] [PubMed] [Google Scholar]
- Scott TM, Byrd D, Rentería MA, Coulehan K, Miranda C, Fuentes A, & Mindt MR (2018). The Combined Roles of Nonsomatic Depressive Symptomatology, Neurocognitive Function, and Current Substance Use in Medication Adherence in Adults Living With HIV Infection. Journal of the Association of Nurses in AIDS Care, 29(2), 178–189. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Statistical Analysis Systems, JMP. (2012). Version 10. SAS Institute Inc., Cary, NC. [Google Scholar]
- Thaler NS, Sayegh P, Arentoft A, Thames AD, Castellon SA, & Hinkin CH (2015). Increased neurocognitive intra-individual variability is associated with declines in medication adherence in HIV-infected adults. Neuropsychology, 29(6), 919–925. doi: 10.1037/neu0000191 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Tractenberg RE, & Pietrzak RH (2011). Intra-individual variability in Alzheimer's disease and cognitive aging: definitions, context, and effect sizes. PLoS One, 6(4), e16973. doi: 10.1371/journal.pone.0016973 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Troyer AK, Vandermorris S, & Murphy KJ (2016). Intraindividual variability in performance on associative memory tasks is elevated in amnestic mild cognitive impairment. Neuropsychologia, 90, 110–116. [DOI] [PubMed] [Google Scholar]
- Turchan J, Anderson C, Hauser KF, Sun Q, Zhang J, Liu Y, … Nath A (2001). Estrogen protects against the synergistic toxicity by HIV proteins, methamphetamine and cocaine. BMC Neurosci, 2, 3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Venkatesan A, Nath A, Ming GL, & Song H (2007). Adult hippocampal neurogenesis: regulation by HIV and drugs of abuse. Cell Mol Life Sci, 64(16), 2120–2132. doi: 10.1007/s00018-007-7063-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Vriesendorp R, Cohen A, Kristanto P, Vrijens B, Rakesh P, Anand B, … Stiekema J (2007). Adherence to HAART therapy measured by electronic monitoring in newly diagnosed HIV patients in Botswana. Eur J Clin Pharmacol, 63(12), 1115–1121. doi: 10.1007/s00228-007-0369-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wilkinson GS (1993). WRAT-3: Wide range achievement test administration manual. Wide Range, Incorporated. [Google Scholar]
- Woods SP, Moore DJ, Weber E, & Grant I (2009). Cognitive neuropsychology of HIV-associated neurocognitive disorders. Neuropsychol Rev, 19(2), 152–168. doi: 10.1007/s11065-009-9102-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Woods SP, Rippeth JD, Frol AB, Levy JK, Ryan E, Soukup VM, … Heaton RK (2004). Interrater reliability of clinical ratings and neurocognitive diagnoses in HIV. J Clin Exp Neuropsychol, 26(6), 759–778. doi: 10.1080/13803390490509565 [DOI] [PubMed] [Google Scholar]
- World Health Organization. (1997). Composite International Diagnostic Interview (Version 2.1): Geneva: WHO. [Google Scholar]


