Cognitive Intra-individual Variability in HIV: An Integrative Review

Abstract

Nearly 30–50% of people living with HIV (PLWH) experience HIV-Associated Neurocognitive Disorder (HAND). HAND indicates performance at least one standard deviation below the normative mean on any two cognitive domains. This method for diagnosing/classifying cognitive impairment has utility; however, cognitive intraindividual variability (IIV) provides a different way to understand cognitive impairment. Cognitive IIV refers to the scatter in cognitive performance within repeated measures of the same cognitive test (i.e., inconsistency) or across different cognitive tests (i.e., dispersion). Cognitive IIV is associated with cognitive impairment and cognitive decline in various clinical populations. This integrative review of 13 articles examined two types of cognitive IIV in PLWH, inconsistency and dispersion. Cognitive IIV appears to be a promising approach to detect subtle cognitive impairments that are not captured by traditional mean-based neuropsychological testing. Greater IIV in PLWH has been associated with: 1) poorer cognitive performance and cognitive decline; 2) cortical atrophy, both gray and white matter volume, 3) poorer everyday functioning (i.e., driving simulation performance), specifically medication adherence; and 4) even mortality. This inspires future directions for research. First, greater cognitive IIV may reflect a greater task demand on executive control to harness and regulate cognitive control over time; by improving executive functioning through cognitive training, it may reduce cognitive IIV which could slow down cognitive decline. Second, cognitive IIV may be reconsidered in prior cognitive intervention studies in which only mean-based cognitive outcomes were used; it is possible that such cognitive interventions may actually improve cognitive IIV, which could have clinical relevance.

By 2030 in the United States and the Netherlands, nearly 70% of people living with HIV (PLWH) will be 50 and older (Smit et al., 2015; Wing, 2016). As PLWH age, they become vulnerable to age-related comorbidities that contribute to both normal and pathological cognitive aging (i.e., dementia). Current estimates indicate that approximately 42.6% of PLWH meet the diagnostic criteria for HIV-Associated Neurocognitive Disorder (HAND) (Wang et al., 2020). However, HIV-Associated Dementia, the most severe form of HAND, has decreased from 15% to 5% [95% confidence interval (CI) = 3.5 – 6.8] in the current era of antiretroviral therapy. However, Asymptomatic Neurocognitive Impairment characterized by mild cognitive deficits with no change in everyday function persists in 24% [95% CI = 20.3 – 26.8] of PLWH and Mild Neurocognitive Disorder characterized by mild cognitive deficits with mildly decreased everyday function persists in about 13.3% (95% CI = 10.6 – 16.3) of PLWH (Wang et al., 2020). HAND and its sub-diagnostic categories are diagnosed on the Frascati criteria, revised by a group of neuroHIV experts in Frascati, Italy (Antinori et al., 2007).

To diagnose HAND, performance impairments must be identified in at least two cognitive domains (Blackstone et al., 2012). In this mean-based approach, norm-based (e.g., age & education) neuropsychological measures are assessed in at least five cognitive domains (e.g., attention, verbal learning). Asymptomatic Neurocognitive Impairment is diagnosed if neuropsychological scores are one or more standard deviations (SD) below the adjusted norm-based mean in two or more cognitive domains. If this cognitive impairment impacts everyday functioning (e.g., shopping, finances, bathing), Mild Neurocognitive Disorder is diagnosed. Finally, if there is a marked impairment of everyday functioning and neuropsychological scores are 2 or more SDs below the adjusted norm-based mean in two or more cognitive domains, a diagnosis of HIV-Associated Dementia is given. It is important to note, that in making these HAND diagnoses, an additional requirement is to rule out that the cognitive and everyday functioning impairments are not due to other comorbidities such as heart disease, diabetes, hypertension, and hepatitis C. Unfortunately, such comorbidities often impact cognition and are quite common in PLWH, especially as they age (Vance et al., 2011); frequently, it is not clear how much such impairments are influenced by HIV, a comorbidity, or the synergistic combination. Despite this requirement, what is often observed in the neuroHIV literature is that this comorbidity requirement is suspended and just the cognitive criteria are applied as has been seen in many HIV cohort and experimental studies (e.g., Heaton et al., 2010; Vance et al., 2017) that also compare the prevalence of HAND to those without HIV who meet these same cognitive criteria.

Although this mean-based nosology of cognitive neuroHIV has been useful in quantifying and standardizing cognitive impairment in HIV, a 2016 international HAND task force of neuroHIV experts identified some limitations (Cysique et al., 2017). One of the major limitations is that HAND diagnoses are highly dynamic, fluctuating over time. For example, in the CNS HIV Anti-Retroviral Therapy Effect Research (CHARTER) cohort study, using laboratory, medical, and neuropsychological measures, researchers followed 436 PLWH every six months (16–72 months; M_months = 35) and observed that cognitively 60.85% remained stable, 16.5% improved, and 22.7% declined (Heaton et al., 2015). Furthermore, these fluctuations were observed in multiple cognitive domains including speed of processing, spatial reasoning, verbal memory, and others (Fellows et al., 2014; Maki et al., 2018; Waldrop et al., 2021).

Despite the utility of the HAND nosology, other ways to examine neuropsychological function have been proposed. One approach that has been increasingly used over the past two decades is cognitive intra-individual variability (IIV) (Hultsch et al., 2000). Although normal levels of cognitive IIV have not yet been established, it is understood that most individuals have cognitive strengths and weaknesses, with many individuals having poor performance in some cognitive domains (e.g., 1.3 SD below the adjusted mean) but cognitive strengths in others (e.g., 1.7 SD above the adjusted mean). Impairments in cognitive and functional abilities are traditionally diagnosed using deficits in an individual’s mean performance on a task or in a cognitive domain, often comparing performance to normative data. Similarly, traditionally research on cognition in HIV has focused on mean-based cognitive function, usually examining group means.

In cognitive aging research, the cognitive IIV literature has established the context in which this can be applied to other clinical populations such as HIV. Several studies have shown that increased cognitive IIV predicts cognitive decline and even mortality. In a sample of 134 community-dwelling older adults (64+), Hilborn et al. (2009) using both measures of inconsistency and dispersion, types of cognitive IIV, found that higher levels of cognitive IIV were observed in old-old adults and those with worse cognitive function. In a sample of 212 community-dwelling older adults, Bielak et al. (2010) used reaction time inconsistency at baseline and found that higher inconsistency was predictive of developing cognitive impairment 5 years later; this was more predictive than baseline reaction time. Similarly, within a prospective cohort of 897 community-dwelling older adults 70+, Holtzer et al. (2008) found that baseline dispersion predicted the development of dementia three years later, even when adjusting for mean-based cognitive performance at baseline. Finally, in a sample of 44 healthy older adults (M_age = 72.0 years), using a measure of dispersion and diffusion tensor imaging (DTI), Halliday et al. (2019) found greater cognitive IIV was associated with poorer white matter integrity, specifically in the left superior longitudinal fasciculus, anterior corona radiata bilaterally, and the body and genu of the corpus callosum. Unfortunately, with the way cognitive IIV is measured from study to study, there are no established population norms for cognitive IIV yet; but this approach does provide new predictive insights into cognition and behavior.

As in cognitive aging research above, research on cognitive IIV over the past several years (e.g., Ettenhofer et al., 2010; Levine et al., 2008) raises the possibility that beyond using traditional mean-based cognitive assessment which has established utility, cognitive IIV may provide complementary information to examine cognitive function, cognitive decline, and other non-cognitive behaviors in PLWH. In fact, several studies suggest that cognitive IIV may be more sensitive to detect cognitive decline, driving simulation performance, mortality, and medication adherence above that of mean-based cognitive measures (e.g., Anderson et al., 2018; Levine et al., 2008; Morgan et al., 2014). It may thus be important to examine patterns of cognitive performance within individuals, either over time on the same task (inconsistency) or across cognitive domains (dispersion). These two varieties of cognitive IIV have been shown to be positively correlated (Hultsch et al., 2002), suggesting they may share an underlying mechanism, most likely due to poor executive control and resulting dysregulation of other cognitive abilities (Bellgrove et al., 2004; Morgan et al., 2012a). Neural areas associated with executive functioning (e.g., frontal-striatal system, dorso-lateral prefrontal cortex, and caudate nucleus) have been observed to be compromised in PLWH, especially as they age (Corrêa et al., 2016; du Plessis et al., 2015). A recent meta-analysis of 37 neuropsychological HIV studies (n = 3,935 HIV+; n = 2,483 HIV-) found executive dysfunction across multiple subdomains of executive functioning including working memory, set-shifting, inhibition, decision-making, and apathy in PLWH (Walker & Brown, 2018).

Cognitive IIV has been demonstrated in numerous clinical populations including traumatic brain injury, neurodegenerative diseases, and healthy aging, where it persists even when reaction time latency is controlled (Bangen et al., 2019; Dykiert et al., 2012). In a healthy aging sample, cognitive IIV was associated with reduced white matter volume and increased frontal lobe white matter hyperintensities on MRI (Bunce et al., 2007), as well as increased frontal activation during Go/No-go response inhibition task performance (Bellgrove et al., 2004).

This review was thus undertaken to critically evaluate the literature on cognitive IIV in HIV, including its sensitivity to HAND, neurobiological substrates, and relevance to everyday functioning. To do this, a historical and conceptual overview of cognitive IIV is provided, with emphasis on inconsistency and dispersion. Then, the methodology is described, followed by the actual integrative review. The article concludes with a synthesis of the findings and a discussion of implications for clinical practice and research.

Historical and Conceptual Overview of Cognitive IIV

Historically, the concept of cognitive IIV, dispersion specifically, has long been debated from multiple perspectives, with Plake, Reynolds, and Gutkin (1981) proposing an index of scatter called Profile Variability Index which was applied to the WAIS-R, to Silverstein (1993) cautioning the use of scatter or score variability patterns, to McLean, Reynolds, and Kaufman (1990) reevaluating the Profile Variability Index which was touted as “a more sensitive index of profile variability for clinicians” (p. 289), to Matarazzo (1990) and others (Kline et al., 1993) providing convincing evidence that such cognitive IIV does not measure what we think it does. In fact, Matarazzo observed that greater variability between cognitive tests was associated with greater IQ, not less as expected. Since then, revivals of its use have periodically resurfaced in various definitions and formulas of cognitive IIV, extending from dispersion to moment-to-moment variability (i.e., inconsistency), where it has gained more appeal in predicting cognitive decline, everyday functioning and behaviors, and even study attrition rates and mortality (Bielak et al., 2010).

In neuropsychological research, a large battery of neuropsychological tests, covering multiple cognitive domains, is administered and normatively-derived mean scores are used to evaluate and describe the cognitive profile of a population of interest. While this approach has great utility, it can miss clinically useful information about variability across the person’s distribution of cognitive performance scores. This variation between successive trials (trial-to-trial), such as reaction time variability on a single test, or across various cognitive measures from a cognitive test battery, may provide further insight to one’s overall cognitive functioning.

Cognitive IIV has been described as a transient but “lawful” within-person fluctuation in behavior (i.e., cognition) over a specified period (Nesselroad,1991). In healthy individuals, some variability on or across tasks is common (Schretlen et al., 2008). This is intuitive as we would not expect someone to score at the 50^th percentile on every test of a neuropsychological battery, and on reaction time tests, for example, we would not expect a person to press the spacebar precisely at the same time following each presentation of a stimulus. There will always be a distribution of scores around a person’s mean performance. While IIV is the norm rather than the exception, higher IIV may reflect the inability to maintain complex top-down attention, associated with frontal and temporal lobe dysfunction (Bellgrove et al., 2004; Troyer et al., 2016); unfortunately, there is not an accepted standard of what constitutes “high” vs “low” IIV as these seem to vary from study sample to study sample. Consistent with the purview of this article, prior research has also shown IIV to be a useful metric to evaluate cognition in PLWH (Anderson et al., 2018; Arce Renteria et al., 2020). Aberrant IIV has been conceptualized as the degradation of cognitive control or attention processes (MacDonald et al., 2006; Morgan et al., 2012a; Vasquez et al., 2018), which has important clinical implications as it may be a bellwether for declining cognitive function.

Cognitive IIV can be thought of in two ways: inconsistency and dispersion (Abdi, 2010; Stuss et al., 2003; Tractenberg & Pietrzak, 2011). IIV consistency refers to variability related to a single person’s performance on a single task across multiple instances or trials, such as the variability observed across a person’s distribution of reaction time on a cognitive task (e.g., stimulus detection) (Abdi, 2010; Stuss et al., 2003; Tractenberg & Pietrzak, 2011). IIV dispersion refers to variability related to a person’s performance across multiple measures within the same time period, such as a neuropsychological test battery. Since cognitive IIV metrics are related to the variance of a distribution relative to the individual, the minimum score is 0 (reflecting no variability), and the value will always be positive (Abdi, 2010; Tractenberg & Pietrzak, 2011).

Two common calculations for IIV-inconsistency and IIV-dispsersion are the intra-individual standard deviation (iSD) and the coefficient of variation (CoV) (Tractenberg & Pietrzak, 2011). The iSD is calculated in three steps. Step 1, each person’s performance score is normed by either transforming into a z-score using the formula (X – M) / SD, (where X is the person’s score, M is the sample mean, and SD is the standard deviation). Alternatively, if normative data exist, one can also use scaled, standard, or T-scores in lieu of z-scores. This is followed by step 2, computing an overall test battery mean (OTBM) for the individual, and step 3, calculating the SD for the individual’s OTBM (Lindenberger & Baltes, 1997). With this approach, lower iSD reflects less variability and higher iSD reflects more variability (Schretlen & Sullivan, 2013).

Another common IIV calculation is the coefficient of variation (CoV), calculated with the following equation: CoV = iSD / iM (Christensen et al., 2005). With the CoV calculation, the iSD is divided by the individual OTBM (Adbi, 2010), thus making the metric proportional to their OTBM. Of note, if z-scores were used, it is possible to obtain a negative CoV value if an individual’s mean z-score is below zero. Similarly, if a person has an OTBM of zero, then their CoV value cannot be computed. Using T-scores can avoid this issue.

The primary interest in IIV-inconsistency is performance consistency. For example, consider the case of a simple reaction time task where the participant is instructed to press a button immediately after seeing a red circle presented 100 times and achieves 100% accuracy. Across all 100 trials, a mean reaction time of 327 milliseconds is calculated (summation of button press response times divided by 100). Further, the SD of reaction times of these correct trials (the square root of the average sum of SDs of correct reaction time trials from the mean reaction time) is 27.5. Because this SD represents an index of within-person (or intra-individual) variability of scores, it is often referred to as the intra-individual SD (iSD), which is one way to determine IIV-inconsistency. If the iSD is the desired IIV-inconsistency metric, then this process is repeated for each study participant and it can be used as a variable in statistical analyses. If the CoV is the desired IIV metric, then one would take the SD (27.5) and divide this by the mean (327 ms), which yields 0.084. This number can be multiplied by 100 to make it a percentage (8.4%).

The mathematics are similar for IIV-dispersion but the conceptualization is different. Rather than focusing on how consistently a person performs on a single test, the focus is instead on the extent of variability across performance on an array of measures (e.g., a test battery comprised of multiple tests). For example, both clinical and research neuropsychology involve the administration of a large array of cognitive tests, many of which may have multiple outcome scores. To calculate the iSD and CoV, one must first take all the available scores and apply the appropriate normative data to get a scaled score or T-score. As an example, using age-matched norms for three hypothetical tests, we have calculated scaled scores (ss) with a mean of 10, SD of 3, yielding the following values – Test₁ ss = 10, Test₂ ss = 6, and Test₃ ss = 12. This person’s OTBM is 9.333 and the iSD is 2.49. To calculate the CoV, divide the iSD (2.49) by the OTBM (9.33), resulting in a CoV value of 0.267. This value can also be expressed as a percentage of 26.7%. The iSD or CoV for each study participant can then be entered into statistical models.

Tractenberg and Pietrzak (2011) reported three definitions of dispersion, each calculated slightly differently. The first definition is computed as the SD for items or for standardized test scores per person. Referred to as intra-individual standard deviation (iSD), larger iSD reflects greater cognitive IIV. The second definition is a coefficient of variation computed as the SD divided by the mean, either over all items or over standardized test scores. Lastly, dispersion has been defined using mean-independent variations; it is important to note that this variability metric is the value after partialling out the shared variance between the OTBM and iSD or CoV.

Methodology of Integrative Review

Using the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) approach (Moher et al., 2009), on November 25, 2020, MEDLINE (via PubMed), Embase, and Scopus databases were searched for research studies on cognitive IIV in HIV. To structure this integrative review, population/problem, intervention, control, and outcome for quantitative studies was used (Higgins & Green, 2011). The population was PLWH; the problem was cognitive impairment. There was no intervention. The comparison was of participants with HIV to themselves or to adults without HIV. The outcome was cognitive IIV as measured by neuropsychological testing. This search was restricted to articles published in English. The detailed search terms and final search strategies for the three database searches are displayed in Table 1. Keywords for all three searches were “Intra-individual Variability (IIV)”, “Dispersion”, “Inconsistency”, “HIV”, and “Cognition”. The search was not limited by year of publication. A total of 129 articles were retrieved including 18 duplicates which were removed; two authors (DEV & JSF) discussed these articles to confirm their appropriateness for the study. Two additional articles were found through review of reference lists, leaving 111 articles to be reviewed by title and abstract. Of these, 97 articles were excluded, primarily for not addressing either IIV or cognition. Fourteen were screened more thoroughly and one was removed because it did not focus on cognitive IIV specifically. Thirteen articles were then included in the final review. The PRISMA diagram is displayed in Figure 1. Summaries of these articles are presented in the next two sections and in Table 2. Inconsistency articles are presented first followed by dispersion articles.

Table 1.

Terms Used to Search for Targeted Articles in PubMed, Embase and Scopus

Database	Search
PubMed – Search conducted on November 25, 2020	((((“intra-individual variability”[ti] OR “intra-individual variabilities”[ti] OR intra-individual-variabilit*[ti] OR intra-individual-variability[ti] OR intra-individual*[ti] OR variable*[ti] OR variabilit*[ti] OR “Within-Person Variation”[ti] OR “Within-Person Variations”[ti] OR “within-person variability”[ti] OR “within-person variabilities”[ti] OR “within-subject variations”[ti] OR “within-subject variation”[ti] OR dispersion*[ti] OR inconsisten*[ti])) AND ((“HIV”[Mesh] OR “HIV Infections”[Mesh] OR HIV[tiab] OR HIV-infected[ti] OR “HIV-seropositive”[ti] OR “Human Immunodeficiency Virus”[ti] OR “Human T Cell Lymphotropic Virus Type III”[ti] OR “Human T-Cell Leukemia Virus Type III”[ti] OR LAV-HTLV-III[ti] OR “Lymphadenopathy Associated Virus”[ti] OR “Human T Lymphotropic Virus Type III”[ti] OR AIDS[ti] OR AIDS-virus*[ti] OR “Acquired Immune Deficiency Syndrome Virus”[ti] OR “Acquired Immunodeficiency Syndrome Virus”[ti] OR HTLV-III[ti] OR HIV-1[ti] OR HIV-2[ti] OR “HIV-1”[Mesh] OR “HIV-2”[Mesh] OR HIV-I[ti] OR HIV-II[ti])))) AND (((“Cognition Disorders”[Mesh] OR Cognition[tiab] OR cognitive*[tiab] OR neuropsychological[ti] OR impairment[ti] OR dependence[ti] OR attention[ti] OR everyday-function*[ti] OR neurocognitive[ti] OR memory[ti])))
Embase Search conducted on Nov. 25, 2020	(‘intra individual variability’:ti,ab,kw OR iiv:ti,ab,kw OR ‘within-person variation*’:ti,ab,kw OR ‘within-subject variation*’:ti,ab,kw OR dispersion:ti,ab,kw OR inconsistenc*:ti,ab,kw) AND (‘human immunodeficiency virus’:ti,ab,kw OR hiv:ti,ab,kw) AND (cognition:ti,ab,kw OR ‘cognitive defect’:ti,ab,kw OR cognit*:ti,ab,kw)
Scopus Search conducted on Nov. 25, 2020	TITLE-ABS(“intra-individual variability” OR IIV OR “intra-individual variabilities” OR intra-individual* OR “Within-Person Variation” OR “Within-Person Variations” OR “within-person variability” OR “within-person variabilities” OR “within-subject variations” OR “within-subject variation” OR dispersion* OR inconsisten*) AND TITLE-ABS(“HIV” OR “Human Immunodeficiency Virus”[ti] OR “Human T Cell Lymphotropic Virus Type III” OR “Human T-Cell Leukemia Virus Type III” OR LAV-HTLV-III OR “Lymphadenopathy Associated Virus” OR “Human T Lymphotropic Virus Type III” OR AIDS OR “Acquired Immune Deficiency Syndrome Virus” OR “Acquired Immunodeficiency Syndrome Virus” OR HTLV-III OR HIV-1 OR HIV-2 OR HIV-I OR HIV-II) AND TITLE-ABS(Cognition OR cognitive* OR neuropsychological OR impairment OR dependence OR attention OR everyday-function* OR neurocognitive OR memory)

Figure 1.

PRISMA Diagram Demonstrating Screening Method for Literature Search (Adapted from Moher et al., 2009)

Table 2.

Summary of Intra-Individual Variability in HIV Cognitive Studies

Study	Participants	Intra-Individual Variability Calculation	Design/Protocol	Findings	Strengths/Limitations
1. Levine et al. (2008) Elements of Attention in HIV-infected Adults: Evaluation of an Existing Model	N = 104 HIV+ Adults (Mage = 40.9) Entry Criteria • No significant neurological or psychiatric disease; no substance use at time of testing (by self-report)	• Inconsistency Cognitive IIV • The PCA derived “stabilize” factor, which includes inconsistency and omission errors, is one factor in a five-factor model of attention. Performance on this factor was measured using the Conners’ Continuous Performance Test (CPT). The factor structure of the five attention measures was explored using principal components analysis.	• Cross-sectional study with 1 group Primary Outcome • Association between five attentional factors and NP performance, medication adherence, and virologic variables	• Lower inconsistency on the Conners’ CPT was associated with better medication adherence as measured by MEMS cap over six months.	Strengths • Use of MEMS cap to measure medication adherence Limitations • The “sustain” factor was not a pure measure of IIV as it also included errors of omission. • The “stabilize” factor is also comprised of only CPT measures, which are all inter-correlated. • Lack of HIV-negative control group • Lack of longitudinal data
2. Ettenhoffer et al. (2010) Reaction Time Variability in HIV-Positive Individuals	N = 46 HIV+ Adults (Mage = 41.52) Entry Criteria • Taking ART; no current substance use disorder or significant neurological or psychiatric disease	• Inconsistency Cognitive IIV • Reaction time variability across the Conners’ CPT	• Cross-sectional study with 1 group Primary Outcome • Associations between RT variability and global cognition, medication adherence, and peak immunological dysfunction	• RT variability was associated with lower NP performance, most recent and nadir CD4 count and viral load, and worse medication adherence.	Strengths • Use of MEMS cap to measure medication adherence • RT IIV on one test is quicker and more convenient than dispersion on NP battery for detecting incipient cognitive decline. Limitations • Lack of HIV-negative control group • Lack of longitudinal data
3. Morgan et al. (2014) Elevated Intraindividual Variability in Methamphetamine Dependence Is Associated with Poorer Everyday Functioning	N = 90 Men • Methamphetamine Dependent (MD), n = 35 (Mage = 40.5, 54.3% HIV+) • Non-MD, n = 55 (Mage = 48.1, 43.6% HIV+) Entry Criteria • Male; English speaking; no major psychiatric or neurologic illness	• Inconsistency Cognitive IIV • Reaction time variability across the Conners’ CPT	• Cross-sectional study comparing 2 groups Primary Outcomes • To determine the relationship between methamphet-amine dependence and RT consistency on everyday outcome measures	• Higher IIV was related to recency of methamphetamine use, psychomotor slowing, executive dysfunction, driving simulator performance, cognitive complaints, and poorer laboratory-based functional declines. • HIV status was not a strong predictor in these analyses.	Strengths • Use of several measures of everyday functioning Limitations • Lack of longitudinal data • HIV serostatus was not the primary focus • Many statistical analyses, subject to alpha inflation
4. Harrison et al. (2017) The Nature and Consequences of Cognitive Deficits among Tobacco Smokers with HIV: A Comparison to Tobacco Smokers without HIV	N = 173 Adults • HIV+ Smokers, n = 103 (Mage = 47.9) • HIV- Smokers, n = 70 (Mage = 42.7) Entry Criteria • Smokers seeking assistance with cessation; no history of serious psychiatric or neurologic illness; no current substance abuse; no current use of smoking cessation medications; HIV- participants were excluded for heart, kidney, or liver disease	• Inconsistency Cognitive IIV • Inconsistency was computed using the coefficient of variation defined as the SD of reaction time divided by the mean RT on the CPT and N-Back tasks	• Cross-sectional study comparing 2 groups Primary Outcome • To determine whether HIV+ smokers differed from HIV- smokers on measures of cognition	• After controlling for demographic and smoking-related factors, participants with HIV performed worse than uninfected participants on measures of working memory, processing speed, and inconsistency.	Strengths • Inclusion of HIV- smokers as a control group Limitations • Lack of non-smoking control groups • Inclusion criteria differed slightly between HIV+ and HIV- participants
5. Clark et al. (2018) Early Life Stress-related Elevations in Reaction Time Variability Are Associated with Brain Volume Reductions in HIV+ Adults	N = 44 HIV+ Adults • High Early Life Stress (ELS), n = 26 (Mage = 46.08) • Low Early Life Stress, n = 18 (Mage = 44.44) Entry Criteria • Right-handed; native English speaker; no, major psychiatric or neurologic illness; no substance use in recent past; negative urine toxicology at baseline	• Inconsistency Cognitive IIV • RT latency and variability were assessed with the N-back task (test of working memory) administered during fMRI testing. • RT-IIV was calculated using the coefficient of variation for each participant such that CoV = SD across all correct trials divided by mean RT (thus mean RT is controlled).	• Cross-sectional study comparing 2 groups Primary Outcome • To determine the relationship between RT variability and ELS, brain volume, neuropsychiatric symptoms, and self-rated cognitive symptoms	• The high ELS group demonstrated greater RT-IIV than the low ELS group, even after neuropsychiatric symptoms were controlled. • RT-IIV was significantly associated with both gray and white matter volume across both groups, but the high ELS group showed lower gray and white matter volume than the low ELS group. • The negative association between IIV and subjective cognitive ratings was significant.	Strengths • ELS may be common in HIV+ groups and is associated with cognitive impairment. Limitations • Lack of HIV-negative control group • No NP battery to allow for correlation between RT-IIV and objective cognitive impairment • Patterns of neural activation across the two groups were not examined. • Lack of longitudinal data
6. Morgan et al. (2011) Intraindividual Variability in HIV Infection: Evidence for Greater Neurocognitive Dispersion in Older HIV Seropositive Adults	N = 166 Adults • HIV+ Adults, n = 126 (Mage = 45.1, n <50 years = 89) • HIV- Adults, n = 40 (Mage = 43.4, n <50 years = 26) Entry Criteria • No drug/alcohol abuse within six months (and negative urine toxicology screen); no significant neurological or psychiatric disease including current major depression or generalized anxiety disorder	• Dispersion Cognitive IIV • Dispersion was calculated using standard summary measures from 12 cognitive tests from multiple domains. Raw scores were converted into Z-scores and an intraindividual standard deviation was computed across Z-scores. This measure did not correct for level of performance (not mean-adjusted).	• Cross-sectional study comparing 2 groups Primary Outcome • Dispersion across cognitive domains	Analysis revealed a significant HIV x age interaction, with the older HIV+ group showing greater dispersion than the younger HIV+ participants and both young and old HIV-negative groups.	Strengths • IIV is especially relevant in HAND due to characteristic “spotty” pattern of deficits. Limitations • Lack of participants >age 65 • Disease duration not controlled • Lack of longitudinal data • Lack of norms for dispersion in healthy aging • Lack of longitudinal data
7. Morgan et al. (2012) Intra-individual Neurocognitive Variability Confers Risk of Dependence in Activities of Daily Living among HIV-seropositive Individuals without HIV-associated Neurocognitive Disorders	N = 82 HIV+ Adults without HAND (Mage = 45.1) Entry Criteria • No drug/alcohol abuse within six months (and negative urine toxicology screen); no significant neurological or psychiatric disease including current major depression or generalized anxiety disorder	• Dispersion Cognitive IIV • Dispersion across 13 cognitive tests from multiple domains in a single testing session. Raw scores from each measure were converted into T-scores and IIV SDs were computed.	• Cross-sectional study with 1 group Primary Outcome • Predictive value of dispersion for ADLs and a separate measure of self-reported medication adherence	• Positive association between dispersion and increased dependence in ADLs and medication adherence	Strengths • Broad NP battery • The Lawton and Brody ADL questionnaire is widely accepted. Limitations • Self-report measures of ADLs and medication adherence • Lack of longitudinal data precludes examining dispersion as early indicator • No control group with HAND or HIV-negative control group
8. Thaler et al. (2015) Increased Neurocognitive Intra-individual Variability Is Associated with Declines in Medication Adherence in HIV-infected Adults	N = 150 HIV+ Adults (Mage = 41.9) Entry Criteria • No significant history of neurological or psychiatric disease; demonstrated proper use of MEMS cap	• Dispersion Cognitive IIV • Baseline dispersion was calculated as the SD of 18 NP test T-scores. Change in dispersion was calculated by subtracting the dispersion score at 6-month follow-up from the baseline dispersion score and dividing the difference by the baseline score. Change scores for global NP test performance were also examined.	• Longitudinal study with 1 group Primary Outcome • Association between changes in dispersion and overall medication adherence over a six-month period	• After controlling for substance use, mean performance scores and depression, increased dispersion at 6-month follow-up was negatively correlated with medication adherence.	Strengths • Two time points 6 months apart • Use of MEMS cap to measure medication adherence Limitations • Lack of control group(s) • Small effect size
9. Hines et al. (2016) Cortical Brain Atrophy and Intra-individual Variability in Neuropsychological Test Performance in HIV Disease	N = 147 Men 50+ • HIV+ n = 80 (Mage = 59.57) • HIV- (N = 67, Mage = 60.81) Entry Criteria • No history of cardiovascular or cerebrovascular disease	• Dispersion Cognitive IIV • Dispersion across four tests of memory and one test of attention and motor speed. Dispersion was computed as the SD of the T-scores for each participant based on the five chosen tests. • Gray matter volume was measured with MRI.	• Cross-sectional study comparing 2 groups Primary Outcome • Association between total gray matter volume and dispersion	• Dispersion was positively associated with cortical atrophy and with gray matter volume in specific cortical regions, but not with HIV status.	Strengths • HIV-negative control group • Large sample size for imaging study • Dispersion measures across 4 memory tests, and one motor speed test Limitations • Dispersion measured across only two cognitive domains • Not generalizable to younger HIV+ individuals • Lack of longitudinal data
10. Anderson et al. (2018) Intraindividual Variability in Neuropsych- ological Performance Predicts Cognitive Decline and Death in HIV	N = 708 HIV+ Adults (Mage = 43.9 at baseline) Entry Criteria • Participant has data from at least three visits over a period of up to 14 years; diagnosed with severe/advanced HIV disease	• Dispersion Cognitive IIV • Dispersion across 15 cognitive scores at baseline assessment, calculated using the coefficient of variation (within person SD divided by the within person mean).	• Longitudinal study with 1 group Primary Outcome • Association between dispersion at the immediately preceding visit and transition to a more severe stage of HAND or death at the next visit	• Increased dispersion predicted HAND decline and mortality at next visit.	Strengths • Longitudinal trial of long duration (i14 years) • IIV measures independent of mean cognitive scores • Large sample; generalizable Limitations • Participants all had “severe” disease • Lack of control group(s)
11. Jones et al. (2018) Longitudinal Intra-individual Variability in Neuropsych- ological Performance Relates to White Matter Changes in HIV	N = 64 Adults • HIV+ Adults, n = 38 (Mage = 53.9) • HIV- Adults, n = 26 (Mage = 49.9) Entry Criteria • No significant history of neurological or psychiatric disease; negative urine toxicology screen for stimulants or hallucinogens; no current substance use disorder; no past dependence on stimulants	• Dispersion Cognitive IIV • Baseline and follow-up NP dispersion was calculated as the SD of the 14 NP scores for each participant. Change in IIV was calculated as the difference between the two time points. • Measures of white matter integrity were obtained through diffusion tensor imaging.	• Longitudinal study comparing 2 groups Primary Outcome • The longitudinal relationship between change in IIV and white matter integrity in HIV+ and HIV- adults	• The HIV+ group had wider dispersion of scores across domains at baseline but showed no group difference for the global cognition score. • Lower fractional anisotropy (FA) values in the superior longitudinal fasciculus (SLF) at follow-up were significantly related to older age, and IIV in the HIV+ group.	Strengths • Two time points two years apart • Use of imaging in combination with dispersion • Inclusion of a HIV-negative control group Limitations • Small samples
12. Levine et al. (2018) Intraindividual Variability in Neurocognitive Performance: No Influence Due to HIV Status or Self-reported Effort	N = 996 Men • HIV+ Men, n = 522 (Mage = 52) • HIV- Men, n = 474, Mage = 57.9) Entry Criteria • Male; no history of cocaine use in prior six months	• Dispersion Cognitive IIV • Dispersion was examined across a NP battery including measures of working memory, learning, memory, executive functioning, motor functioning, and speed of processing and a mean Z-score of NP functioning was calculated. Dispersion was calculated as the SD of the z-scores.	• Cross-sectional study comparing 2 groups Primary Outcome • To determine whether suboptimal effort as measured by the self-reported Visual Analogue Effort Scale is a predictor of dispersion in HIV+ and HIV- men	• After controlling for age, race, NP functioning, depression, and alcohol use, there was no effect of serostatus or effort on dispersion. The strongest predictor of dispersion was overall NP functioning such that higher NP scores were associated with less dispersion.	Strengths • Large sample size, making it generalizable • Inclusion of HIV-control group Limitations • Participants from this study drew from the same cohort as Hines (2016) and neither included females. • Only one time point • Questions about comparability of HIV samples across studies
13. Arce Renteria et al. (2020) Neurocognitive Intra-individual Variability within HIV+ Adults with and without Current Substance Use	N = 40 HIV+ Adults • Substance Use (SU) Group, n = 17 met criteria for current SU disorder (Mage = 44.29) • Non-SU Group, n = 23, no current or past substance use (Mage = 47.74) Entry Criteria • Latinx or non-Hispanic white race; stable cART for at least 12 weeks	• Dispersion Cognitive IIV • Dispersion across a NP battery. iSD was calculated across each demographically adjusted T-score. The coefficient of variance was then calculated to adjust for mean NP performance.	• Cross-sectional study comparing 2 groups Primary Outcome • Neurocognitive dispersion by HIV serostatus and substance use	• Preliminary evidence demonstrated that substance use is associated with greater NP dispersion among HIV+ adults and it may have a synergistic effect on medication adherence.	Strengths • Use of MEMS cap to measure medication adherence • Use of mean-adjusted measure of dispersion Limitations • Lack of HIV-negative control group • Small sample, mostly Latinx • Lack of longitudinal data

Notes. ADL = Activities of Daily Living; cART = Combined Antiretroviral Therapy; CPT = Connors’ Continuous Performance Test; EEG = Electroencephalogram; ELS = Early Life Stress; FA = Fractional Anisotropy; MEMS = Medication Event Monitoring (MEMS) cap system; MRI = Magnet Resonance Imaging; NP = Neuropsychological; SLF = Superior Longitudinal Fasciculus.

Inconsistency Cognitive IIV Studies

As seen in Table 2, research on inconsistency in PLWH is sparse, with most studies lacking non-HIV control groups and longitudinal measures. These studies are listed in chronological order by publication date.

Levine et al. (2008)

In a cross-sectional study exploring a five-factor attentional model (Mirsky et al., 1991), Levine et al. (2008) examined the association between each of five components of attention, and medication adherence and virologic variables. Participants were 104 PLWH (M_age = 40.9; 16.3% women; mean viral load = 7,848 (SD = 47,713)) with no significant neurological or psychiatric disease who self-reported no substance misuse at the time of testing. Participants engaged in the Conners’ Continuous Performance Test, which requires participants to press the space bar on a keyboard as fast as possible when a target appears on the screen, except when that target is an “X”, in which case the response must be inhibited. Following a principal component analysis, a “stabilize” factor (factor 4) was identified, which includes both inconsistency metrics and omission errors. This factor was significantly associated with medication adherence (R = 0.26, p = 0.01), as measured by the electronic Medication Event Monitoring cap system collected over six months. Stated differently, participants with high medication adherence showed less variability and fewer errors of omission than participants with low medication adherence. The association between the “stabilize” factor and a global neuropsychological score was not significant. As in most of these reviewed studies (see Table 2), this study’s lack of a non-HIV control group was a limitation.

Ettenhoffer et al. (2010)

Ettenhoffer et al. (2010) demonstrated that greater reaction time inconsistency was associated with global cognitive impairment (HIT SE r = −0.37, p = 0.01; Hit SE variability r = −0.44, p = 0.003), most recent CD4 count (Hit SE r = −0.32, p = 0.03), lowest CD4 count (HIT SE block change r = −0.31, p = 0.04), and highest viral load (HIT SE variability r = 0.35, p = 0.03). Participants were a relatively small sample of 46 PLWH (M_age = 41.52; 19.6% women; HIV viral load (most recent), median (interquartile range) = 116.50 (0–1644)) who completed a neuropsychological battery, including the Conners’ Continuous Performance Test and an electronic measure of medication adherence using the medication event monitoring system caps. Analysis of reaction time latencies on the Conners’ Continuous Performance Test revealed a significant negative association between latency and global cognitive ability (r = −0.30, p = 0.04), recent immunological measures (CD4 (r = −0.32, p = 0.03), and most recent viral load (r_s = 0.36, p = 0.02), but not medication adherence. Reaction time inconsistency, in contrast, was associated with medication adherence (HIT SE = −0.36, p = 0.02; Hit SE variability r = −0.38, p = 0.01; Hit SE block change r = −0.29, p = 0.05). These researchers suggested that inconsistency on a reaction time task across trials may provide a sensitive measure of cognitive/functional impairment and neuropathology in PLWH, and may be a marker of future cognitive and/or functional decline. It should be noted that the variability metrics used here are part of the Continuous Performance Test -II output, which is standard clinical practice, and are not directly calculated by the researchers.

Morgan et al. (2014)

Morgan et al. (2014) examined inconsistency in men with (n = 35; M_age = 40.5) and without (n = 55; M_age = 38.1) methamphetamine dependence recruited from a larger study on methamphetamine use. In this sample, 54.3% (mean HIV plasma viral load (log10) = 1.6; 73.7% on combination antiretroviral therapy) of the participants with methamphetamine use and 43.6% (mean HIV plasma load (log10) = 1.7; 60.9% on combination antiretroviral therapy) of the control participants had a diagnosis of HIV infection. Participants completed a comprehensive neuropsychological battery, simulated driving test, the Lawton and Brody Self-rated Activities of Daily Living Scale, and the Patient’s Assessment of Own Functioning Inventory. Cognitive IIV was measured on the Conners’ Continuous Performance Test. As expected, inconsistency was significantly greater in the methamphetamine positive group even when latency and errors were controlled (p = 0.03). Higher levels of inconsistency were associated with more recent methamphetamine use (F(2,32) = 4.81, p = 0.01) as well as poor executive functioning (rho = 0.34, p = 0.04) and slower speed of processing (rho = 0.35, p = 0.04) on neuropsychological testing. Inconsistency also significantly predicted a variety of everyday functions such as worse driving simulator performance (Adjusted R² = 0.4, F = 7.78, p = 0.001), worse performance on laboratory based functioning skills (i.e., USCD Performance-based Skills Assessment), Adjusted R² = 0.19, F = 3.41, p = 0.03), and more cognitive complaints on the Patient’s Assessment of Own Functioning Inventory (Adjusted R² = 0.23, F = 4.1, p = 0.02). The main effect of HIV serostatus was not examined but, unexpectedly, serostatus did not moderate the association between methamphetamine use and inconsistency.

Harrison et al. (2017)

Harrison et al. (2017) hypothesized that smoking might increase the risk for neurodegeneration and cognitive decline in PLWH. Participants were HIV+ smokers (n = 103; M_age = 47.9; 27% women; 100% stable on combination antiretroviral therapy) and HIV- smokers (n = 70; M_age = 42.7; 36% women), seeking assistance with smoking cessation. Visuospatial working memory, processing speed, attention, and inconsistency (computed using the CoV) were measured with the Penn Continuous Performance Test and the N-back task. A stepwise model retained the N-back discrimination index (difference between hits and false alarms) (OR = 0.001, p = 0.001, 95% CI: 0.0002 – 0.05), median reaction time to correct responses (OR = 1.01, p < 0.001, 95% CI 1.0 – 1.01), and the CPT coefficient of variance (OR = 2.1 × 10⁸, p = 0.002, 95% CI: 1,419 – 3.2 X 10¹³). Indicating that participants with HIV performed worse than HIV- participants on measures of working memory and processing speed. HIV+ smokers also demonstrated significantly greater inconsistency across the Penn Continuous Performance Test and the N-back tasks.

Clark et al. (2018)

Clark et al. (2018) noted that exposure to high levels of early life stress (ELS) are associated with greater cognitive impairment in PLWH and thus hypothesized that ELS might be associated with neuropsychiatric symptoms, self-rated cognitive symptoms, and brain volume. Participants were 44 PLWH who either experienced high ELS (n = 26; M_age = 46.08; 35% women; 65% virally suppressed) or experienced low ELS (n = 18; M_age = 44.44; 50% women; 53% virally suppressed). Reaction time latency and variability were measured during a test of working memory (N-back task) administered while the participant was undergoing an fMRI scan (fMRI analyses were not reported in this manuscript, which focused on structural correlates of IIV). IIV-inconsistency was calculated with the CoV. The high ELS group demonstrated greater reaction time-IIV than the low ELS group, even after neuropsychiatric symptoms were controlled (F[1,39] = 6.80, p = 0.013, partial Eta² = 0.15), and a negative association between IIV and subjective cognitive ratings on the Medical Outcomes Survey (MOS-HIV) was significant (β = −0.42 [t = −2.24, p = 0.035]). Nevertheless, MOS-HIV scores were not significantly different across the two groups. Reaction time inconsistency was significantly associated with both total gray and white matter volume across both groups, but the high ELS group showed lower gray and white matter volume than the low ELS group.

Summary of Inconsistency Studies

Overall, these IIV studies show that higher inconsistency is related to poorer cognitive functioning and predicts poorer everyday functioning as reflected by medication adherence, the Patient’s Assessment of Own Functioning Inventory, and self-reported cognitive complaints. Some studies show inconsistency correlates to reduced white and gray matter. Also, higher inconsistency is associated with increasing age, lower education, substance use, and early life stress. Interestingly, a limitation should be noted that measures of IIV inconsistency are generated from attention tests (CPT & N-back); thus, it is difficult to measure other cognitive constructs that would be amenable to inconsistency calculations.

Dispersion Cognitive IIV Studies

Most research on cognitive IIV in PLWH has incorporated dispersion across cognitive domains rather than inconsistency in reaction time within one task. As mentioned, dispersion is more complicated to calculate than inconsistency with multiple formulas for calculation reported in the literature. Here, dispersion has been defined using mean-independent variations (variability independent of the individual’s predicted mean score). These studies are listed in chronological order by publication date.

Morgan et al. (2011)

Morgan et al. (2011) explored the relationship between dispersion and aging in PLWH in a cross-sectional design with an HIV- control group. Participants were 126 HIV+ adults (M_age = 45.1; 11.1% women; plasma viral load (log10) = 2) and 40 HIV- controls (M_age = 43.4; 47.5% women) who were divided into an older (≥ 50 years) and a younger (<50 years) group. Dispersion was calculated based on 12 cognitive measures using the iSD method not adjusted for mean-based performance. Analysis revealed a significant HIV status by age group interaction on dispersion (p = 0.03). Stratified by serostatus, the older HIV+ participants (M = −0.06, SD = 0.12) displayed greater dispersion than the younger HIV+ participants (M = −0.12, SD = 0.11; r = 0.21, p = 0.02). Also, the older HIV+ participants displayed greater dispersion than the older HIV- control group (M = −0.13, SD = 0.1; t ratio = 2.1, p = −0.04, Cohen’s d = 0.6). The interaction remained significant when mean-based performance or global neuropsychological performance was considered and when medical variables were controlled.

Morgan et al. (2012)

In a cross-sectional study, Morgan et al. (2012) tested the hypothesis that higher levels of dispersion are associated with impairment in activities of daily living. Participants in this study consisted of 82 HIV+ individuals without HAND (M_age = 45.1; 9.8% women; plasma viral load (log10) = 2.3 (SD = 1.24)) who completed a broad neuropsychological battery in one testing session as well as the Lawton and Brody ADL questionnaire, and the self-reported “Beliefs Related to Medication Adherence” survey. Analysis revealed that dispersion, calculated as an intra-individual SD computed across neuropsychological T-scores, was a unique predictor of IADL dependence,( Χ² (1) = 9.7, p = 0.002 such that each unit increase in dispersion was associated with a 1.6 times increased risk of dependence in activities of daily living (95% CI: 1.2 – 2.4). Elevated dispersion was also associated with lower ratings of medication adherence (ps < 0.05). These findings suggest that PLWH who show high levels of dispersion on neuropsychological testing may have problems regulating cognitive resources to accomplish functional activities and thus be prone to disability.

Thaler et al. (2015)

Thaler et al. (2015) expanded on the work of Morgan et al. (2012) as described above. They investigated whether PLWH who, as yet, show no mean-based impairment on neuropsychological testing experience changes in dispersion over time that predict functional impairments, in this case in medication management. Participants were 150 PLWH (M_age = 41.9; 18% women; plasma viral load (log10) = 2.2 (SD = 1.9)), recruited from a larger study on adherence in PLWH, who completed a baseline neuropsychological battery and a six-month follow-up assessment. Medication adherence was assessed with the electronic medication event monitoring system. Dispersion was calculated in two ways, both as variability in performance at baseline (where it was computed as the iSD of the 18 neuropsychological test scores) and as change from baseline to most recent testing (six-month follow-up) where it was calculated by subtracting the follow-up score from the baseline score and dividing the difference by the baseline score (Δ iSD). After controlling for drug use, mean performance scores, and depression, increased dispersion at six-month follow-up was negatively correlated with medication adherence (R² change = 0.049, p = 0.005). In fact, over the six-month follow-up interval, more than twice as many participants who showed increased dispersion also became “poor medication adherers” than participants with stable or decreased dispersion, indicating that dispersion might be predictive of future poor adherence. Mean neuropsychological performance and change in mean neuropsychological performance over time did not differentiate between increased, stable, or decreased dispersion groups. The association between change in neuropsychological scores and adherence was not significant but did show a trend in the predicted direction (β = 0.14, p = 0.087).

Hines et al. (2016)

Hines et al. (2016) investigated the relationship between dispersion and neural integrity in older men with HIV. Participants were 147 gay and bisexual men (80 HIV+, M_age = 59.57, mean peak viral load (log10) = 4.57; 67 HIV-, M_age = 60.81) age 50 and older without a history of cardiac or cerebrovascular disease. Based on tests of four memory variables and one test of motor speed, dispersion was computed as the iSD of the demographically adjusted T-scores for each participant. As expected, greater IIV-dispersion was significantly associated with reduced white matter (β = −0.35) and gray matter (β = −0.38) volume (whole brain as well as bilateral inferior frontal gyrus, lateral sulcus, right superior parietal lobe, intraparietal sulcus, and dorsal/ventral regions of the posterior transverse temporal gyrus). In contrast to prediction, dispersion related gray matter changes did not differ by HIV serostatus and were not related to mean performance on cognitive testing.

Anderson et al. (2018)

This study examined whether dispersion over time predicts more severe cognitive and functional impairment, and even death, in a cohort of PLWH followed for up to 14 years. Participants were 708 people with advanced HIV (M_age = 43.9 at baseline; 21.5% women; mean plasma viral load (log10) = 2.22 (SD = 0.744)) who completed at least three neuropsychological evaluations over the follow-up period. Dispersion scores were based on neuropsychological T-scores computed from normative samples for each test and the CoV was calculated. Analysis revealed that a high dispersion score at the immediately preceding visit was a risk factor for transition to a more severe stage of HAND (p < 0.001) at the next visit (along with older age (p < 0.001). High levels of increased dispersion between the preceding and most current assessment also predicted a decline to a more severe HAND stage (p < 0.001) (as did length of time since diagnosis (p < 0.001) and older age (p < 0.001)). Lastly, dispersion at the last available testing was predictive of mortality (p < 0.01). Overall, higher educational attainment (p < 0.001) and higher baseline cognitive functioning (p < 0.001) were associated with less dispersion. This result is consistent with findings that dispersion is associated with markers of immunosuppression, decreased white matter integrity, decreased gray matter volume, and impaired functional abilities (e.g., medication adherence).

Jones et al. (2018)

In a longitudinal study, Jones et al. (2018) examined the relationship between changes in dispersion and white matter integrity among HIV+ and HIV- adults. If elevated IIV is a marker for frontal-subcortical functioning (Bellgrove et al., 2004; Troyer et al., 2016), increases in dispersion on neuropsychological testing should be associated with reduced white matter integrity over time. In fact, in a structural MRI study, Alakkas et al. (2019) observed that abnormal white matter was observed in HAND compared to cognitively normal PLWH. In Jones et al. study, participants were 38 PLWH (M_age = 53.9; 14% women; mean peak viral load (log10) = 10.25 (SD = 3.38)) and 26 HIV- controls (M_age = 49.9; 42% women) who completed neuropsychological testing and a DTI scan at baseline and at a two-year follow-up. Using the iSD method, analysis of neuropsychological results revealed that, despite no difference in global cognition score, the HIV+ group had wider dispersion of scores across domains at baseline (t(62) = 2.03, p = 0.047, Cohen’s D = 0.53). Changes in global cognition and changes in cognitive IIV were mildly correlated (r = −0.241, p = 0.037, r² = 0.058), such that dispersion increased as global cognition declined. Imaging revealed that lower fractional anisotropy (FA) values in the superior longitudinal fasciculus at follow-up were significantly related to greater dispersion in the HIV+ group (β = −0.202, p = 0.007), but not in the HIV- group (β = −0.100, p = 0.376). The superior longitudinal fasciculus is involved in frontal-posterior connections and has been shown to be important in aspects of executive function (Nowrangi et al., 2014). Lower FA values were also seen in the anterior thalamic radiation projections between the anterior and middle portions of the thalamus with the frontal lobe (George & Das, 2020), and with reduced anterior thalamic radiations white matter integrity associated with cognitive deficits in schizophrenia (Mamah et al., 2010). In this study, changes in the mean global cognitive score were not associated with white matter integrity. There was a significant group by IIV dispersion score interaction effect in the superior longitudinal fasciculus demonstrating that the relationship between dispersion and white matter integrity was specific to HIV. These researchers contend that IIV dispersion is likely more sensitive than mean-based performance on cognitive tests in PLWH. Of note, FA values did not decline over the two years, suggesting that changes in white matter integrity may predate the elevated IIV dispersion.

Levine et al. (2018)

Levine et al. (2018) examined whether dispersion measures could be affected by effort on neuropsychological tasks. In this study, 996 male participants (HIV-, n = 474, M_age = 57.9; HIV+, n = 522, M_age = 52, 90% of cases having a viral load < 120) completed a visual analogue scale of perceived effort (rated 0% to 100%) following a comprehensive neuropsychological battery and questionnaires. Dispersion was determined using the calculations followed by Morgan et al. (2012), the iSD method. Elevated dispersion was associated with poorer neuropsychological functioning (Z-score (Beta = −0.040) and race (i.e., being African American) (Beta = 0.014) for the group as a whole but not with self-rated effort and did not differ between the serostatus groups.

Arce Renteria et al. (2020)

Arce Renteria et al. (2020) tested the hypothesis that dispersion would differentiate the unique contribution to cognitive impairment due to substance use. Participants were 40 PLWH (17 with active substance use disorder within the past 12 months and 23 with no history of substance use; M_age = 46.3; 30% women; 29% had undetectable viral load) who completed a neuropsychological battery and a measure of medication adherence (i.e., medication event monitoring system caps). Although analysis revealed that participants in the substance use group had a significantly more impaired mean global deficit score based on neuropsychological testing, this effect was largely attributable to greater depressive symptomatology. Yet, after controlling for depression, the substance use group showed significantly greater dispersion across the neuropsychological battery than the non-substance use group (Cohen’s d = 0.81), t(38) = 2.04, p = 0.049). The relationship between dispersion and medication adherence tended toward significance for the substance use group (b = −75.55, SE = 32.07), 95% CI [−140.90, −4.19], p = 0.039) but the effect was reduced when adjusted for depression.

Summary of Dispersion Studies

Overall, these dispersion studies show that higher dispersion is related to poorer cognitive functioning and predicts poorer everyday functioning such as medication adherence. Some studies show dispersion correlates to poorer integrity of frontal brain regions and the superior longitudinal fasciculus. Also, higher dispersion is often associated with increasing age, lower education, substance use, and sometimes with HIV status. Interestingly, high dispersion scores may help predict which patients will develop HAND, potentially leading to early intervention.

Synthesis and Discussion

Cognitive IIV appears to be a promising approach to examine dynamic processes that are not captured on traditional mean-based neuropsychological testing. Although these ideas of cognitive IIV in the literature have been in the literature for decades, it is only the past few years that the topic has emerged with renewed interest as reflected by more published studies each year expanding this in various clinical populations (Ram et al., 2015). So far, most of the studies described in our systematic review share limitations including small sample size, lack of a non-HIV control group, and the absence of longitudinal data. In addition, inconsistent formulas for calculating dispersion complicates comparison across studies (Table 2). Normative data for cognitive IIV are also lacking, so it is difficult to determine whether a given level of cognitive IIV is normal or not given an individual’s demographic background (Vandermorris & Tan, 2015). Similarly, cognitive IIV in HIV may be affected by factors known to impact cognition itself, such as low education, poor work history, substance use, and other comorbidities such as hepatitis C and cardiovascular disease common to HIV (Acre Renteria et al., 2020; Morgan et al., 2012b; Waldrop et al., 2021).

Cognitive IIV and Structural Neuroimaging

Some studies suggest that cognitive IIV in HIV and other populations may be associated with damage to the frontal-subcortical region and associated neural networks, regions associated with executive functioning. It is important to compare these neuroimaging studies between those with and without HIV to see if the same patterns emerge or if there are disease-related factors at work. Neuroimaging studies in non-HIV samples have shown that cognitive IIV (usually measured as inconsistency) is associated with specific brain morphology. In a sample of 469 community-dwelling older adults 60–64 years old without HIV, Bunce et al. (2007) observed that frontal lobe white matter hyperintensities (derived from T2-weighted MRI) were associated with reaction time inconsistency, but these were not associated with other brain regions (i.e., parietal, temporal, periventricular body, etc.). In a non-HIV sample of 25 adults with focal frontal brain lesions, 11 adults with non-frontal brain lesions, and 12 adults with no brain lesions, Stuss et al. (2003) observed that abnormal dispersion, as measured across different types of reaction tests, was found in only those with frontal lesions. Similarly, in a sample of 42 healthy adults 18 to 46 years old, Bellgrove et al. (2004) observed that higher cognitive inconsistency predicted increased frontal lobe activity as well as recruitment of right inferior parietal and thalamic regions.

The few neuroimaging studies in HIV samples also demonstrate that cognitive IIV is associated with specific brain morphology. In a sample of 80 men with HIV and 67 men without HIV 50 and older, Hines et al. (2016) observed that higher dispersion was associated with less total gray matter volume in specific regions (i.e., the inferior frontal gyrus bilaterally, dorsal/ventral regions of the posterior section of the transverse temporal gyrus, etc.); interestingly, there were no difference by HIV status or cardiovascular disease factors. Using neuroimaging and cognitive evaluation of change in dispersion of 14 cognitive tests at baseline and two years later in a sample of 38 PLWH and 26 without HIV, Jones et al. (2018) found that lower FA values on DTI in the superior longitudinal fasciculus at two-year follow-up were significantly related to older age and cognitive IIV in PLWH. The superior longitudinal fasciculus is involved in frontal-posterior connections and has been shown to be important in executive function. Lower FA values were also seen in anterior thalamic radiations. Arce Renteria et al. (2020) suggest an association between cognitive IIV and dopamine dysfunction related to deterioration in the basal ganglia and related structures.

De Felice and Holland (2018) explain cognitive IIV using the Compensation-Related Utilization of Neural Circuits Hypothesis which recognizes a trade-off between compensatory potential (reserve) and task difficulty (Reuter-Lorenz & Cappell, 2008). Based on the work of Reuter-Lorenz and Cappell (2008), this approach suggests that as a function of a top-down process (e.g., executive functioning) older individuals can compensate for cognitive decline by activating more neural regions as long as the task difficulty does not exceed the cognitive reserve. In this case, cognitive IIV may or may not be higher but overall performance does not suffer. Once the task difficulty exceeds the available cognitive or neurological reserve, performance declines and cognitive IIV increases. This theory was tested by Cappell et al. (2010) who used fMRI to demonstrate high activation of the dorsolateral prefrontal cortex in older adults who were unimpaired on performance of a relatively low-demand memory task but had less dorsolateral prefrontal activation and poor performance when the task’s memory demand became high.

Theories of Cognitive IIV

Cognitive theories proposed to explain cognitive IIV in HIV are similar and not fully elaborated, often attributing both dispersion and inconsistency to a disruption in executive function and overall executive control. For example, Bellgrove et al. (2004) proposed that cognitive IIV is related to changes in executive control. They suggest that the onset of cognitive decline is associated with difficulties in maintaining focus and top-down attentional control on cognitive tasks which, in turn, results in performance variability in patients who do not yet demonstrate mean-based impairment. Thaler et al. (2015) noted that these fluctuations may affect everyday activities such as driving. Anderson et al. (2018) proposed that cognitive IIV may reflect a “greater executive control demand” (p. 971) and reduced brain efficiency in PLWH. In addition, it was observed that both when the brain was at rest (i.e., task-negative) and when it was engaged in cognitive performance (i.e., task-positive), the active brain networks predict cognitive IIV. The finding that network connectivity is associated with cognitive IIV is supported by a study using graph theory, which demonstrated that network organization predicted dispersion (Stevens et al., 2012).

Implications for Clinical Practice

From a clinical perspective, if cognitive IIV is associated with disease severity and progression, as well as medication adherence, elevated cognitive IIV on repeat neuropsychological assessments can be a marker that indicates greater intervention may be warranted by the treating team. An important consideration is that the cognitive deficits and inefficiencies experienced in PLWH may be inadvertently minimized or understated if only normative mean-based scores are used to evaluate cognition. Greater IIV indicates that a person may not perform the same way consistently if re-evaluated and so a psychometrically “average” score on testing in a controlled, quiet environment may lack ecological validity, and might miss day-to-day variability.

This review of the literature suggests that both inconsistency and dispersion have potential for early diagnosis of cognitive and functional impairment in PLWH. Reaction time inconsistency has the advantage of being quick, convenient to administer, and low cost. Dispersion can be calculated from neuropsychological testing that is already being conducted, especially if testing fails to reveal a cause for self-reported symptoms.

Implications for Research

Cognitive IIV in PLWH represents a novel way to examine cognition and perhaps develop or modify cognitive interventions. Future investigations on multiple aspects of IIV in PLWH are warranted. First, none of the reviewed cognitive IIV studies in PLWH examined both dispersion and inconsistency in the same sample; it would be of value to determine whether they are highly correlated as has been observed in non-HIV related studies (Hultsch et al., 2002). Hultsch et al. (2002) examined the relationship between dispersion and inconsistency in four age-groups (young, n = 99, M_age = 23.17; young-old, n = 178, M_age = 60.38; mid-old, n = 361, M_age = 69.56; and old-old, n = 224, M_age = 79.33) on two simple and two complex reaction time tasks. Analysis revealed that both inconsistency and dispersion were significantly higher in the older age groups. A significant correlation between dispersion and inconsistency was maintained when reaction time latency was controlled, demonstrating that participants with the highest dispersion scores were also highly inconsistent. It has been suggested that inconsistency and dispersion may both result from the same mechanism involving “executive dyscontrol” (Morgan et al., 2012a) or the “efficiency with which executive control processes are implemented” (Bellgrove et al., 2004).

Furthermore, cognitive studies should include a reaction time test along with the traditional comprehensive neuropsychological battery; this can be accomplished without much additional cost or burden to study participants. In fact, it would be an additional value to see which measure, reaction time inconsistency or dispersion, has stronger predictive value with factors such as cognitive domain function and global cognitive function, brain chemistry, and brain morphology. For example, there are many biomarkers (i.e., IL-6) that are predictive of cognitive functioning (Fazeli et al., 2020); likewise, compared to domain cognitive scores, reaction time inconsistency and dispersion may be even more associated with certain biomarkers found in the literature.

Second, cognitive IIV has been used as a novel and interesting variable, and it has in some studies outperformed mean-based cognitive measures, to predict non-cognitive behaviors such as medication adherence (Thaler et al., 2015). Although cognitive measures clearly have value in predicting non-cognitive behaviors (i.e., driving), we may find that IIV has a slight incremental advantage in some cases. But that is not to say that individual cognitive measures, such as the Useful Field of View Test (a measure of speed of processing) does not possess unique predictive value in predicting non-cognitive behavior of driving in adults including PLWH (Vance et al., 2014), which it does; but we may find that repeated trials of the Useful Field of View test to produce an IIV inconsistency coefficient could incrementally increase the sensitivity of this already sensitive measure. Yet, it is not clear which type of cognitive IIV, inconsistency or dispersion, is more predictive. Just as cognition is used to predict other behaviors such as decision capacity, risky decision making, automobile driving behaviors, financial management, and health literacy to name a few (Vance et al., 2014; Waldrop et al., 2021), the association between cognitive IIV and other behavioral phenomena should be investigated. In one example of a study relating cognitive IIV to behavior, inconsistency has recently been shown to predict falls in a study of 108 community-dwelling older adults (Bauermeister et al., 2017). Given interest between cognition and falling in PLWH (Sharma et al., 2018), cognitive IIV may also be a more germane, incrementally sensitive predictor of falling beyond mean-based cognitive performance.

Fourth, cognitive IIV may also be related to other types of neurological IIV that could be considered in research. For example, Bauer (2018) used an EEG measure of IIV (EEG readiness amplitude potential) to investigate the relationship between overweight body mass index (BMI), HIV serostatus, IIV-inconsistency_, and the EEG measure. Participants from four groups (HIV+, BMI ≤25 (n = 52, M_age = 40.8); HIV+, BMI >25 (n = 41, M_age = 40.7); HIV-, BMI ≤25, (n = 34, M_age = 38.6); HIV-, BMI >25, (n = 51, M_age = 38.7)) completed a simple stimulus detection/time estimation task while EEG data were collected. As predicted, the HIV+ group showed significantly greater variability in response time and EEG response potential amplitude than the HIV- group. In addition, the high BMI group showed greater reaction time inconsistency and readiness potential amplitude than the normal weight group. Engagement with other such neurological IIV measures may yield further insights.

Fifth, cognitive training protocols are being used with more frequency in PLWH to address cognitive impairments including HAND (Vance, Fazeli et al., 2019). For example, in a sample of 88 PLWH with HAND, Author (Vance, Fazeli et al., 2022) developed an individualized-targeted computerized cognitive training program that, based on the participant’s baseline cognitive performance, prescribed particular cognitive-domain specific games designed to improve function in two specific cognitive domains in order to reverse a diagnosis of HAND (i.e., cognitive domains with performance more than 1 SD below the normative mean). Based on Salthouse’s Diminished Speed of Processing Theory and the Wickens Model of Information Processing, a framework was created to favor training in these domains if impairments were detected in speed of processing and/or attention. Thus, participants received 10 hours of training in each of these domains; if not, training was targeted to the cognitive domains that contributed to the HAND diagnosis but were closest (i.e., least impaired) to being within normal range. It was hoped that by manipulating the Frascati criteria in such a manner, the HAND diagnosis could be simply reversed by slight performance improvements in only two domains. Unfortunately, this study revealed that despite overall improvement in global cognitive functioning, HAND was not reversed. Considering how dispersion is calculated, that is the lowest cognitive performance scores driving greater dispersion, a better therapeutic approach would be to target cognitive domain training to those cognitive domains in which the performance was the worst, with no favor given to speed of processing training or attention training. Nonetheless, if a cognitive domain were to be favored, theoretical assumptions suggest that working memory and executive functioning are the domains that most underlie cognitive IIV (Anderson et al., 2018; Bellgrove et al., 2004; Stevens et al., 2012). Thus, it may be more therapeutically beneficial to target working memory and executive functioning for cognitive training, with the hope of reducing cognitive IIV. Therefore, based on the concept of IIV, there are two approaches to target cognitive training: 1) target the most impaired cognitive domains for training, or 2) apply working memory/executive functioning training. A potential research project would be to test these two approaches to see which one reduces cognitive IIV and produces the best cognitive outcomes.

Sixth, some cognitive interventions currently being used appear to have yielded minimal improvement, suggesting the intervention is not effective. For example, Morrison et al. (2020) examined the cognitive effects of a ketogenic diet on PLWH. Some improvements were observed in executive function and speed of processing, but not in overall global cognitive functioning; however, if dispersion was considered, this study may have found a reduction in cognitive IIV. Thus, researchers of past cognitive intervention studies should consider revisiting their data to see if their intervention reduced cognitive IIV. Likewise, future cognitive intervention studies should consider adding measures of cognitive IIV to examine therapeutic effects of their interventions.

Seventh and finally, other comorbidities (i.e., heart disease, depression, liver disease) common with aging with HIV are also known to independently impact cognition and are likely to exacerbate cognitive IIV in PLWH (Cody & Vance, 2016; Vance et al., 2011). For example, hepatitis C co-occurs in nearly 30% of PLWH (Sun et al., 2020) and it is observed that mean-based cognitive performance is lower in those with co-occurring HIV and hepatitis C than HIV or hepatitis C alone (de Almeida et al., 2018). Moreover, Morgan et al. (2012b) observed that in a demographically-matched sample of 37 adults with and 45 adults without hepatitis C, cognitive IIV was significantly higher in those with hepatitis C, independent of mean-based cognitive function. This suggests that when examining cognitive IIV in PLWH, it is important to take this into consideration and either screen out or control for such comorbidities.

Lingering Practical and Conceptual Questions

In the larger cognitive IIV literature, several lingering practical and conceptual questions have emerged that apply to neuroHIV research and beyond. They are worth considering within the context of the systematic review presented in this article. They include: 1) the use of iSD vs CoV; 2) dispersion metric procedures; 3) whether inconsistency and dispersion are the same dynamic process, and 4) construct validity and reliability.

First, a lingering conceptual question is whether to use iSD vs CoV; however, there appears to be no overall consensus in the literature (yet). iSD is the standard deviation based on the individual’s mean-based performance (i.e., overall test battery mean, which can also be used as a covariate in these instances), but CoV corrects for the mean-based performance (CoV = iSD/iM) (Christensen et al., 2005; Guilford, 1956; Hultsch, MacDonald, Hunter, Levy-Bencheton, & Strauss, 2000). An advantage of CoV is that by removing the unit of measurement, it can adjust for the individual’s overall performance, producing an IIV metric proportional to one’s global performance (Christensen et al., 2005). In fact, it can be transformed to a percentile that aids interpretability. The disadvantage of CoV is that it will be more heavily weighted at the extremes for highly and lowly cognitively functioning individuals (e.g., Alzheimer’s disease). Other researchers seem to favor iSD over CoV. Wojtowicz et al. (2012) asserts that CoV should not be employed because it is crude and only accounts for mean-based performance variation, ignoring other considerations such as learning and practice effects, participant effort, and true group differences. They favor iSD instead, but after incorporating residual scores (Hultsch et al., 2000). Similarly, Morgan et al. (2011, 2012a) assert no meaningful differences between CoV and iSD, except CoV is more complicated and iSD may be sufficient.

Second, concerning the dispersion metric, numerous factors and psychometric properties of the individual cognitive tests may alter cognitive IIV dispersion values. First, some tests may have ceiling or floor effects in which there is little variance. This includes screening measures such as the Mini-Mental Status Exam in which most individuals will score high; screening measures are not normally distributed and should be avoided in such dispersion metrics. Second, depending on the type of study or clinical assessment, some cognitive batteries may be loaded with cognitive measures focused on a specific cognitive domain, so the dispersion metric may be more biased. And third, some cognitive batteries may have more tests than others as has been seen in this systematic review; more tests administered will decrease dispersion, just as increasing a study’s sample size will decrease the standard deviation. Thus, although this is not proposed in the IIV dispersion field (to our knowledge), standardization of dispersion metric procedures would help advance the field. Such standardization could include cognitive tests from representative cognitive domains, tests with a normal distribution (i.e., no ceiling/floor effects), and stating a range in the number of cognitive tests (e.g., 5–9) that should be selected. Such standardization is needed before a discussion of generating IIV norms can be considered.

Third, cognitive inconsistency and dispersion are implicitly assumed by many to be the same underlying dynamic process, but the evidence for that is unclear. We need studies to include both inconsistency and dispersion within their protocols, and not just one or the other. This is especially needed in neuroimaging studies where we can then compare whether inconsistency or dispersion are associated with the same brain imaging indicators; if they are not, that could suggest that these are different dynamic processes after all. In one study of 304 non-demented older adults, Hilborn et al. (2009) incorporated both types of cognitive IIV; dispersion significantly correlated with inconsistency as measured by a complex reaction time test (r = 0.384, p < 0.01) and choice reaction time 1-back test (r = 0.314, p < 0.01); despite the significant correlation, this lack of a large correlation can also suggest that these might be measuring related, but different dynamic processes.

Finally, this brings us to the argument of whether cognitive IIV is a reliable construct. As mentioned, studies incorporate different measurement paradigms and various cognitive assessments with some emphasizing attention more and others more executive functioning and memory tasks to generate values for calculations for inconsistency and dispersion. In that regard, the generated IIV value may be based on a particular cognitive domain more than being a general metric of overall cognitive variability. Embedded within this discussion is the underlying assumption that IIV is a reliable construct. We do observe supportive evidence in studies that higher cognitive IIV is observed in older and/or more clinically compromised samples than younger and healthier samples. Albeit, systematic study of cognitive IIV is needed to examine test-retest across different time points as well as different batteries comprised of alternate measures (e.g., parallel test forms or different tests assessing the same construct).

Conclusion

The history of cognitive IIV in the literature is long and replete with surprising findings, sometimes contradictory. But over the decades, it has provided intriguing insights into how there may be underlying dynamic processes at work in stabilizing our cognitive functioning that may predict future cognitive functioning and behaviors. Yet, whether it is used to examine normal cognitive aging, HIV, or other clinical populations, there is a great deal to resolve concerning measurement issues as well as construct validity and reliability for this field of inquiry to blossom.