Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2012 Feb 1.
Published in final edited form as: Rehabil Psychol. 2011 Feb;56(1):77–84. doi: 10.1037/a0022753

Prospective Memory Deficits Are Associated with Unemployment in Persons Living With HIV Infection

Steven Paul Woods 1, Erica Weber 2, Bradley M Weisz 3, Elizabeth W Twamley 4, Igor Grant 5; the HIV Neurobehavioral Research Programs Group6
PMCID: PMC3264430  NIHMSID: NIHMS349541  PMID: 21401289

Abstract

Objective

To determine whether deficits in prospective memory (i.e., “remembering to remember”) confer an increased risk of unemployment in individuals living with chronic HIV infection.

Methods

Fifty-nine Unemployed and 49 Employed individuals with HIV infection underwent comprehensive neuropsychological and medical evaluations, including measures of prospective memory.

Results

The Unemployed participants demonstrated significantly lower performance on time- and event-based prospective memory, which was primarily characterized by errors of omission. Importantly, prospective memory impairment was an independent predictor of unemployment when considered alongside other neurocognitive abilities, mood disturbance, and HIV disease severity.

Conclusions

Prospective memory impairment is a salient predictor of unemployment in persons living with HIV infection and might be considered in screening for unemployment risk and developing vocational rehabilitation plans.

Keywords: AIDS dementia complex, employment, prospective memory, episodic memory, everyday functioning

Early in the human immunodeficiency virus (HIV) epidemic, the life expectancy for infected individuals who had received an AIDS diagnosis was only approximately one year (e.g., Lemp et al., 1990). However, since the introduction and widespread use of combined antiretroviral therapies (cART) in 1996, survival rates have improved dramatically (Centers for Disease Control [CDC] and Prevention, 2007) and many individuals are now living for decades with chronic HIV infection (The Antiretroviral Therapy Cohort Collaboration, 2008). Thus, although HIV remains a significant cause of death (CDC, 2007), it is increasingly perceived as a manageable, chronic medical condition akin to diabetes or hepatitis C infection. Although cART has greatly reduced mortality and improved systemic HIV disease outcomes (Kitahata et al., 2009), daily functioning capabilities (Goldman & Bao, 2004), and quality of life (Parsons, Braaten, Hall, & Robertson, 2006), individuals with chronic HIV infection nevertheless experience the burdens of a host of medical (e.g., cardiovascular disease, hepatitis C co-infection), neuropsychiatric (e.g., mood disorders, neurocognitive impairment), and psychosocial (e.g., functional independence, stigma) difficulties that are increasingly relevant to rehabilitation psychologists.

Unemployment is one such psychosocial problem that is highly prevalent among individuals living with HIV. It is estimated that one-half to two-thirds of the HIV infected population in the United States are unemployed (e.g., CDC, 2010), which is six times higher than the estimates for the general U.S. population. When considered alongside data on the adverse influence of unemployment on health-related quality of life (e.g., Blalock, McDaniel, & Farber, 2002), such high prevalence estimates underscore the potential importance of identifying novel and potentially remediable predictors of unemployment in persons living with HIV. Research to date has identified a number of important risk factors for unemployment. For example, individuals with lower CD4 cell counts (Blalock et al., 2002; Lem et al., 2005), AIDS diagnoses, and physical disabilities (Rabkin, McElhiney, Ferrando, van Gorp, & Lin, 2004) are all at greater risk for unemployment. Demographic characteristics, including older age (e.g., Burns, Young, & Maniss, 2006) and lower levels of education (Fogarty, Zablotska, Rawstorne, Prestage, & Kippax, 2007), as well as psychiatric comorbidities, such as histories of major depression (Rabkin et al., 2004) or alcohol and substance use disorders (e.g., Lem et al., 2005; van Gorp et al., 2007) have also been associated with a reduced likelihood of employment in HIV.

HIV-associated neurocognitive impairment is another important risk factor for unemployment (e.g., Heaton et al., 1994). Approximately 30-50% of persons living with HIV infection demonstrate global impairment in neurocognitive functioning, which – consistent with the primarily frontostriatal neuropathogenesis of HIV – is most commonly evident in executive functions, psychomotor speed, working memory, and the strategic aspects of episodic memory (e.g., Heaton et al., 2010). In their seminal study, Heaton and colleagues (1994) reported that individuals with global HIV-associated neurocognitive impairment were nearly twice as likely to be unemployed as compared to persons with normal neurocognitive performance. Subsequent studies in the pre-cART (e.g., Albert et al., 1995) and cART (e.g., Heaton et al., 2004; Twamley et al., 2006) eras have demonstrated similar adverse effects of global HIV-associated neurocognitive impairment on vocational outcomes, including return to work (e.g., van Gorp et al., 2007; cf. Chernoff, Martin, Schrock, & Huy, 2010). At the level of specific cognitive domains, HIV-infected individuals with deficits in psychomotor skills (van Gorp, Baerwald, Ferrando, McElhiney, & Rabkin, 1999; van Gorp et al., 2007), executive functions (van Gorp et al., 1999; Rabkin et al., 2004; Chernoff et al., 2010), or episodic memory (van Gorp et al., 1999; 2007) appear to be at particular risk of being unemployed.

In an effort to extend this literature, the goal of the current study was to investigate the relationship between deficits in prospective memory (ProM) and unemployment in individuals living with HIV. ProM is a form of episodic memory that describes one's ability to successfully execute a delayed intention, or “remember to remember.” In other words, ProM describes the complex cognitive process of forming an intention to carry out a specific behavior at a particular future occasion (e.g., take a medication after dinner), maintaining that intention while engaged in other activities (e.g., normal daily life), detecting the proper cue for intention retrieval (e.g., dinner), and retrieving and executing the correct intention (e.g., taking the medication) (Kliegel Jäger, Altgasen, & Shum, 2008). Apparent in this description of the construct is the ubiquitous nature of ProM to daily life; in fact, it has been argued that ProM may be more relevant to many aspects of everyday functioning than other cognitive constructs, such as retrospective memory (i.e., recollection of past events) or executive functions (Woods, Iudicello, et al., 2008).

It is therefore reasonable to posit that ProM may play a key role in vocational functioning, including acquisition and maintenance of paid employment. ProM would be relevant to, for example, submitting job applications, attending interviews, arriving at work on time, and carrying out many assigned work tasks. Yet we are unaware of any prior studies that have examined the association between employment and ProM in HIV disease (or in any other clinical sample, for that matter). This is a relevant undertaking because individuals living with HIV infection report frequent ProM failures in their daily lives (e.g., Woods et al., 2007) and evidence deficits in time- and event-based ProM in the laboratory (e.g., Carey et al., 2006; Martin et al., 2007), the profile of which is suggestive of deficits in strategic encoding and retrieval (e.g., Carey et al., 2006). Moreover, HIV-associated ProM impairment is singly dissociable from deficits in other cognitive functions, including retrospective memory (e.g., list learning) and executive functions. Evidence for the separability of ProM from cognitive (Gupta et al., 2010), neurobiological (Woods, Morgan et al., 2006), and functional (e.g., Woods, Iudicello, et al., 2008) studies. For example, Woods et al. (2006) reported that ProM, but not RetM, was associated with specific biomarkers of HIV neuropathogenesis, including tau. Of particular relevance to the present investigation, several studies now show that ProM provides incremental value over standard neurocognitive tests, psychiatric factors, and disease severity in predicting everyday functioning outcomes in HIV, including dependence in instrumental activities of daily living (Woods, Iudicello et al., 2008) and medication management (e.g., Contardo, Black, Beauvais, Dieckhaus, & Rosen, 2009; Woods, Moran, Carey, et al., 2008; Woods et al., 2009).

The current study was therefore designed to test the hypothesis that ProM deficits confer an increased risk of unemployment in persons living with HIV infection. It was expected that HIV-infected individuals who were unemployed would report more frequent ProM complaints and demonstrate worse performance on a standardized laboratory measure of ProM. It was also hypothesized that the relationship between HIV-associated ProM impairment and unemployment could not be better explained by other neurocognitive deficits, HIV disease severity, or psychiatric factors.

Method

Participants

Participants were selected from among 221 persons enrolled in an NIMH-funded study of prospective memory in HIV infection, which was housed within the University of California, San Diego (UCSD) HIV Neurobehavioral Research Programs (HNRP) and approved by the UCSD Human Research Protections Program. From this larger study, we selected 108 eligible individuals who were infected with HIV (as determined by an enzyme linked immunosorbent assay and a confirmatory Western blot) and were between the ages of 25 and 60 years, in an effort to target individuals in their prime working years. Potential participants were excluded if they reported a history of a psychiatric (e.g., psychotic disorders) or neurological (e.g., seizure disorders, closed head injuries with loss of consciousness greater than 15 minutes, stroke, or active CNS opportunistic infections) disorders known to affect cognition. We also excluded participants who were working part-time or on a strictly volunteer basis. Finally, participants were also excluded if they met diagnostic criteria for a substance use disorder within six months of the examination or tested positive on a urine toxicology screening for illicit drugs (e.g., methamphetamine) or a Breathalyzer test for alcohol on the day of testing.

Participants who reported currently working full-time were classified as “Employed” (n = 49). The remaining 59 participants were not working at the time of evaluation and were classified as “Unemployed.” Thirty-nine (66%) of these individuals were receiving disability benefits. Note that, there were no significant differences in the demographic, psychiatric, or ProM characteristics of the disabled and non-disabled participants in the Unemployed sample (ps > .10). As might be expected, the disabled participants were more likely to carry AIDS diagnoses (80 vs. 50%) and had been infected longer (16 vs. 11 years) than the non-disabled unemployed subsample (ps < .05), but did not differ on any other indicator of HIV disease severity (ps > .10).

The demographic, disease, and psychiatric characteristics for the Employed and Unemployed participants are displayed in Table 1. The study groups did not differ in age, education, gender, estimated VIQ (as determined by the Weschler Test of Adult Reading; Psychological Corporation, 2001), or ethnic identity (ps > .10), but the Unemployed sample had slightly higher Hollingshead scores (p < .10), indicating slightly lower socioeconomic status as estimated by educational and career attainment.

Table 1.

Psychosocial, disease, and psychiatric characteristics of the study samples

Participant Characteristic Employed (n = 49) Unemployed (n = 59) p
Psychosocial
    Age (years) 43.3 (7.5) 44.7 (5.5) 0.285
    Education (years) 13.9 (2.2) 13.2 (2.8) 0.142
    Estimated Verbal IQ 106.6 (11.6) 103.0 (11.1) 0.107
    Hollingshead scorea,b 33.0 (27.5, 38.5) 40.0 (26.0, 47.0) 0.087
    Sex (% Male) 90% 88% 0.785
    Ethnicity (% Caucasian) 65% 66% 0.616
Disease
    Duration of Infection (years)a 11.5 (6.1, 18.8) 15.4 (7.4, 19.7) 0.244
    Nadir CD4 Counta 239.0 (59.0, 332.5) 154.0 (40.0, 285.0) 0.141
    Current CD4 Counta 583.0 (338.0, 840.0) 504.5 (296.3, 697.5) 0.107
    Plasma HIV RNA (% detectable) 31% 28% 0.772
    AIDS (%) 49% 69% 0.030
    cART (%) 71% 90% 0.014
Psychiatric
    POMS Total Mood Disturbancea 41.0 (27.5, 58.5) 55.5 (31.0, 93.3) 0.017
    Major Depressive Disorder
        Current (%) 6% 14% 0.183
        Lifetime (%) 47% 60% 0.191
    General Anxiety Disorder
        Current (%) 0% 4% 0.498
        Lifetime (%) 2% 11% 0.120
    Substance Dependencec (%) 57% 56% 0.917
Open in a new tab

Note.

a

Data are presented as medians (interquartile range)].

b

Hollingshead score = composite score based on education and occupational history with higher scores reflecting lower levels functioning.

c

Lifetime diagnoses.

HIV = Human immunodeficiency virus. CD4 = Cluster of differentiation 4. cART = Combined antiretroviral therapy. POMS = Profile of Mood States.

Procedure

After providing informed consent, all participants underwent comprehensive neuropsychological, psychiatric, and neuromedical research evaluations. ProM was assessed using the Prospective and Retrospective Memory Questionnaire (PRMQ; Smith, Della Sala, Logie, & Maylor, 2000) and the research version of the Memory for Intentions Screening Test (MIST; Raskin, Buckheit, & Sherrod, 2010), which is detailed in Woods, Moran, Dawson, et al. (2008).

MIST

The MIST is a standardized 30-minute task consisting of eight ProM trials that are completed in the context of an ongoing distracter task (i.e., a word search puzzle). There are four cues based on time (e.g., “In 15 min, tell me it is time to take a break.”) and four based on events (e.g., “When I show you a postcard, self address it.”). The primary indices derived from the MIST include a Summary Score (sample range = 15-48), a time-based subscale (sample range = 2-8) and an event-based Scale (sample range = 0-8). Error types were coded as follows: 1) no response (i.e., omission errors), 2) task substitution (e.g., perseverations or intrusions, such as replacing a verbal response with an action or vice versa), 3) loss of content (e.g., acknowledging that a response is required, but failing to recall the particulars), and 4) loss of time (i.e., performing the correct response at the wrong time). After completing the primary task, participants are administered an 8-item, 3-choice recognition trial (sample range = 5-8). Lastly, participants are instructed to call the examiner the next day (i.e., 24 hours later) and leave a telephone message indicating the number of hours they slept the night after the examination (sample range = 0-2). Prior studies support the reliability (e.g., Woods, Moran, Dawson, et al., 2008) and construct validity (e.g., Carey et al., 2006; Gupta et al., 2010; Woods, Iudicello, et al., 2008; Woods et al., 2009; Woods, Dawson, Weber, Grant, & The HNRC Group, 2010) of the MIST in HIV.

PRMQ

The PRMQ is a 16-item, self-report measure that uses a 5-point Likert-type scale to assess the frequency of memory complaints in everyday life. This study focused on the 8-item ProM subscale (sample range = 8-39), which includes four self-cued (e.g., “How often do you forget appointments if you are not prompted by someone else or by a reminder, such as a diary or a calendar?”) (sample range = 4-19) and four environmentally-cued (e.g., “How often do you forget to buy something you planned to buy, like a birthday card, even when you see the shop?”) (sample range = 4-20) ProM complaints. Prior research supports the reliability (Crawford, Smith, Maylor, Della Sala, & Logie, 2003) and construct validity (Smith et al., 2000; Woods et al., 2007) of the PRMQ.

Neuropsychological assessment

Participants were also administered a battery of standardized clinical neurocognitive tests that, consistent with the recommendations of Antinori et al. (2007), assessed the ability areas most commonly affected in HIV-associated neurocognitive disorders. Measures of retrospective learning and memory included Total Trials 1-5 and Long Delay Free Recall from the California Verbal Learning Test (2nd ed., CVLT-2; Delis, Kramer, Kaplan, & Ober, 2000), Logical Memory I and II from Wechsler Memory Scale (3rd ed., WMS-III; Psychological Corporation, 1997), and the Immediate and Delayed Presence and Accuracy scores from the Rey Complex Figure Test, Boston Qualitative Scoring System (BQSS; Stern et al., 1999). Executive functions were assessed with the total time to complete Part B of the Trail Making Test (TMT; Reitan & Wolfson, 1985) and Total Move Score from the Tower of London, Drexel version (ToL; Culbertson & Zillmer, 2001). Tests of information processing speed included TMT Part A (total time) and ToL Total Execution Time. The WMS-III Digit Span subtest (total score) and Self-Ordered Pointing Test total errors (see Morgan et al., 2009) were used to measure attention and working memory. To assess verbal fluency, we administered the letter (C; Benton, Hamsher, & Sivan, 1994), animal (Goodglass & Kaplan, 1972), and action (Woods et al., 2005) fluency tasks. Finally, fine-motor speed and coordination were examined with the dominant and nondominant hand trials of the Grooved Pegboard test (Kløve, 1963). For all of the standard clinical neurocognitive measures, raw scores were converted to population-based z scores derived from the entire study sample (N = 108). These individual test scores were then averaged to create mean domain z scores.

Psychiatric assessment

Psychiatric evaluation included the Composite International Diagnostic Interview (CIDI version 2.1; World Health Organization, 1998), which was administered to provide current and lifetime diagnoses of Major Depressive, Generalized Anxiety, and Substance Use disorders. Participants also completed the Profile of Mood States (POMS; McNair, Lorr, & Droppleman, 1981) as measure of affective distress. The POMS is a 65-item self-report measure of current mood states that include items in six subscales (i.e., Tension, Depression, Anger, Vigor, Confusion, and Fatigue) and a Total Mood Disturbance score, with higher scores signifying greater affective distress. The medical evaluation included standardized physical and neurological examinations, a review of medications and medical history, and blood draw.

Results

Group differences on the MIST, PRMQ, and the various cognitive domains were examined using Wilcoxon rank-sum tests due to the non-normality of the data (Shapiro-Wilk p < .05) and unequal variances (Levene's test p < .05). As shown in Table 2, we observed significant group differences (and medium effect sizes) in the expected direction on the Summary Score (p < .01) and the Event-Based subscale (p < .05) of the MIST. A trend-level difference was evident on the MIST Time-Based subscale (p < .10). Analyses of MIST error types revealed that the Unemployed group committed significantly more Omission errors (p < .05), but did not differ in the rates of Task Substitution, Loss of Content, or Loss of Time errors (ps > 0.10). Similarly, the Employed and Unemployed groups did not differ on the MIST recognition test, ongoing word search, or 24-hour delay task (ps > 0.10). Table 2 also shows a trend-level effect of employment on the PRMQ ProM Environmentally Cued scale (p < .10), but no group differences on the Total or Self-Cued scales of this self-report instrument (ps > .10). The PRMQ ProM Environmentally Cued scale was not significantly correlated with the MIST Event-based scale (Spearman's rho = -0.13, p > .10).

Table 2.

Prospective memory performance in the study samples

ProM Variable Employed (n = 49) Unemployed (n = 59) p d
MIST
    Summary Score 42.0 (36.0, 45.0) 39.0 (30.0, 42.0) 0.008 0.58
        Time-based 6.0 (5.0, 7.0) 6.0 (5.0, 7.0) 0.053 0.41
        Event-based 8.0 (7.0, 8.0) 7.0 (5.0, 8.0) 0.013 0.53
    Error Types
        No Response 0.0 (0.0, 1.0) 1.0 (0.0, 1.0) 0.035 0.48
        Task Substitution 0.0 (0.0, 1.0) 1.0 (0.0, 1.0) 0.498 0.09
        Loss of Content 0.0 (0.0, 1.0) 1.0 (0.0, 1.0) 0.223 0.22
        Loss of Time 0.0 (0.0, 0.0) 0.0 (0.0, 1.0) 0.401 0.12
    Recognition Posttest 8.0 (8.0, 8.0) 8.0 (7.0, 8.0) 0.332 0.34
    Word Search 16.0 (13.0, 18.0) 16.0 (12.0, 19.0) 0.983 0.08
    24-hour Delay 0.0 (0.0, 1.0) 1.0 (0.0, 1.0) 0.188 0.25
PRMQ ProM Scale
    ProM Total 19.0 (16.0, 21.0) 21.0 (15.0, 26.0) 0.190 0.30
        Self 10.0 (8.0, 11.0) 11.0 (8.0, 13.0) 0.292 0.21
        Environment 9.0 (7.0, 10.5) 11.0 (7.0, 13.0) 0.078 0.36
Open in a new tab

Note. Data are presented as median values with the interquartile range in parenthesis unless otherwise indicated. MIST = Memory for Intentions Screening Test. PRMQ = Prospective and Retrospective Memory Questionnaire. ProM = prospective memory.

Table 3 displays the group differences on various cognitive domains. Wilcoxon rank-sums tests revealed a trend-level group effect in the expected direction on Retrospective Learning (p < .10), with significant differences in Executive Functions and Verbal Fluency (ps < .05). The groups did not differ in the domains of Retrospective Memory, Information Processing Speed, Attention/Working Memory, or Motor Coordination (ps > 0.05). A follow-up logistic regression predicting employment group status from Retrospective Learning, Executive Functions, Verbal Fluency, and the MIST Summary Score produced a significant overall model (X2 = 11.9, p = 0.018) in which the ProM variable was the sole cognitive predictor of unemployment (X2 = 4.4, p = .036). The range odds ratio for the MIST Summary Score was 7.8 (95% confidence interval = 1.1, 66.7), as compared to 5.0 (0.5, 55.0) for Verbal Fluency, 1.7 (0.1, 28.0) for Executive Functions, and 1.2 (0.2, 9.9) for Retrospective Learning.

Table 3.

Domain-based neuropsychological scores in study samples

Composite z-scores Employed (n = 49) Unemployed (n = 59) p d
Retrospective Learning 0.1 (-0.4, 0.6) -0.3 (-0.7, 0.5) 0.050 0.37
Retrospective Memory 0.0 (-0.4, 0.7) -0.2 (-0.8, 0.5) 0.066 0.37
Executive Functions 0.4 (0.0, 0.7) 0.1 (-0.7, 0.4) 0.008 0.39
Information Processing Speed 0.1 (-0.2, 0.8) 0.0 (-0.6, 0.7) 0.192 0.31
Attention/Working Memory -0.1 (-0.5, 0.9) -0.1 (-0.8, 0.5) 0.348 0.20
Verbal Fluency 0.1 (-0.5, 0.8) -0.2 (-0.7, 0.4) 0.033 0.45
Motor Coordination 0.1 (-0.2, 0.8) 0.2 (-0.4, 0.5) 0.304 0.21
Open in a new tab

Note. Values reflect composite population-based z-scores whereby higher scores reflect better performance.

As shown in Table 1 above, although the samples were largely comparable with regard to their current immune status (i.e., current CD4 count) and virologic control (i.e., HIV RNA), the Unemployed group had higher prevalence of AIDS diagnoses and was more likely to be prescribed cART (ps < .05). Unemployed participants also reported higher levels of current affective distress than the Employed group as measured by the POMS Total Mood Disturbance (p < .001), but did not differ in rates of Major Depressive Disorder, Generalized Anxiety Disorder, or lifetime Substance Dependence (ps > .10). Consistent with our approach to the cognitive factors above, we conducted a post-hoc logistic regression to evaluate the uniqueness of ProM as a predictor of unemployment in the context of these three significant confounding factors. Again, the overall model (X2 = 25.0, p < 0.0001) and the MIST Summary Score (X2 = 5.0, p = .025) were significant predictors of employment status, this time along with POMS Total Mood Disturbance (X2 = 8.8, p = .003) and cART status (X2 = 6.7, p = .010). In this model, the range odds ratio for the MIST Summary Score was 9.4 (95% confidence interval = 1.3, 85.1), versus 0.04 (0.0, 35.0) for POMS and standard odds ratios of 3.5 (1.2, 10.0) for cART status, and 2.4 (1.1, 5.2) for AIDS status.

Discussion

Unemployment is highly prevalent among persons with HIV-associated neurocognitive disorders and has been reliably linked to executive dysfunction, slowed information processing, and deficits in retrospective memory (Gorman, Foley, Ettenhofer, Hinkin, & van Gorp, 2009). The current study extends that literature to the domain of ProM, which is a complex cognitive construct that describes one's ability to accurately encode, retain, and execute an intention (Kliegel, Jäger, et al., 2008) and is theorized to play an important role in many different aspects of daily functioning (e.g., Woods, Iudicello, et al., 2008). Results from this cross-sectional study showed that HIV-infected persons who were unemployed demonstrated significantly worse overall ProM than employed individuals as measured by a standardized performance-based task. This finding was associated with a medium effect size, with range odds ratios suggesting that individuals with the lowest ProM scores were approximately eight to nine times more likely to be unemployed than individuals with the best ProM performance, even while controlling for other important group differences. These data are commensurate with prior studies showing that HIV-associated ProM deficits may increase the risk of poorer daily functioning outcomes, including medication non-adherence (e.g., Woods et al., 2009) and general declines in IADLs (e.g., Woods, Iudicello, et al., 2008). To the best of our knowledge, this is the first study to demonstrate an association between ProM and unemployment.

Findings suggest that ProM may be an important risk factor for unemployment among persons living with HIV. One interpretation of these data is that individuals with ProM deficits experienced difficulties in the work environment that may have contributed to problems maintaining or achieving employment (e.g., forgetting appointments, missing deadlines). However, due to the cross-sectional (and quasi-experimental) study design, we cannot assert a causal relationship between ProM and unemployment. Indeed, this limitation is inherent to cross-sectional studies that aim to identify clinical correlates of everyday functioning using naturalistic outcomes, such as employment. As such, it is also plausible that individuals who are not presently working do not experience the same ProM demands in their daily lives as employed individuals and are therefore not as practiced in executing future intentions. One might argue that this rationale may also partly explain the weak employment effects on the self-report of ProM failures in daily life and performance on the semi-naturalistic 24-hour task.

Moreover, two-thirds of the unemployed participants were on disability (e.g., persons with longer durations of infection and AIDS diagnoses) and were presumably not seeking gainful employment, which limits direct inferences regarding the predictive value of ProM. The relatively rudimentary employment classification approach used in this study does not allow us to rule out the possibility that some of our unemployed participants were working on a volunteer basis or had some other means of “under the table” income. Such limitations point to the need for longitudinal studies to examine whether ProM is associated with an increased risk of incident unemployment, especially among HIV-infected persons with cognitively demanding jobs. For example, it is not known whether It is also not known whether individuals with HIV-associated ProM deficits may also experience greater challenges in returning to work after unemployment or disability, which has become increasingly common in the cART era (e.g., van Gorp et al., 2007). For example, van Gorp and colleagues (2007) reported that HIV-infected individuals with learning (and executive) deficits were less likely to find employment during a two-year follow-up period. Similarly, Chernoff et al. (2010) reported that HIV-infected persons with executive dysfunction were at mild risk for remaining unemployed over two years. Whether HIV-associated deficits in ProM predict return to work remains to be determined.

Importantly, ProM was a unique predictor of unemployment in this sample, demonstrating incremental validity relative to established risk factors. Consistent with earlier studies (Gorman et al., 2009), unemployment was also associated with lower scores on traditional clinical measures of executive functions, verbal fluency, and retrospective memory (with the latter finding at trend level). Although unemployment was not associated with working memory or information processing speed, as has been demonstrated in some prior studies, it is not uncommon for some degree of predictive variability to be evident across cognitive domains in a given investigation (see Antinori et al., 2007). Also echoing the extant literature, unemployed persons reported greater current affective distress and were more likely to have been diagnosed with AIDS (e.g., Rabkin et al., 2004; Blalock et al., 2002; van Gorp et al., 2007). Nevertheless, ProM remained a significant independent predictor of employment when these other factors were included in the statistical models. Moreover, the association between ProM and Unemployment could not be better explained by other psychiatric comorbidities, HIV disease severity, or demographic characteristics, as the study samples were otherwise well matched in this regard. All told, these data suggest that ProM may be capturing a unique signal in the prediction of unemployment among individuals with HIV.

Although event-based ProM as measured in the laboratory was a significant predictor of employment, its self-reported analogue, environmentally-cued ProM on the PRMQ, only differentiated the groups at a trend-level and was associated with a relatively modest effect size. Moreover, there was no notable association between employment status and the other PRMQ scale. Such null effects of self-reported ProM on a functional outcome echo findings from prior studies in HIV that utilize behavioral (e.g., Woods et al., 2009; Zogg et al., 2010), rather than self-reported (e.g., Woods, Iudicello et al., 2008) indicators of everyday functioning, perhaps suggesting that performance-based measures of ProM may be more sensitive in this regard. In fact, the correlation between self-report and performance-based ProM was weak in this study, as has been reported previously in HIV (Woods et al., 2007). This poor correspondence between cognitive complaints and demonstrated ability is theorized to be driven by affective distress, including fatigue, anxiety, and depression (e.g., Rourke et al., 1999; Woods et al., 2007).

Despite these encouraging findings, this study has several limitations that warrant consideration. First, we used a gross classification of employment, which was derived from self-report during a brief neurobehavioral interview. It will be important to extend these findings to other approaches for assessing vocation, including performance-based or computerized laboratory assessments (e.g., Twamley et al., 2006), as well as perhaps more naturalistic observation-based approaches or virtual reality that might provide valuable information regarding work quality, productivity, and stability (see Sadek & van Gorp, 2009). Other limitations of this study include the characteristics of the study sample, which was highly educated, largely male sample with relatively well-controlled HIV disease. We also excluded participants with many conditions that may themselves be major risk factors for unemployment, including traumatic brain injury, active substance use disorders, and severe mental illness. It remains to be seen whether the observed associations extend to demographically diverse cohorts with more severe HIV disease and higher rates of comorbidities.

Although this is only the first clinical study to examine the role of ProM in vocational functioning, these findings may have clinical implications for rehabilitation psychologists working with HIV-positive individuals. Clinicians might consider including measures of ProM in their assessments of work readiness and return to work. The MIST, for example, has considerable evidence of construct validity (see Raskin, 2009) and is now widely available with age- and education-adjusted normative standards (Raskin et al., 2010). Clinical researchers may wish to examine the extent to which ProM informs the development, deployment, and effectiveness of vocational rehabilitation programs and work-related accommodations. Findings from this study suggest that, at the group level, interventions that enhance cue focality (Kliegel, Phillips & Jäger, 2008), monitoring (e.g., STOP! paradigm; Fish et al., 2007) and detection (e.g., salient alerts and reminders) may be particularly effective. For example, Fish and colleagues (2007) recently reported that content-free cues to increase strategic monitoring improved performance on a semi-naturalistic telephone ProM task in persons with traumatic brain injury. Cognitive remediation training added to vocational rehabilitation and supported employment programs has demonstrated significant success in improving employment outcomes in individuals with severe mental illness (i.e., schizophrenia; e.g., Bell, Zito, Greig, & Wexler, 2008; McGurk, Mueser, Feldman, Wolfe, & Pascaris, 2007), but such programs typically focus on broad cognitive domains (e.g., attention, memory, problem solving). Supplementing vocational rehabilitation programs with such mnemonic strategies to specifically boost ProM performance (e.g., electronic reminders) may improve the relevance of cognitive aspects of training to everyday experiences in the workplace for individuals in this population.

Acknowledgments

The HIV Neurobehavioral Research Programs (HNRP) Group is affiliated with the University of California, San Diego, the Naval Hospital, San Diego, and the Veterans Affairs San Diego Healthcare System, and includes: Director: Igor Grant, M.D.; Co-Directors: J. Hampton Atkinson, M.D., Ronald J. Ellis, M.D., Ph.D., and J. Allen McCutchan, M.D.; Center Manager: Thomas D. Marcotte, Ph.D.; Naval Hospital San Diego: Braden R. Hale, M.D., M.P.H. (P.I.); Neuromedical Component: Ronald J. Ellis, M.D., Ph.D. (P.I.), J. Allen McCutchan, M.D., Scott Letendre, M.D., Edmund Capparelli, Pharm.D., Rachel Schrier, Ph.D.; Neurobehavioral Component: Robert K. Heaton, Ph.D. (P.I.), Mariana Cherner, Ph.D., David J. Moore, Ph.D., Steven Paul Woods, Psy.D.; Neuroimaging Component: Terry Jernigan, Ph.D. (P.I.), Christine Fennema-Notestine, Ph.D., Sarah L., Archibald, M.A., John Hesselink, M.D., Jacopo Annese, Ph.D., Michael J. Taylor, Ph.D.; Neurobiology Component: Eliezer Masliah, M.D. (P.I.), Ian Everall, FRCPsych., FRCPath., Ph.D., T. Dianne Langford, Ph.D.; Neurovirology Component: Douglas Richman, M.D., (P.I.), David M. Smith, M.D.; International Component: J. Allen McCutchan, M.D., (P.I.); Developmental Component: Ian Everall, FRCPsych., FRCPath., Ph.D. (P.I.), Stuart Lipton, M.D., Ph.D.; Clinical Trials Component: J. Allen McCutchan, M.D., J. Hampton Atkinson, M.D., Ronald J. Ellis, M.D., Ph.D., Scott Letendre, M.D.; Participant Accrual and Retention Unit: J. Hampton Atkinson, M.D. (P.I.), Rodney von Jaeger, M.P.H.; Data Management Unit: Anthony C. Gamst, Ph.D. (P.I.), Clint Cushman, B.A., (Data Systems Manager), Daniel R. Masys, M.D. (Senior Consultant); Statistics Unit: Ian Abramson, Ph.D. (P.I.), Reena Deutsch, Ph.D., Florin Vaida Ph.D.

This research was supported by grants R01-MH73419 (Woods) and P30-MH62512 (Grant) from the National Institutes of Mental Health. The views expressed in this article are those of the authors and do not reflect the official policy or position of the Department of the Navy, Department of Defense, nor the United States Government. Aspects of these data were presented at the 3rd International Conference on Prospective Memory in Vancouver, British Columbia, Canada. The authors thank Dr. Sarah Raskin for providing us with the MIST. We are also grateful for the assistance of Matthew Dawson, Nichole A. Duarte, Sarah Gibson, Lisa Moran, and Dr. Catherine L. Carey in collecting, coding, and processing the data for this study.

Contributor Information

Steven Paul Woods, Department of Psychiatry, University of California, San Diego.

Erica Weber, Joint Doctoral Program in Clinical Psychology, San Diego State University and University of California, San Diego.

Bradley M. Weisz, Department of Psychology, San Diego State University.

Elizabeth W. Twamley, Department of Psychiatry, University of California, San Diego

Igor Grant, Department of Psychiatry, University of California, San Diego.

References

  1. Albert SM, Marder K, Dooneief G, Bell K, Sano M, Todak G, Stern Y. Neuropsychologic impairment in early HIV infection: A risk factor for work disability. Archives of Neurology. 1995;52:525–530. doi: 10.1001/archneur.1995.00540290115027. [DOI] [PubMed] [Google Scholar]
  2. Antinori A, Arendt G, Becker JT, Brew BJ, Byrd DA, Cherner M, Wojna VE. Updated research nosology for HIV-associated neurocognitive disorders. Neurology. 2007;69:1789–1799. doi: 10.1212/01.WNL.0000287431.88658.8b. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. The Antiretroviral Therapy Cohort Collaboration Life expectancy of individuals on combination antiretroviral therapy in high-income countries: A collaborative analysis of 14 cohort studies. Lancet. 2008;372:293–299. doi: 10.1016/S0140-6736(08)61113-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Bell MD, Zito W, Greig T, Wexler BE. Neurocognitive enhancement therapy with vocational services: Work outcomes at two-year follow-up. Schizophrenia Research. 2008;105:18–29. doi: 10.1016/j.schres.2008.06.026. [DOI] [PubMed] [Google Scholar]
  5. Benton AL, Hamster K. de S., Sivan AB. Multilingual Aphasia Examination. AJA Associates; Iowa City: 1994. [Google Scholar]
  6. Blalock A, McDaniel S, Farber E. Effect of employment on quality of life and psychological functioning in patients with HIV/AIDS. Psychosomatics. 2002;43:400–404. doi: 10.1176/appi.psy.43.5.400. [DOI] [PubMed] [Google Scholar]
  7. Burns SM, Young LRL, Maniss S. Predictors of employment and disability among people living with HIV/AIDS. Rehabilitation Psychology. 2006;51:127–134. [Google Scholar]
  8. Carey CL, Woods SP, Rippeth JD, Heaton RK, Grant I, The HNRC Group Prospective memory in HIV-1 infection. Journal of Clinical and Experimental Neuropsychology. 2006;28:536–548. doi: 10.1080/13803390590949494. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Centers for Disease Control and Prevention . Clinical and Behavioral Characteristics of Adults Receiving Medical Care for HIV Infection: Medical Monitoring Project, 2005 Pilot Data Collection Cycle. U.S. Department of Health and Human Services, Center for Disease Control and Prevention; Atlanta: 2010. [Google Scholar]
  10. Centers for Disease Control and Prevention . HIV/AIDS Surveillance Report. U.S. Department of Health and Human Services, Center for Disease Control and Prevention; Atlanta: 2007. [Google Scholar]
  11. Chernoff RA, Martin DJ, Schrock DA, Huy MP. Neuropsychological functioning as apredictor of employment activity in a longitudinal study of HIV-infected adults contemplating workforce reentry. Journal of the International Neuropsychological Society. 2010;16:38–48. doi: 10.1017/S1355617709990828. [DOI] [PubMed] [Google Scholar]
  12. Contardo C, Black AC, Beauvais J, Dieckhaus K, Rosen MI. Relationship of prospective memory to neuropsychological function and antiretroviral adherence. Archives of Clinical Neuropsychology. 2009;24:547–554. doi: 10.1093/arclin/acp046. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Crawford JR, Smith G, Maylor EA, Della Sala S, Logie RH. The Prospective and Retrospective Memory Questionnaire (PRMQ): Normative data and latent structure in a large non-clinical sample. Memory. 2003;11:261–275. doi: 10.1080/09658210244000027. [DOI] [PubMed] [Google Scholar]
  14. Culbertson WC, Zillmer EA. The Tower of LondonDX (TOLDX) manual. Multi-Health Systems; North Tonawanda, NY: 2001. [Google Scholar]
  15. Delis DC, Kramer JH, Kaplan E, Ober BA. California Verbal Learning Test – second edition. Adult version. Manual. Psychological Corporation; San Antonio, TX: 2000. [Google Scholar]
  16. Fish J, Evans JJ, Nimmo M, Martin E, Kersel D, Bateman A, Manly T. Rehabilitation of executive dysfunction following brain injury: “Content-free” cueing improves everyday prospective memory performance. Neuropsychologia. 2007;45:1318–1330. doi: 10.1016/j.neuropsychologia.2006.09.015. [DOI] [PubMed] [Google Scholar]
  17. Fogarty AS, Zablotska I, Rawstorne P, Prestage G, Kippax SC. Factors distinguishing employed from unemployed people in the Positive Health Study. AIDS. 2007;21:S37–S42. doi: 10.1097/01.aids.0000255083.69846.59. [DOI] [PubMed] [Google Scholar]
  18. Goldman DP, Bao Y. Effective HIV treatment and employment of HIV+ adults. Health Services Research. 2004;39:1691–1712. doi: 10.1111/j.1475-6773.2004.00313.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Goodglass H, Kaplan E. Assessment of aphasia and related disorders. Lea & Febiger; Philadelphia: 1972. [Google Scholar]
  20. Gorman AA, Foley JM, Ettenhofer ML, Hinkin CH, Van Gorp WG. Functional consequences of HIV-associated neuropsychological impairment. Neuropsychology Review. 2009;19:186–203. doi: 10.1007/s11065-009-9095-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Gupta S, Woods SP, Weber E, Dawson MS, Grant I, The HNRC Group Is prospective memory a dissociable cognitive function in HIV infection? Journal of Clinical and Experimental Neuropsychology. 2010;32:898–908. doi: 10.1080/13803391003596470. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Heaton RK, Clifford DB, Franklin DR, Jr., Woods SP, Ake C, Vaida F, The CHARTER Group HIV-associated neurocognitive disorders persist in the era of potent antiretroviral therapy: CHARTER Study. Neurology. 2010;75:2087–96. doi: 10.1212/WNL.0b013e318200d727. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Heaton RK, Marcotte TD, Mindt MR, Sadek J, Moore DJ, Bentley H, The HNRC Group The impact of HIV-associated neuropsychological impairment on everyday functioning. Journal of the International Neuropsychological Society. 2004;10:317–331. doi: 10.1017/S1355617704102130. [DOI] [PubMed] [Google Scholar]
  24. Heaton RK, Velin RA, McCutchan JA, Gulevich SJ, Atkinson JH, Wallace MR, Grant I. Neuropsychological impairment in human immunodeficiency virus-infection: Implications for employment. HNRC Group. HIV Neurobehavorial Research Center. Psychosomatic Medicine. 1994;56:8–17. doi: 10.1097/00006842-199401000-00001. [DOI] [PubMed] [Google Scholar]
  25. Kitahata MM, Gange SJ, Abraham AG, Merriman B, Saag MS, Justice AC, NAACCORD Investigators Effect of early versus deferred antiretroviral therapy for HIV on survival. New England Journal of Medicine. 2009;360:1815–1826. doi: 10.1056/NEJMoa0807252. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Kliegel M, Jäger T, Altgasen M, Shum D. Clinical neuropsychology of prospective memory. In: Kliegel M, McDaniel MA, Einstein GO, editors. Prospective Memory: Cognitive, Neuroscience, Developmental, and Applied Perspectives. Taylor & Francis Group/Lawrence Erlbaum Associates; New York, NY: 2008. pp. 283–308. [Google Scholar]
  27. Kliegel M, Phillips LH, Jäger T. Adult age differences in event-based prospective memory: A meta-analysis on the role of focal versus non-focal cues. Psychology and Aging. 2008;23:203–208. doi: 10.1037/0882-7974.23.1.203. [DOI] [PubMed] [Google Scholar]
  28. Kløve H. Clinical neuropsychology. In: Forster FM, editor. The Medical Clinics of North America. Saunders; New York: 1963. [PubMed] [Google Scholar]
  29. Lem M, Moore D, Marion S, Bonner S, Chan K, O'Connell J, Hogg R. Back to work: Correlates of employment among persons receiving highly active antiretroviral therapy. AIDS Care. 2005;176:740–746. doi: 10.1080/09540120412331336724. [DOI] [PubMed] [Google Scholar]
  30. Lemp GF, Payne SF, Rutherford GW, Hessol NA, Winkelstein W, Wiley JA, Werdegar D. Projections of AIDS morbidity and mortality in San Francisco. JAMA. 1990;263:1497–1501. [PubMed] [Google Scholar]
  31. Martin EM, Nixon H, Pitrak DL, Weddington W, Rains NA, Nunnally G, Bechara A. Characteristics of prospective memory deficits in HIV-seropositive substance-dependent individuals: Preliminary observations. Journal of Clinical and Experimental Neuropsychology. 2007;29:496–504. doi: 10.1080/13803390600800970. [DOI] [PubMed] [Google Scholar]
  32. McGurk SR, Mueser KT, Feldman K, Wolfe R, Pascaris A. Cognitive training for supported employment: 2-3 year outcomes of a randomized controlled trial. American Journal of Psychiatry. 2007;164:437–441. doi: 10.1176/ajp.2007.164.3.437. [DOI] [PubMed] [Google Scholar]
  33. McNair DM, Lorr M, Droppleman LF. Manual for the Profile of Mood States. Educational and Industrial Testing Service; San Diego: 1981. [Google Scholar]
  34. Morgan EE, Woods SP, Weber E, Dawson MS, Carey CL, Moran LM, The HNRC Group HIV-associated episodic memory impairment: Evidence of a possible differential deficit in source memory for complex visual stimuli. Journal of Neuropsychiatry and Clinical Neurosciences. 2009;21:189–198. doi: 10.1176/appi.neuropsych.21.2.189. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Parsons TD, Braaten AJ, Hall CD, Robertson KR. Better quality of life with neuropsychological improvement on HAART. Health and Quality of Life Outcomes. 2006;24:4–11. doi: 10.1186/1477-7525-4-11. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Psychological Corporation . WAIS-III and WMS-III technical manual. Psychological Corporation; San Antonio, TX: 1997. [Google Scholar]
  37. Psychological Corporation . Wechsler Test of Adult Reading. Psychological Corporation; San Antonio, TX: 2001. [Google Scholar]
  38. Rabkin JG, McElhiney M, Ferrando S, Van Gorp W, Lin SH. Predictors of employment of men with HIV/AIDS: A longitudinal study. Psychosomatic Medicine. 2004;66:72–78. doi: 10.1097/01.psy.0000108083.43147.6d. [DOI] [PubMed] [Google Scholar]
  39. Raskin SA. Memory for Intentions Screening Test: Psychmetric properties and clinical evidence. Brain Impairment. 2009;10:23–33. [Google Scholar]
  40. Raskin S, Buckheit C, Sherrod C. Memory for Intentions Test (MIsT) Psychological Assessment Resources, Inc.; Lutz, FL: 2010. [Google Scholar]
  41. Reitan RM, Wolfson D. The Halstead-Reitan neuropsychological test battery: Theory and clinical interpretation. Neuropsychology Press; Tucson, AZ: 1985. [Google Scholar]
  42. Sadek JR, van Gorp WG. The prediction of vocational functioning from neuropsychological performance. In: Marcotte TD, Grant I, editors. The Neuropsychology of Everyday Functioning. Guilford; New York: 2009. pp. 113–135. [Google Scholar]
  43. Smith G, Della Sala S, Logie RH, Maylor EA. Prospective and retrospective memory in normal aging and dementia: a questionnaire study. Memory. 2000;8:311–321. doi: 10.1080/09658210050117735. [DOI] [PubMed] [Google Scholar]
  44. Stern RA, Javorsky DJ, Singer EA, Singer NA, Somerville JA, Duke LM, Kaplan E. The Boston Qualitative Scoring System for the Rey-Osterrieth Complex Figure: Professional Manual. Psychological Assessment Resources, Inc.; Odessa, FL: 1999. [Google Scholar]
  45. Twamley EW, Narvaez JM, Sadek JR, Jeste DV, Grant I, Heaton RK. Work-related abilities in schizophrenia and HIV infection. The Journal of Nervous and Mental Disease. 2006;194:268–274. doi: 10.1097/01.nmd.0000207362.84045.1b. [DOI] [PubMed] [Google Scholar]
  46. van Gorp WG, Baerwald JP, Ferrando SJ, McElhiney MC, Rabkin JG. The relationship between employment and neuropsychological impairment in HIV infection. Journal of the International Neuropsychological Society. 1999;5:534–539. doi: 10.1017/s1355617799566071. [DOI] [PubMed] [Google Scholar]
  47. van Gorp WG, Rabkin JG, Ferrando SJ, Mintz J, Ryan E, Borkowski T, McElhiney M. Neuropsychiatric predictors of return to work in HIV/AIDS. Journal of the International Neuropsychological Society. 2007;13:80–89. doi: 10.1017/S1355617707070117. [DOI] [PubMed] [Google Scholar]
  48. Woods SP, Carey CL, Moran LM, Dawson MS, Lentendre SL, Grant I, The HNRC Group Frequency and predictors of self-reported prospective memory complaints in individuals infected with HIV. Archives of Clinical Neuropsychology. 2007;22:187–195. doi: 10.1016/j.acn.2006.12.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Woods SP, Dawson MS, Weber E, Gibson S, Grant I, Atkinson JH, The HIV Neurobehavioral Research Center Timing is everything: Antiretroviral nonadherence is associated with impairment in time-based prospective memory. Journal of the International Neuropsychological Society. 2009;15:42–52. doi: 10.1017/S1355617708090012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Woods SP, Dawson MS, Weber E, Grant I, The HNRC Group The semantic relatedness of cue-intention pairings influences event-based prospective memory failures in older adults with HIV infection. Journal of Clinical and Experimental Psychology. 2010;32:398–407. doi: 10.1080/13803390903130737. [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Woods SP, Iudicello JE, Moran LM, Carey CL, Dawson MS, Grant I, The HNRC Group HIV-associated prospective memory impairment increases risk of dependence in everyday functioning. Neuropsychology. 2008;22:110–117. doi: 10.1037/0894-4105.22.1.110. [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Woods SP, Moran LM, Carey CL, Dawson MS, Iudicello JE, Gibson S, The HNRC Group Prospective memory in HIV infection: Is “remembering to remember” a unique predictor of self-reported medication management? Archives of Clinical Neuropsychology. 2008;23:257–270. doi: 10.1016/j.acn.2007.12.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Woods SP, Moran LM, Dawson MS, Carey CL, Grant I, The HNRC Group Psychometric characteristics of the Memory for Intentions Screening Test. The Clinical Neuropsychologist. 2008;22:864–878. doi: 10.1080/13854040701595999. [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. Woods SP, Morgan EE, Marquie-Beck J, Carey CL, Grant I, Letendre SL, The HNRC Group Markers of macrophage activation and axonal injury are associated with prospective memory in HIV-1 disease. Cognitive & Behavioral Neurology. 2006;19:217–221. doi: 10.1097/01.wnn.0000213916.10514.57. [DOI] [PMC free article] [PubMed] [Google Scholar]
  55. Woods SP, Scott JC, Sires DA, Grant I, Heaton RK, Tröster AI, The HNRC Group Action (verb) fluency: Test-retest reliability, normative standards, and construct validity. Journal of International Neuropsychological Society. 2005;11:408–415. [PubMed] [Google Scholar]
  56. World Health Organization . Composite international diagnostic interview. World Health Organization; Geneva, Switzerland: 1998. (CIDI, version 2.1) [Google Scholar]
  57. Zogg JB, Woods SP, Weber E, Iudicello J, Dawson MS, Grant I, The HNRC Group HIV-associated prospective memory impairment in the laboratory predicts failures on a semi-naturalistic measure of health care compliance. The Clinical Neuropsychologist. 2010;24:945–962. doi: 10.1080/13854046.2010.501343. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES