Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2018 Feb 1.
Published in final edited form as: Brain Behav Immun. 2017 May 6;65:161–172. doi: 10.1016/j.bbi.2017.04.024

MMPs/TIMPs imbalances in the peripheral blood and cerebrospinal fluid are associated with the pathogenesis of HIV-1-associated neurocognitive disorders

Yanyan Xing a,b, Nicole Shepherd b, Jie Lan b, Wei Li b, Sushmita Rane b, Samir K Gupta c, Shanxiang Zhang d, Jun Dong a,e,*, Qigui Yu b,c,*
PMCID: PMC5793222  NIHMSID: NIHMS935763  PMID: 28487203

Abstract

HIV-1-associated neurocognitive disorders (HAND) continue to be a major concern in the infected population, despite the widespread use of combined antiretroviral therapy (cART). Growing evidence suggests that an imbalance between matrix metalloproteinases (MMPs) and endogenous tissue inhibitors of MMPs (TIMPs) contributes to the pathogenesis of HAND. In our present study, we examined protein levels and enzymatic activities of MMPs and TIMPs in both plasma and cerebrospinal fluid (CSF) samples from HIV-1 patients with or without HAND and HIV-1-negative controls. Imbalances between MMPs and TIMPs with distinct patterns were revealed in both the peripheral blood and CSF of HIV-1 patients, especially those with HAND. In the peripheral blood, the protein levels of MMP-2, MMP-9, TIMP-1, TIMP-2, and the enzymatic activities of MMP-2 and MMP-9 were increased in HIV-1 patients with or without HAND when compared with HIV-1-negative controls. The enzymatic activity of MMP-2, but not MMP-9, was further increased in plasma samples of HAND patients than that of HIV-1 patients without HAND. Notably, the ratio of MMP-2/TIMP-2 in plasma was significantly increased in HAND patients, not in patients without HAND. In the CSF, MMP-2 activity was increased, but the ratio of MMP-2/TIMP-2 was not altered. De novo induction and activation of MMP-9 in the CSF of HAND patients was particularly prominent. The imbalances between MMPs and TIMPs in the blood and CSF were related to the altered profiles of inflammatory cytokines/chemokines and monocyte activation in these individuals. In addition, plasma from HIV-1 patients directly induced integrity disruption of an in vitro blood-brain barrier (BBB) model, leading to increased BBB permeability and robust transmigration of monocytes/-macrophages. These results indicate that imbalances between MMPs and TIMPs are involved in BBB disruption and are implicated in the pathogenesis of neurological disorders such as HAND in HIV-1 patients.

Keywords: HIV-1, MMPs, TIMPs, Neuropathy, BBB, HIV-1-associated neurocognitive disorders

1. Introduction

The HIV-1 pandemic has claimed over 35 million lives, with approximately 37 million people worldwide living with HIV-1/AIDS (http://aidsinfo.unaids.org/), and will continue to contribute to human morbidity and mortality as there is no vaccine or curative treatment available. HIV-1 is neurovirulent, causing neurological diseases such as HIV-1-associated neurocognitive disorders (HAND) in roughly 20% of infected individuals and 60% of patients in the advanced stages of HIV-1 infection (Clifford, 2008; Lindl et al., 2010; Valcour et al., 2011). The incidence of a subclinical neurological phenotype is much higher. Autopsy studies of patients with AIDS, the most severe phase of HIV-1 infection, have revealed that pathologic abnormalities of the central nervous system (CNS) occur in 75–90% of cases examined (Kay et al., 1991; Trujillo et al., 2005). The widespread use of combined antiretroviral therapy (cART) has dramatically decreased the incidence of severe neurological diseases. However, milder forms of HIV-1-associated neurological disorders have become more prevalent as patients live longer due to the use of cART (Joska et al., 2011; Woods et al., 2009). For example, cART has profoundly changed the clinical phenotypes of HAND that is classified into three categories including (1) asymptomatic neurocognitive impairment (ANI), (2) mild neurocognitive disorder (MND), and (3) HIV-1-associated dementia (HAD). While the incidence of HAD, the most severe form of HAND, has significantly declined in the cART era, the incidence of the milder forms of HAND, such as ANI and MND, persists or even increases (Tan and McArthur, 2012). Pathological studies have indicated that neuroinflammation and neural cell death occur in HAND patients, possibly due to increased transmigration of infected and/or activated monocytes/macrophages across the blood-brain barrier (BBB) into the brain (Lakhan et al., 2013; Louboutin et al., 2010a,b; Louboutin et al., 2011; Louboutin and Strayer, 2012; Power et al., 1993a,b; Rosenberg et al., 1998; Spindler and Hsu, 2012; Strazza et al., 2011; Webster and Crowe, 2006). However, the pathogenesis of this progression remains largely unclear (Shawahna, 2015).

The BBB is a dynamic interface between the peripheral circulation and the CNS, which controls the influx and efflux of biological substances needed for brain metabolic processes as well as neuronal functions. Thus, the structural and functional integrity of the BBB is vital in maintaining brain homeostasis (Bernacki et al., 2008). The BBB is composed of specialized brain microvascular endothelial cells (BMECs) that are surrounded by astrocytes and pericytes to cooperatively form and maintain the cerebral microvasculature (Itoh et al., 2011). It has been reported that the BBB capillaries are 50–100 times tighter than peripheral microvessels as a result of complex tight junctions (TJs) between adjacent cells (Abbott, 2002; Aijaz et al., 2006; Redzic, 2011). TJs are multiprotein complexes that mediate cell-cell adhesion and regulate transportation through the extracellular matrix (ECM). The BBB cells and TJ proteins are linked to the ECM that forms a milieu, thereby playing a prominent role in maintaining the integrity of the BBB and in providing signaling to neural cells. Therefore, BMECs together with the closely associated astrocytes, pericytes, neurons, TJs, and the ECM constitute a ‘‘neurovascular unit” that is essential for the health and function of the CNS (del Zoppo, 2009). It has been shown that the over expression and activation of matrix metalloproteinases (MMPs) can injure the BBB, in part, by digesting the TJs and ECM of the BBB (Louboutin et al., 2010a,b).

In humans, there are 23 related, but distinct, MMPs including 17 soluble MMPs and 6 membrane-type MMPs (MT1-MMP to MT6-MMP) (Klein and Bischoff, 2011; Nagase et al., 2006; Nagase and Woessner, 1999; Nelson and Bissell, 2006; Ra and Parks, 2007; Seiki, 2003; Vincenti, 2001). All MMPs are initially synthesized as enzymatically inactive forms (proMMPs or zymogens), which are subsequently processed to generate the active forms (Klein and Bischoff, 2011; Nagase et al., 2006; Nagase and Woessner, 1999; Nelson and Bissell, 2006; Ra and Parks, 2007; Seiki, 2003; Vincenti, 2001). Collectively, these active MMPs are capable of degrading all types of ECM proteins that form the connective materials between cells and around tissues, thereby playing a key role in tissue remodeling and many other important processes. MMP proteolysis is tightly regulated at multiple levels including inhibition by endogenous tissue inhibitors of MMPs (TIMPs) (Cao et al., 1996; Gomez et al., 1997; Klein and Bischoff, 2011), activation of proMMPs (Galis and Khatri, 2002; Ra and Parks, 2007), and induction of gene expression (Klein and Bischoff, 2011; Nagase and Woessner, 1999; Vincenti, 2001). Excessive expression and activation of MMPs have been linked to many diseases such as rheumatoid arthritis, cancer metastasis, and neurological disorders (Cheng et al., 2000; Klein and Bischoff, 2011; Kontogiorgis et al., 2005; Lakhan et al., 2013; Mun-Bryce and Rosenberg, 1998; Rosenberg et al., 1998; Strazza et al., 2011). Growing evidence suggests that an imbalance between MMPs and TIMPs contributes to HAND by disrupting the BBB to allow the penetration of infected cells and neurotoxic substances (Lakhan et al., 2013; Louboutin et al., 2010a,b; Louboutin et al., 2011; Louboutin and Strayer, 2012; Power et al., 1993a,b; Rosenberg et al., 1998; Spindler and Hsu, 2012; Strazza et al., 2011; Webster and Crowe, 2006) and degrading nerve tissue proteins to induce death of neurons (Webster and Crowe, 2006). In the present study, we examined protein levels and enzymatic activities of MMPs and TIMPs in both plasma and cerebrospinal fluid (CSF) samples from HIV-1 patients with or without HAND and HIV-1-negative controls (HD). We also profiled inflammatory cytokines/chemokines and monocyte/macrophage activation in these clinical samples. We found that imbalances between MMPs and TIMPs with distinct patterns occurred in both the blood and CNS in HIV-1 patients, especially those with neurocognitive disorders. The increased ratio of MMP-2/TIMP-2 in the peripheral blood and de novo induction and activation of MMP-9 in the CSF were especially significant in HAND patients, suggesting that imbalances between MMPs and TIMPs may contribute to BBB disruption and dysfunction in HAND patients.

2. Materials and methods

2.1. Study subjects and ethics statement

Frozen plasma samples from 9 HIV-1 patients who had no neurocognitive disorders (NN) and 19 HAND patients were kindly provided by the National NeuroAIDS Tissue Consortium (NNTC, Rockville, MD) through four collection units including the Texas Repository for AIDS Research (Dr. Gelman BB.), Manhattan HIV Brain Bank at the Mount Sinai Medical Center (Dr. Morgello S.), UCLA National Neurological AIDS Bank (Drs. Lucey G., Im K., and Wei B.), and California NeuroAIDS Tissue Network (CNTN) at UCSD (Dr. Ellis RJ.). The NNTC, funded by NIMH and NINDS, has collected, banked, and distributed samples of central and peripheral nervous system tissues, cerebrospinal fluid (CSF), blood, and other organs from HIV-1-positive and -negative individuals to support researchers worldwide to study pathogenesis of NeuroAIDS disorders. All samples were banked by NNTC according to strictly established protocols to ensure uniformity across the four clinical sites. Upon study entry, participants underwent a comprehensive psychometric evaluation to determine past and current substance use disorders, psychiatric illness, and neuropsychological (NP) functioning, as previously described (Clifford and Ances, 2013). Neurological examinations were also performed along with immunological (CD4 T cell count) and virological (plasma and CSF HIV-1 viral load) testing (Morgello et al., 2001). The NNTC uses an in vivo neurocognitive clinical diagnosis that is based on subjects’ pre-mortem neurological and/or NP evaluation or by post-mortem review of records (if the subject could not be examined). The definitions are as follows: 1) Neurocognitively Normal (NN): Subjects had no significant cognitive complaints, no evidence of impairment on NP testing, and/or no loss of functional capacity; 2) ANI: Subjects had no significant cognitive complaints, but NP testing revealed evidence of mild NP abnormalities that do not impair activities associated with daily living or manifest themselves with clinical symptoms; 3) MND: Subjects or others reported symptoms of cognitive decline, evidence of mild NP impairment, decline in functional capacity that does not reach severity required to diagnose. Subjects may or may not have had a diagnostic evaluation to rule out other causes of cognitive impairment; and 4) HAD: Subjects or others reported symptoms of cognitive decline, evidence of moderate or severe NP impairment on NP testing, decline in functional capacity that reaches the level of dementia. Subjects may or may not have had a diagnostic evaluation to rule out other causes of NP impairment. The demographics and clinical characteristics of these NN and HAND patients are summarized in Table 1 and Table S1. Peripheral blood from 9 HIV-1-negative (HD) individuals who were matched for the gender, age, and ethnicity of HAND patients was collected in BD vacutainer tubes (BD Biosciences, Franklin Lakes, NJ). Blood samples were separated into plasma and peripheral blood mononuclear cells (PBMCs) that were stored at −80°C until use.

Table 1.

Demographics and clinical characteristics of study subjects.

Characteristics HAND
(n = 19)
NN
(n = 9)
p
value
HD
(n = 9)
Age (years) 45 ± 2 48 ± 2 0.4 47 ± 5
Gender (% Male) 78 90 0.4 67
Race (% White) 67 63 0.9 56
CD4 count (cells/µL) 150 ± 34 301 ± 91 0.1 ND
Plasma HIV RNA (log10/mL) 4.0 ± 0.3 3.4 ± 0.5 0.3 ND

Note: HAND, HIV-1-associated neurocognitive disorders; NN, HIV-1 patients who had no neurocognitive disorders; HD, HIV-1-negative donors; ND, not determined.

Frozen CSF samples from 8 HIV-1 patients who were diagnosed with HAND were kindly provided by the California NeuroAIDS Tissue Network (CNTN) at UCSD (Dr. Ellis RJ.) through the NNTC (Rockville, MD) (Table 2). Nine CSF samples from HD individuals who had CNS symptoms due to unidentified pathogens were obtained from the Lee Biosolutions (Maryland Heights, MO) or Indiana University Medical Center (Table 2).

Table 2.

Characteristics of HAND and HIV-1-negative individuals, CSF.

CSF ID Gender (M/F) Age (year) Race NeuroAIDS status CSF viral load (log10/mL) cART
HAND
NNTC005 F 38 NW ANI 4.9 missing
NNTC026 M 44 W ANI missing ABC + APV + EFV
NNTC012 M 42 NW ANI 3.9 missing
NNTC037 M 54 W ANI 4.9 ART-
NNTC010 M 62 W MND missing 3TC + KTA + RTV + TFV
NNTC068 M 60 W MND missing missing
NNTC058 M 47 W HAD 1.7 3TC + ABC + APV
NNTC126 M 61 W HAD 1.7 ATV + EPZ + RTV
HIV-1-negative
T44 M 54
T45 F 57
T51 F 69
T52 F 24
T53 F 74
T54 F 76
T55 F 24
T56 M 91
T57 F 46

Note: M, male; F, female; W, white; NW, non-white; ANI, asymptomatic neurocognitive impairment; MND, mild neurocognitive disorder; HAD, HIV-1-associated dementia; ABC, abacavir; APV, amprenavir; EFV, efavirenz; 3TC, lamivudine; KTA, lopinavir/ritonavir; RTV, ritonavir; TFV, tenofivir; ATV, atazanavir; EPZ, ABC + lamivudine.

Peripheral blood and CSF samples from HD individuals were obtained under an IRB protocol approved by the Indiana University School of Medicine Institutional Review Board. Written informed consent was obtained from each participant before specimen collection.

2.2. Cell lines and cell culture

Human glioblastoma cell line U87MG (kindly provided by Dr. Ivan M., IUSM, Indianapolis, IN) was cultured in high glucose Dulbecco’s modified eagle’s medium (H-DMEM, GE Healthcare Life Sciences, Logan, UT) supplemented with 10% fetal bovine serum (FBS) (Atlanta Biologicals, Flowery Branch, GA), 100 U/mL penicillin, and 100 U/mL streptomycin (GE Healthcare Life Sciences). Human capillary microvascular endothelial cell line hCMEC/D3 (Cellutions Biosystem, Burlington, Ontario, Canada) was cultured in EndoGRO-2MV Complete Culture Media (EMD Millipore, Billerica, MA) supplemented with human basic fibroblast growth factor (bFGF, Sigma-Aldrich, St. Louis, MO), 100 U/mL penicillin, and 100 U/mL streptomycin according to the manufacturer’s protocol. U87MG and hCMEC/D3 cells were cultured at 37 °C in a 5% CO2 incubator, and the medium was changed every 2–3 days.

2.3. Gelatin zymography assay

Gelatinase activity of human plasma and CSF was determined by a sodium dodecyl sulfate-polyacrylamide gel electrophoresis (SDS-PAGE) gelatin zymography assay as previously described (Toth et al., 2012). Briefly, plasma or CSF was mixed with sample buffer in the absence of any protein-reducing agents and loaded onto polyacrylamide gels containing 0.1% (w/v) gelatin for electrophoresis (Novex 10% Zymogram Gelatin Protein Gels, Invitrogen, Carlsbad, CA). After electrophoresis, gels were washed in renaturing buffer, followed by washing in developing buffer, and subsequently incubated in developing buffer at 37 °C for 40 h. Thereafter, gels were stained with a Colloidal Blue Staining Kit (Invitrogen, Carlsbad, CA), and gelatinolytic activity was detected as clear bands on the blue background. The activity of MMPs was determined by densitometric scanning of the bands using an EC3 Imaging System (UVP, Upland, CA). Photographic densities were analyzed using NIH Image J software (National Institutes of Health, Bethesda, MD) to determine the enzymatic activity of MMP-2 (62–72 kDa) and MMP-9 (82–92 kDa).

2.4. Enzyme-linked immunosorbent assay (ELISA)

Total protein concentrations of MMP-2, MMP-9, TIMP-1, TIMP-2, soluble CD14 (sCD14), and soluble CD163 (sCD163) in plasma and CSF samples were determined using ELISAs. All the ELISA kits were purchased from R&D Systems (Minneapolis, MN). ELISA results were documented using a microplate reader system (Bio-Tek, Winooski, VT).

2.5. Multiplex immunoassays

CSF concentrations of 38 human cytokines/chemokines (sCD40L, EGF, Eotaxin, FGF-2, Flt-3 ligand, Fractalkine, G-CSF, GM-CSF, GRO, IFN-α2, IFN-γ, IL-1α, IL-1β, IL-1ra, IL-2, IL-3, IL-4, IL-5, IL-6, IL-7, IL-8, IL-9, IL-10, IL-12p40, IL-12p70, IL-13, IL-15, IL-17A, IP-10, MCP-1, MCP-3, MDC, MIP-1α, MIP-1β, TGF-α, TNFα, TNF-β, and VEGF) were simultaneously measured using a magnetic bead-based multiplex kit (HCYTMAG-60K-PX38, EMD Millipore, Billerica, MA) as per the manufacturer’s instruction. Samples were incubated with beads at 4 °C for 16 h with shaking at 500 rpm, and subsequently washed twice using a hand-held magnetic plate washer (eBioscience, San Diego, CA). The beads were incubated with the biotinylated detection antibody for 1 h at room temperature with shaking at 500 rpm, followed by incubation with streptavidin-PE for 30 min at room temperature with shaking at 500 rpm. After washing twice, the beads were resuspended in shield buffer and read on a Luminex-100 system (EMD Millipore, Billerica, MA) with a setting of 40 beads per bead set and 150 s per well. The standards (3.2–10,000 pg/mL) provided in the HCYTMAG-60 K-PX38 kit were run on each plate in duplicate, and used to calculate the concentrations of cytokines/chemokines using the Bio-Plex Manager Software (Bio-Rad, Hercules, CA).

2.6. In vitro BBB model for analysis of BBB permeability and cell transmigration

An in vitro BBB model consisting of hCMEC/D3 and U87MG cells in co-culture on opposite sides of a collagen-coated, 0.4 µm poresize tissue culture insert (for permeability assay) or 3.0 µm poresize tissue culture insert (for cell transmigration assay) (Corning, Tewksbury, MA) was established as previously described (Eugenin and Berman, 2003; Eugenin et al., 2011). Briefly, U87MG cells were seeded on the Poly-L-Lysine coated basolateral side of the transwell insert at a density of 2 × 104, and hCMVEC/D3 cells were seeded on the apical side of the insert (2 × 104/insert) with type I collagen coating. Co-cultures were maintained for 48–72 h to enable contact between astrocyte endfeet with hCMVEC/D3 cells on the opposite side of the model as previously described (Eugenin et al., 2011).

To analyze the effects of plasma from HIV-1 patients on BBB permeability, 25% (v/v) plasma (in EGM-2MV medium) from healthy donors or HIV-1 patients was added to hCMEC/D3 cells in the transwell inserts and incubated for 24 h at 37 °C in a 5% CO2 incubator. After washing twice with 1x HBSS to remove the medium, FITC-dextran in 1x HBSS at 1 mg/mL was added to the inserts, while 1x HBSS was added to the lower chamber. The HBSS from the basolateral lower chamber was sampled after treatment for 1 h to assess the degree of FITC-dextran flux across the BBB. Fluorescence intensity of FITC-dextran was measured at excitation wavelength 488 nm and at emission wavelength 525 nm using a Synergy 4 microplate reader (Biotek, VT). The degree of diffused FITC-dextran was expressed as fold changes in fluorescence intensity relative to media from vehicle controls (Alabanza and Bynoe, 2012).

To analyze the effects of plasma from HIV-1 patients on cell transmigration across the BBB model, 25% (v/v) plasma (in EGM-2MV medium) from healthy donors or HIV-1 patients was added to hCMEC/D3 cells in the transwell inserts and incubated for 24 h at 37 °C in a 5% CO2 incubator. After washing away the plasma, fresh PBMCs (1 × 106) from healthy donors were gently added to the top chambers of hCMEC/D3 layers growing on the insert filters. Cells were incubated at 37 °C for 4 h to allow cell transmigration to the bottom chambers. The entire population of transmigrated cells present in each bottom chamber was collected by centrifugation at 300g for 10 min. The bottom sides of the insert filters were also rinsed with 2% FBS/2 mM EDTA to detach any adherent cells as previously described (Bahbouhi et al., 2009). MCP-1 (R&D Systems, Minneapolis, MN) at 100 ng/mL was used in every cell transmigration experiment as a positive control, as MCP-1 has been widely used for analyzing cell transmigration across the BBB (Eugenin et al., 2006).

2.7. Flow cytometric analysis

PBMCs and transmigrated cells were subjected to cell surface staining with fluorochrome-conjugated antibodies against CD45 (a marker of human hematopoietic cells or leukocytes) and CD14 (a marker of human monocytes/macrophages) and flow cytometric analysis to determine cell numbers and types. Isotype-matched control antibodies were used as negative controls in each staining. All specific and control antibodies were purchased from BD Biosciences (Franklin Lakes, NJ). Cells were subjected to surface staining with antibodies in the dark at 4 °C for 30 min. After washing 3 times with 2% FBS/PBS, cells were fixed with 2% paraformaldehyde (PFA) and subjected to flow cytometric analysis using a BD Accuri C6 flow cytometer to determine cell numbers and types. Flow cytometric analysis data were analyzed using FlowJo software (Tree Star, San Carlos, CA).

2.8. Proteomic analysis of CSF samples

Proteomic analysis of CSF samples was conducted in the Purdue Proteomics Facility, Purdue University (West Lafayette, IN). Briefly, each sample was denatured by cold acetone, reduced, alkylated, and followed by digestion with a Lys-C/trypsin mixture. Samples were cleaned using a silica-based C18 MacroSpin Column (Supelco, Bellefonte, PA). The final peptide concentration was determined using a BCA assay (Pierce, Rockford, IL). The samples were run on a nanoEksigent 425 HPLC system coupled to the Triple TOF 5600 plus (Sciex, Framingham, MA), and analyzed for 120 min at 300 nL/min over the cHiPLC nanoflex system. The trap column was a Nano cHiPLC 200 µm × 0.5 mm ChromXP C18-CL 3 µm 120 Å followed by the analytical column, the Nano cHiPLC 75 µm × 15 cm ChromXP C18-CL 5 µm 120 Å. Each sample was injected into the Triple TOF 5600 plus through the Nanospray III source equipped with emission tip from New Objective. Peptides from the digestion were eluted from the columns using a mobile phase A of purified H2O/0.1% formic acid (FA) and a mobile phase B of ACN/0.1% FA. With a flow rate of 0.3 µL/min, the method started at 95% A for 1 min followed by a gradient of 5% B to 35% B in 90 min and from 35% B to 80% B in 2 min. 80% B was held for 5 min before being brought to 5% B and held for 20 min. The data acquisition was performed to monitor 50 precursor ions at 250 ms/scan. Mascot Daemon v.2.4.0 (Matrix Science, Boston, MA) was used for database searches against the UniProt_human database.

2.9. Statistical analysis

Statistical analysis was performed using SPSS software 24.0 and GraphPad Prism 6.0 (La Jolla, CA). Data were expressed as mean ± standard error of the mean (SEM) or mean ± standard deviation (SD) unless otherwise indicated. All data were tested for suitability for parametric or non-parametric analysis in SPSS. For comparing the biological factors assessed in this study between the two groups of HD and HAND CSF samples, independent t-test or Mann-Whitney U test was used, as appropriate. For comparison of more than two groups, one-way ANOVA analysis or Kruskal-Wallis test was performed, as appropriate. Dichotomous variables such as gender and ethnicity were compared using Chi-squared test. Correlations were calculated using Spearman’s correlation coefficient analysis. Logistic regression analysis or multiple linear regression analysis was performed for taking into account the effect of covariates of age, sex and race. The p values shown in each figure are adjusted p values. A p value of <0.05 was considered statistically significant. Each in vitro experiment was performed at least 3 times with cells prepared from independent cell cultures.

3. Results

3.1. An imbalance between MMPs and TIMPs was revealed in the peripheral blood of HAND patients

Excessive expression and activation of MMPs are associated with BBB breakdown in many human diseases including HIV-1 infection (Baker et al., 2002; Rosenberg et al., 1998; Louboutin et al., 2010a,b; Louboutin et al., 2011; Louboutin and Strayer, 2012; Power et al., 1993a,b; Spindler and Hsu, 2012; Strazza et al., 2011; Webster and Crowe, 2006). To confirm the increased expression of MMPs in HIV-1 patients and explore whether these MMP alterations were more dramatic in HIV-1 patients with HAND, we determined the protein levels of MMP-2 and MMP-9, two of the most important members of the MMP family, in plasma samples from 9 HD, 9 NN, and 19 HAND individuals. The demographics and clinical characteristics of these study subjects are summarized in Table 1 and Table S1. As shown in Fig. 1A, MMP-2 and MMP-9 were present in all plasma samples analyzed. Compared to HD, both NN and HAND had significantly higher protein levels of MMP-2 (p < 0.01 and p < 0.001) and MMP-9 (p < 0.01) (Fig. 1A). However, there were no significant differences between NN and HAND in their plasma levels of either MMP-2 or MMP-9 (Fig. 1A).

Fig. 1.

Fig. 1

Imbalances between MMPs and TIMPs occurred in the peripheral blood of HAND patients. Plasma samples from 9 HD, 9 NN, and 19 HAND individuals were subjected to ELISAs to determine the levels of MMP-2, MMP-9, TIMP-1, and TIMP-2. A) and B) The scatter plots show the distribution of plasma levels (ng/mL) of MMP-2, MMP-9, TIMP-1, and TIMP-2. C) The ratio of MMP-2/TIMP-2 and MMP-9/TIMP-1 in the plasma samples. Each dot in the plots represents data from a single individual. Lines represent mean ± SEM. HD, HIV-1-negative individuals; NN, HIV-1 patients who had no neurocognitive disorders; HAND, HIV-1-associated neurocognitive disorders; ns, not significant; *p < 0.05; **p < 0.01; ***p < 0.001.

We also quantified the protein levels of TIMP-1 and TIMP-2 in the plasma samples of HD, NN, and HAND individuals, as these two tissue inhibitors of MMPs are mainly involved in the regulation of MMP-9 and MMP-2 activities, respectively (Gomez et al., 1997; Klein and Bischoff, 2011). We found that both TIMP-1 and TIMP-2 were detectable in plasma samples from all subjects studied (Fig. 1B), indicating that they are constitutively present in the peripheral blood. Compared to HD, both NN and HAND had significantly higher protein levels of TIMP-1 (p < 0.001) and TIMP-2 (p < 0.01 and p < 0.001) (Fig. 1B). There were no significant differences between NN and HAND in their plasma levels of either TIMP-1 or TIMP-2 (Fig. 1B). Interestingly, when we further analyzed the ratio of MMP-2/TIMP-2 or MMP-9/TIMP-1, we found that HAND had a significantly higher ratio of MMP-2/TIMP-2 when compared with HD (p < 0.05), whereas NN had a similar ratio of MMP-2/TIMP-2 with that in HD (Fig. 1C). There were no differences in the MMP-9/TIMP-1 ratio among HD, NN, and HAND (Fig. 1C).

3.2. Enzymatic activity of MMP-2 and MMP-9 was increased in the peripheral blood of HAND patients

All MMPs are initially synthesized as enzymatically inactive forms (proMMPs or zymogens), which are subsequently processed to generate the active forms (Klein and Bischoff, 2011; Nagase et al., 2006; Nagase and Woessner, 1999; Nelson and Bissell, 2006; Ra and Parks, 2007; Seiki, 2003; Vincenti, 2001). Since we found that the plasma levels of MMP-2, MMP-9, TIMP-1, and TIMP-2 were dysregulated in HIV-1 patients, we next used the gelatin zymography assay to analyze the enzymatic activity of plasma MMP-2 and MMP-9. The enzymatic activities of both MMP-2 and MMP-9 in NN and HAND samples were significantly higher than those in HD samples (Fig. 2A, B). Notably, MMP-2 activity was even higher in HAND patients when compared with NN individuals (p < 0.05), while MMP-9 activity in NN and HAND samples was not different (Fig. 2B).

Fig. 2.

Fig. 2

Enzymatic activity of MMP-2 and MMP-9 was increased in the peripheral blood of HIV-1 patients. Enzymatic activity of MMP-2 and MMP-9 in the plasma samples from HD, NN, and HAND individuals was evaluated using the gelatin zymography assay. A) Representative zymography results of the plasma samples from 3 HD, 2 NN, and 4 HAND individuals. B) Enzymatic activity of MMP-2 and MMP-9 in the plasma samples from HD, NN, and HAND individuals was quantified using Image J software. Each dot in the plots represents data from a single individual. Lines represent mean ± SEM. A.U., arbitrary unit; HD, HIV-1-negative individuals; NN, HIV-1 patients who had no neurocognitive disorders; HAND, HIV-1-associated neurocognitive disorders; ns, not significant; *p < 0.05; **p < 0.01; ***p < 0.001.

3.3. Elevated plasma levels of MMP-2 and TIMP-2 positively correlated with markers of viral pathogenesis

To determine whether the elevated plasma levels of MMPs and TIMPs were associated with markers of viral pathogenesis, we analyzed the correlations among HIV-1 viral load, CD4 T cell count, and plasma levels of MMPs and TIMPs. We found that the plasma level of MMP-2 positively correlated with viral load (Fig. 3A, r = 0.66, p = 0.0001) and negatively correlated with CD4 T cell count (Fig. 3B, r = −0.40, p = 0.03). Plasma levels of TIMP-2 positively correlated with viral load (Fig. 3C, r = 0.52, p = 0.005), and negatively correlated with CD4 count (Fig. 3D, r = −0.46, p = 0.01). No correlations were found between plasma level of MMP-9 and either viral load or CD4 T cell count (data not shown).

Fig. 3.

Fig. 3

Correlations between plasma MMPs or TIMPs levels and markers of viral pathogenesis in HIV-1 patients. Correlation between plasma level of MMP-2 or TIMP-2 and viral load (A and C) or CD4 count (B and D) in HAND patients. Each dot in the plots represents data from a single individual. Significance was determined using the Spearman correlation test. Spearman correlation coefficient and p value (r and p) are indicated.

3.4. Plasma from HIV-1 patients disrupted the BBB integrity and increased monocyte transmigration across an in vitro BBB model

Since we demonstrated that MMP-2 and MMP-9 levels and proteolytic activities were elevated in plasma samples from HIV-1 patients, we tested whether plasma from HIV-1 patients was able to directly affect BBB integrity. As shown in Fig. 4A, plasma from HIV-1 patients increased the BBB model permeability by 2-fold when compared with plasma from HD individuals (p = 0.05). The increased BBB permeability was partially mediated by excessive expression and activation of MMPs in the plasma, as pretreatment of the BBB with GM6001, a pan-MMP inhibitor, ameliorated the impairment of the BBB permeability (data not shown). To examine whether the impaired BBB integrity induced by plasma from HIV-1 patients could further lead to an increased penetration of leukocytes (CD45+ cells), especially monocytes/macrophages, we carried out a cell transmigration assay by adding PBMCs from HD individuals into upper chambers of the in vitro BBB model treated with plasma from HD or HIV-1 subjects. Compared to HD plasma treatment, treatment with plasma from HIV-1 patients significantly increased the numbers of transmigrated leukocytes in the lower chambers (Fig. 4B, C). Flow cytometric analysis revealed that the majority of transmigrated leukocytes were CD45+CD14+ monocytes/macrophages (Fig. 4B, C). As MCP-1 is a potent monocyte/macrophage attractant and plays a critical role in promoting migration of monocytes/macrophages across the BBB of HIV-1 patients, we also tested whether addition of MCP-1 in lower chambers increased transmigration of monocytes/macrophages. We found that MCP-1 significantly increased transmigration of monocytes/macrophages (Fig. 4C). Our findings indicate that plasma from HIV-1 patients directly impairs the BBB integrity and function, leading to an enhanced transmigration of monocytes/macrophages into the CNS.

Fig. 4.

Fig. 4

Effects of plasma from HIV-1 patients on BBB permeability and PBMC transmigration. Fresh PBMCs from healthy donors were added into inserts of in vitro BBB model that were untreated or treated with plasma from HD, HIV-1 patients, or MCP-1. A) The permeability of the BBB model was analyzed after treatment with plasma from HD or HIV-1 patients for 24 h by quantifying the passage of FITC-conjugated Dextran through co-cultures. B) and C) Transmigrated cells in the lower chamber were collected and stained for flow cytometric analysis to quantify the number of monocytes/macrophages (CD45+CD14+) in the transmigrated population of cells. Forward and side scatter were used to determine transmigrated cell gating. Data from at least 3 donors were expressed as mean ± SEM. *p < 0.05; ***p < 0.001.

3.5. MMPs and TIMPs were dysregulated in the CSF of HAND patients

The increased protein levels and enzymatic activity of MMPs in the peripheral blood of HIV-1 patients are likely involved in over-degradation of ECM components of the BBB, thereby disrupting BBB permeability and function. Damaged BBB allows influx of normally excluded molecules and inflammatory cells into the brain. Consequently, MMPs may be also increased and activated in the CNS of HIV-1 patients, leading to over-degradation of the ECM proteins that are important to cell structure and survival in the CNS (Cinque et al., 1998; Moore et al., 2011). To this end, we obtained 17 CSF samples including those from 8 HAND patients and 9 HD individuals (Table 2) and examined the protein levels and enzymatic activity of MMPs and TIMPs in these CSF samples. As shown in Fig. 5A, MMP-2, TIMP-1, and TIMP-2 were detected in all 17 CSF samples, albeit at various concentrations. Compared to HD, HAND patients had a significantly higher CSF level of TIMP-1 (Fig. 5A). There was a trend that CSF MMP-2 and TIMP-2 levels were also elevated in HAND than in HD, but not significant (Fig. 5A). Strikingly, MMP-9 was detected by ELISA in all 8 HAND CSF samples, but was detected at a very low level in only 1 of the 9 CSF HD samples (Fig. 5A).

Fig. 5.

Fig. 5

Expression and activation of MMPs and TIMPs in the CSF from HIV-1-negative individuals and HAND patients. CSF samples collected from 9 HD and 8 HAND patients were studied. A) The scatter plots showed CSF levels of MMP-2, MMP-9, TIMP-1, and TIMP-2 (ng/mL). B) Enzymatic activity of MMP-2 and MMP-9 from 4 HD and 4 HAND, and quantification data of MMP-2 and MMP-9 in all CSF samples tested. A.U., arbitrary unit; HD, HIV-1-negative individuals; HAND, HIV-1-associated neurocognitive disorders; ns, not significant; **p < 0.05; ***p < 0.001.

The gelatin zymography assay revealed an MMP-2 band in every CSF sample (Fig. 5B). The MMP-2 bands were much stronger in HAND patients than in HD individuals (Fig. 5B). Pooled data showed that the proteolytic activity of MMP-2 was significantly increased in the HAND CSF than that of HD CSF. The active form of MMP-9 was detected in all 8 HAND CSF samples, while none of the HD CSF samples had the active MMP-9 enzyme bands (Fig. 5B).

3.6. Imbalances between MMPs and TIMPs were linked to monocyte/macrophage activation in the peripheral blood, but not CSF, of HAND patients

Over-activation of monocytes/macrophages profoundly contributes to systemic inflammation and BBB damage in HIV-1 patients, leading to HIV-1-associated neuropathogenesis (Anzinger et al., 2014; Gras and Kaul, 2010; Kim et al., 2005). The levels of sCD14 and sCD163 have been used as reliable markers of monocytes/macrophage activation and systemic inflammation in HIV-1 patients (Brenchley et al., 2006; Burdo et al., 2011a,b). We quantified plasma and CSF levels of sCD14 and sCD163 in HD, HAND, and/or NN groups. In agreement with previous reports (Burdo et al., 2011b; Lawn et al., 2000; Sandler et al., 2011), plasma levels of sCD14 and sCD163 were elevated in both NN and HAND patients when compared with HD individuals (Fig. 6A). There was no significant difference between NN and HAND in their plasma levels of sCD14 and sCD163 (Fig. 6A). Since sCD14 and sCD163 are likely shed by MMP proteolytic cleavage at the surface of monocytes/macrophages, we analyzed the correlations of increased plasma levels of sCD14 and sCD163 with plasma levels of MMPs and TIMPs. Spearman correlation analysis demonstrated that plasma levels of MMP-2 and TIMP-2 strongly correlated with the increased plasma levels of both sCD14 (r = 0.46, p = 0.01 and r = 0.55, p = 0.002, Fig. 6B) and sCD163 (r = 0.58, p = 0.001 and r = 0.68, p < 0.0001, Fig. 6C). There were no significant correlations between either sCD14 or sCD163 and plasma concentrations of MMP-9 (data not shown). Both sCD14 and sCD16 were detected in all CSF samples from HD and HAND individuals. However, neither sCD14 nor sCD163 was increased in the CSF of HAND patients when compared with HD individuals (Fig. S1).

Fig. 6.

Fig. 6

Correlations between MMPs and monocyte/macrophage activation markers in the peripheral blood of HIV-1 patients. A) Plasma samples from 9 HD, 9 NN, and 19 HAND individuals were subjected to ELISAs to determine the levels of sCD14 and sCD163 (ng/mL). Each dot in the plots represents data from a single individual. Lines represent mean ± SEM. B) and C) Correlation between plasma sCD14 or sCD163 and MMP-2 or TIMP-2. Significance was determined using the Spearman correlation test. Spearman correlation coefficient and p value (r and p) are indicated.

3.7. Inflammatory cytokines/chemokines affected the production and activation of MMPs in the CNS

To clarify whether inflammatory factors are involved in regulation of production and activation of MMPs in the CNS, we simultaneously quantified CSF levels of 38 cytokines/chemokines. As shown in Table 3, HAND patients had significantly higher CSF levels of 6 cytokines/chemokines including IFN-α2, TNF-α, Fractalkine, IL-1α, TGF-α, and MCP-1 when compared with HD individuals. Interestingly, the upregulated level of CSF MCP-1 was strongly correlated with the CSF level of MMP-2 in HAND patients (Fig. S2), suggesting that there is a complex interplay among the three types of factors (inflammatory cells, cytokines/chemokines, and MMPs) in the CNS of HIV-1 patients. It is important to note that these HIV-1-negative donors had varied CNS symptoms, thereby their CSF samples are not representative of healthy individuals.

Table 3.

Comparison of CSF levels of 38 cytokines/chemokines between HD and HAND patients.

Cytokine/chemokine Groups (Mean ± SEM) (pg/mL)
p value
HD (n = 9) HAND (n = 8)
IFN-α2 16.7 ± 4.2 38.9 ± 3.6 0.0045
TNF-α 1.1 ± 0.2 3.0 ± 0.6 0.0050
MCP-1 1994.0 ± 184.3 3577.0 ± 571.6 0.0207
IL-1α 33.8 ± 12.6 164.1 ± 69.6 0.0267
Fractalkine 69.6 ± 11.6 103.6 ± 14.4 0.0383
TGF-α 2.4 ± 0.3 13.7 ± 7.0 0.0484
CCL5 43.2 ± 0.4 59.6 ± 13.0 0.0660
IP-10 950.2 ± 247.7 3627.0 ± 1581.0 0.0712
Flt-3L 42.2 ± 13.3 208.0 ± 113.5 0.1206
GM-CSF 8.8 ± 1.4 12.8 ± 2.2 0.1857
IL-17α 1.5 ± 0.6 0.4 ± 0.2 0.3448
IL-8 42.7 ± 8.8 72.9 ± 22.7 0.3704
IL-3 8.6 ± 1.6 7.0 ± 0.9 0.4234
IL-6 75.1 ± 60.0 15.0 ± 7.7 0.5946
G-CSF 151.6 ± 83.1 64.9 ± 23.3 0.6371
IL-15 4.4 ± 1.0 4.4 ± 1.1 0.6372
IL-9 3.2 ± 0.7 4.1 ± 1.0 0.6646
TNF-β 0/9 1/8
VEGF 0/9 5/8
IL-12p70 0/9 1/8
IFN-γ 6/9 4/8
GRO 2/9 5/8
IL-10 0/9 2/8
MCP-3 2/9 1/8
IL-12p40 1/9 1/8
MDC 2/9 5/8
IL-13 1/9 1/8
EGF 0/9 3/8
FGF-2 0/9 4/8
Eotaxin 1/9 5/8
sCD40L 5/9 4/8
MIP-1α 1/9 1/8
MIP-1β 6/9 2/8
IL-7 1/9 4/8
IL-4 0/9 1/8
IL-5 0/9 2/8
IL-RA 5/9 5/8
IL-1β 0/9 1/8
IL-2 6/9 2/8

Note: HD, HIV-1-negative donors; HAND, HIV-1-associated neurocognitive disorders; x/9 or x/8, number of CSF samples had detectable levels of cytokines/ chemokines as indicated out of the total of 9 or 8 CSF samples.

The bold section shows all the data that has difference statistically.

To further explore the mechanisms underlying the induction and activation of MMPs in the CNS of HAND patients, we conducted proteomic analysis on the CSF samples. The Venn diagram in Fig. S3A illustrates the overview of the CSF proteome. Among these proteins, 57, 153, and 243 proteins were identified in the CSF of HD only, both HD and HAND, and HAND only, respectively. The 57 unique proteins found in the HD group were mainly defense/immunity proteins, enzyme modulators, extracellular matrix proteins, receptors, and signaling molecules (Fig. S3B). Defense/immunity proteins and signaling molecules that maintain brain homeostasis were the most abundant proteins (23.5%) (Fig. S3B). There were 243 proteins found only in the HAND CSF samples, and these proteins were mainly involved in calcium binding, cell adhesion, membrane traffic, nucleic acid binding and redox process (Fig. S3C). Furthermore, the most abundant proteins were hydrolases (19%) and oxidoreductases (14.3%), indicating that enzymatic proteins and oxidative reactions were upregulated in HAND patients. Both HD and HAND samples had 153 proteins in common and divided into 12 groups based on their functional annotation (Fig. S3D). Among these shared proteins, 24 proteins were statistically different in their levels between HD and HAND samples (Fig. 7). Six and 18 proteins of these 24 proteins were downregulated and up-regulated, respectively, in the CSF of HAND patients when compared with HD individuals. The down-regulated proteins included apolipoprotein A-II, calsyntenin-1, vitamin D-binding protein, complement factor B, C-type lectin domain family 3, and hemopexin, which are mainly associated with the calcium homeostasis and neuroimmunological function in the brain. The up-regulated proteins included 5 epididymis secretory proteins that can be activated by IL-1β or TGF-β, indicating that inflammation occurs in the CNS of HAND patients.

Fig. 7.

Fig. 7

Proteomics analysis of CSF samples from HIV-1-negative donors and HAND patients. Protein profile analysis, as shown in the heat map, reveals the proteins differentially expressed between HIV-1-negative and HAND CSF samples.

4. Discussion

The principal finding of the present work is that imbalances between MMPs and TIMPs with distinct patterns occurred in both the peripheral blood and CNS of HIV-1 patients, especially those with HAND. In the peripheral blood, the protein levels of MMP-2, MMP-9, TIMP-1, TIMP-2, and the enzymatic activity of MMP-2 and MMP-9 were increased in HIV-1 patients with or without HAND when compared with HD individuals (Figs. 1, 2). The enzymatic activity of MMP-2, but not MMP-9, was further increased in HAND than in NN (Fig. 2). Importantly, the ratio of MMP-2/TIMP-2 was significantly increased in HAND patients, not in NN patients (Fig. 1C), which may represent a key risk indicator for HAND. In the CSF, induction and activation of MMP-9 in HAND patients became particularly prominent as MMP-9 expression and enzymatic activity was detected in all 8 CSF samples from HAND patients, but only 1 of 9 CSF samples at a low level from HD individuals (Fig. 5). These results indicate that MMP-9 is not constitutively expressed in the CNS, but is likely induced de novo and activated in the CNS of HAND patients. In addition to MMP-9, MMP-2 enzymatic activity and TIMP-1 expression, but neither expression of MMP-2 and TIMP-2 nor the ratio of MMP-2/TIMP-2, were increased in the CSF samples from HAND patients when compared with HD individuals (Fig. 5). Taken together, the increased ratio of MMP-2/TIMP-2 in the peripheral blood may play a key role in HIV-1-associated neuropathogenesis by disrupting the BBB to increase the penetration of infected cells and neurotoxic substances from the blood into the CNS (Lakhan et al., 2013; Louboutin et al., 2010a,b, 2011; Louboutin and Strayer, 2012; Power et al., 1993a,b; Rosenberg et al., 1998; Singh, 2014; Spindler and Hsu, 2012; Strazza et al., 2011; Webster and Crowe, 2006). Additionally, the robust induction of MMP-9 and activation of MMP-2 and MMP-9 in the CNS of HIV-1 patients may not only damage the BBB, but also degrade nerve tissue proteins to induce death of neurons (Webster and Crowe, 2006). Thus, the imbalances of MMPs and TIMPs occurring in the peripheral blood and CNS of HIV-1 patients likely play a crucial role in the HIV-1-associated neuropathogenesis by disrupting the BBB, increasing influx of inflammatory cells, and degrading neuronal survival molecules.

MMPs and TIMPs are expressed by various cell types, and their gene expressions are regulated by a variety of extracellular factors including cytokines, growth factors, and cell-cell interactions (Lambert et al., 2004; Westermarck and Kahari, 1999). Studies have demonstrated that activated monocytes/macrophages orchestrate these events and play a central role in BBB injury in HIV-1 patients (Anzinger et al., 2014; Gras and Kaul, 2010; Kim et al., 2005). Activated monocytes/macrophages increase the production of MMPs and inflammatory cytokines, enhance infiltration of inflammatory cells and HIV-1-infected cells into the CNS, and accelerate the influx of neurotoxic substances into the brain, thereby promoting HIV-1-associated neuropathogenesis (Anzinger et al., 2014; Gras and Kaul, 2010; Kim et al., 2005). Along these lines, we measured circulating levels of sCD14 and sCD163, two markers of monocyte/macrophage activation and inflammation (Brenchley et al., 2006; Burdo et al., 2011a,b), in the plasma from HIV-1 patients and HD individuals. In agreement with previous reports (Burdo et al., 2011b; Lawn et al., 2000; Sandler et al., 2011), sCD14 and sCD163 were elevated in the plasma of HAND and NN patients when compared with HD individuals (Fig. 6). Interestingly, upregulation of plasma MMP-2 in HIV-1 patients strongly correlated with the increased levels of both sCD14 and sCD163 (Fig. 6), suggesting that sCD14 and sCD163 are likely shed via proteolytic cleavage by MMPs, especially MMP-2, at the surface of monocytes/macrophages. This result provides insight into the molecular mechanisms underlying the increased production of sCD14 and sCD163 in the peripheral blood of HIV-1 patients. In addition, upregulation of plasma MMP-2 positively correlated with viral loads and negatively correlated with CD4 counts in the HIV-1 patients included in this study (Fig. 3). We acknowledge that these results are primarily derived from patients not receiving virologically suppressive cART, and thus these data may not apply to HAND patients who have undetectable plasma viral loads.

To clarify whether the imbalances between MMPs and TIMPs found in both the peripheral blood and CNS of HIV-1 patients, especially those with neurocognitive disorders, were driven by inflammatory factors, we examined the profiles of 38 cytokines/chemokines in the plasma and CSF samples from HAND patients and HD individuals. HAND patients had significantly higher levels of MCP-1 and TNF-α in both the peripheral blood (data not shown) and CSF (Table 3, Fig. S2A). MCP-1 is one of the key chemokines that regulate migration and infiltration of monocytes/macrophages (Deshmane et al., 2009). We found that the upregulated CSF levels of MCP-1 and MMP-2 strongly correlated in HAND patients (Fig. S2B), suggesting that MMPs and MCP-1 may be two key players in the pathogenesis of HAND. It likely occurs that MMPs damage the BBB, allowing increased penetration of activated/infected monocytes/macrophages into the CNS. These migrated monocytes/macrophages directly or indirectly produce MCP-1 and other cytokines/chemokines to attract more activated cells into the CNS, leading to accelerated production of MMPs. Thus, MMPs and MCP-1 generate a positive feedback loop to enhance their neuropathic effects in HIV-1 patients. It is important to note that the CSF samples were obtained from HD individuals who had varied CNS symptoms, thereby not representing that from truly healthy individuals.

Since TNF-α was increased in both the peripheral blood (data not shown) and CSF (Table 3) in HIV-1 patients, we tested whether TNF-α affected production and activation of MMPs by brain cells including astrocytes and brain capillary microvascular endothelial cells. We found that a human astrocyte cell line (U87MG), but not the human brain capillary microvascular endothelial cell line (hCMEC/D3), produced both the latent and active forms of MMP-2 in response to TNF-α stimulation in a time-dependent manner (Fig. S4). However, neither U87MG nor hCMEC/D3 cells produced any forms of MMP-9 (Fig. S4). Therefore, there are still questions remaining about the induction and activation of MMP-9 in the CNS of HIV-1 patients: (1) which types of neural cells can be induced to produce MMP-9, (2) what factors trigger the production and activation of MMP-9, and (3) the molecular mechanisms underlying the robust induction and activation of MMP-9 found in the CNS of HAND patients.

Overwhelming evidence has demonstrated that HIV-1 invades the CNS in part by breaking down the BBB, which represents a key step in HIV-1-associated neuropathy. However, there is no evidence thus far showing that plasma from HIV-1 patients can directly impair the BBB integrity and function. Utilizing the human astrocyte cell line U87MG and brain capillary microvascular endothelial cell line hCMEC/D3 to establish an in vitro BBB model as previously described (Eugenin and Berman, 2003; Eugenin et al., 2011), we demonstrated that plasma from HIV-1 patients was able to directly impair the permeability of the BBB model as FITC-dextran diffusion and transmigration of monocytes/-macrophages across the BBB were significant (Fig. 4). Importantly, pre-treatment of the BBB with GM6001, a pan-MMP inhibitor (Afonso et al., 2013), ameliorated the impairment of the BBB permeability (data not shown), indicating that MMPs in the plasma from HIV-1 patients mediated the impairment of the BBB permeability. To our knowledge, this is the first evidence of plasma from HIV-1 patients directly impairing the BBB permeability and function, and the impairment is at least partially mediated by MMP/TIMP imbalances in the plasma.

In summary, imbalances between MMPs and TIMPs with distinct patterns occur in both the blood and CNS of HIV-1 patients, especially those with neurocognitive disorders. Robust expression and activation of MMPs not only damage the BBB and CNS tissues, but also contribute to inflammation of the CNS, thereby playing a key role in the pathogenesis of HAND. Inhibition of MMP activity may represent a therapeutic strategy to mitigate MMP-related damage during HIV-1 infection. However, it should be noted that the sample size of the individual groups was too small to allow a detailed analysis of each subtype of HAND. Additionally, the CSF samples were obtained from HD individuals who had varied CNS symptoms and thereby may not represent those from healthy individuals. Therefore, the simian immunodeficiency virus (SIV)/macaque animal model of HIV-1 infection may be used to address these issues and to further study the role of dysregulated MMPs and TIMPs in the pathogenesis of neuroAIDS.

Supplementary Material

Supplemental

Acknowledgments

We thank Ms. Shelida Maxison for the human subject study coordination, the HIV-1-infected volunteers for donating their time and blood for these experiments, Dr. Ivan M. (Indiana University School of Medicine, Indianapolis, IN) for gifting the U87MG cells, the Proteomics Facility in Purdue University, and the National NeuroAIDS Tissue Consortium (NNTC) for providing the CSF and plasma samples through four collection units including the Texas Repository for AIDS Research (Dr. Gelman BB.), Manhattan HIV Brain Bank at the Mount Sinai Medical Center (Dr. Morgello S.), UCLA National Neurological AIDS Bank (Drs. Lucey G., ImK., and Wei B.), and California NeuroAIDS Tissue Network (CNTN) at the UCSD (Dr. Ellis RJ.). This publication was made possible from NIH funding through the NIMH and NINDS Institutes by the grant (U24MH100928) to California NeuroAIDS Tissue Network (CNTN). This work was also supported in part by the Grand Challenges Explorations (GCE) Phase II grant through the Bill & Melinda Gates Foundation (OPP1035237 to QY), NIH R21R33AI104268 (QY), NIH R01AI117835 (QY), the Showalter Research Trust Fund (QY), the Research Facilities Improvement Program Grant Number C06 RR015481-01 from the National Center for Research Resources, NIH, to Indiana University School of Medicine, the National Natural Science Foundation of China (81171134 and 81471235 to JD), the Program of Introducing Talents of Discipline to University (B14036 to JD), and the Chinese Scholarship Council (CSC to YX).

Abbreviations

AIDS

acquired immunodeficiency syndrome

ANI

asymptomatic neurocognitive impairment

BBB

blood brain barrier

BMECs

brain microvascular endothelial cells

cART

combined antiretroviral therapy

CNS

central nervous system

CSF

cerebrospinal fluid

ECM

extracellular matrix

HAD

HIV-1-associated dementia

HAND

HIV-1-associated neurocognitive disorders

HIV-1

human immunodeficiency virus type 1

MCP-1

monocyte chemotactic protein 1

MMPs

matrix metalloproteinases

MND

mild neurocognitive disorder

MT-MMPs

membrane-type MMPs

NN

neurocognitively normal

ROS

reactive oxygen species

sCD14

soluble CD14

sCD163

soluble CD163

TIMPs

tissue inhibitors of MMPs

TJ

tight junction

TNF-α

tumor necrosis factor alpha

TGF-β

transforming growth factor beta

Footnotes

Conflicts of interest

We declare that we have no conflicts of interest.

Appendix A. Supplementary data

Supplementary data associated with this article can be found, in the online version, at http://dx.doi.org/10.1016/j.bbi.2017.04.024.

References

  1. Abbott NJ. Astrocyte-endothelial interactions and blood-brain barrier permeability. J. Anat. 2002;200:629–638. doi: 10.1046/j.1469-7580.2002.00064.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Afonso PV, McCann CP, Kapnick SM, Parent CA. Discoidin domain receptor 2 regulates neutrophil chemotaxis in 3D collagen matrices. Blood. 2013;121:1644–1650. doi: 10.1182/blood-2012-08-451575. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Aijaz S, Balda MS, Matter K. Tight junctions: molecular architecture and function. Int. Rev. Cytol. 2006;248:261–298. doi: 10.1016/S0074-7696(06)48005-0. [DOI] [PubMed] [Google Scholar]
  4. Alabanza LM, Bynoe MS. Thrombin induces an inflammatory phenotype in a human brain endothelial cell line. J. Neuroimmunol. 2012;245:48–55. doi: 10.1016/j.jneuroim.2012.02.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Anzinger JJ, Butterfield TR, Angelovich TA, Crowe SM, Palmer CS. Monocytes as regulators of inflammation and HIV-related comorbidities during cART. J. Immunol. Res. 2014;2014:569819. doi: 10.1155/2014/569819. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Bahbouhi B, Berthelot L, Pettre S, Michel L, Wiertlewski S, Weksler B, Romero IA, Miller F, Couraud PO, Brouard S, Laplaud DA, Soulillou JP. Peripheral blood CD4+ T lymphocytes from multiple sclerosis patients are characterized by higher PSGL-1 expression and transmigration capacity across a human blood-brain barrier-derived endothelial cell line. J. Leukoc. Biol. 2009;86:1049–1063. doi: 10.1189/jlb.1008666. [DOI] [PubMed] [Google Scholar]
  7. Baker AH, Edwards DR, Murphy G. Metalloproteinase inhibitors: biological actions and therapeutic opportunities. J. Cell Sci. 2002;115:3719–3727. doi: 10.1242/jcs.00063. [DOI] [PubMed] [Google Scholar]
  8. Bernacki J, Dobrowolska A, Nierwinska K, Malecki A. Physiology and pharmacological role of the blood-brain barrier. Pharmacol. Rep. PR. 2008;60:600–622. [PubMed] [Google Scholar]
  9. Brenchley JM, Price DA, Douek DC. HIV disease: fallout from a mucosal catastrophe? Nat. Immunol. 2006;7:235–239. doi: 10.1038/ni1316. [DOI] [PubMed] [Google Scholar]
  10. Burdo TH, Lentz MR, Autissier P, Krishnan A, Halpern E, Letendre S, Rosenberg ES, Ellis RJ, Williams KC. Soluble CD163 made by monocyte/macrophages is a novel marker of HIV activity in early and chronic infection prior to and after anti-retroviral therapy. J. Infect. Dis. 2011a;204:154–163. doi: 10.1093/infdis/jir214. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Burdo TH, Lo J, Abbara S, Wei J, DeLelys ME, Preffer F, Rosenberg ES, Williams KC, Grinspoon S. Soluble CD163, a novel marker of activated macrophages, is elevated and associated with noncalcified coronary plaque in HIV-infected patients. J. Infect. Dis. 2011b;204:1227–1236. doi: 10.1093/infdis/jir520. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Cao J, Rehemtulla A, Bahou W, Zucker S. Membrane type matrix metalloproteinase 1 activates pro-gelatinase A without furin cleavage of the N-terminal domain. J. Biol. Chem. 1996;271:30174–30180. doi: 10.1074/jbc.271.47.30174. [DOI] [PubMed] [Google Scholar]
  13. Cheng M, De B, Pikul S, Almstead NG, Natchus MG, Anastasio MV, McPhail SJ, Snider CE, Taiwo YO, Chen L, Dunaway CM, Gu F, Dowty ME, Mieling GE, Janusz MJ, Wang-Weigand S. Design and synthesis of piperazine-based matrix metalloproteinase inhibitors. J. Med. Chem. 2000;43:369–380. doi: 10.1021/jm990366q. [DOI] [PubMed] [Google Scholar]
  14. Cinque P, Vago L, Mengozzi M, Torri V, Ceresa D, Vicenzi E, Transidico P, Vagani A, Sozzani S, Mantovani A, Lazzarin A, Poli G. Elevated cerebrospinal fluid levels of monocyte chemotactic protein-1 correlate with HIV-1 encephalitis and local viral replication. AIDS. 1998;12:1327–1332. doi: 10.1097/00002030-199811000-00014. [DOI] [PubMed] [Google Scholar]
  15. Clifford DB. HIV-associated neurocognitive disease continues in the antiretroviral era. Topics HIV Med. 2008;16:94–98. [PubMed] [Google Scholar]
  16. Clifford DB, Ances BM. HIV-associated neurocognitive disorder. Lancet. Infect. Dis. 2013;13:976–986. doi: 10.1016/S1473-3099(13)70269-X. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. del Zoppo GJ. Inflammation and the neurovascular unit in the setting of focal cerebral ischemia. Neuroscience. 2009;158:972–982. doi: 10.1016/j.neuroscience.2008.08.028. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Deshmane SL, Kremlev S, Amini S, Sawaya BE. Monocyte chemoattractant protein-1 (MCP-1): an overview. J. Interferon Cytokine Res. 2009;29:313–326. doi: 10.1089/jir.2008.0027. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Eugenin EA, Berman JW. Chemokine-dependent mechanisms of leukocyte trafficking across a model of the blood–brain barrier. Methods. 2003;29:351–361. doi: 10.1016/s1046-2023(02)00359-6. [DOI] [PubMed] [Google Scholar]
  20. Eugenin EA, Osiecki K, Lopez L, Goldstein H, Calderon TM, Berman JW. CCL2/monocyte chemoattractant protein-1 mediates enhanced transmigration of human immunodeficiency virus (HIV)-infected leukocytes across the blood-brain barrier: a potential mechanism of HIV-CNS invasion and NeuroAIDS. J. Neurosci. 2006;26:1098–1106. doi: 10.1523/JNEUROSCI.3863-05.2006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Eugenin EA, Clements JE, Zink MC, Berman JW. Human immunodeficiency virus infection of human astrocytes disrupts blood-brain barrier integrity by a gap junction-dependent mechanism. J. Neurosci. 2011;31:9456–9465. doi: 10.1523/JNEUROSCI.1460-11.2011. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Galis ZS, Khatri JJ. Matrix metalloproteinases in vascular remodeling and atherogenesis: the good, the bad, and the ugly. Circ. Res. 2002;90:251–262. [PubMed] [Google Scholar]
  23. Gomez DE, Alonso DF, Yoshiji H, Thorgeirsson UP. Tissue inhibitors of metalloproteinases: structure, regulation and biological functions. Eur. J. Cell Biol. 1997;74:111–122. [PubMed] [Google Scholar]
  24. Gras G, Kaul M. Molecular mechanisms of neuroinvasion by monocytes-macrophages in HIV-1 infection. Retrovirology. 2010;7:30. doi: 10.1186/1742-4690-7-30. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Itoh Y, Toriumi H, Yamada S, Hoshino H, Suzuki N. Astrocytes and pericytes cooperatively maintain a capillary-like structure composed of endothelial cells on gel matrix. Brain Res. 2011;1406:74–83. doi: 10.1016/j.brainres.2011.06.039. [DOI] [PubMed] [Google Scholar]
  26. Joska JA, Hoare J, Stein DJ, Flisher AJ. The neurobiology of HIV dementia: implications for practice in South Africa. Afr. J. Psychiatry. 2011;14:17–22. doi: 10.4314/ajpsy.v14i1.65463. [DOI] [PubMed] [Google Scholar]
  27. Kay E, Dinn JJ, Farrell MA. Neuropathologic findings in AIDS and human immunodeficiency virus infection–report on 30 patients. Ir. J. Med. Sci. 1991;160:393–398. doi: 10.1007/BF02957798. [DOI] [PubMed] [Google Scholar]
  28. Kim WK, Avarez X, Williams K. The role of monocytes and perivascular macrophages in HIV and SIV neuropathogenesis: information from non-human primate models. Neurotox. Res. 2005;8:107–115. doi: 10.1007/BF03033823. [DOI] [PubMed] [Google Scholar]
  29. Klein T, Bischoff R. Physiology and pathophysiology of matrix metalloproteases. Amino Acids. 2011;41:271–290. doi: 10.1007/s00726-010-0689-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Kontogiorgis CA, Papaioannou P, Hadjipavlou-Litina DJ. Matrix metalloproteinase inhibitors: a review on pharmacophore mapping and (Q) SARs results. Curr. Med. Chem. 2005;12:339–355. doi: 10.2174/0929867053363243. [DOI] [PubMed] [Google Scholar]
  31. Lakhan SE, Kirchgessner A, Tepper D, Leonard A. Matrix metalloproteinases and blood-brain barrier disruption in acute ischemic stroke. Front. Neurol. 2013;4:32. doi: 10.3389/fneur.2013.00032. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Lambert E, Dasse E, Haye B, Petitfrere E. TIMPs as multifacial proteins. Crit. Rev. Oncol. Hematol. 2004;49:187–198. doi: 10.1016/j.critrevonc.2003.09.008. [DOI] [PubMed] [Google Scholar]
  33. Lawn SD, Labeta MO, Arias M, Acheampong JW, Griffin GE. Elevated serum concentrations of soluble CD14 in HIV- and HIV+ patients with tuberculosis in Africa: prolonged elevation during anti-tuberculosis treatment. Clin. Exp. Immunol. 2000;120:483–487. doi: 10.1046/j.1365-2249.2000.01246.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Lindl KA, Marks DR, Kolson DL, Jordan-Sciutto KL. HIV-associated neurocognitive disorder: pathogenesis and therapeutic opportunities. J. Neuroimmune Pharmacol. 2010;5:294–309. doi: 10.1007/s11481-010-9205-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Louboutin JP, Strayer DS. Blood-brain barrier abnormalities caused by HIV-1 gp120: mechanistic and therapeutic implications. Sci. World J. 2012;2012:482575. doi: 10.1100/2012/482575. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Louboutin J-P, Agrawal Lokesh, Reyes Beverly A, Van Bockstaele EJ, Strayer DS. HIV-1 gp120-Induced Injury to the Blood-Brain Barrier: Role of Metalloproteinases 2 and 9 and Relationship to Oxidative Stress. J. Neuropathol. Experim. Neurol. 2010a;69:801–816. doi: 10.1097/NEN.0b013e3181e8c96f. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Louboutin JP, Reyes BA, Agrawal L, Maxwell CR, Van Bockstaele EJ, Strayer DS. Blood-brain barrier abnormalities caused by exposure to HIV-1 gp120–protection by gene delivery of antioxidant enzymes. Neurobiol. Dis. 2010b;38:313–325. doi: 10.1016/j.nbd.2010.02.007. [DOI] [PubMed] [Google Scholar]
  38. Louboutin JP, Reyes BA, Agrawal L, Van Bockstaele EJ, Strayer DS. HIV-1 gp120 upregulates matrix metalloproteinases and their inhibitors in a rat model of HIV encephalopathy. Eur. J. Neurosci. 2011;34:2015–2023. doi: 10.1111/j.1460-9568.2011.07908.x. [DOI] [PubMed] [Google Scholar]
  39. Mishra R, Singh SK. HIV-1 Tat C phosphorylates VE-cadherin complex and increases human brain microvascular endothelial cell permeability. BMC Neurosci. 2014;15 doi: 10.1186/1471-2202-15-80. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Moore CS, et al. Astrocytic tissue inhibitor of metalloproteinase-1 (TIMP-1) promotes oligodendrocyte differentiation and enhances CNS myelination. J. Neurosci. 2011;31:6247–6254. doi: 10.1523/JNEUROSCI.5474-10.2011. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Morgello S, Gelman BB, Kozlowski PB, Vinters HV, Masliah E, Cornford M, Cavert W, Marra C, Grant I, Singer EJ. The National NeuroAIDS Tissue Consortium: a new paradigm in brain banking with an emphasis on infectious disease. Neuropathol. Appl. Neurobiol. 2001;27:326–335. doi: 10.1046/j.0305-1846.2001.00334.x. [DOI] [PubMed] [Google Scholar]
  42. Mun-Bryce S, Rosenberg GA. Matrix metalloproteinases in cerebrovascular disease. J. Cereb. Blood Flow Metab. 1998;18:1163–1172. doi: 10.1097/00004647-199811000-00001. [DOI] [PubMed] [Google Scholar]
  43. Nagase H, Woessner JF., Jr Matrix metalloproteinases. J. Biol. Chem. 1999;274:21491–21494. doi: 10.1074/jbc.274.31.21491. [DOI] [PubMed] [Google Scholar]
  44. Nagase H, Visse R, Murphy G. Structure and function of matrix metalloproteinases and TIMPs. Cardiovasc. Res. 2006;69:562–573. doi: 10.1016/j.cardiores.2005.12.002. [DOI] [PubMed] [Google Scholar]
  45. Nelson CM, Bissell MJ. Of extracellular matrix, scaffolds, and signaling: tissue architecture regulates development, homeostasis, and cancer. Annu. Rev. Cell Dev. Biol. 2006;22:287–309. doi: 10.1146/annurev.cellbio.22.010305.104315. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Power C, et al. Cerebral white matter changes in acquired immunodeficiency syndrome dementia: alterations of the blood-brain barrier. Ann. Neurol. 1993a;34:339–350. doi: 10.1002/ana.410340307. [DOI] [PubMed] [Google Scholar]
  47. Power C, Kong PA, Crawford TO, Wesselingh S, Glass JD, McArthur JC, Trapp BD. Cerebral white matter changes in acquired immunodeficiency syndrome dementia: alterations of the blood-brain barrier. Ann. Neurol. 1993b;34:339–350. doi: 10.1002/ana.410340307. [DOI] [PubMed] [Google Scholar]
  48. Ra HJ, Parks WC. Control of matrix metalloproteinase catalytic activity. Matrix Biol. 2007;26:587–596. doi: 10.1016/j.matbio.2007.07.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Redzic Z. Molecular biology of the blood-brain and the blood-cerebrospinal fluid barriers: similarities and differences. Fluids Barriers CNS. 2011;8:3. doi: 10.1186/2045-8118-8-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Rosenberg GA, Estrada EY, Dencoff JE. Matrix metalloproteinases and TIMPs are associated with blood-brain barrier opening after reperfusion in rat brain. Stroke. 1998;29:2189–2195. doi: 10.1161/01.str.29.10.2189. [DOI] [PubMed] [Google Scholar]
  51. Sandler NG, Wand H, Roque A, Law M, Nason MC, Nixon DE, Pedersen C, Ruxrungtham K, Lewin SR, Emery S, Neaton JD, Brenchley JM, Deeks SG, Sereti I, Douek DC, Group, I.S.S. Plasma levels of soluble CD14 independently predict mortality in HIV infection. J. Infect. Dis. 2011;203:780–790. doi: 10.1093/infdis/jiq118. [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Seiki M. Membrane-type 1 matrix metalloproteinase: a key enzyme for tumor invasion. Cancer Lett. 2003;194:1–11. doi: 10.1016/s0304-3835(02)00699-7. [DOI] [PubMed] [Google Scholar]
  53. Shawahna R. Physical and metabolic integrity of the blood-brain barrier in hiv infection: a special focus on intercellular junctions, influx and efflux transporters and metabolizing enzymes. Curr. Drug Metab. 2015;16:105–123. doi: 10.2174/138920021602150713114715. [DOI] [PubMed] [Google Scholar]
  54. Spindler KR, Hsu TH. Viral disruption of the blood-brain barrier. Trends Microbiol. 2012;20:282–290. doi: 10.1016/j.tim.2012.03.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  55. Strazza M, Pirrone V, Wigdahl B, Nonnemacher MR. Breaking down the barrier: the effects of HIV-1 on the blood-brain barrier. Brain Res. 2011;1399:96–115. doi: 10.1016/j.brainres.2011.05.015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  56. Tan IL, McArthur JC. HIV-associated neurological disorders: a guide to pharmacotherapy. CNS drugs. 2012;26:123–134. doi: 10.2165/11597770-000000000-00000. [DOI] [PubMed] [Google Scholar]
  57. Toth M, Sohail A, Fridman R. Assessment of gelatinases (MMP-2 and MMP-9) by gelatin zymography. Methods Mol. Biol. 2012;878:121–135. doi: 10.1007/978-1-61779-854-2_8. [DOI] [PubMed] [Google Scholar]
  58. Trujillo JR, Jaramillo-Rangel G, Ortega-Martinez M, Penalva de Oliveira AC, Vidal JE, Bryant J, Gallo RC. International NeuroAIDS: prospects of HIV-1 associated neurological complications. Cell Res. 2005;15:962–969. doi: 10.1038/sj.cr.7290374. [DOI] [PubMed] [Google Scholar]
  59. Valcour V, Sithinamsuwan P, Letendre S, Ances B. Pathogenesis of HIV in the central nervous system. Curr. HIV/AIDS Rep. 2011;8:54–61. doi: 10.1007/s11904-010-0070-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  60. Vincenti MP. The matrix metalloproteinase (MMP) and tissue inhibitor of metalloproteinase (TIMP) genes. Transcriptional and posttranscriptional regulation, signal transduction and cell-type-specific expression. Methods Mol. Biol. 2001;151:121–148. doi: 10.1385/1-59259-046-2:121. [DOI] [PubMed] [Google Scholar]
  61. Webster NL, Crowe SM. Matrix metalloproteinases, their production by monocytes and macrophages and their potential role in HIV-related diseases. J. Leukoc. Biol. 2006;80:1052–1066. doi: 10.1189/jlb.0306152. [DOI] [PubMed] [Google Scholar]
  62. Westermarck J, Kahari VM. Regulation of matrix metalloproteinase expression in tumor invasion. FASEB J. 1999;13:781–792. [PubMed] [Google Scholar]
  63. Woods SP, Moore DJ, Weber E, Grant I. Cognitive neuropsychology of HIV-associated neurocognitive disorders. Neuropsychol. Rev. 2009;19:152–168. doi: 10.1007/s11065-009-9102-5. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplemental

RESOURCES