Skip to main content
Viruses logoLink to Viruses
. 2017 Oct 31;9(11):324. doi: 10.3390/v9110324

Clinical Relevance of Total HIV DNA in Peripheral Blood Mononuclear Cell Compartments as a Biomarker of HIV-Associated Neurocognitive Disorders (HAND)

Vurayai Ruhanya 1,2,*, Graeme B Jacobs 1, Richard H Glashoff 3,4, Susan Engelbrecht 1,5,*
PMCID: PMC5707531  PMID: 29088095

Abstract

The pathogenesis of HIV-associated neurocognitive disorders is complex and multifactorial. It is hypothesized that the critical events initiating this condition occur outside the brain, particularly in the peripheral blood. Diagnoses of HIV-induced neurocognitive disorders largely rely on neuropsychometric assessments, which are not precise. Total HIV DNA in the peripheral blood mononuclear cells (PBMCs), quantified by PCR, correlate with disease progression, which is a promising biomarker to predict HAND. Numerous PCR assays for HIV DNA in cell compartments are prone to variation due to the lack of standardization and, therefore, their utility in predicting HAND produced different outcomes. This review evaluates the clinical relevance of total HIV DNA in circulating mononuclear cells using different published quantitative PCR (qPCR) protocols. The rationale is to shed light on the most appropriate assays and sample types used to accurately quantify HIV DNA load, which predicts severity of neurocognitive impairment. The role of monocytes as a vehicle for trafficking HIV into the CNS makes it the most suitable sample for determining a HAND associated reservoir. Studies have also shown significant associations between monocyte HIV DNA levels with markers of neurodamage. However, qPCR assays using PBMCs are cheaper and available commercially, thus could be beneficial in clinical settings. There is need, however, to standardise DNA extraction, normalisation and limit of detection.

Keywords: HAND, HIV DNA, qPCR, biomarker

1. Introduction and Brief Pathology of HAND

Highly active antiretroviral therapy (HAART) has reduced the prevalence of AIDS related opportunistic infections. Consequently, efforts are being directed at managing long-term HIV complications such as central nervous system (CNS) diseases, which have continued to affect patients, despite adequate virological control [1]. Some of the most frequent complications in patients with HIV-1 are HIV-associated neurocognitive disorders (HAND), which involve impairment or disruption of neurocognitive functioning [2]. It is estimated that 20–69% of all HIV-1 cases have neurocognitive disorders [3,4]. There is a wide range of disorders covered in the description of HAND according to the 2007 Frascati criteria of categorising severity of neurocognitive impairment [5]. Dementia associated with HIV, HAD, is the most severe form of HAND. Mild neurocognitive disorder (MND) is characterised by minor neurocognitive impairment and asymptomatic neurocognitive impairment (ANI) is characterised by two or more cognitive abilities with no functional impairment.

The precise mechanism of the neuropathogenesis of HAND is unknown, but studies have shown that HAND is likely determined by a complex interaction between viral factors, cellular targets and the immune response. Two models have been proposed to explain neurodegradation and the development of symptomatic HAND; namely, the direct and indirect pathways (Figure 1). Both initially require HIV-1 infection of the perivascular macrophages and microglia in the brain. According to the “Trojan horse” hypothesis, HIV-infected perivascular monocytes/macrophages traffic HIV from the peripheral blood to the CNS, leading to the infection of the resident microglial cells [6].

Figure 1.

Figure 1

Mechanism of neuropathogenesis. Two pathways involved shown by arrows: (1) Direct pathway caused by HIV and released HIV proteins. (2) Indirect pathway involving secretion cytokines. (a) Virus particles and viral proteins shed and cross Blood brain barrier (BBB). (b) Neural injury caused by direct viral infection and dysregulation by viral proteins. (c) Infected monocyte infiltrating BBB. (d) Release of cytokines from infected monocytes contributing to disruption of BBB. (e) BBB become more permeable to cytokines present in the periphery. (f) More cytokines released into the brain and (g) cytokines disrupt normal functioning ultimately leading to neuronal apoptosis resulting in different forms of HIV associated neurocognitive disorders (HAND).

The direct pathway proposes that the viral proteins (Env gp120, Tat and Vpr), released from HIV-infected monocyte-derived cells, can cause neuronal death by direct interaction with neurons. Among the direct effects of viral proteins on neurons is neuronal depolarization by direct excitation independent of synaptic interactions, N-methyl-D-aspartate (NMDA) receptor dysregulation and disruption of calcium homeostasis in the case of Tat [7,8]. The indirect pathway suggests that neuronal death is mediated by inflammatory cells targeting HIV proteins released by infected cells. Infected or activated macrophages release a variety of soluble cytokines and chemokines, which are pro-inflammatory, including IL-6, GM TNF-α-CSF and IL-1β, which eventually leads to neuronal apoptosis [9,10].

Since there is no standard protocol to assess the condition optimally, HAND is difficult to diagnose. The screening and diagnosis for HAND relies on multiple clinical and neuro-psychometric methods, such as the World Health Organisation (WHO) Auditory Verbal Learning Task (WHO-AVLT) and the Revised Brief Visual Memory Task (BVMT-R) for learning efficiency [11,12]. These methods have been used to define the scope and severity of HAND. However, there is need to compliment clinical and neuropsychometric assessments with HIV-specific biological markers underlying symptomatic HAND. The availability of reliable biomarkers would help to diagnose patients and bring a better understanding of neuropathogenesis and therapeutic strategies to HAND [13]. Although many proposed potential biomarkers for HAND have been described, there is a lack of sensitivity and some of them are not HIV specific [14]. They include markers of monocyte activation such as soluble CD163 and CD14 (sCD163 and sCD14) [15], cytokine and chemokine receptors as well as markers of neural injury and glial dysfunction [1,16,17,18]. They do not distinguish between the severity and HAND subtype. No longer closely associated with neuropsychometric performance are HIV specific markers such as plasma CD4+ T-cell count and viral load [19] in this era of HAART, where prevalence of HAND in aviremic patients is common. Peripheral blood mononuclear cells (PBMCs) containing HIV DNA has been linked to HAND progression in virally suppressed patients and therapy naïve patients [11,20]. This review summarises recent developments on the potential use of total peripheral blood mononuclear cell-associated HIV DNA as a marker of HAND, focusing on PCR quantification assays. The rationale is to highlight the potential clinical utility of this biomarker for HAND, considering the roles played by different mononuclear cell compartments (lymphocytes compared to monocytes) or unfractionated PBMCs in the proposed HAND pathogenesis models, bringing into context the suitability of different samples used in different PCR assays. A careful analysis of published PCR methods regarding sample type, sample loading quantity and quality, normalisation, robustness, accuracy, sensitivity and specificity is required before total peripheral HIV DNA is used as a biomarker for HAND in clinical settings.

2. Latency and Different Forms of HIV DNA in the Periphery

Immune cells are infected by HIV through CD4 receptors, and the co-receptors CCR5 and/or CXCR4. The primary target cell is the CD4+ T lymphocyte. Following cellular infection, viral genetic material (RNA) is then reverse transcribed into HIV DNA that becomes integrated into host DNA or might exist as non-integrated linear and circular forms (1-LTR and 2-LTR circles) [21]. A subset of these infected immune cells may become memory cells that harbour HIV DNA without the production of virus (i.e., latently infected) [22,23]. This latency state may be disrupted if the cells are re-stimulated/activated or when treatment is interrupted [21,23]. The HIV DNA is persistently stored in a stable form in lineages of memory T lymphocytes and is protected from biochemical decay [23]. Monocytes may be directly infected or may be infected following phagocytic uptake of infected lymphocytes and/or free virions. Infection may occur in the blood or at the bone marrow level [24]. Antiretroviral therapy (ART) reduces HIV RNA particles in circulating blood to undetectable levels, but it cannot eliminate HIV DNA infecting white blood cells [22]. Latently infected cells are disseminated throughout the body, but are differentially concentrated in the lymphoid tissues [25]. Blood is the most accessible tissue, but not the true representative medium for measuring the latent reservoir [26]. Furthermore, the frequency of HIV infected cells is higher in the gut associated lymphoid tissue than in the blood [27,28,29]. Outlined earlier, the systemic HIV DNA reservoir trafficking to the CNS with activated cells (monocyte/macrophage lineages) causes activation-induced neurodamage [30,31].

The easy accessibility of peripheral blood as a clinical specimen makes it more feasible to investigate the contribution of peripheral HIV DNA reservoirs to pathogenesis of neurocognitive impairment [24,28]. Peripheral HIV DNA is an independent predictor of HIV disease irrespective of combined antiretroviral therapy and was widely used in both adult and paediatric cohorts [32,33,34,35]. The HIV DNA in peripheral blood has also been used as a prognostic marker of disease progression, especially in suppressed patients where it is the only biomarker of viral activity [35]. Therefore, it is possible to investigate the role of peripheral HIV DNA in neuropathogenesis by evaluating the association between the clinical categories of HAND (ANI, MND and HAD) and the quantities of HIV DNA. Available assays can quantify both total and unintegrated HIV DNA forms (linear, 1-LTR and 2-LTR) [36,37].

3. What Samples Can Be Used to Measure HIV DNA as a Biomarker for HAND?

Assays to quantify HIV DNA by qPCR have utilised both PBMCs and sorted cell subsets as clinical samples, but choices must be made on the most appropriate samples to collect and process [38]. The appropriate clinical specimen in HIV DNA qPCR should give a true size on the peripheral HIV-1 DNA pool and its composition [36], which enables clinical diagnosis and monitoring disease progression [39,40]. It is important to decide on the type of sample and form of DNA to use as a marker to define and monitor the progression of HAND, considering the size of the HIV reservoir in the sample and its role in the pathology of the disease. An appropriate clinical specimen gives information which reflects on the condition of the disease by correlating with findings from clinical assessments [41]. The choice of the sample should consider the stage of HIV disease as it is known that there are differences between early HIV disease and advanced stages of infection with regard to shift in tropism [42]. Advanced stages of HIV diseases shows depletion of CD4+ T lymphocytes, while there is both expansion and increased turnover of peripheral monocytes, including CD14+/CD16+ subsets known to be associated with increased susceptibility to HIV infection [43]. Therefore, investigations should consider the clinical stage of neurocognitive impairment/severity of HAND since there is likely to be differences in quantities of PBMC subsets and the composition of different forms of HIV DNA in the cells. Sample type is very critical because it has implications, first on the quantity and forms of DNA in the cell type and, second, the role played by the cell type in HAND pathogenesis. All three forms of HIV DNA have been detected in brain tissues of patients having AIDS dementia [44]. Some studies have measured HIV DNA quantities using the whole blood, while others have used PBMC and found good correlation between the two samples [34,36]. To consider peripheral HIV DNA load as a reliable virological marker of HAND in clinical settings, a careful assessment of PBMC and its cell subsets as appropriate samples is required.

3.1. Unfractionated PBMC

Peripheral blood mononuclear cells constitute the cellular part of the blood containing cells with a round nucleus, namely monocytes, T cells, B cells, Natural killer cells (NK) and dendritic cells [45]. Peripheral blood mononuclear cells (PBMCs) are involved in many immune-related diseases and there is growing interest to use them as surrogate markers of several diseases [46,47]. These PBMCs can be obtained relatively easily from routinely collected blood samples without analytical difficulties [48]. Thus, there is a growing interest to use PBMCs diagnostically as surrogates for direct sampling of sites of infection and to investigate other disease processes such as HIV-associated neurocognitive impairment [32,49]. It has been demonstrated that HIV DNA levels in PBMCs correlate with antiretroviral therapy efficacy, suggesting that DNA quantitation is a useful tool to monitor the decay of the HIV reservoir, especially when plasma viremia is undetectable [50].

Significant positive correlations between levels of HIV DNA measured in PBMC and CSF cell pellets has been reported in HAND patients, showing that there was a link between peripheral blood HIV reservoirs which influence CNS HIV DNA quantities [51]. Measurement of HIV DNA in unfractionated PBMCs using PCR is the simplest way of quantifying the reservoir in the peripheral blood [52]. The PBMC HIV DNA levels have been linked to cortical and subcortical grey matter atrophy in patients on HAART with undetectable viral load, and it was also observed that HIV DNA in PBMCs in HAD was 20 times higher than in uninfected individuals [53,54]. Since brain biopsy and CSF are not a clinically feasible option in HAND studies, utility of PBMCs is imperative. The quantity of HIV DNA in unfractionated PBMCs represents the true size of the peripheral HIV reservoir, in fact, because HIV DNA differentially distributed in all cell subsets are considered in PCR assays [31]. However, some studies on the association between HIV DNA in unfractionated PBMC and HAND did not show significant correlation [53]. Therefore, there is need to analyse the role of separate peripheral mononuclear cell subsets (Lymphocytes/monocytes) in neuropathogenesis.

3.2. Lymphocytes

The primary target for HIV infection in the peripheral blood are CD4 T lymphocytes, which makes them the predominant cell type harbouring HIV-1 in the peripheral blood of infected individuals [55,56,57]. The majority of HIV DNA is contained within memory (CD57−CD4+) T cells [57]. Research has also found that all forms of HIV DNA persist in resting CD4 cells, suggesting that there is replenishment of the reservoir, either by reactivation or by infection [21,57]. It has also been demonstrated, both in vitro and in vivo using animal models, that cytotoxic T lymphocytes can be infected by HIV-1 (Stanley et al., 1993). It is thought that the infection takes place during the peripheral interaction between CD4 and CD8 as part of an immune response, thereby allowing direct transmission of HIV-1 to the CD8 lymphocytes [27,56]. Using CD4 and CD8 naive and memory cell populations separated from PBMCs of HIV-seropositive individuals, frequencies of infected cells ranged from 30 to 670 proviral copies/106 CD4+ lymphocytes and from 8 to 500/106 CD8 lymphocytes [56]. The reservoir of HIV in CD4+ and CD8 lymphocytes is of clinical significance due to their important role in HIV pathogenesis, as well as their routine application in diagnostic settings as markers of HIV infection. Using qPCR targeting the HIV gag region, it was shown that memory CD57−CD4+ T cells were always more frequently infected (10–10,000 copies of gag DNA/105 cells) than effector CD57+CD4+ T cells [58]. The quantity of infected memory cells in this study also correlated with plasma viral load, which is a virological marker for monitoring and predicting HIV infection. However, with regard to HAND, there are limited data on correlation studies linking peripheral lymphocyte HIV DNA levels and severity of neurocognitive impairment [42,51]. Investigations using ART naïve participants with mild to moderate neurocognitive impairment, lymphocyte HIV DNA levels were shown to be significantly associated with HAND independent of plasma viral load and CD4+ count [42,54]. There is need for more data on association studies between peripheral lymphocyte HIV DNA reservoirs and HAND, since this is the preferential target of HIV infection [57]. Activated CD4+ T lymphocytes support viral replication and resting cells support establishment of a latent reservoir [58,59]. The result of HIV replication in activated lymphocytes is the production of different forms of HIV DNA (linear, 1-LTR and 2-LTR) and resting cells predominantly constitute integrated HIV DNA, both of which have shown to have clinical consequences in the brain HAND patients [60,61]. Therefore, this cell subset can represent the true size of the reservoir with different forms of HIV DNA.

3.3. Monocytes

There are two major types of monocytes in the blood, classical monocytes that have a high expression of CD14+ cell surface receptor (CD14++ monocytes) and non-classical, pro-inflammatory monocytes with a low CD14+ marker, but with an additional CD16+ marker (CD14+CD16+ monocytes) [62]. However, it has been reported that there is heterogeneity in CD14+CD16+ monocytes which resulted in subdivision of the pro-inflammatory monocytes into intermediate (CD14++CD16+) and non-classical (CD14+CD16++) monocytes [63]. Analysis by cellular subsets revealed that differences in cognitive categories are related to the amount of HIV DNA within inflammatory monocytes (CD14+CD16+) as compared to non-inflammatory monocytes (CD14+CD16−) or the CD14− fraction, which includes the lymphocytes [53,64]. Using a series of immunohistochemical markers, studies have demonstrated that CD14+CD16+ monocytes derived from the periphery accumulated in the perivascular region of the brain in patients with dementia [65]. The infection of monocytes by HIV is thought to arise from virus released from re-activated infected CD4+ T cells in the blood or infection at the bone marrow level [27,62,66].

The CD14+CD16+ monocyte subset in the peripheral blood constitutes only 5–10% of peripheral blood monocytes in healthy individuals, but the percentage increases to approximately 40% in HIV infected people [67]. Both in vitro and in vivo studies demonstrated that CD14+CD16+ monocytes express high amounts of the co-receptor CCR5, which facilitates their infection. It has also been observed that cells of the monocyte lineage are not susceptible to the cytopathic effects of HIV and, once they are infected in the periphery, they produce infectious virions as they enter the CNS and when they differentiate into macrophages [27]. Monocytes circulate in the blood for up to three days before they enter tissues [27,62,66,67]. Since monocytes are only present for 1–3 days in the blood, the presence of viral DNA within these cells indicate recent infection [68]. Monocytes isolated from blood of HIV patients were shown to contain unintegrated circular viral DNA, suggesting transcriptionally active rather than latent infection [69]. However, it has been shown that blood purified monocytes contain HIV DNA over time in both treated and untreated patients [7,27,70,71].

Monocytes circulate freely and patrol blood vessels as they respond to inflammation signals in tissues, such as exposure to viral products such as gp120, which can trigger chemokine production [72]. Upregulation of chemokines lead to movement of monocytes into various tissues and this process is thought to drive trafficking of monocytes through the blood–brain barrier. Studies also observed higher levels of HIV-1 DNA in activated inflammatory monocytes (CD14+/CD16+) than in non-activated monocytes [73]. Accumulating evidence shows that the route of CNS infection appears to involve circulating activated monocytes (CAM) [73]. Increase in peripheral activated monocyte subsets correlates with HIV disease progression and severity of neurocognitive impairment [62,74]. Research has demonstrated that individuals with HAD had higher levels of circulating activated CD14 monocytes and monocytes that co-express CD69, as compared to those without HAD. The finding supports the hypothesis that activated monocytes enter the brain and subsequently trigger neurotoxin production [75]. Individuals with HAD have a higher percentage of circulating activated monocytes/macrophages (M/MΦ), suggesting that HIV-1 DNA harboured in these reservoir cells is transported into the CNS by the same cells [73,76]. This evidence supports that accumulation of activated macrophages/microglia resulting from systemic immune activation trigger neurodegeneration in HAND [72,77]. It is clinically not feasible to sample brain tissue to determine macrophage/microglial viral DNA levels in clinical studies and routine diagnostics. Therefore, blood is the best option. Investigations using some animal models have shown that infected monocyte transmigration into the CNS appear to trigger gene expression of quiescent cells [61]. This finding raises the possibility that latent HIV harboured in peripheral monocytes may have clinical consequences upon BBB transmigration [60]. Viral DNA in blood monocytes in one study was significantly associated with eight of the nine cognitive domains that are frequently impaired in HAND [78,79]. Using magnetic resonance spectroscopy, it was demonstrated that neuronal injury and glial dysfunction was positively linked to peripheral CD14+ HIV DNA [80,81].

4. Quantitative PCR Assays to Detect Total HIV-1 DNA

There are a number of real-time PCR based assays to measure total HIV-1 DNA or cell associated DNA. These assays differ in extraction methods, PCR cycling conditions and amplification targets [82]. Some assays lack the sensitivity to quantify all the HIV-1 subtypes in the major (M) group, due to the high degree of genetic variability of the virus [83]. The genetic differences may result in DNA copy number variations due to probe and primer mismatches. The differences in PCR-based protocols for quantifying cell-associated HIV DNA has made it difficult to compare studies. Some of the confounding factors that result in non-specific variation in proviral load include input template quantity and quality, yields of the extraction process and enzymatic reactions [84]. Therefore, it is very important to prescribe the minimum requirements for a PCR-based assay to determine HIV proviral load. A requirement for quality PCR is that the extraction method produce high purity DNA that is free from haemoglobin, alcohol or high salt concentrations that may interfere with PCR amplification. Another critical requirement for determining HIV DNA proviral load in clinical samples is the assessment of the number of human cells per sample before DNA extraction [85]. The number of cells determined by flow cytometry is used to normalise HIV DNA in a sample [86]. Normalisers are used to ensure that target quantities from an equivalent amount of samples are comparable in absolute quantification, [73,87]. Data normalisation, which can be done by including an endogenous gene (reference gene) in the assay, is one of the essential key elements on the Minimum Information for Publication of Quantitative Real-Time PCR Experiments (MIQE) guidelines [88].

Normalisation is done to reduce non-biological variation as much as possible so that a true biological variation, explaining the phenomenon under investigation, is determined [84,89]. Housekeeping genes, such as Glyceraldehyde phosphate (GAPDH), β-globin, β-actin and Telomerase Reverse Transcriptase (TERT), determined by qPCR, were used for normalisation of HIV DNA to cell equivalents in some studies [28,31,73,90]. These PCR methods are more accurate than cell counts from cell sorting. The normaliser can also be used for quality control purposes since the numbers of cell inputted affect the sensitivity, as well indicating the presence of PCR inhibitors that might alter the sensitivity of the assay [91].

Malnati et al. in 2008, developed a universal HIV-1 group M-specific qPCR with a similar degree of accuracy and sensitivity across all viral subtypes. The designed primers and probe allow correct quantification of virtually all circulating HIV-1 strains. The universal assay amplifies part of the LTR-gag region that is highly conserved among all circulating group M HIV-1 subtypes [92]. The success of this group M-specific HIV-1 qPCR is based on the optimised design of primers and probes that ensure quantification of all the strains in the group. To obtain a standard for construction of a calibration curve, a fragment containing the conserved LTR-gag region was amplified from the T lymhoblastoid cell line using the optimised primers as described by Malnati. The fragment was then cloned into pCRII plasmid, from which serial dilutions were made to make standard curves. An additional qPCR was developed, in this assay, to measure the number of human cells initially present in a sample and to normalise the HIV-1 proviral DNA. This was done by measuring the copy number of a single copy human gene CCR5. The CCR5 assay permits the assessment of the sample quality as well as the number of cells. This protocol performs well if the input sample contains between 1000 and 150,000 cells per reaction. This robust method can quantify HIV-1 DNA from a crude lysate with the same degree of accuracy as with purified DNA. The advantage of a crude lysate is that it circumvents the nucleic acid extraction process, makes the assay take a shorter time and reduces the costs of the assay. The assay becomes suitable for large clinical trial cohorts.

Using cell lysate, Vandergeeten et al. [91] developed a nested real-time to measure total HIV DNA from diverse subtypes. The first round of PCR was pre-quantification duplex PCR with two sets of primers, one for the HIV-1 and the second set for the normaliser CD3 gene. The second round PCR was real qPCR performed separately for HIV quantification and for determination of cell input using the CD3 gene. The primers and probes for this assay, which target the conserved LTR-gag region, allows the precise quantification of total HIV DNA from subtypes, A, B, C, D, CRF01 A/E and CRF02_A/G, with the detection limit of three copies for all subtypes. The assay also showed that HIV DNA copies were detected with similar efficacy if the cell input ranges from 10,000 to 300,000. The assay clearly demonstrated the cell input sensitivity of HIV DNA quantification.

Recently, a simple and reproducible qPCR assay for cell-associated HIV-1 DNA was developed targeting the conserved 3’ part of the integrase region of the pol gene with a sensitivity of three to five copies per million cells [82]. The 3’ end of the HIV-1 pol gene has been shown to be the most highly conserved region, meaning that primers and probes targeting this region should amplify efficiently the largest numbers of samples from HIV-1 infected people without mismatches. The sensitivity of this assay was determined by testing the limiting dilution of DNA standards in accordance with the Poisson distribution. The assay was able to detect three copies of HIV DNA standards in 80–100% of the reactions. Low detection limit (LOD) of three genome copies shows that the assay is very sensitive. Inhibition of HIV DNA PCR by excess nucleic acid was prevented by the dilution of the extract to ≤170 ng/μL. The nucleic acid extract was also diluted by 1:30 for CCR5 qPCR. The control of the nucleic acid input for both HIV DNA and the CCR5 normaliser is important to avoid underestimation of proviral load through inhibition. Many other PCR based assays are used in monitoring the HIV reservoir in clinical trials and HAND studies, but they differ in many ways including target region, normaliser, sample type, sample input quantity, standards, limit of detection and cycling conditions [26,31,82]. Table 1 shows selected methods used in qPCR for HIV DNA showing different parameters, including the ANRS commercial assay, which has been evaluated for inter-laboratory reproducibility.

Table 1.

Selected technologies used in quantitative HIV DNA real time PCR assays.

Assay Normalisation Unique Features and Advantages/Disadvantages PCR Template Loading Quantity LOD Reference
Whole blood ANRS LTR real time PCR Method HIV-DNA copies/µg converted to copies/million leucocytes HIV DNA extracted from re-suspended whole blood cell pellets
No separation of cells
Its quicker and cost serving
Normalisation using cell count/cytometry is not accurate
1 µg DNA, equivalent to 150,000 cells 1 Copy [34]
LTR ANRS LTR real time PCR Method Copies/million/PBMC First assay for estimating HIV reservoir size to be evaluated for inter-laboratory reproducibility. The assay available as a commercial 1 µg DNA, equivalent to 150,000 cells 1 Copy [34]
SYBR Green gag HIV DNA PCR Copies/hundred thousand cells Uses SYBR green fluorescence and amplifies a 142-gag fragment
The assay is not sensitive especially in suppressed patients
300 ng, equivalent to 1,000,000 cells 50 Copies [27]
Cross-Clade Ultrasensitive nested PCR CD3 gene copy cell equivalents Assay is performed on cell lysate, hence circumvents laborious nucleic acid extraction
Allows quantification of HIV DNA A, B, C, D, CRF01_A/E, and CRF02_A/G HIV-1 subtypes, targeting highly conserved LTR-gag region.
Cell input equivalence is accurately quantified by CD3 gene quantitative PCR.
It has a disadvantage of 2 rounds of PCR
100,000 Cells 3 Copies [91]
universal real-time PCR for group-M HIV-1 DNA CCR5 gene copy cell equivalence HIV-1 M-specific quantitative measures the HIV-1
Proviral DNA load in group M with a similar degree of sensitivity and accuracy across subtypes.
Uses cell lysate as a template
Predilution required if loading quantity is above 150,000 cells
1 µg per reaction, equivalent to 150,000 cells 1 Copy [92]
Novel Assay for Total cell associated HIV-1 DNA CCR5 gene copy cell equivalence Sensitive Quantitative real time PCR for HIV-1 Cell associated DNA (CAD) targeting 3’ region of the pol gene.
Assay involves enhanced nucleic extraction by ensuring adequate cell lysis through ultrasonic cell disruption
1.7 µg 3 Copies [82]

5. Relevance of Total HIV DNA qPCR Assays to HAND

Many studies have evaluated and demonstrated the prognostic value of total HIV DNA measurement by qPCR as a marker of disease progression, independent of plasma viral load and CD4+ count [27,36,93,94]. High levels of HIV DNA were associated with rapid progression to AIDS and death in treatment naïve patients [41]. A suitable HIV DNA qPCR assay for HAND should target a clinically relevant marker and should be highly specific, sensitive, reproducible, cost-effective and easy to implement in clinical settings. Both replication-competent and defective forms of HIV DNA have been shown to be clinically relevant to neurocognitive impairment [44,60,61]. It is well understood that the different forms of HIV DNA co-exist in an infected cell, although their distribution varies depending on whether the cells are activated or latent [36,57,93,95]. Therefore, it is important to quantify all forms of HIV DNA, including the defective and unintegrated forms, and utilise them as surrogate biomarkers for HAND pathogenesis. It is also cost effective to have a qPCR that quantifies all the different forms of HIV DNA rather than separately quantifying integrated forms, 1-LTR or 2-LTR. Total HIV DNA qPCR assays can be applied to whole blood, PBMCs, monocytes or lymphocyte subsets and is easy to measure [31,34,42,96]. The assay quantifies all forms of HIV DNA that co-exist in infected cells that are involved in HIV pathogenesis [31,57]. Comparing measures such as 2LTR circles or integrated HIV DNA, total HIV DNA has the advantage of easy quantification, by standardised sensitive, real-time PCR methods [34]. It is suitable for analysing large samples with accuracy, requires a relatively small amount of blood and is not affected by freezing–thawing [96]. However, there is no commercial assay currently available for HIV DNA qPCR [41]. The above-described assays, including the ones that have been used on HAND studies, are a modification of commercial assays used in HIV RNA quantification [82,97]. Thus, an assessment of qPCR applied in HAND studies is required with regard to sensitivity and the sample types that were used in the assays.

6. HIV-1 DNA qPCR Used in HAND Studies

Application of qPCR and digital PCR to determine the association between HIV DNA levels in PBMC and its fractions, in the context of HAND, needs to consider both clinical and demographic variables of the study cohorts as well as the severity of HAND and performance characteristics of the assays. Using a very sensitive qPCR assay (LOD of 1–3 copies), it has been demonstrated that HIV DNA correlated with HAD in both HAART-naïve and HAART treated patients [73,80,81]. These studies also showed that activated monocytes HIV DNA levels in HAD patients were higher than in lymphocytes including the CD4+ cells. The sensitivity of the assay allows detection of low copies in suppressed patients and those who commence ART early in the course of HIV infection. Using a full range of neurocognitive disorders, the assay also showed that people with normal cognition had lower peripheral HIV DNA levels as compared to those with minor motor disorders and HAD [80]. Results of this study showed that the assay was sensitive enough to determine the association between circulating HIV DNA and different stages of HAND from normal cognition to the extreme forms of HAD. The HIV DNA determined by this assay, as a proposed biomarker, has been shown to be proportional to all three levels of neurocognitive function as well as different neuropsychological deficits, making it applicable for both screening and monitoring therapeutic interventions.

The HIV DNA assay targeting the gag region has shown that monocyte HIV DNA levels correlated with all three clinical categories of HAND (ANI, MND and HAD) in HAART treatment naïve patients [11]. The study further demonstrated association between monocyte HIV DNA and glial dysfunction, neuronal injury and CSF immune activation. The important aspect of this study demonstrates the link between both neurobehavioral and neuropathological aspects of HAND and circulating HIV DNA levels. However, there was no association between PBMC HIV DNA and HAND. Different HIV DNA PCR assays including droplet digital PCR used on HAND studies produced different results depending on clinical and demographic characteristics of study participants, as well as performance characteristics of the assays [42,51,80]. Table 2 shows selected PCR assays used in HAND studies with clinical data for study participants and performance characteristics of the assays.

Table 2.

HIV DNA real time PCR assays applied in HAND studies.

Reference Target Sample Clinical Data Study Outcome Advantage/Disadvantage
[73] gag PBMC HAD patients on HAART
20–39 years and 50 years and above
Quantities of HIV DNA correlated with HAD
HIV was higher in activated monocytes than CD14− in two patients
Very sensitive Quantitect Sybr Green PCR assay with detection limit of 1–3 copies and applicable suppressed patients and those with low cell PBMC or cell subset counts
[78] gag PBMC, CD14+, CD14− 15 HAART-naïve HAD patients and 15 Non demented patients
Average age, 34.1
HIV DNA in PBMC was significantly higher in HAD non demented patients
HIV DNA in five HAD patients was higher in CD14/CD16 than CD4
Very sensitive Quantitect Sybr Green PCR assay with detection limit of 1–3 copies and can be applied to suppressed patients and samples with low cell count
[31] pol PBMC HIV Subtype B chronic patients on cART for six months
CD4+ < 350/mL
Median Age 56, 74
HIV DNA was associated with HAD and not
Non demented forms of HAND
Very sensitive assay with one copy as LOD, hence suitable for supressed HAND patients
PCR has a wide dynamic range from three copies to 300,000 copies
High efficiencies for both
HIV DNA and β-actin normaliser
(93.7% and 86.6% respectively)
-Assay restricted to detecting Subtype B
[42] LTR-gag CD14− and CD14+ ART naïve participants (median age, 32) with 17 impaired and 19 unimpaired
Patients had higher HIV DNA in lymphocytes than monocytes
Strong association between HIV DNA levels in lymphocytes and HAND The target region for amplification is highly conserved and the Universal PCR assay has been customised for subtype G and CRF02_AG.
However, performance of the assays not detailed
[79] gag CD14+ 12 HAART treated HAD and 15 non HAD Infected with HIV-1 CRF01_AE
Median Age, 32
Baseline monocyte HIV DNA correlated with HIV DNA and 48 weeks after HAART.
Monocyte HIV DNA level was below detection limit in all non-HAD patients after 48 weeks
Detection limit of the assay was 10 copies/106 cells which is relatively high.
[11] gag CD14+PBMC Treatment naïve with three clinical categories of HAND (ANI, MND, HAD)
CD4+ <350/mL
–CRF AE_01Mean Age, 35
No correlation between PBMC HIV DNA
Before CD14+ enrichment, CD14+ was associated with HAND(ANI, MND, HAD)
–CD14+ HIV DNA was associated with glial dysfunction, neuronal injury and CSF immune activation (neopterin)
Very sensitive assay with a limit of detection of one copy which is suitable for HAND suppressed patients
[51] pol PBMC 22–40 years and 50–71 years ART-supressed HIV subjects Higher HIV DNA levels were associated with more severe neurocognitive impairment in older patients
No association between HIV DNA and HAND in young adults.
Very sensitive droplet digital PCR with Limit of Detection (LOD) of one copy and more precise than qPCR due insensitivity to primer and probe mismatches
absolute quantification, no external standards required

7. Possible Use of Digital PCR to Detect HIV DNA in HAND

The amplification signal is logarithmic and the quantification is based on an external calibration or standard curve in the above described qPCR methods. A relatively newer technology called digital PCR (dPCR) offers simple, linear and digital quantification that is based on the number of positive and negative reactions using Poisson distribution [98]. Digital PCR uses limiting dilutions and sample partitioning into submicrolitre reactions. Each partition represents an isolated end point PCR [99]. The Poisson distribution of template molecule within these partitions is used to deduce the concentration of the target nucleic acid from frequency of negative to partitions. Digital PCR has been shown to be tolerant with sequence variations by higher robustness to primers and probe mismatches with target sequences, giving a superior advantage for HIV-1 DNA quantification and, as a result, dPCR has shown higher accuracy precision and reproducibility over qPCR.

Regarding detection and quantification of HIV proviral DNA in HAART suppressed patients, which are usually very low but relevant to HAND, dPCR can offer an alternative to qPCR based assays [51]. Most patients who are on long term ART, the proviral load ranges from 10 to 1000 copies of HIV DNA per million cells [25,99]. The higher sensitivity of dPCR compared to qPCR will be more relevant to the determination of low quantities of HIV DNA in monocytes in the periphery, which have been shown to be predictive of HAND and thought to be a major source of virus production in the brain [67]. Although dPCR has a great potential to be used as a valid alternative to qPCR, there is need to eliminate false positive droplets, which have been reported in negative template controls [100]. The other hurdle in the utility of dPCR for precise measurement of HIV DNA is the lack of a standardised automated method for determining threshold droplets. Threshold setting is very important to determine baseline fluorescence in order to distinguish between positive and negative droplet populations without bias [101]. There is also a need to work on the reduction of cost so that dPCR can be clinically accredited.

8. Conclusions

Classical virological and immunological parameters such as plasma viral load and CD4+ T-cell counts for HIV normally used to monitor the progression of disease cannot be used to measure and predict the onset of HAND. Most of the proposed markers of HAND are similar to the biomarkers of other neurocognitive disorders due to the similarities in the molecular mechanisms that are involved. The HIV DNA or cell-associated DNA (CAD) is detected in the peripheral blood of HAART suppressed patients and many studies have shown it to correlate with HAND. However, there is a need for an assay that is robust, sensitive and accurately quantifies HIV DNA. Total HIV DNA quantification by real-time PCR from unfractionated PBMCs is very simple, but HIV DNA from inflammatory CD16+ monocytes is linked to the pathogenesis of HAND and, therefore, more suitable as a marker. Regarding quantification of low level HIV DNA in HAART suppressed patients, dPCR offers a superior advantage, but issues to do with standardization of threshold and cost reduction need improvement before utilisation in diagnostic settings. Therefore, qPCR methods remain relevant now for clinical use since research has shown significant association between peripheral HIV DNA levels in cell compartments and severity of HAND. Additionally, evaluated commercial qPCR kits for HIV DNA load are now available. However, more data are required on T cell HIV DNA as an indicator of potential HAND-associated reservoirs in untreated patients.

Acknowledgments

Our work on investigating viral, cellular and immunological factors contributing to the development of HIV associated neurocognitive disorders is supported by the Poliomyelitis Research Foundation (PRF), National Research Foundation (NRF) and the South African Medical Research Council (SAMRC) Collaborating Centre for HIV Laboratory Research (TygHIVLab).

Author Contributions

V.R.: Conceptualization, outline, and preparation of the draft manuscript. G.B.J.: Performed critical reading and editing of the manuscript. R.H.G.: Contributed much of the content in Section 4 of the draft manuscript, critical reading and editing of the manuscript. S.E.: Manuscript preparation, preparation of figures and final assembly of the manuscript.

Conflicts of Interest

The authors declare no conflict of interest.

References

  • 1.Elbirt D., Mahlab-Guri K., Bezel-Rosenberg S., Gill H., Attali M., Asher L. HIV-associated neurocognitive disorders. IMAJI. 2015;17:54–58. [PubMed] [Google Scholar]
  • 2.Sanmarti M., Ibáñez L., Huertas S., Badenes D., Dalmau D., Slevin M., Krupinski J., Popa-Wagner A., Jaen A. HIV-associated neurocognitive disorders. JMP. 2014;2:1–10. doi: 10.1186/2049-9256-2-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Simioni S., Cavassini M., Annoni J.M., Rimbault Abraham A., Bourquin I., Schiffer V., Calmy A., Chave J.P., Giacobini E., Hirschel B., et al. Cognitive dysfunction in HIV patients despite long-standing suppression of viremia. AIDS. 2010;24:1243–1250. doi: 10.1097/QAD.0b013e3283354a7b. [DOI] [PubMed] [Google Scholar]
  • 4.Rao V.R., Ruiz A.P., Prasad V.R. Viral and cellular factors underlying neuropathogenesis in HIV associated neurocognitive disorders (HAND) AIDS Res. Ther. 2014;19:11–13. doi: 10.1186/1742-6405-11-13. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Antinori A., Arendt G., Becker J.T., Byrd D.A., Cherner M., Clifford D.B., Cinque P., Epstein L.G., Goodkin K., Gisslen M., et al. Updated research nosology for HIV-associated neurocognitive disorders. Neurology. 2007;69:1789–1799. doi: 10.1212/01.WNL.0000287431.88658.8b. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Hong S., Banks W.A. Role of the immune system in HIV-associated neuroinflammation and neurocognitive implications. Brain Behav. Immun. 2015;45:1–12. doi: 10.1016/j.bbi.2014.10.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Zhou L., Saksena N.K. HIV Associated Neurocognitive Disorders. Infect. Dis. Rep. 2013;5:s1e8. doi: 10.4081/idr.2013.s1.e8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Kovalevich J., Langford D. Neuronal toxicity in HIV CNS disease. Future Virol. 2012;7:687–698. doi: 10.2217/fvl.12.57. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Appay V., Sauce D. Immune activation and inflammation in HIV-1 infection: Causes and consequences. J. Pathol. 2008;214:231–241. doi: 10.1002/path.2276. [DOI] [PubMed] [Google Scholar]
  • 10.Kamat A., Lyons J.L., Misra V., Uno H., Morgello S., Singer E.J., Gabuzda D. Monocyte activation markers in cerebrospinal fluid associated with impaired neurocognitive testing in advanced HIV infection. J. Acquir. Immune Defic. Syndr. 2012;60:234–243. doi: 10.1097/QAI.0b013e318256f3bc. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Valcour V.G., Ananworanich J., Agsalda M., Sailasuta N., Chalermchai T., Schuetz A., Shikuma C., Liang C.Y., Jirajariyavej S., Sithinamsuwan P., et al. HIV DNA Reservoir Increases Risk for Cognitive Disorders in cART-Naïve Patients. PLoS ONE. 2013;8:e70164. doi: 10.1371/journal.pone.0070164. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Levine A.J., Reynolds S., Cox C., Miller E.N., Sinsheimer J.S., Becker J.T., Martin E., Sacktor N. Neuropsychology Working Group of the Multicenter AIDS Cohort Study. The longitudinal and interactive effects of HIV status, stimulant use, and host genotype upon neurocognitive functioning. J. Neurovirol. 2014;3:243–257. doi: 10.1007/s13365-014-0241-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Colon K., Perez-Laspiur J., Quiles R., Rodriguez Y., Wajna V., Shaffer S.A., Leszyk J., Slolasky R.L., Jr., Melendel L.M. Macrophage secretome from women with HIV-associated neurocognitive disorders. Proteom. Clin. 2016;10:136–143. doi: 10.1002/prca.201400203. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Carroll A., Brew B. HIV-associated neurocognitive disorders: Recent advances in pathogenesis, biomarkers, and treatment. F1000Research. 2017;6:312. doi: 10.12688/f1000research.10651.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.McGuire J.L., Gill A.L., Douglas S.D., Kolson D., CHARTER Group Central and peripheral markers of peripheral neurodegradation and monocyte activation in HIV-associated neurocognitive disorders. J. Neurovirol. 2015;21:439–448. doi: 10.1007/s13365-015-0333-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Brew B.J., Bhalla R.B., Paul M., Gallardo H., McArthur J.C., Schwartz M.K., Price R.W. Cerebrospinal fluid neopterin in human immunodeficiency virus type 1 infection. Ann. Neurol. 1990;28:556–560. doi: 10.1002/ana.410280413. [DOI] [PubMed] [Google Scholar]
  • 17.Burdo T.H., Soulas C., Orzechowski K., Button J., Krishnan A., Sugimoto C., Alvarez X., Kuroda M.J., Williams K.C. Increased monocyte turnover from bone marrow correlates with severity of SIV encephalitis and CD163 levels in plasma. PLoS Pathog. 2010;6:e1000842. doi: 10.1371/journal.ppat.1000842. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Peluso M.J., Meyerhoff D.J., Price R.W., Peterson J., Lee E., Young A.C., Walter R., Fuchs D., Brew B.J., Cinque P., et al. Cerebrospinal fluid and neuroimaging biomarker abnormalities suggest early neurological injury in a subset of individuals during primary HIV infection. J. Infect. Dis. 2013;207:1703–1712. doi: 10.1093/infdis/jit088. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Becker J.T., Kingsley L., Mullen J., Cohen B., Martin E., Miller E.N., Ragin A., Sacktor N., Selnes O.A., Visscher B.R. Multicenter AIDS Cohort Study. Vascular risk factors, HIV serostatus, and cognitive dysfunction in gay and bisexual men. Neurology. 2009;73:1292–1299. doi: 10.1212/WNL.0b013e3181bd10e7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Pierson T., McArthur J., Siliciano R.F. Reservoirs for HIV-1: Mechanisms for viral persistence in the presence of antiviral immune responses and antiretroviral therapy. Annu. Rev. Immunol. 2000;18:665–708. doi: 10.1146/annurev.immunol.18.1.665. [DOI] [PubMed] [Google Scholar]
  • 21.Olivares O., Pernas M., Casado C., Lopez-Galindez C. Human Immunodeficiency Virus Type 1 two-long terminal repeat circles: A subject of debate. AIDS Rev. 2016;18:23–31. [PubMed] [Google Scholar]
  • 22.Koelsch K.K., Liu L., Haubrich R., May S., Havlir D., Gunthard H.F., Ignacio C.C., Campos-Soto P., Little S.J., Shafer R., Robbins G.K., D’Aquila R.T., et al. Dynamics of total, linear nonintegrated, and integrated HIV-1 DNA in vivo and in vitro. J. Infect. Dis. 2008;197:411–419. doi: 10.1086/525283. [DOI] [PubMed] [Google Scholar]
  • 23.Van Lint C., Bouchat S., Marcello A. HIV-1 transcription and latency: An update. Retrovirology. 2013;10:67. doi: 10.1186/1742-4690-10-67. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Valcour V.G., Shiramizu B.T., Cecilia M., Shikuma C.M. HIV DNA in circulating monocytes as a mechanism to dementia and other HIV complications. J. Leukoc. Biol. 2010;87:621–626. doi: 10.1189/jlb.0809571. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Cockerham L.R., Deeks S.G. Biomarker reveals HIV’s hidden reservoir. eLife. 2014;3:e04742. doi: 10.7554/eLife.04742. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Re M.C., Vitone F., Biagetti C., Schiavone P., Alessandrini F., Bon I., de Crignis E., Gibellini D. HIV-1 DNA proviral load in treated and untreated HIV-1 seropositive patients. Clin. Microbiol. Infect. 2010;6:640–646. doi: 10.1111/j.1469-0691.2009.02826.x. [DOI] [PubMed] [Google Scholar]
  • 27.Gibellini D., Borderi M., De Crignis E., Cicola R., Cimatti L., Vitone F., Chiodo F., Re M.C. HIV-1 DNA in peripheral blood monocytes and lymphocytes from naïve and HAART-treated indviduals. J. Infect. 2008;56:219–225. doi: 10.1016/j.jinf.2008.01.001. [DOI] [PubMed] [Google Scholar]
  • 28.Yukl S.A., Shergill A.K., Ho T., Killian M., Girling V., Epling L., Li P., Wong L.K., Crouch P., Deeks S.G., et al. Distribution of HIV DNA and RNA in cell subsets differs in gut and blood of HIV-positive patients on ART: Implications for viral persistence. J. Infect. Dis. 2013;208:1212–1220. doi: 10.1093/infdis/jit308. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Chun T.W., Nickle D.C., Justement J.S., Meyers J.H., Roby G., Hallahan C.W., Kottilil S., Moir S., Mican J.M., Mullins J.I., et al. Persistence of HIV in gut associated lymphoid tissue despite long-term antiviral therapy. J. Infect. Dis. 2008;197:714–720. doi: 10.1086/527324. [DOI] [PubMed] [Google Scholar]
  • 30.Rouzioux C., Richman D. How to best measure HIV reservoirs. Curr. Opin. HIV AIDS. 2013;8:170–175. doi: 10.1097/COH.0b013e32835fc619. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Cysique L.A., Hey-Cunningham W.J., Dermody N., Chan P., Brew B.J., Koelsch K.K. Peripheral Blood Mononuclear Cells HIV DNA Levels Impact Intermittently on Neurocognition. PLoS ONE. 2015;10:e0120488. doi: 10.1371/journal.pone.0120488. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Zhou L., Conceicao V., Gupta P., Saksena N.K. Why are the neurodegenerative disease-related pathways overrepresented in primary HIV-infected peripheral blood mononuclear cells: A genome-wide perspective. Virol. J. 2012;9:308. doi: 10.1186/1743-422X-9-308. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Casabianca A., Gori C., Orlandi C., Forbici F., Federico Perno C., Magnani M. Fast and sensitive quantitative detection of HIV DNA in whole blood leucocytes by SYBR green I real-time PCR assay. Mol. Cell. Probes. 2007;21:368–378. doi: 10.1016/j.mcp.2007.05.005. [DOI] [PubMed] [Google Scholar]
  • 34.Avettand-Fenoel V., Chaix M.L., Blanche S., Burgard M., Floch C., Toure K., Allemon M.C., Warszawski J., Rouzioux C. LTR real-time PCR for HIV-1 DNA quantification in blood cells for early diagnosis in infants born to seropositive mothers treated in HAART area (ANRS CO 01) J. Med. Virol. 2009;81:217–223. doi: 10.1002/jmv.21390. [DOI] [PubMed] [Google Scholar]
  • 35.Ganter P., Mélard A., Damond F., Delaugerre C., Dina J., Gueudin M., Maillard A., Sauné K., Rodallec A., Tuaillon E., et al. Nterlaboratory quality control of total HIV-1 DNA load measurement for multicenter reservoir studies. J. Med. Virol. 2017;89:2047–2050. doi: 10.1002/jmv.24874. [DOI] [PubMed] [Google Scholar]
  • 36.Casabianca A., Orlandi C., Canovari B., Scotti M., Acetoso M., Valentini M., Petrelli E., Magnani M. A real time PCR platform for the simultaneous quantification of total and extrachromosomal HIV DNA Forms in Blood of HIV-1 Infected patients. PLoS ONE. 2012;9:e111919. doi: 10.1371/journal.pone.0111919. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Mexas A.M., Graf E.H., Pace M.J., Yu J.J., Papasavvas E., Azzoni L., Busch M.P., Di Mascio M., Foulkes A.S., Migueles S.A. Concurrent measures of total and integrated HIV DNA monitor reservoirs and ongoing replication in eradication trials. AIDS. 2012;26:2295–2306. doi: 10.1097/QAD.0b013e32835a5c2f. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Wood R., Dong H., Katzenstein D.A., Merigan T.C. Quantification and comparison of proviral load in peripheral monuclear cells and isolated CD4+ T cells. J. Acquir. Immune Defic. Syndr. 1993;6:237–240. [PubMed] [Google Scholar]
  • 39.Katzenstein T.L., Oliveri R.S., Benfield T., Eugen-Olsen J., Nielsen C., Gerstoft J., Copenhagen AIDS Cohort Study Group Cell-associated HIV DNA measured early during infection has prognostic value independent of serum HIV RNA measured concomitantly. Scand. J. Infect. Dis. 2002;34:529–533. doi: 10.1080/00365540110080845. [DOI] [PubMed] [Google Scholar]
  • 40.Wang N., Li T. Factors associated with the size of HIV DNA reservoir. Chin. Med. J. 2017;130:224–230. doi: 10.4103/0366-6999.198009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Alidjinou E.K., Bocket L., Hober D. Quantification of viral DNA during HIV-1infection: A review of relevant clinical uses and laboratory methods. Pathol. Biol. 2015;63:53–59. doi: 10.1016/j.patbio.2014.07.007. [DOI] [PubMed] [Google Scholar]
  • 42.Jumare J., Sunshine S., Ahmed H., El-Kamary S.S., Magder L., Hungerford L., Burdo T., Eyzaguirre L.M., Umlauf A., Cherner M., et al. Peripheral blood lymphocyte HIV DNA levels correlate with HIV associated neurocognitive disorders in Nigeria. J. Neurovirol. 2017;23:474–482. doi: 10.1007/s13365-017-0520-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Ellery P.J., Tippett E., Chiu Y.L., Paukovics G., Cameron P.U., Solomon A., Lewin S.R., Gorry P.R., Jaworowski A., Greene W.C., et al. The CD16+ monocyte subset is more permissive to infection and preferentially harbors HIV-1 in vivo. J. Immunol. 2007;178:6581–6589. doi: 10.4049/jimmunol.178.10.6581. [DOI] [PubMed] [Google Scholar]
  • 44.Pang S., Koyanagi Y., Miles S., Wiley C., Vinters H.V., Chen I.S. High levels of unintegrated HIV-1 DNA in brain tissue of AIDS dementia patients. Nature. 1990;343:85–89. doi: 10.1038/343085a0. [DOI] [PubMed] [Google Scholar]
  • 45.Koncarevic S., Lößner C., Kuhn K., Prinz T., Pike I., Zucht H. In-Depth Profiling of the Peripheral Blood Mononuclear Cells Proteome for Clinical Blood Proteomics. Int. J. Proteom. 2014 doi: 10.1155/2014/129259. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 46.Abbas A.K., Lichtman A.H. Cellular and Molecular Immunology. Elsevier Science; Amsterdam, The Netherlands: 2003. [Google Scholar]
  • 47.Mesko B., Poliska S., Nagy L. Gene expression profiles in peripheral blood for the diagnosis of autoimmune diseases. Trends Mol. Med. 2011;17:223–233. doi: 10.1016/j.molmed.2010.12.004. [DOI] [PubMed] [Google Scholar]
  • 48.Anderson N.L., Anderson A.G. The human plasma proteome: History, character, and diagnostic prospects. Mol. Cell. Proteom. 2002;11:845–867. doi: 10.1074/mcp.R200007-MCP200. [DOI] [PubMed] [Google Scholar]
  • 49.Whitney A.R., Diehn A., Popper S.J., Alizadeh A.A., Boldrick J.C., Relman D.A., Brown P.O. Individuality and variation in gene expression patterns in human blood. Proc. Natl. Acad. Sci. USA. 2003;100:1896. doi: 10.1073/pnas.252784499. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50.Mcdermott J., Giri A.A., Martini I., Bono M., Giacomini M., Campelli A., Tagliaferro L., Cara A., Varnier O.E. Level of Human Immunodeficiency Virus DNA in peripheral Blood Mononuclear cells correlates with efficacy of Antiretroviral Therapy. J. Clin. Microbiol. 1999;37:2361–2365. doi: 10.1128/jcm.37.7.2361-2365.1999. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 51.De Oliveira M.F., Gianella S., Letendre S., Scheffler K., Pond S.L.K., Smith D.M., Strain M., Ellis R.J. Comparative analysis of cell-associated HIV DNA levels in cerebrospinal fluid and peripheral blood by droplet digital PCR. PLoS ONE. 2015 doi: 10.1371/journal.pone.0139510. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 52.Eriksson S., Graf E.H., Dahl C., Strain M.C., Yukl S.A., Lysenko E.S., Bosch R.J., Lai J., Chioma S., Emad F., et al. Comparative analysis of measures of viral reservoirs in HIV eradication studies. PloS Pathog. 2013;9:e1003174. doi: 10.1371/journal.ppat.1003174. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53.Valcour V., Sithinamsuwan P., Letendre S., Ances B. Pathogenesis of HIV in the Central Nervous System. Curr. HIV/AIDS Rep. 2011;8:54–61. doi: 10.1007/s11904-010-0070-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 54.Kallianpur K.J., Kirk G.R., Sailasuta N., Valcour V., Shiramizu B., Nakamoto B.K., Shikuma C. Regional cortical thinning associated with detectable levels of HIV DNA. Cereb. Cortex. 2012;22:2065–2075. doi: 10.1093/cercor/bhr285. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 55.Schnittman S.M., Psallidopoulos M.C., Lane H.C., Thompson L., Baseler M., Massari F., Fox CH., Salzman N.P., Fauci A.S. The reservoir for HIV-1 in human peripheral blood is a T cell that maintains expression of CD4. Science. 1989;245:305–308. doi: 10.1126/science.2665081. [DOI] [PubMed] [Google Scholar]
  • 56.McBreen S., Imlach S., Shirafuji T., Scott G.R., Leen C., Bell J.E., Simmonds P. Infection of the CD45RA1 (naive) Subset of peripheral CD81 lymphocytes by Human immunodeficiency virus type 1 in vivo. J. Virol. 2001;75:4091–4102. doi: 10.1128/JVI.75.9.4091-4102.2001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 57.Murray J.M., Zaunders J.J., McBride K.L., Xu Y., Bailey M., Suzuki K., Cooper D.A., Emery S., Kelleher A.D., Koelsch K.K., et al. HIV DNA subspecies persist in both activated and resting memory CD4− T Cells during antiretroviral therapy. J. Virol. 2014;88:3516–3526. doi: 10.1128/JVI.03331-13. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 58.Brenchley J.M., Hill B.J., Ambrozak D.R., Price D.A., Guenaga F.J., Casazza J.P., Kuruppu J., Yazdani J., Migueles S.A., Connors M., et al. T-Cell Subsets that harbor Human Immunodeficiency Virus (HIV) In Vivo: Implications for HIV Pathogenesis. J. Virol. 2004;78:1160–1168. doi: 10.1128/JVI.78.3.1160-1168.2004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 59.Douek D.C. Disrupting T-cell homeostasis: How HIV-1 infection causes disease. AIDS Rev. 2003;5:172–177. [PubMed] [Google Scholar]
  • 60.Gabuzda D.H., Hess J.L., Small J.A., Clements J.E. Regulation of the visna virus long terminal repeat in macrophages involves cellular factors that bind sequences containing AP-1 sites. Mol. Cell. Biol. 1989;9:2728–2733. doi: 10.1128/MCB.9.6.2728. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 61.Lane J.H., Sasseville V.G., Smith M.O., Vogel P., Pauley D.R., Heyes M.P., Lackner A.A. Neuroinvasion by simian immunodeficiency virus coincides with increased numbers of perivascular macrophages/microglia and intrathecal immune activation. J. Neurovirol. 1996;2:423–432. doi: 10.3109/13550289609146909. [DOI] [PubMed] [Google Scholar]
  • 62.Sasse T., Wu J., Zhou L., Saksena K. Monocytes and their role in Human Immunodeficiency virus pathogenesis. Am. J. Infect. Dise. 2012;8:92–105. [Google Scholar]
  • 63.Jiang X., Zhang L., Liu H., Yuan N., Hou P., Zhang R., Wu T. Expansion of CD14+CD16+ monocytes is related to acute leukemia. Int. J. Clin. Exp. Med. 2015;8:12297–12306. [PMC free article] [PubMed] [Google Scholar]
  • 64.Fischer-Smith T., Bell C., Croul S., Lewis M., Rappaport J. Monocyte/macrophage trafficking in acquired immunodeficiency syndrome encephalitis: Lessons from human and nonhuman primate studies. J. Neurovirol. 2008;14:318–326. doi: 10.1080/13550280802132857. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 65.Fischer-Smith T., Croul S., Sverstiuk A.E., Capini C., L’Heureux D., Regulier E.G., Richardson M.W., Amini S., Morgello S., Khalili K., et al. CNS invasion by CD14+CD16+ peripheral blood-derived monocytes in HIV dementia: Perivascular accumulation and reservoir of HIV infection. J. Virol. 2001;7:528–541. doi: 10.1080/135502801753248114. [DOI] [PubMed] [Google Scholar]
  • 66.Innocenti P., Ottmann M., Morand P., Leclercq P., Seigneurin J.M. HIV-1 in blood monocytes: Frequency of detection of proviral DNA using PCR and comparison with the total CD4 count. AIDS Res. Hum. Retrovir. 1992;8:261–268. doi: 10.1089/aid.1992.8.261. [DOI] [PubMed] [Google Scholar]
  • 67.Williams D.W., Veenstra M., Gaskill P.J., Morgello S., Calderon T.M., Berman J.W. Monocyte mediate HIV neuropathogenesis: Mechanisms that contribute to HIV associated neurocognitive disorders. Curr. HIV Res. 2014;12:85–96. doi: 10.2174/1570162X12666140526114526. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 68.Van Furth R., Cohn Z.A. The origin and kinetics of mononuclear phagocytes. J. Exp. Med. 1968;128:415–435. doi: 10.1084/jem.128.3.415. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 69.Sonza S., Maerz A., Deacon N., Meanger J., Millis J., Crowe S. Human immunodeficiency virus type 1 is blocked prior to reverse transcription and integration in freshly isolated blood monocytes. J. Virol. 1996;70:3863–3869. doi: 10.1128/jvi.70.6.3863-3869.1996. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 70.McElrath M.J., Pruett J.E., Cohn Z.A. Mononuclear phagocytes of blood and bone marrow: Comparative roles as viral reservoirs in human immunodeficiency virus type 1 infections. Proc. Natl. Acad. Sci. USA. 1989;86:675–679. doi: 10.1073/pnas.86.2.675. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 71.Zhu T., Muthui D., Holte S., Nickle D., Feng F. Evidence for human immunodeficiency virus type 1 replication in vivo in CD14+ monocytes and its potential role as a source of virus in patients on highly active antiretroviral therapy. J. Virol. 2002;76:707–716. doi: 10.1128/JVI.76.2.707-716.2002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 72.Lindl K.A., Marks D.R., Kolson D.L., Jordan-Sciutto K.L. HIV—Associated neurocognitive disorder: Pathogenesis and therapeutic opportunities. J. Neuroimmune Pharmacol. 2010;5:294–309. doi: 10.1007/s11481-010-9205-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 73.Shiramizu B., Gartnerb S., Williamsa A., Shikumaa C., Ratto-Kima S., Wattersa M., Aguona J., Valcoura V. Circulating proviral HIV DNA and HIV-associated dementia. AIDS. 2005;19:45–52. doi: 10.1097/00002030-200501030-00005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 74.Pulliam L., Gascon R., Stubblebine M., McGuire D., McGrath M.S. Unique monocyte subset in patients with AIDS dementia. Lancet. 1997;349:692–695. doi: 10.1016/S0140-6736(96)10178-1. [DOI] [PubMed] [Google Scholar]
  • 75.Ryan L.A., Zheng J., Brester M., Bohac D., Hahn F., Anderson J., Ratanasuwan W., Gendelman H.E., Swindells S. Plasma levels of soluble CD14 and tumor necrosis factor-alpha type II receptor correlate with cognitive dysfunction during human immunodeficiency virus type 1 infection. J. Infect. Dis. 2001;184:699–706. doi: 10.1086/323036. [DOI] [PubMed] [Google Scholar]
  • 76.Rappaport J., Volsky D.J. Role of the Macrophage in HIV-Associated Neurocognitive Disorders and Other Comorbidities in Patients on Effective Antiretroviral Treatment. J. Neurovirol. 2015;21:235–241. doi: 10.1007/s13365-015-0346-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 77.Scutari R., Alteri C., Perno C.F., Svicher V., Aquaro S. The Role of HIV Infection in Neurologic Injury. Brain Sci. 2017;7:38. doi: 10.3390/brainsci7040038. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 78.Shiramizu B., Ratto-Kim S., Sithinamsuwan P., Nidhinandana S., Thitivichianlert S., Watt G., de Souza M., Chuenchitra T., Sukwit S., Chitpatima S., et al. HIV DNA and dementia in treatment-naive HIV-1-infected individuals in Bangkok, Thailand. Int. J. Med. Sci. 2006;4:13–18. doi: 10.7150/ijms.4.13. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 79.Valcour V.G., Shiramizu B.T., Sithinamsuwan P., Nidhinandana S., Ratto-Kim S., Ananworanich J., Siangphoe U., Kim J.H., de Souza M., Degruttola V., et al. Age, apolipoprotein E4, and the risk of HIV dementia: The Hawaii Aging with HIV Cohort. J. Neuroimmunol. 2007;157:197–202. doi: 10.1016/j.jneuroim.2004.08.029. [DOI] [PubMed] [Google Scholar]
  • 80.Shiramizu B., Williams A.E., Shikuma C., Valcour V. Amount of HIV DNA in peripheral blood mononuclear cells is proportional to the severity of HIV-1-associated neurocognitive disorders. J. Neuropsychiatry Clin. Neurosci. 2009;21:68–74. doi: 10.1176/jnp.2009.21.1.68. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 81.Kallianpur K.J., Valcour V.G., Lerdlum S., Busovaca E., Agsalda M., Sithinamsuwan P., Chalermchai T., Fletcher J.L.K., Tipsuk S., Shikuma C.M., et al. HIV DNA in CD14+ reservoirs is associated with brain atrophy in cART-Naïve patients. AIDS. 2014;28:1619–1624. doi: 10.1097/QAD.0000000000000306. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 82.Hong H., Aga E., Cillo A.R., Yates A.L., Besson G., Fyne E., Koontz D.L., Jennings C., Zheng L., Mellorsa J.W. Novel Assays for Measurement of Total Cell-Associated HIV-1 DNA and RNA. Novel assays for measurement of total cell-associated HIV-1 DNA and RNA. J. Clin. Microbiol. 2016;54:902–911. doi: 10.1128/JCM.02904-15. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 83.Bogh M., Machuca R., Gerstoft J., Pedersen C., Obel N., Kvinesdal B., Nielsen H., Nielsen C. Subtype specific problems with the qualitative Amplicor HIV-1 DNA PCR test. J. Clin. Virol. 2001;20:149–153. doi: 10.1016/S1386-6532(00)00147-5. [DOI] [PubMed] [Google Scholar]
  • 84.Vandesompele J., Kubista M., Pfaffl M.W. Reference gene validation software for improved normalization. In: Logan J., Edwards K., Saunders N., editors. Real-Time PCR: Current Technology and Applications. Caister Academic Press; Poole, UK: 2009. pp. 47–64. [Google Scholar]
  • 85.Nixon D.E., Landay A.L. Biomarkers of immune dysfunction in HIV. Curr. Opin. HIV AIDS. 2010;5:498–503. doi: 10.1097/COH.0b013e32833ed6f4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 86.Komninakis S.V., Domingos E.M., Santos D.E.M., Santos C., Oliveros M.P.R., Sanabani S., Diaza R.S. HIV-1 Proviral DNA Loads (as Determined by Quantitative PCR) in patients subjected to structured treatment interruption after antiretroviral therapy failure. JCM. 2012;50:2132–2133. doi: 10.1128/JCM.00393-12. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 87.Nakayama Y., Yamaguchi H., Einaga N., Esumi M. Pitfalls of DNA Quantification Using DNA-Binding Fluorescent Dyes and Suggested Solutions. PLoS ONE. 2016;11:e0150528. doi: 10.1371/journal.pone.0150528. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 88.Bustin S.A., Benes V., Garson J.A., Hellemans J., Huggett J., Kubista M., Mueller R., Nolan T., Pfaffl M.W., Shipley G.L., et al. The MIQE guidelines: Minimum information for publication of quantitative real-time PCR experiments. Clin. Chem. 2009;55:611–622. doi: 10.1373/clinchem.2008.112797. [DOI] [PubMed] [Google Scholar]
  • 89.Livak J.K., Schmittgen T. Analysis of relative gene expression data using real-time quantitative PCR and the 2-ΔΔCt method. Methods. 2001;25:402–408. doi: 10.1006/meth.2001.1262. [DOI] [PubMed] [Google Scholar]
  • 90.Agsalda-Garcia M.A., Sithinamsuwan P., Valcour V.G., Chalermchai T., Tipsuk S., Kuroda J., Nakamura C., Ananworanich J., Zhang G., Schuetz A., et al. CD14+ Enriched Peripheral Cells Secrete Cytokines Unique to HIV-associated Neurocognitive Disorders. J. Acquir. Immune Defic. Syndr. 2017;74:454–458. doi: 10.1097/QAI.0000000000001259. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 91.Vandergeeten C., Fromentin R., Merlin E., Lowani M.B., DaFonsesca S., Bakeman W., McNulty A., Ramgopal M., Michael N., Kim J.H., et al. Cross-clade ultrasensitive PCR-based assays to measure HIV persistence in large cohort studies. J. Virol. 2014;88:12385–12396. doi: 10.1128/JVI.00609-14. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 92.Malnati M.S., Scarlatti G., Gatto F., Salvatori F., Cassina G., Rutigliano T., Volpi R., Lusso P. A universal real-time PCR assay for the quantification of group-M HIV-1 proviral load. Nat. Protoc. 2008;3:1240–1248. doi: 10.1038/nprot.2008.108. [DOI] [PubMed] [Google Scholar]
  • 93.Rouzioux C., Hubert J.B., Burgard M., Deveau C., Goujard C., Bary M., Sereni D., Viard J.P., Delfraissy J.F., Meyer L., et al. Early levels of HIV-1 DNA in peripheral blood mononuclear cells are predictive of disease progression independently of HIV-1 RNA levels and CD4+ T cell counts. J. Infect. Dis. 2005;192:46–55. doi: 10.1086/430610. [DOI] [PubMed] [Google Scholar]
  • 94.Minga A.K., Anglaret X., d’Aquin Toni T., Chaix M.L., Dohoun L., Abo Y., Coulibaly A., Duvignac J., Gabillard D., Rouet F., et al. HIV-1 DNA in peripheral blood mononuclear cells is strongly associated with HIV-1 disease progression in recently infected West African adults. J. Acquir. Immune Defic. Syndr. 2008;48:350–354. doi: 10.1097/QAI.0b013e3181775e55. [DOI] [PubMed] [Google Scholar]
  • 95.Suspe’ne R., Meyerhans A. Quantification of unintegrated HIV-1 DNA at the single cell level in vivo. PLoS ONE. 2012;7:e36246. doi: 10.1371/journal.pone.0036246. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 96.Avettand-Fenoel V., Hocqueloux L., Ghosn J., Cheret A., Frange P., Melard A., Viard J.-P., Rouzioux C. Total HIV DNA, a marker of viral reservoir dynamics with clinical implications. Clin. Microbiol. Rev. 2016;29:859–880. doi: 10.1128/CMR.00015-16. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 97.Lillo F.B., Grasso M.A., Lorine S., Bellotti M.G., Colucci G. Few modifications of the Cobus Amplicor HIV Monitor 1.5 test allow reliable quantitation of HIV-1 proviral load in peripheral blood mononuclear cells. J. Virol. Methods. 2004;120:201–205. doi: 10.1016/j.jviromet.2004.05.008. [DOI] [PubMed] [Google Scholar]
  • 98.Pavsic J., Zel J., Milavec M. Assessment of the realtime PCR and different digital PCR platforms for DNA quantification. Anal. Bioanal. Chem. 2016;408:107–121. doi: 10.1007/s00216-015-9107-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 99.Trypsteen W., Kiselinova M., Vandekerckhove L., De Spiegelaere W. Diagnostic utility of digital PCR for HIV reservoir quantification. J. Virus Erad. 2016;2:162–169. doi: 10.1016/S2055-6640(20)30460-X. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 100.Strain M.C., Lada S.M., Luong T., Rought S.E., Gianella S., Terry V.H., Spina C.A. Highly precise measurement of HIV DNA by droplet digital PCR. PLoS ONE. 2013;8:e55943. doi: 10.1371/journal.pone.0055943. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 101.Bosman K.J., Nijhuis M., van Ham P.M., Wensing A.J.M., Vervisch K., Vandekerckhove L., De Spiegelaere W. Comparison of digital PCR platforms and semi-nested qPCR as a tool to determine the size of the HIV reservoir. Sci. Rep. 2015;5:13811. doi: 10.1038/srep13811. [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from Viruses are provided here courtesy of Multidisciplinary Digital Publishing Institute (MDPI)

RESOURCES