Skip to main content
PLOS ONE logoLink to PLOS ONE
. 2019 Apr 18;14(4):e0215620. doi: 10.1371/journal.pone.0215620

HIV disease, metabolic dysfunction and atherosclerosis: A three year prospective study

Hann Low 1, Anh Hoang 1, Tatiana Pushkarsky 2, Larisa Dubrovsky 2, Elizabeth Dewar 3, Maria-Silvana Di Yacovo 4, Nigora Mukhamedova 1, Lesley Cheng 5, Catherine Downs 6, Gary Simon 7, Maria Saumoy 4, Andrew F Hill 5, Michael L Fitzgerald 8, Paul Nestel 1, Anthony Dart 1,3, Jennifer Hoy 6,9, Michael Bukrinsky 2,#, Dmitri Sviridov 1,*,#
Editor: Esther Lutgens10
PMCID: PMC6472799  PMID: 30998801

Abstract

HIV infection is known to be associated with cardiometabolic abnormalities; here we investigated the progression and causes of these abnormalities. Three groups of participants were recruited: HIV-negative subjects and two groups of treatment-naïve HIV-positive subjects, one group initiating antiretroviral treatment, the other remaining untreated. Intima-media thickness (cIMT) increased in HIV-positive untreated group compared to HIV-negative group, but treatment mitigated the difference. We found no increase in diabetes-related metabolic markers or in the level of inflammation in any of the groups. Total cholesterol, low density lipoprotein cholesterol and apoB levels were lower in HIV-positive groups, while triglyceride and Lp(a) levels did not differ between the groups. We found a statistically significant negative association between viral load and plasma levels of total cholesterol, LDL cholesterol, HDL cholesterol, apoA-I and apoB. HIV-positive patients had hypoalphalipoproteinemia at baseline, and we found a redistribution of sub-populations of high density lipoprotein (HDL) particles with increased proportion of smaller HDL in HIV-positive untreated patients, which may result from increased levels of plasma cholesteryl ester transfer protein in this group. HDL functionality declined in the HIV-negative and HIV-positive untreated groups, but not in HIV-positive treated group. We also found differences between HIV-positive and negative groups in plasma abundance of several microRNAs involved in lipid metabolism. Our data support a hypothesis that cardiometabolic abnormalities in HIV infection are caused by HIV and that antiretroviral treatment itself does not influence key cardiometabolic parameters, but mitigates those affected by HIV.

Introduction

HIV infection is accompanied by a range of metabolic co-morbidities, with cardiovascular complications being among the most important in terms of morbidity and mortality [15]. Initially, cardiovascular co-morbidities of HIV infection were attributed to dyslipidaemia caused by protease inhibitor-containing treatment regimens [6, 7]. However, while the pro-atherogenic regimens were gradually phased out as initial therapy, the cardiovascular and metabolic complications persisted [8, 9]. We have suggested that HIV infection itself can cause atherosclerosis and proposed a mechanism whereby the HIV protein Nef reduces the efficiency of reverse cholesterol transport causing accumulation of cholesterol in macrophages and hypoalphalipoproteinemia [10, 11]. We also demonstrated that extracellular Nef, mimicking Nef released from HIV infected cells into circulation, caused atherosclerosis and dyslipidaemia in vivo in the absence of the infection or any other HIV-related factors [12]. While experimental findings point to the important role of HIV itself in the pathogenesis of cardiovascular and metabolic co-morbidities of HIV infection, separation between the effects of HIV and of antiretroviral treatment in the clinical setting is difficult as the overwhelming majority of HIV-positive patients now commence antiretroviral treatment immediately after diagnosis. Furthermore, HIV-positive patients often have overrepresented conventional cardiovascular risk factors, such as high rates of smoking, which may also contribute to the elevated cardiovascular risk [13]. These factors complicate investigation of the causes, pathogenic mechanisms and progression of HIV-associated cardiometabolic co-morbidities.

Most clinical studies investigating the development of cardiovascular disease in the context of HIV infection were cross-sectional [2, 1417], retrospective [18], or short-term prospective studies [1922]; they rarely included patients not treated with antiretroviral regimens. SMART, the only large, outcome based prospective study, while confirming the higher rate of cardiovascular co-morbidities in HIV patients, did not conclusively resolve the issue of its causes [23]. We recently published a prospective study comparing development of atherosclerosis in treated and untreated patients with HIV [21]. However, that study reported only 1 year of follow up and did not include an HIV-negative group. Here, we report the outcomes of a new study, which compared development of atherosclerosis, as well as changes in lipid and glucose metabolism, in HIV-negative and HIV-positive subjects with or without antiretroviral treatment over 3 years of follow up.

Patients and methods

Participant recruitment

Study participants were recruited from 3 sites, a) Infectious Diseases Outpatient Department (including referral of research participants from General Practice clinics with a high case load of people diagnosed with HIV infection) at The Alfred Hospital, Melbourne, Victoria, Australia; b) The George Washington University HIV Clinic, Washington D.C, USA; or c) Bellvitge University Hospital, Barcelona, Spain. Subjects included in HIV-negative group were recruited from healthy volunteers at sites a) and b). Exclusion criteria were age (under 18 or over 60 years), current lipid-lowering medication (including fish oils), history of Familial Hypercholesterolemia or other form of familial dyslipidemia, HCV infection, impaired liver function, alcohol consumption >250 g (>30 units) per week, and confirmed diagnosis of coronary artery disease, carotid/cerebral artery disease or peripheral artery disease. All participants provided written informed consent and the study was approved by the Alfred Hospital Human Research and Ethics Committee (#377/10), George Washington University IRB (#061030) or Hospital Universitario de Bellvitge IRB (#IRB00005523), respectively. The study complied with principles of Declaration of Helsinki. The flow chart of patient recruitment is presented in Fig 1.

Fig 1. Consort diagram of the study population.

Fig 1

At baseline, participants were asked to complete a questionnaire on the presence or absence of lipodystrophy (self-assessed), smoking history, any alcohol consumption, level of physical activity, current use of recreational drugs and medical history. Prior to the baseline visit, HIV-positive participants had made decisions with their treating clinician with regard to commencing treatment or not, based on their viral load, CD4+ T cell counts and percentage. They were allocated into treated and untreated groups, and recruitment continued with the aim of recruiting 30 participants on each group. The decision of whether each individual participant in HIV-positive untreated group remains untreated or commences treatment was made by the treating physician according to the national guidelines at the time of the decision.

Participants were followed-up annually for 3 years, at each visit plasma samples, IMT measurements, anthropometric measurements (height and weight), blood pressure and pulse rate were collected. Twenty mLs of blood was collected into EDTA tubes after a 10 h fast, and plasma was obtained by low speed centrifugation, aliquoted and frozen at −80°C.

Intima-Media thickness

At every visit, participants’ carotid arteries were assessed for carotid wall intima-media thickness (cIMT) by ultrasound using Philips iE33 with 11 MHz Linear transducer as previously described [24]. cIMT was defined as the distance between lumen-intima interface and media-adventitia interface and was measured from the 2D high resolution digital images obtained using an automated border-detection algorithm (Philips QLAB, Netherlands). cIMT was determined in areas identified as without plaque by 6 duplicate consecutive measurements made 1–2 cm proximal to the carotid bulb. IMT of both right and left carotid arteries were measured and mean value of right and left carotid artery IMT was used. All measurements were performed by the same site operator blinded to the subjects’ identity at all time points. Coefficients of variation for repeated measures was 5.7%.

Plasma metabolic profile

Measurements of plasma levels of total cholesterol (TC), high density lipoprotein cholesterol (HDL-C), low density lipoprotein cholesterol (LDL-C), apolipoproteinA-I (apoA-I), apolipoprotein-B (apoB) and triglycerides (TG) were done using the COBAS Integra 400 Plus blood analyser (Roche Diagnostics, Switzerland). Plasma levels of glucose were determined by a Colorimetric assay kit (Cayman Chemical, USA) using 1/5 dilutions, and plasma levels of insulin were determined by ELISA (ALPCO, USA). Plasma levels of LCAT were determined by ELISA (Cell-Biolabs Inc., USA) using 1/10 dilutions and those of CETP were determined by ELISA (Cell-Biolabs Inc., USA) using 1/8 dilutions. hsCRP was determined by ELISA (R&D Systems, USA) using 1/2,500 dilutions; Lp(a) in plasma was determined by ELISA (Mercodia, Sweden) using 1/101 to 1/404 dilutions. Homeostatic model assessments for insulin resistance (HOMA-IR) and insulin sensitivity (HOMA-IS) [25] were calculated for each participant.

Viral load, CD4+ T cell counts, CD4 percentage, and HbA1c were measured using commercial kits and were obtained from participants’ medical records provided to us by the individual recruitment sites.

HDL sub-populations

The distribution of apoA-I among HDL sub-populations was analysed by non-denaturing PAGE (Thermo Fisher Scientific, USA) and immunoblotting using polyclonal anti-human apoA-I as previously described [26].

Cholesterol efflux

Cholesterol efflux assay was performed as previously described [27]. Briefly, THP-1 monocytes were differentiated with 100 ng/ml phorbol 12-myristate 13-acetate (PMA; Merck) and incubated in serum-containing RPMI-1640 (Thermo Fisher Scientific, USA) with 0.6 μCi/ml [3H]-cholesterol (American Radiolabeled Chemicals Inc., USA) for 72 h. Cells were then incubated for 18 h in serum-free RPMI-1640 containing 4 μM LXR agonist TO-901317 (Cayman Chemical, USA). Cholesterol efflux was performed using 1.1% apoB-depleted plasma for 2 h. ApoB-depleted plasma was obtained after precipitation of apoB-containing lipoproteins with Dextran Sulfate (Merck, USA) as previously described [26].

RNA isolation and small RNA deep sequencing

RNA from plasma (400 μl) was extracted using the Qiagen miRNeasy mini kit with the use of TRIzol-LS (ThermoFisher) for organic phase separation and was analysed by an Agilent 2100 Bioanalyzer using Small RNA Chips (Agilent Technologies). Small RNA libraries were constructed from total isolated RNA using the Ion Total RNA-Seq Kit V2 (Life Technologies, Australia) and ligated to adapters containing a unique index barcode according to the manufacturers’ protocol. The yield and size distribution of the small RNA libraries were assessed using the Agilent 2100 Bioanalyzer instrument with the DNA 1000 chip (Agilent Technologies). Equally pooled libraries were prepared for deep sequencing using the Ion Chef system (Life Technologies) and sequenced on the Ion Torrent Ion S5 using Ion 540 chips (Life Technologies). Pre-processing of reads, removal of adapters and barcodes were performed using the Torrent Suite (v.5.8.0). Sequences were analyzed for quality control (FASTQC), aligned to the Human genome (HG19) using the Torrent Suite, and files transferred to Partek Genomic Suite and Flow (Partek Incorporated, Singapore) for mapping against miRBase V.20 and normalization to reads per million reads (RPM). miRNAs identified with at least 30 RPM were used for further analysis on Partek Genomic suite which included statistical analysis and hierarchical clustering.

Statistical analysis

Descriptive statistics (mean ± SD) were calculated for all the variables of interest at baseline. Longitudinal differences between and within groups for primary and secondary outcomes were assessed using mixed-models for repeated measures (MMRM). The MMRM model included group, time point, group by time point interaction, age, sex, BMI, as well as adjustment for baseline measurements by including baseline as a covariate. The model used an autoregressive variance-covariance structure to model within-individual errors. This model was used to analyse all outcomes. Correlations between outcomes were assessed by Pearson Product Movement correlation. Bonferroni correction was performed to adjust for multiple comparisons. Results were reported as mean ± SEM unless indicated otherwise. Statistical analyses were performed using Stata 13 (StataCorp LLC, USA). Sample size calculations for this study were based on previous evidence for primary outcome measures (expected minimum difference = 10%, expected coefficient of variation = 20%, α = 0.05, power > 0.8 for 3 groups) [21].

Results

Participants

Three groups of participants were recruited and investigated in this study. At the time of recruitment, the first group consisted of 27 HIV-negative subjects (control group). The second group consisted of 19 HIV-positive treatment-naïve participants who remained untreated for the duration of the study (36 months) (untreated group), while the third group consisted of 41 HIV-positive treatment-naïve patients who initiated treatment with an NNRTI-based regimen at the beginning of the study and continued treatment for the duration of the study (treated group). There were losses in follow up in all three groups during the course of the study (Fig 1). The number of participants in the HIV-negative group reduced to 23 after 12 months, 22 after 24 months, and 20 after 36 months. The number of participants in the HIV-positive treated group reduced to 38 after 12 months, 34 after 24 months, and 23 after 36 months. The number of participants in the HIV-positive untreated group reduced to 15 after 12 months, 11 after 24 months, and 4 after 36 months; due to limited number of participants, only analysis of HDL metabolism is available for the 36 month time-point in this group. The main reason for the high rate of drop out in this group was initiation of ART as medically indicated.

Baseline characteristics of the participants are shown in Table 1. All but 2 participants (at the Alfred Hospital site) were males and there was no statistically significant difference between the groups in BMI, systolic and diastolic blood pressure and pulse rate. There were higher rates of smoking and use of recreational drugs in both HIV-positive groups. HIV-positive subjects had a lower prevalence of alcohol consumption; most of the participants did not report lipodystrophy. HIV-positive treated participants were marginally younger than HIV-negative participants. All outcomes were therefore adjusted for age during the analysis.

Table 1. Baseline anthropometric and disease related variable.

Variable HIV-negative HIV-positive untreated HIV-positive treated
Number 27 19 41
Age (y) 42 ± 10 39 ± 10 37 ± 9*
Gender (M:F) 27:0 18:1 40:1
BMI (kg/m2) 25.9 ± 4.0 25.3 ± 4.1 24.1 ± 3.2
CD4+ T cell count (cells/μl) N/A 700 ± 284 437 ± 232
CD4% N/A 30.1 ± 10.2 23.3 ± 10.1
Viral load (copies/ml) N/A 35,000 ± 53,000 73,000± 109,000
Systolic BP (mm Hg) 117 ± 10 122 ± 16 122 ± 13
Diastolic BP (mm Hg) 73 ± 11 77 ± 11 76 ± 8
Pulse rate (Beats/min) 70 ± 6 73 ± 15 73 ± 14
Current smoking (y:n:u)1 1:26:0 10:9:0** 10:30:1*#
Alcohol Consumption (y:n:u)1 27:0:0 11:8:0*** 33:7:3*
Exercise (y:n:u)1 23:4:0 9:5:5 21:12:8
Lipodystrophy (y:n:u)1 1:26:0 1:13:5 3:30:8
Recreational Drugs (y:n:u)1 2:25:0 9:10:0** 16:24:1**

Unadjusted values, Mean ± SD are shown

1y:n:u = yes: no: unknown (no answer).

*p<0.05 versus HIV-negative

**p<0.01 versus HIV-negative

***p<0.001 versus HIV-negative

#p<0.05 versus HIV-positive untreated

Progression of HIV disease

At baseline, there was no difference in CD4+ T cell counts or percentages between both HIV-positive groups, however, there was significantly higher viral load in “treated” group (Table 1); the latter was an indication to commence treatment. As expected, after initiation of ART, both CD4+ T cell counts and CD4+ T cell percentage in patients’ blood increased, while viral load decreased to undetectable levels, and these parameters remained stable for the duration of the study (Fig 2A–2C). In the untreated group, CD4+ T cell count gradually declined with time, but CD4+ T cell percentage and viral load remained stable (Fig 2A–2C). There was a statistically significant difference between treated and untreated subjects in all three parameters at all time points after commencement of ART. It is important to recognize a selection bias in assessing progression of HIV disease in untreated subjects, as patients with increasing viral load and declining CD4+ cell counts commenced treatment and were excluded from further follow up.

Fig 2. Changes in CD4+ T lymphocytes and HIV viral load between treated and untreated HIV positive participants.

Fig 2

A–CD4+ cell count; B–CD4+ cell percentage; C–Plasma viral load. “+HIV,—Treatment”–HIV-positive untreated group; “+HIV, + Treatment”–HIV-positive treated group. #p< 0.05 versus HIV-positive untreated group.

Progression of atherosclerosis and diabetes

Progression of atherosclerosis was assessed by monitoring changes in cIMT. There was no statistically significant difference in mean cIMT between the groups at baseline. In HIV-negative subjects cIMT rose slightly after 1 year (p<0.01 versus baseline), and no further rise was observed after 2 and 3 years (Fig 3A). In treated HIV-positive patients cIMT rose slightly after 3 years (p<0.05 versus baseline), and there was no statistically significant difference versus the control (Fig 3A). In untreated HIV-positive patients mean cIMT rose substantially after 2 years (p<0.05 versus baseline and 1 year).

Fig 3. Progression of atherosclerosis and diabetes.

Fig 3

A–cIMT; B—Plasma glucose concentration; C–Plasma insulin concentration; D–HOMA insulin resistance index; E–HOMA insulin sensitivity index; F–Plasma HbA1c concentration; G–Plasma hsCRP concentration. “+HIV,—Treatment”–HIV-positive untreated group; “+HIV, + Treatment”–HIV-positive treated group; “-HIV”–HIV-negative group. *p<0.05 HIV-negative versus HIV positive untreated group; p<0.05 HIV-negative versus HIV-positive treated group.

Development of diabetes was assessed by measuring fasting levels of glucose, insulin and HbA1c and by calculating HOMA indices. No participant entered the study with a clinical diagnosis of diabetes, and at baseline there was no statistically significant difference between the groups in any of these parameters. Mean glucose and insulin levels remained stable in all three groups for the duration of the study (Fig 3B and 3C). Consequently, no statistically significant changes in HOMA-IR and HOMA-IS were detected (Fig 3D and 3E). HbA1c levels in HIV-negative subjects were reduced during the study, while remaining stable in both HIV-positive groups. Mean HbA1c levels in the HIV-positive groups were higher compared with the HIV-negative group after 2 years of follow up (Fig 3F).

There was considerable variability in mean hsCRP levels with no statistically significant differences found between the groups at the baseline or during follow up (Fig 3G).

Lipids and lipoproteins

There was no difference in mean plasma levels of total cholesterol between the groups at the baseline (Fig 4A). Over 2 and 3 years of follow up the level of total cholesterol in plasma of HIV-negative subjects increased by 15.2% and 14.4%, respectively (p<0.001). During the same period, plasma total cholesterol level in HIV-positive treated patients increased by 7.3% (p = 0.08) and 9.2% (p<0.05). There was no significant change in plasma total cholesterol level in HIV-positive untreated patients. After 2 years of follow up plasma total cholesterol levels were higher in HIV-negative subjects than in HIV-positive untreated patients (Fig 4A). There was no difference in mean plasma total triglyceride level between the groups at the baseline or during follow up, and total triglyceride content did not significantly change during follow up in all groups (Fig 4B).

Fig 4. Lipids and lipoproteins.

Fig 4

A–Plasma total cholesterol levels; B–Plasma triglyceride levels; C–Plasma LDL cholesterol levels; D–Plasma apoB levels; E–Plasma Lp(a) levels. “+HIV,—Treatment”–HIV-positive untreated group; “+HIV, + Treatment”–HIV-positive treated group; “-HIV”–HIV-negative group. *p<0.05 HIV-negative versus HIV positive-untreated group; p<0.05 HIV-negative versus HIV-positive treated group.

Changes in mean plasma low density lipoprotein cholesterol (LDL-C) levels were similar to those of total cholesterol, with no difference between the groups at baseline. A gradual increase in mean plasma LDL-C in the HIV-negative group (28%, p<0.01 at 2 and 3 year time points) was found, a moderate increase in LDL-C levels was observed in the HIV-positive treated group (10%, p<0.05 after 2 years, but no significant increase after 3 years) (Fig 4C). There was no significant change in LDL-C levels in HIV-positive untreated group during follow up. Plasma LDL-C levels were higher in HIV-negative subjects than in both groups of HIV-positive participants after 2 years, and higher than in HIV-positive treated patients after 2 and 3 years (Fig 4C). Changes in mean plasma levels of apoB mirrored those in LDL-C, except that the between group differences after 3 years of follow-up did not reach statistical significance (Fig 4D). There was a trend for mean plasma levels of Lp(a) to be higher in HIV-positive treated patients compared to the other two groups at all time points except at baseline, however, the variability was high and the differences did not reach statistical significance (Fig 4E).

High density lipoprotein and parameters of reverse cholesterol transport

At baseline, mean plasma levels of high density lipoprotein cholesterol (HDL-C) and apolipoprotein A-I (apoA-I) in the HIV-positive groups were lower compared to those in HIV-negative subjects (Fig 5A and 5B; p<0.05). There were no statistically significant differences between the groups at all follow-up points (Fig 5A and 5B).

Fig 5. High density lipoproteins.

Fig 5

A–Plasma HDL cholesterol levels; B–Plasma apoA-I levels; C—Proportion of HDL2b particles in total HDL; D—Proportion of HDL2a particles in total HDL; E—Proportion of HDL3a particles in total HDL; F—Proportion of HDL3b particles in total HDL; G—Proportion of HDL3c particles in total HDL. “+HIV,—Treatment”–HIV-positive untreated group; “+HIV, + Treatment”–HIV-positive treated group; “-HIV”–HIV-negative group. #p< 0.05 HIV-positive treated versus HIV-positive untreated group; *p<0.05 HIV-negative versus HIV-positive untreated group; p<0.05 HIV-negative versus HIV positive treated group.

We used non-denaturing gradient gel electrophoresis to analyse changes in the distribution of apoA-I among HDL particles of different sizes between baseline and after 3 years of follow up. We found no difference between the groups at baseline, but a significant increase in proportion of small HDL3b and HDL3c particles at the expense of large HDL2a particles in HIV-positive untreated patients when compared to the other two groups (Figs 5C–5G).

Functionality of HDL was measured as a capacity of 1.1% serum depleted of apoB-containing lipoproteins to support cholesterol efflux from differentiated THP-1 macrophages (see Materials and Methods for details). There was no difference in HDL functionality between the groups at baseline (Fig 6A). HDL functionality remained unchanged in the HIV-positive treated group for the duration of the study, while gradually declined in the HIV-negative group, and in the HIV-positive untreated group (Fig 6A; p<0.05 baseline versus 2 and 3 years). There was statistically significant difference between the HIV-positive treated and HIV-negative groups after 3 years of follow up (Fig 6A). We also calculated functionality of individual HDL particles by relating values of cholesterol efflux to concentration of apoA-I (Fig 6B). Similar to the overall HDL functionality, this parameter declined during follow up in HIV-negative group (Fig 6B, p<0.01), but not in treated HIV-positive group. The variability of this parameter was high and the only statistically significant difference between the groups was that between HIV-negative and HIV-positive untreated groups at baseline (Fig 6B).

Fig 6. Reverse cholesterol transport.

Fig 6

A–HDL functionality defined as the capacity of HDL (apoB-depleted plasma) to support cholesterol efflux from THP-1 macrophages; B–Functionality of individual HDL particles (defined as capacity of apoB-depleted plasma to support cholesterol efflux normalised to apoA-I level); C–Plasma concentration of LCAT; D–Plasma concentration of CETP. “+HIV,—Treatment”–HIV-positive untreated group; “+HIV, + Treatment”–HIV-positive treated group; “-HIV”–HIV-negative group. *p<0.05 HIV-negative versus HIV-positive untreated group; p<0.05 HIV-negative versus HIV-positive treated group, #p<0.05 HIV-positive treated versus HIV-positive untreated group.

There was no difference in plasma concentration of lecithin cholesterol acyltransferase (LCAT) between the groups or within the groups during follow up (Fig 6C). Plasma concentration of cholesteryl ester transfer protein (CETP) was similar for all three groups at baseline and did not change during follow up in HIV-negative and treated HIV-positive groups, but gradually increased in the HIV-positive untreated participants (Fig 6D).

Plasma microRNAs

We and others have previously suggested that the effects of HIV on lipid metabolism may be mediated in part through differences in microRNAs released into bloodstream [28, 29]. To determine whether the effects of HIV on lipid metabolism are mediated through microRNAs, we analysed differences in plasma microRNA profile in a subgroup of HIV-negative (n = 14) and HIV-positive (n = 7) participants at a single time-point (6 months). Small RNA sequencing revealed 72 highly expressed microRNAs displaying more than 30 reads per million (RPM) across all samples (S1 File). Three microRNAs were shown to be significantly differently expressed between HIV-positive participants compared to HIV-negative group, with differences greater than 1.5 fold and p<0.05 (Fig 7 and Table 2). Two of these microRNAs, hsa-miR-27a-3p and hsa-miR-126-5p, were overrepresented, while hsa-miR-1307-3p was under-represented in plasma of HIV-positive subjects (Fig 7 and Table 2).

Fig 7. Hierarchical clustering of significantly differentially expressed miRNA in HIV-positive and HIV-negative samples.

Fig 7

Table 2. MicroRNAs differentially abundant in plasma of HIV-positive versus HIV-negative participants.

microRNA P value HIV+/HIV- ratio
hsa-miR-27a-3p 0.00112 1.58
hsa-miR-126-5p 0.01038 1.58
hsa-miR-1307-3p 0.04281 0.55

Raw sequencing reads were aligned to the human genome (HG19) and mapped to miRBase V.20. Reads were then normalised to reads per million (RPM) and miRNA species with greater than 30RPM were used for ANOVA analysis. Significantly expressed miRNA expressing greater than 1.5 fold changes and p-value < 0.05 are shown here. Red indicates high expression and blue indicates low expression.

Associations

To investigate an association between the severity of HIV infection and changes in surrogate markers of atherosclerosis or metabolic parameters, we analysed correlations between all metabolic parameters and IMT versus CD4+ T cell count and HIV viral load. For this analysis we combined all data for both groups of HIV-positive patients. We found a statistically significant negative association between viral load and plasma levels of total cholesterol (r = -0.21, p<0.001), LDL cholesterol (r = -0.13, p<0.001), HDL cholesterol (r = -0.19, p<0.001), apoA-I (r = -0.24, p<0.001) and apoB (r = -0.16, p<0.001). We also found a negative association between viral load and cIMT (r = -0.18, p<0.05), but this was mainly due to a sharply decreased viral load combined with modest rise in IMT in treated HIV-positive patients. We found a statistically significant positive association between CD4+ T cell count and proportion of HDL3c particles and combined HDL3 (HDL3a+HDL3b+HDL3c) (r = 0.28, p<0.01), and a negative association between CD4+ T cell count and proportion of combined HDL2 (HDL2a+HDL2b) particles (r = -0.28, p<0.01).

Discussion

In this study, we investigated metabolic dysfunction and atherosclerosis in people newly diagnosed with HIV infection who commenced antiretroviral therapy (ART) or remained untreated for 3 years. This study was performed prior to the recommendation that all HIV-positive patients commence ART immediately after HIV diagnosis regardless of CD4 cell count. A significant rise in cIMT was observed in untreated HIV-infected patients after 2 years. In uninfected group, there was a modest rise in cIMT after 1 year, which did not progress further, and in treated patients a modest rise was seen after 3 years. This finding indicates that progression of atherosclerosis, as assessed by changes in cIMT, seen in HIV-positive patients, was mitigated by ART. This is consistent with a suggestion that HIV infection itself, rather than side-effects of ART, is a major cause of elevated cardiovascular risk in HIV-positive patients [10, 30]. It is also consistent with the view that controlling HIV disease with modern ART regimens effectively restrains progression of atherosclerosis without bringing an additional iatrogenic cardiovascular risk. The changes in atherosclerosis progression took at least two years to manifest, consistent with no change in cIMT after one year of follow up reported in our previous study [21]. Differences in lifestyle factors between HIV-negative and treated HIV-positive subjects apparently did not contribute significantly to the progression of atherosclerosis, a finding consistent with our previous report [13]. Recognizing limitations of IMT as a predictor of future cardiovascular events [31], our findings tentatively support a hypothesis that HIV infection contributes to the pathogenesis of atherosclerosis, as proposed in a number of cross-sectional and prospective studies [2, 5, 1416, 2022] and confirmed in recently published meta-analysis [9].

Analysis of diabetes markers, plasma glucose, insulin and HbA1c levels as well as HOMA indices, showed no significant increase in progression to diabetes related to HIV infection, ART or lifestyle factors. This is in contrast to several reports showing increased risk of diabetes in HIV-positive patients [32, 33]; the differences likely attributable to exposure to older treatment regimens, age, lifestyle factors or genetic background of the study population. There was also no effect of HIV or ART on a marker of inflammation, hsCRP, although variability of this parameter was high. This finding does not rule out low grade inflammation in HIV-positive patients.

We found a consistent difference in plasma levels of total cholesterol, LDL cholesterol and apoB between HIV-negative and HIV-positive participants. Hypocholesterolemia in untreated HIV-positive patients has been reported previously [34, 35] while HIV-associated hypercholesterolemia was predominantly attributed to the PI-containing treatment regimens [35, 36], the latter were not used in this study. It appears that the effect on plasma level of LDL was related to HIV and was not alleviated by ART. We did not find any effect of HIV or ART on plasma triglycerides, a finding contrasting with some reports [34, 37] but consistent with previous findings from our laboratory [21] and also consistent with lack of diabetes in our study population. Of note, patients on lipid-lowering or anti-diabetic medications were excluded from the study ruling out the effect of these treatments.

Consistent with findings from our and other laboratories [21, 34, 35, 37, 38], HIV-positive patients had hypoalphalipoproteinemia at baseline. However, with slight decrease of HDL levels in the HIV-negative group and slight increase in HIV-positive groups, the difference disappeared during follow up. Instead, we found redistribution in HDL sub-populations, with increased proportion of smaller HDL particles at the expense of larger HDL particles in untreated HIV-positive patients. This is consistent with our previous findings in SIV-infected monkeys [39] and may be a result of increased plasma levels of CETP in this group, the latter being consistent with our published reports [21, 38]. The changes in HDL functionality were not evident at baseline, consistent with our previous study [21], but progressed during 3 years of follow-up. When normalized to the number of HDL particles, the functionality of HDL gradually declined in all groups and the differences were no longer statistically significant, indicating that changes in overall HDL functionality is a combination of changes in functionality of the particles and in their numbers.

Relative to control group, the HIV-infected groups had higher prevalence of smoking, recreational drug use and lower prevalence of alcohol consumption, lifestyle factors that can affect many parameters measured in this study. However, the two HIV-positive groups were similar for these lifestyle factors and most differences seen in HIV-infected untreated group were not seen in HIV-infected treated group over the course of the study suggesting that the differences between the two HIV-infected groups were unlikely to be due to these lifestyle factors.

An interesting finding of this study is the effect of HIV infection on plasma microRNA profile. Three most strongly affected microRNAs were found to regulate pathways related to the development of atherosclerosis and metabolic dysfunction. MiR-27 plays a prominent role in lipid metabolism: it is one of the key regulators of the expression of ABCA1 and LDL receptor [40, 41] and is also involved in regulation of adipogenesis [42]. Polymorphism in miR-27 was found to be associated with higher body mass in preeclamptic HIV-infected women [43] MiR-126 is an important element of anti-inflammatory and anti-atherogenic response to the lipid-induced inflammation [44, 45], but was shown to be a positive regulator of inflammatory response in HIV-positive monocytes and was upregulated in chronic HIV infection [46]. Differential expression of miR-1307 in monocytes was identified as a signature of diabetes [47]. Many miRNAs are transported in plasma by HDL, and it is unclear if changes in the abundance of miRNAs are a direct result of infection or of changes in HDL structure induced by infection.

This study has several important limitations. First, the group size was limited, especially that of HIV-positive untreated group. Evolving guidelines demanded initiation of treatment at earlier stages of the disease, depleting this group as the study progressed and introducing a bias by removing patients with more rapid progression of the disease from the untreated group. Thus, negative findings of this study should be treated with caution. Second, we did not differentiate between different treatment regimens. Treatment regimens were determined by the treating physicians and very few patients in HIV-positive treated group received identical regimens; moreover, regimens were frequently changed during the follow up period. Finally, it is important to recognize that cardiometabolic co-morbidities may take more than 3 years to develop and their development may depend on the length of the period between the actual infection, diagnosis and commencement of ART.

Overall, this study demonstrates that HIV infection is associated with disturbances of several aspects of cholesterol metabolism and with advancement in the progression of atherosclerosis. Both of these effects are most likely caused by the virus itself. Modern anti-retroviral therapy had mitigated most of the cardiometabolic effects of HIV, at least in short-term, and had no adverse contribution to the cardiometabolic risk. Lifestyle factors also did not have a significant contribution to the cardiometabolic risk.

Supporting information

S1 File. Database 1.

Sequencing database of miRNA in plasma of study subjects.

(XLSX)

S2 File. Database 2.

Manuscript data.

(XLSX)

Acknowledgments

We thank the participants for their contributions to regular follow-up in the study, and the clinical trial co-ordinators for organising the visits and collection of clinical data, routine HIV monitoring information and specimens.

Data Availability

All relevant data are within the manuscript and its Supporting Information files.

Funding Statement

The study was supported by the grant from the National Institutes of Health (HL101274) (www.nih.gov) to MB and DS and in part by the Victorian Government’s OIS Program. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

References

  • 1.Lake JE, Currier JS. Metabolic disease in HIV infection. Lancet Infect Dis. 2013;13:964–975. 10.1016/S1473-3099(13)70271-8 [DOI] [PubMed] [Google Scholar]
  • 2.Hsue PY, Ordovas K, Lee T, Reddy G, Gotway M, Schnell A, et al. Carotid intima-media thickness among human immunodeficiency virus-infected patients without coronary calcium. Am J Cardiol. 2012;109:742–747. 10.1016/j.amjcard.2011.10.036 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Baker JV, Lundgren JD. Cardiovascular implications from untreated human immunodeficiency virus infection. Eur Heart J. 2011;32:945–951. 10.1093/eurheartj/ehq483 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Boccara F. Cardiovascular complications and atherosclerotic manifestations in the HIV-infected population: type, incidence and associated risk factors. AIDS. 2008;22 Suppl 3:S19–26. [DOI] [PubMed] [Google Scholar]
  • 5.Triant VA, Perez J, Regan S, Massaro JM, Meigs JB, Grinspoon SK, et al. Cardiovascular Risk Prediction Functions Underestimate Risk in HIV Infection. Circulation. 2018;137:2203–2214. 10.1161/CIRCULATIONAHA.117.028975 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Friis-Moller N, Sabin CA, Weber R, d'Arminio Monforte A, El-Sadr WM, Reiss P, et al. Combination antiretroviral therapy and the risk of myocardial infarction. N Engl J Med. 2003;349:1993–2003. 10.1056/NEJMoa030218 [DOI] [PubMed] [Google Scholar]
  • 7.Depairon M, Chessex S, Sudre P, Rodondi N, Doser N, Chave JP, et al. Premature atherosclerosis in HIV-infected individuals—focus on protease inhibitor therapy. Aids. 2001;15:329–334. [DOI] [PubMed] [Google Scholar]
  • 8.Bavinger C, Bendavid E, Niehaus K, Olshen RA, Olkin I, Sundaram V, et al. Risk of Cardiovascular Disease from Antiretroviral Therapy for HIV: A Systematic Review. PLoS ONE. 2013;8:e59551 10.1371/journal.pone.0059551 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Shah ASV, Stelzle D, Lee KK, Beck EJ, Alam S, Clifford S, et al. Global Burden of Atherosclerotic Cardiovascular Disease in People Living with the Human Immunodeficiency Virus: A Systematic Review and Meta-Analysis. Circulation. 2018. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Bukrinsky M, Sviridov D. HIV and cardiovascular disease: contribution of HIV-infected macrophages to development of atherosclerosis. PLoS Med. 2007;4:e43 10.1371/journal.pmed.0040043 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Mujawar Z, Rose H, Morrow MP, Pushkarsky T, Dubrovsky L, Mukhamedova N, et al. Human Immunodeficiency Virus Impairs Reverse Cholesterol Transport from Macrophages. PLoS Biol. 2006;4:e365 10.1371/journal.pbio.0040365 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Cui HL, Ditiatkovski M, Kesani R, Bobryshev YV, Liu Y, Geyer M, et al. HIV protein Nef causes dyslipidemia and formation of foam cells in mouse models of atherosclerosis. FASEB J. 2014;28:2828–2839. 10.1096/fj.13-246876 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Saumoy M, Alonso-Villaverde C, Navarro A, Olmo M, Vila R, Maria Ramon J, et al. Randomized trial of a multidisciplinary lifestyle intervention in HIV-infected patients with moderate-high cardiovascular risk. Atherosclerosis. 2016;246:301–308. 10.1016/j.atherosclerosis.2016.01.014 [DOI] [PubMed] [Google Scholar]
  • 14.van Wijk JP, de Koning EJ, Cabezas MC, Joven J, op't Roodt J, Rabelink TJ, et al. Functional and structural markers of atherosclerosis in human immunodeficiency virus-infected patients. J Am Coll Cardiol. 2006;47:1117–1123. 10.1016/j.jacc.2005.09.073 [DOI] [PubMed] [Google Scholar]
  • 15.Lorenz MW, Stephan C, Harmjanz A, Staszewski S, Buehler A, Bickel M, et al. Both long-term HIV infection and highly active antiretroviral therapy are independent risk factors for early carotid atherosclerosis. Atherosclerosis. 2008;196:720–726. 10.1016/j.atherosclerosis.2006.12.022 [DOI] [PubMed] [Google Scholar]
  • 16.Grunfeld C, Delaney JA, Wanke C, Currier JS, Scherzer R, Biggs ML, et al. Preclinical atherosclerosis due to HIV infection: carotid intima-medial thickness measurements from the FRAM study. Aids. 2009;23:1841–1849. 10.1097/QAD.0b013e32832d3b85 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Holloway CJ, Ntusi N, Suttie J, Mahmod M, Wainwright E, Clutton G, et al. Comprehensive Cardiac Magnetic Resonance Imaging and Spectroscopy Reveal a High Burden of Myocardial Disease in HIV Patients. Circulation. 2013;128:814–822. 10.1161/CIRCULATIONAHA.113.001719 [DOI] [PubMed] [Google Scholar]
  • 18.Tseng ZH, Secemsky EA, Dowdy D, Vittinghoff E, Moyers B, Wong JK, et al. Sudden Cardiac Death in Patients With Human Immunodeficiency Virus Infection. J Am Coll Cardiol. 2012;59:1891–1896. 10.1016/j.jacc.2012.02.024 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Hsue PY, Lo JC, Franklin A, Bolger AF, Martin JN, Deeks SG, et al. Progression of Atherosclerosis as Assessed by Carotid Intima-Media Thickness in Patients With HIV Infection. Circulation. 2004;109:1603–1608. 10.1161/01.CIR.0000124480.32233.8A [DOI] [PubMed] [Google Scholar]
  • 20.Mangili A, Polak JF, Skinner SC, Gerrior J, Sheehan H, Harrington A, et al. HIV infection and progression of carotid and coronary atherosclerosis: the CARE study. J Acquir Immune Defic Syndr. 2011;58:148–153. 10.1097/QAI.0b013e31822d4993 [DOI] [PubMed] [Google Scholar]
  • 21.Rose H, Low H, Dewar E, Bukrinsky M, Hoy J, Dart A, et al. The effect of HIV infection on atherosclerosis and lipoprotein metabolism: A one year prospective study. Atherosclerosis. 2013;229:206–211. 10.1016/j.atherosclerosis.2013.04.010 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Maggi P, Bellacosa C, Leone A, Volpe A, Ricci ED, Ladisa N, et al. Cardiovascular risk in advanced naïve HIV-infected patients starting antiretroviral therapy: Comparison of three different regimens—PREVALEAT II cohort. Atherosclerosis. 2017;263:398–404. 10.1016/j.atherosclerosis.2017.05.004 [DOI] [PubMed] [Google Scholar]
  • 23.Phillips AN, Carr A, Neuhaus J, Visnegarwala F, Prineas R, Burman WJ, et al. Interruption of antiretroviral therapy and risk of cardiovascular disease in persons with HIV-1 infection: exploratory analyses from the SMART trial. Antivir Ther. 2008;13:177–187. [DOI] [PubMed] [Google Scholar]
  • 24.Liang YL, Teede H, Kotsopoulos D, Shiel L, Cameron JD, Dart AM, et al. Non-invasive measurements of arterial structure and function: repeatability, interrelationships and trial sample size. Clin Sci (Lond). 1998;95:669–679. [DOI] [PubMed] [Google Scholar]
  • 25.Matthews DR, Hosker JP, Rudenski AS, Naylor BA, Treacher DF, Turner RC. Homeostasis model assessment: insulin resistance and beta-cell function from fasting plasma glucose and insulin concentrations in man. Diabetologia. 1985;28:412–419. [DOI] [PubMed] [Google Scholar]
  • 26.Hoang A, Drew BG, Low H, Remaley AT, Nestel P, Kingwell BA, et al. Mechanism of cholesterol efflux in humans after infusion of reconstituted high-density lipoprotein. Eur Heart J. 2012;33:657–665. 10.1093/eurheartj/ehr103 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Low H, Hoang A, Sviridov D. Cholesterol Efflux Assay. J Vis Exp. 2012:e3810 10.3791/3810 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Asahchop EL, Akinwumi SM, Branton WG, Fujiwara E, Gill MJ, Power C. Plasma microRNA profiling predicts HIV-associated neurocognitive disorder. AIDS. 2016;30:2021–2031. 10.1097/QAD.0000000000001160 [DOI] [PubMed] [Google Scholar]
  • 29.Low H, Cheng L, Di Yacovo M-S, Churchill MJ, Meikle P, Bukrinsky M, et al. Lipid metabolism in patients infected with Nef-deficient HIV-1 strain. Atherosclerosis. 2016;244:22–28. 10.1016/j.atherosclerosis.2015.10.103 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Oliviero U, Bonadies G, Apuzzi V, Foggia M, Bosso G, Nappa S, et al. Human immunodeficiency virus per se exerts atherogenic effects. Atherosclerosis. 2009;204:586–589. 10.1016/j.atherosclerosis.2008.10.012 [DOI] [PubMed] [Google Scholar]
  • 31.van den Oord SCH, Sijbrands EJG, ten Kate GL, van Klaveren D, van Domburg RT, van der Steen AFW, et al. Carotid intima-media thickness for cardiovascular risk assessment: Systematic review and meta-analysis. Atherosclerosis. 2013;228:1–11. 10.1016/j.atherosclerosis.2013.01.025 [DOI] [PubMed] [Google Scholar]
  • 32.Calza L, Colangeli V, Magistrelli E, Rossi N, Rosselli Del Turco E, Bussini L, et al. Prevalence of metabolic syndrome in HIV-infected patients naive to antiretroviral therapy or receiving a first-line treatment. HIV Clin Trials. 2017:1–8. [DOI] [PubMed] [Google Scholar]
  • 33.Worm SW, De Wit S, Weber R, Sabin CA, Reiss P, El-Sadr W, et al. Diabetes Mellitus, Preexisting Coronary Heart Disease, and the Risk of Subsequent Coronary Heart Disease Events in Patients Infected With Human Immunodeficiency Virus: The Data Collection on Adverse Events of Anti-HIV Drugs (D:A:D Study). Circulation. 2009;119:805–811. 10.1161/CIRCULATIONAHA.108.790857 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Crook MA, Mir N. Abnormal lipids and the acquired immunodeficiency syndrome: is there a problem and what should we do about it? Int J STD AIDS. 1999;10:353–356. [PubMed] [Google Scholar]
  • 35.Riddler SA, Smit E, Cole SR, Li R, Chmiel JS, Dobs A, et al. Impact of HIV Infection and HAART on Serum Lipids in Men. JAMA. 2003;289:2978–2982. 10.1001/jama.289.22.2978 [DOI] [PubMed] [Google Scholar]
  • 36.Stein JH, Klein MA, Bellehumeur JL, McBride PE, Wiebe DA, Otvos JD, et al. Use of Human Immunodeficiency Virus-1 Protease Inhibitors Is Associated With Atherogenic Lipoprotein Changes and Endothelial Dysfunction. Circulation. 2001;104:257–262. [DOI] [PubMed] [Google Scholar]
  • 37.El-Sadr W, Mullin C, Carr A, Gibert C, Rappoport C, Visnegarwala F, et al. Effects of HIV disease on lipid, glucose and insulin levels: results from a large antiretroviral-naive cohort. HIV Med. 2005;6:114–121. 10.1111/j.1468-1293.2005.00273.x [DOI] [PubMed] [Google Scholar]
  • 38.Rose H, Hoy J, Woolley I, Tchoua U, Bukrinsky M, Dart A, et al. HIV infection and high density lipoprotein metabolism. Atherosclerosis. 2008;199:79–86. 10.1016/j.atherosclerosis.2007.10.018 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Asztalos BF, Mujawar Z, Morrow MP, Grant A, Pushkarsky T, Wanke C, et al. Circulating Nef induces dyslipidemia in simian immunodeficiency virus-infected macaques by suppressing cholesterol efflux. J Infect Dis. 2010;202:614–623. 10.1086/654817 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Goedeke L, Rotllan N, Ramirez CM, Aranda JF, Canfran-Duque A, Araldi E, et al. miR-27b inhibits LDLR and ABCA1 expression but does not influence plasma and hepatic lipid levels in mice. Atherosclerosis. 2015;243:499–509. 10.1016/j.atherosclerosis.2015.09.033 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Zhang M, Wu J-F, Chen W-J, Tang S-L, Mo Z-C, Tang Y-Y, et al. MicroRNA-27a/b regulates cellular cholesterol efflux, influx and esterification/hydrolysis in THP-1 macrophages. Atherosclerosis. 2014;234:54–64. 10.1016/j.atherosclerosis.2014.02.008 [DOI] [PubMed] [Google Scholar]
  • 42.Sun L, Trajkovski M. MiR-27 orchestrates the transcriptional regulation of brown adipogenesis. Metabolism. 2014;63:272–282. 10.1016/j.metabol.2013.10.004 [DOI] [PubMed] [Google Scholar]
  • 43.Maharaj NR, Ramkaran P, Pillay S, Chuturgoon AA. microRNA-27a rs895819 is associated with obesity in HIV infected preeclamptic Black South African women on HAART. BMC Med Genet. 2016;17:92 10.1186/s12881-016-0353-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Sun C, Alkhoury K, Wang YI, Foster GA, Radecke CE, Tam K, et al. IRF-1 and miRNA126 Modulate VCAM-1 Expression in Response to a High-Fat Meal / Novelty and Significance. Circ Res. 2012;111:1054–1064. 10.1161/CIRCRESAHA.112.270314 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 45.Schober A, Nazari-Jahantigh M, Wei Y, Bidzhekov K, Gremse F, Grommes J, et al. MicroRNA-126-5p promotes endothelial proliferation and limits atherosclerosis by suppressing Dlk1. Nat Med. 2014;20:368–376. 10.1038/nm.3487 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 46.Huang J, Zhu L, Qiu C, Xu X, Zhang L, Ding X, et al. MicroRNA miR-126-5p Enhances the Inflammatory Responses of Monocytes to Lipopolysaccharide Stimulation by Suppressing Cylindromatosis in Chronic HIV-1 Infection. J Virol. 2017;91. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47.Collares CV, Evangelista AF, Xavier DJ, Rassi DM, Arns T, Foss-Freitas MC, et al. Identifying common and specific microRNAs expressed in peripheral blood mononuclear cell of type 1, type 2, and gestational diabetes mellitus patients. BMC Res Notes. 2013;6:491 10.1186/1756-0500-6-491 [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

S1 File. Database 1.

Sequencing database of miRNA in plasma of study subjects.

(XLSX)

S2 File. Database 2.

Manuscript data.

(XLSX)

Data Availability Statement

All relevant data are within the manuscript and its Supporting Information files.


Articles from PLoS ONE are provided here courtesy of PLOS

RESOURCES