Skip to main content
The Journal of Infectious Diseases logoLink to The Journal of Infectious Diseases
. 2015 Feb 26;212(4):585–595. doi: 10.1093/infdis/jiv123

Factors Associated With Plasma IL-6 Levels During HIV Infection

Álvaro H Borges 1, Jemma L O'Connor 2, Andrew N Phillips 2, Frederikke F Rönsholt 3, Sarah Pett 2,4,5, Michael J Vjecha 6, Martyn A French 7,8, Jens D Lundgren 1, for the INSIGHT SMART and ESPRIT Study Groups and the SILCAAT Scientific Committee
PMCID: PMC4598808  PMID: 25722296

Abstract

Background. Elevated interleukin 6 (IL-6) levels have been linked to cardiovascular disease, cancer and death. Persons with human immunodeficiency virus (HIV) infection receiving treatment have higher IL-6 levels, but few data are available on factors associated with circulating IL-6.

Methods. Participants in 3 trials with IL-6 measured at baseline were included (N = 9864). Factors associated with IL-6 were identified by linear regression. Demographic and HIV variables (nadir/entry CD4+ cell count, HIV RNA level, antiretroviral therapy regimen) were investigated in all 3 trials. In the SMART (Strategies for Management of Anti-Retroviral Therapy) trial, CD4/CD8 ratio, smoking, comorbid conditions, serum lipids, renal function (estimated glomerular filtration rate [eGFR]), and educational level were assessed.

Results. Demographics associated with higher IL-6 levels were older age and lower education, whereas black race was associated with lower IL-6. Higher HIV RNA levels were associated with higher IL-6 levels, and higher nadir CD4+ cell counts with lower IL-6 levels. Compared with efavirenz, protease inhibitors were associated with higher and nevirapine with lower IL-6 levels. Smoking and all comorbid conditions were related to higher IL-6. IL-6 levels increased with decreasing eGFR and decreasing serum lipids.

Conclusions. Higher levels of IL-6 were associated with older age, nonblack race, higher body mass index, lower serum lipid levels, HIV replication, low nadir CD4+ cell count, protease inhibitor use, comorbid conditions, and decreased eGFR. Multiple factors affect inflammation in HIV and should be considered in studies of IL-6 as a biomarker of clinical outcomes.

Keywords: HIV, IL-6, inflammation, cancer, cardiovascular disease


Interleukin 6 (IL-6) is a proinflammatory cytokine that regulates various physiological processes [1, 2]. It plays a key role in the acute phase response [1] and in the transition from acute to chronic inflammation [3]. Evidence has accrued to suggest that dysregulation of IL-6 production is a major contributor to the pathogenesis of chronic inflammatory and autoimmune diseases [2, 4].

Human immunodeficiency virus (HIV) infection has long been shown to induce expression and secretion of IL-6 [5] by monocytes and macrophages [6]. Even when virologically suppressed, treated HIV-infected persons have significantly higher plasma levels of IL-6 than well-matched uninfected controls [7, 8]. Activated inflammation, as demonstrated by persistently higher IL-6 levels, may have profound and far-reaching clinical implications. Recent reports involving both HIV-infected and HIV-uninfected persons have linked increased plasma IL-6 levels to a variety of adverse clinical outcomes, including anemia [9, 10], cancer [11, 12], cardiovascular disease [1315], and death [16, 17].

The reasons HIV infection is associated with a chronic inflammatory state are not entirely understood. In a random sample of 499 individuals participating in the Strategies for Management of Anti-Retroviral Therapy (SMART) trial [17], older age and higher body mass index (BMI) were independently associated with elevated IL-6 levels. Among SMART participants coinfected with hepatitis C virus (HCV), IL-6 levels were particularly higher in those with preexisting hepatic fibrosis, as defined by abnormally elevated levels of the liver fibrosis marker hyaluronic acid [18]. Identifying additional factors associated with higher IL-6 levels could elucidate the mechanisms driving inflammation during HIV infection. The main purpose of the present study was to identify factors independently associated with plasma levels of IL-6 among a large number of HIV-infected individuals participating in 3 international HIV trials.

MATERIAL AND METHODS

This is a cross-sectional study involving HIV-infected persons participating in 3 randomized controlled trials conducted by the International Network for Strategic Initiatives in Global HIV Trials: the SMART (NCT00027352) [19], ESPRIT (Evaluation of Subcutaneous Proleukin in a Randomized International Trial; NCT00004978), and SILCAAT (Subcutaneous Recombinant, Human Interleukin-2 in HIV-Infected Patients With Low CD4+ Counts Under Active Antiretroviral Therapy; NCT00013611) [20] trials, which have been described in detail elsewhere. The SMART, ESPRIT, and SILCAAT studies, including the consent for stored specimens, were approved by the institutional review board or ethics committee of each clinical site and the University of Minnesota. Written informed consent was obtained from all participants involved in the 3 trials.

Biomarker Measurements

Biomarkers were measured at baseline, before randomization, in all participants who consented to storing blood samples for future research (N = 9864). For SMART participants, measurements were performed at the Laboratory for Clinical Biochemistry Research at the University of Vermont in Burlington. In ESPRIT and SILCAAT participants, laboratory measurements were performed by SAIC-Frederick. IL-6 was measured with the same method at both laboratories (Chemiluminescent Sandwich ELISA; R&D Systems).

In the SMART trial, D-dimer levels were measured using immunoturbidometric methods with the Sta-R analyzer (Liatest D-DI; Diagnostic Stago) and high-sensitivity C-reactive protein (hsCRP) was measured with an NBTMII nephelometer, N Antiserum to Human CRP (Siemens Diagnostics). In the ESPRIT and SILCAAT trials, D-dimer and hsCRP were both measured using an enzyme-linked fluorescent assay (VIDAS instrument [bioMerieux] for D-dimer and ELISA system [R&D Systems] for hsCRP). As explained in detail elsewhere [15, 21], the assays used to measure D-dimer and hsCRP, while different, compared well in duplicates. The median plasma levels measured at each laboratory were comparable and well correlated [15].

SMART and ESPRIT participants were screened for coinfection with hepatitis B virus (HBV) or HCV, as described elsewhere [22, 23]. Briefly, baseline plasma samples obtained from individuals with antibody tests positive for HBV and HCV were analyzed for levels of HCV RNA and HBV DNA using branched DNA assays (Versant HCV RNA 3.0 and Versant HBV DNA 3.0, respectively; Bayer Diagnostics). Participants with positive HBV/HCV antibody and viral load test results were considered to have hepatitis virus coinfection. Baseline levels of hyaluronic acid, a validated marker of hepatic fibrosis, were measured in coinfected patients with an enzyme-linked binding protein assay (Corgenix) [23]. The estimated glomerular filtration rate (eGFR) was calculated using the chronic kidney disease-epidemiology collaboration equation [24] in ESPRIT and SMART participants. Leevels of triglycerides, total cholesterol, low-density lipoprotein cholesterol (LDL-C), and high-density lipoprotein cholesterol (HDL-C) were measured in the SMART trial by Quest Diagnostics, using standard enzymatic methods.

Statistical Analyses

Factors independently associated with IL-6 levels were identified by multivariable linear regression models. Because the IL-6 distribution was skewed to the right, log2-transformed IL-6 levels were used as the dependent variable. Back-transformed exponentiated estimates (ie, corresponding to fold differences in IL-6 levels per unit or category difference in the covariates) with 95% confidence intervals (CIs) were calculated to assess the contribution of baseline variables to the variance of IL-6. To account for interlaboratory variability and inherent differences among participants in each trial, all models were adjusted for study type. Goodness of fit was assessed using the adjusted R2 coefficient, and multicollinearity was assessed using variance inflation factors.

The variables investigated were chosen on the basis of their epidemiological importance and biological plausibility. Because the variables of interest were not routinely collected in all 3 trials, models were fitted to 3 data sets: (1) SMART, ESPRIT, and SILCAAT participants (N = 9864): age, sex, ethnicity, BMI, CD4+ cell counts (nadir and baseline), markers of inflammation (hsCRP) and activated coagulation (D-dimer), antiretroviral therapy (ART) use, and ART regimens; (2) SMART and ESPRIT participants (n = 6938): comorbid conditions (HBV or HCV infection, diabetes mellitus, cardiovascular disease) and renal function (eGFR); and (3) SMART participants only (n = 4498): smoking, level of education (less than high school, high school, and bachelor's degree or above), triglyceride levels, and cholesterol levels (LDL-C and HDL-C). Among the SMART participants with CD8+ T-cell counts available (n = 2399), we also investigated the association between the CD4/CD8 ratio and IL-6 levels.

Because the 3 trials involved participants with different baseline characteristics, we investigated interactions between studies (SMART, ESPRIT, and SILCAAT) and demographic characteristics (sex, race, and age). We also investigated interactions between age and HIV RNA, between age and biomarkers (IL-6, hsCRP, and D-dimer), and between sex and BMI.

Additional preplanned analyses were performed, stratified by sex, HIV RNA (<500, 501–30 000; 30 001–100 000, or >100 000 copies/mL), and current ART use (yes or no) and regimen. These analyses were done to compare factors associated with IL-6 levels in men and women, in virologically suppressed and unsuppressed participants, and in those treated with different ART regimens. ART regimens were categorized into nonnucleoside reverse-transcriptase inhibitor (NNRTI)– and protease inhibitor (PI)–based regimens. PI-based regimens were further split into ritonavir-boosted and unboosted regimens. The relationship between IL-6 levels and specific NNRTI and PI drugs (ritonavir boosted or unboosted) was also analyzed. In these analyses, 2 covariates were fitted to model a participant's regimen: (1) the nucleoside/nucleotide reverse-transcriptase inhibitor (NRTI) backbone drugs and (2) the third drug (ie, NNRTI or PI) in the regimen. Only participants receiving NNRTI- or PI-based ART regimens (but not both drug classes together) were included in these analyses of specific drugs (n = 7291). Because of collinearity between ART use and HIV RNA, these variables could not be entered into the same models.

Although the impact of liver fibrosis on circulating IL-6 had been characterized in SMART participants with HCV coinfection [18], the relative contribution of replication of hepatitis viruses to variance in IL-6 levels had not been previously investigated among HBV- and HCV-coinfected persons, to our knowledge. We thus further adjusted models restricted to HBV- and HCV-coinfected persons for hyaluronic acid (measured at baseline in 245 study participants coinfected with HBV and 860 coinfected with HCV), HBV DNA, and HCV RNA levels.

RESULTS

Baseline characteristics of study participants are summarized separately for each of the 3 data sets included in the analyses and are presented in Table 1. There were marginally significant differences in terms of sex distribution across the 3 studies (P = .06 for interaction between study and sex), although it is unlikely that these differences were large enough to constitute clinical significance. There were significant interactions between study and race (P < .001) and between study and age (P < .001). The addition of an increasing number of covariates in multiple regression models on different data sets did not substantially change the power to estimate the variance of IL-6 levels (adjusted R2 values ranged from 0.23 to 0.27).

Table 1.

Baseline Characteristics by Data Set in ESPRIT, SILCAAT, and SMART Trial Participants

Characteristic SMART, ESPRIT, and SILCAAT (N = 9864) SMART and ESPRIT (n = 6938) SMART (n = 4498)
IL-6, median (IQR), pg/mL 1.80 (1.20–2.89) 1.81 (1.17–2.90) 1.72 (1.07–2.93)
Demographics
 Age, median (IQR), y 42 (36–49) 42 (36–49) 44 (38–50)
 Female sex, No. (%) 2150 (21.8) 1609 (23.2) 1147 (25.5)
 Black race, No. (%) 1913 (19.4) 1478 (21.3) 1250 (27.8)
 BMI, median (IQR), kg/m2 24.34 (22.12–27.00) 24.45 (22.15–27.30) 24.99 (22.50–28.09)
 Educational level, No. (%)a
  Less than high school NA NA 1138 (25.3)
  High school or some college/university NA NA 2550 (56.7)
  Bachelor's degree or above NA NA 810 (18.0)
HIV-specific variables
 Baseline CD4+ cell count, median (IQR), cells/mm3 490 (368–671) 540 (422–722) 601 (470–799)
 Nadir CD4+ cell count, median (IQR), cells/mm3 200 (84–316) 229 (121–335) 250 (154–358)
 Plasma HIV RNA ≤500 copies/mL, No. (%) 7536 (76.4) 5293 (76.3) 3297 (73.3)
 CD4/CD8 ratio, median (IQR)b NA NA 0.69 (0.48–0.98)
ART regimen, No. %
 None 824 (8.4) 701 (10.1) 683 (15.2)
 Other 1749 (17.7) 1200 (17.3) 683 (15.2)
 PI/r based 1261 (12.8) 1020 (14.7) 841 (18.7)
 PI basedc 2430 (24.6) 1311 (18.9) 589 (13.1)
  Amprenavir-fosamprenavir 137 (1.9) 91 (1.8) 60 (1.9)
  Atazanavir 267 (3.7) 244 (4.9) 240 (7.7)
  Indinavir 920 (12.6) 485 (9.6) 170 (5.4)
  Lopinavir 721 (9.9) 574 (11.4) 436 (13.9)
  Nelfinavir 1006 (13.8) 571 (11.3) 354 (11.3)
  Saquinavir 352 (4.8) 188 (3.7) 103 (3.3)
  Other/multiple PIs 288 (4.0) 179 (3.6) 65 (2.1)
 NNRTI based 3600 (36.5) 2699 (38.9) 1704 (37.9)
  Nevirapine 1505 (20.6) 1116 (22.2) 688 (22.0)
  Efavirenz 2082 (28.6) 1578 (31.4) 1011 (32.3)
  Other NNRTI 13 (0.2) 9 (0.2) 5 (0.2)
 NRTI backbonec
  Zidovudine + TTC 2841 (39.0) 1958 (38.9) 1203 (38.4)
  Abacavir + TTC 414 (5.7) 292 (5.8) 202 (6.5)
  Tenofovir + TTC 471 (6.5) 437 (8.7) 414 (13.2)
  Didanosine + TTC 240 (3.3) 171 (3.4) 114 (3.6)
  Stavudine + TTC 1667 (22.9) 1037 (20.6) 496 (15.8)
  Zidovudine + lamivudine + abacavir 181 (2.5) 167 (3.3) 147 (4.7)
  Other NRTI drugs 1451 (19.9) 955 (19.0) 543 (17.3)
  No NRTI 26 (0.4) 18 (0.4) 13 (0.4)
Biomarker of inflammation or activated coagulation, median (IQR), µg/mLd
 hsCRP 1.59 (0.70–3.67) 1.61 (0.71–3.76) 1.70 (0.71–4.07)
 D-dimer 0.24 (0.15–0.38) 0.22 (0.15–0.37) 0.20 (0.13–0.36)
Comorbid conditions, No. (%)
 Cardiovascular diseasee NA 187 (2.7) 162 (3.6)
 Diabetes mellituse NA 368 (5.3) 301 (6.7)
 Hepatitis Be NA 257 (3.7) 99 (2.2)
  Hyaluronic acid, median (IQR), ng/mLf NA 23.80 (14.63–43.69) 18.15 (12.33–32.18)
  HBV DNA, median (IQR), IU/mLf NA 71 704 (2000–100 000 000) 5251 (357–119 226 071)
 Hepatitis Ce NA 999 (14.4) 603 (13.4)
  Hyaluronic acid, median (IQR), ng/mLg NA 33.17 (18.75–59.82) 32.33 (19.95–59.82)
  HCV RNA, median (IQR), IU/mLg NA 2 576 804 (583 936–7 610 964) 579 088 (217 236–1 538 882)
 Smokinge NA NA 1822 (40.5)
Renal function, median (IQR)
 eGFR, mL/min per 1.73 m2e NA 111.56 (100.66–121.03) 110.82 (100.25–120.56)
 Cystatin C, mg/dLa NA NA 0.81(0.71–0.92)
Serum lipids, median (IQR), mg/dLa
 Cholesterol
  Total cholesterol NA NA 192 (164–222)
  LDL-C NA NA 112 (90–137)
  HDL-C NA NA 40 (33–51)
 Triglycerides NA NA 166 (107–265)

Abbreviations: ART, antiretroviral therapy; BMI, body mass index; eGFR, estimated glomerular filtration rate; ESPRIT, Evaluation of Subcutaneous Proleukin in a Randomized International Trial; HBV, hepatitis B virus; HCV, hepatitis C virus; HDL-C, high-density lipoprotein cholesterol; HIV, human immunodeficiency virus; hsCRP, high-sensitivity C-reactive protein; IL-6, interleukin 6; IQR, interquartile range; LDL-C, low-density lipoprotein cholesterol; NA, not available; NNRTI, nonnucleoside reverse-transcriptase inhibitors; NRTI, nucleoside/nucleotide reverse-transcriptase inhibitor; PI, protease inhibitor; PI/r, ritonavir-boosted PI; SILCAAT, Subcutaneous Recombinant, Human Interleukin-2 in HIV-Infected Patients With Low CD4+ Counts Under Active Antiretroviral Therapy; SMART, Strategies for Management of Anti-Retroviral Therapy; TTC, lamivudine or emtricitabine.

a Not ascertained for SILCAAT or ESPRIT participants.

b SMART participants with CD8 data (n = 2339).

c Including only those receiving a nucleoside/nucleotide backbone plus a third drug (PI or NNRTI). Percentages for the drug in question are for this subset of participants in each data set.

d hsCRP is the biomarker of inflammation, and D-dimer the biomarker of activated coagulation.

e Not ascertained for SILCAAT participants.

f Data available for 245 participants.

g Data available for 860 participants.

Demographics

The demographic factors found to be independently correlated with higher IL-6 levels were older age, higher BMI, and lower educational level (Figure 1). Overall, similar results were seen consistently across multiple models using different data sets and after adjustment for an increasing range of covariates. Black race, on the other hand, was found to be associated with higher IL-6 levels only in univariable analysis. In adjusted models, black race became associated with lower IL-6 levels. Men and women had similar IL-6 levels.

Figure 1.

Figure 1.

Fold differences in interleukin 6 (IL-6) levels associated with demographic factors. Fold differences associated with older age are expressed per 10 years older and fold differences associated with higher body mass index (BMI) are expressed per 5 kg/m2 higher. aSMART (Strategies for Management of Anti-Retroviral Therapy) participants only (n = 4498): adjusted as described in footnote b and also for comorbid conditions (hepatitis B and hepatitis C virus infection, diabetes mellitus, cardiovascular disease), renal function (estimated glomerular filtration rate), smoking status, educational level (less than high school, high school/some college, or bachelor's degree or above), and cholesterol levels (low- and high-density lipoprotein cholesterol); bSMART, ESPRIT (Evaluation of Subcutaneous Proleukin in a Randomized International Trial), and SILCAAT (Subcutaneous Recombinant, Human Interleukin-2 in HIV-Infected Patients With Low CD4+ Counts Under Active Antiretroviral Therapy) participants (N = 9864): models adjusted for age, sex, ethnicity, BMI, CD4+ cell counts (nadir and baseline), markers of inflammation (high-sensitivity C-reactive protein) and activated coagulation (D-dimer), and antiretroviral therapy (ART) use and regimens; cSMART, ESPRIT, and SILCAAT participants (N = 9864): univariable analysis. Abbreviations: CI, confidence interval; HIV, human immunodeficiency virus.

In analyses stratified by sex, higher BMI was associated with higher IL-6 levels in women but not in men (P for interaction, <.001).

HIV-Specific Variables

HIV replication was associated with significantly increased IL-6 levels. There was apparently a dose-response effect with increasing fold differences of IL-6 with higher HIV RNA plasma levels (Figure 2). No significant associations were seen between baseline CD4+ cell counts and IL-6 levels. We did find, however, a negative and independent correlation between nadir CD4+ cell counts and IL-6 levels: higher nadir CD4+ cell counts were associated with lower IL-6 levels. There was no evidence of an interaction between age and plasma HIV RNA levels (P = .89). Higher CD4/CD8 ratios ere associated with lower IL-6 levels in univariable analyses only, but there was no significant association after adjustment (Figure 2).

Figure 2.

Figure 2.

Fold differences in interleukin 6 (IL-6) levels associated with human immunodeficiency virus (HIV)–specific variables. Fold differences associated with the CD4/CD8 ratio are expressed per 1 log2 higher ratio; fold differences associated with higher baseline and nadir CD4+ cell counts are expressed per 100 cells/mm3 higher. aSMART (Strategies for Management of Anti-Retroviral Therapy), ESPRIT (Evaluation of Subcutaneous Proleukin in a Randomized International Trial), and SILCAAT (Subcutaneous Recombinant, Human Interleukin-2 in HIV-Infected Patients With Low CD4+ Counts Under Active Antiretroviral Therapy) participants receiving antiretroviral therapy (ART) at baseline (n = 9027): models adjusted for age, sex, ethnicity, body mass index (BMI), CD4+ cell counts (nadir and baseline), and markers of inflammation (high-sensitivity C-reactive protein [hsCRP]) and activated coagulation (D-dimer). Because of collinearity between ART use and HIV RNA, these variables could not be entered into the same models; bSMART, ESPRIT, and SILCAAT participants (N = 9864): models adjusted for age, sex, ethnicity, BMI, CD4+ cell counts (nadir and baseline), markers of inflammation (hsCRP) and activated coagulation (D-dimer), and ART use and regimens; cSMART, ESPRIT, and SILCAAT participants (N = 9864): univariable; dSMART participants with CD8 data (n = 2339): adjusted for age, sex, ethnicity, BMI, CD4+ cell counts (nadir and baseline), markers of inflammation (hsCRP) and activated coagulation (D-dimer), ART use and regimens, comorbid conditions (hepatitis B and hepatitis C virus infection, diabetes mellitus, cardiovascular disease), renal function (estimated glomerular filtration rate), smoking, and cholesterol levels (low- and high-density lipoprotein cholesterol); eSMART participants with CD8 data (n = 2339): univariable. Abbreviation: CI, confidence interval.

ART Regimens

Among participants receiving ART, PI-based regimens, ritonavir boosted or unboosted, were independently related to higher IL-6 levels when compared with NNRTI-based regimens (Figure 3). This was a drug class effect, and all individual PI drugs were associated with higher IL-6 levels when compared with efavirenz, although this association was not significant for atazanavir. Nevirapine, on the other hand, was associated with significantly lower IL-6 levels. Drugs composing the NRTI backbone were not associated with IL-6 levels. Participants not receiving ART at baseline had significantly increased IL-6 levels.

Figure 3.

Figure 3.

Fold differences in interleukin 6 (IL-6) levels associated with antiretroviral therapy (ART) and regimens and individual antiretroviral drugs. aSMART (Strategies for Management of Anti-Retroviral Therapy), ESPRIT (Evaluation of Subcutaneous Proleukin in a Randomized International Trial), and SILCAAT (Subcutaneous Recombinant, Human Interleukin-2 in HIV-Infected Patients With Low CD4+ Counts Under Active Antiretroviral Therapy) participants (N = 9864): models adjusted for age, sex, ethnicity, body mass index (BMI), CD4+ cell counts (nadir and baseline), markers of inflammation (high-sensitivity C-reactive protein) and activated coagulation (D-dimer); ART regimens were categorized as off ART, protease inhibitor (PI) based, ritonavir-boosted PI (PI/r) based, nonnucleoside reverse-transcriptase inhibitor (NNRTI) based, or other; bAdjusted as described in footnote a but including only participants receiving an NNRTI- or PI-based ART regimen (but not both) (SMART, ESPRIT, and SILCAAT participants; n = 7291); ART regimens were categorized as off ART, PI based (boosted and unboosted), NNRTI based, or other; cOther nucleoside/nucleotide reverse-transcriptase inhibitors (NRTIs) include adefovir (used in some hepatitis B virus–coinfected participants), zalcitabine, any of the drugs in our chosen NRTI pairs without lamivudine or emtricitabine, or multiple NRTIs, including a pair of NRTIs that are not among our chosen NRTI pairs. Other PIs include tipranavir, darunavir, and multiple PI drugs; other NNRTIs, delavirdine; other ART, ART regimens including NRTI drugs only. Abbreviations: CI, confidence interval; TTC, lamivudine or emtricitabine.

Biomarkers of Inflammation and Activated Coagulation

Levels of hsCRP and D-dimer were independently and positively correlated with IL-6 levels. In multivariable models involving participants from all 3 trials, higher levels of hsCRP (adjusted fold difference [95% CI], 1.16 [1.11–1.13] per 1 log2 µg/mL; P < .001) and D-dimer (1.12 [1.11–1.13] per 1 log2 µg/mL; P < .001) were associated with higher IL-6 levels. There was no evidence of an interaction between the effects of age and these biomarkers on IL-6 (P = .69 for hsCRP and P = .89 for D-dimer).

Comorbid Conditions and Smoking

Smoking and all the comorbid conditions investigated, namely, prior cardiovascular disease, diabetes mellitus, and HBV and HCV infection, were found to be associated with higher IL-6 levels in univariable and adjusted analyses (Figure 4). Comorbid conditions were associated with larger fold differences in IL-6 levels among virologically suppressed persons (HIV RNA, ≤500 copies/mL) than among those with HIV RNA levels >500 copies/mL (data not shown).

Figure 4.

Figure 4.

Fold differences in interleukin 6 (IL-6) levels associated with comorbid conditions and smoking. aSMART (Strategies for Management of Anti-Retroviral Therapy) and ESPRIT (Evaluation of Subcutaneous Proleukin in a Randomized International Trial) participants (n = 6938): models adjusted for age, sex, ethnicity, body mass index (BMI), CD4+ cell counts (nadir and baseline), markers of inflammation (high-sensitivity C-reactive protein [hsCRP]) and activated coagulation (D-dimer), antiretroviral therapy (ART) use and regimens, comorbid conditions (hepatitis B virus [HBV] and hepatitis C virus [HCV] infection, diabetes mellitus, cardiovascular disease), and renal function (estimated glomerular filtration rate [eGFR]); bSMART and ESPRIT participants (n = 6938): univariable; cSMART participants (n = 4498): models adjusted for age, sex, ethnicity, BMI, CD4+ cell counts (nadir and baseline), markers of inflammation (hsCRP) and activated coagulation (D-dimer), ART use and regimens, comorbid conditions (HBV and HCV infection, diabetes mellitus, cardiovascular disease), renal function (eGFR), smoking, educational level (less than high school, high school/some college and bachelor's degree or above), and cholesterol levels (low- and high-density lipoprotein cholesterol); dSMART participants (n = 4498): univariable. Abbreviation: CI, confidence interval.

Among HCV-coinfected participants, the degree of liver damage, as demonstrated by higher hyaluronic acid levels (fold difference [95% CI], 1.14 [1.11–1.18] per 1 log2 ng/mL; P < .001), but not HCV RNA plasma levels (1.00 [0.99–1.01] per 1 log2 IU/mL; P = .56), was found to be positively correlated with IL-6 levels. Among participants with HBV coinfection, on the other hand, neither hyaluronic acid levels (0.99 [0.97–1.01] per 1 log2 ng/mL; P = .45), nor HBV DNA (1.01 [1.00–1.02] per 1 log2 IU/mL; P = .18) were associated with lower IL-6 levels.

Renal Function and Serum Lipids

In adjusted analyses, higher eGFR values at baseline were found to be significantly associated with lower IL-6 levels (0.98 [0.97–1.00] per 10 mL/min per 1.73 m2; P = .01). We also observed a negative correlation between serum cholesterol levels and IL-6 levels: both higher LDL-C levels (0.99 [0.99–0.99] per 10 mg/dL, P < .001) and higher HDL-C levels (0.98 [0.96–0.99] per 10 mg/dL; P < .001) were associated with lower IL-6 levels. We found no association between triglyceride levels (1.01 [1.00–1.02] per 100 mg/dL; P = .24) and IL-6 levels in multivariable analyses.

DISCUSSION

In this study of almost 10 000 HIV-infected persons from across the globe, we identified demographic, viroimmunological, and clinical factors associated with plasma levels of IL-6. The factors associated with plasma IL-6 levels had been partially characterized in a small subset of randomly selected SMART participants [17] and among those with HCV coinfection [18], but there was still a need for systematic studies with adequate information on potential confounders involving large numbers of HIV-infected persons. We found that demographic factors, HIV RNA plasma levels, markers of inflammation and coagulation, smoking, comorbid conditions, renal function, and serum lipid levels were independently associated with IL-6 levels. A possible effect of ART regimen on plasma IL-6 levels was also observed. Taken together, these findings suggest that the inflammatory state seen during HIV infection has several possible causes. This enhanced inflammation, as demonstrated by elevated IL-6 plasma levels, is probably determined by a synergistically deleterious interaction of HIV-related and HIV-unrelated morbidity.

Congruent with previous reports [8, 17, 2527], we found that older age is associated with higher IL-6 levels, consistent with the view that the aging process is tightly integrated with inflammation [28]. The lower IL-6 levels that we observed among black participants may reflect, in part, the fact that circulating IL-6 levels are genetically determined [13, 29], although interactions between race and other contributing factors we were unable to capture and hence adjust for may play a role as well. The association we observed between lower education levels and increased IL-6 levels has not been reported before. Lower education levels may be associated with increased inflammation through indirect mechanisms involving poorer socioeconomic status and health status. Similarly to studies involving both HIV-infected persons [17, 30] and the general population [25, 31], we found BMI to be positively correlated with IL-6. Indeed, the subcutaneous adipose tissue can produce, in some instances, up to 25% of circulating IL-6 [32, 33]. The effect of BMI on IL-6 was significantly influenced by sex, which can be attributable to sexual dimorphism in body fat distribution [34] and percentage [35] that is not captured by BMI.

Among the HIV-specific variables that we investigated, we found that lower nadir CD4+ cell count and HIV RNA plasma levels, but not baseline CD4+ cell count or CD4/CD8 ratio, were related to higher IL-6 levels. This suggests that CD4+ cell depletion before ART initiation and ongoing HIV replication are more important drivers of inflammation than immunodeficiency. Other studies, however, have observed increased IL-6 levels in those with lower CD4+ cell counts [8, 26]. Inherent differences in patient cohorts and differences in variables adjusted for may explain these discrepant findings. Our study, however, is the largest investigation into factors associated with IL-6 published so far, and we had extensive information on confounding factors.

The lack of association between CD4/CD8 ratio and IL-6 levels suggests that this ratio is a poor surrogate for ongoing inflammation among the HIV-infected persons with relatively high CD4+ cells enrolled in SMART. This is in agreement with the report by Serrano-Villar and colleagues [36], in which no correlation between CD4/CD8 ratio and IL-6 was observed among individuals with CD4+ cell counts ≥500/mmm3.

Among participants receiving ART, we found that PI-based regimens were independently associated with higher IL-6 levels, whereas nevirapine use was associated with lower IL-6 levels, compared with efavirenz. These findings raise the possibility that specific ART regimens can influence inflammation, which may have important clinical implications. Although levels of inflammatory biomarkers were similar between persons initiating NNRTI- and PI-based ART in small subsets of randomized trials [37], switching away from PI-based regimens led to significant reductions in plasma levels of IL-6 [38]. Nevirapine-based regimens, on the other hand, were associated with significantly lower IL-6 levels in this study. NNRTI drugs, particularly nevirapine, have better penetration into body compartments than PIs [39], and the use of nevirapine is associated with lower plasma HIV RNA levels than the use of efavirenz or lopinavir-ritonavir in patients with HIV RNA levels <50 copies/mL when an assay with a lower limit of detection is used [40]. Our findings, however, should be interpreted with caution. Because participants were not randomized to PI- or NNRTI-based regimes, we cannot rule out the possibility that the choice of a PI regimen could be a marker for certain participant characteristics, such as increased visceral adiposity [41], that may be associated with higher IL-6 levels. Future studies should address the question as to whether some ART regimens may suppress HIV-associated inflammation more than others.

In accordance with other studies, we found that smoking [8, 25, 34, 42], diabetes mellitus [43, 44], prior cardiovascular disease [13, 45], and hepatitis virus coinfection [26] were associated with increased IL-6. At least for cardiovascular disease, mendelian randomization studies suggest that the link may be causal [13]. Therefore, HIV-unrelated morbidity may contribute to enhanced inflammation during HIV infection. This effect was stronger among virologically suppressed individuals, which indicates that once HIV replication is suppressed by ART, the relative association of non-HIV comorbid conditions with inflammation becomes more prominent. Among participants with HCV coinfection, we observed that hepatic fibrosis, as demonstrated by elevated hyaluronic levels but not HCV RNA plasma levels, was independently associated with higher IL-6 levels. As others have already pointed out [18], this suggests that the enhanced proinflammatory state associated with HCV coinfection is mainly mediated by liver fibrosis. Hyaluronic acid, however, is not a perfect marker of hepatic fibrosis, and information on other measures of liver fibrosis found to be associated with IL-6 by other investigators (eg, fibrosis 4 score [46]) would have been helpful.

There was a negative correlation between eGFR and IL-6, indicating that poorer renal function is associated with increased inflammation, which may just reflect a proinflammatory state inherently associated with renal dysfunction [47]. We also observed a negative correlation between IL-6 and serum lipid levels. IL-6 infusions have been shown to reduce cholesterol levels [48], and a polymorphism of the IL-6 gene associated with higher IL-6 circulating levels and lower serum lipids has been described [29].

Our study had several limitations. First, its cross-sectional design and the lack of HIV-uninfected controls hampered our ability to study the temporal direction of associations and to infer causality. Second, although the assays used to measure biomarkers compared well and the main associations that we observed held within large sub–data sets using the same assay, there may have been residual confounding due to different assays used. Third, serum levels of inflammatory markers may not accurately reflect end-organ inflammation and damage. Fourth, only a single biomarker was measured; however, other studies of SMART participants have shown that no other cytokine or chemokine predicts mortality better than IL-6 [49]. Fifth, we did not have information on hormonal replacement therapy, physical exercise, and alcohol use, which were shown to be significantly associated with IL-6 levels in HIV-uninfected women [25]. Finally, structural equation modeling (SEM) can test models with multiple dependent variables using several regression equations simultaneously [50] and is a more robust technique than multivariable linear regression, which we used in this study. Through causal modeling or path-analysis methods, SEM can deal with nonnormally distributed and autocorrelated data and would have allowed us to make more causal inferences from our data set. A reanalysis of factors associated with IL-6 levels among HIV-infected persons using SEM is therefore warranted.

To conclude, higher IL-6 levels in HIV-infected persons were associated with older age, nonblack race, higher BMI, lower serum lipid levels, ongoing HIV replication, low nadir CD4+ cell counts, comorbid conditions, and decreased renal function. Furthermore, PI-based ART regimens were associated with higher IL-6 levels, whereas nevirapine-based regimens were associated with lower IL-6 levels. These findings suggest that there are multiple determinants of inflammation and/or IL-6 production during HIV infection. These determinants should be considered when investigating IL-6 as a biomarker of clinical outcomes in HIV-infected persons.

Notes

Acknowledgments. We would like to acknowledge the SMART, ESPRIT, and SILCAAT participants and investigators (see reference 19 for the complete list of SMART investigators and reference 20 for the complete list of ESPRIT and SILCAAT investigators). We are also indebted to Lars Peters, Centre for Health & Infectious Diseases Research, for critically reading the manuscript.

Author contributions. A. H. B., J. L. O., A. N. P., and J. D. L. conceived the study; J. L. O. and A. N. P. performed all statistical analyses; and A. H. B. drafted the manuscript. All authors contributed to data interpretation, critically revised the manuscript, and approved the final version.

Disclaimer. The funders had no role in study design, data collection and analysis, the decision to publish, or preparation of this manuscript.

Financial support. This study was funded by the National Institutes of Health (grants U01AI46957 and U01AI068641 [ESPRIT and SMART] and U01AI042170 and U01AI46362 [SMART]). SILCAAT was supported by grants from Chiron and Novartis. This project was also supported by the Danish National Research Foundation and from the Research Council at Rigshospitalet (grant DNRF126).

Potential conflicts of interest. All authors: No potential conflicts of interest.

All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.

References

  • 1.Heinrich PC, Castell JV, Andus T. Interleukin-6 and the acute phase response. Biochem J 1990; 265:621–36. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Tanaka T, Kishimoto T. The biology and medical implications of interleukin-6. Cancer Immunol Res 2014; 2:288–94. [DOI] [PubMed] [Google Scholar]
  • 3.Kaplanski G, Marin V, Montero-Julian F, Mantovani A, Farnarier C. IL-6: a regulator of the transition from neutrophil to monocyte recruitment during inflammation. Trends Immunol 2003; 24:25–9. [DOI] [PubMed] [Google Scholar]
  • 4.Atreya R, Mudter J, Finotto S, et al. Blockade of interleukin 6 trans signaling suppresses T-cell resistance against apoptosis in chronic intestinal inflammation: evidence in Crohn disease and experimental colitis in vivo. Nat Med 2000; 6:583–8. [DOI] [PubMed] [Google Scholar]
  • 5.Nakajima K, Martínez-Maza O, Hirano T, et al. Induction of IL-6 (B cell stimulatory factor-2/IFN-beta 2) production by HIV. J Immunol 1989; 142:531–6. [PubMed] [Google Scholar]
  • 6.Breen EC, Rezai AR, Nakajima K, et al. Infection with HIV is associated with elevated IL-6 levels and production. J Immunol 1990; 144:480–4. [PubMed] [Google Scholar]
  • 7.Neuhaus J, Jacobs DR, Jr, Baker JV, et al. Markers of inflammation, coagulation, and renal function are elevated in adults with HIV infection. J Infect Dis 2010; 201:1788–95. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Armah KA, McGinnis K, Baker J, et al. HIV status, burden of comorbid disease, and biomarkers of inflammation, altered coagulation, and monocyte activation. Clin Infect Dis 2012; 55:126–36. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Raj DS. Role of interleukin-6 in the anemia of chronic disease. Semin Arthritis Rheum 2009; 38:382–8. [DOI] [PubMed] [Google Scholar]
  • 10.Borges AH, Weitz JI, Collins G, et al. Markers of inflammation and activation of coagulation are associated with anaemia in antiretroviral-treated HIV disease. AIDS 2014; 28:1791–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Grivennikov SI, Karin M. Inflammatory cytokines in cancer: tumour necrosis factor and interleukin 6 take the stage. Ann Rheum Dis 2011; 70:104–8. [DOI] [PubMed] [Google Scholar]
  • 12.Borges ÁH, Silverberg MJ, Wentworth D, et al. Predicting risk of cancer during HIV infection: the role of inflammatory and coagulation biomarkers. AIDS 2013; 27:1433–41. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Interleukin-6 Receptor Mendelian Randomisation Analysis (IL6R MR) Consortium Hingorani AD, Casas JP. The interleukin-6 receptor as a target for prevention of coronary heart disease: a mendelian randomisation analysis. Lancet 2012; 379:1214–24. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Duprez DA, Neuhaus J, Kuller LH, et al. Inflammation, coagulation and cardiovascular disease in HIV-infected individuals. PLoS One 2012; 7:e44454. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Nordell AD, McKenna M, Borges ÁH, Duprez D, Neuhaus J, Neaton JD; INSIGHT SMART, ESPRIT Study Groups; SILCAAT Scientific Committee. Severity of cardiovascular disease outcomes among patients with HIV is related to markers of inflammation and coagulation. J Am Heart Assoc 2014; 3:e000844. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Harris TB, Ferrucci L, Tracy RP, et al. Associations of elevated interleukin-6 and C-reactive protein levels with mortality in the elderly. Am J Med 1999; 106:506–12. [DOI] [PubMed] [Google Scholar]
  • 17.Kuller LH, Tracy R, Belloso W, et al. Inflammatory and coagulation biomarkers and mortality in patients with HIV infection. PLoS Med 2008; 5:e203. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Peters L, Neuhaus J, Duprez D, et al. Biomarkers of inflammation, coagulation and microbial translocation in HIV/HCV co-infected patients in the SMART study. J Clin Virol 2014; 60:295–300. [DOI] [PubMed] [Google Scholar]
  • 19.El-Sadr WM, Lundgren JD, Neaton JD, et al. CD4+ count–guided interruption of antiretroviral treatment. N Engl J Med 2006; 355:2283–96. [DOI] [PubMed] [Google Scholar]
  • 20.Abrams D, Lévy Y, Losso MH, et al. Interleukin-2 therapy in patients with HIV infection. N Engl J Med 2009; 361:1548–59. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Borges AH, O'Connor JL, Phillips AN, et al. Factors associated with D-dimer levels in HIV-infected individuals. PLoS One 2014; 9:e90978. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Tedaldi E, Peters L, Neuhaus J, et al. Opportunistic disease and mortality in patients coinfected with hepatitis B or C virus in the Strategic Management of Antiretroviral Therapy (SMART) study. Clin Infect Dis 2008; 47:1468–75. [DOI] [PubMed] [Google Scholar]
  • 23.Peters L, Neuhaus J, Mocroft A, et al. Hyaluronic acid levels predict increased risk of non-AIDS death in hepatitis-coinfected persons interrupting antiretroviral therapy in the SMART Study. Antivir Ther 2011; 16:667–75. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Levey AS, Stevens LA, Schmid CH, et al. A new equation to estimate glomerular filtration rate. Ann Intern Med 2009; 150:604–12. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Bermudez EA, Rifai N, Buring J, Manson JE, Ridker PM. Interrelationships among circulating interleukin-6, C-reactive protein, and traditional cardiovascular risk factors in women. Arterioscler Thromb Vasc Biol 2002; 22:1668–73. [DOI] [PubMed] [Google Scholar]
  • 26.Salter ML, Lau B, Mehta SH, Go VF, Leng S, Kirk GD. Correlates of elevated interleukin-6 and C-reactive protein in persons with or at high risk for HCV and HIV infections. J Acquir Immune Defic Syndr 2013; 64:488–95. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Lacerda HR, Falcão Mda C, de Albuquerque VM, et al. Association of inflammatory cytokines and endothelial adhesion molecules with immunological, virological, and cardiometabolic disease in HIV-infected individuals. J Interferon Cytokine Res 2014; 34:385–93. [DOI] [PubMed] [Google Scholar]
  • 28.Deeks SG. HIV infection, inflammation, immunosenescence, and aging. Annu Rev Med 2011; 62:141–55. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Fernández-Real JM, Broch M, Vendrell J, Richart C, Ricart W. Interleukin-6 gene polymorphism and lipid abnormalities in healthy subjects. J Clin Endocrinol Metab 2000; 85:1334–9. [DOI] [PubMed] [Google Scholar]
  • 30.Koethe JR, Dee K, Bian A, et al. Circulating interleukin-6, soluble CD14, and other inflammation biomarker levels differ between obese and nonobese HIV-infected adults on antiretroviral therapy. AIDS Res Hum Retroviruses 2013; 29:1019–25. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Straub RH, Hense HW, Andus T, Schölmerich J, Riegger GA, Schunkert H. Hormone replacement therapy and interrelation between serum interleukin-6 and body mass index in postmenopausal women: a population-based study. J Clin Endocrinol Metab 2000; 85:1340–4. [DOI] [PubMed] [Google Scholar]
  • 32.Mohamed-Ali V, Goodrick S, Rawesh A, et al. Subcutaneous adipose tissue releases interleukin-6, but not tumor necrosis factor-alpha, in vivo. J Clin Endocrinol Metab 1997; 82:4196–200. [DOI] [PubMed] [Google Scholar]
  • 33.Pal M, Febbraio MA, Whitham M. From cytokine to myokine: the emerging role of interleukin-6 in metabolic regulation. Immunol Cell Biol 2014; 92:331–9. [DOI] [PubMed] [Google Scholar]
  • 34.Fernandez-Real JM, Vayreda M, Richart C, et al. Circulating interleukin 6 levels, blood pressure, and insulin sensitivity in apparently healthy men and women. J Clin Endocrinol Metab 2001; 86:1154–9. [DOI] [PubMed] [Google Scholar]
  • 35.Peters MJ, Ghouri N, McKeigue P, Forouhi NG, Sattar N. Circulating IL-6 concentrations and associated anthropometric and metabolic parameters in South Asian men and women in comparison to European whites. Cytokine 2013; 61:29–32. [DOI] [PubMed] [Google Scholar]
  • 36.Serrano-Villar S, Sainz T, Lee SA, et al. HIV-infected individuals with low CD4/CD8 ratio despite effective antiretroviral therapy exhibit altered T cell subsets, heightened CD8+ T cell activation, and increased risk of non-AIDS morbidity and mortality. PLoS Pathog 2014; 10:e1004078. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.McComsey GA, Kitch D, Daar ES, et al. Inflammation markers after randomization to abacavir/lamivudine or tenofovir/emtricitabine with efavirenz or atazanavir/ritonavir. AIDS 2012; 26:1371–85. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Martínez E, D'Albuquerque PM, Llibre JM, et al. Changes in cardiovascular biomarkers in HIV-infected patients switching from ritonavir-boosted protease inhibitors to raltegravir. AIDS 2012; 26:2315–26. [DOI] [PubMed] [Google Scholar]
  • 39.Antinori A, Perno CF, Giancola ML, et al. Efficacy of cerebrospinal fluid (CSF)-penetrating antiretroviral drugs against HIV in the neurological compartment: different patterns of phenotypic resistance in CSF and plasma. Clin Infect Dis 2005; 41:1787–93. [DOI] [PubMed] [Google Scholar]
  • 40.Bonora S, Nicastri E, Calcagno A, et al. Ultrasensitive assessment of residual HIV viraemia in HAART-treated patients with persistently undetectable plasma HIV-RNA: a cross-sectional evaluation. J Med Virol 2009; 81:400–5. [DOI] [PubMed] [Google Scholar]
  • 41.Curran A, Martinez E, Saumoy M, et al. Body composition changes after switching from protease inhibitors to raltegravir: SPIRAL-LIP substudy. AIDS 2012; 26:475–81. [DOI] [PubMed] [Google Scholar]
  • 42.Tappia PS, Troughton KL, Langley-Evans SC, Grimble RF. Cigarette smoking influences cytokine production and antioxidant defences. Clin Sci 1995; 88:485–9. [DOI] [PubMed] [Google Scholar]
  • 43.Kado S, Nagase T, Nagata N. Circulating levels of interleukin-6, its soluble receptor and interleukin-6/interleukin-6 receptor complexes in patients with type 2 diabetes mellitus. Acta Diabetol 1999; 36:67–72. [DOI] [PubMed] [Google Scholar]
  • 44.Pradhan AD, Manson JE, Rifai N, Buring JE, Ridker PM. C-reactive protein, interleukin 6, and risk of developing type 2 diabetes mellitus. JAMA 2001; 286:327–34. [DOI] [PubMed] [Google Scholar]
  • 45.Liuzzo G, Biasucci LM, Gallimore JR, et al. Enhanced inflammatory response in patients with pre infarction unstable angina. J Am Coll Cardiol 1999; 34:1696–703. [DOI] [PubMed] [Google Scholar]
  • 46.Fuster D, Tsui JI, Cheng DM, et al. Interleukin-6 is associated with noninvasive markers of liver fibrosis in HIV-infected patients with alcohol problems. AIDS Res Hum Retroviruses 2013; 29:1110–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47.Shlipak MG, Fried LF, Crump C, et al. Elevations of inflammatory and procoagulant biomarkers in elderly persons with renal insufficiency. Circulation 2003; 107:87–92. [DOI] [PubMed] [Google Scholar]
  • 48.Veldhuis GJ, Willemse PH, Sleijfer DT, et al. Toxicity and efficacy of escalating dosages of recombinant human interleukin-6 after chemotherapy in patients with breast cancer or non-small-cell lung cancer. J Clin Oncol 1995; 13:2585–93. [DOI] [PubMed] [Google Scholar]
  • 49.French MA, Cozzi-Lepri A, Arduino RC, Johnson M, Achhra AC, Landay A; for the INSIGHT SMART Study Group. Plasma levels of cytokines and chemokines and the risk of mortality in HIV-infected individuals: a case-control analysis nested in a large clinical trial. AIDS 2015; 29:847–51. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50.Kline RB. Principles and practice of structural equation modelling. 2nd ed New York, NY: Guilford Press, 2005. [Google Scholar]

Articles from The Journal of Infectious Diseases are provided here courtesy of Oxford University Press

RESOURCES