Abstract
Background
We assessed whether key biomarkers of endothelial activation and hemostasis/thrombosis were elevated in individuals receiving effective antiretroviral therapy (ART) in the year before ischemic stroke.
Methods
We conducted a case–control study nested in the CFAR Network of Integrated Clinical Systems cohort, comparing 42 adjudicated cases with ischemic stroke with 83 controls matched for ART regimen. Angiopoietin-1, angiopoietin-2, C-reactive protein, interleukin-6, plasminogen activation inhibitor–1, P-selectin, serum amyloid–A, soluble CD14, ICAM-1, VCAM-1, apolipoprotein A1, ADAMTS13, and von Willebrand factor (VWF) were measured in stored plasma collected before the stroke event. We used conditional logistic regression to identify associations with ischemic stroke, with and without adjustment for Atherosclerotic Cardiovascular Disease (ASCVD) and Veterans Aging Cohort Study (VACS) scores.
Results
After adjustment for age and sex, higher plasma viral load and higher angiopoeitin-2, soluble CD14, and VWF were associated with increased odds of ischemic stroke; higher nadir CD4 count was associated with decreased odds of ischemic stroke. VWF remained associated with subsequent ischemic stroke after adjustment for ASCVD score (adjusted odds, 1.74; 95% CI, 1.01–2.98 per log2 increment). In a separate model adjusting for VACS score, only VWF (adjusted odds, 1.80; 95% CI, 1.04–3.12 per log2 increment) was associated with subsequent ischemic stroke. In a sensitivity analysis excluding participants with viral load ≥400 copies/mL, associations between VWF and ischemic stroke were attenuated, with risk estimates ranging from 1.59 to 1.64 per log2 increment.
Conclusions
Endothelial activation and related release and attachment of VWF may play an important role in ischemic stroke among persons with treated HIV infection.
Keywords: endothelial activation, hemostasis/thrombosis, HIV infection, ischemic stroke, von Willebrand factor
While antiretroviral therapy (ART) has greatly decreased the risk of opportunistic infections among persons with HIV infection (PWH) and increased survival, there is growing recognition that PWH are at increased risk for cardiovascular disease (CVD), including ischemic stroke [1, 2]. Although the mechanistic pathways linking chronic HIV infection and long-term ART with stroke are not yet clear, mounting evidence points to a key role for chronic activation of inflammatory and hemostatic pathways [3–8]. Even prolonged effective ART may not normalize biomarkers of inflammation and coagulation [3], so a better understanding of the relationship between these pathways and the pathogenesis of HIV-associated CVD risk is urgently needed.
Endothelial activation, in particular, appears to be a critical link between immune activation, inflammation, thrombosis, and CVD in HIV infection [9]. Plasma biomarkers of endothelial activation, including soluble forms of intercellular adhesion molecule–1 (sICAM-1), vascular cell adhesion molecule–1 (sVCAM-1), and E-selectin, as well as the angiopoietin-2 to angiopoietin-1 (ANG-2:ANG-1) ratio, increase soon after HIV-1 acquisition [10], and elevated plasma sVCAM-1 and ANG-2 levels are associated with increased risk of HIV disease progression and death [11, 12]. Because endothelial cells are involved in many critical aspects of vascular biology, including barrier function, immune surveillance, inflammation, blood clotting, and atherosclerosis, ongoing endothelial activation likely accelerates development of CVD. For example, both sICAM-1 and sVCAM-1 have been implicated as biomarkers of symptomatic atherosclerotic plaque [13, 14].
Chronic endothelial activation is also accompanied by increased release and persistent attachment of von Willebrand factor (VWF) to the luminal surface of the endothelium, enabling platelets to adhere to the vessel wall, thereby promoting subclinical microangiopathy [12]. Elevations of VWF, especially of large multimers that represent its most adhesive forms, correlate with disease severity in several inflammatory conditions, including malaria [15], sickle cell disease [16], and prothrombotic conditions such as antiphospholipid syndrome [17]. Acute HIV infection increases plasma VWF levels, which correlate with plasma viral load, disease progression, and death [18–21]. Moreover, elevated plasma VWF levels have been associated with ischemic stroke in both PWH and populations uninfected by HIV [22–25]. We previously reported that total active VWF, a parameter that takes into account the amount and adhesive activity of VWF, and ADAMTS13, the protease that regulates adhesive activity by cleaving VWF, were positively correlated with plasma HIV-1 RNA levels in PWH [12].
The objective of the present study was to determine whether plasma biomarkers of endothelial activation (ANG-1, ANG-2, sICAM-1, sVCAM-1) and hemostasis/thrombosis (VWF antigen, ADAMTS13) are elevated among PWH receiving effective ART before the development of ischemic stroke. Our hypothesis was that endothelial activation and hemostasis/thrombosis biomarkers, including VWF levels, would be elevated in the pre-event plasma of ART-treated PWH who subsequently develop ischemic stroke relative to controls (ie, PWH who did not develop a subsequent stroke) selected from the same cohort who were matched for ART treatment regimen.
METHODS
Study Population
The CFAR Network of Integrated Clinical Systems (CNICS) is a national network of 8 HIV clinical sites that integrates data for ~30000 participating PWH from electronic health record systems and other sources into a single research database [26]. All patients in care at each site are eligible for CNICS enrollment, which corresponds to the date each patient began HIV care at the participating site. Because CNICS data reflect clinical practice, they are less subject to volunteer and nonresponse biases than data collected in traditional cohort studies [26]. The CNICS Data Management Core has established standards for data terminology, format, verification, and quality assurance for clinical diagnoses, laboratory results, and medication data. Each site received human subjects approval for CNICS [26].
Study Design
A matched case–control design was used to evaluate associations of the biomarkers studied with subsequent ischemic stroke. Cases were CNICS participants who had attained viral suppression (ie, <400 copies/mL) on ART after enrollment, experienced an ischemic stroke ≥6 months after ART initiation, and had stored plasma available within 12 months before the stroke event. We selected controls who had attained viral suppression on ART, had not experienced an ischemic stroke, and had stored plasma available following viral suppression. In addition, controls were matched to cases by ART regimen prescribed on the date of sampling. CNICS participants who had experienced a myocardial infarction after ART initiation were excluded. Two controls were identified for each case; in some cases, controls were used for >1 case. Participants were followed in CNICS between 1997 and 2015.
Primary Outcome
Ischemic stroke events were adjudicated using a state-of-the-art protocol based on Multi-Ethnic Study of Atherosclerosis (MESA) criteria [27, 28]. Briefly, for all potential events, sites assembled de-identified packets with physician notes, radiology reports, and procedure and laboratory results and uploaded them to a central web-based platform for review by 2 neurologists, followed by a third reviewer if discrepancies occurred. Reviewers categorized each stroke as ischemic vs hemorrhagic. Ischemic strokes were further classified by whether a predisposing factor (eg, infection, illicit drug use) was present when the stroke occurred and by ischemic stroke subtype (ie, cardioembolic, large vessel atherosclerosis, or small vessel) [29].
Biomarker Predictors
A 400-µL aliquot of stored plasma from all included participants was shipped from each participating CNICS site to Seattle on dry ice and stored at –80°C before testing. The Meso Scale Discovery (Rockville, MD, USA) immunoassay platform was used to measure concentrations of ANG-1, ANG-2, C-reactive protein (CRP), interleukin 6 (IL-6), plasminogen activation inhibitor–1, P-selectin, serum amyloid A (SAA), soluble CD14, sICAM-1, sVCAM-1, apolipoprotein A1, ADAMTS13, and VWF. Biomarkers were log2-transformed to normalize skewed data and increase biological relevance, as a log2 increase corresponds to doubling of concentration (and a log2 decrease corresponds to a 50% decrease in concentration). Biomarkers of HIV disease status were also evaluated as predictors, including most recent CD4 count and viral load, peak viral load, and CD4 count nadir.
Comorbidity and Health-Related Variables
Variables from the CNICS data repository included demographic characteristics (sex, age, and race/ethnicity) and clinical data including stroke risk factors, vital signs, and laboratory measures. Hypertension was defined as mean systolic blood pressure (BP) >140 mmHg or diastolic BP >90 mmHg in the previous 6 months or use of antihypertensive drugs [30]. Diabetes was defined as a hemoglobin A1c level >6.5% or use of a diabetes-specific medication such as insulin or a diabetes-related medication frequently but not exclusively used to treat diabetes (eg, biguanides) in the setting of a diabetes diagnosis [31]. To optimize power given the relatively small sample size, calculated risk scores were evaluated as potential confounders in separate models due to the overlap in score components. The Atherosclerotic Cardiovascular Disease (ASCVD) score, which predicts 10-year risk for atherosclerotic CVD [32], is based on sex, age, race, systolic blood pressure, hypertension treatment status, diabetes status, tobacco use, total cholesterol, and high-density lipoproteins. The Veterans Aging Cohort Study (VACS) index, which predicts both all-cause and cardiovascular mortality in PWH [33], is based on sex, age, race, hemoglobin, platelet count, creatinine, aspartate aminotransferase, alanine aminotransferase, hepatitis C status, CD4 count, and plasma viral load.
Data Analysis
To retain analytic power, missing data were imputed before regression analysis using multiple imputation by chained equations. Demographic and clinical data at the time of sample collection were summarized using descriptive statistics. Dot plots were created to visualize differences in the distribution of biomarker levels between stroke cases and matched controls. Pearson correlations were used to evaluate associations between biomarkers, with Bonferroni adjustment due to multiple comparisons. A heat map was generated to show the strength and direction of correlations.
Conditional logistic regression was performed to determine if individual biomarkers were independently associated with ischemic stroke in unadjusted analyses and after adjustment for age and sex (Model 1), for ASCVD score (Model 2), and for VACS score (Model 3). Odds ratios (ORs) from these analyses indicate the increased odds for each log2 increase in biomarker level, log10 increase in viral load or 100-cell/μL increase in CD4 count. Because some participants had viremia on the date of sampling despite attaining viral suppression after CNICS enrollment, we conducted a sensitivity analysis in which the sample was restricted to those individuals with a plasma viral load <400 copies/mL. The impact of adjustment for years on ART was evaluated for each analysis. Stata, version 14.2 (StataCorps, College Station, TX, USA), was used, and P values<.05 were considered significant.
RESULTS
Study Population
The study population consisted of 42 cases (ie, individuals with a subsequent ischemic stroke event) and 83 matched controls, as 1 control sample had insufficient volume for testing. Table 1 presents demographic and clinical characteristics of cases and controls, indicating which variables are included in the ASCVD and VACS scores. Most stroke cases (30; 71.4%) had no precipitating factor identified, while 7 (16.7%) had an infection such as endocarditis or sepsis and 5 (11.9%) occurred in PWH who used illicit drugs such as cocaine. Stroke subtypes included small vessel (13; 31.0%), other (10; 23.8%), unknown (7; 16.7%), cardioembolic (5; 11.9%), large vessel extracranial atheroembolic (4; 9.5%), and large vessel intracranial aeroembolism (3; 7.1%). The median time on ART (interquartile range [IQR]) was 5.1 (2.3–11.7) years for cases and 5.8 (2.1–11.5) years for controls. For ART, 45.6% of participants were taking a protease inhibitor (PI)–based regimen; 32.8% a non-nucleoside reverse transcriptase inhibitor (NNRTI)–based regimen; 8% a regimen including both PI and NNRTI drugs; 6.4% a “salvage-type” regimen including a PI, NNRTI, and integrase strand transfer inhibitor (INSTI); and 2.4% a triple nucleotide reverse transcriptase inhibitor (NRTI)–based regimen without other classes. Overall, 84% of participants had a viral load <400 copies/mL at the time of sampling; 10 cases and 10 controls had viral loads over this threshold. In general, cases were older, non-Caucasian, and had lower CD4 counts, higher viral loads, and higher ASCVD and VACS scores than controls.
Table 1.
Characteristics of Cases and Controls at the Time of Sample Collectiona
| Characteristic | No. (%) or Mean (SD) | |
|---|---|---|
| Stroke Cases (n=42) | Controls (n=83) | |
| Matching variable | ||
| ART regimen type | ||
| PI | 19 (45.2) | 38 (45.8) |
| NNRTI | 13 (31.0) | 28 (33.7) |
| NNRTI/PI | 4 (9.5) | 6 (7.2) |
| INSTI/NNRTI/PI | 3 (7.1) | 5 (6.0) |
| NRTI | 1 (2.4) | 2 (2.4) |
| None | 2 (4.8) | 4 (4.8) |
| Sociodemographic characteristics | ||
| Male sexb,c | 32 (76.2) | 65 (78.3) |
| Age at specimen collection, yb,c | 51.8 (11.0) | 45.3 (9.4) |
| Raceb,c | ||
| White | 9 (21.4) | 31 (37.4) |
| Black | 28 (66.7) | 44 (53.0) |
| Hispanic | 4 (9.5) | 8 (9.6) |
| Other | 1 (2.4) | 0 |
| Clinical characteristics | ||
| Diabetes historyb | 12 (28.6) | 8 (9.6) |
| Hypertension historyb | 23 (54.8) | 26 (31.3) |
| Systolic blood pressureb | 130 (22) | 125 (14) |
| Tobacco useb | 17 (40.5) | 21 (25.3) |
| Laboratory values | ||
| Hemoglobinc | 12.7 (2.1) | 14.1 (1.9) |
| Plateletsc | 253 (131) | 234 (64) |
| Creatininec | 1.1 (0.5) | 1.4 (1.8) |
| Alanine aminotransaminasec | 45.5 (53.7) | 46.3 (35.0) |
| Aspartate aminotransferasec | 42.6 (43.8) | 41.8 (32.1) |
| Total cholesterolb | 183.0 (62.6) | 177.9 (43.5) |
| High-density lipoproteinb | 49.3 (17.7) | 48.9 (15.6) |
| Hepatitis C infection statusc | 17 (40.5) | 33 (39.8) |
| HIV-related characteristics | ||
| Years on ART | 6.7 (5.5) | 7.2 (5.9) |
| Log10 HIV viral load, IU/mLc | 2.3 (1.3) | 1.8 (0.8) |
| Log10 peak HIV viral load, IU/mL | 4.9 (1.1) | 4.6 (1.1) |
| CD4 count, cells/μLc | 418 (332) | 516 (305) |
| Nadir CD4, cells/μL | 122 (132) | 192 (164) |
| Viral suppression (<400 copies/mL) | 32 (76.2) | 73 (88.0) |
| Composite risk scores | ||
| ASCVD score | 0.14 (0.10) | 0.06 (0.07) |
| VACS score | 41.3 (21.5) | 22.0 (19.5) |
| Biomarkers | Median (IQR) | |
| Angiopoietin-1, ng/mL | 3.5 (1.7–7.9) | 4.2 (1.8–7.9) |
| Angiopoietin-2, ng/mL | 13.8 (8.9–19.6) | 9.6 (6.9–14.5) |
| C-reactive protein, mcg/mL | 2.7 (0.7–9.3) | 1.6 (0.5–5.7) |
| Interleukin-6, pg/mL | 1.1 (0.6–2.1) | 0.6 (0.4–1.5) |
| Plasma activation inhibitor–1, mcg/mL | 0.7 (0.2–1.0) | 0.7 (0.4–1.0) |
| P-selectin, ng/mL | 47.9 (31.8–60.1) | 44.1 (28.6–58.3) |
| Soluble ICAM-1, mcg/mL | 0.3 (0.3–0.4) | 0.3 (0.2–0.4) |
| Soluble VCAM-1, mcg/mL | 0.3 (0.3–0.5) | 0.3 (0.3–0.5) |
| Serum amyloid A, mcg/mL | 4.2 (1.7–12.7) | 2.6 (1.4–5.3) |
| Soluble CD14, mcg/mL | 2.4 (1.8–3.0) | 2.1 (1.8–2.7) |
| Apolipoprotein A1, mcg/mL | 0.09 (0.07–0.13) | 0.10 (0.07–0.14) |
| ADAMTS-13, ng/mL | 0.15 (0.10–0.20) | 0.14 (0.11–0.17) |
| Von Willebrand factor, ng/mL | 18.0 (12.6–25.5) | 14.4 (9.5–19.9) |
Abbreviations: ART, antiretroviral therapy; ASCVD, Atherosclerotic Cardiovascular Disease score; CD4, cluster of differentiation 4; CNICS, CFAR Network of Integrated Clinical Systems; INSTI, integrase strand transfer inhibitor; IQR, interquartile range; NNRTI, non-nucleotide reverse transcriptase inhibitor; NRTI, nucleoside/nucleotide reverse transcriptase inhibitor; PI, protease inhibitor; VACS, Veterans Aging Cohort Study.
Missing values were imputed using multiple imputation with chained equations. Imputation included all Table 1 variables, in addition to CNICS site, alcohol use, marijuana use, illicit drug use, coronary artery disease, heart failure, warfarin use, statin use, dyslipidemia, body mass index, glomerular filtration rate, and triglycerides.
ASCVD score component.
VACS score component.
Correlations
Figure 1 presents a heat map of correlations between biomarkers, ASCVD score, and VACS score. Other than expected correlations between CD4 count and viral load measures, correlations that were significant at P<.01 after Bonferroni adjustment were the following: ANG-2 and IL-6 (r=0.4455), ANG-2 and VACS score (r=0.5449), IL-6 and SAA (r=0.4634), and VACS score and ASCVD score (r=0.4145). Moderate positive correlations (r between 0.2 and 0.4) were seen between VWF and ASCVD score and between WVF, sCD14, and IL-6, none of which were significant after Bonferroni adjustment.
Figure 1.
Heat map of correlations between biomarkers, ASCVD score, and VACS score. Biomarkers included are measures for which cases and controls differed in unadjusted conditional logistic regression analysis by P<.20. Negative correlations are in red, with a perfect negative correlation (ie, a Pearson coefficient of –1) in dark red. Positive correlations are in green, with a perfect positive correlation (ie, a Pearson coefficient of +1) in dark green. Nonsignificant correlations are in gray. Abbreviations: ANG-2, angiopoietin-2; ASCVD, Atherosclerotic Cardiovascular Disease score; CD4, cluster of differentiation 4; IL-6, interleukin-6; SAA, serum amyloid A; sCD14, soluble CD14; VACS, Veterans Aging Cohort Study; VL, viral load; VWF, von Willibrand factor.
Regression Analysis
Table 2 presents the results of bivariable and multivariable conditional logistic regression. Figure 2 presents dot plots of biomarker levels in cases and controls that differed at P<.20 in unadjusted analysis. In the model adjusted for age and sex only (Model 1), higher plasma viral load and higher ANG-2, sCD14, and VWF were associated with increased odds of ischemic stroke; higher nadir CD4 count was associated with decreased odds of ischemic stroke. In the model adjusted for ASCVD score (Model 2), plasma viral load (adjusted odds ratio [AOR], 2.11; 95% CI, 1.16–3.84 per log10 increment), nadir CD4 count (AOR, 0.65; 95% CI, 0.45–0.95 per 100 cells), ANG-2 (AOR, 2.07; 95% CI, 1.16–3.68 per log2 increment), and VWF (AOR, 1.74; 95% CI, 1.01–2.98 per log2 increment) remained associated with ischemic stroke; IL-6 and SAA became significant predictors (AOR, 1.77; 95% CI, 1.15–2.73; and AOR, 1.26; 95% CI, 1.01–1.57, respectively), and sCD14 was not significant. After adjustment for VACS score (Model 3), only VWF (AOR, 1.80; 95% CI, 1.04–3.12) was associated with ischemic stroke. These results did not change with adjustment for years on ART.
Table 2.
Associations Between Biomarkers and Ischemic Stroke Using Conditional Logistic Regression With the Imputed Data Set and All Participants Included (n=125)
| Biomarker | OR (95% CI) | P Value | Model 1: Adjusted for Age and Sex AOR (95% CI) | P Value | Model 2: Adjusted for ASCVD Score AOR (95% CI) | P Value | Model 3: Adjusted for VACS Score AOR (95% CI) | P Value |
|---|---|---|---|---|---|---|---|---|
| Log10 viral load | 1.84 (1.13–3.01) | .01 | 2.15 (1.18–3.92) | .01 | 2.11 (1.16–3.84) | .01 | 1.29 (0.73–2.27) | .38 |
| Log10 peak viral load | 1.33 (0.90–1.97) | .16 | 1.45 (0.94–2.24) | .09 | 1.33 (0.86–2.07) | .20 | 1.25 (0.78–2.00) | .35 |
| CD4 count, per 100 cells | 0.90 (0.79–1.03) | .12 | 0.87 (0.75–1.01) | .06 | 0.87 (0.74–1.02) | .09 | 1.06 (0.91–1.24) | .43 |
| Nadir CD4 count, per 100 cells | 0.67 (0.49–0.94) | .02 | 0.69 (0.49–0.98) | .04 | 0.65 (0.45–0.95) | .03 | 0.84 (0.57–1.24) | .37 |
| Log2 angiopoietin-1 | 0.91 (0.68–1.24) | .57 | 0.92 (0.67–1.28) | .64 | 0.90 (0.63–1.29) | .57 | 1.05 (0.74–1.48) | .79 |
| Log2 angiopoietin-2 | 1.63 (1.05–2.52) | .03 | 1.73 (1.05–2.86) | .03 | 2.07 (1.16–3.68) | .01 | 0.97 (0.57–1.67) | .92 |
| Log2 C-reactive protein | 1.11 (0.95–1.30) | .20 | 1.12 (0.93–1.36) | .23 | 1.14 (0.95–1.37) | .15 | 1.02 (0.84–1.24) | .85 |
| Log2 interleukin-6 | 1.42 (1.02–1.96) | .04 | 1.36 (0.95–1.94) | .10 | 1.77 (1.15–2.73) | .01 | 1.06 (0.71–1.57) | .79 |
| Log2 plasma activation inhibitor-1 | 0.79 (0.54–1.13) | .20 | 0.86 (0.57–1.31) | .49 | 0.81 (0.51–1.29) | .37 | 0.93 (0.58–1.51) | .78 |
| Log2 P-selectin | 1.30 (0.69–2.46) | .42 | 1.16 (0.57–2.35) | .68 | 0.97 (0.47–2.01) | .94 | 1.18 (0.55–2.53) | .67 |
| Log2 soluble ICAM-1 | 1.49 (0.81–2.75) | .20 | 1.42 (0.74–2.71) | .29 | 1.94 (0.91–4.10) | .08 | 0.94 (0.43–2.05) | .87 |
| Log2 soluble VCAM-1 | 1.10 (0.65–1.86) | .73 | 1.16 (0.67–2.02 | .60 | 1.51 (0.75–3.03) | .24 | 0.54 (0.26–1.14) | .11 |
| Log2 serum amyloid A | 1.17 (0.97–1.41) | .10 | 1.20 (0.96–1.48) | .10 | 1.26 (1.01–1.57) | .04 | 1.07 (0.86–1.34) | .53 |
| Log2 soluble CD14 | 1.91 (0.84–4.35) | .13 | 2.57 (1.01–6.49) | .05 | 2.69 (0.92–7.85) | .07 | 0.86 (0.32–2.33) | .77 |
| Log2 apolipoprotein A1 | 0.75 (0.33–1.71) | .50 | 0.78 (0.31–1.93) | .59 | 0.77 (0.29–2.04) | .60 | 0.62 (0.22–1.77) | .37 |
| Log2 ADAMTS-13 | 1.37 (0.66–2.84) | .39 | 1.59 (0.72–3.52) | .26 | 2.25 (0.86–5.86) | .10 | 1.48 (0.60–3.64) | .40 |
| Log2 von Willebrand factor | 1.75 (1.09–2.79) | .02 | 1.75 (1.03–2.97) | .04 | 1.74 (1.01–2.98) | .05 | 1.80 (1.04–3.12) | .04 |
Abbreviations: AOR, adjusted odds ratio; ASCVD, Atherosclerotic Cardiovascular Disease score; CD4, cluster of differentiation 4; OR, odds ratio; VACS, Veterans Aging Cohort Study.
Figure 2.
Dot plots of biomarker levels in cases and controls. Individual data points are graphed with gray circles, a horizontal line of black Xs indicates the median, and horizontal dashed lines indicate the upper and lower quartiles. Biomarkers presented are measures for which cases and controls differed in unadjusted conditional logistic regression analysis by P<.20; P values are also presented. Abbreviations: ANG-2, angiopoietin-2; CD4, cluster of differentiation 4; IL-6, interleukin-6; SAA, serum amyloid A; sCD14, soluble CD14.
Sensitivity Analysis
Table 3 presents the results of a sensitivity analysis restricted to the 89 individuals with plasma viral load <400 copies/mL who had at least 1 matched case or control. In this analysis, the association of higher levels of VWF with an increased odds of ischemic stroke was of borderline significance in unadjusted analysis, but was not significant in adjusted models, with or without further adjustment for years on ART.
Table 3.
Associations Between Biomarkers and Ischemic Stroke Using Conditional Logistic Regression With the Imputed Data Set, Restricted to Matched Cases and Controls Whose Viral Load Was <400 Copies/mL (n=89)
| Biomarker | OR (95% CI) | P Value | Model 1: Adjusted for Age and Sex, AOR (95% CI) | P Value | Model 2: Adjusted for ASCVD Score, AOR (95% CI) | P Value | Model 3: Adjusted for VACS Score, AOR (95% CI) | P Value |
|---|---|---|---|---|---|---|---|---|
| Log10 viral load | 2.79 (0.65–11.9) | .17 | 3.56 (0.62–20.4) | .16 | 2.74 (0.36–21.2) | .33 | 1.34 (0.24–7.38) | .74 |
| Log10 peak viral load | 1.10 (0.72–1.68) | .66 | 1.18 (0.75–1.86) | .48 | 0.90 (0.51–1.61) | .73 | 1.15 (0.71–1.86) | .56 |
| CD4 count, per 100 cells | 0.98 (0.85–1.14) | .83 | 0.97 (0.82–1.13) | .66 | 0.98 (0.81–1.18) | .83 | 1.12 (0.93–1.33) | .23 |
| Nadir CD4 count, per 100 cells | 0.74 (0.50–1.10) | .14 | 0.77 (0.51–1.15) | .21 | 0.78 (0.49–1.25) | .30 | 0.83 (0.54–1.29) | .41 |
| Log2 angiopoietin-1 | 0.90 (0.64–1.27) | .55 | 0.93 (0.64–1.36) | .72 | 0.87 (0.56 –1.36) | .54 | 1.05 (0.72–1.54) | .80 |
| Log2 angiopoietin-2 | 1.33 (0.85–2.06) | .21 | 1.46 (0.87–2.46) | .15 | 1.72 (0.96–3.10) | .07 | 0.95 (0.57–1.60) | .86 |
| Log2 C-reactive protein | 1.13 (0.92–1.39) | .25 | 1.18 (0.92–1.52) | .20 | 1.15 (0.89–1.48) | .28 | 1.08 (0.85–1.38) | .51 |
| Log2 interleukin-6 | 1.16 (0.81–1.67) | .41 | 1.15 (0.77–1.72) | .49 | 1.31 (0.79–2.18) | .30 | 0.96 (0.63–1.48) | .86 |
| Log2 plasma activation inhibitor–1 | 0.73 (0.46–1.15) | .17 | 0.81 (0.48–1.37) | .44 | 0.70 (0.37–1.34) | .29 | 0.92 (0.52–1.63) | .79 |
| Log2 P-selectin | 1.19 (0.58–2.46) | .64 | 0.97 (0.43–2.19) | .94 | 0.72 (0.28–1.88) | .50 | 1.06 (0.48–2.34) | .89 |
| Log2 soluble ICAM-1 | 1.30 (0.58–2.90) | .52 | 1.19 (0.50–2.88) | .69 | 1.48 (0.53–4.15) | .46 | 0.98 (0.36–2.64) | .97 |
| Log2 soluble VCAM-1 | 0.98 (0.51–1.91) | .96 | 1.00 (0.49–2.07) | .99 | 1.24 (0.54–2.84) | .61 | 0.41 (0.15–1.11) | .08 |
| Log2 serum amyloid A | 1.09 (0.86–1.40) | .47 | 1.15 (0.86–1.54) | .36 | 1.15 (0.86–1.55) | .34 | 1.05 (0.80–1.38) | .71 |
| Log2 soluble CD14 | 1.35 (0.52–3.49) | .54 | 1.89 (0.61–5.88) | .27 | 1.84 (0.51–6.65) | .35 | 0.60 (0.18–2.00) | .40 |
| Log2 apolipoprotein 1 | 0.77 (0.27–2.17) | .62 | 0.73 (0.23–2.35) | .60 | 0.93 (0.24–3.57) | .92 | 0.58 (0.16–2.02) | .39 |
| Log2 ADAMTS-13 | 1.29 (0.55–3.02) | .56 | 1.41 (0.55–3.62) | .47 | 2.16 (0.63–7.36) | .22 | 1.62 (0.60–4.38) | .35 |
| Log2 von Willebrand factor | 1.61 (0.99–2.62) | .06 | 1.60 (0.90–2.82) | .11 | 1.64 (0.88–3.05) | .12 | 1.59 (0.93–2.74) | .09 |
Abbreviations: AOR, adjusted odds ratio; ASCVD, Atherosclerotic Cardiovascular Disease score; CD4, cluster of differentiation 4; OR, odds ratio; VACS, Veterans Aging Cohort Study.
DISCUSSION
In this case–control study, plasma viral load, ANG-2, sCD14, and VWF levels were all elevated, after adjustment for age and sex, in treated PWH in the 12 months before an ischemic stroke, compared with PWH taking the same ART regimen who did not subsequently develop a stroke. The nadir CD4 count was also significantly lower among those who experienced a stroke. While plasma viral load, nadir CD4 count, ANG-2, IL-6, SAA, and VWF were all independently associated with ischemic stroke after adjustment for ASCVD score (a measure based on traditional CVD risk factors), only VWF was associated with ischemic stroke after adjustment for VACS score, a measure that incorporates viral load and CD4 count. In a sensitivity analysis restricted to 89 participants with a plasma viral load <400 copies/mL at the time of sampling, the association between VWF and ischemic stroke was attenuated (risk estimates ranging from 1.59 to 1.64 instead of 1.74 to 1.80), suggesting that VWF as a risk factor may be weaker when viral load is suppressed to very low levels; however, this analysis lacked sufficient power.
PWH have been found to be at increased risk for CVD events, despite effective ART [3, 4, 34–36]. The SMART trial found an increased risk of CVD events in patients undergoing CD4-guided intermittent ART compared with patients on continuous ART, accompanied by increases in the inflammatory and hemostatic biomarkers IL-6 and D-dimer [5, 6, 37, 38]. In addition, the SMART study and subsequent cohort studies demonstrated a ~40% increase in the risk of ischemic stroke in PWH compared with uninfected individuals [3, 4]. Notably, although an increasing burden of morbidity and mortality due to stroke is likely as PWH age, ischemic stroke risk may be highest in younger PWH compared with age-matched controls. In a population-based study in Taiwan, HIV infection was associated with an elevated risk of developing any stroke (adjusted hazard ratio [AHR], 1.57; 95% CI, 1.15–2.14) and specifically ischemic stroke (AHR, 1.91; 95% CI, 1.25–2.91) in patients 45 years of age and younger, but no association was observed in older age groups [39]. While some data suggest that the newer INSTI-based regimens may be associated with a lower risk of ischemic stroke than older NNRTI- and PI-based regimens [40], our study population included only 8 participants (6.4%; 3 cases and 5 controls) with regimens including the first approved INSTI (ie, raltegravir), and our method of matching by regimen precluded analysis of regimen type as a predictor.
The evidence supporting an increased risk of CVD in PWH has raised the question of whether adjunctive anti-inflammatory treatments should be combined with ART to decrease mortality [6, 41, 42]. One mechanism for inflammation in HIV relates to the death of abortively infected CD4 T cells through pyroptosis, a highly inflammatory process that leads to release of intracellular content from dying cells [43]. This content stimulates the danger-associated innate immune response, which activates endothelial cells and triggers the release of inflammatory mediators—a process that persists even after virologic suppression by ART [11]. Activated endothelial cells express adhesion receptors for leukocytes, the first step in leukocyte egress from the blood, and release VWF from their storage granules (Weibel-Palade bodies), which allows platelets to adhere to the endothelial surface [44]. These events favor pathology that accelerates CVD [45]. In large vessels, leukocyte and platelet adhesion are both early steps in the development of atherosclerosis [46]. In small vessels, adherent platelets form aggregates that can occlude the vessels and produce microinfarcts [47]. Similar to our findings, a prior case–control study demonstrated that levels of VWF were elevated in young PWH with stroke, compared with both uninfected patients and PWH without stroke [25]. Samples in that study were collected after the event, and low levels of ADAMTS13 were also associated with stroke [25]. In contrast, we did not find any significant differences in ADAMTS13 between cases and controls in our study, in which samples were collected in the 12 months before the stroke had occurred.
This study has a number of limitations. First, the number of cases was small, a consequence of sample identification during the first year of stroke adjudication for CNICS. Because of this small sample size, we included 12 cases (28.6% of the total) for which a potential precipitating factor—either infection or ongoing drug use—was identified. Second, study participants were followed before modern INSTI regimens were available; this is reflected in the relatively low rates of viral suppression. Third, in open-cohort studies with replacement of participants who do not return, a difference in participants who are retained from those lost to follow-up can be a limitation. Fortunately, loss to follow-up in CNICS is relatively uncommon, at <10% [26]. Fourth, data on traditional CVD risk factors, such as body mass index, systolic blood pressure, and lipid values, were missing for up to 22% of participants, potentially limiting our ability to adjust for these factors. However, by creating multiple predictions for each missing value, multiple imputation with chained equations takes into account the data uncertainties, yielding accurate standard errors [48]. Finally, we did not have complete information on physical activity levels or use of some medications, such as aspirin and hormone therapies, that may have affected CVD risk in the sample. Nevertheless, we were able to adjust for comprehensive and validated predictors of stroke risk, including the ASCVD score, the most commonly used assessment of CVD risk [32]. While the generalizability of our results to PWH not meeting our inclusion criteria, especially those not taking ART or in the early months of treatment, may be limited, the strengths of this study include rigorously adjudicated clinical outcomes in a prospective cohort study; matching of cases and controls by ART regimen, an important potential confounder; and the collection of plasma before the stroke event.
In conclusion, our study demonstrated that plasma VWF levels were elevated in treated PWH in the 12 months preceding an ischemic stroke event, compared with PWH who were taking the same regimen who did not experience a stroke. This association persisted after adjustment for traditional CVD risk factors and for VACS score, which includes CD4 count and plasma viral load. Assessment of circulating levels of VWF may identify subgroups of PWH with increased risk for development of adverse CVD events, including stroke. Further work to validate these findings in other clinical cohorts and to investigate potential interventions, such as the use of antiplatelet medications (eg, acetylsalicylic acid [aspirin], clopidogrel, and dipyridamole), as primary prophylaxis to reduce CVD risk in PWH with elevated VWF levels is warranted.
Acknowledgments
We would like to thank the CNICS cohort participants for sharing their clinical data and specimens. Additionally, we would like to acknowledge the many clinical, laboratory, and administrative staff who assisted with this project at the CNICS sites. We also thank Yu Ni for her help identifying samples for testing, Susanna Harju-Baker and Victoria Dmyterko for their laboratory work, and Joseph Delaney for his advice regarding multiple imputation and data analysis.
Author contributions. S.M.G., H.M.C., W.C.L., and J.A.L. designed the study, and S.M.G. and J.A.L. acquired funding. H.M.C., F.C.C., and J.N.M. oversaw CNICS data collection, and D.L.T., J.R.Z., C.M.M., E.L.H., and A.H. adjudicated cases. R.M.N. assisted with identification of eligible cases and controls, and S.M.G. and R.M.N. analyzed the data. J.L. organized sample collection and processing, and laboratory testing was overseen by J.C., D.W.C., M.M.W., and J.A.L. A.S.R. provided input on relevance to the HIV, stroke, and aging literature. S.M.G. wrote the initial draft of the manuscript. All authors contributed to the final draft and approved the manuscript for submission.
Patient consent. All participants provided written informed consent to participate in the CNICS cohort. Each participating site received human subject approval for CNICS.
Financial support. This study was funded by the National Institute on Aging (NIA) as a subaward of grant R24 AG044325 and by National Heart, Lung, and Blood Institute (NHLBI) grant R21 HL129526. Additional support came from the National Institute of Allergy and Infectious Diseases (CNICS grant R24 AI067039, University of Washington Center for AIDS Research [CFAR] grant P30 AI027757; Johns Hopkins University CFAR grant P30 AI094189; University of Alabama at Birmingham CFAR grant P30 AI027767, and University of California at San Francisco CFAR grant P30 AI027763) and NHLBI grant R01 HL126538. A.S.R. was supported by a Veterans Affairs RR&D Senior Research Career Scientist Award and the Claude D. Pepper Older Americans Independence Center (NIA grant P30 AG028747).
Potential conflicts of interest. All authors: no reported conflicts of interest. All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.
References
- 1.Marcus JL, Leyden WA, Chao CR, et al. HIV infection and incidence of ischemic stroke. AIDS 2014; 28:1911–9. [DOI] [PubMed] [Google Scholar]
- 2.Chow FC. HIV infection, vascular disease, and stroke. Semin Neurol 2014; 34:35–46. [DOI] [PubMed] [Google Scholar]
- 3.Neuhaus J, Jacobs DR Jr, Baker JV, et al. Markers of inflammation, coagulation, and renal function are elevated in adults with HIV infection. J Infect Dis 2010; 201:1788–95. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Kuller LH, Tracy R, Belloso W, et al. ; INSIGHT SMART Study Group. Inflammatory and coagulation biomarkers and mortality in patients with HIV infection. PLoS Med 2008; 5:e203. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Baker J, Quick H, Hullsiek KH, et al. Interleukin-6 and d-dimer levels are associated with vascular dysfunction in patients with untreated HIV infection. HIV Med 2010; 11:608–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Baker J, Ayenew W, Quick H, et al. High-density lipoprotein particles and markers of inflammation and thrombotic activity in patients with untreated HIV infection. J Infect Dis 2010; 201:285–92. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Baker JV, Neuhaus J, Duprez D, et al. ; INSIGHT SMART Study Group. Inflammation predicts changes in high-density lipoprotein particles and apolipoprotein A1 following initiation of antiretroviral therapy. AIDS 2011; 25:2133–42. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Dubé MP, Sattler FR.. Inflammation and complications of HIV disease. J Infect Dis 2010; 201:1783–5. [DOI] [PubMed] [Google Scholar]
- 9.Graham SM, Mwilu R, Liles WC.. Clinical utility of biomarkers of endothelial activation and coagulation for prognosis in HIV infection: a systematic review. Virulence 2013; 4:564–71. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Graham SM, Rajwans N, Jaoko W, et al. Endothelial activation biomarkers increase after HIV-1 acquisition: plasma vascular cell adhesion molecule-1 predicts disease progression. AIDS 2013; 27:1803–13. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Graham SM, Rajwans N, Tapia KA, et al. A prospective study of endothelial activation biomarkers, including plasma angiopoietin-1 and angiopoietin-2, in Kenyan women initiating antiretroviral therapy. BMC Infect Dis 2013; 13:263. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Graham SM, Chen J, Le J, et al. Von Willebrand factor adhesive activity and ADAMTS13 protease activity in HIV-1-infected men. Int J Med Sci 2019; 16:276–84. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Mocco J, Choudhri TF, Mack WJ, et al. Elevation of soluble intercellular adhesion molecule-1 levels in symptomatic and asymptomatic carotid atherosclerosis. Neurosurgery 2001; 48:718–21; discussion 721–2. [DOI] [PubMed] [Google Scholar]
- 14.Weinkauf CC, Concha-Moore K, Lindner JR, et al. Endothelial vascular cell adhesion molecule 1 is a marker for high-risk carotid plaques and target for ultrasound molecular imaging. J Vasc Surg 2018; 68:105–13S. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Angchaisuksiri P. Coagulopathy in malaria. Thromb Res 2014; 133:5–9. [DOI] [PubMed] [Google Scholar]
- 16.Chen J, Hobbs WE, Le J, et al. The rate of hemolysis in sickle cell disease correlates with the quantity of active von Willebrand factor in the plasma. Blood 2011; 117:3680–3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Groot E, de Groot PG, Fijnheer R, Lenting PJ.. The presence of active von Willebrand factor under various pathological conditions. Curr Opin Hematol 2007; 14:284–9. [DOI] [PubMed] [Google Scholar]
- 18.Aukrust P, Bjørnsen S, Lunden B, et al. Persistently elevated levels of von Willebrand factor antigen in HIV infection. Downregulation during highly active antiretroviral therapy. Thromb Haemost 2000; 84:183–7. [PubMed] [Google Scholar]
- 19.Schved JF, Gris JC, Arnaud A, et al. Von Willebrand factor antigen, tissue-type plasminogen activator antigen, and risk of death in human immunodeficiency virus 1-related clinical disease: independent prognostic relevance of tissue-type plasminogen activator. J Lab Clin Med. 1992; 120:411–9. [PubMed] [Google Scholar]
- 20.van Vonderen MG, Hassink EA, van Agtmael MA, et al. Increase in carotid artery intima-media thickness and arterial stiffness but improvement in several markers of endothelial function after initiation of antiretroviral therapy. J Infect Dis 2009; 199:1186–94. [DOI] [PubMed] [Google Scholar]
- 21.Jong E, Louw S, van Gorp EC, et al. The effect of initiating combined antiretroviral therapy on endothelial cell activation and coagulation markers in South African HIV-infected individuals. Thromb Haemost 2010; 104:1228–34. [DOI] [PubMed] [Google Scholar]
- 22.Greisenegger S, Segal HC, Burgess AI, et al. Biomarkers and mortality after transient ischemic attack and minor ischemic stroke: population-based study. Stroke 2015; 46:659–66. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.McCabe DJ, Murphy SJ, Starke R, et al. Relationship between ADAMTS13 activity, von Willebrand factor antigen levels and platelet function in the early and late phases after TIA or ischaemic stroke. J Neurol Sci 2015; 348:35–40. [DOI] [PubMed] [Google Scholar]
- 24.Kovacevic KD, Mayer FJ, Jilma B, et al. Von Willebrand factor antigen levels predict major adverse cardiovascular events in patients with carotid stenosis of the ICARAS study. Atherosclerosis 2019; 290:31–6. [DOI] [PubMed] [Google Scholar]
- 25.Allie S, Stanley A, Bryer A, et al. High levels of von Willebrand factor and low levels of its cleaving protease, ADAMTS13, are associated with stroke in young HIV-infected patients. Int J Stroke 2015; 10:1294–6. [DOI] [PubMed] [Google Scholar]
- 26.Kitahata MM, Rodriguez B, Haubrich R, et al. Cohort profile: the Centers for AIDS Research Network of Integrated Clinical Systems. Int J Epidemiol 2008; 37:948–55. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Crane HM, Nance RM, Avoundjian T, et al. Types of stroke among people living with HIV in the United States. J Acquir Immune Defic Syndr 2021; 86:568–78. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Longstreth WT Jr, Gasca NC, Gottesman RF, et al. Adjudication of transient ischemic attack and stroke in the multi-ethnic study of atherosclerosis. Neuroepidemiology 2018; 50:23–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Harding BN, Avoundjian T, Heckbert SR, et al. HIV viremia and risk of stroke among people living with HIV who are using antiretroviral therapy. Epidemiology 2021; 32:457–64. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Crane HM, Grunfeld C, Harrington RD, Kitahata MM.. Lipoatrophy and lipohypertrophy are independently associated with hypertension. HIV Med 2009; 10:496–503. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Crane HM, Kadane JB, Crane PK, Kitahata MM.. Diabetes case identification methods applied to electronic medical record systems: their use in HIV-infected patients. Curr HIV Res 2006; 4:97–106. [DOI] [PubMed] [Google Scholar]
- 32.Lloyd-Jones DM, Braun LT, Ndumele CE, et al. Use of risk assessment tools to guide decision-making in the primary prevention of atherosclerotic cardiovascular disease: a special report from the American Heart Association and American College of Cardiology. Circulation 2019; 139:e1162–77. [DOI] [PubMed] [Google Scholar]
- 33.Salinas JL, Rentsch C, Marconi VC, et al. Baseline, time-updated, and cumulative HIV care metrics for predicting acute myocardial infarction and all-cause mortality. Clin Infect Dis 2016; 63:1423–30. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Phillips AN, Neaton J, Lundgren JD.. The role of HIV in serious diseases other than AIDS. AIDS 2008; 22:2409–18. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Lifson AR, Belloso WH, Carey C, et al. ; INSIGHT Cause of Death Writing Group. Determination of the underlying cause of death in three multicenter international HIV clinical trials. HIV Clin Trials 2008; 9:177–85. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Hasse B, Ledergerber B, Furrer H, et al. ; Swiss HIV Cohort Study. Morbidity and aging in HIV-infected persons: the Swiss HIV Cohort Study. Clin Infect Dis 2011; 53:1130–9. [DOI] [PubMed] [Google Scholar]
- 37.El-Sadr WM, Lundgren JD, Neaton JD, et al. ; Strategies for Management of Antiretroviral Therapy (SMART) Study Group. CD4+ count-guided interruption of antiretroviral treatment. New Engl J Med 2006; 355:2283–96. [DOI] [PubMed] [Google Scholar]
- 38.Phillips AN, Carr A, Neuhaus J, et al. Interruption of antiretroviral therapy and risk of cardiovascular disease in persons with HIV-1 infection: exploratory analyses from the SMART trial. Antivir Ther 2008; 13:177–87. [DOI] [PubMed] [Google Scholar]
- 39.Lin HL, Muo CH, Lin CY, et al. Incidence of stroke in patients with HIV infection: a population-based study in Taiwan. PLoS One 2019; 14:e0217147. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.O’Halloran JA, Sahrmann J, Butler AM, et al. Brief report: integrase strand transfer inhibitors are associated with lower risk of incident cardiovascular disease in people living with HIV. J Acquir Immune Defic Syndr 2020; 84:396–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Nixon DE, Landay AL.. Biomarkers of immune dysfunction in HIV. Curr Opin HIV AIDS 2010; 5:498–503. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Baker JV, Neuhaus J, Duprez D, et al. ; INSIGHT SMART Study Group. Changes in inflammatory and coagulation biomarkers: a randomized comparison of immediate versus deferred antiretroviral therapy in patients with HIV infection. J Acquir Immune Defic Syndr 2011; 56:36–43. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Doitsh G, Galloway NL, Geng X, et al. Cell death by pyroptosis drives CD4 T-cell depletion in HIV-1 infection. Nature 2014; 505:509–14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Fiedler U, Augustin HG.. Angiopoietins: a link between angiogenesis and inflammation. Trends Immunol 2006; 27:552–8. [DOI] [PubMed] [Google Scholar]
- 45.Shim CY, Liu YN, Atkinson T, et al. Molecular imaging of platelet-endothelial interactions and endothelial von Willebrand factor in early and mid-stage atherosclerosis. Circ Cardiovasc Imaging 2015; 8:e002765. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Blankenberg S, Barbaux S, Tiret L.. Adhesion molecules and atherosclerosis. Atherosclerosis 2003; 170:191–203. [DOI] [PubMed] [Google Scholar]
- 47.Wootton DM, Ku DN.. Fluid mechanics of vascular systems, diseases, and thrombosis. Annu Rev Biomed Eng 1999; 1:299–329. [DOI] [PubMed] [Google Scholar]
- 48.Azur MJ, Stuart EA, Frangakis C, Leaf PJ.. Multiple imputation by chained equations: what is it and how does it work? Int J Methods Psychiatr Res 2011; 20:40–9. [DOI] [PMC free article] [PubMed] [Google Scholar]


