Skip to main content
Journal of the American Heart Association: Cardiovascular and Cerebrovascular Disease logoLink to Journal of the American Heart Association: Cardiovascular and Cerebrovascular Disease
. 2022 Feb 18;11(5):e024142. doi: 10.1161/JAHA.121.024142

Grayscale Ultrasound Texture Features of Carotid and Brachial Arteries in People With HIV Infection Before and After Antiretroviral Therapy

Christina M Hughey 1,*, Belinda W Vuong 1,*, Heather B Ribaudo 2, Carol C K Mitchell 1, Claudia E Korcarz 1, Howard N Hodis 3, Judith S Currier 4, James H Stein 1,
PMCID: PMC9075086  PMID: 35179037

Abstract

Background

We aimed to investigate novel grayscale ultrasound characteristics of the carotid and brachial arteries in people with HIV infection before and after starting initial antiretroviral therapy (ART).

Methods and Results

We performed grayscale ultrasound image analyses of the common carotid artery (CCA) and brachial artery before and after receipt of 1 of 3 randomly allocated ART regimens. We measured arterial wall echogenicity (grayscale median), contrast (gray‐level difference statistic method), and entropy. These measures and their changes were compared with atherosclerotic cardiovascular disease risk factors, measures of HIV disease severity, and inflammatory biomarkers before and after ART. Changes in the grayscale measures were evaluated within and between ART arms. Among 201 ART‐naïve people with HIV, higher systolic blood pressure, higher body mass index, lower CD4+ T cells, and non‐Hispanic White race and ethnicity were associated independently with lower CCA grayscale median. Changes in each CCA grayscale measure from baseline to 144 weeks correlated with changes in soluble CD163: grayscale median (ρ=−0.17; P=0.044), gray‐level difference statistic–contrast (ρ=−0.19; P=0.024), and entropy (ρ=−0.21; P=0.016). Within the atazanavir/ritonavir arm, CCA entropy increased (adjusted β=0.023 [95% CI, 0.001–0.045]; P=0.04), but no other within‐arm changes in grayscale measures were seen. Correlations of brachial artery grayscale measures were weaker.

Conclusions

In ART‐naïve people with HIV, CCA grayscale ultrasound measures were associated with atherosclerotic cardiovascular disease risk factors and lower grayscale median was associated with lower CD4+ T cells. Reductions in soluble CD163 with initial ART were associated with improvements in all 3 CCA grayscale measures, suggesting that reductions in macrophage activation with ART initiation may lead to less arterial injury.

Registration

URL: https://clinicaltrials.gov/; Unique identifiers: NCT00811954; NCT00851799

Keywords: antiretroviral therapy, atherosclerosis, brachial arteries, carotid arteries, HIV, ultrasound, vascular disease

Subject Categories: Atherosclerosis, Cardiovascular Disease, Imaging, Ultrasound, Vascular Disease


Nonstandard Abbreviations and Acronyms

CCA

common carotid artery

GLDS‐CON

gray‐level difference statistic–contrast

GSM

grayscale median

PWH

people with HIV

sCD163

soluble CD163

Clinical Perspective

What Is New?

  • Early arterial wall changes, detected using novel grayscale ultrasound imaging, permit characterization of the arterial wall beyond macroscopic changes.

  • In antiretroviral therapy–naïve people with HIV infection, common carotid artery grayscale ultrasound measures were associated with atherosclerotic cardiovascular disease risk factors and lower grayscale median pixel intensity was associated with lower CD4+ T cells.

  • Reductions in soluble CD163 with initial antiretroviral therapy were associated with improvements in all 3 common carotid artery grayscale measures.

What Are the Clinical Implications?

  • These findings suggest that reductions in macrophage activation with antiretroviral therapy initiation may lead to less arterial injury.

  • Antiretroviral therapy regimens that contain atazanavir may lead to less common carotid arterial injury.

Since the development and widespread use of effective antiretroviral therapy (ART), people with HIV (PWH) infection have longer life expectancies; and non‐AIDS illnesses, such atherosclerotic cardiovascular disease (ASCVD), are increasingly prevalent causes of morbidity and mortality. 1 , 2 , 3 Indeed, the 2018 guidelines on the management of blood cholesterol consider HIV‐positive serostatus an ASCVD “risk‐enhancing factor.” 4 Proposed explanations for increased ASCVD risk in PWH include a greater prevalence of traditional ASCVD risk factors, chronic inflammation with altered immune cell activation, and adverse effects of ART. 5 , 6

Because traditional risk factors underestimate ASCVD risk in PWH, investigators have focused on using biochemical and imaging markers of arterial injury to investigate the pathophysiological characteristics of HIV‐associated ASCVD. Elevated inflammatory biomarkers, such as CRP (C‐reactive protein), D‐dimer, and interleukin‐6, are associated with overall mortality and ASCVD in PWH. 7 , 8 , 9 Markers of monocyte and macrophage activation, such as soluble CD163 (sCD163), also have been linked to ASCVD and subclinical atherosclerosis in PWH. 10 , 11 , 12 , 13 Several noninvasive measures of arterial function, injury, and plaque burden also have shed light on the pathophysiological characteristics of ASCVD in PWH; however, each has limitations that have been reviewed elsewhere, 6 including use of ionizing radiation and insensitivity for detecting early changes in the arterial wall that occur before macroscopic changes can be detected. These early changes may shed light on the pathogenesis of ASCVD in PWH.

Recent advances in grayscale ultrasound imaging permit characterization of the arterial wall beyond macroscopic changes. Historically, ultrasound tissue characterization measured arterial wall echogenicity (brightness), as quantified by its grayscale median (GSM) pixel intensity. 14 , 15 More recently, techniques that quantify arterial grayscale texture using variation measures and interpixel grayscale relationships (eg, entropy and contrast) have emerged as independent predictors of future ASCVD events 16 and identify arterial changes with therapeutic interventions, even when more traditional arterial measures do not. 17 , 18 For example, among women with HIV in the WIHS (Women’s Interagency HIV Study), common carotid artery (CCA) GSM levels were associated with ASCVD risk factors. 19 However, in the MESA (Multi‐Ethnic Study of Atherosclerosis), CCA texture features, such as entropy, contrast, and angular second moment, had stronger associations with ASCVD risk factors than GSM; they also predicted incident cardiovascular disease events, even after adjustment for ASCVD risk factors, CRP, and carotid intima‐media thickness. 16 In adults with suppressed HIV replication, brachial artery entropy and contrast improved after 24 weeks of anti‐inflammatory therapy with low‐dose methotrexate; these changes were associated with changes in CD4+ T‐cell and D‐dimer concentrations, in the absence of detectable improvements in flow‐mediated dilation. 17 Similarly, in people with peripheral arterial disease who completed an exercise regimen, improvements in 6′ walking distance correlated with increases in brachial artery entropy and contrast, but not flow‐mediated dilation. 18

The objective of this study was to investigate associations of these novel grayscale ultrasound measures in the CCA and brachial artery in PWH before and after starting their initial ART. We achieved this objective by performing grayscale arterial analyses of images from participants in ACTG (AIDS Clinical Trials Group) Study A5260s, a prospective, randomized clinical trial of initial ART in PWH.

Methods

Study Design and Participants

The design and primary findings from ACTG Study A5260s have been published previously. 20 , 21 The ACTG Study A5260s was a prospective, 144‐week longitudinal study in which 328 ART‐naive PWH without known cardiovascular disease or diabetes, uncontrolled thyroid disease, or use of lipid‐lowering medications were randomized to tenofovir disoproxil fumarate‐emtricitabine plus atazanavir/ritonavir, or darunavir/ritonavir, or raltegravir as part of the A5257 trial. 22 Randomization was stratified by screening HIV‐1 RNA level (>100 000 or <100 000 copies/mL) and Framingham 10‐year coronary heart disease risk score (<6% or ≥6% risk). The institutional review boards at all participating institutions approved the parent study and substudy (ClinicalTrials.gov identifiers: NCT00811954 and NCT00851799); all participants provided written informed consent. Data, analytic methods, and study material are accessible by the public and other investigators using standard operating procedures from the National Institute of Health’s AIDS Clinical Trials Group (https://actgnetwork.org/clinical‐trials/access‐published‐data).

Carotid and Brachial Artery Ultrasonography and Laboratory Testing

B‐mode images of the distal right CCA were acquired by centrally trained and certified ultrasonographers using a high‐resolution linear array ultrasound transducer with simultaneous electrographic tracings before ART initiation, then after 48, 96, and 112 to 144 weeks. 21 Brachial artery ultrasound images were obtained before ART initiation and after 24 and 48 weeks. 21 This study focused on measurements made from CCA images obtained pre‐ART initiation, at week 48, and at week 112 to 144 (pre‐ART and weeks 24 and 48 for brachial artery images). Blood samples at each of these visits (pre‐ART, weeks 24, 48, 96, and 112–144) were obtained from fasting participants and sent to core laboratories for analyses, as described previously. 21

Grayscale Artery Image Analysis

Digital images were converted to bitmaps for grayscale analysis using AccessPoint Web (Freeland Systems, LLC, Version 8.2, Alpharetta, GA). Grayscale analyses and texture feature extraction were performed using LifeQ Medical Carotid Plaque (Nicosia, Cyprus). 16 , 17 , 23 Grayscale texture features analyzed for this study were GSM, contrast as measured using gray‐level difference statistics (GLDS‐CON), and entropy. Analytic steps included normalization, standardization, segmentation, and feature extraction. Images were normalized so that the blackest area in the blood was assigned a grayscale value of 0 and the brightest white portion in the middle of the adventitia a grayscale value of 190. Next, images were standardized to a pixel density of 20/mm. 16 , 23 Segmentation was performed by identifying a reproducible, horizontal 1‐cm segment of the distal CCA or the brachial artery with clear “double lines” indicating the blood‐intima and media‐adventitia interfaces. The far wall of the artery was segmented by tracing the blood‐intima interface and media‐adventitia interface over 1 cm, then grayscale texture features were extracted from the wall segment. GSM and entropy are first‐order statistics derived from the grayscale histogram. 23 GSM is the median grayscale value within the segmented region of interest of the arterial wall and represents echogenicity. Entropy is a measure of randomness or uncertainty of how grayscale values are distributed in the image using the formula: ENT=ipilog(pi), where p(i) is the probability that a grayscale value i is contained within the region of interest. 23 GLDS methods measure the distribution of grayscale values and assess the heterogeneity of the region of interest. Differences in grayscale values between pixels at different distances and directions are computed using the formula: CON=i2pδi, where i is the difference between 2 pixels and pδi is the probability that a grayscale value will occur at a given distance. 23

Statistical Analysis

Distributions of all participant measures were examined visually and described using medians and interquartile ranges; levels of inflammatory biomarkers and of GLDS‐CON were log10 transformed to normalize their distributions. Associations between the baseline (pre‐ART) variables in Table 1 and imaging measures as well as their changes were assessed using Spearman correlations; associations of baseline (pre‐ART) variables also were included in mixed‐effect models described below. Because we were interested in changes in outcomes in response to ART and to minimize confounding attributable to lack of control of HIV‐1 viremia, the primary analyses were restricted to the ACTG Study A5260s–defined successfully treated population who successful maintained suppression of HIV replication throughout the duration of the study 21 , 24 ; supportive analyses used an intention‐to‐treat approach.

Table 1.

Baseline Participant Characteristics by Population and Grayscale Ultrasound Image Availability

Optimally treated group Grayscale data available
Baseline characteristics Total (N=328) No (N=94) Yes (N=234) No (N=33) Yes (N=201)
Treatment group
Atazanavir/ritonavir 109 (33) 41 (44) 68 (29) 7 (21) 61 (30)
Raltegravir 106 (32) 24 (26) 82 (35) 13 (39) 69 (34)
Darunavir/ritonavir 113 (34) 29 (31) 84 (36) 13 (39) 71 (35)
Age, y
Median (Q1 to Q3) 36 (28 to 45) 34 (25 to 42) 36 (28 to 45) 32 (27 to 39) 38 (29 to 46)
Sex
Women 34 (10) 10 (11) 24 (10) 2 (6) 22 (11)
Race and ethnicity
White, non‐Hispanic 144 (44) 32 (34) 112 (48) 16 (48) 96 (48)
Black, non‐Hispanic 105 (32) 37 (39) 68 (29) 12 (36) 56 (28)
Hispanic 65 (20) 20 (21) 45 (19) 4 (12) 41 (20)
Other 14 (4) 5 (5) 9 (4) 1 (3) 8 (4)
Body mass index, kg/m²
Underweight (<18) 5 (2) 3 (3) 2 (1) 0 (0) 2 (1)
Normal (18–<25) 167 (51) 49 (52) 118 (50) 15 (45) 103 (51)
Overweight (25–<30) 103 (31) 28 (30) 75 (32) 7 (21) 68 (34)
Obese (≥30) 53 (16) 14 (15) 39 (17) 11 (33) 28 (14)
Systolic blood pressure, mm Hg
Median (Q1 to Q3) 117 (108 to 125) 114 (104 to 120) 118 (109 to 126) 119 (108 to 125) 118 (109 to 126)
Smoking history
Yes 194 (59) 65 (69) 129 (55) 19 (58) 110 (55)
Time since HIV‐1 diagnosis, y
<1 179 (55) 41 (44) 138 (59) 23 (70) 115 (57)
1–5 69 (21) 29 (31) 40 (17) 3 (9) 37 (18)
5–10 30 (9) 8 (9) 22 (9) 3 (9) 19 (9)
≥10 14 (4) 3 (3) 11 (5) 2 (6) 9 (4)
Baseline CD4, cells/mm3
Median (Q1 to Q3) 349 (203 to 455) 398 (238 to 500) 338 (191 to 448) 293 (191 to 366) 340 (197 to 451)
Baseline HIV‐1 RNA, copies/mL
<10 000 80 (24) 20 (21) 60 (26) 10 (30) 50 (25)
10 000–99 999 158 (48) 47 (50) 111 (47) 14 (42) 97 (48)
>100 000 90 (27) 27 (29) 63 (27) 9 (27) 54 (27)
Total cholesterol, mg/dL
Median (Q1 to Q3) 152 (133 to 177) 143 (127 to 171) 156 (136 to 178) 140 (121 to 159) 160 (137 to 179)
Triglycerides, mg/dL
Median (Q1 to Q3) 107 (78 to 151) 111 (84 to 165) 105 (74 to 142) 90 (70 to 144) 106 (76 to 140)
High‐density lipoprotein cholesterol, mg/dL
Median (Q1 to Q3) 37 (31 to 45) 35 (28 to 45) 38 (32 to 45) 37 (28 to 42) 38 (32 to 46)
Low‐density lipoprotein cholesterol, mg/dL
Median (Q1 to Q3) 89 (73 to 109) 77 (61 to 101) 92 (75 to 110) 84 (73 to 97) 93 (77 to 112)
Homeostasis model: insulin resistance, log molar units
Median (Q1 to Q3) −0.21 (−0.40 to 0.14) −0.20 (−0.41 to 0.17) −0.22 (−0.40 to 0.14) −0.09 (−0.39 to 0.37) −0.22 (−0.40 to 0.12)
Total bilirubin, ×upper limit normal
Median (Q1 to Q3) 0.50 (0.40 to 0.70) 0.50 (0.40 to 0.70) 0.50 (0.40 to 0.70) 0.45 (0.35 to 0.60) 0.50 (0.40 to 0.70)
CRP, log10 µg/mL
Median (Q1 to Q3) 0.13 (−0.16 to 0.47) 0.05 (−0.23 to 0.39) 0.17 (−0.11 to 0.50) 0.32 (−0.19 to 0.59) 0.16 (−0.10 to 0.50)
Interleukin‐6, log10 pg/mL
Median (Q1 to Q3) 0.25 (0.06 to 0.46) 0.27 (0.05 to 0.48) 0.24 (0.07 to 0.45) 0.21 (0.08 to 0.48) 0.24 (0.07 to 0.44)
D‐dimer, log10 µg/mL
Median (Q1 to Q3) −0.59 (−0.85 to −0.24) −0.49 (−0.77 to −0.19) −0.59 (−0.85 to −0.26) −0.54 (−0.70 to −0.19) −0.59 (−0.85 to −0.27)
sCD14, log10 ng/mL
Median (Q1 to Q3) 3.23 (3.16 to 3.31) 3.24 (3.16 to 3.30) 3.23 (3.16 to 3.31) 3.23 (3.17 to 3.30) 3.23 (3.16 to 3.31)
sCD163, log10 ng/mL
Median (Q1 to Q3) 3.05 (2.9 to −3.20) 3.07 (2.92 to 3.20) 3.04 (2.89 to 3.19) 3.16 (3.00 to 3.24) 3.02 (2.88 to 3.19)

Data are given as number (percentage), unless otherwise indicated. CRP indicates C‐reactive protein; Q1, quartile 1; Q3, quartile 3; sCD14, soluble CD14; and sCD163, soluble CD163.

The treatment group effect of interest was the difference in the rate of change (ie, the slope) of the grayscale measures over time (ie, the interaction between the treatment group and time) with the week 0 grayscale measure in the outcome vector; supportive analyses estimated changes from baseline to each time point adjusted for the baseline measures. Values are presented as the estimated rate of change per year (β) and difference in rate of change (Δ) with 95% CIs. To accommodate repeated measurements (pre‐ART and weeks 48 and 112–144 for the CCA; pre‐ART and weeks 24 and 48 for the brachial artery), analyses used mixed‐effect models with random intercepts and slopes, without adjustment for baseline factors for the CCA and brachial artery measures. Subsequent models for the CCA grayscale measures adjusted for baseline sex, race and ethnicity, smoking history (yes/no), body mass index, reported duration of HIV infection, and baseline CD4+ T‐cell count.

Consistent with the ACTG Study A5260s design, treatment group comparisons first assessed the difference between the 2 boosted protease inhibitor regimens. 21 If no difference was apparent, the raltegravir regimen was then compared with the pooled protease inhibitor effect. In the event of a difference between the protease inhibitor regimens, all pairwise comparisons were performed. A post hoc analysis comparing the atazanavir regimen with the raltegravir regimen also is presented.

Results

Participant Characteristics

The first participant enrolled in June 2009. The final study visit occurred in June 2013. Of the 328 HIV‐1 ART‐naïve PWH, 234 were successfully treated (adequate suppression defined as undetectable RNA level) and 201 had grayscale data available for analysis. Baseline (before ART initiation) participant characteristics, ASCVD risk factors, measures of HIV disease severity, and inflammatory biomarker are in Table 1.

Associations of CCA Grayscale Measures at Baseline and on Treatment

At baseline, CCA GSM correlated strongly with several traditional ASCVD risk factors, age (ρ=−0.26; P<0.001) and body mass index (ρ=−0.39; P<0.001), and more modestly with systolic blood pressure (ρ=−0.28; P<0.001), low‐density lipoprotein cholesterol (ρ=−0.16; P=0.045), and total bilirubin (ρ=−0.14; P=0.042). GLDS‐CON was correlated with body mass index only (r=−0.16; P=0.041). In adjusted analyses of baseline associations, higher systolic blood pressure, higher body mass index, lower CD4+ T cells, and non‐Hispanic White race and ethnicity (compared with non‐Hispanic Black race and ethnicity) were associated with lower GSM (Table 2). In adjusted models, Hispanic ethnicity also was associated with higher GLDS‐CON (β=0.140 [95% CI, 0.051–0.230]; P=0.002) and entropy (β=0.120 [95% CI, 0.021–0.210]; P=0.020) than non‐Hispanic Black race and ethnicity.

Table 2.

Adjusted Baseline Associations With CCA Grayscale Median

Variable β 95% Lower CI 95% Upper CI P values
Age, y
18–29 Reference 0.08
30–39 0.28 −6.66 7.22 0.94
≥40 −6.07 −12.8 0.65 0.08
Sex
Women Reference 0.51
Men 2.93 −5.86 11.7 0.51
Race and ethnicity
Black, non‐Hispanic Reference 0.004
Hispanic, other −5.78 −13.4 1.78 0.13
White, non‐Hispanic −13.5 −20.3 −6.83 0.004
Systolic blood pressure, mm Hg −1.67 −2.67 −0.67 0.001 0.001
Smoking history
No Reference
Yes 1.40 −3.91 6.70 0.60 0.60
Body mass index, kg/m2
<25 Reference 0.002
25–<30 −7.25 −13.0 −1.50 0.01
≥30 −15.8 −25.6 −6.03 0.002
CD4+ T cell, /mm3
>350 Reference 0.008
200–349 −3.80 −10.3 2.72 0.25
<200 −11.2 −18.1 −4.24 0.002
Time since HIV‐1 diagnosis, y
<1 Reference 0.46
1–5 −0.20 −7.56 7.16 0.96
≥30 −3.95 −10.5 2.57 0.23

Adjusted for age, sex, race and ethnicity, smoking history, body mass index, duration of HIV infection, and baseline CD4+ T cells. βindicates estimated rate of change per year; and CCA, common carotid artery.

Similar correlations for GSM and GLDS‐CON were seen after up to 144 weeks of ART; however, GSM also correlated with insulin resistance (ρ=−0.25; P=0.002) and CRP (ρ=−0.19; P=0.018). Entropy weakly correlated with insulin resistance (ρ=0.16; P=0.041) and interleukin‐6 (ρ=0.10; P=0.02) at baseline only. No other statistically significant correlations between the variables in Table 1 and the CCA grayscale measures were identified at baseline or after up to 144 weeks.

Changes in all 3 grayscale measures from baseline to 144 weeks correlated consistently with changes in sCD163: GSM (ρ=−0.17; P=0.044), GLDS‐CON (ρ=−0.19; P=0.024), and entropy (ρ=−0.21; P=0.016). Changes in sCD163 over time are in Figure 1. Changes in GSM also correlated with changes in triglycerides (ρ=−0.18; P=0.034). No other statistically significant correlations with changes in CCA grayscale measures were identified.

Figure 1. Soluble CD163 (sCD163) over time, by treatment.

Figure 1

A, Measured values. B, Changes. Mean and 95% CI are plotted over raw data. The y axis is trimmed at the 5th and 95th percentiles of the observed data. ATV indicates atazanavir; DRV, darunavir; RAL, raltegravir; and RTV, ritonavir.

Changes in CCA Grayscale Measures Within and Between ART Groups

After 144 weeks of effective ART, use of the atazanavir‐containing ART regimen was associated with an increase in entropy (adjusted β=0.023 [95% CI, 0.001–0.045]; P=0.04), but not GSM or GLDS‐CON (Figure 2 and Tables 3 and 4). Changes in CCA entropy were not observed in the raltegravir arm (adjusted β=−0.006 [95% CI, −0.026 to 0.011]; P=0.56); changes for the darunavir‐containing regimen were intermediate (β=0.014 [95% CI, −0.006 to 0.033]; P=0.17) but similar to the atazanavir‐containing regimen (P=0.51). The improvements in entropy with the atazanavir‐containing regimen (Δ=−0.029 [95% CI, −0.062 to 0.004]; P=0.05) and with the boosted protease inhibitor regimens (Δ=−0.024 [95% CI, −0.051 to 0.003]; P=0.05) were greater than with the raltegravir regimen. Significant within‐ and between‐arms differences in changes in CCA GSM and GLDS‐CON were not identified (Tables 5, 6, 7, 8). Similar results were obtained in the intention‐to‐treat analysis (data not shown).

Figure 2. Changes from baseline in common carotid artery grayscale measures over time by treatment group.

Figure 2

A, Entropy. B, Gray‐level difference statistic: contrast. C, Grayscale median. Mean and 95% CI are plotted over raw data. The y axis is trimmed at the 10th and 90th percentiles of the observed data. ATV indicates atazanavir; DRV, darunavir; RAL, raltegravir; and RTV, ritonavir.

Table 3.

Distributions of CCA Entropy, Overall and by Treatment

Week Total Atazanavir/ritonavir Darunavir/ritonavir Raltegravir
(N=234) (N=68) (N=84) (N=82)
0 4.40 (4.17 to 4.60) 4.41 (4.16 to 4.59) 4.37 (4.19 to 4.62) 4.42 (4.26 to 4.59)
48 4.38 (4.20 to 4.53) 4.38 (4.20 to 4.57) 4.40 (4.19 to 4.52) 4.32 (4.23 to 4.52)
144 4.42 (4.26 to 4.56) 4.48 (4.31 to 4.60) 4.44 (4.31 to 4.54) 4.39 (4.19 to 4.52)
Change from baseline
48 0.02 (−0.13 to 0.15) 0.07 (−0.09 to 0.17) 0.03 (−0.13 to 0.12) −0.01 (−0.15 to 0.14)
144 0.04 (−0.12 to 0.15) 0.06 (−0.05 to 0.15) 0.08 (−0.09 to 0.15) −0.06 (−0.23 to 0.15)

Data are given as median (quartile 1 to quartile 3). CCA indicates common carotid artery.

Table 4.

Adjusted Treatment Group Differences in Ratef of Change in CCA Entropy

Variable β 95% Lower CI 95% Upper CI P value
Rate of change
Atazanavir/ritonavir 0.023 0.001 0.045 0.04
Darunavir/ritonavir 0.014 −0.006 0.033 0.17
Raltegravir −0.006 −0.026 0.014 0.56
Treatment group comparisons
Darunavir/ritonavir vs atazanavir/ritonavir −0.009 −0.041 0.023 0.51

Raltegravir vs Protease inhibitor/ritonavir

(atazanavir/ritonavir and darunavir/ritonavir)

−0.024 −0.051 0.003 0.05
Raltegravir vs atazanavir/ritonavir −0.029 −0.062 0.004 0.05

Adjusted for age, sex, race and ethnicity, smoking history, body mass index, duration of HIV infection, and baseline CD4+ T cells. β indicates estimated rate of change per year; and CCA, common carotid artery.

Table 5.

Distributions of CCA GLDS‐CON, Overall and by Treatment

Week

Total

(N=234)

Atazanavir/ritonavir

(N=68)

Darunavir/ritonavir

(N=84)

Raltegravir

(N=82)

0

112

(76.1 to 167)

113

(76.1 to 175)

109

(77.7 to 154)

114

(76.1 to 182)

48

114

(79.2 to 161)

120

(83.1 to 170)

114

(76.0 to 165)

101

(74.6 to 141)

144

104

(73.2 to 159)

124

(78.4 to 172)

102

(65.6 to 154)

98.3

(78.1 to 155)

Change from baseline
48

0.35

(−31.2 to 29.1)

5.4

(−35.8 to 46.6)

6.6

(−24.8 to 36.0)

−5.41

(−59.7 to 23.0)

144

−1.42

(−41.3 to 27.1)

0.54

(−34.1 to 29.4)

−1.42

(−42.3 to 25.7)

−7.04

(−47.9 to 26.3)

Data are given as median (quartile 1 to quartile 3). CCA indicates common carotid artery; and GLDS‐CON, gray‐level difference statistic–contrast.

Table 6.

Adjusted Treatment Group Differences in Rate of Change in CCA GLDS‐CON

Variable β 95% Lower CI 95% Upper CI P value
Rate of change
Atazanavir/ritonavir 0.004 −0.017 0.025 0.73
Darunavir/ritonavir −0.010 −0.029 0.008 0.27
Raltegravir −0.010 −0.029 0.009 0.32
Treatment group comparisons
Darunavir/ritonavir vs atazanavir/ritonavir −0.014 −0.045 0.017 0.31

Raltegravir vs Protease inhibitor/ritonavir

(atazanavir/ritonavir and darunavir/ritonavir)

−0.007 −0.033 0.020 0.57
Raltegravir vs atazanavir/ritonavir −0.013 −0.045 0.018 0.33

Adjusted for age, sex, race and ethnicity, smoking history, body mass index, duration of HIV infection, and baseline CD4+ T cells. β indicates estimated rate of change per year; CCA, common carotid artery; and GLDS‐CON, gray‐level difference statistic–contrast.

Table 7.

Distributions of CCA Grayscale Median, Overall and by Treatment

Week

Total

(N=234)

Atazanavir/ritonavir

(N=68)

Darunavir/ritonavir

(N=84)

Raltegravir

(N=82)

0 63.7 (48.3 to 80.6) 64.5 (51.1 to 80.7) 62.8 (45.0 to 74.2) 65.5 (51.7 to 84.8)
48 64.3 (51.6 to 77.8) 61.3 (51.9 to 78.9) 67.9 (52.2 to 76.3) 63.7 (49.2 to 78.5)
144 67.6 (53.4 to 78.8) 67.7 (53.4 to 76.6) 67.2 (55.0 to 82.9) 65.9 (49.2 to 80.9)
Change from baseline
48

0.48

(−7.94 to 9.0)

−1.25

(−7.94 to 9.0)

−1.25

(−7.94 to 9.0)

−1.25

(−7.94 to 9.0)

144

1.05

(−12.2 to 14.1)

1.33

(−12.2 to 14.1)

1.33

(−12.2 to 14.1)

1.33

(−12.2 to 14.1)

Data are given as median (quartile 1 to quartile 3). CCA indicates common carotid artery.

Table 8.

Adjusted Treatment Group Differences in Rate of Change in CCA Grayscale Median

Variable β 95% Lower CI 95% Upper CI P value
Rate of change
Atazanavir/ritonavir −0.005 −1.96 1.95 1.00
Darunavir/ritonavir 1.26 −0.46 2.98 0.15
Raltegravir −0.51 −2.31 1.29 0.57
Treatment group comparisons
Darunavir/ritonavir vs atazanavir/ritonavir 1.26 −1.56 4.09 0.31

Raltegravir vs Protease inhibitor/ritonavir

(atazanavir/ritonavir and darunavir/ritonavir)

−1.14 −3.54 1.26 0.28
Raltegravir vs atazanavir/ritonavir −0.51 −3.40 2.38 0.69

Adjusted for age, sex, race and ethnicity, smoking history, body mass index, duration of HIV infection, and baseline CD4+ T cells. β indicates estimated rate of change per year; and CCA, common carotid artery.

Brachial Artery Grayscale Measures

Correlations of brachial artery grayscale measures with ASCVD risk factors, markers of HIV infection, and inflammatory biomarkers at baseline and on treatment were weaker and differed slightly from those observed with the CCA grayscale measures. GSM correlated with systolic blood pressure (ρ=−0.21; P=0.003), low‐density lipoprotein cholesterol (ρ=−0.20; P=0.007), and CD4+ T cells (ρ=0.22; P=0.002). GLDS‐CON did not correlate significantly with any biomarker. Entropy correlated with CRP (ρ=−0.18; P=0.013) and interleukin‐6 (ρ=−0.14; P=0.046). After 48 weeks of ART, these correlations no longer were significant, although entropy on ART correlated with high‐density lipoprotein cholesterol (ρ=0.22; P=0.003). Changes in GSM over 48 weeks correlated with changes in CD4+ T cells (ρ=0.19; P=0.007) and CRP (ρ=−0.17; P=0.022). No other significant correlations of changes in brachial artery grayscale measures with changes in any biomarker were observed.

Correlations Between Grayscale Measures, Carotid Artery Intima‐Media Thickness, and Brachial Artery Flow‐Mediated Dilation

At the baseline visit, carotid artery intima‐media thickness correlated significantly with carotid artery GSM (ρ=−0.20; P=0.003) and GLDS‐CON (ρ=−0.23; P<0.001); correlations at week 144 were somewhat stronger: GSM (ρ=−0.25; P<0.001) and GLDS‐CON (ρ=−0.26; P<0.001). In the brachial arteries, none of the grayscale measures correlated significantly with brachial artery flow‐mediated dilation at baseline or at week 48. CCA and brachial artery GSM correlated significantly at baseline (ρ=0.40; P<0.001) and after 48 weeks (ρ=0.32; P<0.001). CCA and brachial artery GLDS‐CON correlated significantly at baseline (ρ=0.16; P=0.020), but not at week 48. No treatment‐related changes were apparent (all P>0.10).

Discussion

In ART‐naïve PWH, CCA grayscale ultrasound measures were associated with ASCVD risk factors, although with different patterns between grayscale measures. Lower GSM was associated with lower CD4+ T cells. On ART, CCA entropy improved in the atazanavir arm but not the other arms, although we did observe a nonsignificant improvement in the darunavir arm. Accordingly, the atazanavir and pooled protease inhibitor arms had significantly greater change in CCA entropy than the raltegravir arm. Arterial entropy is a first‐order statistic derived from the grayscale histogram that describes randomness of the pixels in an arterial image. In the MESA, CCA entropy predicted future ASCVD events, although less robustly than other measures, such as GSM and GLDS‐CON. 16 Overall and within arms, changes in CCA entropy were not associated with changes in any risk marker, including changes in bilirubin, with the exception of sCD163, a marker of macrophage activation. Indeed, our most remarkable finding was the consistent association of improvements in each of the 3 CCA grayscale measures with reductions in sCD163 levels. In ACTG Study A5260s, ART reduced sCD163 levels in all 3 arms, 24 suggesting that reduced macrophage activation attributable to effective ART in treatment‐naïve PWH is associated with consistent improvements in grayscale carotid artery texture features. These findings extend a robust body of literature in PWH, suggesting that macrophage activation, as measured by levels of sCD163, is related to carotid artery atherosclerotic plaque burden, coronary artery stenosis, noncalcified coronary artery, atherosclerotic plaque, and aortic inflammation. 10 , 11 , 12 , 13 In our study, CCA GSM and GLDS‐CON correlated inversely with carotid intima‐media thickness.

Although the CCA grayscale measures only changed modestly with ART, entropy appeared to improve in the atazanavir but not in the other arms, consistent with the primary finding of ACTG Study A5260s that participants assigned to the atazanavir arm had slower progression of carotid intima‐media thickness. 21 The marginal statistical significance of the effect of ART on CCA entropy in the atazanavir arm may be attributable to limited power, as our sample size was approximately one third less than in ACTG Study A5260s. Limited power also may explain the borderline levels of significance of many of the correlations we observed, considering the high biological and test variability of the laboratory parameters we measured.

Associations with the brachial artery grayscale measures were fewer and less strong; these measures also did not change with ART. Their weak correlations with CCA grayscale measures suggest that early structural changes in the brachial artery wall may not reflect those in the carotid arteries and grayscale changes between arterial beds are not interchangeable, perhaps because the brachial artery differs from the carotid artery in size, geometry, and cellular composition. 25

Our article has limitations. Ultrasound images were acquired from >20 field centers, which used different ultrasound machines. Grayscale ultrasound measures are highly sensitive to machine settings, which may have led to a null bias. 26 Reported time since diagnosis of HIV‐1 infection was not associated with any of the CCA grayscale measures at baseline or over time; however, this is an imprecise measure, because the exact date of seroconversion often is not known and duration of infection can affect arterial injury.27 Our study participants mostly were younger men, all of whom were ART naïve at its inception, which may limit generalizability. We also could not reliably assess the influence of cocaine, marijuana, and other illicit substances, which might influence changes in the arterial walls. Given the exploratory nature of our article, we did not adjust for multiple comparisons.

Conclusions

In ART‐naïve PWH, CCA grayscale ultrasound measures were associated with ASCVD risk factors and lower GSM was associated with lower CD4+ T cells. These grayscale ultrasound measures were not affected dramatically by ART, although CCA entropy improved with atazanavir‐containing ART. Reductions in sCD163 with initial ART were associated with improvements in all 3 CCA grayscale measures, suggesting that reductions in macrophage activation with ART initiation may lead to less arterial injury.

Sources of Funding

This work was supported by the National Institute of Allergy and Infectious Diseases of the National Institutes of Health (AI068634, AI068636, and UM1 AI106701). This research also was supported by the National Institutes of Health (HL146199, HL095132, HL095126, AI69471, AI56933, and OD010569) from the National Heart, Lung, and Blood Institute, the National Institute of Allergy and Infectious Diseases, and the Office of the Director. Pharmaceutical support was provided by Bristol‐Myers Squibb Company, Gilead Sciences, Inc, Merck & Co, Inc, and Tibotec Therapeutics. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Disclosures

Dr. Mitchell disclosed that she is a textbook author for Davies Publishing, Inc, authored a textbook chapter for Elsevier and Wolters‐Kluwer, and has received contracted research grants through the University of Wisconsin–Madison from W. L. Gore & Associates.

Acknowledgments

The assistance of the AIDS Clinical Trials Group (ACTG) Statistical and Data Analysis Center and the ACTG Optimization of Antiretroviral Therapy Committee, as well as the clinical trials support and other clinical trials specialists at Social and Scientific Systems, Inc, are greatly appreciated. Principal contributions of each author were as follows: Dr Hughey: data analysis and drafted manuscript. Dr Vuong: data analysis and drafted manuscript. Dr Ribaudo: design, conduct of study, data analysis, and critical revision of manuscript. Dr Mitchell: data analysis and critical revision of manuscript. Dr Korcarz: data analysis and critical revision of manuscript. Dr Hodis: design, conduct of study, and critical revision of manuscript. Dr Currier: conception, design, obtained funding, conduct of study, and critical revision of manuscript. Dr Stein: conception, design, obtained funding, conduct of study, data analysis, and critical revision of manuscript.

The following AIDS Clinical Trials Units participated in this study: 103: Beth Israel Deaconess Medical Center ACTG Clinical Research Site 6; 107: Brigham and Women’s Hospital Therapeutics ACTG Clinical Research Site 5; 201: Johns Hopkins University Clinical Research Site 11; 401: New York University HIV/AIDS Clinical Research Site 11; 601: University of California‐Los Angeles CARE Center Clinical Research Site 8; 603: Harbor‐University of California‐Los Angeles Medical Center Clinical Research Site 24; 801: UCSF AIDS Clinical Research Site 4; 1001: University of Pittsburgh Clinical Research Site 4; 1101: University of Rochester ACTG Clinical Research Site 4; 1108: Trillium Health ACTG Clinical Research Site 8; 1201: University of Southern California Clinical Research Site 30; 1401: University of Washington AIDS Clinical Research Site 18; 1601: Duke University Medical Center Adult Clinical Research Site 3; 2101: Washington University Therapeutics Clinical Research Site 23; 2301: Ohio State University AIDS Clinical Research Site 9; 2401: University of Cincinnati Clinical Research Site 28; 2501: Case Western Reserve Clinical Research Site 12; 2503: MetroHealth Clinical Research Site 1; 2701: Northwestern University Clinical Research Site 23; 2702: Rush University Medical Center ACTG Clinical Research Site 8; 3201: Chapel Hill Clinical Research Site 15; 3652: Vanderbilt Therapeutics Clinical Research Site 17; 5802: Ponce de Leon Center Clinical Research Site 3; 6101: University of Colorado Hospital Clinical Research Site 40; 31473: Houston AIDS Research Team Clinical Research Site 10; 31477: New Jersey Medical School–Adult Clinical Research Center Clinical Research Site 9.

For Sources of Funding and Disclosures, see page 11.

References

  • 1. Feinstein MJ, Bahiru E, Achenbach C, Longenecker CT, Hsue P, So‐Armah K, Freiberg MS, Lloyd‐Jones DM. Patterns of cardiovascular mortality for HIV‐infected adults in the United States: 1999 to 2013. Am J Cardiol. 2016;117:214–220. doi: 10.1016/j.amjcard.2015.10.030 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2. Shah ASV, Stelzle D, Lee KK, Beck EJ, Alam S, Clifford S, Longenecker CT, Strachan F, Bagchi S, Whiteley W, et al. Global burden of atherosclerotic cardiovascular disease in people living with HIV: systematic review and meta‐analysis. Circulation. 2018;138:1100–1112. doi: 10.1161/CIRCULATIONAHA.117.033369 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3. Freiberg MS, Chang C‐C, Kuller LH, Skanderson M, Lowy E, Kraemer KL, Butt AA, Bidwell Goetz M, Leaf D, Oursler KA, et al. HIV infection and the risk of acute myocardial infarction. JAMA Intern Med. 2013;173:614–622. doi: 10.1001/jamainternmed.2013.3728 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4. Grundy SM, Stone NJ, Bailey AL, Beam C, Birtcher KK, Blumenthal RS, Braun LT, de Ferranti S, Faiella‐Tommasino J, Forman DE, et al. 2018 AHA/ACC/AACVPR/AAPA/ABC/ACPM/ADA/AGS/APHA/ASPC/NLA/PCNA guideline on the management of blood cholesterol: a report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines. Circulation. 2019;139:e1082–e1143. doi: 10.1161/CIR.0000000000000625 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5. Kearns A, Gordon J, Burdo TH, Qin X. HIV‐1‐associated atherosclerosis: unraveling the missing link. J Am Coll Cardiol. 2017;69:3084–3098. doi: 10.1016/j.jacc.2017.05.012 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6. Stein JH, Currier JS, Hsue PY. Arterial disease in patients with human immunodeficiency virus infection: what has imaging taught us? JACC Cardiovasc Imaging. 2014;7:515–525. doi: 10.1016/j.jcmg.2013.08.019 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7. Triant VA, Meigs JB, Grinspoon SK. Association of C‐reactive protein and HIV infection with acute myocardial infarction. J Acquir Immune Defic Syndr. 2009;51:268–273. doi: 10.1097/QAI.0b013e3181a9992c [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8. Kuller LH, Tracy R, Belloso W, Wit SD, Drummond F, Lane HC, Ledergerber B, Lundgren J, Neuhaus J, Nixon D, et al. Inflammatory and coagulation biomarkers and mortality in patients with HIV infection. PLoS Med. 2008;5:e203. doi: 10.1371/journal.pmed.0050203 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9. Emery S, Neuhaus JA, Phillips AN, Babiker A, Cohen CJ, Gatell JM, Girard PM, Grund B, Law M, Losso MH, et al. Major clinical outcomes in antiretroviral therapy (ART)‐naive participants and in those not receiving ART at baseline in the smart study. J Infect Dis. 2008;197:1133–1144. doi: 10.1086/586713 [DOI] [PubMed] [Google Scholar]
  • 10. Hanna DB, Lin J, Post WS, Hodis HN, Xue X, Anastos K, Cohen MH, Gange SJ, Haberlen SA, Heath SL, et al. Association of macrophage inflammation biomarkers with progression of subclinical carotid artery atherosclerosis in HIV‐infected women and men. J Infect Dis. 2017;215:1352–1361. doi: 10.1093/infdis/jix082 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11. McKibben RA, Margolick JB, Grinspoon S, Li X, Palella FJ, Kingsley LA, Witt MD, George RT, Jacobson LP, Budoff M, et al. Elevated levels of monocyte activation markers are associated with subclinical atherosclerosis in men with and those without HIV infection. J Infect Dis. 2015;211:1219–1228. doi: 10.1093/infdis/jiu594 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12. Burdo TH, Lo J, Abbara S, Wei J, DeLelys ME, Preffer F, Rosenberg ES, Williams KC, Grinspoon S. Soluble CD163, a novel marker of activated macrophages, is elevated and associated with noncalcified coronary plaque in HIV‐infected patients. J Infect Dis. 2011;204:1227–1236. doi: 10.1093/infdis/jir520 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13. Subramanian S, Tawakol A, Burdo TH, Abbara S, Wei J, Vijayakumar J, Corsini E, Abdelbaky A, Zanni MV, Hoffmann U, et al. Arterial inflammation in patients with HIV. JAMA. 2012;308:379–386. doi: 10.1001/jama.2012.6698 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14. Wohlin M, Sundström J, Andrén B, Larsson A, Lind L. An echolucent carotid artery intima‐media complex is a new and independent predictor of mortality in an elderly male cohort. Atherosclerosis. 2009;205:486–491. doi: 10.1016/j.atherosclerosis.2009.01.032 [DOI] [PubMed] [Google Scholar]
  • 15. Andersson J, Sundström J, Gustavsson T, Hulthe J, Elmgren A, Zilmer K, Zilmer M, Lind L. Echogenecity of the carotid intima‐media complex is related to cardiovascular risk factors, dyslipidemia, oxidative stress and inflammation: the Prospective Investigation of the Vasculature in Uppsala Seniors (PIVUS) study. Atherosclerosis. 2009;204:612–618. doi: 10.1016/j.atherosclerosis.2008.10.038 [DOI] [PubMed] [Google Scholar]
  • 16. Mitchell CC, Korcarz CE, Gepner AD, Nye R, Young RL, Matsuzaki M, Post WS, Kaufman JD, McClelland RL, Stein JH. Carotid artery echolucency, texture features, and incident cardiovascular disease events: the MESA study. J Am Heart Assoc. 2019;8:e010875. doi: 10.1161/JAHA.118.010875 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17. Stein JH, Yeh E, Weber JM, Korcarz C, Ridker PM, Tawakol A, Hsue PY, Currier JS, Ribaudo H, Michell CKC. Brachial artery echogenicity and grayscale texture changes in HIV‐infected individuals receiving low‐dose methotrexate. Arterioscler Thromb Vasc Biol. 2018;38:2870–2878. doi: 10.1161/ATVBAHA.118.311807 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18. Berroug J, Korcarz CE, Mitchell CK, Weber JM, Tian L, McDermott MM, Stein JH. Brachial artery intima‐media thickness and grayscale texture changes in patients with peripheral artery disease receiving supervised exercise training in the PROPEL randomized clinical trial. Vasc Med. 2019;24:12–22. doi: 10.1177/1358863X18804050 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19. Jung M, Parrinello CM, Xue X, Mack WJ, Anastos K, Lazar JM, Selzer RH, Shircore AM, Plankey M, Tien P, et al. Echolucency of the carotid artery intima‐media complex and intima‐media thickness have different cardiovascular risk factor relationships: the women’s interagency HIV study. J Am Heart Assoc. 2015;4:e001405. doi: 10.1161/JAHA.114.001405 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20. Stein JH, Brown TT, Ribaudo HJ, Chen Y, Yan M, Lauer‐Brodell E, McComsey GA, Dubé MP, Murphy RL, Hodis HN, et al. Ultrasonographic measures of cardiovascular disease risk in antiretroviral treatment‐naive individuals with HIV infection. Aids. 2013;27:929–937. doi: 10.1097/QAD.0b013e32835ce27e [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21. Stein JH, Ribaudo HJ, Hodis HN, Brown TT, Tran TTT, Yan M, Brodell EL, Kelesidis T, McComsey GA, Dube MP, et al. A prospective, randomized clinical trial of antiretroviral therapies on carotid wall thickness. Aids. 2015;29:1775–1783. doi: 10.1097/QAD.0000000000000762 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22. Lennox JL, Landovitz RJ, Ribaudo HJ, Ofotokun I, Na LH, Godfrey C, Kuritzkes DR, Sagar M, Brown TT, Cohn SE, et al. Efficacy and tolerability of 3 nonnucleoside reverse transcriptase inhibitor‐sparing antiretroviral regimens for treatment‐naive volunteers infected with HIV‐1: a randomized, controlled equivalence trial. Ann Intern Med. 2014;161:461–471. doi: 10.7326/M14-1084 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23. LifeQMedical . Carotid plaque texture analysis research software for ultrasonic arterial wall and atherosclerotic plaques measurements. Operation Manual Version. 2013;Version;4.5. [Google Scholar]
  • 24. Kelesidis T, Tran TT, Stein JH, Brown TT, Moser C, Ribaudo HJ, Dube MP, Murphy R, Yang OO, Currier JS, et al. Changes in inflammation and immune activation with atazanavir‐, raltegravir‐, darunavir‐based initial antiviral therapy: ACTG 5260s. Clin Infect Dis. 2015;61:651–660. doi: 10.1093/cid/civ327 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25. Sorensen KE, Kristensen IB, Celermajer DS. Atherosclerosis in the human brachial artery. J Am Coll Cardiol. 1997;29:318–322. doi: 10.1016/S0735-1097(96)00474-3 [DOI] [PubMed] [Google Scholar]
  • 26. Steffel CN, Brown R, Korcarz CE, Varghese T, Stein JH, Wilbrand SM, Dempsey RJ, Mitchell CC. Influence of ultrasound system and gain on grayscale median values. J Ultrasound Med. 2019;38:307–319. doi: 10.1002/jum.14690 [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from Journal of the American Heart Association: Cardiovascular and Cerebrovascular Disease are provided here courtesy of Wiley

RESOURCES