Cardiovascular Disease Risk Prediction in the HIV Outpatient Study

Abstract

Background

Cardiovascular disease (CVD) risk prediction tools are often applied to populations beyond those in which they were designed when validated tools for specific subpopulations are unavailable.

Methods

Using data from 2283 human immunodeficiency virus (HIV)–infected adults aged ≥18 years, who were active in the HIV Outpatient Study (HOPS), we assessed performance of 3 commonly used CVD prediction models developed for general populations: Framingham general cardiovascular Risk Score (FRS), American College of Cardiology/American Heart Association Pooled Cohort equations (PCEs), and Systematic COronary Risk Evaluation (SCORE) high-risk equation, and 1 model developed in HIV-infected persons: the Data Collection on Adverse Effects of Anti-HIV Drugs (D:A:D) study equation. C-statistics assessed model discrimination and the ratio of expected to observed events (E/O) and Hosmer-Lemeshow χ² P value assessed calibration.

Results

From January 2002 through September 2013, 195 (8.5%) HOPS participants experienced an incident CVD event in 15 056 person-years. The FRS demonstrated moderate discrimination and was well calibrated (C-statistic: 0.66, E/O: 1.01, P = .89). The PCE and D:A:D risk equations demonstrated good discrimination but were less well calibrated (C-statistics: 0.71 and 0.72 and E/O: 0.88 and 0.80, respectively; P < .001 for both), whereas SCORE performed poorly (C-statistic: 0.59, E/O: 1.72; P = .48).

Conclusions

Only the FRS accurately estimated risk of CVD events, while PCE and D:A:D underestimated risk. Although these models could potentially be used to rank US HIV-infected individuals at higher or lower risk for CVD, the models may fail to identify substantial numbers of HIV-infected persons with elevated CVD risk who could potentially benefit from additional medical treatment.

Cardiovascular disease (CVD) risk is increased by 40%–75% among human immunodeficiency virus (HIV)–infected adults compared with non-HIV-infected individuals after accounting for traditional risk factors such as age, sex, hypertension, proatherogenic hyperlipidemia, smoking, and diabetes mellitus [1–5]. HIV-specific factors such as low CD4⁺ T-lymphocyte (CD4) counts or exposure to specific antiretroviral therapy (ART) have been found to be independent risk factors for incident CVD, with an effect magnitude comparable to that of several traditional CVD risk factors [1, 4, 6, 7].

Prediction equations based on traditional CVD risk factors have been developed in the general population [8–14] and are commonly used to aid in clinical management and estimate CVD risk in research studies. These CVD risk prediction tools were not designed specifically for use with HIV-infected persons, who appear to have independent HIV-related risk factors [15–17]. In practice, validated tools are often applied to populations beyond those in which they were developed when subpopulation-specific tools are unavailable.

The Data Collection on Adverse Events of Anti-HIV Drugs (D:A:D) Study CVD risk prediction equation was developed specifically for HIV-infected populations, taking into account traditional CVD risk factors plus exposure to specific ART [2, 18]. Recently, the D:A:D equation was updated addressing some critiques of the prior model by including CD4 count or removing ART use [19]. The D:A:D equations could be of use in HIV clinical and research settings, but evaluation of the tool in external cohorts has been limited. Because the availability of CVD risk prediction tools specifically designed for use in an HIV-infected population is limited, the objective of the present analysis was to assess the performance of the D:A:D tool and 3 other commonly used CVD risk prediction equations in a large, diverse cohort of HIV-infected US adults to determine if current tools can be used to adequately predict CVD risk in this population.

METHODS

Study Population

We analyzed data from the HIV Outpatient Study (HOPS), an ongoing, open, prospective cohort study of HIV-infected adults (aged ≥18 years) receiving care in HIV specialty clinics in the United States, described in detail elsewhere [20–22]. Information was abstracted from outpatient charts, entered electronically by trained staff, compiled centrally, reviewed, and edited before analysis. The vital status of participants without a care visit for at least 18 months was verified by periodic searches of the Social Security Death Index database. The HOPS protocol has been approved and renewed annually by the ethical review board of each participating institution and the Centers for Disease Control and Prevention. All study participants provided written informed consent.

Among 10 084 patients enrolled in the HOPS database as of 30 September 2013, we identified 2 283 participants (22.6%) who met the following criteria for inclusion in this analysis (Figure 1): attended ≥2 HOPS office visits on or after 1 January 2002 with first HOPS visit no later than 1 October 2010 (date selected a priori), had at least 1 year of follow-up, had no prior CVD events, and had at least 1 measurement each for total and high-density lipoprotein (HDL) cholesterol, and at least 2 systolic blood pressure measurements in the window between 12 months before and up to 9 months after the baseline date, defined as the later of the first HOPS visit or 1 January 2002 [1]. The follow-up observation period extended from the end of the baseline window of observation until the first of these occurred: a CVD event, death, or 30 September 2013.

Figure 1.

Flowchart of the included HIV Outpatient Study population. Abbreviations: CVD, cardiovascular disease; HDL, high-density lipoprotein; HIV, human immunodeficiency virus; HOPS, HIV Outpatient Study.

Cardiovascular Risk Prediction Equations

Four commonly used risk prediction equations were selected for evaluation. The Framingham general cardiovascular Risk Score (FRS) was developed to calculate 10-year CVD risk for men and women [10]. The American College of Cardiology/American Heart Association Pooled Cohort equations (PCEs) were recommended for use in 2013 to calculate 5- and 10-year atherosclerotic CVD risk and to help guide clinical treatment decisions specifically related to the use of cholesterol-lowering medications (statins) [13, 23, 24]. The Systematic COronary Risk Evaluation (SCORE) tool was developed in pooled cohorts from 12 European countries to estimate 10-year fatal CVD risk [12]. European guidelines for the management of dyslipidemias recommend the same low-density lipoprotein (LDL) cholesterol goal for HIV-infected and high-risk persons [25]; therefore, we applied the SCORE equation for high-risk populations in our main analyses and examined the equation for low-risk populations in a sensitivity analysis [25]. The D:A:D equation was developed using data from >22 000 HIV-infected adults from the United States, Europe, Argentina, and Australia and used to predict 5- and 10-year CVD risk [18]. Recently, updated D:A:D prediction models were published to include CD4 count in the full model and remove ART exposure in the reduced model [19]. We used the original model in analyses and later examined performance of the updated models in supplemental sensitivity analyses.

Independent (Predictor) Variables for Calculating Cardiovascular Risk

CVD risk factors included as predictors vary across equations (Table 1). Tobacco smoking (smoking) was ascertained from medical record at entry into HOPS and medical record and supplemental patient survey during HOPS observation. We defined diabetes as having ≥2 fasting blood glucose levels >125 mg/dL or 2 nonfasting glucose levels >200 mg/dL within a 6-month period, taking insulin or other antidiabetic medications for at least 30 days, having a hemoglobin A1c result >7%, or having a diabetes diagnosis. We coded participants as having no family history of CVD if such was not documented in their medical records. Use of ART was defined per standard criteria described previously [22]. Statin use was defined as prescription or reported use for at least 30 days, and aspirin use as any dose taken daily, during the baseline or follow-up observation periods.

Table 1.

Comparison of Study Populations, Risk Factors/Covariates Included, Measured Outcomes, and Target Populations of Selected Cardiovascular Disease Risk Prediction Equations

CVD Risk Prediction Equation	Year	Study Population	Risk Factors/Covariates	Measured Outcomes	Target Population
Framingham General Cardiovascular Risk Score [10]	2008	US adults from the Framingham Heart Study and Framingham Offspring Study	Age, sex, SBP, BP Rx, smoking, total cholesterol, HDL-C, diabetes	Composite CHD (coronary death, MI, coronary insufficiency, angina), cerebrovascular events (stroke, TIA), PAD, heart failure	Men and women aged 30–74 y
ACC/AHA Pooled Cohort equations [13]	2013	Adults from the general, noninstitutionalized population in 4 US-based studies	Age, sex, SBP, BP Rx, smoking, total cholesterol, HDL-C, LDL-C, diabetes, race/ethnicity	MI, fatal or nonfatal stroke, CHD death	African American and white men and women aged 40–79 y
Systematic Coronary Risk Evaluation (SCORE) high-risk equation [12]	2003	Adults from 12 European countries	Age, sex, SBP, smoking, total cholesterol	CHD death, fatal stroke	Men and women in Europe
Data Collection on Adverse Effects of Antiretroviral Drugs (D: A:D) Study equation [18]	2010	HIV-infected adults from clinics in the US, Europe, Argentina, and Australia	Age, sex, SBP, smoking, total cholesterol, diabetes, ever smoke, family history of CVD, years of use of indinavir or lopinavir, current abacavir use	Composite CVD (fatal or nonfatal MI including sudden death, stroke, TIA, invasive coronary artery procedures including CABG, angioplasty, death from other CHD)	HIV-infected adults prescribed antiretroviral therapy

CHD death included fatal MI, peripheral vascular disease (PVD), or coronary artery disease (CAD). PVD and CAD diagnoses were confirmed on the basis of aortography, angiography, or arterial Doppler.

Abbreviations: ACC/AHA, American College of Cardiology/American Heart Association; BP Rx, blood pressure treatment; CABG, coronary artery bypass graft; CHD, coronary heart disease; CVD, cardiovascular disease; HDL-C, high-density lipoprotein cholesterol; HIV, human immunodeficiency virus; LDL-C, low-density lipoprotein cholesterol; MI, myocardial infarction; PAD, peripheral arterial disease; SBP, systolic blood pressure; TIA, transient ischemic attack.

Outcome Measures

The CVD outcomes for each equation varied (Table 1). The outcomes were identified using standardized diagnostic and procedure codes (Supplementary Table 1), abstracted from HOPS patient charts, and confirmed by clinician review (K. A. L.). For patients who experienced multiple CVD events, the first event was defined as the incident event for the risk equation.

Statistical Analyses

We summarized the distribution of sociodemographic characteristics and CVD events using event counts and proportions or median values and interquartile ranges (IQRs), for categorical and continuous variables, respectively. We calculated individual risk according to the study-specific published formula for each risk prediction equation [10, 12, 13, 18, 19, 24]. We calculated C-statistics to estimate discrimination between individuals who did compared with those who did not experience a CVD event [26, 27] and considered a C-statistic between 0.50 and 0.59 to be poor; 0.60 and 0.69, moderate; 0.70 and 0.79, acceptable; and ≥0.80, very good to excellent [28]. Model calibration was determined using the ratio of expected to observed (E/O) CVD events; a Hosmer-Lemeshow χ² statistic with a low P value suggested poor model fit [29].

The highest risk stratum for each equation is the level at which drug intervention for cholesterol management was recommended in addition to lifestyle intervention. We categorized individuals into this highest risk stratum, then noted statin and aspirin use among these individuals to assess the proportion of HIV-infected persons at elevated CVD risk who are or are not receiving standard treatments for primary prevention of CVD according to each prediction equation. We examined concordance of risk score stratification (highest risk stratum vs other), using the κ coefficient as a measure of agreement; Pearson χ² was used to evaluate the difference in proportions classified in the highest risk stratum between FRS and each of the other prediction equations.

Sensitivity analyses examined model fit for 5-year CVD risk calculations for the PCE and D:A:D equations [18, 19, 24]. As the PCE was developed among persons aged 40–79 years, we limited the first sensitivity analysis to participants in this age range, then further restricted the sensitivity analysis to persons without diabetes, with LDL cholesterol level ≤190 mg/dL, and not prescribed statins during baseline or follow-up because patients without those high-risk conditions but with a CVD risk score ≥7.5% would be targeted as candidates for statin therapy [23, 24].

Analyses were conducted for the overall study population and were repeated for the subpopulation with ≥10 years of follow-up, to determine if the models performed better in HIV-infected adults with longer duration of observation. All statistical analyses were performed using SAS software, version 9.3 (SAS Institute, Cary, North Carolina).

RESULTS

Of the 2283 HOPS participants who met our inclusion criteria, 692 (30.3%) had ≥10 years of follow-up (Table 2). At baseline, comorbid conditions were common, with high prevalence of overweight or obesity (50.0%), hypertension (47.8%), and hypercholesterolemia (16.9%), and 13.3% had been prescribed statins. One-third (33.9%) had a prior AIDS-defining illness, the median nadir CD4 count was 211 cells/μL, and most (86.9%) participants used combination ART.

Table 2.

Characteristics and Antiretroviral Exposure of Patients in the Human Immunodeficiency Virus Outpatient Study, January 2002 –September 2010

Participant Characteristicsa	Median (IQR) or Proportion
	All HOPS Participants (N = 2283)	HOPS Participants With ≥10 y of Follow-up (n = 692)b
Year of HOPS entry
1998 or earlier	809 (35.4)	458 (66.2)
1999–2005	906 (39.7)	234 (33.8)
2006–2010	568 (24.9)	0 (0.0)
Age, y, median (IQR)	42.2 (36.4–48.4)	43.0 (38.1–48.4)
Male sex	1732 (75.9)	510 (73.7)
Race and ethnicity
White, non-Hispanic	1144 (50.1)	386 (55.8)
Black, non-Hispanic	775 (34.0)	207 (29.9)
Hispanic	295 (12.9)	77 (11.1)
Other	69 (3.0)	22 (3.2)
Insurance
Private	1219 (53.4)	398 (57.5)
Public, other	1064 (46.6)	294 (42.5)
History of injection drug use	222 (9.7)	71 (10.3)
Smokingc	952 (41.7)	241 (34.8)
Alcohol use
>14 drinks/week	104 (4.6)	22 (3.2)
7–14 drinks/week	141 (6.2)	42 (6.1)
<7 drinks/week	797 (34.9)	251 (36.3)
None (“never” or “previously”)	858 (37.6)	333 (48.1)
Missing information	383 (16.8)	44 (6.4)
Body mass index
Overweight or obese (≥25 kg/m2)	1141 (50.0)	383 (55.3)
Normal weight or underweight (<25 kg/m2)	1058 (46.3)	301 (43.5)
Missing information	84 (3.7)	8 (1.2)
Diabetes mellitusd	378 (16.6)	137 (19.8)
Hypertensione	1091 (47.8)	371 (53.6)
Systolic blood pressure, mm Hg, median (IQR)	120 (112–130)	120 (112–130)
Diastolic blood pressure, mm Hg, median (IQR)	80 (70–84)	80 (70–84)
Hypercholesterolemiaf	386 (16.9)	170 (24.6)
Total cholesterol, mg/dL, median (IQR)	185 (155–221)	200 (167–235)
LDL cholesterol, mg/dL, median (IQR)	104 (79–131)	112 (82–142)
HDL cholesterol, mg/dL, median (IQR)	40 (33–51)	41 (35–50)
Non-HDL cholesterol, mg/dL, median	139 (112–176)	158 (126–190)
Triglycerides, mg/dL, median	153 (100–256)	188 (118–288)
Statin useg	303 (13.3)	118 (17.1)
Aspirin useg	109 (4.8)	31 (4.5)
Family history of cardiovascular diseaseh	25 (1.1)	0 (0.0)
Years since HIV diagnosis, median (IQR)	6.1 (1.7–11.4)	8.0 (4.5–12.0)
Prior AIDS-defining illness	773 (33.9)	285 (41.2)
Nadir CD4+ cell count, cells/μL, median (IQR)	211 (68–370)	185 (50–333)
Baseline CD4+ cell count, cells/μL, median (IQR)	396 (229–600)	424 (300–660)
Viral load, copies/mL, median (IQR)	490 (25–21 600)	147 (25–1833)
ART use at baselineg	2055 (90.0)	661 (95.5)
Combination ARTi	1984 (86.9)	640 (92.5)
NRTIs	2025 (88.7)	655 (94.7)
Abacavir	620 (27.2)	201 (29.0)
Didanosine	388 (17.0)	163 (23.6)
NNRTIs	1072 (47.0)	358 (51.7)
Efavirenz	731 (32.0)	217 (31.4)
Boosted protease inhibitors	853 (37.4)	213 (30.8)
Indinavirj	321 (14.1)	172 (24.9)
Lopinavir	527 (23.1)	157 (22.7)
Cumulative years of ART use among usersk, median (IQR)	4.4 (2.1–6.5)	5.0 (3.1–6.9)
Combination ART	3.7 (1.8–5.0)	4.1 (2.5–5.1)
NRTIs	4.4 (2.1–6.4)	5.0 (3.0–6.7)
Abacavir	1.1 (0.5–2.3)	1.1 (0.4–2.1)
Didanosine	1.4 (0.6–3.0)	1.6 (0.7–3.4)
NNRTIs	1.6 (0.7–3.0)	1.8 (0.8–2.9)
Efavirenz	1.2 (0.4–2.5)	1.3 (0.5–2.4)
Boosted protease inhibitors	0.9 (0.4–1.5)	0.8 (0.4–1.2)
Indinavir	2.2 (1.0–4.0)	2.5 (1.1–4.4)
Lopinavir	0.7 (0.4–1.3)	0.7 (0.4–1.1)

Data are presented as No. (%) unless otherwise indicated.

Abbreviations: ART, antiretroviral therapy; HDL, high-density lipoprotein; Hg, hemoglobin; HIV, human immunodeficiency virus; HOPS, HIV Outpatient Study; IQR, interquartile range; LDL, low-density lipoprotein; NNRTI, nonnucleoside reverse transcriptase inhibitor; NRTI, nucleoside reverse transcriptase inhibitor.

^aBaseline date is the later of first HOPS visit or 1 January 2002. Baseline window of observation is from 365 days before through 275 days after baseline date.

^bSince the later of first HOPS visit or 1 January 2002.

^cTobacco smoking (smoking) was ascertained per medical record at entry into the HOPS and by medical record and supplemental patient survey during HOPS observation.

^dTwo fasting glucose levels >125 mg/dL or 2 nonfasting glucose levels >200 mg/dL within a 6-month period, taking insulin or other antidiabetic medications for at least 30 days, having a hemoglobin A1c result >7% but not during pregnancy, or having a diagnosis of diabetes.

^eBlood pressure ≥140 mm Hg systolic or ≥90 mm Hg diastolic or current use of blood pressure medications.

^fTotal cholesterol ≥240 mg/dL or LDL ≥ 160 mg/dL.

^gAny use during baseline window, defined above.

^hIf information was unavailable, the participant was coded as having no family history of cardiovascular disease.

ⁱCombination antiretroviral therapy was defined as the following regimens: (1) any combination of 3 antiretrovirals that included a protease inhibitor (PI); NNRTI; a fusion, entry, or integrase inhibitor; or a CCR5 antagonist; (2) any combination of 3 NRTIs that included abacavir (ABC) or tenofovir disoproxil fumarate (TDF), with the exception of the following combinations: ABC + TDF + lamivudine and didanosine + TDF + lamivudine; (3) 2 full-dose PIs; (4) a ritonavir-boosted PI combined with either an NNRTI or fusion inhibitor; (5) an integrase inhibitor combined with either a PI, an NNRTI, an entry inhibitor or a CCR5 antagonist [21].

^jOf the HOPS patients who ever took indinavir during the baseline window, 137 (42.7%) of all participants and 67 (39.0%) of those with ≥10 years of follow-up took it with either full-dose ritonavir or low-dose ritonavir.

^kFrom start of antiretroviral use to baseline date.

During the 15 056 person-years of observation (median follow-up, 6.5 years [IQR, 3.3–10.4 years]), 195 (8.5%) participants experienced an incident CVD event, and 107 participants (4.7%) died; 18 deaths (16.8%) were attributed to fatal myocardial infarction, stroke, peripheral vascular disease, or coronary artery disease (Table 3). The number of events included in each model was 199, 161, 18, and 220 among the entire population and 79, 64, 0, and 71 among participants with ≥10 years of follow-up, for the FRS, PCE, SCORE, and D:A:D, respectively.

Table 3.

Follow-up Information and Incident Cardiovascular Disease Events of Patients in the Human Immunodeficiency Virus Outpatient Study, January 2002–September 2013

Participant Characteristic	HOPS Participants
	Total (N = 2283)	≥10 y of Follow-upa (n = 692)
Person-years of follow-up	15 056	7495
Years of follow-up, median (IQR)	6.5 (3.3–10.4)	10.9 (10.5–11.2)
Statin use during follow-upb	666 (29.2)	305 (44.1)
Aspirin use during follow-upc	331 (14.5)	152 (22.0)
Patients with incident cardiovascular disease	195 (8.5)	80 (11.6)
Nonfatal cardiovascular disease events/proceduresd
Angina	28 (14.4)	13 (16.3)
Angioplasty	11 (5.6)	7 (8.8)
Cerebrovascular accident (stroke)	28 (14.4)	12 (15.0)
Coronary artery bypass graft	7 (3.6)	5 (6.3)
Coronary artery disease	94 (48.2)	46 (57.5)
Coronary stent (diagnosis or treatment)	6 (3.1)	4 (5.0)
Heart failure	10 (5.1)	2 (2.5)
Myocardial infarction	34 (17.4)	16 (20.0)
Peripheral vascular disease	22 (11.3)	8 (10.0)
Transient ischemic attack	17 (8.7)	5 (6.3)
Fatal cardiovascular diseasee event	18 (0.8)	0 (0.0)
Noncardiovascular disease cause of death	89 (3.9)	3 (0.4)
Events included as outcomes, by risk prediction equation
Framingham general cardiovascular risk score	199 (8.7)	79 (11.4)
ACC/AHA Pooled Cohort equations	151 (6.6)	64 (9.2)
SCORE equation	18 (0.8)	0 (0.0)
D:A:D Risk Equation	220 (9.6)	71 (10.3)

Data are presented as No. (%) unless otherwise indicated. Study-specific outcomes are as follows: Framingham general cardiovascular Risk Score—composite coronary heart disease (coronary death, myocardial infarction [MI], coronary insufficiency, angina), cerebrovascular events (stroke, transient ischemic attack), peripheral arterial disease, heart failure; ACC/AHA Pooled Cohort equations—MI, stroke, coronary artery disease (CAD); SCORE—fatal MI, stroke, peripheral vascular disease (PVD), CAD; D:A:D Study equation—fatal or nonfatal MI (including sudden death), CAD, stroke, death from other coronary heart disease. A total of 195 persons experienced an incident cardiovascular disease (CVD) event that was included in at least 1 of the CVD risk prediction equations. The number of events exceeds the number of individuals because outcomes differed by study equation. For patients who experienced multiple CVD events, the first event was defined as the incident event for the risk equation.

Abbreviations: ACC/AHA, American College of Cardiology/American Heart Association; D:A:D:, Data Collection on Adverse Effects of Antiretroviral Drugs; HOPS, HIV Outpatient Study; IQR, interquartile range; SCORE, Systematic COronary Risk Evaluation.

^aSince the later of first HOPS visit or 1 January 2002.

^bDefined as prescription or reported use of statins for at least 30 days since baseline.

^cDefined as prescribed or reported use of any dose of aspirin daily for at least 6 months during follow-up.

^dAmong the patients with incident CVD.

^eIncluded: fatal MI, fatal stroke, fatal PVD, and fatal CAD.

Among participants with any length of follow-up, 3 risk prediction equations demonstrated moderate to acceptable discrimination (Table 4), distinguishing persons with higher vs lower risk for CVD (C-statistics: 0.66, 0.71, and 0.72 for FRS, PCE, and D:A:D, respectively). The FRS was well calibrated for this population (E/O: 1.01; P = .89) but the PCE and D:A:D were less well calibrated, underestimating the 10-year CVD risk by 12% to 20% (E/O: 0.88 and 0.80; P < .001, P < .001 respectively). SCORE high-risk equation demonstrated poor discrimination (C-statistic: 0.59) and calibration, overestimating risk of CVD death by 72% (E/O: 1.72; P = .48). Among participants with ≥10 years of follow-up, we found no improvement in measures of calibration or discrimination for FRS, PCD, or D:A:D and were unable to fit the model for SCORE due to lack of observed events.

Table 4.

Performance of Equations for Predicting Cardiovascular Disease Outcomes in the Human Immunodeficiency Virus Outpatient Study Cohort

HOPS Participants	Framingham General Cardiovascular Risk Score	10-Year Cardiovascular Disease Risk Estimation
		ACC/AHA Pooled Cohort Equations	SCORE (High-Risk) Equation	D:A:D Equation
Any length of follow-up (N = 2283)
Median risk score (IQR)	6.3 (3.3–11.7)	3.3 (1.2–7.3)	0.5 (0.1–1.4)	4.7 (2.3–9.7)
Expected/observed events	201/199	133/151	31/18	175/220
Ratio expected/observeda	1.01	0.88	1.72	0.80
Hosmer-Lemeshow χ2	2.97	28.19	7.51	35.44
Hosmer-Lemeshow P valueb	0.89	<0.001	0.48	<0.001
C-statisticc	0.66	0.71	0.59	0.72
≥10 y of follow-up (n = 692)
Median risk score (IQR)	6.7 (3.3–13.2)	3.7 (1.4–7.9)	0.6 (0.2–1.5)	5.7 (2.9–10.3)
Expected/observed events	64/79	41/64	10/0	55/71
Ratio expected/observeda	0.81	0.64	NRd	0.77
Hosmer-Lemeshow χ2	6.89	11.32	NRd	14.28
Hosmer-Lemeshow P valueb	0.44	0.18	NRd	0.07
C-statisticc	0.68	0.68	NRd	0.70

Study-specific outcomes are as follows: Framingham general cardiovascular Risk Score—composite coronary heart disease (coronary death, myocardial infarction [MI], coronary insufficiency, angina), cerebrovascular events (stroke, transient ischemic attack), peripheral arterial disease, heart failure; ACC/AHA Pooled Cohort equations—MI, stroke, coronary artery disease (CAD); SCORE—fatal MI, stroke, peripheral vascular disease, CAD; D:A:D Study equation—fatal or nonfatal MI (including sudden death), CAD, stroke, death from other coronary heart disease.

Abbreviations: ACC/AHA, American College of Cardiology/American Heart Association; D:A:D:, Data Collection on Adverse Effects of Antiretroviral Drugs; HOPS, HIV Outpatient Study; IQR, interquartile range; NR, not reported; SCORE, Systematic COronary Risk Evaluation.

^aA ratio of expected/observed events of 1.0 indicates that the number of expected events equals the number of observed events. More events are observed than expected if the ratio expected/observed is <1.0; that is, a ratio of 0.6 is 40% less than 1.0 and would indicate a decrease or underestimation of 40%.

^bA Hosmer-Lemeshow χ² P value <.05 indicates poor model calibration.

^cThresholds to determine discriminative ability of our models: a C-statistic between 0.50 and 0.59, poor; 0.60 and 0.69, moderate; 0.70 and 0.79, good; ≥0.80, very good to excellent.

^dDue to small number of expected and observed events in SCORE, the ratio of expected/observed events, Hosmer-Lemeshow χ² P value, and C-statistic are not reported.

Each equation classified approximately one-quarter of all participants into the high-risk stratum (Table 5), and we found high concordance in cardiovascular risk score stratification (Supplementary Table 2), demonstrated by κ coefficients >0.90 comparing FRS with PCE, SCORE, and D:A:D. Among all participants classified as being at high risk at baseline, approximately half were prescribed statins and nearly one-third were prescribed aspirin during follow-up. Among persons with ≥10 years of follow-up, 24.3%–27.9% were classified in the highest risk stratum, approximately two-thirds were prescribed statins, and approximately 2 of 5 were prescribed aspirin during follow-up.

Table 5.

Prevalence of Statin and Aspirin Use at Follow-up Among Human Immunodeficiency Virus Outpatient Study Participants at Highest 10-Year Risk of Cardiovascular Disease Events, by Risk Prediction Equation

Risk Equation	Definition of Highest Risk Stratum	Any Length of Follow-up (N = 2283)			≥10 y of Follow-up (n = 692)
		Highest Risk Stratum	Use at Follow-up Among Highest Risk Stratum		Highest Risk Stratum	Use at Follow-up Among Highest Risk Stratum
			Statins	Aspirin		Statins	Aspirin
Framingham general cardiovascular Risk Score [10]	>20% risk or a CVD risk equivalenta	491 (21.5)	251 (51.1)	153 (31.2)	168 (24.3)	112 (66.7)	68 (40.5)
ACC/AHA Pooled Cohort equations [13]	≥7.5% risk	553 (24.2)	258 (46.7)	169 (30.6)	184 (26.6)	112 (60.9)	74 (40.2)
SCORE high-risk equation [12]	≥10% risk or has diabetes, CKD, or very high levels of individual risk factorsb	556 (24.4)	264 (47.5)	150 (27.0)	193 (27.9)	123 (63.7)	66 (34.2)
D:A:D Study equation [18]	>10% risk	549 (24.0)	254 (46.3)	159 (29.0)	185 (26.7)	115 (62.2)	74 (40.0)

Data are presented as No. (%). Study-specific outcomes are as follows: Framingham general cardiovascular Risk Score—composite coronary heart disease (coronary death, myocardial infarction [MI], coronary insufficiency, angina), cerebrovascular events (stroke, transient ischemic attack), peripheral arterial disease, heart failure; ACC/AHA Pooled Cohort equations—MI, stroke, coronary artery disease (CAD); SCORE—fatal MI, stroke, peripheral vascular disease, CAD; D:A:D Study equation—fatal or nonfatal MI (including sudden death), CAD, stroke, death from other coronary heart disease.

Abbreviations: ACC/AHA, American College of Cardiology/American Heart Association; CKD, chronic kidney disease; CVD, cardiovascular disease; D:A:D:, Data Collection on Adverse Effects of Antiretroviral Drugs; HIV, human immunodeficiency virus; HOPS, HIV Outpatient Study; SCORE, Systematic COronary Risk Evaluation.

^aCoronary heart disease risk equivalents included diabetes mellitus or prior peripheral or central arterial disease.

^b

Individuals at very high risk include those with any of the following:

• Documented CVD by invasive or noninvasive testing, previous MI, acute coronary syndrome, coronary and other arterial revascularization procedures, ischemic stroke, or peripheral arterial disease.
• Patients with type 2 diabetes or patients with type 1 diabetes with target organ damage (such as microalbuminuria).
• Patients with moderate to severe CKD (glomerular filtration rate <60 mL/minute/1.73 m²).

Estimation of 5-year CVD risk using the PCE and D:A:D equations revealed greater underestimation of events than in the main findings, and the updated D:A:D models failed to improve event estimation in the HOPS cohort (Supplementary Table 3). The PCE demonstrated poor to moderate discriminatory ability and underestimated CVD events when we limited the analysis to persons 40–79 years of age (Supplementary Table 4), and when we further restricted the analysis to persons without diabetes, with an LDL cholesterol level <190 mg/dL, and who were not prescribed statins at baseline or during follow-up (Supplementary Table 5). Finally, the SCORE equation for low-risk populations underestimated fatal CVD events (E/O: 0.94; P = .13) and had poor discrimination (C-statistic: 0.55) (Supplementary Table 6).

DISCUSSION

In this large, diverse cohort of HIV-infected US adults receiving medical care, 4 widely used risk prediction equations inaccurately predicted CVD risk. The FRS, PCE, and D:A:D equations were able to adequately distinguish and rank US HIV-infected adults with higher or lower risk for CVD as evidenced by the C-statistic but only the FRS accurately estimated risk of CVD events at the population level; PCE and D:A:D underestimated risk. Although nearly 1 in 4 HIV-infected adults in this cohort were classified at highest risk for CVD, statin and aspirin therapy were underprescribed. Progress is needed to more consistently identify HIV-infected US adults at risk for CVD to improve treatment for primary prevention of CVD.

Risk prediction tools for CVD and other conditions provide information that can help patients understand their risk of a life-threatening illness and can help clinicians tailor recommendations for lifestyle changes, specific medication therapy, or other preventive interventions to patients according to risk [23, 30–32]. These risk prediction tools may provide accurate estimates of long-term risk [23], but the PCE particularly has been shown to overestimate CVD risk in general populations [33, 34]. In contrast, our findings support another recent study that reported that the FRS and PCE underestimate CVD risk in HIV-infected populations [35]. The difference in ability of these models to predict CVD in the general population vs an HIV-infected population may lie in a yet-undetermined set of HIV-specific CVD risk factors such as low CD4 count, exposure to antiretroviral medications, inflammation, or endothelial dysfunction. We also found that the D:A:D risk equation underestimated CVD risk in the HOPS cohort; in contrast, internal–external validation studies in the D:A:D cohort reported that the earlier and recently updated D:A:D equations more accurately predicted CVD risk than the FRS, which overestimated risk [18, 19]. The reasons for differences in study findings are not clear but may be due to population differences in cohorts studied, different prescribing practices in Europe and the United States, or differences in calendar periods studied and the extent to which participants were exposed to proatherogenic antiretroviral medications. Not surprisingly, performance of predictive equations typically demonstrates worse fit when applied to a data set or population that is different than the validation sample; however, they may still be useful if discriminatory ability is adequate.

We found concordance in the proportion of individuals classified as high risk by each of the CVD prediction equations we evaluated but exceeded the proportion of HIV-infected patients classified in the highest CVD risk stratum in other reports [36, 37]. Other studies have reported moderate or low concordance in CVD risk score stratification among risk prediction equations [36–38]. The HOPS cohort had a higher prevalence of overweight or obese participants than that reported in other HIV-infected populations to which the risk prediction equations have been applied [18, 19, 37, 38]. The HOPS cohort may have differed from these other populations in additional important ways, including by sociodemographic and insurance status, or healthcare-seeking behaviors that we could not assess. SCORE for high-risk populations was the only risk prediction equation that overestimated CVD risk among HOPS participants, but this equation was developed in European cohorts in which there was a lower prevalence of cardiovascular risk factors and disease. SCORE used the most limited set of risk factors in the prediction model and only examined fatal CVD events. The predictive ability of SCORE may have been diminished by the paucity of fatal CVD events in the HOPS population.

Candidate factors for inclusion in a new risk prediction equation for US HIV-infected persons may include nontraditional factors associated with increased CVD risk and routinely or easily monitored as part of HIV-related clinical care, such as CD4 counts and HIV RNA levels [1, 18, 19]. Although the D:A:D prediction equation was recently updated to include CD4 count (full model) or to remove ART exposure (reduced model) [19] and may improve prediction in a predominantly European cohort, the updated equation failed to provide substantially improved prediction in the US-based HOPS cohort (Supplementary Table 3), potentially due to the population-level differences previously described. HIV-associated inflammation and endothelial dysfunction may underlie the excess CVD risk among HIV-infected populations [1, 4, 39]; however, inclusion of any single measure of inflammation or endothelial dysfunction has not improved CVD risk prediction in models developed in the general population [10, 12]. Lipodystrophy has also been associated with increased CVD risk [40], and simple anthropometric measurements such as body mass index or waist-to-hip ratios can be calculated during routine clinical visits. Revisions to existing multivariable prediction models to recalibrate them for US HIV-infected adults, include HIV-specific CVD risk factors, or develop new prediction models should be explored to improve CVD prediction in HIV-infected populations [30]. Currently HIV infection is not considered a CVD risk equivalent; further research to develop a CVD risk prediction equation tailored to HIV-infected populations may more accurately predict true CVD risk in this patient group and thereby enhance clinical decision making.

We note several limitations to our study. First, CVD events were abstracted from existing medical records of patients cared for in clinical practice settings, not collected through scheduled periodic laboratory and physical exams. Family history of CVD was not reliably recorded for all HOPS participants; this limitation could account for some of the underestimation in CVD risk we found using the D:A:D equation. About half of all participants in the HOPS database were not assessed for eligibility for this analysis because they were no longer active at our baseline date, had inadequate follow-up, or had prior CVD; another one-quarter were excluded because they were missing measurements (Figure 1). Individuals included in the analysis differed from those excluded (Supplementary Tables 7 and 8), resulting in an analysis sample and findings that may not be generalizable to all HIV-infected adults in care the United States. Fasting status at time of blood draw was frequently not documented, and an excess of nonfasting specimens may have led to an overestimation of the prevalence of diabetes and elevated LDL cholesterol. The performance of the predictive equations will vary with the prevalence of treatments for primary prevention. None of the equations accounted for statin or aspirin use and had poorer discrimination among those who were not on statins at baseline or follow-up. Failure to account for some statin use may have contributed to the inability of the equations to adequately distinguish between persons who did or did not have an event. Finally, not all patients were observed for 10 years or longer, and we observed fewer events than would be expected. Consequently, our results may provide conservative estimates of the extent to which the current CVD risk prediction equations underestimate risk in US HIV-infected patients.

In conclusion, current CVD risk prediction equations were able to adequately distinguish US HIV-infected adults with higher or lower CVD risk, but inaccurately predicted CVD events. Among those at highest CVD risk, less than half reported statin and/or aspirin use. These models may fail to identify substantial numbers of HIV-infected persons with elevated CVD risk who could potentially benefit from additional medical treatment.