Lessons Learned From the Design and Implementation of Myocardial Infarction Adjudication Tailored for HIV Clinical Cohorts

Abstract

We developed, implemented, and evaluated a myocardial infarction (MI) adjudication protocol for cohort research of human immunodeficiency virus. Potential events were identified through the centralized Centers for AIDS Research Network of Integrated Clinical Systems data repository using MI diagnoses and/or cardiac enzyme laboratory results (1995–2012). Sites assembled de-identified packets, including physician notes and results from electrocardiograms, procedures, and laboratory tests. Information pertaining to the specific antiretroviral medications used was redacted for blinded review. Two experts reviewed each packet, and a third review was conducted if discrepancies occurred. Reviewers categorized probable/definite MIs as primary or secondary and identified secondary causes of MIs. The positive predictive value and sensitivity for each identification/ascertainment method were calculated. Of the 1,119 potential events that were adjudicated, 294 (26%) were definite/probable MIs. Almost as many secondary (48%) as primary (52%) MIs occurred, often as the result of sepsis or cocaine use. Of the patients with adjudicated definite/probable MIs, 78% had elevated troponin concentrations (positive predictive value = 57%, 95% confidence interval: 52, 62); however, only 44% had clinical diagnoses of MI (positive predictive value = 45%, 95% confidence interval: 39, 51). We found that central adjudication is crucial and that clinical diagnoses alone are insufficient for ascertainment of MI. Over half of the events ultimately determined to be MIs were not identified by clinical diagnoses. Adjudication protocols used in traditional cardiovascular disease cohorts facilitate cross-cohort comparisons but do not address issues such as identifying secondary MIs that may be common in persons with human immunodeficiency virus.

Many questions remain unanswered about the risk of cardiovascular disease (CVD), and particularly myocardial infarction (MI), in persons living with human immunodeficiency virus (HIV). Antiretroviral therapy has reduced morbidity and mortality rates among persons living with HIV (1–4), but it has also been thought to increase CVD risk (5, 6). However, the Strategies for Management of Antiretroviral Therapy (SMART) Study found that CVD incidence is lower among persons who receive uninterrupted treatment than among those who receive delayed or intermittent antiretroviral therapy (7). HIV infection may alter lipid metabolism and worsen endothelial function, and it is associated with greater carotid intima medial thickness, which can lead to CVD (8–11). Additional studies are needed to further understand the impact of HIV and its treatment on CVD.

The study of CVD requires clearly defined clinical endpoints and accurate identification of events. Identification of CVD events among HIV-uninfected individuals using data from electronic health records (EHRs) is complex and prone to errors (12). Therefore, some studies have conducted CVD event adjudication (13–15) because it improves accuracy in comparison with more streamlined endpoint ascertainment (16–18). However, in many studies of CVD and HIV, investigators relied on administrative diagnosis codes and other nonadjudicated outcomes (19–24) that are known to lead to the misclassification and overestimation of true event rates (25–27).

We developed and implemented an MI adjudication protocol for HIV cohort research in the Centers for AIDS Research Network of Integrated Clinical Systems (CNICS) cohort (28). The MI protocol addresses issues that are specific to persons with HIV and enables comparisons with traditional CVD cohort studies. A central, independent review committee adjudicates all suspected MIs identified by screening comprehensive clinical data while blinded to potential confounders, such as use of particular antiretroviral medications (ARVs). We evaluated the sensitivity and positive predictive value of different criteria that could be used to ascertain potential MI events, using adjudication as the gold standard.

METHODS

Study cohort

The present observational study was conducted in the CNICS cohort, which includes more than 27,000 persons living with HIV and receiving care at 8 clinical sites across the United States from 1995 to the present (28). All participating sites provide primary and subspecialty care and have EHRs that span both inpatient and outpatient settings. Potential MI events from 5 sites (University of Alabama at Birmingham, Birmingham, Alabama; University of Washington, Seattle, Washington; University of California at San Diego, San Diego, California; University of North Carolina at Chapel Hill, Chapel Hill, North Carolina; and Johns Hopkins University, Baltimore, Maryland) were included in this analysis. We included all events adjudicated between March 2010 and December 2012; event dates were between January 1, 1996, and January 6, 2012. Sites received institutional review board approval for the CNICS Study.

CNICS data

The CNICS data repository systematically captures demographic, clinical, medication, and laboratory data for all patients receiving care at each CNICS site from the EHRs and other institutional data systems (28). Clinical diagnoses are recorded in the EHRs by the treating physician using standardized diagnoses and International Classification of Diseases, Ninth Revision, coding. Data quality assessment is conducted at the sites before data transmission and at the time of submission to the Data Management Core before insertion into the central repository. Data undergo extensive quality-assurance procedures, and data quality issues are reported to sites to be investigated and corrected. Data from each site are updated, reviewed, and integrated into the repository quarterly (28).

Ascertainment of potential MI events

Potential MI events are identified centrally for review based on a standard protocol applied to laboratory and diagnosis data in the CNICS data repository. Criteria for ascertainment of a potential MI event included: 1) inpatient or outpatient clinical MI diagnosis of an acute or unspecified timeframe (such as International Classification of Diseases, Ninth Revision, codes 410.00, 410.01, and 410.10) and/or 2) cardiac enzyme elevation above the laboratory-specific upper limit of normal for troponin-I, troponin-T, and creatine kinase MB. Use of specific cardiac enzyme tests varied by site and over time. One or more elevated values of any of these was sufficient to meet ascertainment criteria.

As part of protocol development, we also examined all potential events (n = 48) at one site (University of Washington) that could be identified by additional diagnosis and procedure codes (such as codes 37.22, 37.23, 411.0, 411.1, 414.04, and 428.0), including congestive heart failure, cardiac catheterization, or coronary artery bypass graft surgery. Adding these criteria resulted in no additional adjudicated MIs beyond those already identified; therefore, these additional codes were not included in the ascertainment criteria.

MI review packet assembly

For every potential MI event that was identified, investigators from each site assembled a standardized set of computerized clinical information (in the form of Adobe PDF or compressed documents) for central review that contained the following:

• Physician's notes made closest to the potential MI date, including admission, transfer, discharge, clinic, and emergency department notes, inpatient cardiology consultation notes, and autopsy reports;

• Documentation pertaining to the first 3 outpatient cardiology consultations or visits after the potential MI date;

• Baseline electrocardiogram (ECG) (before the MI date), if available;

• First 2 ECGs after admission or event date (includes ECGs obtained in the emergency department), the last ECG before discharge, and the last ECG recorded on day 3 (or the first ECG thereafter) after admission or in-hospital event;

• Results from related procedure and diagnostic tests performed around and after the potential MI event, including stress test, cardiac echocardiography, cardiac radionuclide imaging, cardiac magnetic resonance imaging, cardiac computed tomography, and cardiac catheterization results, as well as operative reports from coronary artery bypass graft surgery; and

• Related laboratory values measured near the potential MI event, including creatine kinase MB and troponin results.

Information regarding which ARVs had been prescribed was redacted from the packet. Completed packets were uploaded to the CNICS web-based MI platform. Investigators at the sites were asked to document reasons for missing and incomplete packets, such as potential events that occurred outside the hospital system, and were asked to make 2 attempts to obtain outside records before declaring information unavailable. Investigators could also document when the ascertainment of an MI was an error, thereby precluding the need to assemble data. Finally, if ascertainment identified an event that was determined to have occurred previously, investigators were asked to identify the approximate timing of the earlier event. A new packet request was generated with the earlier event date.

Packet data were available for review to users with appropriate permissions managed through an open standards-compliant, secure, federated single sign-on identity service (http://www.protectnetwork.org/), using an application that permits review of the data and assignment of adjudication results into a database linked by CNICS study identifiers. This supports easy, distributed access to the adjudication packets and efficient management of adjudication results while allowing reviewers to be selected without regard for physical location.

MI adjudication criteria

The criteria used to classify events included reports of chest pain, elevated cardiac enzymes, and abnormal ECGs. The CNICS MI criteria were adapted from the Multi-Ethnic Study of Atherosclerosis (MESA) (14) and the universal MI definition (29) with input from experts in the field, including MESA investigators. A web application designed for CNICS MI adjudication enabled reviewers to classify the MI event and enter additional standardized data. Capturing standardized review criteria provided the flexibility to apply different operational MI definitions. Cardiac chest pain was defined as an episode of ischemic pain, tightness, pressure, or discomfort in the chest, arm, or jaw of any duration. Atypical pain that was determined to be due to coronary ischemia also qualified as cardiac chest pain. Chest pain was considered absent if the pain was due to a clear noncardiac cause. Cardiac enzyme concentrations were classified as normal, equivocal, or abnormal based on the most elevated value. Different cutpoints were used for cardiac enzyme levels in patients who had undergone a coronary artery bypass graft surgery or percutaneous transluminal coronary angioplasty within the prior 48 hours or who had muscle trauma. Reviewers also identified whether a patient had regional wall abnormalities identified on imaging (if available).

MI adjudication

Members of the CNICS MI adjudication committee are physicians participating in MESA and other CVD cohort studies and are thus experienced MI event reviewers. Training calls were conducted to ensure understanding of the protocol. All reviewers adjudicated the same initial group of events, and discrepancies between reviewers were discussed. After the initial training period, 2 reviewers adjudicated each event, followed by a third reviewer in cases of discrepancy. Each potential event was categorized by reviewers as definite, probable, no/absent, or no with resuscitated cardiac arrest. For potential events classified as non-MIs (no/absent), reviewers documented whether the patient had undergone a cardiac intervention, such as a coronary artery bypass graft surgery, percutaneous transluminal coronary angioplasty, or stent placement. Adapting and expanding prior MI protocols, reviewers categorized probable/definite MIs as primary spontaneous events or secondary events that occurred because of some other cause, such as sepsis, and selected a cause for each secondary MI from a standardized list. For some potential MI events, cardiac enzyme elevation was felt to be related to a cause other than MI; reviewers labeled such events as “false positives” and specified the suspected other cause.

Analysis

We examined the number and percentages of events that required third reviews, were adjudicated to be primary versus secondary events, or had false-positive enzyme elevation. We calculated κ statistics as a measure of agreement between 2 reviewers beyond chance alone (30). A priori, we defined κ < 0.4 as poor to fair agreement, 0.41–0.60 as moderate agreement, 0.61–0.80 as substantial agreement, and 0.81–1.0 as excellent agreement, as was done previously (15, 31). We calculated the positive predictive value, which is the proportion of MIs reported by each ascertainment method (clinical MI diagnosis, elevated cardiac enzymes) that were verified by adjudication, and the sensitivity, which is the proportion of true MIs verified by adjudication that were identified as positive by each ascertainment method (30). We combined probable and definite MIs for these analyses. We report exact binomial 95% confidence intervals.

Our primary analyses focused on persons with diagnoses or elevated cardiac enzymes that were suggestive of an MI. Additionally, to further evaluate our approach, a random sample of 100 patients from one site without diagnoses or elevated cardiac enzyme levels was also reviewed for MI events.

RESULTS

There were 1,119 potential MI events identified by an elevated enzyme level and/or clinical MI diagnosis adjudicated during the study period. Among all potential events, 75% were in men, the mean age of subjects was 47 (standard deviation, 9) years, and the current mean CD4 cell count (at the time of the potential MI event) was 298 (standard deviation, 265) cells/mm³ (Table 1).

Table 1.

Clinical and Demographic Characteristics of Patients Who Met Ascertainment Criteria for a Potential Myocardial Infarction Event by Event Adjudication Outcome (n = 1,119), Centers for AIDS Research Network of Integrated Clinical Systems, 1995–2012

Characteristic	No MIa (n = 825)		Probable MI (n = 92)		Definite MI (n = 202)		P Valueb
	No.	%	No.	%	No.	%
Sex							0.1
Male	634	77	62	67	157	78
Female	191	23	30	33	45	22
Age, years							0.02
<40	152	18	23	25	28	14
40–50	368	45	43	47	85	42
50–60	242	29	20	22	60	30
≥60	63	8	6	7	29	14
Race							0.6
White	270	33	31	34	79	39
African-American	485	59	56	61	109	54
Hispanic	48	6	3	3	11	5
Other	22	3	2	2	3	2
Risk factor for HIV transmission							<0.01
Being a MSM	205	25	28	30	67	33
Injection drug use	392	48	40	44	59	29
Being heterosexual	203	25	20	22	61	30
Other/unknown	18	2	4	4	15	7
CD4 cell count,c current cells/mm3							0.3
0–200	357	43	47	51	84	42
201–350	172	21	17	19	35	17
>350	295	36	28	30	83	41
CD4 cell count,c nadir cells/mm3							0.1
0–200	573	70	69	75	133	66
201–350	151	18	11	12	32	16
>350	100	12	12	13	37	18
HIV-1 RNA,d copies/mL							0.8
>100,000	191	23	24	26	43	21
10,000–100,000	163	20	21	23	39	20
<10,000	468	57	47	51	119	59
Body mass indexe							0.3
<18.5	80	10	11	12	25	13
18.5–24	411	50	53	58	92	46
25–30	215	26	16	18	51	25
>30	116	14	11	12	33	16

Abbreviations: HIV, human immunodeficiency virus; MI, myocardial infarction; MSM, man who has sex with men.

^a Falsely positive events were included in the no MI category.

^b P value for the difference between definite/probable and no MI.

^c CD4 values were available for 1,118 events.

^d HIV RNA levels were available for 1,115 events.

^e Weight in kilograms divided by height in meters squared. Measurements were available for 1,114 events. Height measurements were available for 957 events. Median height by sex and age group were imputed for the remaining 157 events.

Of 1,119 potential events, 202 (18%) were adjudicated as definite MIs, 92 (8%) were adjudicated as probable MIs, and another 9 (<1%) were adjudicated as resuscitated cardiac arrests using the MESA definition. Of the 294 definite/probable MIs, 153 (52%) were adjudicated to be primary and 141 (48%) were considered secondary. Sepsis (38 MIs, 34% of secondary MIs) and vasospasm related to use of cocaine or other illicit drugs (20 MIs, 14%) were the most common causes of secondary MIs. Other common causes (ranging from 5% to 10% of the secondary MIs) included hypotension, hypertensive urgency/emergency, arrhythmias, and gastrointestinal bleeding.

We examined the standardized review criteria entered by reviewers for the 294 definite/probable MIs. Chest pain was identified for 164 (56%), abnormal ECGs were present for 190 (65%), and elevated cardiac enzymes were identified for 287 (98%). Only 64 (22%) events had regional wall abnormalities identified on imaging.

Ascertainment criteria

Adjudicated events could have been identified by more than one ascertainment/identification criterion from the CNICS data repository. Of the 294 definite/probable MIs, 89% were identified by elevated cardiac enzymes, of which 78% had elevated troponin levels and 65% had elevated creatine kinase MB levels (some had both). Less than half (44%) of adjudicated definite/probable MIs were identified by a clinical MI diagnosis (Table 2). Among the 294 definite/probable MIs, 70% of primary MIs were identified by a clinical MI diagnosis, whereas only 23% of secondary MIs were. Elevated troponin values had the highest positive predictive value (57%) (Table 2). Sensitivity and positive predictive value did not differ meaningfully by age for persons with elevated troponin or creatine kinase MB values; however, among patients less than 50 years of age, the sensitivity and positive predictive value for clinical MI diagnoses (39% and 42%, respectively) were lower than those among patients 50 years of age or older (51% and 50%, respectively).

Table 2.

Ascertainment (Test) Criteria Used to Identify Events Categorized by Myocardial Infarction Adjudication Outcome (n = 1,119), Centers for AIDS Research Network of Integrated Clinical Systems, 1995–2012

Test Criteria	Reference Standard							Sensitivitya,b		Positive Predictive Valuea,b
	No Event		False Positive		MI (Probable or Definite)		P Value	%	95% CI	%	95% CI
	No.	%	No.	%	No.	%
MI diagnosis
No	549	79	121	95	166	56	<0.001	44	38, 49	45	39, 51
Yes	148	21	7	5	128	44
Elevated CK-MB value
No	196	28	82	64	102	35	<0.001	65	60, 71	26	23, 29
Yes	501	72	46	36	192	65
Elevated troponin value
No	639	92	15	12	65	22	<0.001	78	73, 83	57	52, 62
Yes	58	8	113	88	229	78
Any elevated cardiac enzyme (CK-MB or troponin)
No	158	23	0	0	31	11	<0.001	89	85, 93	28	25, 31
Yes	539	77	128	100	263	89

Abbreviations: CI, confidence interval; CK-MB, creatine kinase MB; MI, myocardial infarction.

^a A false positive was recoded as no event.

^b Estimates were assessed among persons who tested positive on at least 1 of 3 criteria. No patients identified as negative by the 3 criteria were included in the analysis; as such, estimates of specificity and negative predictive value are unavailable.

There were 128 events (11%) that met MI criteria but were identified clinically as occurring in persons who had an enzyme elevation because of a cause other than acute cardiac ischemia, such as renal failure (Table 3). As the result of the adjudication process, these events were categorized as falsely positive and not included as MIs.

Table 3.

Causes of Potential False Positive Results (n = 128), Centers for AIDS Research Network of Integrated Clinical Systems, 1995–2012

Cause	No.	%
Renal failure	75	59
Congestive heart failure	15	12
Severe sepsis/shock	12	9
Pulmonary embolism	6	5
Pericarditis	4	3
Myocarditis	1	1
All other causes	16	12

Discrepancies in some aspect of the review requiring a third review were present for 255 possible events (22%) (MI vs. no MI; definite vs. probable MI; primary vs. secondary MI; falsely positive vs. not). κ statistics between the first 2 reviewers were 78%–84%, indicating substantial to excellent agreement. Of 100 randomly selected patients without any ascertainment criteria suggesting MI, no events were detected (0%, 95% confidence interval: 0, 3.6).

DISCUSSION

We developed, implemented, and evaluated a MI adjudication protocol for HIV that was conducted by centralized expert reviewers who were blinded to the exposures of interest. Among the first 1,119 potential events, 294 were adjudicated as definite/probable MIs. Ascertainment criteria for 89% of definite/probable MIs included elevated cardiac enzymes, and 78% of patients with definite/probable MIs had elevated troponin values. Only 44% had a clinical MI diagnosis, demonstrating the insensitivity of clinical diagnoses and the importance of multiple ascertainment criteria. Furthermore, secondary MIs (48%) were almost as common as primary events (52%), highlighting the importance of identifying MI type. After reviewing packets that could include information from outside medical records, 98% of adjudicated MIs included elevated cardiac enzymes as MI criteria.

Definitions

These analyses used MESA MI case definitions. Case definitions vary across studies and continue to evolve, such as with the universal MI definition, which has led to the appearance of increasing MI incidence rates over time (29, 32–37). Although the impact is small, MESA and universal MI definitions use different thresholds to define cardiac enzyme elevations after a coronary artery bypass graft surgery. This has been a controversial area with limited data (38), suggesting a need to reevaluate current definitions (39). Reviewers enter MI criteria for each case into the adjudication platform, facilitating future analyses that use different criteria as MI definitions continue to evolve (33, 34).

Ascertainment and adjudication

Comprehensive clinical data in the CNICS repository facilitated more focused, thorough, and efficient MI ascertainment than was possible in previous studies (13, 15, 40) that may have had to obtain medical records from essentially every hospitalization. Consistent with prior studies, we found that using multiple criteria to identify potential events increased the number of events that were identified (41). However, we did not find that including related CVD diagnoses or procedures increased the number of cases, which was in contrast to prior studies (15, 42). The enhanced sensitivity of our approach may be due to the use of cardiac enzyme levels in addition to diagnoses.

Adjudication resulted in identifying false-positive events that would have been coded as an MI event in studies relying on nonadjudicated definitions, such as MI billing diagnosis codes (41, 43–45). One study found that diagnosis codes underestimated MIs identified through chart review by half (46). We found that only 44% of true events were identified by MI diagnosis codes, and these codes worked less well among younger patients. Secondary MIs in particular were frequently missed. In fact, MI diagnoses identified more non-MI events than events adjudicated to be definite/probable MIs. Elevated cardiac enzymes, particularly troponin levels, provided the highest level of accuracy but were still problematic, as 11% of potential events had falsely positive enzyme elevations. This highlights the need for adjudication and additional studies comparing event rates between adjudicated and nonadjudicated events.

Central adjudication is preferable to local event adjudication with or without secondary central review. Previous studies found that outcome rates vary depending on whether committees merely confirm events reported by local sites or adjudicate all potential events identified by systematic review of comprehensive clinical data as we did. The higher MI rates found in studies using central adjudication versus those relying on onsite clinicians/investigator reviews (47–50) have resulted in meaningful differences across studies (47, 50–52). MI misclassification has been attributed to reluctance by reviewers to apply all aspects of standard MI definitions (41) and to misreporting of MI endpoints (49, 52, 53). Central ascertainment and adjudication has been particularly important in heterogeneous geographic regions and settings in which there is risk of applying varying MI definitions despite established protocols (51).

Secondary MIs

It is important to distinguish primary spontaneous MI events from events that occur secondary to other clinical syndromes in the setting of ischemia due to increased oxygen demand or decreased supply, such as severe hypotension (54). Secondary MIs are categorized as type 2 MIs by the universal MI definition (with the exception of those that are secondary to certain procedures, which are type 4 or 5) (37) and have treatment implications distinct from those of primary MIs (55). We found high numbers of secondary MIs due to sepsis or vasospasm associated with cocaine use. Although few studies have examined type 2 MIs, we found that close to half of the events in a population of persons living with HIV were type 2 events, in contrast to a recent general population study which found that a quarter of events were type 2 (56). Identifying these events may be particularly important among persons living with HIV, in whom secondary MI rates may be higher than in other patient populations.

False-positive MIs

False-positive events were due to isolated enzyme elevations without other evidence for MI. Reasons identified included pericarditis, pulmonary embolism, heart failure, or renal failure, though reasons for elevations may differ for troponin-T versus troponin-I (57). Many studies cannot distinguish these events from MIs. Identifying these events may be particularly important in studies of persons living with HIV; this highlights the need for adjudication that incorporates clinical context rather than just using endpoint ascertainment.

Blinding

The CNICS MI protocol blinds reviewers to key covariates, such as which individual ARVs were prescribed to patients. Blinding addresses the possibility of differential adjudication based on the reviewer's belief that a particular ARV, such as abacavir, may be more likely to be associated with MI (41, 58, 59). The influence of a reviewer's knowledge of ARV exposure on adjudication is not known and may be of particular importance given the number of conflicting studies regarding ARVs, such as abacavir, and MIs (60–64). Blinding reviewers to individual ARVs requires additional time for redaction of primary records. Blinded central adjudication addresses differential misclassification only if the site clinicians/investigators are ascertaining and reporting information without bias (41). Therefore, after centralized event ascertainment, sites upload primary notes and reports centrally for distribution rather than synthesizing information onto event forms, which could result in bias because site clinicians are not blinded to ARVs (41).

Web-based adjudication system

The CNICS web-based system used to adjudicate MIs and other endpoints improves review efficiency by eliminating the need to mail forms. Electronic data entry facilitates automated checks for missing data and comparisons between reviews. This system allows skip patterns such that reviewers are only asked about adjudication criteria for persons found to have had an event and allows use of dropdown menus that include common reasons for false-positive events, thereby decreasing review time and improving data quality.

Limitations

A potential limitation is the lack of self-report for MIs, which could allow missed cases. However, one study found that of 276 self-reported CVD events, only 68% were verified by adjudication of medical records (65). Among self-reported MIs, 50% were confirmed, 29% were adjudicated as angina or other non-MI coronary event, and 21% were determined not to be a coronary event (65). Sensitivity for self-reported MI is low (66). Medical record data has traditionally been used as the gold standard for self-report validation (66, 67), and thus it is unclear if self-report would increase identification beyond the use of comprehensive EHR data. Symptomatic MIs tend to present as dramatic painful events that require hospitalization, and therefore information should be readily available in both medical record data and self-report. However, the potential for ascertainment of secondary MI events occurring in the setting of sepsis or other hospitalizations suggests that medical record data ascertainment may be more comprehensive in studies of persons living with HIV.

A limitation of our approach is that research costs continue to climb and MI adjudication consumes resources. Not all studies have demonstrated that endpoint adjudication has a clear impact on estimates (41), although there is little doubt that it increases accuracy (68). Although we did not conduct a formal cost estimate, the CNICS MI protocol is more time-intensive than using coded EHR data without adjudication. However, the number of related diagnosis codes makes endpoint ascertainment alone complicated and inaccurate without adjudication (12). The CNICS web-based adjudication platform greatly reduced administrative time and burden, thereby decreasing costs. The requirement for 2 reviewers for each event also contributes to cost. However, although some protocols have selected subsamples of charts for review (33), we found that double review of all charts was worth the investment because for approximately 1 of every 5 of events, there were discrepancies between reviewers.

We did not systematically ascertain ECGs to look for silent events. Using ECGs to look for past MIs has very low detection probability, even among patients with confirmed MIs (69). Researchers minimized the number of missed events by requesting records from outside hospitals when possible, but there may still have been events missed. The use of cardiac enzymes in clinical care and MI definitions has changed over time, leading to increasing numbers of events meeting broader MI definitions (55). To minimize the impact of changing clinical definitions, we ascertained potential events using cardiac enzymes and not just diagnoses.

Finally, even though this protocol validation includes data from multiple sites across the United States, the findings have not been replicated more broadly. Although we reported several measures of accuracy, we note that measures such as positive predictive value depend on the disease prevalence within the population (70). However, because we reviewed records for every patient with elevated enzymes, which is the key to modern MI definitions, it is likely that few nonsilent MIs were missed. Because we included only potential events, negative results were not included, and thus we did not calculate specificity and negative predictive value. However, no events were identified among 100 randomly sampled patients from one site who had neither clinical diagnoses nor elevated cardiac enzymes, which suggests that given the low prevalence of the outcome, the estimated specificity for each ascertainment criteria would be 97% or higher.

Strengths

Event adjudication conducted in the present study was based on the MESA protocol (71) and universal MI definition (29). As noted above, because reviewers identified relevant MI criteria for each event, we were able to apply each definition or other definitions to facilitate cross-study comparisons. Although ECGs and procedure notes must still be obtained for adjudication, the use of comprehensive EHR data, including inpatient and outpatient information, greatly decreases the likelihood of there being missing data. CNICS includes a large and ethnically diverse cohort of patients with the full range of HIV disease. CNICS addresses several limitations of prior cohort studies. It includes predominantly prospective data collection, which decreases recall bias and exposure misclassification, and uses standardized data collection, verification, and adjudication, which minimizes misclassification and missing data.

North American AIDS Cohort Collaboration on Research and Design

We have extended the MI protocol used in CNICS to multiple sites in the North American AIDS Cohort Collaboration on Research and Design (NA-ACCORD) cohort (72), the largest multicenter HIV cohort collaboration worldwide. Applying the same MI protocol and outcome definitions across CNICS and NA-ACCORD strengthens future collaborations that use combined data for analyses. Although there is some overlap between the 2 cohorts, each also has independent sites, which greatly enhanced the potential analytic power. In addition, data collected in the 2 cohorts are complementary. CNICS provides detailed patient-reported data such as the routine measurement of physical activity levels that is not available in other cohorts. NA-ACCORD provides the very large sample size required to answer key questions related to HIV and CVD that cannot be addressed in smaller cohorts.

Conclusions

The MI adjudication protocol described here includes systematic centralized ascertainment and comprehensive central adjudication. We demonstrated that adjudication impacts conclusions drawn about MIs in HIV cohort research. Clinical diagnoses alone are insufficiently sensitive for MI event ascertainment, as over half of events adjudicated to be probable/definite MIs were not identified by an MI diagnosis code. Secondary events were almost as common as primary events. These findings suggest that failure to differentiate primary MIs from secondary MIs could lead to misclassification and false conclusions. Cardiac enzyme elevations that were determined using comprehensive laboratory data led to better capture of events than did clinical diagnoses alone. Nevertheless, 11% of events were in patients with false-positive enzyme elevation. Blinding to key covariates, identifying discrepancies through the use of multiple centralized reviewers, and distinguishing primary from secondary MI events were key strengths of the approach. Ongoing careful event adjudication and comprehensive clinical data in CNICS will facilitate a nuanced and thorough investigation of the epidemiology of MIs among persons living with HIV across the United States.