Misclassification of Incident Hospitalized and Outpatient Heart Failure in Administrative Claims Data: The Atherosclerosis Risk in Communities (ARIC) Study

Abstract

Purpose

To quantify the influence of the length of the look-back period on misclassification of heart failure (HF) incidence in Medicare claims available for participants of a population-based cohort.

Methods

Atherosclerosis Risk in Communities (ARIC) participants with ≥3 years of continuous fee-for-service Medicare enrollment from 2000–2012 was assigned an index date 36 months after enrollment separating the time-in-observation into the look-back and the incidence periods. Incident HF events were identified using ICD-9-CM code algorithms as the first observed hospitalization claim or the second of two HF outpatient claims occurring within 12 months. Using 36 months as a referent, the look-back period was reduced by 6-months increments. For each look-back period, we calculated the incidence rate, percent of prevalent HF events misclassified as incident, and loss in sample size.

Results

We identified 9,568 ARIC participants at risk for HF. For hospitalized and outpatient HF, the number of events misclassified as incident increased and the total number of incident events decreased with increased length of the look-back period. The incident rate (per 1,000 person years) decreased with increased length of the look-back period from 6 to 36 months and had a greater impact on outpatient HF; for example, from 11.2 to 10.6 for ICD-9-CM 428.xx hospitalization in the primary position and 10.5 to 7.9 for outpatient HF.

Conclusion

Our estimates can be used to optimize trade-offs between the degree of misclassification and number of events in the estimation of incident HF from administrative claims data, as pertinent to different study questions.

Background

Administrative healthcare data, such as the Centers for Medicare and Medicaid Services (CMS) claims, are widely used in epidemiologic research. Claims data can be used to estimate incidence, prevalence, and associations of chronic and acute conditions^1–5. A limitation of claims data in this regard is the paucity of patients’ disease history and laboratory findings, patient-reported outcomes data, age restrictions, such as those encountered with the CMS Medicare data, and changes in enrollment status due to mandatory annual re-enrollment. Thus, identifying initial occurrences, i.e., true incident events, from administrative claims data can pose challenges.

Establishing incidence of heart failure (HF) is particularly challenging as this complex condition can be diagnosed in the inpatient as well as the outpatient setting, using a range of diagnostic criteria. Repeated outpatient visits and hospital admissions are common for individuals with chronic diseases, and notably in patients living with chronic HF, while the identification of incident events requires a look-back period sufficient to identify the incident nature of an event and to exclude prevalent cases. Consequently, in analyses of administrative claims data a time has to be set as the start of enrollment, or the at risk period of incidence, to differentiate incident events from prevalent events whose onset preceded the look-back period.

No standards for the optimal length of a look-back period in identifying incident HF from administrative claims data have been established. A one-year or shorter look-back period has been commonly used in research to date^6–10 although this conventional time frame may be insufficient to avoid misclassification of incident HF¹¹. Furthermore, the varying lengths of look-back periods used by investigators may contribute to conflicting estimates of incidence. To accurately classify incident HF, lengthening the look-back period may be necessary, but the impact of the length of the look-back period on the accuracy of the estimation of HF incidence has not been thoroughly examined.

The objective of this study was to characterize the impact of varying lengths of the look-back period on the observed incidence and misclassification of HF diagnosed in the inpatient and outpatient setting using Medicare administrative claims data linked with the Atherosclerosis Risk in Communities (ARIC) study.

Methods

Study Participants

The cohort component of the ARIC study is a community-based prospective study of cardiovascular disease among individuals sampled from 4 U.S. communities: Forsyth County, North Carolina; Jackson, Mississippi; Washington County, Maryland; and the northwest suburbs of Minneapolis, Minnesota¹². Recruitment occurred in 1987–89 and included 15,792 mostly Caucasian and African American participants aged 45–64 years at baseline. Institutional Review Boards at each participating institution approved the study and all participants provided written informed consent at each examination.

Data for ARIC cohort participants were linked with Medicare claims for the years 1991–2012 using a finder file that included participants’ social security numbers, gender, and date of birth. From the total number of study participants with available social security numbers (n=15,744), 238 died before 1991 and 607 died after 1991 but before reaching the Medicare eligibility age of 65 years, leaving 14,899 eligible ARIC study participants. A crosswalk file was used to identify ARIC cohort participants eligible for Medicare coverage. The crosswalk file between the ARIC study finder file and the Medicare Beneficiary Summary file yielded 14,702 ARIC cohort IDs for analyses (98.7% match).

Information concerning ARIC study participant enrollment in fee-for-service (FFS) Medicare was obtained from monthly indicators of enrollment in Part A, Part B, and Medicaid buy-in available from annual Medicare Beneficiary Summary files. Continuous enrollment periods were created to indicate uninterrupted FFS Medicare coverage, defined as enrollment in Medicare Part A and Part B as well as the lack of enrollment in a Medicare Advantage (HMO) plan. Study participants with continuous Medicare Advantage enrollment or less than 36 months of continuous Medicare FFS enrollment were excluded from the study. We included the last enrollment period extending ≥36 months for participants with gaps in coverage resulting from the discontinuation of enrollment FFS, enrollment in a Medicare Advantage plan, or from missing enrollment information. All inpatient and outpatient claims were linked.

Observation Period

Due to limited number of HF events from 1991–1999, we included participants with Medicare claims for the years 2000–2012 and followed participants from the date of FFS enrollment to FFS disenrollment, death, or December 31, 2012, whichever came first. Each participant was assigned an index date exactly 36 months after enrollment. The index date separated the time-in-observation into two phases: the look-back and incidence periods (Figure 1). The referent look-back period was defined as the first 36 months of FFS enrollment prior to the index date and used to identify prevalent cases of heart failure. The incidence period was defined as the time in FFS enrollment after the index date.

Figure 1.

Study participant’s hospitalized and outpatient heart failure classification scheme, ARIC cohort members enrolled in Medicare Fee-for-Service, 2000–2012

Two outpatient heart failure visits occurring within 12 consecutive months are considered a diagnosis of heart failure in the outpatient setting. An outpatient heart failure diagnosis can be considered prevalent if preceded by either an outpatient heart failure diagnosis or a heart failure hospitalization in the look-back period.

HF Events

HF identified from an ICD-9-CM code in the primary position has high specificity but poor sensitivity; thus, ICD-9-CM code 428.xx in the primary position probably underestimates numbers of true HF hospitalizations¹³. Indeed, hospitalizations with a non-primary ICD-9-CM code 428.xx are three times more common than those with a primary code of 428.xx^14,15 and codes other than 428.xx may also identify patients with HF^13,16. Therefore, hospitalized HF was identified from annual Medicare Provider Annual Review (MedPAR) records using three International Classification of Disease, 9^th Edition Clinical Modification (ICD-9-CM) code definitions for HF: ICD-9-CM code of 428.xx in the primary position; ICD-9-CM code of 428.xx in any position; and the Chronic Condition Data Warehouse (CCW) chronic condition algorithm for HF (ICD-9-CM codes: 398.91, 402.01, 402.11, 402.91, 404.01, 404.11, 404.91, 404.03, 404.13, 404.93, and 428.xx) in any position. An incident hospitalized HF was defined as the first observed hospitalization claim in the incidence period, following the index date (Figure 1).

Outpatient HF was identified from claims with Evaluation and Management service codes for new and established outpatient visits (99201–99205), consultations (99241–99245), and established preventive medicine visits (99395–99397) matched with date of service found in the Carrier (Part B) claims as well as claims for outpatient services provided through the Federally Qualified Healthcare Center (520–521). Similar to inpatient HF, outpatient HF events were identified using HF-specific ICD-9-CM codes 428.xx in any position in the medical record. A diagnosis of HF in the outpatient setting was defined as the presence of two outpatient HF claims within 12 consecutive months. The date of the first of those two HF outpatient claims was considered as the date of outpatient HF diagnosis (Figure 1)^1,2. A 30-day interval between the two outpatient encounters was used to ensure that the two outpatient encounters were separate events. An incident outpatient HF was defined as the first observed outpatient HF diagnosis in the incidence period following the index date.

Analysis

ARIC study participants were included in analyses if they had at least one MedPAR hospitalization record or at least one outpatient claim. Misclassification of HF incidence was estimated separately for hospitalized and outpatient HF. The referent look-back period was defined as 36 months following enrollment.

For identification of incident and prevalent hospitalized and outpatient HF, each participant was classified as having one of four HF incidence and prevalence conditions, based on the 36 month referent look-back period: (1) a HF event in the look-back and incidence periods, (2) a HF event in in the look-back period only, (3) a HF event in the incidence period only, or (4) neither (Figure 1). Similar classifications have been used in previous studies¹¹. Participants with a HF event in the incidence period only were considered as having “true incident HF events”. The incidence rate (IR, per 1,000 person-years) was calculated as the number of true HF events divided by the total person-time for all participants at risk. Participants were considered at risk for a heart failure event if they did not have a HF event in the defined look-back period. Participants’ person-time was measured from the index date to (1) an incident HF event, (2) death, (3) disenrollment, or (4) December 31, 2012, whichever came first. Participants with a HF event in both the look-back and incidence period and participants with a HF event in the look-back period only were excluded from the IR analyses.

The look-back period was progressively reduced in 6-month intervals, from the referent 36 months to a minimum 6 months before the index date. When the look-back period was reduced, membership in the “neither”, “look-back only”, “incidence only”, and “both” categories changed based on the dates of their HF events. The incident HF events observed among participants who were moved to the “incidence only” group in the process of reducing the look-back period were considered to be “false positives”. When assessing outpatient HF, a hospitalized HF or outpatient HF diagnosis in the look-back period reclassified a participant as “look-back only” or “both” from the “neither” or “incidence only” category, respectively. For each shortened look-back period, we recalculated the IR of HF events, calculated the percent of prevalent HF events misclassified as incident, percent of false positives, and the resulting loss in events. Participants with a HF event in both the look-back and incidence period or with a HF event in the look-back period only were excluded from all analyses. Misclassification of HF incidence was calculated as the proportion of the number of true incident HF events among the incident HF events in a given look-back period. False positives were calculated as 1-positive predicted value (the number of true incident HF events divided by the total number of reported incident HF events).

All analyses were conducted using SAS V9.3 (SAS Inc. Cary, North Carolina).

Results

Among the 14,702 ARIC cohort participants whose records were matched to Medicare claims, we identified 9,568 participants with a hospitalization or outpatient claim who were enrolled in FFS Medicare for at least 36 months from January 1, 2000 to December 31, 2012 and met aforementioned inclusion criteria. The average age of participants at the index date was 71 (SD=4.2) years (Table 1). The average length of the FFS Medicare enrollment for participants at risk was 106 months. Fifty-eight percent of participants were female and 27% were African American. There were 617 participants who had a HF hospitalization with an ICD-9-CM code 428.xx in primary position after the index date. We also observed 1,290 and 1,366 participants who had a HF hospitalization with ICD-9-CM code 428.xx in any position or an ICD-9-CM code corresponding to the HF CCW algorithm occurring after the index date respectively. Claims for HF diagnoses in the outpatient setting occurring after the index date were observed among 729 participants. Within the first 12 months following the index date and a non-event outpatient claim, 261 (2.7%) and 138 (1.4%) participants died, respectively.

Table 1.

Participant characteristics, ARIC cohort members enrolled in Medicare Fee-for-Service, 2000–2012

Characteristic	CMS enrolled ARIC participants (N = 9568)
	Mean (SD)
Age at index date (years)	71.1 (4.2)
Length in enrollment (months)	105.5 (100.2)
	N (%)
% Female	5558 (58.1)
% African American	2564 (26.8)
Study region
Forsyth County, North Carolina	1989 (20.8)
Jackson, Mississippi	2315 (24.2)
Minneapolis, Minnesota	2160 (22.6)
Washington County, Maryland	3104 (32.4)
Hospitalized events
ICD-9-CM code of 428.xx in the primary position	617 (6.4)
ICD-9-CM code of 428.xx in any position	1290 (13.5)
CCW ICD-9-CM code algorithm for heart failure	1366 (14.3)
Outpatient events
ICD-9-CM code of 428.xx	766 (8.0)

Abbreviations: Atherosclerosis Risk in Communities (ARIC), Centers for Medicare and Medicaid Services (CMS), standard deviation (SD)

Hospitalized HF (428.xx in the primary position)

Using the referent look-back period of 36 months, we observed 577 HF events classified as incident hospitalized HF (IR = 10.6 per 1,000 person-years, Table 2). Under this referent condition, 414 participants were classified as having an incident event (Figure 2). As the length of the look-back period was progressively shortened the number of HF hospitalizations classified as prevalent decreased and the numbers of HF hospitalizations misclassified as incident increased with a subsequent increase of classified HF events (Figure 2). For example, when the look-back period was shortened to 24 months, 26 (4.2%) of the putatively incident events were excluded as prevalent HF cases. The misclassification of 14 prevalent hospitalized HF events as incident (2.4% false positives) resulted in an overestimation of the IR by 2.4% from 10.6 to 10.9 events per 1,000 person-years. Further shortening the look-back period to 12 months led to the exclusion of 9 (1.5%) participants as having prevalent HF, thus resulting in 31 (5.1%) false positive incident HF hospitalizations and an overestimation of the IR by 5.3% (IR = 11.2 per 1,000 person-years).

Table 2.

Incidence of hospitalized and outpatient heart failure by length of look-back period, ARIC cohort members enrolled in Medicare Fee-for-Service, 2000–2012 (N = 9568)

Look-back period (monthsb)	Hospitalized heart failure				Outpatient heart failure
	428.xx in the primary position		428.xx in any position		Observed incident outpatient HF	Incidence ratec
	Observed incident hospitalized HF	Incidence ratec	Observed incident hospitalized HF	Incidence ratec
6	609	11.20	1255	24.15	549	10.46
12	608	11.18	1223	23.54	507	9.67
18	598	11.00	1190	22.91	473	9.03
24	591	10.87	1161	22.37	448	8.56
30	583	10.72	1139	21.95	431	8.24
36a	577	10.61	1112	21.44	414	7.92

^aReferent

^bMonths available prior to index date

^cPer 1,000 person years

Figure 2.

Proportion of false positives and number of events for hospitalized and outpatient heart failure, ARIC cohort members enrolled in Medicare Fee-for-Service, 2000–2012

The look-back period is used to exclude patients with prevalent heart failure prior to the index date. Proportion of false positives was calculated as the proportion of incident HF events in a given look-back period relative to the number of true incident HF events.

Hospitalized HF (428.xx in any position)

Using the referent look-back period of 36 months we observed 1,112 incident hospitalized HF events with the ICD-9-CM code of 428.xx in any position (IR = 21.4 per 1,000 person-years, Table 2), leading to the classification of 178 (13.8%) participants as having prevalent HF (Figure 2). As the length of the look-back period was shortened, the number of hospitalized HF events misclassified as incident increased (Figure 2), similar to what was observed using the more restrictive definition of HF based on the presence of ICD-9-CM code 428.xx in the primary position. Shortening the look-back period to 24 months resulted in the exclusion of 129 (10.0%) participants with prevalent hospitalized HF (7.8% false positives). Thus, incidence was overestimated by 4.5% from 21.4 to 22.4 events per 1,000 person-years, about double the overestimation observed when using a HF code in the primary position. When the look-back period was further shortened to 12 months, 67 (5.2%) participants were excluded as prevalent cases and the incidence was overestimated by 8.9% (IR = 23.5 events per 1,000 person-years). This resulted in 111 (9.1%) false positive incident hospitalized HF events.

Results were similar when assessing the CCW chronic condition algorithm for HF (Supplementary Table and Figure).

Outpatient HF

Using the referent look-back period of 36 months we observed 414 incident outpatient HF events (7.92 events per 1,000 person-years, Table 2). Similar to estimates obtained for hospitalized HF, the incidence of outpatient HF increased as the look-back period length was reduced (Figure 2) although the increase in the number of events was more substantial: under the referent condition, 352 (46%) participants were categorized as having prevalent HF (Figure 2). Shortening of the look-back period to 24 months resulted in the exclusion from analyses of 318 (42%) participants with prevalent outpatient HF. Incidence was overestimated by 7.5% from 7.92 to 8.56 events per 1,000 person-years and 34 prevalent outpatient HF events were misclassified as incident. When the look-back period was shortened to 12 months, 259 (33.8%) participants were excluded to account for prevalent outpatient HF, and the overestimation of incidence increased by 18.1% (9.67 events per 1,000 person-years). This resulted in 135 false positive incident outpatient HF events.

Discussion

Evaluations of the impact of the length of the look-back period employed in the estimation of incident events from administrative claims are uncommon, and none specific to HF have been reported to our knowledge. Consistent with a previous report⁷, our results suggest that lengthening a look-back period to exclude pre-existing HF from the risk set when using Medicare claims decreases the misclassification of prevalent HF events as incident. The use of short look-back periods thus tends to overestimate the incidence of HF by incorporating prevalent events; however, the impact is low for hospitalized HF.

By examining the effect of the length of look-back periods on misclassification of hospitalized HF as well as outpatient HF we contribute new information on misclassification of HF in administrative claims. The misclassification of outpatient HF was approximately double that of the misclassification observed for hospitalized HF using a more specific definition, and five times compared to observed misclassification of hospitalized HF defined by an ICD-9-CM code of 428.xx in any position. The greater impact of the look-back period on misclassification of outpatient, as compared to hospitalized HF, likely stems from the definition of an outpatient diagnosis requiring two HF claims within 12 consecutive months. This differential misclassification suggests that there is value in separately addressing the incidence of outpatient and hospitalized HF.

We compared different conventions commonly used for identification of HF events in claims data, i.e. the use of a HF code (428.xx) in the primary position versus more inclusive definitions such as 428.xx any position and the CCW chronic condition algorithm for HF. Our data suggest that if misclassification is of concern, the use of a more specific code algorithm (e.g. 428.xx in the primary position) may be desirable to reduce the number of false positives.

While misclassification is reduced as the length of the look-back period increases, the number of events is also reduced, particularly in the estimation of the incidence of outpatient HF. Misclassification bias is thus reduced when the length of the look-back period is increased although the resulting loss in the number of events likely affects precision. In general, a look-back period that is not congruent with the natural history of the event under consideration, or the risk characteristic of the population under study, may select observations that are not representative of the population of cases from which the events arise. As our results suggest, a choice of the desirable length of a look-back period would be informed by a trade-off between the costs assigned to misclassification bias vs. the loss of events, which would be influenced by the study question being addressed. If a short look-back period is necessary and the loss in study size is substantial the source of administrative claims used may not be a suitable resource.

Detailed medical record information is not available in administrative claims, nor is anamnestic information that would allow for the assessment of prior events or a time course for the event recorded as a claim. The inability to validate events as true incident events requires that the investigator establish a look-back period to identify and exclude prevalent events. In the absence of established standards for a look-back period we examined the impact of a practical range of lengths of look-back periods on misclassification of HF events as incident and the association reduction in the number of events. For comparability to previous evaluations of look-back periods, we defined as “true” incident HF events those preceded by a 36-month look-back period. While our definition of “true” incident HF events is not calibrated to a gold standard, the average time between HF hospitalizations and outpatient visits in this population was observed to be <1 year and <6 months respectively.

The trade-off between misclassification and the impact on precision of a reduced number of events can be an important consideration when assessing the merit of various look-back periods. Thus, we present information that can assist researchers in gauging the divergent effects of length of the look-back period by comparing misclassification and precision, as best suited to a study question. We also infer from our results that the short look-back periods commonly used in the analysis of administrative claims data may overestimate HF incidence.

Conclusion

Estimating hospitalized or outpatient HF using Medicare claims requires the use of a look-back period predating a HF event to exclude prior events and thus avoid misclassification bias. The length of the look-back period can substantially influence misclassification of prevalent hospitalized and outpatient HF as incident, as well as the size of the population available for study. Our results, coupled with other evaluations¹¹ suggest that published reports of the incidence of HF derived from administrative claims may be overestimated as a result of the short lengths of prevalent periods used. An optimal trade-off between protection against misclassification bias and loss of events in the estimation of incident HF from administrative claims can be informed by the results presented in this report, and adjusted to different study questions as appropriate.

Supplementary Material

Supp info

Supplementary Figure. Proportion of false positives and number of events for hospitalized heart failure (CCW), ARIC cohort members enrolled in Medicare Fee-for-Service, 2000–2012

Footnote: The Chronic Conditions Data Warehouse (CCW) chronic condition algorithm for heart failure includes International Classification of Disease, 9^th Edition Clinical Modification codes 398.91, 402.01, 402.11, 402.91, 404.01, 404.11, 404.91, 404.03, 404.13, 404.93, 428.0, 428.1, 428.20, 428.21, 428.22, 428.23, 428.30, 428.31, 428.32, 428.33, 428.40, 428.41, 428.42, 428.43, 428.9 in any position. The look-back period is used to exclude patients with prevalent heart failure prior to the index date. Proportion of false positives was calculated as the proportion of incident HF events in a given look-back period relative to the number of true incident HF events.

Supplementary Table. Incidence of hospitalized heart failure (CCW) by length of look-back period, ARIC cohort members enrolled in Medicare Fee-for-Service, 2000–2012 (N = 9568)

Key Points.

• Estimating incidence of HF using administrative claims requires the use of a look-back period predating a HF event to exclude prevalent events

• Implementation of a look-back period will decrease the amount of bias created by misclassification

• The length of the look-back period influences misclassification of prevalent HF events as incident

• An optimal trade-off between protection against misclassification bias and loss of events can be based on information presented in this report and adjusted to the study question as appropriate