Provider verification of electronic health record receipt and non-receipt of direct-acting antivirals for the treatment of hepatitis C virus infection

Abstract

Purpose.

Pharmacoepidemiologic studies using electronic health record (EHR) data could serve an important role in assessing safety and effectiveness of direct-acting antiviral (DAA) therapy for chronic hepatitis C virus (HCV) infection, but the validity of these data needs to be determined. We evaluated the accuracy of pharmacy fill records in the national Veterans Health Administration (VA) Corporate Data Warehouse (CDW) as compared to facility-level EHR.

Methods.

Patients prescribed a DAA regimen at five VA sites between 2014-2016 were randomly selected and reviewed. A random sample of patients with chronic HCV infection without evidence of HCV treatment during the study period also underwent chart review. We calculated positive predictive value (PPV) and negative predictive value (NPV) overall and by site.

Results.

Of the 501 patients who received a total of 2,416 prescriptions, 494 were validated using data extracted from CDW six months after the study period, yielding a PPV of 98.6% (95% confidence interval [CI], 97.6%-99.6%). Of the 100 patients with chronic HCV infection without prescriptions for HCV treatment, 99 were confirmed not to have received antiviral treatment (NPV, 99.0%; 95% CI, 97.1%-100%).

Conclusions.

These findings provide assurance to researchers who use national VA CDW data for retrospective cohort studies that the CDW contains accurate information on HCV therapies in the modern treatment era.

INTRODUCTION

Hepatitis C virus (HCV) infection is a major public health priority in the United States (US), with an estimated 2.7 million chronically infected residents [1]. New direct-acting antiviral (DAA) agents for the treatment of chronic HCV were approved by the US Food and Drug Administration in 2014 [2, 3]. Clinical trials demonstrated that >90% of patients achieve viral cure with typically no more than 12 weeks of treatment [2, 4]. However, clinical trials often underrepresent individuals with pre-existing conditions, older patients, and minorities, thus attenuating their generalizability to routinely treated populations. Observational data could serve an important role in pharmacoepidemiologic studies assessing the safety and effectiveness of modern therapies for chronic HCV infection [5].

The Veterans Health Administration (VA) is the largest provider of chronic HCV care in the US, with an HCV seroprevalence of 5.4% among Veterans, which is five times higher than that of the general US population [1,6]. In late 2013, VA providers began prescribing DAA medications in regimens to treat HCV infection. However, in August 2014, the Veterans Choice Program (Choice) was established and allowed eligible Veterans to receive health care, including HCV treatment, from a community provider rather than wait for a VA appointment or travel to a VA facility. Between November 2014 and January 2017, 1.5 million Veterans representing 17% of all patients in VA care received care through Choice [7]. Accordingly, non-VA providers may have written prescriptions for HCV treatment during this time period. While it is plausible prescriptions of DAAs made off-site were filled by a VA pharmacy due to their high cost, it remains unclear if these Choice prescriptions and pharmacy fills are documented in the national VA Corporate Data Warehouse (CDW), the primary source of data for researchers. Additionally, electronic data from each VA facility go through multiple extracts, transforms, and loads before reaching the CDW, all of which may compromise validity of the original data [8].

The purpose of this study was to compare DAA pharmacy fill records in CDW with facility-level electronic health record (EHR) data at five VA sites with high HCV prevalence. We also sought to characterize the frequency of Choice prescriptions and their impact on the accuracy of CDW data in this sample. We hypothesized that we would find overall high agreement between data sources.

METHODS

Design and data sources

We conducted a retrospective cohort study among chronic HCV-infected patients in the Veterans Aging Cohort Study (VACS) between 1 January 2014 and 30 June 2016 [9, 10]. VACS consists of all HIV-infected patients receiving care at VA medical facilities across the US, matched to two HIV-uninfected patients on age, sex, race/ethnicity, and site. Available data include diagnoses, laboratory results, and pharmacy fill data from CDW as well as access to facility-level EHR information including electronic pharmacy records, clinical progress notes, and Choice referrals. For this analysis, we compared CDW pharmacy fill data with facility-level EHR data at five VA sites with high HCV prevalence (i.e., Atlanta, GA; Bronx, NY; Houston, TX; Los Angeles (LA), CA; Washington, DC). This study was approved by Institutional Review Boards of Yale University and each participating VACS site.

Patients selected for validation

All chronic HCV-infected patients (determined by positive HCV RNA) who had pharmacy fill data for HCV treatment in the CDW at any of the five VA sites between 1 January 2014 and 30 June 2016 were eligible. We estimated that a target sample size of 500 patients would yield a minimal (≤4%) margin of error at any level of agreement >50% between CDW and facility EHR data. We included all patients treated for HCV infection at Atlanta (79) and LA (92) and randomly selected 110 patients from each remaining site. We also randomly selected 20 chronic HCV-infected patients at each site, without evidence of HCV treatment in CDW pharmacy data, to estimate frequency of HCV treatment not captured in CDW.

HCV treatment agents

The original search for pharmacy fill data of HCV treatment in CDW occurred on 7 July 2016, or seven days after the study period, and included the following DAA agents: boceprevir, telaprevir, simeprevir, sofosbuvir, daclatasvir, ledipasvir/sofosbuvir, ombitasvir/paritaprevir/ritonavir, ombitasvir/paritaprevir/ritonavir/dasabuvir, elbasvir/grazoprevir, sofosbuvir/velpatasvir (fixed-dose combinations indicated by / between medication names). We also evaluated for older HCV therapies, including interferon alfa-2a and -2b, pegylated interferon alfa-2a and -2b, and ribavirin. CDW data list each prescription of each agent or fixed-dose combination on a separate record. Therefore, an individual receiving a standard treatment of 12 weeks of ledipasvir/sofosbuvir, would have three records (one per 30-day fill) to validate.

Confirmation of HCV Treatment

For patients who received prescriptions for HCV treatment in CDW pharmacy fill data, we populated a validation database with name and date for each prescription found, including fill, release, return, and stop dates, and sent the encrypted database to each site. An experienced infectious disease (ID) physician at each facility was asked to confirm that each prescription was dispensed using the facility-level EHR.

For patients without evidence of HCV treatment in CDW pharmacy fill data, ID physicians were asked to determine whether these patients had any evidence of receiving HCV treatment in the facility or outside the VA system during the study period based on a review of the facility EHR. For patients who received treatment outside VA (e.g., through Choice or other), details on their treatment (i.e., antiviral regimen prescribed, start and stop dates) were recorded.

Statistical analyses

Demographic and clinical characteristics were compared across four distinct groups of patients: (1) treatment in CDW validated by EHR, (2) treatment in CDW not validated by EHR, (3) absence of treatment in CDW validated by EHR, and (4) absence of treatment in CDW not validated by EHR. For these comparisons, patients with treatment in CDW were deemed validated if all prescriptions were identified and not validated if at least one prescription was not identified.

The primary outcome for patients with HCV treatment recorded in CDW was a patient-level indicator that denoted whether or not all pharmacy fill records for each patient were noted in the facility EHR. The primary outcome for sampled chronic HCV-infected patients without evidence of HCV treatment in CDW was a patient-level indicator that denoted any evidence of HCV treatment noted in the facility EHR, regardless of source of treatment. We calculated positive predictive value (PPV; proportion of patients with prescriptions in CDW that were validated) and negative predictive value (NPV; proportion of patients without prescriptions in CDW that were validated) with 95% confidence intervals (CIs) overall and by site. Lastly, we repeated the data extraction from CDW six months after the initial data pull to account for CDW data that may have been subsequently updated or corrected, and we re-calculated PPV and NPV. All statistical analyses were performed using SAS version 9.4 (SAS Institute Inc., Cary, NC, USA).

RESULTS

We reviewed facility-level EHR data from the original data extraction for 501 patients who had a combined 2,416 prescriptions for HCV treatment between 1 January 2014 and 30 June 2016, as well as 100 patients without evidence of HCV treatment in CDW. The median age was 61 years (interquartile range [IQR] 58-64), 592 (98.5%) were male, 399 (66.4%) were black/African-American, 124 (20.6%) were white, and 275 (45.8%) were HIV co-infected (Table 1).

Table 1.

Characteristics of chronic HCV-infected patients with and without evidence of pharmacy fill data for HCV treatment in the national Veterans Health Administration (VA) corporate data warehouse (CDW)

	Treatment in CDW		Absence of treatment in CDW
Characteristic	Validated n=493	Not validated n=8	Validated n=97	Not validated n=3
Age, years	60.9 (58.0-64.3)	56.0 (53.4-59.2)	61.2 (57.6-65.0)	55.2 (51.7-59.7)
Male sex	488 (99.0)	8 (100.0)	95 (97.9)	1 (33.3)
Race/ethnicity
Black/African-American	327 (66.3)	3 (37.5)	67 (69.1)	2 (66.7)
White	103 (20.9)	5 (62.5)	15 (15.5)	1 (33.3)
Hispanic	51 (10.3)	0 (0)	10 (10.3)	0 (0)
Other/unknown	12 (2.4)	0 (0)	5 (5.2)	0 (0)
HIV co-infection	230 (46.7)	2 (25.0)	43 (44.3)	0 (0)

Abbreviations: HCV - hepatitis C virus; HIV - human immunodeficiency virus

Note: continuous measures reported as median (interquartile range), categorical measures reported as n (%)

Of 2,416 HCV prescriptions reviewed, 1,268 (52.1%) were ledipasvir/sofosbuvir, 399 (16.4%) were ribavirin, 294 (12.1%) were sofosbuvir, 245 (10.1%) were ombitasvir/paritaprevir/ritonavir/dasabuvir, 159 (6.5%) were simeprevir, and the remaining medications included elbasvir/grazoprevir, daclatasvir, pegylated interferon, sofosbuvir/velpatasvir, boceprevir, and telaprevir.

Overall, 493/501 patients (or 2,387/2,416 prescriptions) with evidence of treatment in CDW were validated yielding a PPV of 98.4% (95% confidence interval [CI] 97.3%-99.5%). Across the five sites, PPVs ranged from 96.7% to 100% (p=0.60) (Table 2). Twelve (2.4%) patients received HCV care from a non-VA provider through Choice, and all of their HCV prescriptions (n=34) were present in CDW.

Table 2.

Positive and negative predictive values of pharmacy fill data of HCV treatment in the national VA CDW

	Positive predictive value (PPV)	Negative predictive value (NPV)
Overall, % (95% CI)	98.4 (97.3, 99.5)	97.0 (93.7, 100)
By site
Atlanta	100 (100, 100)	100 (100, 100)
Bronx	99.1 (97.3, 100)	100 (100, 100)
DC	98.2 (95.7, 100)	95.0 (85.5, 100)
Houston	98.2 (95.7, 100)	90.0 (76.9, 100)
Los Angeles	96.7 (93.1, 100)	100 (100, 100)
	p=0.60	p=0.51

Abbreviations: HCV - hepatitis C virus; VA - Veterans Health Administration; CDW - Corporate Data Warehouse; CI - confidence interval; DC - Washington, D.C.

Note: Fisher’s Exact tests were used to test for heterogeneity across sites

Of 100 patients without evidence of HCV treatment in CDW, 97 (97.0%) were confirmed not to have been treated and 3 (3.0%) were found to have received HCV treatment (22 prescriptions, all within the VA) during the study period, resulting in an overall NPV of 97.0% (95% CI 93.7%-100%). NPV results were similar across sites (p=0.51).

After an updated data pull from CDW on 9 January 2017, or six months after the study period, one patient who was not originally validated among those treated in the CDW was found to agree with the facility-level report, and all but one of the 22 prescriptions were found for the three patients who originally had no treatment records in the CDW. These findings corresponded to an updated PPV of 98.6% (95% CI 97.6%-99.6%) and NPV of 99.0% (95% CI 97.1%-100%).

DISCUSSION

Our validation process suggests high levels of agreement between CDW and EHR data at five VA sites for 601 randomly selected VACS patients. The PPV among treated patients was 99% and the NPV among untreated patients was 97% from a data extraction one week after the study period. After repeated data extraction six months after the study period, PPV remained at 99% while NPV increased to 99%.

There was modest utilisation of Choice in our sample. All HCV prescriptions that were ordered by a non-VA provider through Choice were captured in CDW. Providers found evidence of HCV treatment for three patients who had no evidence of HCV treatment in CDW, but their HCV care was received within the VA system and not through Choice.

The sample used in this validation study had good representation of older patients, racial minorities, and those with HIV co-infection, which are underrepresented and often excluded populations in clinical trials of DAAs [11, 12].

Our study had potential limitations. First, we chose five large VA sites with high HCV prevalence. Other VA sites that utilize Choice more frequently may not be represented. Second, there was a potential for bias arising from the study’s design that had site physicians validate a pre-populated list of medications found in CDW. However, this concern was minimized by the sole use of electronic data that did not burden physicians to search non-electronic (i.e., paper charts), as well as the detailed notes frequently added to the validation database by the experienced physicians. Finally, these findings may not be generalised to other national EHR systems, such as Medicare or Kaiser Permanente. However, our methodology could be used to test validity of prescription fill data in other EHR systems.

Observational data could serve an important role in pharmacoepidemiologic studies assessing safety and effectiveness of modern therapies for chronic HCV infection. These findings provide assurance to researchers who use national VA CDW data for retrospective cohort studies that the CDW contains accurate information, particularly after a six-month lag for data extraction, on HCV therapies in the modern treatment era, even for VA patients who obtained treatment from non-VA providers. Future analyses should examine comparative effectiveness of DAA regimens, adverse effects of importance (e.g., acute liver injury, HBV reactivation), and the impact of unhealthy alcohol use on receipt of DAAs and subsequent treatment outcomes.

ABBREVIATIONS

CDW

Corporate Data Warehouse

Choice

Veterans Choice Program

DAA

direct-acting antiviral

DC

Washington, DC

EHR

electronic health record

HBV

hepatitis B virus

HCV

hepatitis C virus

ID

infectious disease

LA

Los Angeles

NPV

negative predictive value

PPV

positive predictive value

RNA

ribonucleic acid

US

United States

VA

Veterans Health Administration

VACS

Veterans Aging Cohort Study