Effect of health information exchange on recognition of medication discrepancies is interrupted when data charges are introduced: results of a cluster-randomized controlled trial

Abstract

Objectives

To determine the effect of health information exchange (HIE) on medication prescribing for hospital inpatients in a cluster-randomized controlled trial, and to examine the prescribing effect of availability of information from a large pharmacy insurance plan in a natural experiment.

Methods

Patients admitted to an urban hospital received structured medication reconciliation by an intervention pharmacist with (intervention) or without (control) access to a regional HIE. The HIE contained prescribing information from the largest hospitals and pharmacy insurance plan in the region for the first 10 months of the study, but only from the hospitals for the last 21 months, when data charges were imposed by the insurance plan. The primary endpoint was discrepancies between preadmission and inpatient medication regimens, and secondary endpoints included adverse drug events (ADEs) and proportions of rectified discrepancies.

Results

Overall, 186 and 195 patients were assigned to intervention and control, respectively. Patients were 60 years old on average and took a mean of 7 medications before admission. There was no difference between intervention and control in number of risk-weighted discrepancies (6.4 vs 5.8, P = .452), discrepancy-associated ADEs (0.102 vs 0.092 per admission, P = .964), or rectification of discrepancies (0.026 vs 0.036 per opportunity, P = .539). However, patients who received medication reconciliation with pharmacy insurance data available had more risk-weighted medication discrepancies identified than those who received usual care (8.0 vs 5.9, P = .038).

Discussion and Conclusion

HIE may improve outcomes of medication reconciliation. Charging for access to medication information interrupts this effect. Efforts are needed to understand and increase prescribers’ rectification of medication discrepancies.

BACKGROUND AND SIGNIFICANCE

Unintentional medication discrepancies are common at the time of hospital admission and discharge,^1,2 and they can cause adverse events that lengthen hospital stays³ and cause readmission.⁴ Studies have shown that structured medication reconciliation at the time of hospital admission and discharge helps correct medication discrepancies and prevent adverse drug events (ADEs).^5–9 Although medication reconciliation is a safety standard of the Joint Commission¹⁰ and the World Health Organization,¹¹ many organizations have difficulty implementing it.^12,13 Barriers include health-record systems that impede high-quality medication reconciliation, lack of provider and management support, patients’ incomplete knowledge or understanding of their medications, and competing higher-priority tasks.^14–16

A key step in medication reconciliation is information-gathering from sources that include patients, family members, providers’ offices, health care facilities, pharmacies, and prescription coverage plans. If each of these sources is accessed or contacted separately, the medication reconciliation process can be prohibitively time-consuming.¹⁷ Recently, regional health information exchanges (HIEs) have been established that consolidate information from multiple sources, including facilities, providers, and/or insurance plans, and make this information available to credentialed providers.¹⁸ HIEs could improve medication safety by facilitating reconciliation of medication information from multiple sources at once at the time of a patient encounter.

OBJECTIVE

The objective of this cluster-randomized trial was to determine the effect of real-time HIE on medication reconciliation in hospitalized patients at a US Department of Veterans Affairs (VA) hospital that is an early adopter of HIE. The VA system has a nationwide information system, but care information from outside the VA is not easily available to VA providers. This is important, because among patients treated at a VA facility who have other insurance (eg, Medicare), 53–80% utilize non-VA services,^19,20 and in our prior study, non-VA service use was a risk factor for medication discrepancy adverse events.⁵ We hypothesized that HIE would raise the impact of medication reconciliation for hospitalized veterans who utilize VA and non-VA services on discrepancies between preadmission and inpatient medication regimens (primary outcome) and reduction of ADEs (secondary outcome). This study also features an unplanned natural experiment: prescribing information was available in the HIE both from providers and from a pharmacy insurance plan for the first 10 months of the study, but only from providers for the last 21 months, when data charges were imposed by the insurance plan.

MATERIALS AND METHODS

Study design and participants

Patients admitted to 1 of the 4 inpatient units at the James J Peters VA Medical Center (JJP VA) between January 25, 2012, and August 25, 2014, were screened for study enrollment. Patients were eligible if they used non-VA health care services in the last 2 years, as indicated by an identity match in the Bronx Regional Health Information Organization (RHIO) system, a regional HIE. Identity matching in the HIE was based on name and birth date. Patients were excluded if they were admitted to an intensive care unit, were transferred to a study unit from a non-study unit, or did not remain in the hospital at least 24 h. The Bronx RHIO HIE receives information from the largest hospitals in the region (Montefiore Health System, Saint Barnabas Hospital Health System, and Bronx-Lebanon Hospital Center), as well as ambulatory care centers, individual physician offices, and long-term care, home care, community, and other organizations. Collectively, these providers deliver the majority of health care received by the Bronx’s 1.4 million residents, including over 95% of annual hospital discharges. For the first 10 months of the study, the Bronx RHIO also contained comprehensive prescription information from the largest pharmacy insurance plan (Surescripts), but for the remaining 21 months of the 31-month enrollment period, this information feed was stopped.

Ethics approval

The Institutional Review Board of the JJP VA approved study activities (project no. BOO-10‐076). All participants provided written informed consent before taking part.

Group assignment

Patients were assigned to intervention or control according to the unit to which they were admitted. At study start, 2 units were randomly assigned to intervention and 2 to control. Subsequently, units crossed over between intervention and control every 3 months, such that 2 of the 4 units were always intervention units and 2 were control units. Crossover dates were chosen to coincide with house staff rotation changeovers, to reduce the likelihood that a house staff provider had patients in both intervention and control groups and to minimize contamination. Admitted patients were recruited on business days by a research assistant who was blinded to study hypotheses and group assignment. Patients admitted on non-business days were recruited the next business day.

Patients who were hospitalized more than once during the enrollment period were eligible to be re-enrolled 30 days after hospital discharge. We allowed patients to re-enter the study, because medication prescribing decisions during hospitalizations more than 30 days apart are reasonably independent of each other and permitting re-entry of patients maximized sample size efficiently.

HIE-enhanced medication reconciliation

For patients assigned to the intervention group, an intervention pharmacist conducted HIE-enhanced medication reconciliation, following a structured protocol (Supplementary Appendix 1). In brief, the intervention pharmacist reviewed the patient’s medication information from the VA record and the Bronx RHIO HIE and interviewed the patient and/or caregiver upon hospital admission. The intervention pharmacist generated a best possible preadmission medication list and asked the patient and/or caregiver to clarify or explain differences between VA and outside data sources. The intervention pharmacist then identified discrepancies between the medications the patient was receiving as a hospital inpatient and the preadmission medication list. The intervention pharmacist recorded discrepancies in a progress note in the VA electronic health record and alerted the house staff immediately of any potentially important unintended discrepancies (eg, unexplained omission of warfarin). For non-urgent unexplained discrepancies (eg, a small difference in dosage of an antihypertensive medication), the house staff were not alerted in real time, but they were invited to read and cosign the pharmacist’s medication reconciliation note. The intervention pharmacist, who was the same individual throughout the study, was a VA staff pharmacist who saw study patients and completed study medication reconciliation procedures (<5 h/week) outside of his regular work duties and hours.

Usual care

For patients assigned to usual care, the intervention pharmacist performed the structured medication reconciliation protocol but without access to the Bronx RHIO HIE. To preserve blinding of the house staff and the outcomes assessors, the intervention pharmacist did not indicate in his medication reconciliation note whether he had accessed the Bronx RHIO HIE.

Measures

Patient characteristics

Patient age, gender, race, and chronic conditions, preadmission number of medications, illness severity at the time of index hospital admission,²¹ time of day of index admission, and responsible house staff specialty (medicine, surgery, or psychiatry) were abstracted from the VA electronic health record.

Processes

In the intervention and control groups, time from hospital admission to medication reconciliation was calculated and house staff orders that rectified or addressed discrepancies in the medication reconciliation note were identified. In the intervention group, we recorded how often medication information was present in the Bronx RHIO HIE and how often it led to identification of discrepancies that would otherwise have been undetected.

Outcomes

Medication discrepancies (primary outcome). Medication discrepancies were defined as differences between a patient’s prehospital medication list and medications received in the hospital, and were initially identified and recorded by the unblinded intervention pharmacist at the time of admission medication reconciliation (see HIE-enhanced medication reconciliation, above). All discrepancies were then reassessed and confirmed by blinded outcomes assessors by retrospective chart review at least 30 days after hospital discharge. Outcomes assessors were 5 research pharmacists who were separate from the intervention pharmacist and were blinded to group assignment. Outcomes assessors used a detailed study manual and completed at least 10 training cases in which their responses could be compared to reference responses before completing any study cases. Quality control over the course of the study, including data abstraction and ratings consistency, was maintained by weekly review and discussion of cases with the principal investigator. Disagreements were resolved by consensus. The assessors examined prescribing records in the hospital electronic health record and in the Bronx RHIO HIE to create a reference prehospital medication list, and then examined the inpatient medication administration record to create a comparison medication list. Differences were considered discrepancies if the comparison list omitted medications on the reference list; if there were switches to new medications for the same diagnostic indication; or if there were differences in dosage, route, or scheduling (eg, scheduled vs as-needed) of the same medication. Medications added to the comparison list for new diagnoses were not considered discrepancies.

For the primary outcome, assessors rated each medication discrepancy on a 4-point scale reflecting potential to cause harm to the patient (none, small, moderate, or great), using the study manual. Assessors were asked to consider the likelihood that harm might occur and the severity of the potential harm. Short-term (ie, during the hospital stay)and long-term (≥1 month after hospital discharge) risk were both rated. The ratings were summed for an overall risk-weighted medication discrepancy score for that admission. Interrater reliability of this measure is good (kappa 0.57–0.74), and each additional point confers a 40% increased likelihood of ADE (odds ratio [OR] = 1.40; 95% confidence interval [CI], 1.1–1.9).²² In addition, counts of total medication discrepancies and medication discrepancies in high-risk drug classes (opioid analgesics, nonsteroidal anti-inflammatory drugs, digoxin, insulin, antipsychotics, sedatives/hypnotics, and anticoagulants^23–25) were calculated for each admission.

Adverse drug events (secondary outcome). Outcomes assessors examined records in the electronic health record and the Bronx RHIO HIE during the hospital stay through 30 days after hospital discharge for adverse events that satisfied preset definitions (eg, new systolic blood pressure < 80). They rated whether an event was caused by an admission medication discrepancy using structured implicit review.^26–28 Causal criteria were adapted from Naranjo et al.²⁹ and included: (1) timing of the medication discrepancy and the event, (2) competing causes of the event, and (3) whether the patient’s condition improved after resolution of the medication discrepancy. The assessor assigned a score indicating the overall certainty that the event was caused by the medication discrepancy (unlikely, possible, probable, or definite),³⁰ and ADEs were defined as those with causal ratings of probable or definite.

Medication appropriateness index (secondary outcome). Outcomes assessors ascertained potential drug-drug interactions and inappropriate duplications using items from the Medication Appropriateness Index (MAI).³¹ The MAI is a reliable 10-item instrument that assesses appropriateness of drug indication, effectiveness, dosage, directions, practicality, interactions, duplication, duration, and cost, with a 3-level Likert scale response. For this study, only the items for drug-drug interactions and inappropriate duplications were used.

Analyses

The unit of observation was hospitalization episode. With 144 observations in each group, the study had 80% power to detect an effect size of 0.33 in the primary outcome with an α of 0.05. This effect size is equal to a risk-weighted medication discrepancy score (primary outcome) difference of 2, with a standard deviation of 6. A difference of 2 points is equal to 1 additional medication discrepancy with a “moderate” risk of harm.

Descriptive statistics were used to describe patient and hospitalization characteristics, time from hospital admission to medication reconciliation, and house staff rectification of medication discrepancies, by study group. In the intervention group, descriptive statistics were used to characterize the frequency that medication information was present in the Bronx RHIO HIE and whether that information led to identification of discrepancies that would otherwise have been undetected.

To assess intervention efficacy, we used multivariable regression, with risk-weighted medication discrepancy score as the dependent variable, group assignment as the independent variable, and age, number of chronic conditions, responsible house staff specialty, number of preadmission medications, admission illness severity, and hospital length of stay as covariates. We used generalized linear models (SAS Inc., Cary, NC, USA) and generated robust variance estimates to account for within-provider correlations, since some providers had more than 1 patient. Similar models were estimated for secondary outcome measures (eg, MAI). Multivariable logistic regression was used for the outcome of ADE (yes/no). Main models were intent-to-treat models. To examine the effect of the stoppage of prescription information from the pharmacy insurance plan partway through the study, an as-treated analysis was conducted, in which we stratified patients according to whether pharmacy insurance plan information was available at the time of the medication reconciliation.

RESULTS

Patients

After 6 patients withdrew, 311 patients with 381 hospital admissions (186 and 195 admitted to intervention and control units, respectively) were included (Figure 1 ). Patients were 96% male, 60 years old on average, 48% black, 31% white, and took a mean of 7 medications before admission. The median hospital length of stay was 6 days. Overall, 60% of patients were cared for by the medical service, 32% by the psychiatry service, and 8% by the surgical service. There were no significant differences between intervention and control groups in baseline characteristics (Table 1).

Figure 1.

CONSORT diagram

Table 1.

Characteristics of hospitalized veterans by study arm

Measure	Intervention	Control	P-value
Patients, Na	150	161
Age, mean (s.d.), years	59.7 (14.4)	60.8 (14.5)	0.502
Male (%)	96.7	95.7	0.643
Race (%)
White	32.0	29.8	0.866
Black	62.7	63.4	–
Other	5.3	6.8	–
Ethnicity (%)
Hispanic	23.3	26.1	0.585
Chronic conditions, no. mean (s.d.)	2.47 (1.77)	2.56 (1.74)	0.667
Hospitalizations, Na	186	195
Preadmission medications, no. mean (s.d.)	7.4 (4.3)	7.3 (4.2)	0.920
Admission illness severity, score, mean (s.d.)	1.85 (2.69)	2.05 (2.58)	0.467
Admission after 6 p.m. (%)	58.6	50.3	0.102
Admission service specialty
General medicine (%)	55.4	65.1	0.140
Psychiatry (%)	36.6	27.7	–
Surgery (%)	8.0	7.2	–
Hospital length of stay, median (IQR), days	6 (3–11)	5 (3–9)	0.805
Days from hospital admission to medication reconciliation, mean (s.d.)	2.9 (2.2)	2.9 (2.7)	0.923

^aPatients who were hospitalized more than once during the study period were eligible to be re-enrolled.

Medication reconciliation process and physician alerting

The mean time from hospital admission to medication reconciliation in both intervention and control groups was 2.9 (±2.4) days. For 40 study patients (10.4%), the intervention pharmacist immediately alerted a physician verbally of a potentially important unintended discrepancy. For 36 (9.4%), the pharmacist alerted a physician by inviting a cosignature to the medication reconciliation note. For 9 (2.4%), the pharmacist did both. There were no differences between intervention and control in numbers of verbal or cosignature alerts that the intervention pharmacist provided to physicians. Of 36 notes invited for cosignature, 23 (64%) were cosigned. Verbal alerting was associated with physicians being more likely to rectify a discrepancy than invite a cosignature alone: there were 10 rectifications (0.25 per patient) associated with verbal alerts and 1 rectification (0.04 per patient) associated with inviting a cosignature alone (P = .008 for comparison between groups). There were also 3 rectifications (0.01 per patient) when the pharmacist did neither. Overall, non-VA information was present in the Bronx RHIO HIE outpatient and inpatient medication sections for 22% and 23% of intervention patients, respectively. However, when a pharmacy benefits data feed was available to the HIE, outpatient medication information was present for 29% of patients, vs 19% when a pharmacy benefits data feed was not available to the HIE (P = .172 for comparison between time periods).

Effect of HIE on medication discrepancies and their rectification

In intent-to-treat analyses adjusted for covariates and accounting for provider clustering, there was no difference in risk-weighted medication discrepancies (beta coefficient = 0.60; 95% CI, −0.27, 1.5; P = .175), total medication discrepancies (0.14; −0.23, 0.50; P = .452), medication discrepancies in high-risk drug classes (0.01; −0.16, 0.19; P = .895), or MAI (0.26; −021, 0.74; P = .280) between intervention and control (Table 2). There were 6 house-staff orders that rectified medication discrepancies in intervention patients (among 231 medication discrepancies recorded by the intervention pharmacist = 0.026 rectified orders per opportunity) and 8 that rectified medication discrepancies in control patients (among 224 medication discrepancies recorded by the intervention pharmacist = 0.036 rectified orders per opportunity) (P = .539).

Table 2.

HIE effect on medication discrepancies and appropriateness: intent-to-treat analysis

Measure	Intervention	Control	Beta coefficient (95% CI)a	P-value
N	186	195
Medication discrepancy outcomes, mean (s.d.)
Discrepancies	3.2 (2.6)	3.0 (2.4)	0.14 (−0.23, 0.50)	0.452
Risk-weighted discrepancies (primary outcome)	6.4 (5.9)	5.8 (5.0)	0.60 (−0.27, 1.5)	0.175
Discrepancies in high-risk drug classes	0.75 (.95)	0.70 (0.95)	0.01 (−0.16, 0.19)	0.895
Medication appropriateness, mean (s.d.)	2.5 (2.9)	2.1 (2.8)	0.26 (−0.21, 0.74)	0.280

^aAdjusted for age, number of chronic conditions, house staff service specialty, number of preadmission medications, admission illness severity, and hospital length of stay.

Effect of HIE with and without pharmacy insurance data

Fifty-one patients (28%) received HIE-enhanced medication reconciliation with pharmacy insurance data available and 131 (72%) without (Figure 1). In as-treated analyses adjusted for covariates and accounting for provider clustering, patients who received HIE-enhanced medication reconciliation with pharmacy insurance data available had greater risk-weighted medication discrepancies identified than those who received usual care: 8.0 vs 5.9, respectively (beta coefficient 1.4; 95% CI, 0.08, 2.7; P = .038). In addition, among intervention patients, there were 10 medication discrepancies in 51 patients (0.196 discrepancies per patient) that would otherwise not have been recognized when pharmacy insurance data were available, and 2 discrepancies in 131 patients (0.015 discrepancies per patient) when pharmacy insurance data were not available (P = .002). However, there were no differences in total medication discrepancies (0.16; −0.40, 0.73; P = .568), medication discrepancies in high-risk drug classes (−0.15; −0.42, 0.11; P = .261), or MAI (0.13; −0.61, 0.86; P = .734) (Table 3).

Table 3.

HIE effect on medication discrepancies and appropriateness: as-treated analysis

Measure	(1) HIE used; (2) pharmacy insurance data available	(1) HIE used; (2) pharmacy insurance data not available	(1) HIE not used	Beta coefficient (95% CI)a	P-value
N	51	131	199
Medication discrepancy outcomes, mean (s.d.)
Discrepancies	3.6 (2.6)	3.1 (2.6)	3.0 (2.4)	0.16 (−0.40, 0.73)	0.568
Risk-weighted discrepancies	8.0 (6.6)	5.9 (5.6)	5.7 (4.9)	1.4 (0.08, 2.7)	0.038
Discrepancies in high-risk drug classes	0.67 (0.77)	0.79 (1.0)	0.70 (0.94)	−0.15 (−0.42, 0.11)	0.261
Medication appropriateness, mean (s.d.)	2.6 (2.9)	2.5 (3.0)	2.1 (2.8)	0.13 (−0.61, 0.86)	0.734

^aAdjusted for age, number of chronic conditions, house staff service specialty, number of preadmission medications, admission illness severity, and hospital length of stay.

Effect of HIE on ADEs

Thirty-seven patients (9.7%) experienced 41 ADEs caused by medication discrepancies. All ADEs were characterized by temporary symptoms (eg, pain) or temporary organ dysfunction (eg, a rise in creatinine); no ADE caused serious or permanent harm. There were no differences in ADEs between those assigned to HIE-enhanced medication reconciliation and those assigned to usual care (OR 1.0; 95% CI, 0.49–2.1; P = .964) (Table 4), or between those who received HIE-enhanced medication reconciliation with pharmacy insurance plan data available and those who received usual care (0.67; 0.22–2.1; P = .480) (Table 4).

Table 4.

HIE effect on adverse drug events (ADEs) caused by medication discrepancies

Group	N	ADEs per admission	Odds ratio (95% CI)a	P-value
Intent-to-treat analysis
Intervention	186	0.102	−	−
Control	195	0.092	1.0 (0.49–2.1)	0.964
As-treated analysis
HIE used and pharmacy  insurance data available	51	0.098	−	−
HIE not used	199	0.101	0.67 (0.22–2.1)	0.480

^aAdjusted for age, number of chronic conditions, house staff service specialty, number of preadmission medications, admission illness severity, and hospital length of stay.

DISCUSSION

This study reports on the impact of HIE on the process and outcomes of medication reconciliation at the time of hospital admission to an urban US VA hospital. We hypothesized that HIE would raise the impact of medication reconciliation on discrepancies between preadmission and inpatient medication regimens, prompting resolution of these discrepancies by the house staff team and causing fewer discrepancy-related ADEs. Study findings indicate that patients who received HIE-enhanced medication reconciliation with an active pharmacy insurance data feed had on average 1 additional moderate-risk medication discrepancy identified than those who received usual care. In addition, for 1 in 5 patients who received HIE-enhanced medication reconciliation, discrepancies were detected that would not have been detected without access to the HIE. This effect was attenuated when the pharmacy insurance data feed to the HIE was stopped, leading to no difference in outcomes in intent-to-treat analysis.

Strengths of our study are that it tested the effect of HIE in potentially high-impact circumstances (medication prescribing at the time of hospital admission) and did not depend on voluntary HIE access by the user (the intervention pharmacist was obligated to access HIE for all intervention patients). This is important, because recent studies suggest that providers access HIE in only 5% or less of patient encounters where it is available.^32–34 Our study’s findings of a positive effect of HIE on detection of medication discrepancies, but only when medication information was available from a large pharmacy insurance plan, demonstrate the potential of HIE to have a positive effect on prescribing. However, when the insurance plan requested a transactional payment that was unaffordable to the HIE, the pharmacy insurance data feed was interrupted, significantly reducing the HIE’s impact on prescribing.

This case is an example of commercial interests adversely affecting HIE. Charging for access to prescription information led to information blocking, with measurable deleterious results. The importance of HIEs’ having comprehensive information that may not be available elsewhere cannot be overstated, in part because their viability depends on adding value and creating demand for services,³⁵ and because patients and providers rely on this information for sound decision-making. Aligning government HIE policy to facilitate and ensure inclusion of all relevant information is essential to the success and survival of HIEs. In this regard the Senate Committee on Health, Education, Labor, and Pensions proposed legislation in 2016 that ultimately was not passed that would have given the Office of Inspector General the authority to investigate and establish deterrents to information-blocking practices that interfere with appropriate sharing of electronic health information.³⁶

An important limitation of our study was low house staff responsiveness to medication discrepancy information. Of a total of 455 medication discrepancies identified by the intervention pharmacist, only 14 were rectified by house staff prescribing orders, and most rectifications came after our intervention pharmacist spoke directly with the physician. Furthermore, 36% of medication reconciliation notes designated for cosignature were never cosigned by a physician. This inertia may reflect house staff not accessing the intervention pharmacist’s medication reconciliation note, or accessing the note but deciding not to act, and illustrates the limited impact of asynchronous notifications common in electronic health records. Of note, in this study, only 3.5% of medication discrepancies caused ADEs. This is consistent with previous studies showing that, although medication prescribing errors are common, only a small percentage of them cause harm.^37–39 This low rate at which medication discrepancies cause ADEs helps explain house staff inertia in addressing medication discrepancies.

A second limitation that may have lessened impact was that the mean time from hospital admission to the intervention pharmacist’s medication reconciliation was 2.9 (±2.4) days. However, this delay, which was the time required to identify and enroll participants, was the same in both intervention and control groups. In addition, our study had an intervention pharmacist conducting a structured medication reconciliation process in the usual care group, which may have raised the quality of usual care and attenuated the difference between groups. Finally, the presence of medication information in the Bronx RHIO HIE was relatively low. This likely reflects utilization patterns of patients with VA coverage: some study subjects may have relied primarily on the VA pharmacy for prescriptions, and the HIE was not needed for providers to access this information.

CONCLUSION

HIE may provide an incremental benefit over usual medication reconciliation, but charging for information may result in information blocking, reducing the impact of HIE. Our findings indicate that efforts to improve medication reconciliation likely need to prioritize high-risk discrepancies in alerts to providers. Knowing when prescribers read, understand, and do or do not agree to rectify medication discrepancies is important to informing improvement efforts. There may also be benefits to implementing HIE other than health, including time saved during information-gathering, which provides a strong case for supporting HIE.

Funding

Financial support for the study was provided by the US Department of Veterans Affairs Health Services Research and Development Service (grant no. IIR-10‐146). This work was supported with resources and the use of facilities at the James J Peters VA Medical Center, Bronx, NY. The study sponsor and the Bronx RHIO had no role in the study design; in the collection, analysis, and interpretation of the data; in the writing of the report; or in the decision to submit the paper for publication. The contents do not represent the views of the US Department of Veterans Affairs or the United States government.

Author Contributions

KSB: conception and design, acquisition of data, analysis and interpretation of data, drafting the article and revising it critically for important intellectual content; WH: acquisition, analysis, and interpretation of data, revising article critically for important intellectual content; JP: acquisition, analysis, and interpretation of data, revising article critically for important intellectual content; KDiP: acquisition, analysis, and interpretation of data, revising article critically for important intellectual content; JW: acquisition, analysis, and interpretation of data, revising article critically for important intellectual content; JP: acquisition, analysis, and interpretation of data, revising article critically for important intellectual content; JN: conception and design, drafting the article and revising it critically for important intellectual content; WH: conception and design, analysis and interpretation of data, drafting the article and revising it critically for important intellectual content.

COMPETING INTERESTS

All authors have no competing interests to declare.

Clinical Trials registration

This study is registered at ClinicalTrials.gov (no. NCT01239121).

SUPPLEMENTARY MATERIAL

Supplementary material is available at Journal of the American Medical Informatics Association online.