Use of Health Information Technology to Advance Evidence-Based Care: Lessons from the VA QUERI Program

ABSTRACT

BACKGROUND

As the Department of Veterans Affairs (VA) Health Services Research and Development Service’s Quality Enhancement Research Initiative (QUERI) has progressed, health information technology (HIT) has occupied a crucial role in implementation research projects.

OBJECTIVES

We evaluated the role of HIT in VA QUERI implementation research, including HIT use and development, the contributions implementation research has made to HIT development, and HIT-related barriers and facilitators to implementation research.

PARTICIPANTS

Key informants from nine disease-specific QUERI Centers.

APPROACH

Documentation analysis of 86 implementation project abstracts followed up by semi-structured interviews with key informants from each of the nine QUERI centers. We used qualitative and descriptive analyses.

RESULTS

We found: (1) HIT provided data and information to facilitate implementation research, (2) implementation research helped to further HIT development in a variety of uses including the development of clinical decision support systems (23 of 86 implementation research projects), and (3) common HIT barriers to implementation research existed but could be overcome by collaborations with clinical and administrative leadership.

CONCLUSIONS

Our review of the implementation research progress in the VA revealed interdependency on an HIT infrastructure and research-based development. Collaboration with multiple stakeholders is a key factor in successful use and development of HIT in implementation research efforts and in advancing evidence-based practice.

INTRODUCTION

Health information technology (HIT) is prominently identified as a way to significantly improve the provision of health care^1,2. For all the recognized potential, realizing the benefits of HIT remains problematic^3–5. Implementation science involves efforts to improve the quality of health care by focusing on the system of care rather than individual components, such as physicians⁶. Although many implementation science studies have focused on discovering how to incorporate HIT into systems of care ^7–13, we sought to comprehensively evaluate the role of HIT across a continuum of implementation science studies.

We reviewed the use of HIT by researchers in studies conducted as part of the Department of Veterans Affairs (VA), Veterans Health Administration (VHA), Quality Enhancement Research Initiative (QUERI). This landmark program began in 1998 as a key element of VHA’s strategy to systematically examine and enhance the quality of VA medical care¹⁴ and had 360 active or recently completed individual projects at the time of this study. QUERI is a multidisciplinary, data-driven quality improvement program with the mission to translate research discoveries and innovations into better patient care and systems improvements¹⁵. This process has been described as a sequential and iterative six-step approach with additional activities for methods development and evidence development (see Fig. 1: Steps 1 to 6, Step M and Step C, respectively). Over 18 months, we evaluated HIT used in VA QUERI projects. We describe HIT use and identify specific contributions of the use of HIT in implementation science studies to the VA information architecture. We identify barriers and facilitators to the use and development of HIT in VA implementation research. Finally, we discuss lessons learned and implications for use of HIT in future health-care quality improvement research efforts.

Figure 1.

Health information technology needs at each step in the QUERI process. Modifications to this framework include Step M, Developing Methods, Measures and Data Sources, and Step C, Developing Evidence, are considered to be outside of the core QUERI process, although they support the process.

METHODS

We focused on nine QUERI Centers (see Table 1) operational for at least 1 year in 2005. We reviewed the QUERI Center Strategic Plans and Annual Reports for 2005 and 2006 to prepare a list of each Center’s research projects. We identified all implementation research designated as QUERI Step 4 (Implement Improvement Programs) or QUERI Step 5/6 (Evaluate Improvement Programs) projects. We extracted information about the use of HIT from the Step 4/5/6 projects’ abstracts and used the abstracted information as a basis for semi-structured group interviews with each QUERI Center’s key personnel, including the Research Coordinator, Clinical Coordinator, Implementation Research Coordinator, and other project staff. We conducted semi-structured interviews during September 2005-March 2006, seeking to identify (1) VA and non-VA data sources being used or developed, (2) any information system being utilized or developed, (3) any role in governance of data or information systems, (4) expected changes in data sources or information systems as a result of implementation research efforts, (5) identify perceived barriers regarding HIT, and (6) solicit a prioritized list of HIT needs to support the QUERI Center’s implementation research efforts. We performed descriptive and qualitative analysis of the interview data, including project-level analysis. This analysis allowed us to group our results into three categories of HIT use and development within the VA: (1) the role of HIT in providing data and information for implementation research, (2) the role of implementation research in the use and development of VA HIT, and (3) HIT barriers and facilitators to implementation research.

Table 1.

Name and Abbreviations of QUERI Centers

QUERI Center Disease Area Focus	Abbreviation
Chronic heart failure	CHF
Colorectal cancer	CRC
Diabetes mellitus	DM
HIV/hepatitis	HIV
Ischemic heart disease	IHD
Mental health	MH
Spinal cord injury	SCI
Stroke	STR
Substance use disorders	SUD

RESULTS

We identified 360 research projects from the nine QUERI Centers and determined that 271 of these projects met criteria consistent with QUERI Steps 4–6 implementation projects. We excluded 185 projects that had been completed prior to October 2004 in order to focus on the projects with the most recently available information. The project-level analysis focused on the remaining 86 implementation research projects.

HIT Role in Providing Data and Information for Implementation Research

A key aspect of HIT use was to provide data that either directly guided implementation projects or informed the impact of an intervention. We identified four data collection approaches used among the 86 implementation research projects: (1) VA-wide databases built with extractions from local health-care facility data systems (35 projects), (2) local or remote extraction from local electronic health records (EHR) (59 projects), (3) collection of primary data directly from research subjects (55 projects), and (4) collection of data from non-VA databases (6 projects). Most projects used a combination of approaches.

The Role of Implementation Research in the Use and Development of HIT

QUERI researchers’ use of HIT was concentrated in five distinct areas: (1) clinical decision support systems (CDSS); (2) HIT tools targeted for specific users; (3) telehealth applications; (4) design and maintenance of VA-wide databases; (5) integration and innovative use of extant VA-wide data (Table 2).

Table 2.

Examples of QUERI Projects for Each HIT Category

HIT category	Project example	QUERI Center	Project description	HIT features
1. Clinical decision support systems	Clinical reminder—alcohol misuse screening	Substance Use Disorder QUERI	Clinical reminder in VA electronic health record to screen patients for alcohol-related risks	• Reminder offers advice, brief intervention, or referral
				• Provider can insert free text as a progress note documenting the alcohol intervention
2. HIT tools targeted for patients	Personal health record –Web-based requests	Stroke QUERI	Web page on the VA personal health record for veterans to use in requesting stroke-related educational materials	• Addition to VA’s Web-based personal health record (My HealtheVet)
3. Tele-health	Telemedicine consultation—diabetes	Diabetes Mellitus QUERI	Intervention using teleconferencing to evaluate and document the process of outreach consultation	• Teleconferencing is used by diabetes specialists to hold consultations with patients from multiple sites
				• Evaluates short-term medical care utilization and costs associated with the intervention
4. New VA-wide databases	Departmental data collection– catheterization laboratory	Ischemic Heart Disease QUERI	Software application for standardized report generation, national data repository, and national quality improvement program for VA catheterization labs	• Tracks all VA catheterization labs to provide cardiac measurements
				• Automatically incorporates data from the EHR (medical history, medications, vitals, labs)
5. Extant VA-wide databases	Algorithm development–HIV case finding	HIV/Hepatitis QUERI	Usability for case finding of the Immunology Case Registry (ICR, which is the VA’s HIV disease registry)	• Developed, tested, and validated procedures for HIV case finding by analyzing HIV testing data
				• Evaluated the effects of enrollment into the ICR in determining access to quality HIV care and outcomes

HIT = Health Information Technology; QUERI = VA Quality Enhancement Research Initiative

Use and development of CDSS in implementation research projects included a range of informatics tools that utilized the EHR as a platform for providing cues to clinicians to support clinical care. These tools included clinical reminders for specific actions and/or follow-up, alerts for medication issues, decision aids, and knowledge tools. QUERI researchers used CDSS in 23 implementation research projects (27%). Notably all of the QUERI programs had at least one research project in their portfolio that included use or development of CDSS. Use of CDSS tools included both the modification of VA’s existing clinical reminder software within the VA EHR¹⁶ and development of customized CDSS.

Two projects exemplify the range of HIT innovation with CDSS that QUERI teams developed. The Substance Use Disorder QUERI Center (SUD) evaluated a questionnaire to serve as a scaled marker for screening alcohol-related risks. The SUD placed the questionnaire in a new alcohol screening reminder within existing clinical reminders software and integrated with the VA EHR.¹⁷ The SUD convened researchers, VA clinical informatics experts from the Office of Health Information, and leadership from the clinical and policy services (Office of Mental Health Service, Office of Quality and Performance, and Patient Care Services) to develop and evaluate the alcohol screening clinical reminder.

A QUERI collaboration developed a new hypertension management CDSS outside of the VA EHR platform. The collaboration focused on the development of the Assessment and Treatment of Hypertension: Evidence-Based Automation (ATHENA) CDSS and involved multiple VA investigators including the Stanford University Medical Informatics group. ATHENA is a CDSS that utilizes clinical practice guidelines to assist clinicians in lipid management and blood pressure control in primary care.¹⁸ The researchers implemented and tested ATHENA within a clinical trial to identify difficulties in integrating the CDSS within the information architecture of VA EHR.

Another group of HIT tools used and developed in QUERI research projects provided generalized reference or educational material to clinician, patient, or administrative staff user groups. Unlike CDSS tools that link an event or processes with decisions or choice sets for action, these HIT tools were not linked to any specific patient information or specific clinical decision but instead left the decision about applicability to the user. We found 17 QUERI projects (20%) developed HIT tools to provide support to clinicians (nine projects), educate veterans (five projects), or monitor system performance (three projects). Internet-based toolkits were a particularly frequent example of HIT tools to support clinicians. For example, SUD conducted a study to identify acceptance by physicians of an Opiate Monitoring System used to help opiod agonist therapy clinics assess concordance with evidence-based practices. SUD developed and evaluated an online toolkit to help physicians understand key issues in prescribing buprenorphine for treatment of opioid addiction.¹⁹ Unlike a CDSS, this online toolkit was not linked to specific actions or processes of clinical care, yet it provided important information to clinicians in an easily accessible format. Projects focused on veterans included enhancement of the veterans’ personal health record Website. For example, the Stroke QUERI (STR) developed a Web page to alert users of the VA personal health record to stroke-related educational materials. The application allowed veterans to request the stroke educational materials from the Website.

We found eight QUERI projects (9%) that evaluated home telehealth initiatives. QUERI telehealth initiative examples included: Diabetes Mellitus QUERI center (DM) efforts to test telemedicine consultations among general medicine and diabetes specialists to improve health care utilization among diabetic patients, and a SUD randomized trial of recruitment into a VA-wide smoking cessation program that utilized a telephonic ‘Quitline’ and telephone care management^20,21. Studies involving veteran-focused telehealth utilized varying levels of technological sophistication with some studies using videoconferencing or robotic technology.²²

We found seven QUERI projects (8%) focused on efforts to create new VA-wide databases. For example, the Ischemic Heart Disease QUERI center (IHD) used the development of a comprehensive clinical documentation and information system/database for the VA cardiac catheterization laboratories²³ as an occasion to study the process of implementing HIT across the expanse of the VA.

The final category of researchers’ involvement in the development of HIT applications was efforts to integrate existing data into more useful formats. Implementation research projects in this category involved improving existing data. The HIV/Hepatitis QUERI Center (HIV) evaluated the use of a new case-finding algorithm to populate the VA HIV/AIDS Immunology Case Registry (ICR) to improve tracking of VA antiretroviral use and outcomes.²⁴ A DM project studied the use of diabetes registry data to improve the assignment of patients to a clinic group responsible for diabetes care.²⁵

HIT Barriers and Facilitators to Implementation Research

We discussed barriers and facilitators in the use and development of HIT in our semi-structured group interview with QUERI Center key personnel. We identified three major categories of barriers that, if resolved, would facilitate HIT use: (1) lack of local patient-level clinical data in VA-wide databases; (2) lack of VA-wide data in a standardized and analyzable format; (3) gaps in capture of clinical data in the EHR.

All of the QUERI Centers identified the need for additional patient-level clinical data that are captured locally in the VA EHR but were not included in standardized system-wide databases. Examples of specific types of such data included patient-level vital signs data, body mass index, weight, microbiology laboratory results, cancer pathology and diagnostic data, and smoking status. These data elements were identified as necessary for identifying target populations, tracking effects of implementation research interventions, and tracking national trends. Although the VA EHR is a VA-wide electronic health record, many clinical data elements are stored only at the local hospital level rather than in a national database. This situation creates challenges for multi-site studies because there may be no means for capturing data from multiple local EHR systems into a central repository for review and analysis during an implementation research project.

In our interviews, it was noted that two factors often made the option of extracting data from local EHR systems to a study-specific research repository untenable. First, regulations on human subject research and privacy protections required approvals from the Institutional Review Board of each participating site. The variability of IRB procedures introduces costs and delays that are particularly detrimental to “real-time” implementation research efforts.²⁶ Second, coordination of multiple separate data extractions may also increase error rates compared to a single extract from a national database. Where possible, QUERI researchers attempted to utilize existing VA-wide patient-level clinical data, such as laboratory results and pharmacy data as noted previously. Another barrier we identified was the lack of standardized and analyzable data that make it “research-ready.” The most prevalent problem reported was that data were not standardized across facilities and thus were more difficult to collect, prepare, and analyze.

Finally, regarding gaps in capture of clinical information in the EHR, there were several examples shared that identified long-standing unmet needs for data collection. Implementation researchers petitioned to add additional data to the EHR that would then make nationwide clinical data available for research use. For example, researchers desired changes to the VA EHR to collect patient-centered outcomes data using standard assessment instruments, such as the SF-36 for health-related quality of life, and various specific mental health and substance use screening tools. Researchers reported that separate data collection outside of the EHR was necessary to meet research timelines.

DISCUSSION AND LESSONS LEARNED

Impact of Implementation Research on HIT

VA implementation researchers needed to form collaborations with the clinical, administrative, and technical VA organizational units in order to use HIT in their research projects. Our evaluation showed a high level of interaction between QUERI implementation research projects and HIT development. Almost all QUERI programs collaborated with researchers, policy-makers, practitioners, or administrators on HIT issues. QUERI Centers collaborated most frequently with the VA’s Office of Health Information and various clinical policy leaders within the Office of Patient Care Services. It is important to note that implementation research-driven HIT enhancements had direct benefits for clinicians (e.g., providing documentation and decision-support tools) and for system leaders (e.g., supporting quality and performance monitoring and improvement) in addition to implementation researchers.

The dependence on HIT development for successful implementation research is consistent with prior research in non-VA settings. Nagykaldi and Mold²⁷ found HIT necessary for the delivery of primary care preventive services. In their implementation of a preventive services reminder system, they found that rural practices that were stable and ready for change were most able to successfully utilize and integrate new HIT in their practices and that facilitated improvements in preventive care delivery. In their review of HIT in US health care, Doebbeling et al.²⁸ noted that the informatics infrastructure is integral to implementation of new evidence. In particular, they describe a range of HIT including data acquisition methods, health care standards, including standardized terminologies, data repositories, clinical event monitors, data-mining techniques, digital sources of evidence, and communication technologies, as specific examples shown to be integral to implementation of evidence-based practices.

HIT Barriers and Facilitators

QUERI researchers reported that while some clinical information was available for look-ups and clinical decision making in the EHR, sometimes data were stored in text fields or in images that required chart review and record abstraction in order to reprocess the data for analysis. In some cases we identified that clinical data needed for implementation research were not collected in the electronic record, so new data collection processes were undertaken. New data collection processes integrated within the EHR have a long timeline from development to deployment. The data collection cardiac catheterization laboratory system, a powerful tool for clinicians, managers, and researchers, took more than 5 years to develop. Such a sustained engagement between implementation researchers and the health care delivery system is a unique attribute of QUERI and is difficult to replicate at sites dependent on more traditional 3- to 5-year research funding cycles.

The VA QUERI experience reinforces the findings from research and operations partnerships in other settings. A research collaboration among 17 primary care sites serving a diverse population of insured and uninsured patients throughout Indianapolis faced the problem of obtaining data to support an evidenced-based collaboration.²⁹ Similar to the VA implementation research efforts, the solution was to improve data management and identification of eligible patients by applying key principles of data standards and centralized electronic data management to ensure successful deployment. Chaudhry et al. reviewed the literature on HIT use and found HIT prominently used to provide measures for primary and secondary screening.³⁰ This finding is consistent with the VA’s use of HIT to support performance monitoring and tracking.

QUERI investigators also identified changing operational priorities as a barrier to implementation research as the shifts in priority led to decreased support for HIT projects tied to research. DM joined with an organizational partner to develop a repository to support both operations and diabetes implementation research. The organizational partners withdrew their support for replenishment of the repository, and subsequently the repository fell into disuse because aging of the data made it too old for effective use in research. Integration of QUERI-developed databases and information technology is crucial for sustainability of research impact. As such, it is necessary to ensure sufficient and steady resources to maintain valuable data resources and integrate them with other VA information systems.

Changing priorities in the business side of HIT are to be expected. The main business processes that have traditionally driven HIT development are clinical care requirements and administrative requirements. Implementation research introduces new considerations for HIT development. In particular, implementation research focuses on how to implement evidence-based practices that originate from research in the medical community, often from outside of an organization. Implementation research thus becomes an agent of HIT change because of its focus on introducing new practices from outside of an organization. VA QUERI investigators are dependent on HIT but must rely on others for HIT execution—for example, the VA Office of Health Information is needed to address barriers to data collection and serve as advocate, motivator, and negotiator for data and information technology activities. The Office of Patient Care Services is needed to serve as an advocate for clinical policy priorities (and hence HIT priorities) that are grounded in the evidence-based principles that QUERI researchers are attempting to foster and disseminate. Not surprisingly, our study found the greatest successes in cases where QUERI HIT reinforced traditional clinical or administrative business processes or could draw from pre-existing HIT products. The greatest challenges occurred where QUERI HIT was unrelated to traditional business processes or introduced new HIT products. In such instances, success required simultaneous coordination of implementation research, clinical, and administrative business processes over a sustained period of time to optimize VA databases and information technology.

CONCLUSION

The strength of our evaluation is that it represents the implementation research experience in the VA, one of the early adopting EHR organizations and one of four organizations that has contributed substantially to the HIT efficacy literature.³⁰ Overall, this work supports the call for more information about what happens when implementation research drives process redesign that involves deployment and evaluation of HIT.^31,32 Further research aimed at fostering innovative uses of HIT that support health care quality improvement within the VA and in other health care settings is needed.