Abstract
Information systems are increasingly important for measuring and improving health care quality. A number of integrated health care delivery systems use advanced information systems and integrated decision support to carry out quality assurance activities, but none as large as the Veterans Health Administration (VHA). The VHA's Quality Enhancement Research Initiative (QUERI) is a large-scale, multidisciplinary quality improvement initiative designed to ensure excellence in all areas where VHA provides health care services, including inpatient, outpatient, and long-term care settings. In this paper, we describe the role of information systems in the VHA QUERI process, highlight the major information systems critical to this quality improvement process, and discuss issues associated with the use of these systems.
Information systems are increasingly important for measuring and improving health care quality. A number of integrated health care delivery systems use advanced information systems and integrated decision support to carry out quality assurance activities, of which the Veterans Health Administration (VHA) may be the largest.
The Institute of Medicine report Crossing the Quality Chasm noted that scientific evidence about best care is not applied systematically or expeditiously in clinical practice. It is estimated that an average of 17 years is necessary for new knowledge generated by randomized, controlled trials to be incorporated into clinical practice.1,2 The gap between scientific evidence and clinical practice can be bridged only by influencing clinician behavior to translate research findings into routine clinical practice.3,4 Information technology can facilitate the translation of research findings into practice. The 2001 President's Information Technology Advisory Committee in its report “Transforming Health Care Through Information Technology” noted that information technology “offers the potential to expand access to health care significantly, to improve its quality, to reduce its costs, and to transform the conduct of biomedical research.”5 The future of a cost-effective, evidence-based clinical care system is dependent on significant investment in information technology that will support clinical research and strengthen clinical practice.
Investment in information technology is key to the VHA Quality Enhancement Research Initiative (QUERI) and is the focus of this overview paper. The development and use of information technology for VHA QUERI can be characterized as what Berg6 described as a “socio-technical” approach that incorporates organizational factors in a process involving iterative technical design in response to organizational input and explicit attention to the political context of technology implementation. As has been described by Demakis and McQueen and their colleagues, the VHA QUERI utilizes methodologies for translation research with an emphasis on organizational structures and process considerations.7,8 It includes a variety of coordinated and specifically targeted efforts to integrate information technology into the context of health care. Each QUERI coordinating center works with the existing information technology and organizational context to achieve their goals but may also create new information technology to meet their needs. In this paper, we describe the role of information technology in the VHA QUERI process. We highlight the major information systems critical to this quality improvement process and discuss ongoing issues.
VHA's Quality Enhancement Strategy and Information Technology
As described in the overview manuscript by McQueen and colleagues,8 the VHA QUERI approach comprises an iterative series of steps aimed at achieving and maintaining continuous quality improvement in specific chronic disease areas. Throughout these steps, information technology plays a critical role. ▶ depicts the information technology needs at each step in the QUERI process. During the initial steps (steps 1–3), activities focus on identifying extant data and need for new data collection to describe and analyze the Department of Veterans Affairs (VA) health care practices and performance. QUERI coordinators need to ascertain the reliability, sensitivity, and completeness of extant databases and anticipate future data needs for their target patient conditions. Component methods for identifying patients include case findings and screening algorithms. This process may include the use of existing clinical or administrative data or may require new data collection, for example, through patient or provider surveys. Development, expansion, or improvement of existing patient registries and cohort databases may be needed and may include primary identification/enrollment of patients and informatics innovations.
Figure 1.
Information technology needs at each step in the QUERI process.
As the QUERI process progresses to implementation and evaluation (steps 4–6), each registry or cohort database system must be evaluated and validated for completeness, accuracy, and utility for specific routine uses. Enhancements to the information architecture necessary to create standardized data and support quality improvement must also be considered. For example, QUERI coordinators might consider the extent to which feedback reports at the clinician or clinic level will be needed, how the utility of feedback reports might be evaluated, and whether specific clinical procedures need to be reviewed in real time or retrospectively. Linkages with other information systems (e.g., inpatient and long-term care facility data) or linkages of different types of data (e.g., patient prescriptions, laboratory test results) need to be considered. Since the clinical quality improvement process focuses on patient outcomes, there must be linkages with clinical and satisfaction outcomes at multiple levels (e.g., provider, clinical unit, hospital, region). These linkages may require additional data collection to fill outcome measurement gaps.
In addition, information technology may serve as an intervention component to improve a performance gap. The interventional use of information technology might require access to patient medical record data, organizational data, and clinical decision support tools. The QUERI teams must consider best approaches to ensure that the information technology system and the health care providers can work well together to meet point of care needs while moving toward quality improvement goals.
The VHA information technology architecture provides a core of resources and linkages for this quality improvement effort. The next section provides an overview of the key clinical and health care event information systems used by QUERI.
VHA Clinical Information Systems
The VHA clinical information system, begun in 1982 as the Decentralized Hospital Computer Program (DHCP) and now known as VistA (Veterans Health Information Systems and Technology Architecture), was developed internally to meet the VHA's unique needs. VistA has evolved into a very rich health care information system9,10 that provides the information technology framework for VHA's approximately 1,300 sites of care, including medical centers, outpatient clinics, and nursing homes.11 It also provides the central information technology infrastructure for VHA's QUERI.
VistA supports day-to-day operations at local VA health care facilities, with the primary focus on direct patient care. VistA provides the source data for many other administrative, financial, and workload databases available through local hospital information systems and databases maintained at other locations, including the National Patient Care Database and the Decision Support System and their associated SAS® extracts12 (SAS® is a registered trademark of SAS Institute Inc., Cary, NC). VistA also includes clinical, administrative/financial, and infrastructure-related data that are not available in other VHA information systems9,13(▶). These process-related data are key to the QUERI initiative in ascertaining impacts and interactions between clinical outcomes and process factors, such as staffing patterns, volume of care, and appointment delays.
Table 1.
Sample Data Elements Included in VistA Databases Not Currently Available in National VA Databases
| Data Category and Elements | Type of Information Needed | VistA Source Package |
|---|---|---|
| Homocysteine/Vitamin B | RESULT | Lab |
| Microalbumin/creatinine ratio | FACT & RESULT | Lab |
| Potassium chloride | RESULT | Lab |
| Blood pressure | RESULT | Vitals/measures |
| Height | FACT & RESULT | Vitals/measures |
| Pulse | FACT & RESULT | Vitals/measures |
| Weight | FACT & RESULT | Vitals/measures |
FACT = date, time, and site of test performed; RESULT = Currently, these test results are entered in patients' records in text form. The CPRS re-engineering project will create encoded data elements for recording many standard test results.
VistA is built on a client-server architecture that ties together workstations and personal computers with nationally mandated and locally adapted software accessed through a graphical user interface (GUI) at VHA facilities. One aspect of VistA that has served the QUERI well is its ability to incorporate, within the national VistA software suite, locally developed software tools to test and collect unique data. For example, the Spinal Cord Injury and Disorders (SCI&D) QUERI team augmented the local VistA software suite at two facilities to collect information specific to influenza vaccination of patients with SCI.14 VistA also includes links that allow the incorporation of commercial off-the-shelf products.
VistA comprises more than 100 applications (e.g., pharmacy, radiology, nursing, laboratory) (▶) that clinicians access via the Computerized Patient Record System (CPRS) GUI to pull all the clinical information together from the underlying, facility-based, M programming environment and database (formerly known as the Massachusetts General Hospital Utility Multiprogramming System or MUMPS). CPRS provides a single interface for health care providers to review and update a patient's medical record as well as to place orders for various services including medications, special procedures, x-rays, patient care nursing orders, diets, laboratory tests, and consults. CPRS is flexible enough to be used in a wide variety of settings by a broad spectrum of health care workers and provides a consistent, event-driven, windows-style interface. More than 100,000 VA health care providers nationwide currently use CPRS. At the many VA medical centers now using the computer system exclusively, providers rarely need a paper medical chart.
Table 2.
Veterans Health Information System Technology Architecture (VistA): Highlights of Components
| Clinical Packages | Administration/Financial Packages |
|---|---|
| Admission, discharge, transfer (ADT/registration) | Accounts Receivable |
| Registration* | |
| Computerized Patient Record System (CPRS) | Automated Information Collection System (AICS) |
| Dentistry | Automated Medical Information Exchange (AMIE) |
| Dietetics | Automated Safety Incident Surveillance Tracking System (ASISTS) |
| Home-Based Primary Care (HBPC) | Clinical Monitoring System |
| Immunology Case Registry (ICR) Overview | Current Procedural Terminology (CPT) |
| Intake and Output | Decision Support System (DSS) Extracts |
| Laboratory | Diagnostic Related Group (DRG) Grouper |
| Lexicon Utility | Engineering |
| Medicine | Equal Employment Opportunity (EEO) |
| Mental Health | Equipment/Turn-in Request |
| Nursing | Event Capture |
| Oncology | Fee Basis |
| Patient Care Encounter (PCE) | Generic Code Sheet |
| Pharmacy* | Hospital Inquiry (HINQ) |
| Primary Care Management Module (PCMM) | Incident Reporting |
| Prosthetics | Income Verification Match |
| Quality: Audiology and Speech Analysis and Reporting (QUASAR) | Integrated Funds Distribution, Control Point Activity, Accounting and Procurement |
| Radiology/Nuclear Medicine | Integrated Patient Funds |
| Remote Order Entry System | Integrated Billing |
| Resident Assessment Instrument Minimum Dataset (RAI/MDS) | Library |
| Social Work | Missing Patient Registry |
| Spinal Cord Dysfunction | Occurrence Screen |
| Surgery | Patient Representative |
| Veteran Identification Card (VIC) | Personnel and Accounting Integrated Data (PAID) |
| VistA Imaging System | Police and Security |
| Visual Impairment Service Team (VIST) | Record Tracking |
| Vitals/Measurements | Voluntary Timekeeping |
| Women's Health |
These packages are key foundational components.
CPRS organizes and presents all relevant data on a patient in a way that directly supports clinical decision making. The comprehensive cover sheet displays timely, patient-centric information, including active problems, allergies, current medications, recent laboratory results, vital signs, hospitalization, and outpatient clinic history. This information is displayed immediately when a patient is selected and provides an accurate overview of the patient's current status before any clinical interventions are ordered.
CPRS capabilities include
A real-time order checking system that alerts clinicians during the ordering session that a possible problem could exist if the order is processed;
A notification system that alerts clinicians about clinically significant events;
A patient posting system, displayed on every CPRS screen, that alerts clinicians to issues related specifically to the patient, including crisis notes, warnings, adverse reactions, and advance directives;
A clinical reminder system that allows caregivers to track and provide preventive health care for patients and ensure that timely clinical interventions are initiated;
A remote data view functionality that allows clinicians to view information from other VA facilities that is relevant to a local patient.
VistA uses standard coding for much of its data, including ICD-9, CPT, DSM-III, LOINC, and other universal and standards-based coding methodologies. All VHA sites are required to map local data to these standard coding schemes. The VHA's VistA is continuing to establish more rigorous coding support to improve the ability of all users to readily analyze VistA data.
In addition to CPRS, VistA Imaging is operational at more than 115 VA medical centers. VistA Imaging provides a multimedia, online patient record that integrates traditional medical chart information with medical images of all kinds including x-rays, pathology slides, video views, scanned documents, cardiology examination results, wound photos, dental images, and endoscopies. The VistA Imaging application allows users to view, from any qualified workstation, images captured via digital means. Many commercially available, complex medical diagnostic systems (e.g., for pulmonary function testing and gastroenterology) now provide the means for clinicians to capture images and create complex clinical reports. VistA Imaging combined with the Clinical Procedures application now under development will be used to transfer the completed reports to VistA so they can be viewed via CPRS and VistA Imaging. Clinical Procedures will also establish complex interfaces with these products to capture, where feasible, the discrete data within many test results (e.g., ejection fractions measured during cardiac catheterization).
A new dimension of VistA, the Health Data Repositories (HDR), will support a full electronic patient medical record, integrating clinical data from all VHA sites of care. In addition, the HDR will be able to receive data electronically from other health care entities such as the U.S. Department of Defense, private health care, and any reference facilities (such as specialty laboratories). All data in the HDR will be fully encoded. Data warehouses and data marts will be created using data extracted from the HDR. The development team includes VA research scientists to ensure that the HDR will be able to facilitate medical research.
As a research tool, VistA provides comprehensive, patient-level, clinical data and hospital organizational information, such as staffing and specialty units. VistA provides one source of information across the spectrum of care provided at the local facility. Individual identifiers used in VistA are compatible with those in other VA information systems. VistA provides several avenues to access and extract data. New Care Management software includes a query tool for sorting and exporting data for multiple patients simultaneously. The VA FileMan, Health Summary, and Patient Data Exchange (PDX) utilities all provide access to patient data. MUMPS or M Code can also be written to extract subsets of data for use on approved platforms. Within specific VistA software programs such as Clinical Reminders and within the Decision Support System, there are also capabilities for exporting data in other formats. Exported data can then be analyzed on other computer platforms using various specific commercial statistical analysis packages (e.g. SAS®, SPSS®) or other software tools (SPSS® is a registered trademark of SPSS Inc., Chicago, IL).
VistA has some limitations affecting its use in research. Ad hoc queries are challenging because of the size and complexity of the VistA database. VistA has more than 1,940 files and more than 44,960 data fields. Accurate extraction of data requires expert knowledge about the underlying data structure and programming environment. Since each local VistA only includes information about services provided at that facility, multisite studies require data extraction from all facilities involved. The local Institutional Review Board, Information Resource Management Chief, and sometimes other local authorities must authorize data extraction from remote locations. Some clinical data are currently stored outside the VistA system and therefore are not extractable using VistA-based tools. Examples include results and images from radiology and cardiac catheterization tests. This limitation has been an obstacle for several QUERI teams' efforts to obtain measures for cardiac ejection fractions useful in characterizing severity of heart disease and ascertaining clinical outcomes. While the QUERI teams are working closely with the Office of Information to eliminate this data gap, in the short term, the data must be collected in a separate data collection process, generally by individual medical record review.
Another limitation of VistA is that some text data, such as patient history and physical information and discharge notes, are extractable only as narrative text, and require subsequent formatting/coding to enable analysis (▶). Also, some data may not be coded consistently across all sites, although this situation is improving. For example, laboratory tests may be coded differently (single versus panels) and reported in different units (mg/dL versus positive or negative or percentage). Knowledge of the specific standards used and the possible data values is required for accurate interpretation of the data.
Table 3.
Sample Data Elements in VistA Retrievable as Text Data
| Data Category and Elements | Type of Information | VistA Source Package |
|---|---|---|
| Cardiac imaging | Narrative text interpretation | Medicine, but will be replaced by clinical procedures |
| Pathology reports | Interpretive diagnostic information | Pathology |
| Surgery reports | Interpretive diagnostic information | Surgery |
The Health Data and Informatics Program of the VHA's Office of Information promotes VistA data quality, fostering best practices for ensuring uniformity of policy and practice. Evidence of the validity or reliability observed in VistA data has been described previously.13 Several recent articles focus specifically on the completeness and accuracy of particular elements of VistA data.15,16,17
In summary, the VHA VistA provides the informatics architecture that brings the clinical and process data together at the point of health care delivery. The common architecture that VistA provides supports implementation of health care quality improvement efforts across multiple points of care in the VHA.
National Database Systems
VistA provides the source data for several VA national aggregate databases and registries. These national datasets are populated through systematic transmissions of data from all VistA systems across the country and provide key resources for national-level evaluation of health care quality improvement efforts. Below, we highlight some of the key national databases used in QUERI.
National Patient Care Database SAS® System Files: Inpatient and Outpatient Datasets
The VHA's National Patient Care Database (NPCD) system was designed to consolidate legacy data systems into one national reporting system based at the VA Austin Automation Center. Since 1996, full diagnostic and procedural data for outpatients have been extracted from the local facilities' VistA systems and transmitted to the Austin Center where they are translated directly into the NPCD system using an Oracle™ relational database. Data marts and SAS® datasets12,18,19 are then created from the NPCD. The NPCD inpatient and outpatient SAS® datasets provide patient-level data on health care use across all VA points of care. The inpatient datasets, also known as the patient treatment files, include patient demographics, diagnoses, and admission and discharge dates, as well as a number of VA-specific variables (e.g., Prisoner of War status, means test category). The outpatient SAS® datasets, also referred to as the outpatient clinic datasets, include patient demographics, visit dates, and information on the number and types of clinic stops made by the patient. The outpatient SAS® datasets generated are visit (SF), event (SE), procedure (SC), and diagnoses (SG). They are linkable through use of a common identifier.
The quality improvement use of NPCD datasets has focused on identifying specific cohorts of patients who may benefit from quality improvement efforts and ongoing evaluation efforts. The datasets provide clinical detail for each discharge or ambulatory encounter20 and details on patients' health care use by the different types of providers in VHA. The information can be linked with other VHA patient-level data systems and across years to generate episodes of care for individual patients. These NPCD datasets provide information across all of the VHA, an advantage over using data from only one local hospital's VistA system since veterans often use VHA resources in more than one location.
Decision Support System (DSS)
The DSS brings together VHA clinical, resource use, and financial data at each VA facility. The DSS integrates data from existing VA clinical and administrative data systems, providing information about patients' episodes of care and the resources used. Each facility's DSS data are stored on computer systems at VA's Austin Automation Center, and the Center also provides national SAS® extracts of DSS data. The national clinical DSS datasets are particularly useful for QUERI because of the wealth of patient-level clinical data that are not available in any other VHA national dataset. Four clinical DSS extracts became available in FY 2002: (1) outpatient laboratory event file (Fact of Lab), (2) outpatient radiology event file (Fact of Radiology), (3) laboratory results file, and (4) outpatient prescription event file (Pharmacy). These datasets include diagnostic coding, provider information, and cost information that are linkable with other VHA data. The DSS holds great promise for quality improvement evaluation efforts because it provides a mechanism for systematic review of treatment patterns and resource use at the encounter, patient, provider, department, or hospital level. For example, ordering of specific screening tests such as prostate-specific antigen assay or mammography can be tracked over time. Trends in ordering at the physician, department, and hospital level can be reported. The DSS can be used to track compliance with specific standards of clinical care. For example, the DSS can be used to evaluate whether patients with a diagnosis of diabetes mellitus have had their blood glucose levels and glycosylated hemoglobin (Hb A1c) levels monitored during the year. Involvement of clinical administrative and research personnel in the ongoing DSS implementation and improvement process should ensure clinical relevance of data elements and processes. Plans underway to use the DSS to create new financial and clinical data marts will make these data even more useful to evaluate quality improvements at multiple levels.
In summary these national data systems provide key resources to the quality improvement initiatives underway in the VHA.
Linking Research Tools and Developments with Clinical Operations and Policy
The QUERI efforts have enhanced or expanded several existing VHA information technology resources by developing entirely new databases and informatics tools, validating and refining existing databases, and analyzing and interpreting their contents. As shown in ▶, each of the QUERI programs has developed unique databases to support identification of performance gaps. The resulting databases and informatics tools and findings have proven to be extremely valuable for QUERI and its stakeholders in their quality measurement and quality improvement activities.
Table 4.
Informatics Strategies and Data Sources of Selected Veterans Affairs Quality Enhancement Research Initiative Disease-Focused Service Directed Projects*
| QUERI Disease Focus Research Project Title | Research Objectives | Data Collection Approaches | Informatics Tools |
|---|---|---|---|
| IHD: Translation plan to improve lipid management for patients with IHD | (1) Improve health of veterans with IHD in VISN 20 via increases in lipid level measurement, appropriate use of lipid-lowering agents, number of patients with low-density lipoprotein levels at guideline recommendation | Audiotaped interviews with clinicians; vitals, laboratory data, recruitment data, and clinical reminders retrieved from existing databases | Lipid clinical reminders created and implemented in CPRS; vitals, laboratory data, recruitment, and clinical reminders data retrieved via CPRS |
| (2) Further evaluate and refine intervention from Lipid Measurement and Management System pilot in VISN 20 using continuous quality improvement | |||
| (3) Develop and evaluate Lipid Clinical Reminder | |||
| (4) Design, test, and implement national data reporting system for lipid measurement and management | |||
| MH: MH QUERI translation plan to improve antipsychotic treatment | (1) Compare QUERI-recommended intervention to basic intervention regarding medication management in schizophrenia | Chart reviews to determine medication adjustment needs pre- and post- intervention; survey for clinicians regarding guidelines and use of informatics tools | Clinical reminders created and implemented within CPRS for medication prescribing practices; VistA Structured Query Language |
| (2) Prepare MH QUERI translation package for pilot test of national rollout strategy in two VISNs | |||
| (3) Facilitate a pilot of the QUERI-recommended intervention in two VISNs | |||
| SCI: Increasing influenza vaccination rates in the veteran SCI population | (1) To increase influenza vaccination rates and decrease morbidity and mortality due to respiratory infection in veterans with SCI at eight pilot sites | Vaccination rates: patient surveys and data supplied by the Office of Quality and Performance External Peer Review Program chart review program; hospital discharges and visits for respiratory infections–NPCD | NPCD; Spinal Cord Dysfunction Registry database |
QUERI = quality enhancement research initiative; IHD = ischemic heart disease; VISN = Veterans Integrated System Network; MH = mental health; SCI = spinal cord injury; CPRS = Computerized Patient Record System; NPCD = National Patient Care Database.
Abbreviated list in hard print, full listing available as an online data supplement at www.jamia.org.
QUERI's development of information technology tools is an important component of (and contributor to) VHA's overall informatics strategy and exemplifies the socio-technical approach. The resulting databases and data elements and information technology tools provide valuable information for patient care, quality improvement, research, and management decisions, including high-level policy formulation and resource allocation. An example from the Ischemic Heart Disease (IHD) QUERI's experience can illustrate this point. When the IHD QUERI team proposed to develop a national clinical reminder for lipid level management, it quickly became clear that the then-existing formal protocols and procedures for clinical reminder development were incomplete. Relationships between the VHA information technology offices and the clinical practice guideline development offices required realignment. This organizational realignment resulted in formal links and collaborations between QUERI, the VHA's Office of Quality and Performance, and the National Clinical Practice Guidelines Council, leading to formalized, well-documented processes for developing, testing, and implementing national clinical reminders in the clinical information system. The combined efforts and productive collaboration of the Office of Information, IHD QUERI, VA Information Resource Center, and the Guidelines Council led to the development of efficient, well-documented policies whereby the Council now champions the development of national clinical reminders. The Council also has led development of the electronic medical record and associated databases that will make guidelines and clinical data readily available for clinical management and system-wide performance evaluation.
Another example is described in the paper by Goldstein et al.21 The authors describe the integration of the new clinical decision support software ATHENA DSS in VA primary care clinics for the management of hypertension. They describe both the technical features of the implementation developed in an iterative process responding to organizational input and the interrelated process of attending to the organizational context by obtaining and maintaining endorsement of the project by the administrative and clinical leadership of the organization.
Integration of clinical needs and technology capacity is a dynamic and ongoing process. QUERI efforts and resources are expected to integrate with other VHA activities to ensure broad applicability with VHA's efforts to implement best practices. Databases and informatics provide tools to inform best practices and improve care of veterans, but integration is needed for this information to be used efficiently by practitioners, managers, quality officers, administrators, and others. Mechanisms are also needed for reporting and feedback.
Standards and processes for information system development are still evolving. QUERI centers are collecting information and storing it in a variety of ways. For example, Mental Health and IHD QUERI teams are using the clinical reminder software initiatives that include transmission of HL7 messages to Austin, with data then stored in SAS® files. A transition to a more efficient, flexible approach to quality measurement and analysis (such as storing data in a database for multidimensional analysis) is also desirable to facilitate additional analyses and data mining.
In conclusion, the health care quality improvements underway in the VHA rely in large part on the data and information systems that bring the essential quality measures and decision points to the relevant stakeholders. Point of care systems must be linked with national summary systems to use the data and information at different levels and steps in the quality improvement process. The challenges that the VHA faces in undertaking QUERI are not unlike other large- and small-scale health care quality improvement efforts. By describing the VHA's quality improvement efforts and the role of information technology in this dynamic process, we hope to share lessons that are applicable to other public and private sector health care systems undertaking similar programs aimed at improving the quality of health care for their patients.
Supplementary Material
References
- 1.Balas EA, Boren SA. Managing clinical knowledge for health care improvement. In: Bemmel J, McCray AT (eds). Yearbook of Medical Informatics. Stuttgart, Germany: Schattauer Publishing Company; 2000, pp. 65–70. [PubMed]
- 2.Institute of Medicine, Committee on Quality of Health Care in America. Crossing the Quality Chasm—A New Health System for the 21st Century. Washington, DC: National Academy Press, 2001.
- 3.Bero LA, Grilli R, Grimshaw JM, Harvey E, Oxman AD, Thomson MA. Closing the gap between research and practice: an overview of systematic reviews of interventions to promote the implementation of research findings. The Cochrane Effective Practice and Organization of Care Review Group. BMJ. 1998; 317(7156):465–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Eisenberg JM, Kamerow DB. The Agency for Healthcare Research and Quality and the U.S. Preventive Services Task Force: public support for translating evidence into prevention practice and policy. Am J Prev Med. 2001; 20(3 suppl):1–2. [DOI] [PubMed] [Google Scholar]
- 5.President's Information Technology Advisory Committee. Panel on Transforming Health Care. Transforming Health Care through Information Technology. Arlington, VA: National Coordinating Office for Information Technology Research and Development, 2001.
- 6.Berg M. Patient care information systems and health care work: a sociotechnical approach. Int J Med Inf. 1999;55:87–101. [DOI] [PubMed] [Google Scholar]
- 7.Demakis JG, McQueen L, Kizer KW, Feussner JR. Quality Enhancement Research Initiative (QUERI). A collaboration between research and clinical practice. Med Care. 2000; 38(6 suppl 1):I17–25. [PubMed] [Google Scholar]
- 8.McQueen L, Mittman BS, Demakis JG. Overview of the Veterans Health Administration (VHA) Quality Enhancement Research Initiative (QUERI). J Am Med Inform Assoc. 2004;11:339–43. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Department of Veterans Affairs, Office of Information. VistA Monograph, 2003–2004. Department of Veterans Affairs, Office of Information, September, 2002. Available at: http://vista.med.va.gov/monograph/2002/12_02_03-04.pdf Accessed July 7, 2004.
- 10.Kolodner R (ed). Computerizing Large Integrated Health Networks: The VA Success. New York, NY: Springer-Verlag, 1997.
- 11.Department of Veterans Affairs, Office of Public Affairs, Media Relations. VA Fact Sheet: Facts about the Department of Veterans Affairs. Washington, DC: Department of Veterans Affairs, Office of Public Affairs; April2003. Available at: http://www1.va.gov/opa/fact/vafacts.html Accessed July 7, 2004.
- 12.Murphy PA, Cowper DC, Seppala G, Stroupe KT, Hynes DM. Veterans Health Administration inpatient and outpatient care data: an overview. Eff Clin Pract. 2002; 5(3 suppl):E4. [PubMed] [Google Scholar]
- 13.Hynes DM, Joseph G, Pheil C. Veterans Health Information Systems and Technology Architecture (VistA) as a research tool. VIReC Insights. 2002; 3(1):1–8. [Google Scholar]
- 14.Weaver FM, Hatzakis M, Evans CT, et al. A comparison of multiple data sources to identify vaccinations for veterans with spinal cord injuries and disorders. J Am Med Inform Assoc. 2004:377–9. [DOI] [PMC free article] [PubMed]
- 15.Best WR, Khuri SF, Phelan M, et al. Identifying patient preoperative risk factors and postoperative adverse events in administrative databases: results from the Department of Veterans Affairs National Surgical Quality Improvement Program. J Am Coll Surg. 2002;194:257–66. [DOI] [PubMed] [Google Scholar]
- 16.Cowper DC, Kubal JD, Maynard C, Hynes DM. A primer and comparative review of major US mortality databases. Ann Epidemiol. 2002;12:462–8. [DOI] [PubMed] [Google Scholar]
- 17.Szeto HC, Coleman RK, Gholami P, Hoffman BB, Goldstein MK. Accuracy of computerized outpatient diagnoses in a Veterans Affairs general medicine clinic. Am J Managed Care. 2002;8:37–43. [PubMed] [Google Scholar]
- 18.VIReC Research User Guide: FY2002 VHA Medical SAS® inpatient datasets. Hines, IL: Veterans Affairs Information Resource Center; 2003.
- 19.VIReC Research User Guide: FY2002 VHA Medical SAS® Outpatient Datasets. Hines, IL: Veterans Affairs Information Resource Center; 2003.
- 20.Hynes DM, Cowper D, Kerr M, Kubal J, Murphy PA. Database and informatics support for QUERI: current systems and future needs. Quality Enhancement Research Initiative. Med Care. 2000; 38(6 suppl 1):I114–28. [PubMed] [Google Scholar]
- 21.Goldstein M, Coleman RW, Tu SW, et al. Translating research into practice: sociotechnical integration of automated decision support for hypertension in three medical centers. J Am Med Inform Assoc. 2004;11:368–76. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.