Informatics Implementation in the Veterans Health Administration (VHA) Healthcare System to Improve Quality of Care

Abstract

We describe VHA’s information technology (IT) implementation from the providers’ perspective, and identify factors influencing its effective implementation to improve care. We surveyed a stratified random national sample of 4227 clinicians from three VHA primary care provider groups: 1) physicians; 2) nurse practitioners, physician assistants; and 3) nurses. Facility-level IT support availability was rated across six dimensions: 1) access to literature/evidence, 2) computerized decision support, 3) computerized clinical data, 4) error reduction, 5) provider communication, and 6) patient communication. Factor analysis identified a 5-item scale (IT clinical support, α = 0.76). Generalized estimating equation models identified factors influencing IT clinical support. Complete data from 123 hospitals (1777 providers) were included. The final model showed IT clinical support was higher in hospitals that were urban (p<0.05) and had cooperative cultures (p<0.01). Opportunities exist to enhance effective use of IT to support clinical decision making, electronic communication with patients and access to recommendations while delivering care.

Introduction

The Institute of Medicine (IOM) and National Academy of Engineering have advocated widespread adoption of information technology (IT) to improve quality, evidence-based practice and to reduce medical errors.^1–3 IOM-recommended IT dimensions include: point-of-care access to health literature and evidence-based guidelines; computer-assisted decision support systems; computerized patient clinical data; automation of decisions to reduce errors; and electronic communication between providers and between providers and patients.²

The VHA electronic health record (EHR), is one of the most widely implemented in the United States. The VHA is a leader in the use of IT in clinical support systems.⁴ In 1995, the VHA initiated a reengineering effort to ensure the predictable and consistent provision of high-quality care throughout the system and to optimize the value of VHA health care.^{5, 6} The VHA has developed the informatics infrastructure necessary to support an EHR, the Computerized Patient Record System (CPRS), which enables clinicians to review and analyze patient clinical data, order laboratory tests and medications, document care, review radiology and other data and support clinical decision-making. It has been widely implemented throughout the VHA’s healthcare system of acute care medical centers, outpatient care clinics and long-term care facilities.

Little data exists regarding the institutional factors or conditions which influence adoption of key components of information systems within health care systems or individual medical centers. As the largest integrated healthcare delivery system in the U.S, the VHA provides an excellent forum in which to study IT factors that may help promote evidence-based practices. We sought to describe the state of IT within VHA nationally from the providers’ perspective and to identify institutional factors that influence the implementation of IT to improve quality of care.

Methods

Databases

We used three data sources: the 2000 American Hospital Association (AHA) Annual Survey Database, our 2001 Quality Manager Survey and our 2003 VHA Provider Survey data. The AHA’s Annual Survey Database, contains institutional-level data on multiple hospital characteristics, which includes ownership, accreditations, bed size, clinical services and equipment availability, staffing levels, and inpatient and outpatient utilization. Survey development was based on a literature review, our existing instruments from prior studies of clinical practice guideline (CPG) implementation in community hospitals and VA hospitals, and additional factors from our multi-institutional qualitative study of using IT in CPG implementation.^7–9 The surveys utilized 5-point Likert-type response scales that ranged from “not at all” (1) to “very great” (5), wherever appropriate. These databases were linked at the facility level to create an analytic dataset.

Quality Manager Survey (QMS)

Our QMS selected one or more key respondents at VAMCs and assessed efforts to improve quality and implement national VHA clinical practice guidelines.¹⁰

Provider Survey

Our Provider Survey selected and surveyed a sample of VHA providers at all acute care VAMCs nationally regarding their experience, work conditions, attitudes and experiences with guideline implementation, hospital culture, quality improvement efforts and IT tools at their facility. The survey included items assessing: 1) hospital’s provision of IT support across 6 dimensions to improve quality; 2) provider knowledge of and agreement with CPGs; 3) provider support for CPGs; 4) dissemination approaches; 5) patient care workload; 6) hospital culture, cooperation, and structure; 7) use of technology in implementation and maintenance; 8) performance feedback.

Providers rated the availability of various IT tools at their facility by responding to the following statement: “To what extent your hospital provides…”: 1) Access to literature/evidence based medicine (EBM) while delivering care; 2) Computer assisted decision support systems (e.g., condition-specific lab tests or medications); 3) Computerized patient clinical data (e.g., problem list, history of adverse drug reactions, or electronic medical records); 4) Automation of decisions to reduce errors (e.g., potential adverse drug interactions, correct dosages); 5) Electronic communication between providers (e.g., e-mail, automated result reporting); 6) Electronic communication between providers and patients (e.g., e-mail, automated result reporting).²

Provider Sample

The sampling frame included 139 VAMC acute care hospitals with primary care clinics and AHA Survey Data. VA administrative data from the Personnel Accounting Integrated Data (PAID) system were used to identify physicians, physician assistants (PA), nurses, and nurse practitioners (NP) in primary care, ambulatory care, medical service, nursing service, and geriatrics. Several reviewers independently reviewed the multiple occupational titles and service codes used, in order to aggregate the sampling frame to reflect active clinicians primarily based in ambulatory care. Where such a designation was not available, we aggregated and selected the sample to reflect the Medicine and Geriatrics services for physicians and physician assistants, and Medical and Nursing services for nurses. Based on power calculations, we sought to identify at least 8 physicians, 8 nurses, and 4 PAs and/or NPs, if available, at each facility.

An Institutional Review Board and VHA research committees approved the study and all procedures. Surveys were sent to 401 quality managers or other key personnel involved in guideline implementation at 143 VHA hospitals. At least one respondent provided data from 90% of VAMCs. We selected 4,621 VA providers. Providers not currently providing care were removed (N=394) resulting in final sample of 4227 providers: 1770 physicians, 1643 nurses, and 814 PAs or NPs. There were 2438 respondents (response rate = 58%). After linking the all three surveys, the total sample size included 2190 clinicians across 123 facilities. Surveys with missing outcome data were excluded for a final N of 1777.

Statistical Analyses

Descriptive Statistics were generated using facility averages. Organizational characteristics of responding and non-responding hospitals were compared. For normally distributed, continuous variables, the t-test was used to compare groups. When distributions of continuous variables deviated from normality, comparisons were made using the Wilcoxon rank sum test. For binary variables, statistically significant differences between group proportions were evaluated using the Chi-square test or Fisher’s exact test, as appropriate. Alpha was set at 0.05 and p-values were two-tailed. All analyses were conducted using SAS, version 8.02 (Cary, NC).

Factor analysis

The principal components method of factor analysis with varimax rotation was utilized to identify primary underlying dimensions and as a data reduction technique. Factor analyses identified a single factor with five items, which was reproducible across the three provider groups. Individual items were then summed to create separate scales. Factor analysis identified a single factor which included five IT items derived from the IOM report.² The survey item regarding electronic communication with patients was excluded due to the low rate of use and a low factor loading. The internal reliability was assessed using Cronbach’s alpha coefficient. Responses to the remaining five items were summed to create the factor scale, IT support (alpha=0.76).

Multivariate Modeling

Generalized Estimating Equation (GEE) regression models were constructed to identify characteristics that influence IT clinical support. GEE is a multivariate modeling procedure that takes into account correlations between different respondents at a facility to minimize the risk of identifying spurious associations. Predictor variables from the linked dataset (representing structure and processes hypothesized to be important) were grouped into two conceptual domains: 1) Organizational context; and 2) Change management and quality improvement. The multivariate modeling used stepwise regression with backward elimination. Bivariate correlates associated with the outcome at p<0.25 were evaluated in separate domain models. Using backward stepwise regression, variables from the domain models significant at p<0.15 were used to construct a final multivariate model (p<0.05).

Results

The 123 hospitals that provided institutional and provider-level data used in these analyses were widely distributed across the U.S. and had similar characteristics to other VHA hospitals. Notably, nearly half (46%) of the hospitals were members of the Council of Teaching Hospitals and had residency-training programs. The mean total bed size was 329, and 15% of hospitals were located in rural areas.

The mean age of providers surveyed was 48 years. The sample was predominantly female (61%) and had a mean duration of employment at the facility of 10 years. Physicians represented 40% of those participating, and nurses 37%. Internal Medicine was the most common clinical specialty reported (55%). Respondents spent 2 ½ days in a typical week providing primary care in an ambulatory clinic (49%), and one-day per week (20%) providing specialty care in an ambulatory clinic. Respondents spent about one-third of their time providing preventive services (30%) and even more (41%) documenting care. Two-thirds (61%) of their time involved providing direct patient care and 14% supervising residents.

Providers at nearly all hospitals agreed there was widespread availability of electronic medical record data. Nearly all (99%) providers (Table 1) indicated that their hospitals had computerized patient clinical data. Similarly, availability of electronic communication was nearly universally available (98%). However, there was significant variation between provider groups regarding the availability of other IT dimensions to improve quality of care. Automation of decisions to reduce errors was commonly reported (84%). Slightly less than half (46%) of the providers indicated that their hospital provided access to literature and EBM while delivering care. Less than a quarter (22%) of providers reported that computer assisted decision support systems were available.

Table 1.

Availability of IT at Facility (N=136).

Survey Item	Mean
Computerized patient clinical data	4.3
Electronic communication between providers	4.1
Automation of decisions to reduce errors	3.8
Access to literature/EBM during care	3.4
Computer assisted decision support systems	3.2
Electronic communication b/w providers & pts.	2.4

Respondents rated the extent to which IT and other measures were helpful in adhering to clinical practice guidelines in VA hospitals. Attitudes regarding the use of automated clinical reminders were very positive. Nearly three-quarters (74%) of providers believed that computerized clinical reminders to provide specific care were helpful in adhering to clinical guidelines. Tools or templates to facilitate documentation were rated nearly as high. Interdisciplinary teamwork was also considered extremely important in facilitating adherence.

Despite their perceived utility, computerized clinical reminders were not widely implemented across VA facilities. Approximately one quarter reported the availability of computer reminders to provide recommended services for chronic obstructive pulmonary disease, diabetes, and major depressive disorder. Nearly half of facilities had computerized clinical reminders for diabetes mellitus (44%). One-fifth to one-third of facilities used computer tools to document recommended services. When asked whether the computer reminders actually impacted the care that was delivered, the percentages were similar, but slightly lower. Personal digital assistants, such as Palm Pilots, were used infrequently as part of the hospital IT system (less than 2% of facilities).

Teaching hospitals more often provided access to literature, and supported using EBM tools and approaches, than non-teaching hospitals (p<0.05). Similarly, larger hospitals (> 100 beds) more often provide computer-assisted decision support (p<0.05) systems and electronic communication between providers (p<0.1) than smaller hospitals.

A number of institutional factors were associated with the IT Clinical Support scale. In bivariate GEE associations (Table 2), urban location, a generally cooperative culture and cooperation specifically between management and physicians were all highly positively associated with IT clinical support scale (p<0.01). Organizational context factors such as teaching hospital status, number of personnel (FTEs), and network (Veterans Integrated Service Network, VISN) approached significance. Within the change management dimension, the availability of sufficient resources approached significance. Staffing ratios, hospital type, and change management measures were unrelated to our measure of IT clinical support.

Table 2.

Bivariate Associations w/IT Clinical Support.

Variable	Beta	P-value
Teaching Hospital	0.32	0.12
Personnel	0.0002	0.10
Urban Location	0.89	<0.01
VISN Influence	−0.03	0.12
Cooperative Culture	0.08	<0.01
Sufficient Resources	0.17	0.16
Participation	0.15	0.23

The final model showed IT Clinical Support was higher in hospitals that were urban (p<0.05) and had cooperative cultures (p<0.01).

Discussion

VA has widely implemented an electronic medical record and email communication among providers in its health care system. VA’s Computerized Patient Record System is now the most widely implemented electronic medical record in a federal health care system, and is being considered for wider implementation elsewhere. VA providers, however, report considerable variation in the adoption of other key IT dimensions recommended to support improved quality of care and incorporation of evidence-based practice at their facilities. Industry organizations such as the Leapfrog Group, as well as professional associations, also support implementation of electronic medical records to improve service delivery and decrease costs. The variation in implementation we observed provides an important opportunity to better understand what facility-level factors, other than cost, influence the widespread adoption of IT support within a health care system.

Clinicians at urban hospitals and those with more cooperative cultures reported higher levels of clinical support across the spectrum of recommended IT dimensions. This finding may be partially explained by the concept of “resource slack”, in that larger urban hospitals may have the necessary resources and experience to facilitate adaptation of the IT system to meet the local practice standards and needs of providers. Similarly, a cooperative culture, particularly between physicians and management may help make the different needs and perspectives of both groups known and facilitate shared decision-making. Further study is needed to determine which IT system dimensions contribute most to improved quality and how best to effectively implement them. These data demonstrate that opportunities exist within the VA system for improvement in IT support for: 1) access to literature/EBM; 2) computer assisted decision support systems; and 3) automation to reduce errors. Variation in the extent of availability of these IT dimensions across facilities nationally may represent differences in the level of investment locally, differences in local adaptation of these tools, or variation in procedures to roll-out and train providers in the routine use of these tools in practice. Microsystem factors, such as the patient workload, and computer skills of the provider workforce (e.g., typing skills) likely influence their adoption and use.

Decision support systems are only as good as the quality and quantity of the data available. If such data are not accurate and complete, the system loses credibility and the intervention is ignored. The VA health care system has been a leader in the integration of electronic clinical and administrative data into its electronic medical record system. These data suggest there are still opportunities to improve the incorporation of important clinical data to support clinical and management decision making.

While accurate and usable data is a necessary foundation for assuring that new technology is reliable, there is no guarantee that the innovation will result in improved care or increased efficiency. For a technology to be accepted, and readily used, adaptation and customization for the intended environment and users are crucial. New technologies sometimes create new work for busy clinicians—mainly in the form of additional documentation tasks, such as manual entry of diagnosis code designations on electronic orders. Thus, any healthcare system that installs an electronic medical record should carefully consider what is asked of providers. In particular, one goal should be to try to quantify and minimize additional work burdens on providers from use of new IT. This will allow clinicians to accomplish the highest-priority clinical tasks and maximize their time with patients. We believe that analysis of adherence to clinical performance measures using IT data should be used primarily to improve systems of care—not to evaluate individual provider performance.

When implementing new technologies, healthcare systems should also recognize that various human factors need to be considered and evaluated during implementation. Many providers, for example, have difficulty with typing. Not surprisingly, as long as typing is required, many providers will view electronic medical records and decision support systems as a burden and their potential for improving efficiency will not be fully realized and/or providers may find ways to circumvent intended use (for example by typing very short notes that convey less information then their hand-written notes). Hence, understanding the culture of medical practice and its realities is as important as understanding the benefits of the technology. As a corollary, systems that can streamline the process of adaptation, so that new technologies are incorporated swiftly and effectively, will likely do better than those providing little IT support.

VA is quickly entering a new era of using IT to engage providers, patients, and managers in improving the health of our population. Integrating information technology (IT) into primary care provides both great opportunities and challenges from managerial, clinical and research perspectives. These data suggest that a culture of collaboration in facilities is particularly important in influencing widespread adoption and implementation of IT to improve evidence-based practice.

Our results suggest that further study is needed to determine how best to adapt IT to the patient care setting and effectively use IT to improve the quality and efficiency of care. Effective modification and implementation of IT into the work flow offers tremendous opportunities to improve the effectiveness and safety of primary care, as well as access to that care. Partnerships between managers, clinicians, informaticians and other researchers can help meet the challenges of effectively incorporating new technologies to improve quality of care.

Interdisciplinary collaborations are important to help IT implementation reach its maximum potential. These issues should be explored by a range of modalities, including qualitative tools, such as ethnographic interviews and human factors observation, and quantitative methods, such as linking survey and database data to better measure health care processes and the success of implementation. Providers and managers alike need to be involved in these efforts.

Conclusions

Our results suggest that the use of different dimensions of IT to improve quality varies dramatically across facilities, even within an integrated delivery system like VA. Thus, VA can serve as a model for understanding facilitators and barriers to implementing IT clinical support in other settings, by providing multiple opportunities to identify patterns of usage, determine the best means for integrating IT into patient care delivery, and develop interventions for internal transfer of best practices.

As IT is more widely implemented in U.S. health care systems, it will be important to focus on ways to more effectively implement IT to improve care delivery on a population and individual basis. Through incorporating IT capabilities effectively in healthcare systems, healthcare professionals, patients, and administrators can collaborate to craft a system of care that is efficient, economical, high quality and enhances the interpersonal relationship between the physician and patient.