Comparing VA and Non-VA Quality of Care: A Systematic Review

Abstract

Background

The Veterans Affairs (VA) health care system aims to provide high-quality medical care to veterans in the USA, but the quality of VA care has recently drawn the concern of Congress. The objective of this study was to systematically review published evidence examining the quality of care provided at VA health care facilities compared to quality of care in other facilities and systems.

Methods

Building on the search strategy and results of a prior systematic review, we searched MEDLINE (from January 1, 2005, to January 1, 2015) to identify relevant articles on the quality of care at VA facilities compared to non-VA facilities. Articles from the prior systematic review published from 2005 and onward were also included and re-abstracted. Studies were classified, analyzed, and summarized by the Institute of Medicine’s quality dimensions.

Results

Sixty-nine articles were identified (including 31 articles from the prior systematic review and 38 new articles) that address one or more Institute of Medicine quality dimensions: safety (34 articles), effectiveness (24 articles), efficiency (9 articles), patient-centeredness (5 articles), equity (4 articles), and timeliness (1 article). Studies of safety and effectiveness indicated generally better or equal performance, with some exceptions. Too few articles related to timeliness, equity, efficiency, and patient-centeredness were found from which to reliably draw conclusions about VA care related to these dimensions.

Discussion

The VA often (but not always) performs better than or similarly to other systems of care with regard to the safety and effectiveness of care. Additional studies of quality of care in the VA are needed on all aspects of quality, but particularly with regard to timeliness, equity, efficiency, and patient-centeredness.

Electronic supplementary material

The online version of this article (doi:10.1007/s11606-016-3775-2) contains supplementary material, which is available to authorized users.

INTRODUCTION

Providing high-quality health care is central to our nation’s commitment to veterans. The Veterans Affairs health care system (VA), the nation’s largest integrated health care system, provides comprehensive health services to US military veterans who are enrolled. However, the quality of VA care is a longstanding area of concern.^{1 , 2} As a result, the Veterans Access, Choice and Accountability Act (VACAA), passed in 2014, mandated an independent assessment of health care capabilities and resources of the Veterans Health Administration.³ As part of this assessment, the Interim Under Secretary for Health for VA called for a comprehensive evaluation of “VA’s ability to deliver high-quality health care to Veterans.”⁴

Previous studies have systematically reviewed aspects of care at the VA.^{5 – 7} Two peer-reviewed articles based on one of these⁷ summarized the available evidence on medical⁸ and surgical⁹ care quality through 2009. This review indicated that in most studies, the care provided in the VA compares favorably to non-VA systems. However, quality may have changed in the last 6 years since this review, and concerns about quality of care have continued to mount.¹⁰ To assess the quality of care provided by the VA, we performed a systematic review of published comparisons of the quality of care in VA facilities to other settings.

METHODS

Search Strategy

We defined quality of care using the Institute of Medicine’s (IOM) definition, which is “the degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge.”¹¹ Given the existence of a recent systematic review on the quality of health care delivered in VA versus comparable non-VA settings, we chose to explicitly build upon and expand this work.⁷ We use consistent methods, including the same search terms (Online Appendix) and database (MEDLINE). The dates searched were limited to the past 10 years (January 1, 2005, to January 1, 2015) to ensure a reasonable scope and up-to-date studies.

Article Screening and Data Abstraction

Titles and abstracts identified by our literature search were screened by two reviewers with experience screening and abstracting data for health-related systematic reviews (CO, CH) under the supervision of a senior member of the study team with extensive clinical and systematic review experience (CG) for inclusion in full-text screening. If the article presented comparisons of quality of health care in the VA and non-VA settings in the US, the full text of each article was reviewed independently by both reviewers (CO, CH). We extracted the data sources, geographical areas, clinical conditions, quality dimensions, and comparability of measures in each sample. Studies were excluded if they included no original data; were case reports; or used non-contemporaneous comparisons, unequal or non-representative samples, or dissimilar or indirect quality measures. Data from each article, including the size of the VA and non-VA samples, years of data collection, control variables, and primary and secondary outcomes, were abstracted by one reviewer and double-checked by another reviewer (CO, CH). Disagreements were resolved by a senior team member (CG).

Quality Assessment

Article quality was assessed using the criteria developed in the original review for assessing comparative health care quality studies.⁷ Briefly, six elements were used, including whether: (1) time periods during which the VA and non-VA patients were studied were sufficiently contemporaneous; (2) VA and non-VA samples were comparable in size/scope; (3) quality measurements were assessed using specified and identical indicators with a similar assessment format; (4) quality measures were meaningfully associated with outcomes; (5) measures were clinically relevant and important; and (6) the statistical methodology was sound. Each factor was graded (A for excellent, B for acceptable, C for unacceptable). The overall subjective assessment of the article was created, but this assessment was not an “average” of individual component grades. Thus, an article with a critical flaw could be graded “C” and rejected, even if it scored well on other criteria. Articles were graded by one reviewer and double-checked by the other reviewer (CO, CH). Disagreements were resolved in consultation with the senior team member (CG).

Study Classification

Although we employed a nearly identical search and abstraction strategy, we organized article categories differently than the original review,⁷ which split articles into medical⁸ (non-surgical) and surgical⁹ care and by Donabedian’s quality triad of structure, process, and outcome.¹² To guide decision making about how to improve quality at the VA, we organized our review using the IOM’s six characteristics of high-quality care: safe, timely, equitable, effective, efficient, and patient-centered,¹¹ and then grouped similar kinds of outcomes together. Timeliness as related to delays or wait times for receiving necessary medical care (e.g., time from admission to emergency surgery) was included in this review. Timeliness as it relates solely to access (e.g., wait time for obtaining an appointment)—while a critical issue currently facing the VA¹³—was not addressed because no studies directly comparing VA and non-VA settings were identified.¹⁴ We included articles on efficiency if they assessed appropriateness (necessary care versus overuse). We excluded cost-efficiency articles, as cost comparisons between VA and other settings have a number of limitations.¹⁵

We classified each study (both new and from the prior review) according to the direction of the statistically significant differences in performance measures for the VA care relative to a non-VA comparison group. If VA quality of care was shown to be better than non-VA care or if multiple results were reported and VA quality of care was better in some instances and the same in others, the study was classified as “VA better.” If multiple quality measures were reported and VA care was better than non-VA on some and worse on others, the study was classified as “mixed.” If the quality of care in VA and non-VA did not differ, the study was classified as “same.” If VA quality of care was shown to be worse than non-VA, the study was classified as “VA worse,” as were studies with multiple results reported where the quality of care was worse in some instances and the same in others.

Ensuring Comparability of Populations

Wherever possible, we report results adjusted for risk, comorbidities, demographics, or other variables. Unadjusted results are noted. We included and emphasized studies that compare VA patients to veterans receiving care in non-VA settings. We also included studies that compared veterans receiving VA care (referred to as “VA patients”) with individuals who are not identified as veterans receiving non-VA care (referred to as “non-veterans”).

RESULTS

Searches yielded 461 articles, 306 of which were rejected at abstract screening; 155 proceeded to full-text screening. Data were extracted from 69 articles; 31 of these were also included in the previous review,⁷ yielding 38 new articles (Fig. 1). Included articles were categorized by quality dimension: safety (34), timeliness (1), equity (4), effectiveness (24), efficiency (9), and patient-centeredness (5). Studies were sufficiently heterogeneous to preclude pooling or other meta-analysis, so results are presented narratively. The 34 articles on safety and 24 on effectiveness are summarized in Table 1.

Fig. 1.

Flow Diagram of Included and Excluded Studies. Of the 461 articles identified using our search strategies, 306 were excluded at title/abstract screening and 155 were screened for inclusion using the full text of the article. Of these, 86 were excluded, leaving 38 new articles for data abstraction. These articles were combined with the 31 articles from the prior review that met inclusion criteria, resulting in the 69 articles included in this systematic review.

Table 1.

Evidence table for safety and effectiveness

Author, year	Quality dimension(s) [assessment*]	Data level	VA sample [sample size]	Years collected	Data level	Non-VA sample [sample size]	Years collected	Conditions	Outcomes	Primary findings†	Final grade‡
New articles abstracted for this review
Bean-Mayberry B, et al., 2007	Effectiveness [mixed]	Mult ctrs	VA comprehensive Womens’ Health Centers [N = 8]	2003	Nat’l	Department of Health and Human (DHHS) Services National Centers of Excellence [N = 13]	2003	None	Availability of services	Preventive cancer screening and general reproductive services were uniformly available at all centers, although DHHS centers offered extensive reproductive services on-site more frequently, and VA centers more often had on-site mental health care	B
Boitano LT, et al., 2012	Safety [same]	Single ctr	Veterans at a VA hospital [N = 322]	2006–2009	Single ctr	Patients at Northwestern Memorial Hospital [N = 269]	2006–2009	Vascular surgery	Postoperative outcomes (morbidity and mortality), adjusted	Multivariate analysis showed that hospital setting was not an independent predictor of complications, major adverse events, or death, suggesting no difference in outcomes between the VA and private sector	A/B
Bond CA, et al., 2008	Effectiveness [better]	Nat’l	VA hospitals [N = 84]	2006	Nat’l	Non-VA hospitals [N = 1,041]	2006	None	Clinical pharmacy services offered	8/15 clinical pharmacy services were more common in VA hospitals. In-service education was higher by 25 %, clinical research 154 %, drug protocol management 28 %, drug therapy counseling 80 %, participation on rounds 38 %, and admission drug histories 310 % (P < 0.003)	A
Borzecki AM, et al., 2010	Safety [better]	Nat’l	IQI-related discharge at VA [N = 403,828]	2003–2007	Nat’l	HCUP-NIS IQI-related discharges [sample size not reported]	2003–2007	Multiple	Inpatient quality indicators	Comparing VA and HCUP-NIS risk-adjusted rates from 2003 through 2007, slope estimates for AMI, stroke, hip fracture, pneumonia, and hip replacement mortality rates declined more rapidly in the VA	A
Chakkera HA, et al., 2005	Safety [worse]	Nat’l	VA patients who received pretransplant care in VA or paid by VA [N = 1,646]	1991–2001	Nat’l	Non-VA patients who did not receive pretransplant care in VA or paid for by VA [N = 77,715]	1999–2001	Kidney transplant	Graft and patient survival after kidney transplant, adjusted	Among all recipients, VA users had a 20 % higher risk for graft failure (RR 1.21; 95 % CI 1.12–1.30) and 14 % higher risk of mortality (RR 1.14; 95 % CI 1.07–1.22) compared with non-VA users	A
Choi JC, et al., 2009	Safety [same]	Single ctr	VA patients [N = 682]	2002–2006	Nat’l	Non-veteran non-VA patients [N = 34,572]	2004	Cardio	In-hospital mortality rate after CABG, unadjusted	No significant difference in in-hospital mortality rate for male patients (1.6 % versus 2.4 %; P = 0.20)	B
Fihn SD, et al., 2009	Safety [same]	Nat’l	VA patients [N = 27,494]	2000–2005	Nat’l	Medicare patients in private sector hospitals [N = 789,400]	2000–2005	IHD	Adjusted 30-day mortality following AMI	Adjusted relative odds of death were not significantly different for VA or private sector hospitals (OR 1.02; 95 % CI 0.96–1.08)	A
French DD, et al., 2012a	Safety [worse]	Nat’l	Patients in VA database [N = 20,191]	2007	Nat’l	Patients in Medicare database [N = 137,726]	2007	Cataract surgery	90-Day rates of cataract procedure complications	Adjusted results resulted in significant odds increases of corrective procedures for routine cataract surgeries performed in VA (OR 1.70; 95 % CI 1.58–1.82) and for complex cataract surgery (OR 2.68; 95 % CI 2.24–3.20)	A
French DD, et al., 2012b	Safety [better]	Nat’l	Patients in VA database [N = 19,721]	2007	Nat’l	Patients in Medicare database [N = 129,302]	2007	Cataract surgery	All-cause mortality following cataract surgery, adjusted	Mortality risk did not differ significantly between the two cohorts at time points within 6 months following cataract surgery. Medicare patients experienced 13–17 % excess mortality in both routine and complex cases at 270 and 365 days	A
Gonzales R, et al., 2006	Effectiveness [worse]	Mult ctrs	Patients at VA ED [N = 1,125]	2003–2004	Mult ctrs	Patients at matched non-VA EDs [N = 1,138]	2003–2004	Pulmonary	Antibiotic prescribing patterns in acute respiratory infections	Clinical setting (VA vs. non-VA) was not independently associated with antibiotic prescribing. In four cities, VA and non-VA EDs were very similar, but far apart in four others. In discordant cases, the VA ED sites showed higher adjusted rates of antibiotic treatment	B
Hynes DM, et al., 2006	Effectiveness [better]	Mult ctrs	VA hemodialysis patients [N = 177]	2001–2003	Mult ctrs	Private sector hemodialysis patients [N = 131]	2001–2003	ESRD	Compliance with erythropoietin guidelines, adjusted	Erythropoietin was administered predominantly subcutaneously for 52 % of patients in VA versus 15 % in private-sector facilities (P < 0.001)	B
Keating NL, et al., 2010	Effectiveness [better]	Nat’l	Male VA patients [N = 2,913]	2001–2005	Nat’l	Propensity-matched male Medicare patients [N = 2,913]	2001–2005	Lung cancer, colorectal cancer	End-of-life care indicators, propensity adjusted	Men treated at VA were less likely than those in private sector to receive chemo within 14 days of death (4.6 % vs. 7.5 %; P < 0.001) or to be admitted to an ICU within 30 days of death (12.5 vs. 19.7; P < 0.001), and similarly likely to have >1 ER visit within 30 days of death (13.1 vs. 14.7; P = 0.09)	A
Keating NL, et al., 2011	Effectiveness [better]	Nat’l	VA patients [N = 50,573]	2001–2005	Nat’l	FFS Medicare patients [N = 143,504]	2001–2005	Colorectal, lung, or prostate cancer; lymphoma; or multiple myeloma	High-quality cancer care process measures, propensity adjusted	For colon cancer, VA had higher rates of diagnosis at earlier stage (P < 0.001) and resection (P = 0.01), but similar rates of adjuvant chemo. For rectal cancer, VA had higher rates of diagnosis at earlier stage (P = 0.007), but similar rates of resection or adjuvant chemo/radiation. Lung cancer outcomes were non-significant. Outcomes were mixed for prostate and hematologic cancer	A/B
Landrum MB, et al., 2012	Safety [better]	Nat’l	Males over 65 diagnosed/treated for certain cancers at VA [N = 26,718]	2001–2005	Nat’l	Males over 65 with certain cancers using Medicare FFS [N = 118,195]	2001–2005	Colorectal, lung, or prostate cancer; lymphoma; or multiple myeloma	Time to all-cause death and cancer-attributable death), propensity adjusted	VA patients had higher survival rates of colon cancer (HR 0.87; 95 % CI 0.82–0.93) and non–small-cell lung cancer (HR 0.91; 95 % CI 0.88–0.95) and similar survival rates of rectal cancer, small-cell lung cancer, diffuse large–B-cell lymphoma, and multiple myeloma compared to similar FFS Medicare patients	A
Liu CF, et al., 2008	Effectiveness [worse]	Nat’l	Veterans who are primary care users at 76 VA-staffed community clinics [N = 17,060]	2000–2001	Nat’l	Veterans who are primary care users at 32 non-VA contract community clinics receiving capitation [N = 6,842]	2000–2001	DM, pulmonary	Outpatient service utilization and receipt of primary care services, adjusted	Odds ratios for the proportion of contract and VA-staffed clinic diabetic patients receiving a retinal exam were (OR 0.72; 95 % CI 0.55-0.93) and COPD patients receiving a flu shot (OR 0.73; 95 % CI 0.55–0.99)	A
Lu H, et al., 2010	Effectiveness [better]	Mult ctrs	Veterans who died in a VA facility [N = 520]	NR	Mult ctrs	Veterans who died in a non-VA facility [N = 89]	NR	None	Perceptions of the care and services during last month of life	Patients who died in the VA were more likely to have had a palliative care consult (67 % vs. 21 %; P < 0.001) and to have died in a palliative care or hospice unit (47 % vs. 16 %; P < 0.001), but they were more likely to die in ICU (26 % vs. 13 %; P = 0.01) and less likely to die in a nursing home (0 % vs. 26 %; P < 0.001)	B
Lynch CP, et al., 2010	Effectiveness [better]	Nat’l	Veterans who had used VA facilities in the last year [N = 1,342]	2003	Nat’l	Veterans who had not used VA facilities in the last year [N = 3,159]	2003	DM, Preventive care	Quality of diabetes care, adjusted	VA users were more than twice as likely to have received foot exams by a provider (OR 2.59; 95 % CI 1.76–3.83), pneumonia shots (OR 2.30; 95 % CI 1.68–3.14), and flu shots (OR 2.05; 95 % CI 1.44–2.92). VA users had 60–70 % greater likelihood of A1c tests, and 2+ provider visits in the last 12 months	A
Nelson KH, et al., 2011	Effectiveness [same]	Mult ctrs	VA patients [N = 150]	2009	Mult ctrs	Patients at an academic medical center [N = 150]	2009	IHD	Appropriate use of stress/rest myocardial perfusion imaging (MPI), unadjusted	The majority of the studies were ordered for appropriate indication (67.3 % in VA vs. 74 % in academic practice) (P = 0.272)	B
Parikh DS, et al., 2011	Effectiveness [same]	Mult VISNs	VA patients [N = 378]	2005–2006	Mult VISNs	Medicare patients [N = 25,534]	2005–2006	ESRD	Method of vascular access for first outpatient dialysis	Adjusting for patient demographics and comorbidities only, VA patients had greater likelihood of arteriovenous fistulas (AVF) use (preferred by guidelines) (OR = 1.70; 95 % CI 1.31–2.20), but accounting for pre end-stage renal disease care removed the significant difference (OR 1.28; 95 % CI 0.98–1.66)	A/B
Richardson KK, et al., 2013	Safety[worse]	Nat’l	VA users [N = 9,308]	2002–2008	Nat’l	Veterans at non-VA hospitals [N = 1,881]	2002–2008	Orthopedic surgery	Time between admission and repair, 1-year mortality	Likelihood of death within 1 year of admission was 21 % less for veterans admitted to non-VA hospitals (RR = 0.79; 95 % CI 0.71–0.88; P < 0.001)	A
Rivard PE, et al., 2010	Safety [worse]	Nat’l	VA hospitals [N = 116]	2003–2004	Nat’l	Non-VA hospitals (HCUP-NIS) [N = 992]	2003–2003	None	Patient Safety Indicators (PSIs), adjusted	VA had higher composite PSI [VA: 1.118 95 % CI (1.071–1.164), HCUP-NIS: 0.987 95 % CI (0.977–0.997)]. VA also had higher foreign body left in during procedure, iatrogenic pneumothorax, postoperative hemorrhage, postoperative wound dehiscence. HCUP-NIS had more postoperative sepsis than VA, with a small overlap of confidence intervals. No significant differences for other PSIs evaluated	A
Rosen AK, et al., 2005	Safety [mixed]	Nat’l	VA users [N = 281,423]	2000–2001	Nat’l	Non-VA (HCUP-NIS and Medicare) [sample sizes not reported]	2000–2000	None	PSIs, risk-adjusted	VA-risk-adjusted rates are lower than HCUP-NIS and Medicare event rates for decubitus ulcer, infection due to medical care, postoperative respiratory failure, and postoperative sepsis. VA PSI event rates were higher than HCUP-NIS and Medicare event rates for postoperative physiologic and metabolic derangements and technical difficulty with procedure. VA PSI event rates were higher than HCUP-NIS event rates, but lower than Medicare event rates, for the remaining indicators	A
Selim AJ, et al., 2010	Safety [better]	Nat’l	VA users [N = 35,876]	1999–2003	Nat’l	Medicare Advantage cohort [N = 71,424]	1999–2003	None	2-Year mortality, probability same or better physical or mental health at 2 years; adjusted	2-Year mortality rates were 11.8 % and 9.9 % for the Medicare and VA, respectively; probability of being alive with the same or better physical health at 2 years higher VA compared to Medicare; same or better mental health at 2 years was also significantly higher at VA than in Medicare. Propensity score matched analyses had comparable results	A
Tarlov E, et al., 2012	Safety [same]	Nat’l	VA users [N = 1,465]	1999–2001	Nat’l	Medicare FFS users [N = 1,042]	1999–2001	Colon cancer	Overall and event-free 36-month survival, adjusted	Overall survival hazard ratios were similar at stages 1–3. Event-free survival hazard ratios were also not significantly different	A
Trivedi AN, et al., 2011	Effectiveness [better]	Nat’l	VA patients [N = 293,554]	2000–2007	Nat’l	Medicare Advantage (MA) enrollees [N = 5,768,573]	2000–2007	DM, IHD, HTN, Preventive care	Healthcare Effectiveness Data and Information Set (HEDIS), External Peer Review Program (EPRP) indicators	The VA had higher aggregate performance than MA for 10 of 11 measures in the initial year of assessment, and all 12 measures in the final year. Adjusted differences range from 4.3 percentage points (95 % CI 3.2–5.4) for cholesterol testing in CHD to 30.8 percentage points (95 % CI 28.1–33.5) for colorectal cancer screening	A/B
Vaughan-Sarrazin MS, et al., 2007	Safety [worse]	Nat’l	Male VA users [N = 139,331]	1996–2002	Nat’l	Male Medicare patients [N = 1,212,729]	1996–2002	Cardio	30-, 90-, 365-Day mortality, adjusted	Adjusted mortality after CABG was higher (P < 0.001) in VA users compared with nonusers at 30, 90, and 365 days (ORs 1.07, 1.07, 1.09). For PCI, adjusted mortality at 30 and 90 days was similar (NS), but higher for VA users at 365 days (OR 1.09)	A
Wang A, et al., 2005	Effectiveness [better]	Nat’l	VA users [N = 3,391]	2000	Nat’l	Non-VA users [N = 178,735]	2000	Overweight/obesity	Association between obesity and weight advice, adjusted	Obese VA users were twice as likely to have received professional advice to lose weight (OR 2.06; 95 % CI 1.64 to 2.59) and as likely to have received professional advice to maintain weight (OR 1.72; 95 % CI 0.75 to 3.97)	B
Wang V, et al., 2013	Safety [same]	Mult VISNs	VA dialysis users [N = 381]	2007–2008	Mult VISNs	Veterans who received outpatient dialysis exclusively in VA-outsourced settings [N = 659]	2007–2008	ESRD	Adjusted all-cause hospitalization and mortality at 1 year, adjusted	There was no difference in mortality outcomes among veterans who received outpatient dialysis exclusively in VA-outsourced compared to VA dialysis users (OR 0.80; 95 % CI 0.48–1.3)	A
Weeks WB, et al., 2009	Effectiveness [better]	Nat’l	VA patients [sample size not reported]	2005–2006	Nat’l	Medicare FFS patients [sample size not reported]	2005–2006	Multiple	Outpatient and inpatient quality measures	The VA outperformed Medicare fee-for-service performance in one measure of mammography and two measures of outpatient diabetic management. In 2005 and 2006, the VA performed better than hospitals contributing to Hospital Compare	B
West A, et al., 2006	Effectiveness [better]	Nat’l	VA users [N = 1,928]	2000	Nat’l	Veteran VA non-users and non-veterans [N = 12,461]	2000	None	Routine checkup within last 2 years	VA patients reported higher rates of seeing a doctor for a routine checkup within the past 2 years (91.6 %; P < 0.001)	B
Articles from previous review (with additional information abstracted as necessary)
Bansal D, 2005	Effectiveness [better]	Single ctr	VA patients [N = 117]	2002	Nat’l	Registry not further described [sample size not reported]	2002	IHD	Use of certain medications among patients with MI	Use of aspirin, beta-blockers, ACE inhibitors, heparin, GP2b/3a inhibitors was higher in the Little Rock VA compared to the rest of Arkansas and the entire US	B
Barnett MJ, et al.,2006	Effectiveness [better]	Nat’l	VA patients [N = 123,633]	2002–2003	Nat’l	Medicare HMO patients [N = 157,517]	2000–2001	Other safety	Use of potentially inappropriate medications	Compared with private sector patients, VA patients were less likely to receive any inappropriate medication (21 % vs. 29 %; P < 0.001), and in each classification: always avoid (2 % vs. 5 %; P < 0.001), rarely appropriate (8 % vs. 13 %; p < 0.001), and some indications (15 % vs. 17 %; P < 0.001)	B
Berlowitz DR, et al., 2005	Safety [mixed]	One VISN	Veterans in VA nursing homes [N = 3,802]	1997–1999	Lrg geo area	Veterans in contract nursing homes [N = 961]	1997–1999	Other medical/ nonsurgical condition	Risk-adjusted rates of pressure ulcer, functional and behavioral decline, mortality	Veterans in VA nursing homes were significantly (P < 0.05) less likely to develop a pressure ulcer (OR 0.63) but more likely to experience functional decline (OR 1.6) than veterans in community nursing homes. Veterans in VA nursing homes were similar likely to die experience behavioral decline	A
Bilimoria KY, et al., 2007	Safety [same]	Nat’l	VA patients [N = 513]	1985–2004	Nat’l	Academic hospital patients [N = 12756]	1985–2004	General surgical, surgical oncology	60-Day and 3-year mortality	Unadjusted and adjusted mortality rates at 60 days and 3 years were comparable between VA, academic and community hospital settings for resection of stage I and II pancreatic cancer	B
						Community hospital patients [N = 18,299]
Campling BG, et al., 2005	Safety [worse]	One VISN	Male VA patients [N = 862]	1995–1999	Lrg geo	Male non-VA patients [N = 27,936]	1995–1999	Cancer	Survival following diagnosis of lung cancer	The median survival was 6.3 months for VA patients compared with 7.9 months for patients in the rest of the state, and the 5-year overall survival rate was 12 % for VA patients compared with 15 % for patients in the rest of the state. The hazard ratio for VA patients compared with non-VA patients is 1.22 (P < 0.001)	B
Chi RC, et al.,2006	Effectiveness [better]	Nat’l	VA users [N = 3,265]	2003	Nat’l	Veteran non-VA users [N = 10,677]	2003	Preventive care	Influenza and pneumococcal vaccination	Among veterans, influenza and vaccination rates higher for VA users compared to non-users. For veterans, VA care was independently associated with influenza vaccination (adjusted OR 1.8; 95 % CI 1.5–2.2) and pneumococcal vaccination (adjusted OR 2.4; 95 % CI 2.0–2.9)	A
						Non-veterans [N = 40,331]
Fink AS, et al., 2007	Safety [better]	Nat’l	Female VA patients [N = 5,157]	2001–2004	Mult. ctrs	Female private sector patients [N = 27,467]	2001–2004	General surgical	30-Day postoperative morbidity and mortality	Risk-adjusted mortality rates are comparable between PS and VA patients, although setting of care did not enter the mortality regression model. Risk-adjusted morbidity was higher in the PS compared with the VA OR 0.8 (95 % CI 0.71–0.90)	B
Gill JS, et al., 2007	Effectiveness [worse]	Nat’l	VA patients [N = 7,395]	1995–2004	Nat’l	Privately insured patients [N = 144,651]	1995–2004	Other surgical	Time to treatment	Both VA-insured and Medicare/Medicaid-insured patients were 35 % less likely to receive transplants than privately insured patients (HR 0.65; 95 % CI 0.60–0.70; P < 0.001). VA patients were less likely to be placed on the wait-list (HR 0.71; 95 % CI 0.67–0.76), but even those listed received transplants less frequently than the privately insured (HR 0.89; 95 % CI 0.82–0.96)	A
						Medicare/Medicaid patients [N = 357,345]
Glasgow RE, et al.,2007	Safety [worse]	Nat’l	VA patients [N = 377]	2001–2004	Mult. ctrs	Private sector patients [N = 692]	2001–2004	Other surgical	Postoperative outcomes	Adjusting for case mix differences, postoperative morbidity and mortality rates for pancreatectomy were higher in the VA compared with the PS (OR 1.58; 95 % CI 1.08–2.31 and 2.53 95 % CI 1.02–6.29 respectively)	A/B
Hall BL, et al., 2007	Safety [same]	Nat’l	VA patients [N = 2,814]	2001–2004	Mult. ctrs	Private sector patients [N = 4,268]	2001–2004	General surgical, head and neck	30-Day morbidity and mortality;	Overall 30-day morbidity and mortality do not differ significantly in the VA vs. PS in risk-adjusted model. Mortality event rate is too low to accurately evaluate, odds ratio for morbidity associated with VA care is 1.25 (95 % CI 0.87–1.78)	B
									Adverse event rates, LOS
Henderson WG, et al., 2007	Safety [worse]	Nat’l	Male VA patients [N = 9,409,818]	2001–2004	Mult. ctrs	Male private sector patients [N = 18,399]	2001–2004	General surgical	30-Day postoperative morbidity and mortality	After risk adjustment for patient comorbidities and severity of illness, the odds of mortality at 30 days were higher in the VA compared with the PS (OR 1.23; 95 % CI 1.08–1.41). There was no significant difference in morbidity at 30 days among the sites	A/B
Hutter MM, et al., 2007	Safety [better]	Nat’l	Male VA patients [N = 30,058]	2001–2004	Mult. ctrs	Male private sector patients [N = 5,174]	2001–2004	Vascular	30-Day postoperative morbidity and mortality	Risk-adjusted mortality was comparable among the two groups. Accounting for comorbidities and severity of illness, postoperative morbidity rates were lower in the VA population, OR 0.84 (95 % CI 0.78–0.92)	A/B
Jha AK, et al., 2007	Effectiveness [better]	Nat’l	VHA patients [N = 33,504-74,250]	1995–2003	Nat’l	Representative community sample [sample size not reported]	1995–2003	Preventive care	Vaccination rates	Trends in influenza and pneumonia vaccination rates were significantly different in the VA compared to in the BRFSS (P < 0.001). Pneumonia hospitalization decreased by 50 % among elderly VA enrollees but increased among Medicare enrollees by 15 % (P < 0.001)	B
Johnson RG et al., 2007	Safety [better]	Nat’l	VA patients [N = 458]	2001–2004	Mult. ctrs	Private sector patients [N = 3,535]	2001–2004	Vascular	30-Day postoperative morbidity and mortality	After risk adjustment, no significant difference in 30-day mortality rates among VA and PS female vascular patients. After adjusting for severity of illness, 30-day complication/morbidity rates were significantly lower in VA compared with PS (OR 0.60; 95 % CI 0.44–0.81)	B
Keyhani S, et al., 2007	Effectiveness [better]	Nat’l	Veterans receiving VHA/VHA and FFS Medicare/ VHA and Medicare HMOs [N = 171/1,009/145]	2000–2003	Nat’l	Veterans receiving FFS Medicare/Medicare HMO [N = 3,552/576]	2000–2003	Preventive care	Influenza a and pneumonia vaccination, cholesterol screening	Veterans receiving care through VA reported 10 % greater use of influenza vaccination (P < 0.05), 14 % greater use of pneumococcal vaccination (P < 0.01), and a non-significant 6 % greater use of serum cholesterol screening (P = 0.1) than veterans receiving care through Medicare HMOs	B
Krein SL, et al., 2007	Safety [better]	Nat’l	VA hospitals [N = 119]	2005	Nat’l	Non-VA hospitals [N = 421]	2005	Other medical/ nonsurgical condition	Regular use of specific prevention modalities and a composite measure	Adjusted findings revealed that VA hospitals were significantly more likely to use chlorhexidine gluconate (OR 4.8; 95 % CI 1.6–15.0) and/or to use a composite approach (OR 2.1; 95 % CI 1.0–4.2) as compared with non-VA hospitals	B
Lancaster RT, et al., 2007	Safety [same]	Nat’l	Procedures at VA hospitals [N = 237]	2001–2004	Mult. ctrs	Procedures at university hospitals [N = 783]	2001–2004	General surgical	Postoperative 30-day morbidity and mortality LOS, need for re-operation, postoperative events	Risk-adjusted outcomes suggest that 30-day postoperative morbidity and mortality rates in the VA compared with the PS for hepatic resections do not vary significantly. After risk adjustment, morbidity rates and mortality were comparable in VA and PS. Comparing morbidity of VA w/ PS OR was 0.94 (95 % CI 0.62–1.42) and mortality OR was 1.62 (95 % CI 0.61–4.32)	A/B
Lautz DB, et al., 2007	Safety [worse]	Nat’l	VA patients [N = 374]	2001–2004	Mult. ctrs	Private sector patients [N = 2,064]	2001–2004	Other surgical	30-Day postoperative outcomes: morbidity and mortality, LOS	No significant difference in postoperative morbidity or mortality among women in the VA versus non-VA settings (16.07 vs. 12.02 %; P = 0.21 and 0.89 vs. 0.42 %; P = 0.47). Unadjusted and adjusted morbidity rates were higher among men treated at the VA versus non-VA (OR 2.77; 95 % CI 1.78–4.31 unadjusted and OR 2.29; 95 % CI 1.28–4.10 adjusted)	A/B
Nelson KM, et al., 2005	Effectiveness [better]	Nat’l	Veterans with some VA care [N = 254]	2000	Nat’l	Adults with other insurance types [N = 10,632]	2000	DM	Diabetes self- management and preventive care practices	Persons who received care through the VA were more likely to report taking a diabetes education class and HbA1c testing than those covered by private insurance	B
			Veterans with all VA care [N = 281]
Neumayer L, et al., 2007	Safety [same]	Nat’l	VA patients [N = 644]	2001–2004	Mult. ctrs	Private sector patients [N = 3,179]	2001–2004	General surgical	30-Day postoperative morbidity and mortality, LOS	After adjusting for comorbidities and preoperative factors, there was no significant difference in 30-day morbidity or mortality in female patients at the VA compared with the PS (OR 1.40; 95 % CI 0.89–2.20)	B
Rehman SU, et al., 2005	Effectiveness [better]	One VISN	VA patients [N = 12,366]	2001–2003	Lrg geo	Non-VA patients [N = 7,734]	2001–2003	HTN	Control of blood pressure below 140/90 mmHg	Blood pressure control was comparable among white hypertensive men at VA (55.6 %) and non-VA (54.2 %) settings (P = 0.12). In contrast, BP control was higher among African American hypertensive men at VA (49.4 %) compared with non-VA (44.0 %) settings (P < 0.01), even after risk adjustment	A
Ross JS, et al., 2008	Effectiveness [better]	Nat’l	Adults receiving care at VAMCs [N = 10,007]	2000, 2004	Nat’l	Adults receiving care elsewhere [N = 393,873]	2000, 2004	DM, IHD, HTN, preventive care	Self-reported use of 17 recommended health care services	VA care was associated with greater use of recommended services in both years of study (6/17 services more used in 2000, 12/17 more used in 2004)	B
Selim AJ, et al., 2007	Safety [better]	Nat’l	VHA patients [N = 16,725 at baseline and 12,177 at follow-up]	1998–2000	Nat’l	Medicare Advantage Program patients [N = 62,614 at baseline and 26,225 at follow-up]	1998–2000	None	Risk-adjusted 2-year mortality, change in physical and mental health status	Lower risk-adjusted mortality in the VA compared to MA (2-year mortality 7.6 % in VA vs. 9.2 % in MA); VA patients had a slightly higher probability than Medicare patients of being alive with the same or better mental health (71.8 % vs. 70.1 %; P = 0.002)	B
Selim AJ, et al., 2006	Safety [better]	Nat’l	VHA patients [N = 420514]	1999–2004	Nat’l	Medicare Advantage Program [N = 584294]	1998–2004	Other medical/nonsurgical condition	Risk-adjusted mortality	After adjusting for case mix, the HR for mortality in Medicare was significantly higher than that in VA (HR, 1.40; 95 % CI = 1.38–1.43)	B
Selim, AJ, et al., 2009	Safety [better]	Nat’l	Medicaid-eligible VHA patients [N = 2,361]	1999–2000	Nat’l	Medicaid-eligible Medicare Advantage patients [N = 1,912]	1999–2000	Other medical/ nonsurgical condition	3-Year risk-adjusted mortality rate	The adjusted HR of mortality in the MA dual enrollees was significantly higher than in VA dual enrollees (HR 1.26; 95 % CI 1.04–1.52)	B
Turrentine FE, et al., 2007	Safety [same]	Nat’l	VA patients [N = 178]	2001–2004	Mult. ctrs	Private sector patients [N = 371]	2001–2004	Other surgical	30-Day morbidity and mortality	Mortality event rate was too low for adjustment. Adjusting for comorbidities, the 30-day postoperative morbidity ratio in VA versus the PS was no longer significant (adjusted OR1.33, 95 % CI 0.49–3.61 compared with unadjusted OR 2.75; 95 % CI: 1.55–4.91)	B
Weeks WB, et al., 2008	Safety [mixed]	One VISN	Male VA enrollees receiving care within VA [N = 50,429]	1998-2000	Lrg geo	Male VA enrollees receiving care outside VA [N = 74,017]	1998-2000	Patient Safety Indicators	Risk-adjusted rates of non- obstetric PSIs	Rates similar for 9 of 15 PSIs, ulcer, sepsis, iatrogenic infection, postoperative respiratory failure, postoperative metabolic derangement lower in VA, mortality higher in VA for low-risk DRGs	B
Weiss JS, et al., 2006	Safety [same]	One VISN	VA patients [N = 140]	1997–2002	Lrg geo	Private sector patients [N = 6,949]	1997–2002	Vascular	Perioperative mortality, stroke, and cardiac complications	After risk adjustment, having surgery at the VA was not a significant predictor of death (OR 2.98; 95 % CI 0.51-17.6), stroke (OR 0.95; 95 % CI 0.3–3.4), or cardiac complications (OR 1.07; 95 % CI 0.37–3.10)	B

Table abbreviations: BRFSS Behavioral Risk Factor Surveillance System, CI confidence interval, COPD chronic obstructive pulmonary disease, CABG coronary artery bypass grafting, DM diabetes mellitus, DRG diagnosis-related group, ED emergency department, FFS fee for service, ESRD end stage renal disease, HR hazard ratio, HCUP-NIS Healthcare Cost and Utilization Project Nationwide Inpatient Sample, HTN hypertension, IQI inpatient quality indicator, ICU intensive care unit, IHD ischemic heart disease, MA Medicare Advantage, MI myocardial infarction, NR not reported, OR odds ratio, PSI patient safety indicator, PS private sector, RR relative risk, TIA transient ischemic attack, VA Veterans Affairs, VISN Veterans Integrated Service Network

*We assessed each study in the review according to the statistically significant differences in performance on quality of care measures for VA care relative to a non-VA comparison group. If the VA quality of care was shown to be better than quality for non-VA care, the study was classified as “VA better.” If VA quality of care was better in some instances and the same in other instances compared to non-VA care in the same study, the study was also classified as “VA better.” If multiple quality measures were reported in the study and VA care was better than non-VA on some and worse on others, the study was classified as “mixed.” If the quality of care in the VA and non-VA did not differ, the study was classified as “same.” If the VA quality of care was shown to be worse than non-VA, the study was classified as “VA worse” (as were studies where the quality of care was worse in some instances and the same in other instances)

†The Primary Findings text has been drawn directly from the reviewed articles and in some cases may be similar or identical to the article’s text

‡Each article was given an overall assessment of quality shown in the Final Grade column. This assessment was based on the following criteria: time frames, samples (both VA and non-VA), quality measurements, outcomes, importance of measures, and statistical methods. Each of these factors was assigned a grade (A, B, or C) based on the data abstraction grading guidelines developed in the original systematic review. The overall assessment was predicated on the global assessment of the article considering the individual components, but was not an average. Thus, an article that had, for example, a critical flaw in methodology would be a “C,” even if other issues were satisfactory. Articles that received an overall grade of “C” were rejected from the review

Safety

Safety measures focused on avoiding illness/injury to patients resulting from medical care, such as complications following surgical procedures. Studies of morbidity and mortality were also included in this category, as were studies about adherence to safety guidelines or best practices. In 22 of 34 articles on safety, VA generally performed as well (11 studies) or better (11 studies) than non-VA settings. VA fared worse in nine studies and had mixed performance in three studies.

Mortality

VA facilities generally performed comparably or favorably relative to other settings in terms of mortality. Mortality rates associated with specific conditions^{16 – 19} or following surgery^{20 – 26} were often similar for VA patients compared with non-veterans, and occasionally lower.¹⁷ However, some studies found higher surgical mortality among VA patients compared with non-veterans.^{19 , 22 , 26 – 29} Lower mortality rates were observed after cataract surgeries for VA patients compared with veterans who are Medicare fee-for-service beneficiaries.³⁰ Mortality among veterans residing in VA nursing homes and community nursing homes was similar.³¹ Veterans undergoing dialysis treatment in VA and non-VA settings experienced similar mortality.³² Adjusted mortality was lower among male VA patients compared with male Medicare Advantage beneficiaries over 65 years old;^{33 – 36} results were similar for females.³⁴ Mortality within 1 year of admission after hip fracture was 21 % lower among veterans admitted to non-VA hospitals compared with VA patients.³⁷ Mortality rates have declined more quickly in VA over time than in non-VA settings.³⁸

Morbidity

VA facilities had mixed results with respect to most studies of morbidity. Postoperative morbidity was lower for VA patients compared with non-veterans in some surgeries,^{22 , 25 , 39} while for other surgeries it was similar.^{40 – 44} In a study comparing quality of care in nursing homes, veterans in VA nursing homes were less likely to develop a pressure ulcer than veterans in community nursing homes.³¹ However, other studies found that morbidity among VA patients is worse after pancreatectomy²⁸ and for male patients only after gastric bypass, compared to patients in private sector hospitals (unadjusted).⁴¹

Complications

VA facilities had mixed results with respect to complications after surgery. Surgical complication rates were similar among VA patients and non-veterans.^{21 , 23 , 29} Higher complication rates were observed for VA patients undergoing cataract surgeries than veterans who were Medicare fee-for-service beneficiaries.⁴⁵ Among all kidney transplant recipients, VA patients had higher graft failure risk than non-veterans.²⁷

Other Safety Measures

VA hospitals were more likely to follow best practices for central venous catheter bloodstream infection prevention compared with non-VA hospitals.⁴⁶ However, performance on the Agency for Healthcare Research and Quality’s (AHRQ) patient safety indicators was found to be a mix of higher, lower, and similar rates at VA hospitals compared with non-VA hospitals.^{47 – 49}

Effectiveness

Seventeen studies showed better performance in VA facilities, while three had similar performance, one was mixed, and three were worse than non-VA settings.

Outpatient Care

Outpatient care was generally strong in VA facilities. VA patients received higher quality care than non-veterans for one measure of preventive care mammography (85–90 % vs. 64–77 %) and two measures of outpatient diabetic management (annual HbA1c test: 94–96 % vs. 70–81 %; annual eye examination: 67–85 % vs. 68–74 %) (no statistical tests).⁵⁰ VA patients also received more effective care than non-veterans based on 10 of 11 quality measures in the first study year and all 12 quality measures in the second study year assessing diabetes, cardiovascular, and cancer screening care, with rate differences ranging from 4.3 percentage points (95 % confidence interval [CI] 3.2–5.4) for cholesterol testing in coronary heart disease to 30.8 (95 % CI 28.1–33.5) for colorectal cancer screening.⁵¹ Receipt of diabetes education and annual HbA1c tests was higher among VA patients compared with veterans in non-VA care.⁵² VA patients were more likely than veterans receiving care outside VA to receive recommended diabetes care, including being twice as likely to have a foot examination and 60–70 % more likely to have an eye examination, two or more A1c tests, and two or more providers visits,⁵³ a routine checkup within 2 years (91.6 % among VA patients compared to 80.6 % overall; P < 0.001),⁵⁴ and influenza and pneumonia vaccinations (rates increased 10–240 %),^{53 , 55 , 56} but similar rates of cholesterol screening.⁵⁵ Blood pressure control was higher among male African-American VA patients than male African-American non-VA patients (49.4 % vs. 44.0 %, P < 0.01) though similar among Caucasians.⁵⁷ VA patients were more likely than non-VA patients to receive recommended ambulatory preventive and disease management services,⁵⁸ including influenza and pneumococcal vaccination.^{58 , 59} These studies used a national sample comparison group of non-VA patients, which may have included both veterans and non-veterans. Obese VA patients were more than twice as likely to have received advice to lose weight as veterans receiving non-VA care and non-veterans and equally likely to have received advice to maintain weight.⁶⁰ However, in a study of veterans receiving primary care at VA-staffed versus contract community clinics, veterans with diabetes at VA-staffed clinics were less likely to receive a retinal examination (odds ratio [OR] 0.72, 95 % CI 0.55–0.93) and veterans with chronic obstructive pulmonary disease at VA-staffed clinics were less likely to receive a flu shot (OR 0.73, 95 % CI 0.55–0.99).⁶¹

Non-Ambulatory Care

In non-ambulatory settings, VA care was generally of similar quality to care provided in non-VA facilities. Compared with non-VA patients from the Medicare cancer patient database, VA patients had earlier diagnosis of colon and rectal cancers, higher rates for three quality measures, similar rates for nine, and lower rates for one.⁶² Comparison of an academic practice and a VA hospital found that appropriate use of stress/rest myocardial perfusion imaging studies did not differ between settings.⁶³ Rates of hemodialysis via arteriovenous fistulas (which are preferred by guidelines over other methods) were not different between VA patients and Medicare patients after accounting for pre-end-stage renal disease care.⁶⁴ VA patients received higher quality care than non-veterans for nine out of ten measures of inpatient care and performed similarly on one measure.⁵⁰

Medication Management

Medication management was examined in a number of settings, including outpatient, emergency department, and inpatient. Elderly VA patients were less likely to receive inappropriate medication than were patients in Medicare HMOs,⁶⁵ and VA patients with acute myocardial infarction were more likely to receive appropriate medications than were non-VA patients.⁶⁶ Observed compliance by providers with erythropoietin administration guidelines was higher at VA than in the private sector.⁶⁷ Antibiotic prescribing practices were generally similar between seven VA and seven non-VA emergency departments; however, in the three cities in which prescription rates were not comparable between VA and non-VA sites, VA sites had much higher rates of antibiotic prescriptions.⁶⁸

Availability of Services

The structure of women’s health care differed at eight VA women’s health centers and 13 Department of Health and Human Services Centers of Excellence. Preventive cancer screening and general reproductive services were available at all centers, while VA centers were less likely to offer extensive reproductive services but more likely to offer mental health care.⁶⁹ Eight of 15 clinical pharmacy services were more commonly provided in VA hospitals than non-VA hospitals [in-service education was 25 % higher (P = 0.003), clinical research 154 % higher (P < 0.0001), drug protocol management 28 % higher (P < 0.0001), drug therapy counseling 80 % higher (P < 0.0001), participation on rounds 38 % higher (P = 0.001), and admission drug histories 310 % higher (P < 0.0001)].⁷⁰ VA patients and Medicare/Medicaid-insured patients were less likely to receive kidney transplants than were patients with private insurance.⁷¹

End-of-Life Care

VA tended to avoid inappropriate care and utilize palliative care at end of life. Increasing use of chemotherapy at the end of life is associated with higher rates of in-hospital deaths and later admission to hospice, which are linked to lower quality end-of-life care.^{72 – 75} When comparing male VA patients and Medicare patients with lung and colorectal cancer, VA patients were less likely to receive chemotherapy within 14 days of death or to be admitted to an ICU within 30 days of death, and they were similarly likely to have more than one emergency room visit within 30 days of death.⁷⁶ Among veterans who died in VA facilities, palliative care consults (67 % vs. 21 %, P < 0.001) and death in a dedicated palliative care, hospice unit, or intensive care unit were more common (47 % vs. 16 %, P < 0.001), and death in a nursing home was less common (10 % vs. 26 %, P < 0.001) than among veterans who died in non-VA facilities (all unadjusted results).⁷⁷

Timeliness, Equity, Efficiency, and Patient-Centeredness

Since fewer than ten studies were found that related to each of the dimensions of timeliness, equity, efficiency, and patient-centeredness, we do not discuss the results of these studies in detail (results and evidence tables in Online Appendix). The single study that addressed timeliness of care showed worse performance among veterans in VA facilities relative to non-VA facilities for time between hospital admission for hip fracture and surgical repair.³⁷ Four studies examined equity, with one study showing better performance on several chronic disease and preventive measures,⁵¹ one study showing worse performance in VA facilities for emergency room visits among cancer patients in the last month of life,⁷⁶ and two studies showing similar equity in 30-day post-admission mortality⁷⁸ and graft failure.²⁷ Nine articles compared efficiency using utilization, with six studies showing worse performance in VA facilities on outcomes such as average length of stay and service utilization for patients on dialysis,^{32 , 61 , 79 – 82} two showing better performance for visits/admissions⁸³ and generic drug utilization,⁸⁴ and one with mixed results for utilization-related inpatient quality indicators. Five studies examined patient-centeredness; three studies demonstrated better patient and family satisfaction in VA facilities^{77 , 85 , 86} and two demonstrated similar performance in perceptions of racial discrimination and satisfaction in transitional programs for homeless veterans.^{87 , 88}

DISCUSSION

In this systematic review, we identified 69 studies comparing quality of care provided in VA facilities to non-VA care. Studies of safety and effectiveness indicated generally favorable performance by VA facilities compared to non-VA facilities, with 22 of 34 studies of safety and 20 of 24 studies of effectiveness showing the same or better quality of care in VA facilities and the remainder showing mixed (four studies) or worse (12 studies) performance. In terms of safety, VA facilities performed similarly or better in most, but not all, studies comparing morbidity and mortality. Results from studies about complications and patient safety indicators were more mixed. In terms of effectiveness, VA facilities had similar or superior quality to non-VA facilities with respect to preventive, recommended, and end-of-life care, as well as managing medications. Non-ambulatory care studies indicated similar care quality between VA and non-VA settings. However, studies on service availability had mixed results. A comparison of studies included in the previous review, which included studies from 1990 to 2009, and those identified in this review (2005 to 2014) showed that while numerous studies have been published in the last 5 years, no stark trends can be observed in terms of whether older or newer studies demonstrate systematically better or worse performance in VA settings (Table 2).

Table 2.

Comparison of studies included in the prior and current review

	Outcomes				Articles
	Better	Same	Mixed	Worse	Total
Safety
Studies included in prior review	7	6	2	4	19
Studies identified in current review	4	5	1	5	15
Effectiveness
Studies included in prior review	8	0	0	1	9
Studies identified in current review	10	2	2	2	16

In keeping with the findings of the prior review, we find that VA is generally more adherent to recommended care processes than other systems of care. However, better processes did not necessarily achieve better outcomes, as we observed few differences in adjusted mortality. Although easy to obtain, mortality may not be the optimal outcome for comparison for several reasons, some of which were noted in the prior review.^{7 – 9} While nearly every study we included attempted to risk-adjust the veteran and non-veteran populations, some excess baseline mortality risk may still be present, since veterans have comparatively worse health status than the general population.^{89 – 91} Mortality rates are also known not to be sensitive in detecting differences in the quality of health care provided.^{92 – 94} Other outcomes, such as complications specific to a disease or procedure of interest, might be more revealing about meaningful quality differences in VA settings than those included in this review.

Our review builds on a previous review conducted in this area,^{7 – 9} but both updates and expands upon it. Updating the results of the prior review up to 2015 is critically important given the ongoing interest in the quality of care provided at the VA.⁹⁵ In addition to categorizing the studies published since the prior review, we also categorized the studies contained in the prior review to provide a broad picture of quality at the VA over the last decade along the dimensions of health care quality proposed by the IOM.¹¹. This categorization allows us to highlight the gaps in research by dimension, which may help inform the investment of resources for research and improvement by VA stakeholders.

This study is subject to a number of limitations. The search strategy employed was narrowly defined to align with the methods of the previous review, but this may have excluded relevant studies. Most studies were not conducted with perfectly matched comparison groups (i.e., veterans receiving care in VA and non-veterans in non-VA settings). While we used risk-adjusted or otherwise comparable results wherever possible, this may not have accounted for unobservable differences between veterans and non-veterans. Although we included confidence intervals or levels of significance in the evidence tables (Table 1 and Online Appendix), it was not always possible to determine whether lack of a significant difference was due to an underpowered study versus evidence of a lack of a difference. However, while many of the studies used large, nationally representative samples, some of the smaller studies did observe significant differences. We also did not assess whether the magnitudes of differences were clinically significant, as the thresholds for clinical differences for many of the outcomes investigated may be subjective. In deciding which studies to include, we assessed the study design of each (Table 1) and excluded low-quality studies from this review. However, we did not systematically assess each study’s risk of bias beyond that. It is notable that all but 4 of the 69 articles included had at least one VA-affiliated author or were funded directly by VA.

Nonetheless, the available data indicate overall comparable health care quality in VA facilities compared to non-VA facilities with regard to safety and effectiveness. Rates of complications and availability of services had the least favorable results, but these results were mixed rather than consistently poor. The overall number of studies comparing VA and non-VA care was small, and study quality varied. More studies that examine and compare the quality of VA care with respect to timeliness, equity, efficiency, and patient-centeredness are needed to better assess VA facilities’ performance on these quality dimensions.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Compliance with Ethical Standards

Conflict of Interest

The authors declare no conflicts of interest.