In defense of Direct Care: Limiting access to military hospitals could worsen quality and safety

Abstract

Objective

Ongoing health care reforms within the US Military Health System (MHS) are expected to shift >1.9 million MHS beneficiaries from military treatment facilities (MTFs) into local civilian hospitals over the next 1–2 years. The objective of this study was to examine how such health care reforms are likely to affect the quality of MHS care.

Data sources

Adult MHS beneficiaries, aged 18–64 years, treated in MTFs (under a program known as Direct Care) were compared against (1) MHS beneficiaries treated in locally available civilian hospitals (under a program known as Purchased Care) and (2) similarly‐aged adult civilian patients across the United States. MHS beneficiaries in Direct and Purchased Care were identified from fiscal‐year 2016–2018 MHS inpatient claims. National inpatients were identified in the 2017 Nationwide Readmissions Database.

Study design

Retrospective cohort.

Data collection

Differences in quality were compared using two sets of quality metrics endorsed by the US Agency for Healthcare Research and Quality (AHRQ): Inpatient Quality Indicators, 19 quality metrics that look at differences in in‐hospital mortality, and Patient Safety Indicators, 18 quality metrics that look at differences in in‐hospital morbidity and adverse events. Among MHS beneficiaries (Direct and Purchased Care), we further simulated what changes in quality indicators might look like under various proposed scenarios of reduced access to Direct Care.

Principal findings

A total of 502,252 MHS admissions from 37 MTFs and surrounding civilian hospitals were included (326,076 Direct Care, 179,176 Purchased Care). Nationwide, 9.34 million adult admissions from 2453 hospitals were included. On average, MHS beneficiaries treated in MTFs experienced better inpatient quality and improved patient safety compared with MHS beneficiaries treated in locally available civilian hospitals (e.g., summary observed‐to‐expected ratio for medical mortality: 0.98 vs. 1.03, p < 0.001) and adult patients across the United States (0.98 vs. 1.02, p < 0.001). Simulations of proposed changes resulted in consistently worse outcomes for MHS patients, whether reducing MTF access by 10%, 20%, or 50% nationwide; limiting MTF access to active‐duty beneficiaries; or closing MTFs with the worst performance on patient safety (p < 0.001 for overall quality indicators for each).

Conclusions

Reducing access to MTFs could result in significant harm to MHS patients. The results underscore the importance of health‐policy planning based on evidence‐based evaluation and the need to consider the consequential downstream effects caused by changes in access to care.

What is known on this topic

• The US Military Health System (MHS) has long served as a proving ground for theories about the importance of access to care.

• In October 2019, major health care reforms within the MHS led to a substantial restructuring of MHS beneficiaries' access to military treatment facilities.

What this study adds

• Ongoing MHS health care reforms could lead to marked changes in AHRQ‐endorsed quality metrics for inpatient quality (Inpatient Quality Indicators) and patient safety (Patient Safety Indicators), resulting in significantly worse care for MHS beneficiaries.

• The results underscore the importance of health‐policy planning based on evidence‐based evaluation and need to consider the consequential downstream effects caused by changes in access to care.

1. INTRODUCTION

As one of the largest health systems in the United States and the single largest provider of government‐sponsored universal health insurance coverage for adults aged <65 years, the US Department of Defense's Military Health System (MHS) has long served as a proving ground for theories about the importance of access to care. ¹ , ² It is tasked with ensuring both the health‐related readiness of the US Armed Forces (readiness mission) and the adequate provision of health care to support the health needs of its 9.5 million beneficiaries (benefits mission). ³ Funded by taxpayer money, the resultant dual mission of the MHS is complex, ³ requiring a sophisticated and evolving organizational structure that is able to accomplish both of its aims while simultaneously addressing federal pressure to reduce costs, improve quality, and ensure adequate training of its active‐duty providers. ⁴

Since the MHS's creation in 1966, the national health system has grown to include an international network of military hospitals known as military treatment facilities (MTFs) and a centrally‐administered health insurance program currently known as TRICARE that together enable the MHS's population of active‐duty, reservist, retired, and civilian‐dependent beneficiaries to individually choose to receive care at either MTFs operated by the MHS (Direct Care) or at civilian hospitals with MHS payment of claims (Purchased Care). ⁵ , ⁶ As of October 2018, the MHS employed >82,000 military and 61,000 civilian providers and staff, operating an annual budget of $53 billion (approximately 10% of all US military spending). ⁷

In January 2016, much of that began to change when Congress enacted major reforms to the organizational structure of the MHS. ⁴ , ⁸ , ⁹ Set to begin in October 2019, when completed, the changes are expected to result in marked reductions in MHS beneficiaries' access to Direct Care. Early reports anticipate the removal of >15,000 military providers and staff (18.3% of current levels), reductions in coverage for civilian dependents, and limited provision of certain types of noncombat‐oriented health services. ⁸ , ⁹ A February 2020 interim report to Congress, ¹⁰ formally acknowledged the beginning of “MTF Realignment” and its inclusion of both realized and future MTF closings, shift toward active‐duty‐only coverage at certain MTFs, and plan to further develop specifics on additional “right‐sizing [of] MTF staff.”

The objective of this study was to examine how continued proposed rollouts of MTF Realignment could affect the quality of MHS care. Using national Direct Care MHS claims, we measured variations in US Agency for Healthcare Research and Quality's (AHRQ's) endorsed quality metrics for inpatient quality and patient safety among MHS beneficiaries treated at US‐based MTFs and compared their outcomes with those of (a) MHS beneficiaries treated in locally available civilian hospitals under Purchased Care and (b) national civilian adult inpatient claims. We further simulated what changes in quality indicators might look like for MHS beneficiaries under various proposed scenarios of reduced access to Direct Care.

2. METHODS

2.1. Data source and study population

MHS Direct and Purchased Care inpatient claims for fiscal years 2016–2018, the most recent years of data available following the introduction of ICD‐10‐CM diagnosis and ICD‐10‐PCS procedure codes, were queried for patients meeting eligibility requirements for inclusion in at least one of AHRQ's endorsed quality metrics for inpatient quality and patient safety. ¹¹ Within the MHS, the majority of beneficiaries aged ≥65 years receive health insurance through Medicare. For this reason, included patients were limited to adults aged 18–64 years. Included patients were further limited to those treated in the 37 US‐based MTFs (Direct Care) and in locally available civilian hospitals (Purchased Care), defined as hospitals falling within an MTF's catchment area of 40 miles where indicator‐eligible MHS patients received Purchased Care. MHS beneficiaries receiving Purchased Care in other parts of the country without ready access to an MTF (~20% of the total US‐based MHS population) were excluded from the analysis.

Variations in quality metrics within the national adult civilian population were assessed using inpatient claims meeting the same diagnostic and age‐based inclusion criteria contained within the 2017 Nationwide Readmissions Database (NRD). Drawn from hospital records in 28 states, NRD represents all hospitalizations for patients admitted to hospitals in participating states, regardless of age or insurance. It accounts for >18 million annual hospitalizations, 60.0% of the total US population, and 58.2% of all US hospital admissions. ¹² ICD‐10 claims from January to December 2017 were the most recent data available, situated approximately halfway through the time period included for MHS claims. Deidentified hospital IDs change within NRD each year, precluding the ability for data from 2015 to 2016 to be included.

2.2. Variables and outcomes

AHRQ evaluates inpatient care based on four sets of reliability‐adjusted quality metrics: Inpatient Quality Indicators (IQI; hospital‐specific in‐hospital outcomes for surgical and medical mortality), Patient Safety Indicators (PSI; hospital‐specific in‐hospital outcomes for potentially preventable complications), Prevention Quality Indicators (access to care‐related hospitalizations among regional populations at risk), and Pediatric Quality Indicators. ¹¹ They are designed for use with administrative claims. Each set consists of ≥10 reliability‐adjusted condition‐specific rates (risk‐adjusted rates of adverse events per 1000 hospital admissions, accounting for differences in known covariates including potential confounding due to age, gender, and condition‐specific preexisting medical conditions/prior medical care received, as well as empirical Bayes shrinkage corrections for unstable estimates due to small sample size) ¹³ and ≥1 observed‐to‐expected (O/E) ratio summary measure(s) (calculated as weighted averages of observed vs. expected numbers of adverse events for select condition‐specific rates). ¹⁴ , ¹⁵ , ¹⁶ , ¹⁷ Summary O/E ratios >1.0 indicate more adverse events than expected (worse quality); ratios <1.0 indicate fewer adverse events than expected (better quality). Technical specifications of each metric are available online; quality metrics used in this study were calculated using v2020. ¹¹

For the purposes of this study, pediatric indicators were omitted given insufficient hospital‐specific numbers of pediatric adverse events taking place at hospitals within the MHS. Prevention indicators were omitted given the option for MHS beneficiaries to choose to receive Direct or Purchased Care, a reality that limits meaningful calculation of regional populations at risk.

Using SAS programs from AHRQ, ¹¹ reliability‐adjusted performance on inpatient quality and patient safety was calculated for each MTF, corresponding MHS Purchased Care catchment area, and hospital included in NRD. Due to the nature of the models defined by AHRQ, all indicators considered in‐hospital outcomes occurring during index hospitalization. Transfers were handled in accordance with model specifications. Variations in summary O/E ratios for surgical mortality (IQI), medical mortality (IQI), and patient safety (PSI) and corresponding reliability‐adjusted condition‐specific rates served as the study's primary outcome measures. Receipt of Direct versus Purchased Care within the MHS was the primary explanatory variable. Secondary comparisons looked at differences between MTFs and adults treated at hospitals across the United States.

2.3. Statistical analysis

Differences in summary O/E ratios and reliability‐adjusted condition‐specific rates between MTFs and (a) MHS beneficiaries treated in locally available civilian hospitals and (b) national inpatient claims were assessed using volume‐weighted linear regression models. Analyzing data in this way allowed for varying hospitals/catchment areas to receive their appropriate patient volume‐determined weight.

In order to simulate potential changes in inpatient quality and patient safety within the MHS under various proposed scenarios of reduced access to MTFs, outcomes of MHS beneficiaries treated at MTFs and in locally available civilian hospitals were combined using a second set of counterfactual volume‐weighted linear regression models. Following similar methodology to that described above, current summary ratios and condition‐specific rates for quality metrics with significant differences between Direct and Purchased Care were compared with theoretical versions of the same quality metrics attained following six simulated changes in MHS beneficiaries' access to MTFs: reduction in MTF access nationwide by (1) 10%, (2) 20%, and (3) 50%; (4) restriction of MTF access to active‐duty beneficiaries (no retirees, reservists, or dependents); and closure of worst‐performing MTFs as measured by differences in patient safety ([5] O/E ratios >1.5 [highest quartile] and [6] MTFs where O/E ratios exceeded those of locally available Purchased Care). Differences between current and theoretical O/E ratios and condition‐specific rates were defined as absolute mean differences and 95% confidence intervals (95% CI). Relative increases were calculated based on this change and current values (difference/current).

Initial data analysis, cleaning, and calculation of quality metrics were conducted using Statistical Analysis Software (SAS): Version 9.4. Volume‐weighted modeling was conducted using Stata Statistical Software: Version 16.1. Graphs were plotted in R. Two‐sided p values < 0.05 were considered significant. Missing data required to calculate AHRQ quality metrics (<1.0% for all variables combined) were assumed to be missing completely at random and analyzed using a complete‐case analysis approach. The work was deemed exempt from full review by the Institutional Review Boards of Brigham & Women's Hospital and the Uniformed Services University of the Health Sciences.

3. RESULTS

A total of 502,252 adult MHS beneficiaries admitted to hospitals in fiscal year 2016–2018 met inclusion criteria for at least one quality metric. Of these, 326,076 received care at MTFs (64.9%), and 179,176 were managed in locally available civilian hospitals (35.7%). All 37 US‐based MTFs reported by the MHS in fiscal year 2018 were represented in the data. ⁷ In 2017, NRD included hospital admissions for 9.34 million adults (unweighted) from 2453 hospitals meeting inclusion criteria for at least one quality metric. Overall demographic differences among included patients are presented in Table S1. Generally, the three populations were similar, with adult MHS beneficiaries receiving treatment at MTFs being the least likely to present with documented comorbidities (mean number of reported Elixhauser Comorbidity Index comorbidities per patient: Direct Care 1.7, Purchased Care 2.0, and US adult civilian patients 1.9).

3.1. Quality of care in MTFs compared with hospitals across the United States

Summary indicators for each MTF and their comparison to summary indicators of civilian adults treated in hospitals across the United States are presented in Figure 1. With regard to surgical mortality (Figure 1A), most MTFs outperformed risk‐adjusted expectations, yielding values that primarily fell with the lower half of US hospitals (better quality). Similar results were seen for medical mortality (Figure 1B). The results for patient safety were mixed (Figure 2A), suggesting better than expected outcomes for overall patient safety in the majority of MTFs but a marked increase in O/E patient safety ratios among MTFs falling within the worst‐performing quartile (O/E ratio >1.5). More exaggerated increases in O/E ratios for patient safety were seen for hospitals across the United States (Figure S1), indicating that despite a trend toward higher than expected rates of average patient safety among some MTFs, MTFs as a whole remained among the best performing hospitals for patient safety in the United States.

FIGURE 1.

Variation in summary observed‐to‐expected (O/E) ratios for (A) surgical mortality and (B) medical mortality among adults aged 18–64 years comparing reliability‐adjusted outcomes among Military Health System (MHS) patients treated in military treatment facilities under Direct Care (MTFs; gray diamonds) and general population adults treated in hospitals across the United States (black circles). Black lines indicate observed‐to‐expected (O/E) ratios of 1.00 where the number of observed events equaled the number of expected events. Values <1.00 suggest fewer adverse events than expected (better quality); values >1.00 suggest more events than expected (worse quality)

FIGURE 2.

(A) Variation in summary observed‐to‐expected (O/E) ratios for patient safety among adult Military Health System (MHS) beneficiaries aged 18–64 years treated at military treatment facilities under Direct Care (MTFs). (B) Region‐specific comparison of summary O/E ratios for patient safety among adult MHS beneficiaries treated at MTFs under Direct Care and in locally available civilian hospitals using Purchased Care (n = 30/37 regions, 80.1%, experienced better patient safety at MTFs)

When comparing volume‐weighted mean differences in summary indicators and reliability‐adjusted condition‐specific rates based on linear regression models (Table 1), MTFs had significantly better condition‐specific rates in 8 of 11 inpatient quality metrics (e.g., surgical mortality: CABG mortality rate per 1000 hospital admissions, mean difference [95% CI]: 8.33 [4.25–12.41]; medical mortality: heart failure mortality per 1000 hospital admissions, 1.21 [0.62–1.80]) and 7 of 9 patient safety indicators (e.g., postoperative respiratory failure rate per 1000 hospital admissions, 0.35 [0.18–0.52]). They presented with significantly better (i.e., lower) O/E ratios for surgical mortality (0.91 vs. 1.00, p < 0.001), medical mortality (0.98 vs. 1.02, p < 0.001), and patient safety (1.18 vs. 2.98, p < 0.001).

TABLE 1.

Differences in reliability‐adjusted inpatient quality indicators and patient safety indicators comparing (1) MHS patients treated at US‐based military treatment facilities (MTFs) under Direct Care and (2) adults treated at hospitals across the United States

	Nationwide in 2017	US military treatment facilities (MTFs)	Mean difference	95% CI
Inpatient quality indicators (rate per 1000 admissions)
Better outcomes at MTFs
CABG mortality rate	25.0	16.7	8.3	4.2	12.4
Acute stroke mortality rate	76.8	69.1	7.7	3.9	11.5
Pancreatic resection mortality rate	26.2	22.5	3.7	1.9	5.6
Percutaneous coronary intervention mortality rate	28.9	26.1	2.8	1.4	4.2
Heart failure mortality rate	28.3	27.0	1.2	0.6	1.8
AAA repair mortality rate	38.0	37.3	0.7	0.4	1.1
Carotid endarterectomy mortality rate	4.5	3.9	0.6	0.3	0.9
Gastrointestinal hemorrhage mortality rate	23.6	23.4	0.2	0.1	0.3
Better outcomes among US adults
Pneumonia mortality rate	26.4	27.0	−0.6	−0.3	−0.8
AMI mortality rate	50.0	51.0	−1.0	−0.5	−1.5
Mortality for selected conditions (medicine), O/E ratio	1.02	0.98	p value	<0.001
Mortality for selected procedures (surgery), O/E ratio	1.00	0.91	p value	<0.001
Patient safety indicators (rate per 1000 admissions)
Better outcomes at MTFs
Death rate among surgical inpatients with serious complications	146.2	145.1	1.1	0.5	1.6
Perioperative pulmonary embolism or deep vein thrombosis rate	4.7	3.7	1.0	0.5	1.5
Accidental puncture or laceration rate	1.1	0.9	0.2	0.1	0.2
Death rate in low‐mortality DRG	0.5	0.2	0.3	0.2	0.5
Postoperative physiologic and metabolic derangement rate	0.9	0.7	0.2	0.1	0.3
Postoperative respiratory failure rate	5.7	5.4	0.3	0.2	0.5
Postoperative sepsis rate	4.4	4.0	0.4	0.2	0.7
Better outcomes among US adults
Postoperative wound dehiscence rate	0.7	0.7	0.0	0.0	−0.1
Perioperative hemorrhage or hematoma rate	2.3	2.4	−0.1	0.0	−0.1
Patient safety for selected indicators, O/E ratio	2.98	1.18	p value	<0.001

Note: Mean differences and 95% confidence intervals were derived from volume‐weighted linear regression models.

Summary O/E ratios are based on reliability‐adjusted metric weights derived for the 2017 US population.

• Summary metrics for medical mortality were based on weighted contributions from reliability‐adjusted mortality rates for AMI, heart failure, acute stroke, gastrointestinal hemorrhage, hip fracture (omitted due to age‐eligibility), and pneumonia.
• Summary metrics for surgical mortality were based on reliability‐adjusted mortality rates for esophageal resection (omitted due to low sample size), pancreatic resection, AAA repair, CABG, percutaneous coronary intervention, and carotid endarterectomy.
• Summary metrics for patient safety were based on weighted contributions from reliability‐adjusted adverse event rates for pressure ulcer (omitted due to low sample size), iatrogenic pneumothorax (omitted due to low sample size), in‐hospital fall with hip fracture (omitted due to age‐eligibility), perioperative hemorrhage or hematoma, postoperative acute kidney injury requiring dialysis (omitted due to low sample size), postoperative respiratory failure, perioperative pulmonary embolism or deep vein thrombosis, postoperative sepsis, postoperative wound dehiscence, and accidental puncture or laceration.

Abbreviations: AAA, abdominal aortic aneurysm; AMI, acute myocardial infarction; CABG, coronary artery bypass graft; DRG, diagnosis‐related groups; MTF, military treatment facility; US, United States.

3.2. Differences between US‐based MTFs and locally available civilian hospitals

Mean differences in summary indicators and reliability‐adjusted condition‐specific rates between MTFs and MHS beneficiaries receiving Purchased Care at locally available civilian hospitals are presented in Table 2. Akin to the results between MTFs and the general US population, MHS beneficiaries receiving care at MTFs, on average, experienced better reliability‐adjusted outcomes in 4 of 11 inpatient quality indicators and 5 of 9 patient safety indicators. The results for the remaining condition‐specific rates were not significantly different between treatment groups. Summary O/E ratios told a similar story, demonstrating significantly better outcomes for MTF patients in terms of overall medical mortality (0.98 vs. 1.03, p < 0.001) and patient safety (1.18 vs. 1.68, p < 0.001). When compared on a region‐specific basis (Figure 2B), 80.1% (n = 30/37) of MTFs experienced better overall patient safety.

TABLE 2.

Differences in reliability‐adjusted inpatient quality indicators and patient safety indicators comparing (1) MHS patients treated at US‐based military treatment facilities (MTFs) under Direct Care and (2) MHS patients treated in locally available civilian hospitals using Purchased Care

	Locally available civilian	US military treatment facilities (MTFs)	Mean difference	95% CI
Inpatient quality indicators (rate per 1000 admissions)
Better outcomes at MTFs
Heart failure mortality rate	32.3	27.0	5.2	2.1	8.4
CABG mortality rate	18.9	16.7	2.2	0.0	4.4
Pneumonia mortality rate	28.5	27.0	1.5	0.1	3.0
Pancreatic resection mortality rate	23.6	22.5	1.1	0.0	2.2
No significant difference
Acute stroke mortality rate	71.8	69.1	2.7	−2.5	7.9
Gastrointestinal hemorrhage mortality rate	23.6	23.4	0.2	−0.1	0.4
Percutaneous coronary intervention mortality rate	26.1	26.1	0.0	−1.0	1.1
Carotid endarterectomy mortality rate	3.9	3.9	0.0	0.0	0.0
AMI mortality rate	50.6	51.0	−0.4	−0.9	0.2
AAA repair mortality rate	36.3	37.3	−0.9	−12.2	10.4
Mortality for selected conditions (medicine), O/E ratio	1.03	0.98	p value	<0.001
Mortality for selected procedures (surgery), O/E ratio	0.93	0.91	p value	<0.001
Patient safety indicators (rate per 1000 admissions)
Better outcomes at MTFs
Postoperative respiratory failure rate	12.6	5.4	7.3	2.4	12.1
Death rate in low‐mortality DRG	0.6	0.2	0.4	0.1	0.8
Postoperative sepsis rate	4.3	4.0	0.3	0.1	0.6
Postoperative physiologic and metabolic derangement rate	0.9	0.7	0.1	0.0	0.2
Accidental puncture or laceration rate	1.0	0.9	0.1	0.0	0.1
No significant difference
Perioperative pulmonary embolism or deep vein thrombosis rate	3.8	3.7	0.1	−0.2	0.4
Postoperative wound dehiscence rate	0.7	0.7	0.0	−0.1	0.0
Perioperative hemorrhage or hematoma rate	2.3	2.4	−0.1	−0.3	0.1
Death rate among surgical inpatients with serious complications	143.3	145.1	−1.8	−3.9	0.2
Patient safety for selected indicators, O/E ratio	1.68	1.18	p value	<0.001

Note: Mean differences and 95% confidence intervals were derived from volume‐weighted linear regression models.

Summary O/E ratios are based on reliability‐adjusted metric weights derived for the 2017 US population.

• Summary metrics for medical mortality were based on weighted contributions from reliability‐adjusted mortality rates for AMI, heart failure, acute stroke, gastrointestinal hemorrhage, hip fracture (omitted due to age‐eligibility), and pneumonia.
• Summary metrics for surgical mortality were based on reliability‐adjusted mortality rates for esophageal resection (omitted due to low sample size), pancreatic resection, AAA repair, CABG, percutaneous coronary intervention, and carotid endarterectomy.
• Summary metrics for patient safety were based on weighted contributions from reliability‐adjusted adverse event rates for pressure ulcer (omitted due to low sample size), iatrogenic pneumothorax (omitted due to low sample size), in‐hospital fall with hip fracture (omitted due to age‐eligibility), perioperative hemorrhage or hematoma, postoperative acute kidney injury requiring dialysis (omitted due to low sample size), postoperative respiratory failure, perioperative pulmonary embolism or deep vein thrombosis, postoperative sepsis, postoperative wound dehiscence, and accidental puncture or laceration.

Geographic proximity was defined as hospitals treating MHS patients within 40 miles of an MTF managing similar patients.

• MHS‐defined hospital catchment area for MTFs.

Abbreviations: AAA, abdominal aortic aneurysm; AMI, acute myocardial infarction; CABG, coronary artery bypass graft; DRG, diagnosis‐related groups; MTF, military treatment facility.

3.3. Implications for MHS beneficiaries

Anticipated changes in summary measures and reliability‐adjusted condition‐specific rates following various proposed reductions in MHS beneficiaries' access to MTFs are presented in Table S2. Relative percent changes from current values for summary measures and exemplary condition‐specific rates (two quality metrics from each set) are highlighted in Figure 3.

FIGURE 3.

Simulated changes in summary observed‐to‐expected (O/E) ratios (black) and exemplary condition‐specific metrics (gray) among adult Military Health System (MHS) beneficiaries following various proposed reductions in MHS beneficiaries' access to military treatment facilities (MTFs). Results are reported as relative changes (absolute values are presented in Table S2). Each was significantly based on a two‐sided p value < 0.001

Among US‐based MHS beneficiaries, reducing access to MTFs by as little as 10% nationwide could result in significantly worse surgical mortality (e.g., 0.7% increase in CABG mortality, mean difference [95% CI]: 0.13 [0.07–0.19] additional deaths per 1000 hospital admissions), medical mortality (e.g., 1.3% increase in heart failure mortality, 0.36 [0.18–0.54] additional deaths per 1000 hospital admissions), and patient safety (e.g., 4.5% increase in postoperative respiratory failure, 0.39 [0.20–0.59] additional complications per 1000 hospital admissions). More prominent increases in adverse outcomes were seen following reducing MTF access by 20% (e.g., 9.1% increase in postoperative respiratory failure) and 50% (22.6% increase in postoperative respiratory failure) nationwide. Limiting MTF access to active‐duty beneficiaries increased adverse patient safety by 23.0% (postoperative respiratory failure rates by 13.6%), while closing MTFs with the worst performance on patient safety increased adverse patient safety by 16.5% (postoperative respiratory failure rates by 16.4%) owing to worse outcomes for MHS beneficiaries treated in locally available civilian hospitals. Removing MTF access in regions with worse overall patient safety than locally available civilian hospitals did result in a 5.1% reduction in adverse patient safety (3.9% reduction in postoperative respiratory failure rates). However, it was accompanied by a 1.1% increase in surgical mortality (e.g., 0.5% increase in CABG mortality corresponding to 0.09 [0.05–0.13] additional deaths per 1000 hospital admissions) and 2.0% increase in medical mortality (e.g., 1.3% increase in heart failure mortality corresponding to 0.37 [0.19–0.55] additional deaths per 1000 hospital admissions).

4. DISCUSSION

The results of this study show that ongoing health care reforms ⁸ , ⁹ stemming from an organizational overhaul of the MHS in 2016 ⁴ could result in significant harm to MHS patients. Quality assessment of inpatient outcomes conducted in accordance with the US AHRQ between Direct Care MTFs and locally available civilian Purchased Care revealed better mortality and patient safety for MHS beneficiaries treated in military hospitals. While room for improvement remains among MTFs with higher than expected rates of adverse patient safety, assessment of various proposed reductions in access to MTFs consistently showed that suddenly shifting MTF patients into local civilian hospitals would likely result in worse patient safety and significant increases in both medical and surgical mortality. Compared with outcomes of adult patients treated in hospitals across the United States, MTFs consistently performed among the best hospitals in the United States.

Critics have alleged that the MHS's US operations cost too much, deliver an uneven quality of care, and do not attract enough complex cases or sufficient volume to maintain the skills that active‐duty providers need to serve on deployment. ¹⁸ , ¹⁹ Nevertheless, available evidence suggests that limiting access to MTFs could decrease the quality of care for active‐duty service members, their families, and military retirees. Repeated assessments of systemic and hospital quality by the US government ²⁰ and American College of Surgeons ²¹ point toward comparable performance between the MHS and some of the country's top health systems. ²⁰ They indicate that Direct Care MTFs are among the best performing surgical hospitals in the United States ²¹ —a finding with which our results agreed. Prior studies of MHS patients have further documented a lack of racial disparities seldom found in the civilian sector, ¹ , ² , ²² revealed that for select conditions MHS patients treated by military doctors underwent fewer procedures but experienced better outcomes, ²³ and highlighted potential concerns that forcibly shifting MHS beneficiaries from Direct into Purchased Care could result in decreased efficiency ²⁴ and uncertain cost savings. ²⁵ The results of our study build on this work, demonstrating that for each of AHRQ's validated adult sets of hospital‐specific quality metrics, US‐based MTFs operating within the Direct Care system consistently performed equivalent to or better than locally available Purchased Care.

Described as the “most sweeping overhaul of the [MHS] in a generation,” ²⁶ ongoing health care reforms within the MHS were designed to provide a “set of interdependent and nested initiatives” that optimize the delivery of the quadruple aim: improved readiness, better health, better care, and lower cost. ²⁷ The challenge lies in figuring out if it can be achieved and discerning how it should be accomplished. Lessons learned from health care reform in the civilian sector over the past decade suggest that unilateral improvement is seldom simple. Well‐intended programs can fail ²⁸ or have unintended consequences that adversely affect the very patients they were designed to protect. ²⁹ , ³⁰ Improvements in value often face trade‐offs between a desire to reduce costs and the expenditures necessary to make improvements in quality. ³⁰ It is unknown where the ongoing changes taking place within the MHS will ultimately fall on this scale. Much likely depends on the manner in which proposed changes are implemented. In looking toward health‐policy planning ahead, the results of our study suggest that, in terms of quality as measured by changes in inpatient quality and patient safety, additional caution is needed in order to protect the health and safety of US servicemen and women as well as that of their loved ones that service members serve to protect. Changes in cost and, resultantly, taxpayer funding will largely depend on whether reductions in MTF overhead are offset by increased spending on adverse outcomes within the locally available civilian sector and the MHS's ability to act as a “smart purchaser” of Purchased Care.

Lack of sufficient foresight and planning is a common theme in health‐policy evaluation—one repeatedly evidenced by the mixed cycles of successes and failures that continue to dominate political discussions of health in the United States today. As an example of current health‐policy change, the results underscore the importance of health‐policy planning based on evidence‐based evaluation and the need to consider the consequential downstream effects caused by sudden changes in access to care. The answers are not easy, and few if any perfect solutions exist. Yet in monitoring the real‐time rollout of ongoing health‐policy change in one of the country's largest, government‐run, universally insured health systems, there are important lessons to be learned. Changes taking place within the MHS affect not only the future and health of the country's MHS patients but also those of similar patients treated in the civilian sector who are likely to be affected by forthcoming discussions about, for example, the importance of ensuring access to care, potential development of government‐sponsored health insurance coverage of younger adults, and implications of major health‐system change on the development of larger regionalized centers of excellence versus the closure of smaller locally available community hospitals. In shifting patients from MTFs into locally available, primarily smaller civilian hospitals, ongoing health care reforms within the MHS have the added complication of adding >1.9 million additional beneficiaries (~20% of current US‐based Direct Care coverage) into the catchment areas and corresponding workloads of many smaller hospitals located in areas previously served by MTFs in communities across the United States. It is also possible that increasing volumes within smaller hospitals could, over time, lead to regional improvements in the quality of care.

Future studies are encouraged to more closely look at the role that hospital volume plays on the quality of MHS patients' care and to move beyond in‐hospital outcomes to also assess MTF Realignment's implications on longer‐term outcomes‐of‐care. To bolster the quality of outcomes experienced by displaced MHS patients based on what is currently known, policy makers might consider focusing future development efforts on creating the following: (1) larger centers of excellence at remaining MTFs where MHS patients with more complicated presentations can be transported for care; (2) resources to ensure the timely transfer of MHS patients shifted outside of MTFs to larger, regional civilian centers able to provide comparable care; and (3) infrastructure in surrounding civilian hospitals where displaced MHS patients are likely to receive Purchased Care.

The study has limitations. The most important reflect its reliance on administrative claims, including the potential for absence or misreporting of events and limited access to clinical detail. While MHS beneficiaries are similar to the US population, the MHS is a health system composed of predominately younger patients, making it difficult to apply quality metrics developed for use among older adults in Medicare or the Veterans Health Administration. The use of validated hospital‐specific quality metrics endorsed by AHRQ and designed for use in administrative claims among patients of all ages helps to address this concern. However, it is worth noting that due to low patient volumes and/or adverse event numbers for certain conditions, some quality metrics were omitted and others faced regional exclusions when unstable condition‐specific rates defaulted to the reliability‐adjusted average rate. In combining outcomes across military and civilian hospitals, systemic differences in comorbidity screening and reporting and/or motivating incentives based on differences in reimbursement schemes could have influenced the baseline differences in comorbidities observed. ³¹ AHRQ quality metrics are designed to account for differences in the reported case mix. Unreported differences in severity or other systemic contributing confounding factors could still be at play.

As one of the largest health systems in the United States and the single largest provider of government‐sponsored universal health insurance coverage for adults aged <65 years, the US Department of Defense's MHS is tasked with ensuring both the health‐related readiness of the US Armed Forces and the adequate provision of health care to support the health needs of its 9.5 million beneficiaries. The results of this study show that ongoing health care reforms stemming from an organizational overhaul of the MHS in 2016 intended to better achieve these aims could result in significant harm to MHS beneficiaries requiring inpatient care should they proceed as currently planned, including marked reductions in inpatient quality and patient safety. As an example of ongoing health‐policy change, the results underscore the importance of health‐policy planning based on evidence‐based evaluation and the critical need to consider the consequential downstream effects caused by changes in access to care.

CONFLICT OF INTEREST

The authors declare no conflicts of interest.

AUTHOR CONTRIBUTIONS

Cheryl K. Zogg, Judith H. Lichtman, Tracey Perez Koehlmoos, Joel S. Weissman, and Zara Cooper made substantial contributions to the conception or design of the work. Cheryl K. Zogg and Tracey Perez Koehlmoos participated in the acquisition and analysis of the data. Cheryl K. Zogg, Judith H. Lichtman, Michael K. Dalton, Peter A. Learn, Andrew J. Schoenfeld, Tracey Perez Koehlmoos, Joel S. Weissman, and Zara Cooper contributed toward the interpretation of data for the work. Cheryl K. Zogg drafted the manuscript, and Judith H. Lichtman, Michael K. Dalton, Peter A. Learn, Andrew J. Schoenfeld, Tracey Perez Koehlmoos, Joel S. Weissman, and Zara Cooper critically revised the manuscript for intellectual content. All authors provided final approval of the version to be published and agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

DISCLOSURE

The contents of this publication are the sole responsibility of the authors and do not reflect the views, assertions, opinions, or policies of the Uniformed Services University of the Health Sciences, Department of Defense, or Departments of the Army, Navy, or Air Force. Mention of trade names, commercial products, or organizations does not imply endorsement by the US Government.

Supporting information

Appendix S1. Supporting Information.

Zogg CK, Lichtman JH, Dalton MK, et al. In defense of Direct Care: Limiting access to military hospitals could worsen quality and safety. Health Serv Res. 2022;57(4):723‐733. doi: 10.1111/1475-6773.13885