Outcomes of primary care delivery by nurse practitioners: Utilization, cost, and quality of care

Abstract

Objective

To examine whether nurse practitioner (NP)‐assigned patients exhibited differences in utilization, costs, and clinical outcomes compared to medical doctor (MD)‐assigned patients.

Data Sources

Veterans Affairs (VA) administrative data capturing characteristics, outcomes, and provider assignments of 806 434 VA patients assigned to an MD primary care provider (PCP) who left VA practice between 2010 and 2012.

Study Design

We applied a difference‐in‐difference approach comparing outcomes between patients reassigned to MD and NP PCPs, respectively. We examined measures of outpatient (primary care, specialty care, and mental health) and inpatient (total and ambulatory care sensitive hospitalizations) utilization, costs (outpatient, inpatient and total), and clinical outcomes (control of hemoglobin A1c, LDL, and blood pressure) in the year following reassignment.

Principal Findings

Compared to MD‐assigned patients, NP‐assigned patients were less likely to use primary care and specialty care services and incurred fewer total and ambulatory care sensitive hospitalizations. Differences in costs, clinical outcomes, and receipt of diagnostic tests between groups were not statistically significant.

Conclusions

Patients reassigned to NPs experienced similar outcomes and incurred less utilization at comparable cost relative to MD patients. NPs may offer a cost‐effective approach to addressing anticipated shortages of primary care physicians.

What this study adds.

What is already known on this topic

• Prior research suggests that primary care NPs provide comparable clinical care compared to MDs.

• No comprehensive assessment has been conducted of the clinical and economic differences in care provided by primary care NPs versus MDs.

What this study adds

• NP patients did not differ from MD patients in VA health care costs.

• Compared to MD patients, NP patients had less utilization of primary care, specialty care, and inpatient services.

• NP and MD patients achieved similar quality of care in chronic disease management.

1. INTRODUCTION

Nurse practitioners (NPs), accounting for about one‐fifth of the primary care workforce in the United States1 serve a significant role in primary care delivery. The Veterans Health Administration (VA), one of the largest integrated health care systems in the United States,2 employs more than 5000 nurse practitioners in 142 VA medical centers (VAMCs) and over 800 community‐based outpatient clinics (CBOCs) across 50 states, the District of Columbia, and Puerto Rico.3 NPs in the VA play a key role in the delivery of team‐based primary care, serving as primary care team leaders,4 and provide nearly 20 percent of primary care visits, approximately two million visits annually.2, 5 VA is one of the largest employers of NPs in the United States.

Several studies have identified current and expected future shortages of primary care physicians in the United States, and increased use of NPs as primary care providers has been suggested as a potential solution to alleviate this shortage.6, 7, 8 Indeed, states have steadily increased the practice authority of NPs over the past decade.9 For national health care systems such as VA, NPs may offer cost‐effective care delivery, as the average annual salary for NPs in the United States in 2017 was $107 480 compared to $198 370 for medical doctor (MD) general internists.10, 11 NPs can increase primary care capacity provided they deliver care that is at least comparable to that of MDs. In addition, the broader use of NPs may only be more effective for subgroups within a patient population that include younger, less complex patients. This has been posed as a concern as NPs typically receive less extensive training and may be less equipped to care for more complex patients.12

Previous research suggests that relative to MDs, NPs provide care with comparable clinical outcomes2 and quality of care,13, 14, 15, 16 particularly for patients with chronic conditions.13, 14, 15, 16, 17 However, prior observational studies used the observed care provided by an NP vs an MD as the exposure variable. This could have led to biased estimates of the comparative effectiveness of NP‐provided care if more seriously ill patients self‐selected to receive care from MDs. In addition, previous studies did not assess a comprehensive set of clinical and economic outcomes.

In this study, we conducted a comprehensive assessment of patient outcomes between primary care NPs and MDs, including utilization, cost, and quality of care. To address patient selection bias, we applied a unique quasi‐experimental approach. We identified patients who were reassigned to either an MD or an NP as their PCP after their prior MD provider left VA primary care practice. This natural experiment introduces an element of random assignment to NP or MD care because the reassignment process necessarily depends in part on the panel capacities of other clinicians in the clinic, which is exogenous to a reassigned patient’s health. Additionally, we applied instrumental variable models, leveraging natural variation in the supply of NPs at VA facilities to increase the plausibility of inferences derived from our primary quasi‐experimental approach. The comparison of outcomes between patients receiving primary care from NPs compared to MDs can inform health systems and policy makers on whether NPs can effectively meet the increasing primary care demand in the United States at comparable cost and quality.

2. METHODS

2.1. Data and study sample

We conducted a retrospective cohort study using VA electronic health records from the VA Corporate Data Warehouse (CDW). PCPs and patients were identified from the Primary Care Management Module (PCMM), which contains longitudinal data on all VA primary care panels and tracks PCPs by type (eg, MDs and NPs). Primary care at the VA is team‐based. PCMM allows VA administrators to define PCP‐led teams and assign patients to a specific PCP. National guidelines recommend a target panel size of 1200 for MDs and 900 for NPs. Panel size can be adjusted based on patient and facility characteristics.

We present the algorithm for deriving the study sample in (Figure 1). We first identified primary care MDs who left VA primary care practice from 2010 to 2012, defined as those MDs without a patient panel for two consecutive quarters. We then identified all patients assigned to these MDs who were subsequently reassigned to a new PCP. We then applied several exclusions including dropping patients who were reassigned to a PCP that was not an MD or NP. The final sample consisted of 806 434 patients in 530 VA facilities, of whom 696 404 patients had a new MD PCP (hereafter MD patients) and 110 030 had a new NP PCP (hereafter NP patients).

Figure 1.

Algorithm for defining sample of VA primary care patients reassigned to NPs and MDs, respectively

To assess clinical outcomes for quality of care, we identified three disease‐specific cohorts from the final sample based on one outpatient or inpatient diagnosis code in the year prior to the PCP reassignment: diabetes (n = 388 231: 26 363 NP patients and 180 934 MD patients), ischemic heart disease (IHD) (n = 113 260: 14 413 NP patients and 98 847 MD patients), and hypertension (n = 357 987: 45 806 NP patients and 312 181 MD patients).

2.2. Study design and explanatory variables

We employed a difference‐in‐difference approach, leveraging natural variation arising from the reassignment of patients whose provider left VA primary care practice. The independent variable of interest was a binary measure denoting type of PCP that patients were reassigned to, as indicated within PCMM (=0 for reassignment to an MD and =1 for reassignment to an NP). Other than PCP panel capacity, there is no national guidance on how to reassign a patient to a PCP. To facilitate comparisons in pre‐ and postchanges between groups, we interacted the indicator for reassignment to NP with a second indicator denoting time period (=1 for post‐reassignment, =0 for pre‐reassignment).

2.3. Outcome measures

We examined nine utilization and cost outcome categories using VA administrative data capturing patient encounters during one year after the reassignment to a new PCP.

2.3.1. Utilization and cost measures

We assessed three types of outpatient utilization: primary care, which includes visits at Geriatrics Primary Care and Comprehensive Women’s Primary Care clinics; specialty care; and mental health visits. Classification of outpatient visits was based on VA clinic stop codes, which identify the type of services rendered at a specific physical location (eg, primary care, cardiology, psychiatry, etc).18 We examined (a) a binary measure denoting whether patients had any visits in a given year and (b) number of visits among users for each outpatient visit type.

We assessed two measures of inpatient utilization: any inpatient admission and any admission for Prevention Quality Indicators (PQIs). PQIs are a set of ambulatory care sensitive conditions for which high‐quality outpatient care can potentially prevent hospitalization.19 If NPs provide a different quality of care compared to MDs, we would expect higher rates of hospitalization, particularly for PQIs, and that MD‐NP differences would increase with patient complexity.

Two cost measures were assessed: total outpatient cost and total cost, both measured using VA’s Managerial Cost Accounting System, an activity‐based cost allocation system that generates estimates of the cost of individual VA hospital stays and health care encounters.20 All cost estimates were inflation‐adjusted to 2016 dollars using the Consumer Price Index.21

Finally, we examined whether patients received two types of diagnostic tests, metabolic panel and echocardiogram or stress test, which were identified using a previously developed algorithm classifying categories of CPT procedure codes.22, 23 Previous studies have suggested that NPs order more diagnostic tests and imaging.24, 25 We hypothesized that NPs, who may have fewer years of formal training than MDs, were more likely to order diagnostic tests that are more frequent and low cost (metabolic panel), as well as less frequent and high cost (echocardiogram or stress test), particularly for newly assigned patients. The diagnostic tests are not disease cohort specific.

2.3.2. Clinical quality

We examined eight process and outcome measures routinely used by VA to measure the quality of primary care.26, 27 Four measures were assessed for the diabetes cohort: (a) receipt of any hemoglobin A1c (HbA1c) test, (b) normal HbA1c defined as HbA1c < 9 percent, (c) receipt of any low‐density lipoprotein (LDL) measure, and (e) normal LDL defined as LDL <100 mg/dL. For HbA1c or LDL, we used the value of the last measure if there were multiple measures in a given year. Two measures were assessed for the IHD cohort: (a) receipt of any LDL test and (b) normal LDL defined as LDL <100 mg/dL. Two measures were assessed for the hypertension cohort: (a) record of any blood pressure (BP) measurement and (b) normal BP defined as systolic BP <140 mm Hg and diastolic BP <90 mm Hg.26

2.4. Control variables

We controlled for a large set of confounders that may potentially influence outcomes and NP assignment. These included patient demographics (age, gender, race, ethnicity, marital status), exemption from VA copayments, non‐VA health coverage (Medicaid and Medicare), distance to the closest VA facility, geographic locations (urban/rural, Census division), length of time with the new PCP (<90, 91‐180, 181‐270, 271‐365, >365 days), facility type (VA Medical Center [VAMC] vs CBOC), MD supply in the parent VAMC (number of full‐time equivalent MDs/1000 patients), and baseline comorbidity measures. The baseline period was defined as one year prior to the assignment of the new PCP. We controlled for comorbidity using the validated Gagne index, which is a composite weighted average of 20 conditions derived by combining conditions in the Charlson28 and Elixhauser29 comorbidity scores.30 We controlled for the number of drug classes of prescriptions taken by patients and an indicator denoting the presence of any psychiatric condition. Finally, we included lagged outcomes for all outcomes; lagged values for LDL and HbA1c were coded as high, normal, or not present.

2.5. Statistical analysis

We applied generalized linear models (GLMs) to estimate differences in outcomes between patients reassigned to NPs and MDs. We modeled binary outcomes (any outpatient visits, hospitalizations, diagnostic tests, and meeting of clinical quality measures) using GLMs assuming a binomial distribution and a logit link. For counts of outpatient visits, we assumed a negative‐binomial distribution and a logit link. For cost outcomes, we integrated GLMs within two‐part models to account for the large proportion of zero observations.31, 32 In the first part, we modeled whether patients incurred nonzero costs as a binary outcome using the assumptions described above. In the second‐part model, we modeled the number of visits among nonzero observations using GLM assuming a gamma distribution and a log link, which were determined using the Pregibon Link test33 and the modified Park test.34 For all outcomes, we examined whether differences in outcomes varied by level of patient comorbidity by interacting Gagne score categories (<0, 0‐2, 3‐10, and ≥11) with indicator variables for reassigned PCP type and period. Standard errors for GLM parameter estimates were estimated using a bootstrap procedure with 1000 replications. All standard error estimates adjusted for intracluster correlation at the facility level.

For inference, we calculated average marginal effects (AMEs), which reflect average differences in outcomes between patients reassigned to NPs and MDs. AMEs are presented on the probability scale for dichotomous outcomes, visit counts for utilization measures, and dollars for cost measures. For analyses of differential effects, we estimated AMEs conditional on the four levels of comorbidity. Standard errors for AME estimates were estimated using the delta method, and all hypotheses tests assumed a nominal P‐value of .01 due to multiple comparisons.

We conducted sensitivity analyses by applying instrumental variable (IV) models to explore the potential for unobserved confounders affecting PCP assignment and outcomes. This approach seeks to isolate the pseudorandom component of PCP assignment that is related to an IV and not correlated with unobserved variables influencing the outcome.35 The IV we employed was the proportion of NPs (the number of NPs/the total number NPs and MDs) at VA primary care clinics. The set of NPs from which the VA can hire is smaller in states with more restrictive NP scope of practice.9 We hypothesized the IV would be strongly correlated with the explanatory variable as greater availability of NPs at a clinic should increase the probability of being reassigned to an NP. We also hypothesized that after controlling for clinic characteristics, the proportion of NPs should be uncorrelated with unobserved variables affecting outcomes as a result of this variation in the IV influenced by state laws.

For dichotomous outcomes, we implemented the IV approach using bivariate probit models. In these models, the probability of being reassigned to an NP and the probability of a positive outcome were estimated as separate, but correlated outcomes. NP reassignment was estimated as a function of the IV (ie, the proportion of PCPs at a clinic that were NPs) and control variables. Dichotomous patient outcomes were simultaneously modeled as a function of NP reassignment and control variables. Instrument strength was assessed by comparing the partial F‐statistic for the IV with a cutoff of 10, defined by previous studies as a minimum threshold.36 For IV models with dichotomous treatment and outcome variable, the bivariate probit model has been shown to minimize bias when estimating average treatment effects.37

For continuous outcomes, IV models were implemented using the two‐stage residual inclusion estimator.38 In the first stage, we modeled NP reassignment as a function of the IV and control variables using a GLM with a binomial distribution and logit link. We then estimated the second stage as a function of NP reassignment, control variables and the response residual from the first‐stage model, defined as the difference between the observed outcome and expected probability of NP reassignment. For utilization and cost analyses, two‐stage residual inclusion was applied to each part of the two‐part model previously specified.

This study was approved by the Institutional Review Boards at the authors’ affiliated institutions.

3. RESULTS

3.1. Patient characteristics

In (Table 1), we report standardized differences, which compare means in units of the pooled standard deviation and can be used to compare balance in measured variables. With few exceptions, standardized differences in patient characteristics between patients reassigned to NPs and MDs were below 10, indicating negligible differences between the two groups.39, 40 Both groups had a mean age of 62 years and exhibited small differences in VA copayment status, Medicaid and Medicare coverage, and comorbidity. Compared to MD patients, NP patients were less likely to be black (17.4 percent vs 13.2 percent) and stay with the new PCP for more than 1 year (41.7 percent vs 46.6 percent).

Table 1.

Baseline patient characteristics of patients reassigned to NPs and MDs

	NP (n = 110 030)		MD (n = 696 404)		Std. diff. (NP‐MD)
	Mean or %	SD	Mean or %	SD
Age (mean, y)	62.3	15.5	62.0	15.1	1.6
Age ≥ 65 (%)	45.9	49.8	44.7	49.7	2.6
Female (%)	7.7	26.7	6.2	24.1	5.9
Race
White (%)	78.2	41.3	74.0	43.9	9.8
Black (%)	13.2	33.8	17.4	37.9	−11.8
Other (%)	4.6	2.1	4.9	21.7	−1.5
Unknown (%)	4.0	19.6	3.6	18.7	2.1
Ethnicity: Hispanic (%)	4.3	20.2	6.2	24.2	−8.9
VA copayment status
Copay exempt due to disability (%)	42.3	49.4	43.5	49.6	−2.2
Copay exempt due to low income (%)	36.3	48.1	37.4	48.4	−2.2
No copay exempt (%)	21.3	41.0	19.1	39.3	5.4
Married (%)	55.9	49.6	55.1	49.7	1.6
Distance to the closest VA facility
<5 miles (%)	22.7	41.9	21.6	41.1	1.1
5‐10 miles (%)	16.8	37.4	19.5	39.6	−7.0
11‐20 miles (%)	17.5	38.0	20.4	40.3	−7.3
21‐40 miles (%)	18.7	39.0	17.5	38.0	3.2
>40 miles (%)	24.0	42.7	20.8	40.6	7.7
Missing distance	0.3	5.5	0.3	5.4	0.3
Medicaid (%)	3.6	18.5	3.3	17.9	1.3
Medicare coverage
Fee for service (%)	38.3	48.6	37.1	48.3	2.4
HMO (%)	10.0	30.0	9.9	29.9	3.0
No Medicare (%)	51.7	50.0	53.0	49.9	−2.5
Any psychiatric condition (%)	47.6	49.9	48.5	50.0	−1.8
Comorbidity—Gagne score
<0 (%)	21.1	40.8	21.2	40.9	−0.3
1‐2 (%)	67.7	46.8	67.0	47.0	1.5
3‐10 (%)	11.1	31.4	11.6	32.0	−1.8
11+ (%)	0.2	4.4	0.2	4.3	0.2
NOSOS risk score	0.56	0.97	0.58	0.95	−0.02
Diabetes cohort (%)	24.0	42.7	26.0	43.9	−4.7
Ischemic heart disease cohort (%)	13.1	33.8	14.2	34.9	−3.1
Hypertension cohort (%)	41.6	49.3	44.8	49.7	−6.5
Length of time with new provider (%)
<90 d	20.6	40.4	21.8	41.3	−3.0
90‐179 d	15.2	35.9	11.5	31.9	10.9
180‐269 d	11.5	31.9	10.4	30.5	3.7
270‐364 d	11.0	31.2	9.7	29.6	4.0
≥365 d	41.7	49.3	46.6	49.9	−9.8
Region
West (%)	27.7	44.8	22.6	41.8	5.7
Midwest (%)	30.0	45.8	19.0	39.2	25.9
South (%)	32.4	46.8	50.4	50.0	−37.2
Northeast (%)	9.9	29.8	8.0	27.1	6.6
Rural (%)	23.9	42.7	14.7	35.5	23.4
Facility type: VAMC (%) vs CBOC	53.1	49.9	52.7	49.9	0.8
FTE MDs/1000 patients at parent VAMC
Nonsurgical specialists	47.6	21.1	53.3	19.6	−29.9
Surgical specialists	30.6	15.9	36.0	15.3	−35.1
Primary care	77.1	15.2	80.9	14.8	−22.7
Outcome data collected in baseline period
Any visit (%)
Primary care	90.7	29.0	90.4	29.5	1.2
Specialty care	34.0	47.4	38.0	48.5	−8.3
Mental health	27.9	44.9	28.8	45.3	−1.9
Number visits (mean)
Primary Care	4.5	4.9	4.6	4.9	0.0
Specialty care	1.6	4.7	2.0	5.1	−0.1
Mental health	4.3	18.7	4.3	18.1	0.0
Test or procedure (%)
Any metabolic panels	62.9	48.3	65.4	47.6	−5.3
Any echocardiogram or stress test	3.9	19.4	4.8	21.3	−4.1
Hospitalization (%)
Any hospitalization	7.9	27.0	8.9	28.4	−3.4
Any PQI	1.1	10.4	1.2	11.0	−1.3
VA costs ($)
Outpatient costs	4096	6900	4659	7941	−0.08
Total costs	6385	18 943	7111	24 007	−0.03
Clinical outcomes: diabetes (%)
Any HbA1c	86.4	34.2	87.5	33.0	−3.2
HbA1c control: HbA1c < 9%	87.5	33.0	87.1	33.5	1.3
Any LDL	83.1	37.5	83.7	36.9	−1.6
LDL control: LDL < 100	71.2	45.3	72.8	44.5	−3.5
Clinical outcomes: IHD (%)
Any LDL	78.0	41.5	79.8	40.2	−4.4
LDL control: LDL < 100	74.3	43.7	76.3	42.5	−4.7
Clinical outcomes: hypertension (%)
Any BP measure	94.4	23.0	94.3	23.1	0.4
BP control Systolic < 140 & diastolic < 90	73.1	44.4	73.0	44.4	0.3

Abbreviations: BP, blood pressure; CBOC, community‐based outpatient clinic; FTE, full‐time equivalent; HbA1c, hemoglobin A1c; HMO, health maintenance organization; IHD, ischemic heart disease; LDL, low‐density lipoprotein; MD, medical doctor, NP, nurse practitioner; PQI, prevention quality indicator; Std. Diff., standardized difference; VA, Veterans Affairs; VAMC, VA Medical Center.

There were larger differences in rural/urban and geographic regions between the two groups. NP patients were more likely to reside in the Midwest than MD patients (30.0 percent vs 19.0 percent), while MD patients were more likely to reside in the South than NP patients (50.4 percent vs 32.4 percent). NP patients were more likely to reside in rural areas than MD patients (23.9 percent vs 14.7 percent). While there were 53 percent of VAMC patients in the both groups, MD patients were more likely to be with a parent VAMC with more MDs, including primary care and surgical and nonsurgical specialists, than NP patients. For baseline period outcomes, standardized differences were less than 10 for all measures.

3.2. Utilization and cost outcomes

3.2.1. Descriptive analysis

The only outcome with a standardized difference greater than 10 was use of specialty care (Table 2). Most patients had at least one primary care visit (91.5 percent for NP patients vs 92.7 percent for MD patients). About one‐third had ≥1 specialty care visit (31.2 percent for NP patients vs 36.5 percent for MD patients). Less than 30 percent had ≥1 mental health visit (28.1 percent for NP patients vs 28.9 percent for MD patients). About two‐third of patients had any metabolic test (63.2 percent for NP patients vs 64.5 percent for MD patients), while only a small proportion had any echocardiogram or stress test (4.6 percent for NP patients vs 5.0 percent for MD patients).

Table 2.

Differences in utilization and cost outcomes between patients reassigned to NPs and MDs

Outcome	Unadjusted			Adj. Diff. (NP‐MD) [99% CI]
	NP (n = 110 030)	MD (n = 696 404)	SMD (NP‐MD)
Any outpatient visit (%)
Primary care	91.5	92.7	−4.4	−1.33** [−2.11, −0.55]
Specialty care	31.2	36.5	−11.3	−2.19** [−3.35, −1.03]
Mental health	28.1	28.9	−0.2	−0.06 [−0.88, 0.76]
Number of visits among users by type of visit (counts)
Primary care	4.54	4.86	−0.1	−0.36** [−0.58, −0.13]
Specialty care	4.53	4.91	−0.06	−0.19* [−0.35, −0.03]
Mental health	14.87	14.86	0.003	−0.46 [−1.07, 0.14]
Any test or procedure (%)
Any metabolic panels	63.2	64.5	−0.2	0.04 [−4.83, 4.06]
Any echocardiogram or stress test	4.6	5.0	−1.9	−0.22 [−0.71, 0.28]
Inpatient utilization (%)
Any inpatient	7.9	8.9	−3.7	−0.86** [−1.37, −0.34]
Any inpatient for PQIs	1.1	1.3	−2.3	−0.21** [−0.37, −0.06]
Costs
Outpatient costs	$4447	$4894	−0.06	−$98 [−251, 55]
Total costs	$6972	$7432	−0.02	−$289 [−696, 117]

Abbreviations: Adj. Diff, adjusted difference; MD, medical doctor; NP, nurse practitioner; PQI, prevention quality indicator; SMD, standardized mean difference.

^* P < .01.

^** P < .001.

Fewer than 1‐in‐10 patients had any hospitalization (7.9 percent for NP patients vs 8.9 percent for MD patients). Any hospitalization for PQIs was 1.1 percent for NP patients and 1.3 percent for MD patients, respectively. Mean VA outpatient costs were $4447 for NP patients compared to $4894 for MD patients. Mean total VA costs were $6972 for NP patients and $7432 for MD patients.

3.2.2. Multivariable analysis

The first‐part results showed that NP patients were less likely to have at least one primary care visit (AME = −1.33 percentage points, 99% CI: −2.11 to −0.55 percentage points) and any specialty care (AME = −2.19 percentage points, 99% CI: −3.35 to −1.03 percentage points) than MD patients. Count models showed that NPs patients had fewer primary care visits (AME = −0.36 visits, 99% CI: −0.58 to −0.13) and specialty care visits (AME = −0.19 visits, 99% CI: −0.35 to −0.03) than MD patients among respective subpopulations of patients using each service type. We did not identify statistically significant differences in use of mental health services.

In the analysis of differential effects across level of comorbidity, the probability of at least one primary care visit was lower for NP patients, compared to MD patients with a Gagne score from 0‐2 (AME = −1.76 percentage points, 99% CI: −2.74 to −0.79) (Figure 2). Among users of primary care, NP reassignment was associated with fewer primary care visits for patients with a Gagne score <0 (AME = −0.37 visits, 99% CI: −0.60 to −0.15) and from 0‐2 (AME = −0.35 visits, 99% CI: −0.55 to −0.14), respectively (Table S1). The probability of ≥1 specialty care visit was 2.63 (99% CI: −4.12 to −1.11) and 2.15 (99% CI: −3.31 to −0.99) percentage points lower for NP patients with a Gagne score < 0 and 0‐2, respectively, relative to analogous patients reassigned to MDs. Among users of specialty care, NP reassignment was associated with fewer visits for patients with a Gagne score from 0‐2 (AME = −0.18 visits, 99% CI: −0.33 to −0.03) and from 3‐10 (AME = −0.32 visits, 99% CI: −0.60 to −0.04), respectively.

Figure 2.

Differences in utilization between NP and MD patients by level of comorbidity [Color figure can be viewed at wileyonlinelibrary.com]

NP patients were less likely to have any hospitalization (AME = −0.86 percentage points, 99% CI: −1.37 to −0.34) and any PQI hospitalization (AME = −0.21 percentage points, 99% CI: −0.37 to −0.06). The interaction terms between provider type and Gagne score categories showed consistent NP‐MD differences across levels of comorbidity, with most of the effects occurring in the midrange Gagne categories (0‐2 and 3‐10). We identified no significant differences in any metabolic panel test, any diagnostic echocardiogram or stress test, and total VA cost in both main effect and differential effect estimates. The significant differences in use but not costs were likely due to the relatively small differences in use and the large variance in cost. Finally, while NP reassignment was not associated with outpatient costs overall, a statistically significant difference was identified among patients with a Gagne score from 3‐10 (AME = −$341, 99% CI: −$579 to −$104).

The first stage of the IV model found a partial F‐statistic on the instruments of 36.3. Overall, IV models produced wider CIs than GLM models (Table [Link], [Link]). As a result, no statistically significant differences between NP and MD patients were identified. However, AME estimates for use of primary care and specialty care were directionally similar to analogous GLM estimates.

3.3. Clinical outcomes

3.3.1. Descriptive analysis

All standardized differences in clinical outcomes were less than 10, indicating negligible differences between NP and MD patients (Table 3). Among the diabetes cohort, the majority of patients had any HbA1c test (87.9 percent for NP patients vs 89.3 percent for MD patients), normal HbA1c (87.9 percent for NP patients vs 87.6 percent for MD patients), any LDL test (84.7 percent for NP patients vs 85.6 percent for MD patients), and normal LDL (70.4 percent for NP patients vs 71.7 percent for MD patients). Among the IHD cohort, the majority had any LDL test (80.7 percent for NP patients vs 82.7 percent for MD patients) and normal LDL (74.1 percent for NP patients vs 75.6 percent for MD patients). Among the hypertension cohort, almost all patients had a BP measure (97.4 percent for NP patients vs 97.7 percent for MD patients) and the majority had normal BP (72.1 percent for NP patients vs 72.9 percent for MD patients).

Table 3.

Differences in clinical outcomes between patients reassigned to NPs and MDs

Outcome	Unadjusted			Adj. Diff (NP‐MD) [99% CI]
	NP	MD	SMD (NP‐MD)
Diabetes cohort	(n = 26 363)	(n = 180 934)
Any HbA1c test (%)	87.9	89.3	−4.4	−1.00 [−2.15, 0.17]
HbA1c control: HbA1c < 9 (%)	87.9	87.6	−0.9	−0.02 [−0.72, 0.69]
Any LDL (%)	84.7	85.6	−2.4	−0.25 [−2.16, 1.66]
LDL control: LDL < 100 (%)	70.4	71.7	−2.7	−0.60 [−1.75, 0.55]
Ischemic heart disease	(n = 14 413)	(n = 98 847)
Any LDL Test (%)	80.7	82.7	−5.2	−0.71 [−2.88, 1.45]
LDL control: LDL < 100 (%)	74.1	75.6	−3.6	−0.70 [−1.85, 0.46]
Hypertension	(n = 45 806)	(n = 312 181)
Any BP measurement (%)	97.4	97.7	−2.4	—
BP control (%): systolic < 140 and diastolic < 90	72.1	72.9	−1.8	−0.74 [−2.22, 0.74]

Abbreviations: Adj. Diff, adjusted difference; BP, blood pressure; HbA1c, hemoglobin A1c; LDL, low‐density lipoprotein; MD, medical doctor; NP, nurse practitioner; SMD, standardized mean difference, – means model did not converge.

^* P < .01.

^** P < .001.

3.3.2. Multivariable analysis

Both the GLM and IV analysis indicated NP‐MD differences in clinical outcomes were not statistically significant across all measures (Table 3).

4. DISCUSSION

This study conducted a comprehensive assessment of economic and clinical outcomes between primary care NPs and MDs in the VA system. Study findings suggest utilization and cost of NP primary care were largely comparable to MD primary care.12, 41 While prior studies show mixed results in health care utilization,42, 43 we identified small, but statistically significant reductions in use of primary care and specialty care among patients reassigned to NPs relative to comparable patients reassigned to MDs. The marginal effects for use of any primary care and specialty care translated to 1.4 percent and 6.1 percent of the respective means. Similarly, we found NP patients were less likely to have any hospitalization overall and for PQIs than MD patients. While the differences were small, these findings are consistent with other studies.41, 42, 43, 44

One potential explanation for fewer hospitalizations, especially for PQIs, may be increased access to NP providers due to their smaller panel size. An NP’s panel size at VA is expected to be 75 percent of an MDs in order to give NPs sufficient time to manage their patients.45 Reduced panel size may translate to greater access and time spent with individual patients, which in turn may translate into fewer hospitalizations. Differences in utilization were present across the risk distribution, suggesting NPs may be effective at reducing outpatient utilization and hospitalizations for different subpopulations of patients.

The novel approach employed in this study compared outcomes of care provided by NPs and MDs using natural variation arising from the reassignment of patients following the departure of an existing provider. Characteristics of patients reassigned to NPs and MDs were generally balanced, which is supportive of pseudorandom reassignment. Nevertheless, we conducted sensitivity analyses using IV models to address the potential for other unobserved confounders; finding outcomes among NP patients were no worse than those of MD patients. However, IV models have the limitation of less precise AME estimates.

In contrast to prior studies that showed NPs were more likely to order tests,14, 46, 47, 48, 49 this study showed no significant differences in the ordering of two diagnostic tests (metabolic panel and echocardiogram or stress test). Previous studies show mixed results in cost of NP care,14, 50, 51, 52, 53, 54 while our study shows that NP and MD patients had similar outpatient cost and total cost. Finally, we found NPs and MDs achieved similar clinical outcomes among patients with chronic diseases, including diabetes, IHD, and hypertension, which is consistent with prior VA and non‐VA studies.2, 17, 55, 56, 57

Several systematic reviews have highlighted the challenges in summarizing the literature examining outcomes between patients receiving care from NPs compared to MDs because of methodological limitations, including small sample sizes.12, 42 Our study extends previous research with several methodological contributions. First, we addressed patient selection bias by focusing on patients of MDs who were reassigned to a new NP or MD when their MD providers left VA primary care practice. This is important because provider reassignment likely introduced some pseudorandom variation into the type of provider responsible for patients’ primary care. It is pseudorandom in that at least some of the assignment was due to idiosyncratic operational considerations (eg, patients assigned to whichever PCPs have slots available for new patients) rather than to specific characteristics of the patient that might affect health care outcomes. Although we do not observe the decision process for reassigning patients, the average MD PCP panel at the VA consists of 1200 patients; reassignment of this many patients based on characteristics of the patient would be a complex undertaking for a busy clinic. Second, this study examined a large, nationally representative cohort of 806 434 patients and assessed outcomes in one year after reassignment to a new PCP. This expands on prior research that examined utilization and clinical outcomes simultaneous to provider assignment.2 Finally, we identified patient–PCP relationships using panel assignments tracked within VA administrative data. PCPs in VA are responsible for managing overall health of their patient panel, irrespective of whether a patient receives care from a different PCP at a given encounter. Specifically, PCPs are evaluated on whether they meet specific performance targets for their patient panel.58 This improves on prior research that assigns patients based on the provider of record for a single encounter or the provider rendering the majority of visits, which may not reflect the provider responsible for managing the patient’s care.

The VA system provides a unique environment to assess effectiveness of care provided by NPs and offers several advantages. Because NPs comprise approximately 20 percent of all PCPs in VA, there is sufficient sample size to examine differences in outcomes between patients managed by MDs and NPs and to explore whether differences vary across patient characteristics.2 Like MDs, VA primary care NPs have their own patient panels and have independent practice authority. The VA is federally funded and, while it bills third party payers, the vast majority of its funding is from federal appropriations. In addition, the VA electronic health records system comprehensively captures services provided by NPs, identified in administrative data using distinct provider type codes. In contrast, commercial or Medicare claims may underestimate care provided by NPs because of higher reimbursement rates for MDs; a significant proportion of NP services are billed through their supervising MDs.59

Taken together, study findings suggest that the general use of NPs as primary care providers may be a high‐value solution to increasing access to care, which has been emphasized as a top VA priority.60 We also found that comparable or better outcomes achieved at similar costs for patients across differing levels of comorbidity, suggesting NPs as PCPs need not be limited to less complex patients. For health systems deciding how to expand capacity, consideration should also be given to differences in labor costs, panel sizes, and spillover effects outside of primary care, such as referrals, each of which was outside the scope of this study. Future research should also examine potential heterogeneous differences in outcomes between patients reassigned to NPs vs MDs across subpopulations defined by characteristics other than comorbidity.

This study has several limitations. First, this was an observational study, not a randomized controlled trial. However, we applied methodological approaches to improve causal inference including the use of a study design that leveraged pseudorandom variation in patient assignment to primary care providers introduced through the attrition of VA PCPs who left primary care practice, in order to address potential patient selection bias. Sensitivity analyses using IV methods also supported the finding that care provided by NPs was at least comparable to MDs. Second, this study examined the population of VA primary care patients, which may differ from other non‐VA populations because of unique VA enrollment requirements (eg, prior military service). However, prior research has found similarity between VA and non‐VA populations, including Medicare.61 Third, we did not measure differences between NP and MD patients in their satisfaction with their PCP, and we did not measure requests to be reassigned to a new PCP. Fourth, we assumed that the administratively reassigned PCP was responsible for differences in patient outcomes; we did not address the extent to which other clinicians in primary or specialty care helped orchestrate a patient’s care. Fifth, future studies should assess whether differences in clinical outcomes are evident over follow‐up times longer than the 1 year in the present study. Finally, we did not observe health care use provided by community VA providers through a Veteran’s Medicare benefits. This could affect the results if patients reassigned to MDs and NPs differed in their subsequent reliance on the VA for care.

In conclusion, this study shows that NP patients did not differ from MD patients in VA health care costs, but had less utilization of primary care, specialty care, and inpatient services.

Further, NP and MD reassigned patients achieved similar quality of care in chronic disease management. This study supports the evidence that use of NPs can improve access to primary care with similar quality and cost of care.

CONFLICT OF INTEREST

All other authors declare no conflict of interest.

Supporting information

 

 

 

Liu C‐F, Hebert PL, Douglas JH, et al. Outcomes of primary care delivery by nurse practitioners: Utilization, cost, and quality of care. Health Serv Res. 2020;55:178–189. 10.1111/1475-6773.13246