Comparing Use of Low-Value Health Care Services Among U.S. Advanced Practice Clinicians and Physicians

Abstract

Background

Many physicians believe that advanced practice clinicians (APCs [nurse practitioners and physician assistants]) provide care of relatively lower value.

Objective

To compare use of low-value services among U.S. APCs and physicians.

Design

Service use after primary care visits was evaluated for 3 conditions after adjustment for patient and provider characteristics and year. Patients with guideline-based red flags were excluded and analyses stratified by office-versus hospital-based visits, acute versus nonacute presentations, and whether clinicians self-identified as the patient’s primary care provider (PCP).

Setting

National Ambulatory Medical Care Survey (NAMCS) and National Hospital Ambulatory Medical Care Survey (NHAMCS), 1997 to 2011.

Patients

Patients presenting with upper respiratory infections (URIs), back pain, or headache.

Measurements

Use of guideline-discordant antibiotics (for URIs), radiography (for URIs and back pain), computed tomography (CT) or magnetic resonance imaging (MRI) (for headache and back pain), and referrals to other physicians (for all 3 conditions).

Results

12 170 physician and 473 APC office-based visits and 13 359 physician and 2947 APC hospital-based visits were identified. Although office-based clinicians saw similar patients, hospital-based APCs saw younger patients (mean age, 42.6 vs. 45.0 years; P < 0.001), and practiced in urban settings less frequently (49.7% vs. 81.7% of visits; P < 0.001) than hospital-based physicians. Unadjusted and adjusted results revealed that APCs ordered antibiotics, CT or MRI, radiography, and referrals as often as physicians in both settings. Stratification suggested that self-identified PCP APCs ordered more services than PCP physicians in the hospital-based setting.

Limitation

NHAMCS reflects hospital-based APC care; NAMCS samples physician practices and likely underrepresents office-based APCs.

Conclusion

APCs and physicians provided an equivalent amount of low-value health services, dispelling physicians’ perceptions that APCs provide lower-value care than physicians for these common conditions.

Primary Funding Source

U.S. Health Services and Research Administration, Ryoichi Sasakawa Fellowship Fund, and National Institutes of Health.

The U.S. health care system faces a looming shortage of primary care providers (PCPs) (1, 2), and estimates suggest that it might reach up to 20 000 physicians by 2020 (3). The causes of the projected shortage include both demand- and supply-related factors, such as an increased demand for primary care services from the millions of newly insured patients after passage of the Patient Protection and Affordable Care Act, as well as fewer trainees electing to enter primary care (2, 4–7).

In response to these trends, many have advocated for expanding the role of advanced practice clinicians (APCs [nurse practitioners and physician assistants]) in primary care as a potential policy solution to mitigate the workforce shortage (1, 8–10). These practitioners represent the fastest-growing segment of the primary care workforce, and they are more likely to serve minority and underserved patients (1). Studies suggest that APCs may be a cost-saving alternative to physicians in primary care, even after adjustment for case complexity and productivity, mainly because of their lower salaries (10–13). Although previous research and systematic reviews suggest that APCs provide care of the same quality as that of physicians (14–17), little is known about their effect on providing wasteful or low-value health services. According to a recent national survey, most physicians believe that APCs provide lower-quality care than they do, and nearly one quarter think that expanding nurse practitioners’ roles in U.S. practice would decrease the efficiency and value of health care (18).

Despite these perceptions, no previous research compared APCs and physicians directly with regard to ordering potentially guideline-discordant and low-value health services, which has important implications for the quality and efficiency of care delivery in the U.S. health care system. For example, an estimated 30% of U.S. health care spending is wasteful (19), and much of this waste is driven by use of low-value health services, which is defined as patient care that typically portends a greater probability of harm than benefit (20–22). In this context, we used national data on ambulatory visits to providers to compare the use of potentially low-value health services between APCs and physicians in the management of common conditions in the primary care setting.

Methods

Data Sources

We used data from the National Ambulatory Medical Care Survey (NAMCS) and the National Hospital Ambulatory Medical Care Survey (NHAMCS) on ambulatory visits to clinicians from 1 January 1997 through 31 December 2011. The NAMCS comprises probability samples of outpatient visits to nonfederal, office-based physician practices, whereas the NHAMCS contains data on visits to nonfederal, hospital-based outpatient departments, including outpatient clinics and emergency departments. Developed in parallel, the NAMCS and NHAMCS have common designs, variables, and visit weights. They both are structured to represent ambulatory visits to physicians across the United States, but they also contain data on visits to APCs (23, 24). Importantly, the NAMCS samples physician practices, some of which include APCs, but does not include independently practicing APCs. The NHAMCS samples hospital-based outpatient departments, which more commonly include APCs, including those who practice independently. Thus, APCs in our sample are represented predominantly by the NHAMCS data. Further details on the NAMCS and NHAMCS are available in the Methods section in the Appendix (including Appendix Figures 1 and 2, available at www.annals.org) and from the National Center for Health Statistics (23).

Data Collection Procedures

The data are collected by a standardized survey form, which the provider or other staff completes soon after each ambulatory visit. The NAMCS and NHAMCS both collect information on the patient’s primary reason for the visit (for example, the chief symptom); 2 other nonprimary reasons for the visit; up to 3 diagnoses, derived from the International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM); the payer for the visit (for example, commercial or Medicare insurance); whether the provider self-identified as the patient’s PCP; patient and provider demographic information; tests and treatments ordered; and medications listed during the visit.

Study Sample

We selected outpatient visits (excluding those to the emergency department) for 3 common conditions seen in the primary care setting (for example, general medicine, family medicine, or internal medicine physician offices or outpatient hospital departments; see the Methods section in the Appendix for more details) that frequently are associated with use of low-value services, namely upper respiratory infections (URIs [such as acute bronchitis and pharyngitis]), back pain, and headache.

For all chief symptoms, the NAMCS and NHAMCS provide information on the reasons for patient visits, divided into 5 categories: new-onset symptoms (<3 months duration), acute-on-chronic flare-up, chronic routine visit, routine or preventive care, and pre- or postoperative visit. We defined acute visits by combining the categories new-onset symptoms and acute-on-chronic flare-up.

APCs

Both the NAMCS and NHAMCS identify the clinician who provided care during the visit as a physician, a nurse practitioner, a physician assistant, a nurse midwife, a registered or licensed practical nurse, or another provider. For this study, we excluded registered and licensed practical nurses, other providers, and midwives (see the Appendix for details on excluding midwives), because they typically do not have the authority to order such services as antibiotic treatment, imaging, and referrals. For our primary analysis, we classified nurse practitioners and physician assistants as APCs. As a secondary analysis, we also evaluated results by APC visit subtype (for example, APCs seeing the patient alone vs. alongside the physician) and profession subtype (for example, nurse practitioner vs. physician assistant) and compared them with results from physicians seeing patients without an APC, who made up the reference group (Appendix). Although training of nurse practitioners and physician assistants, as well as the regulatory environment in which they practice, differs between these groups and among the states in which they practice, we chose a combined APC variable, because both types of clinicians often play similar roles in the primary care setting (25) and because it made interpretation easier. In addition, the findings of the subgroup categories described earlier largely were consistent with our overall results.

Outcome Measures

We studied 3 types of outcomes widely considered to be low-value services in most cases, as identified in previous studies or guidelines—use of antibiotics (for URIs); plain radiography (for URIs and back pain); and advanced imaging, including both magnetic resonance imaging (MRI) and computed tomography (CT) (for back pain and headache)—as well as 1 overall outcome that applied to all 3 conditions. Our cohort excluded visits from patients with red flags (for example, symptoms or diagnoses of neurologic deficit in those with back pain or headache, concomitant chronic obstructive pulmonary disease, or HIV in those with acute bronchitis) and was derived from previous research by our group and others using NAMCS and NHAMCS data (20, 21, 26, 27), which were based on relevant clinical and regulatory agency guidelines during the study period (28–32). In addition, we evaluated referrals to other physicians (for all 3 conditions), which generally are not required for these illnesses and might indicate more unnecessary downstream utilization.

Potential Confounders

From the data recorded for each visit, we assessed for patient age (measured as a continuous variable), sex, race or ethnicity, modified Charlson comorbidity count (33), symptom acuity, insurance status, urban location, geographic region, and year. The clinicians who completed the survey defined race and ethnicity as 2 separate variables; we reclassified patients to a single 4-level race or ethnicity variable (white, black, Hispanic, and other).

We also considered whether the clinician self-identified as the patient’s PCP, which we determined as follows: From 1997 through 2004, the NAMCS and NHAMCS asked, “Are you the patient’s primary care physician?” From 2005 through 2011, the NAMCS changed the question to “Are you the patient’s primary care physician/provider?” During 2005 and 2006, the NHAMCS asked, “Are you the patient’s primary care physician/provider?” Finally, in 2007 to 2011, the NHAMCS changed the question to “Is this clinic the patient’s primary care provider?”

For the NAMCS analyses, we also identified practice setting types, which we combined into 3 main categories: private solo or group continuity practices; noncontinuity, freestanding clinics or urgent care centers; and other continuity-type clinics (such as federally qualified health centers, community health centers, and nonfederal government clinics).

Statistical Analysis

We pooled all years and conducted separate analyses for our NAMCS and NHAMCS samples because they each use distinct sampling approaches with regard to APC practice, as explained earlier. All analyses were weighted and accounted for the complex sampling. We used chi-square tests and unadjusted linear regression to compare visit characteristics among APCs and physicians, respectively. For our unadjusted analysis, we pooled all years within each outcome category and used unadjusted logistic regression models for each outcome, focusing on APC provider as the exposure of interest. For adjusted analyses, we estimated similar models but included the aforementioned demographic, clinical, and provider variables to control for potential confounders. We also controlled for year.

To evaluate important potential effect modification based on symptom acuity and whether the clinician self-identified as the PCP, we also stratified our results by symptom acuity and PCP status. Because of the wording change associated with the PCP variable, as noted earlier, we conducted sensitivity analyses to ensure that stratified results were similar when we compared data after 2005 with our main findings (see the Appendix for details). Finally, we also evaluated practice settings in which APCs predominantly practiced independently versus those in which physicians practiced alone (Appendix).

We performed all our analyses with SAS-Callable SUDAAN, version 11.0 (RTI International), using SUDAAN subpopulation procedures. These functions use information from the entire NAMCS and NHAMCS sample to account for the complex study design and sampling weights to produce national estimates (24).

The Harvard Medical School Committee on Human Studies determined that this study was exempt from review.

Role of the Funding Source

The funders had no role or influence in the design and conduct of the study; collection, management, analysis, and interpretation of the data; decision to publish; or preparation, review, or approval of the manuscript.

Results

We identified 28 949 U.S. primary care visits for the 3 conditions, including 25 529 physician and 3420 APC visits, representing an estimated 681 million total visits from 1997 to 2011. Of these, 89.9% reflected visits to clinicians in office-based physician practices (data from the NAMCS) and 10.1% to those practicing in hospital-based outpatient clinics (data from the NHAMCS). Table 1 presents relevant patient and visit characteristics by clinician subtype, and Appendix Tables 1 and 2 (available at www.annals.org) present these characteristics by additional APC subgroupings. Although office-based APCs and physicians saw largely similar patients, hospital-based APCs treated younger patients (mean age, 42.6 vs. 45.0 years; P < 0.001) and delivered care in an urban setting less frequently (49.7% vs. 81.7% of visits; P < 0.001) than hospital-based physicians.

Table 1.

Patient Demographic Characteristics, by Clinician Type^*

Characteristic	Office-Based			Hospital-Based
	Physicians	APCs	P Value	Physicians	APCs	P Value
Sample/weighted population, n/N	12 170/586 million	473/25.8 million		13 359/54.2 million	2947/14.6 million
Mean age, y	48.9	47.9	0.34	45.0	42.6	<0.001
Female	62.7	63.6	0.78	64.6	66.3	0.27
Race/ethnicity			0.067			<0.001
White	76.9	77.2		70.6	79.6
Black	10.3	10.8		16.7	9.2
Hispanic	8.3	10.0		9.7	7.5
Other	4.6	1.9		3.0	3.6
Mean Charlson comorbidity count	0.047	0.044	0.77	0.049	0.022	<0.001
Acute symptoms	72.7	77.5	0.096	75.0	88.0	<0.001
Insurance			0.077			0.050
Private	57.6	65.1		46.0	42.0
Medicare/Medicaid	26.7	21.0		33.4	32.7
Other	15.8	13.9		20.6	25.3
Provider identifies as PCP	79.9	71.0	0.21	52.8	52.5	0.96
Urban location	82.1	78.9	0.46	81.7	49.7	<0.001
Region			0.040			0.31
Northeast	19.1	17.1		17.6	17.5
Midwest	24.2	23.6		37.2	25.5
South	35.5	47.6		30.8	40.2
West	21.2	11.7		14.4	16.8
Symptom group
URIs	35.6	44.1	0.015	37.1	42.0	0.016
Back pain	47.4	39.9	0.011	46.0	41.5	0.029
Headache	18.2	17.8	0.85	18.1	18.1	0.99

APC = advanced practice clinician; PCP = primary care provider; URI = upper respiratory infection.

^*Values are percentages unless otherwise indicated and may not sum to 100 due to rounding. Boldface values indicate comparisons with P < 0.05.

Unadjusted Frequencies of Use

In the unadjusted analyses (Table 2), APCs were not significantly more likely to order antibiotics, CT or MRI, or radiography, or to refer patients to other physicians, than physicians in office- or hospital-based settings, although the rates of these outcomes differed across these settings. We also examined visits among the APC subgroups (Appendix Tables 3 and 4, available at www.annals.org), which revealed few differences.

Table 2.

Unadjusted Frequencies of Utilization, by APC Versus Physician Status and Office-Versus Hospital-Based Clinics^*

Variable	Physicians	APCs	P Value
Weighted population, n
Office-based	586 million	25.8 million
Hospital-based	54.2 million	14.6 million
Antibiotics
Office-based	1988/4074(49.1)	98/191 (51.3)	0.71
Hospital-based	2233/4742 (51.6)	665/1281 (53.9)	0.46
CT/MRI
Office-based	511/8156(6.3)	12/283(3.3)	0.111
Hospital-based	87/1680 (7.5)	639/8666 (8.5)	0.52
Radiography
Office-based	45/399 (9.4)	987/10 093(10.1)	0.74
Hospital-based	1390/11 015 (13.2)	299/2436(11.6)	0.24
Referral to other physician
Office-based	1066/12 170 (8.1)	44/473 (9.2)	0.54
Hospital-based	2546/13 359 (18.4)	582/2947 (21.5)	0.35

APC = advanced practice clinician; CT = computed tomography; MRI = magnetic resonance imaging.

^*Values are sample numerator/sample denominator (percentages) unless otherwise indicated. Proportions do not match percentages because the percentages represent national estimates based on population weighting.

Multivariable-Adjusted and Stratified Results

After multivariable adjustment, practice patterns among APCs and physicians remained consistent with the unadjusted findings in both office- and hospital-based settings (Table 3). Appendix Tables 5 and 6 (available at www.annals.org) present multivariable-adjusted results by visit and profession category.

Table 3.

Multivariable-Adjusted Frequencies of Utilization, by APC Versus Physician Status and Office-Versus Hospital-Based Clinics^*

Variable	Physicians	APCs	P Value
Sample/weighted population, n/N
Office-based	12 170/586 million	473/25.8 million
Hospital-based	13 359/54.2 million	2947/14.6 million
Antibiotics
Office-based	49.2	49.4	0.96
Hospital-based	52.0	52.3	0.92
CT/MRI
Office-based	5.7	2.8	0.086
Hospital-based	7.3	6.4	0.48
Radiography
Office-based	9.8	8.7	0.65
Hospital-based	12.2	11.8	0.73
Referral to other physician
Office-based	7.4	8.1	0.64
Hospital-based	14.7	18.6	0.091

APC = advanced practice clinician; CT = computed tomography; MRI = magnetic resonance imaging; NAMCS = National Ambulatory Medical Care Survey.

^*Values are percentages unless otherwise indicated. Models are adjusted for age, sex, race/ethnicity, insurance status, modified Charlson comorbidity count, symptom acuity, and whether the provider self-identified as the patient’s primary care provider. Office-based (NAMCS) models also adjusted for practice setting.

Table 4 presents results stratified by PCP versus non-PCP visits. Hospital-based PCP APCs ordered more antibiotics (52.8% vs. 46.0%; P = 0.043) and made more referrals (11.8% vs. 8.3%; P = 0.018) than hospital-based PCP physicians. However, only referral differences remained significant in a sensitivity analysis of data recorded in the surveys from 2005 to 2011, after the wording change in the question regarding clinicians self-identifying as PCPs (see the Appendix for details). Test results for interaction were positive only for antibiotic use (P = 0.019 for interaction), suggesting that differences in antibiotic use between PCP APCs and PCP physicians were greater than those between non-PCP APCs and non-PCP physicians in the hospital setting. Table 5 presents results stratified by acute versus nonacute visits. Symptom-acuity stratification demonstrated that hospital-based APCs referred patients to other physicians for nonacute care more often than hospital-based physicians (20.3% vs. 11.4%; P = 0.022). Tests for interaction between acuity and APCs were nonsignificant for all outcomes. Practice settings with more independent APCs had results that were not substantively different from our main findings. Detailed results are presented in Appendix Table 7 (available at www.annals.org).

Table 4.

Multivariable-Adjusted Frequencies of Utilization, by APC Versus Physician Status, Office-Versus Hospital-Based Clinics, and PCP Versus Non-PCP Status^*

Variable	PCP			Non-PCP
	Physicians	APCs	P Value	Physicians	APCs	P Value
Sample/weighted population, n/N
Office-based	9592/468 million	366/18.3 million		2578/118 million	107/7.5 million
Hospital-based	6547/28.6 million	1410/7.6 million		6812/25.6 million	1537/6.9 million
Antibiotics
Office-based	48.7	51.9	0.52	50.9	45.2	0.61
Hospital-based	46.0	52.8	0.043	56.9	51.1	0.23
CT/MRI
Office-based	5.7	3.0	0.24	3.0	4.2	0.67
Hospital-based	7.1	6.4	0.70	4.4	5.8	0.34
Radiography
Office-based	10.0	10.9	0.71	8.5	3.3	0.097
Hospital-based	10.5	11.1	0.66	13.9	12.5	0.49
Referral to other physician
Office-based	7.6	7.8	0.88	6.6	8.6	0.57
Hospital-based	8.3	11.8	0.018	25.1	29.6	0.35

APC = advanced practice clinician; CT = computed tomography; MRI = magnetic resonance imaging; NAMCS = National Ambulatory Medical Care Survey; PCP = primary care provider.

^*Values are percentages unless otherwise indicated. Boldface values indicate comparisons with P < 0.05. Models are adjusted for age, sex, race/ethnicity, insurance status, modified Charlson comorbidity count, and symptom acuity. Office-based (NAMCS) models also adjusted for practice setting.

Table 5.

Multivariable-Adjusted Frequencies of Utilization, by APC Versus Physician Status, Office-Versus Hospital-Based Clinics, and Acute Versus Nonacute Visits^*

Variable	Acute			Nonacute
	Physicians	APCs	P Value	Physicians	APCs	P Value
Sample/weighted population, n/N
Office-based	8804/427 million	360/20 million		3366/160 million	113/5.8 million
Hospital-based	9847/40.7 million	2581/12.8 million		3512/13.6 million	366/1.7 million
Antibiotics†
Office-based	51.1	52.1	0.87	34.6	26.7	0.60
Hospital-based	54.1	53.8	0.93	22.9	29.9	0.40
CT/MRI
Office-based	6.8	2.7	0.061	3.7	3.4	0.91
Hospital-based	7.9	6.4	0.30	5.2	6.4	0.52
Radiography
Office-based	10.7	9.0	0.49	7.0	8.3	0.76
Hospital-based	12.0	11.6	0.75	11.7	12.1	0.86
Referral to other physician
Office-based	7.4	8.1	0.65	7.3	8.2	0.80
Hospital-based	15.5	18.5	0.27	11.4	20.3	0.022

APC = advanced practice clinician; CT = computed tomography; MRI = magnetic resonance imaging; NAMCS = National Ambulatory Medical Care Survey; PCP = primary care provider.

^*Values are percentages unless otherwise indicated. Boldface values indicate comparisons with P < 0.05. Models are adjusted for age, sex, race/ethnicity, insurance status, modified Charlson comorbidity count, and whether the provider self-identified as the patient’s PCP. Office-based (NAMCS) models also adjusted for practice setting.

^†Positive for interaction (P = 0.019), suggesting that differences in antibiotic use between PCP APCs and PCP physicians were greater than those between non-PCP APCs and non-PCP physicians in the hospital setting.

Discussion

This large national comparison between APCs and physicians on the use of potentially low-value health services revealed that APCs and physicians in both office- and hospital-based primary care settings provided equivalent amounts of guideline-discordant low-value care, including antibiotics, CT or MRI, plain radiography, and referrals to other physicians. Moreover, our results remained largely consistent when stratified by important potential effect modifiers, such as symptom acuity and whether the clinician self-identified as the PCP, although results did suggest that self-identified PCP APCs used more services than PCP physicians in the hospital-based setting. These findings have important implications for ongoing efforts to improve the quality and value of health care delivered in the United States as well as for the expanding role of APCs in primary care delivery.

Previous studies and meta-analyses showed that APC-provided care is similar in quality to that of physician-provided care, with equal or better outcomes (15). Despite this evidence, a recent national survey revealed that most physicians believe APCs provide lower-quality care than physicians, and nearly one quarter of those surveyed think expanding nurse practitioners’ role in U.S. practice would decrease the efficiency and value of care (18). Few studies, however, have compared APCs and physicians specifically in providing inefficient or low-value care. The existing literature is limited and shows mixed results, with some studies showing increased value and others showing no net change or lower value associated with APCs (15, 34, 35). Moreover, these studies focused mainly on health care costs, and few investigated the use of guideline-discordant and low-value health services as we did in our study. As debate continues over whether APCs’ scope of practice should expand (36), our analysis adds important data to the literature, suggesting that U.S. APCs seem to provide care equal to that of physicians in value and efficiency of delivery.

Our finding that APCs order antibiotics, CT or MRI, radiography, and referrals as frequently as physicians is reassuring given recent efforts to expand the number of APCs, as well as their role, to meet the increasing demand for primary care while the primary care physician workforce continues to shrink. Our results on antibiotic use among APCs and physicians are consistent with those of studies from a decade ago (37, 38). Our study builds on previous work by also demonstrating the similarities between APCs and physicians regarding potentially low-value referrals they make for conditions commonly encountered in primary care. The NAMCS and NHAMCS data we used also provide better clinical granularity and validity than earlier claims-based research, which frequently underrepresented APCs, who often bill under a physician’s name (39). The lack of differences in imaging use between APCs and physicians contrasts with recent research using Medicare claims data that showed greater imaging use among APCs than physicians (40). That study, however, included patients with multiple different presenting complaints of varying complexity. Consistent with our study, the authors found little to no difference among patients presenting with back pain or URI, which are relatively straightforward conditions with clearly defined guidelines. These findings are consistent with the hypothesis that APCs order imaging more frequently for patients whose conditions might be more complex than those we studied or for which guidelines are lacking.

Our analysis is subject to several limitations. First, because the NAMCS samples ambulatory visits to physician practices, it may exclude visits to more independent APCs or to APC-led practices, clearly underrepresenting APC care. However, we note that by law, physician assistants cannot practice without physician supervision, and previous research suggests that few nurse practitioners lead their own practices nationally (39). In addition, because the NHAMCS samples visits to outpatient departments rather than to physicians, it is considered more nationally representative of APCs in those settings and should be generalizable to APCs who practice at nonfederal outpatient hospital clinics (41). Moreover, although the NHAMCS represents a smaller proportion of care than office-based visits, our finding that NAMCS results were consistent with our NHAMCS results is reassuring.

Second, we could not identify low-value care precisely because we lacked granular clinical data. Thus, in some instances, such care might be justified. However, prominent clinical guidelines, specialty societies, and regulatory bodies previously identified these measures as being low-value in most circumstances. Moreover, our analyses excluded clinical red flags and controlled for age, symptom acuity, and comorbid conditions; thus, it is unlikely that variations in case complexity are confounding our results.

Third, we lacked longitudinal data at the patient level, as the NAMCS and NHAMCS are cross-sectional, visit-based surveys; thus, we could not assess whether various tests or referrals ordered were completed. However, we have no reason to suspect that completion rates would differ between APCs and physicians.

Fourth, we also could not account for variations in state-level scope-of-practice laws across the United States, which may limit an APC’s ability to order diagnostic tests (40, 42, 43). However, controlling for larger geographic regions did not affect our findings, and importantly, a recent rigorous analysis of the effect of state scope-of-practice laws on APC utilization revealed only a modest association at best (44).

We found that APCs and physicians ordered potentially guideline-discordant and low-value health services with similar frequency, dispelling physicians’ perceptions that APCs provide lower-value care than they do. We also acknowledge, however, that attributes of specific clinicians or practice settings might lead to practice patterns that differ from those we observed overall. As APCs rapidly expand their role in primary care, these findings have important implications for clinicians, practice leaders, and policymakers who have a stake in improving access to primary care services and ensuring the delivery of high-value care.

Appendix: Further Details and Subanalyses

Methods

Complex Design of the NAMCS and NHAMCS

The NAMCS and NHAMCS both use a multistage probability sample design to acquire nationally representative samples of U.S. ambulatory visits to physicians (45). During the first stage, 112 geographic primary sampling units were selected from among those originally used in the National Health Interview Survey. For the second stage, physician office practices (NAMCS) or hospitals (NHAMCS) were chosen within these primary sampling units. In the last stage, a subset of visits to these office practices or outpatient departments was sampled during a prespecified period. This complex design allowed calculation of national-level estimates for physician visits and associated SEs by using the survey weights provided by the National Center for Health Statistics. See Appendix Figures 1 and 2 for details on imputation methods to handle missing values.

Excluding Midwives

From 1997 through 2000, the NAMCS and NHAMCS included midwives and nurse practitioners as separate variables; from 2001 through 2011, they were combined into a single element. We examined visits to midwives and nurse practitioners for the selected conditions from 1997 to 2000 in our cohort and identified 14 midwife and 709 nurse practitioner visits, suggesting that the combined variable from 2001 to 2011 captures predominantly nurse practitioner visits in our primary care–focused cohort.

NAMCS and NHAMCS Wording Change on Whether Clinician Self-Identified as the PCP

From 1997 through 2004, the NAMCS and NHAMCS asked, “Are you the patient’s primary care physician?” Therefore, some APCs may have answered “no” because they were not physicians. From 2005 through 2011, the NAMCS question was changed to “Are you the patient’s primary care physician/provider?” In addition, from 2005 through 2006, the NHAMCS question became “Are you the patient’s primary care physician/provider?” Finally, from 2007 through 2011, the NHAMCS asked, “Is this clinic the patient’s primary care provider?” We therefore compared the frequency of APCs self-identifying as the PCP from 1997 to 2004 versus 2005 to 2011. We found that the frequency increased on both the NAMCS (59.5% from 1997 to 2004, to 78.3% from 2005 to 2011; P = 0.21) and NHAMCS (42.4% from 1997 to 2004, to 59.6% from 2005 to 2011; P = 0.030). Therefore, we evaluated whether our findings that PCP APCs ordered more antibiotics and referrals than PCP physicians in the hospital setting remained true after the surveys’ wording change starting in 2005. We found that although differences in referral patterns remained significant (18.2% among PCP APCs vs. 10.5% among PCP physicians; P < 0.001), differences in antibiotic use no longer were significant (47.8% among PCP APCs vs. 50.9% among PCP physicians; P = 0.60) in the hospital setting from 2005 to 2011.

Selecting Generalists in the Primary Care Setting by Using NAMCS and NHAMCS Data

The NAMCS provides detailed specialty data, allowing us to restrict our analysis to general, family, and internal medicine physicians and APCs. Although the NHAMCS does not provide detailed specialty information on the physician or APC, it does allow restriction of the analysis to “general medical clinics” as opposed to surgical or other specialty clinics. General medical clinics exclude most specialists who treat URIs (such as otolaryngologists), back pain (such as orthopedic surgeons, neurosurgeons, anesthesiologists, and pain specialists), and headache (such as neurologists and pain specialists).

To confirm that general medicine clinics excluded relevant specialists (general medical clinics include some other medical subspecialists), we used the NAMCS to examine all visits for the identified symptoms to determine the extent to which these symptoms were seen by specialists. Most patients presenting with back pain, headache, and URIs were treated by generalists, internists, and family physicians, with the rest being seen predominantly by orthopedic/neurosurgeons; neurologists; and ear, nose, and throat physicians, respectively, none of whom are part of the NHAMCS general medical clinics (rather, they are part of “surgical” and “other” clinics).

Results

Subcategory Analysis

Findings in the subcategory analysis were not substantively different from our main results, with few exceptions (see Appendix Tables 3 to 6 for details). For example, nurse practitioners seeing patients alongside physicians were less likely to order antibiotics (with multivariable-adjusted rates of 27.3% vs. 49.2%; P = 0.034) than physicians alone in the office-based setting (Appendix Table 5). However, nurse practitioners seeing patients alongside physicians were more likely to refer patients to other physicians (multivariable-adjusted rates of 23.5% vs. 14.8%; P = 0.044) than physicians in the hospital-based setting (Appendix Table 6).

Practice Setting Sensitivity Analyses

Adding the practice setting variable to our NAMCS multivariable models did not change our results, and interacting practice setting with APC provider in our NAMCS subset revealed no significant findings, suggesting that practice setting in the NAMCS was not associated with differences in provider behavior. In addition, when we evaluated physician practices or hospitals in which nurse practitioners and physician assistants predominantly practiced independently (Appendix Table 7; see columns for “Nurse Practitioners: Always Alone” and “Physician Assistants: Always Alone”), our findings were consistent with our overall results, with few exceptions. For example, physician assistants who predominantly see patients alone ordered antibiotics more frequently than physicians practicing alone (71.6% vs. 49.1%; P = 0.020).

We note, however, that these subanalyses are imperfect because of small sample sizes, and we remain concerned about multiple testing and potential bias in creating these categories. Hence, we limit these results to the Appendix.

Appendix Figure 1.

NAMCS missing data procedures.

Some missing data were imputed by randomly assigning a value from a patient record survey form with similar characteristics. Imputations were based on physician specialty, geographic region, and 3-digit ICD-9-CM codes for primary diagnosis. ICD-9-CM = International Classification of Diseases, Ninth Revision, Clinical Modification; NAMCS = National Ambulatory Medical Care Survey.

Appendix Figure 2.

NHAMCS outpatient department missing data procedures.

Some missing data were imputed by randomly assigning a value from a patient record survey form with similar characteristics. Imputations were based on outpatient department volume by clinic type, geographic region, and 3-digit ICD-9-CM codes for primary diagnosis. ICD-9-CM = International Classification of Diseases, Ninth Revision, Clinical Modification; NHAMCS = National Hospital Ambulatory Medical Care Survey.

Appendix Table 1.

Demographic Characteristics, by Clinician Type, in Physician Offices (NAMCS)^*

Characteristic	Physician (Reference) (n = 12 170)	Nurse Practitioner Only (n = 70)	P Value	Nurse Practitioner and Physician (n = 103)	P Value	Physician Assistant Only (n = 82)	P Value	Physician Assistant and Physician (n = 218)	P Value
Mean age, y	48.9	47.0	0.36	46.8	0.37	47.1	0.29	49.1	0.91
Female	62.7	83.9		61.7		63.8		56.7	0.068†
Race/ethnicity									0.029†
White	76.9	85.5		71.9		70.4		79.3
Black	10.3	9.1		9.8		13.0		10.9
Hispanic	8.3	5.3		16.4		14.7		7.0
Other	4.6	0.01		1.9		1.9		2.8
Mean Charlson comorbidity count	0.047	0.049	0.95	0.032	0.41	0.047	NA	0.046	0.93
Acute symptoms	72.7	84.3		71.7		77.7		77.4	0.37†
Insurance									0.21†
Private	57.6	61.0		56.4		72.9		66.7
Medicare/Medicaid	26.7	21.4		21.6		15.3		23.2
Other	15.8	17.6		22.0		11.9		10.0
Provider identifies as PCP	79.9	88.5		70.9		68.9		65.5	0.193†
Urban location	82.1	54.1		61.3		79.2		95.4	0.011†
Region									0.062†
Northeast	19.1	13.9		12.8		19.5		19.0
Midwest	24.2	28.7		36.8		8.6		23.0
South	35.5	49.3		45.0		60.3		42.3
West	21.2	8.1		5.4		11.6		15.7
Symptom group
URIs	35.7	54.0		48.9		41.1		39.9	0.054†
Back pain	47.4	32.8		33.2		32.3		48.7	0.003†
Headache	18.2	15.9		22.2		26.7		12.5	0.108†

NA = not available; NAMCS = National Ambulatory Medical Care Survey; PCP = primary care provider; URI = upper respiratory infection.

^*Values are percentages unless otherwise indicated and may not sum to 100 due to rounding. Boldface values indicate comparisons with P < 0.05.

^†Represents global chi-square tests among all clinician subtypes.

Appendix Table 2.

Demographic Characteristics, by Clinician Type, in Hospital Clinics (NHAMCS)^*

Characteristic	Physician (Reference) (n = 13 359)	Nurse Practitioner Only (n = 1400)	P Value	Nurse Practitioner and Physician (n = 191)	P Value	Physician Assistant Only (n = 1152)	P Value	Physician Assistant and Physician (n = 204)	P Value
Mean age, y	45.0	42.2	0.003	46.3	0.58	42.1	0.006	44.2	0.63
Female	64.4	67.1		58.3		66.7		63.7	0.51†
Race/ethnicity									0.002†
White	70.6	81.4		59.2		82.0		67.2
Black	16.7	8.3		17.7		9.3		8.1
Hispanic	9.7	6.3		20.0		6.3		14.7
Other	3.0	4.0		3.1		2.5		10.0
Mean Charlson comorbidity count	0.049	0.021	<0.001	0.059	0.68	0.015	<0.001	0.039	0.70
Acute symptoms	75.0	89.6		75.4		88.7		81.2	<0.001†
Insurance									0.007†
Private	46.0	41.8		45.8		43.4		28.3
Medicare/Medicaid	33.4	34.5		37.4		28.7		41.2
Other	20.6	23.7		16.8		27.9		30.5
Provider identifies as PCP	52.8	59.3		28.0		49.6		38.0	0.027†
Urban location	81.7	51.2		83.9		39.5		74.3	<0.001†
Region									0.126
Northeast	17.6	19.4		12.9		14.4		28.1
Midwest	37.2	26.0		33.1		25.1		16.6
South	30.8	45.4		37.6		35.8		26.6
West	14.4	9.3		16.5		24.7		28.6
Symptom group
URIs	37.1	43.8		35.7		42.0		31.8	0.046†
Back pain	46.0	38.0		53.1		43.3		47.6	0.060†
Headache	18.1	20.0		12.9		16.2		20.6	0.178†

NHAMCS = National Hospital Ambulatory Medical Care Survey; PCP = primary care provider; URI = upper respiratory infection.

^*Values are percentages unless otherwise indicated and may not sum to 100 due to rounding. Boldface values indicate comparisons with P < 0.05.

^†Represents global chi-square tests among all clinician subtypes.

Appendix Table 3.

Unadjusted Proportions of Utilization, by Clinician Type, in the NAMCS Only^*

Variable	Physician (Reference) (n = 12 170)	Nurse Practitioner Only (n = 70)	P Value	Nurse Practitioner and Physician (n = 103)	P Value	Physician Assistant Only (n = 82)	P Value	Physician Assistant and Physician (n = 218)	P Value
Antibiotics	1988/4074(49.1)	15/32 (48.7)	0.96	18/47 (29.0)	0.057	23/34(61.0)	0.29	42/78 (59.4)	0.083
CT/MRI	511/8156(6.3)	0/39 (0)	NA	4/56 (8.8)	0.60	2/48 (2.2)	0.123	6/140 (2.9)	0.22
Radiography	987/10 093 (10.1)	8/61 (10.7)	0.92	10/88 (6.4)	0.28	8/62 (13.8)	0.45	19/188 (8.4)	0.62
Referral to other physician	1066/12 170 (8.1)	4/70 (8.8)	0.85	11/103(9.3)	0.75	7/82 (7.2)	0.82	22/218 (10.2)	0.46

CT = computed tomography; MRI = magnetic resonance imaging; NA = not available; NAMCS = National Ambulatory Medical Care Survey.

^*Values are sample numerator/sample denominator (percentages). Proportions may not match percentages because the percentages represent national estimates based on population weighting.

Appendix Table 4.

Unadjusted Proportions of Utilization, by Clinician Type, in the NHAMCS Only^*

Variable	Physician (Reference) (n = 13 359)	Nurse Practitioner Only (n = 1400)	P Value	Nurse Practitioner and Physician (n = 191)	P Value	Physician Assistant Only (n = 1130)	P Value	Physician Assistant and Physician (n = 204)	P Value
Antibiotics	2233/4742 (51.6)	335/637 (55.5)	0.35	28/64(48.4)	0.66	271/516(53.0)	0.73	31/64 (47.4)	0.66
CT/MRI	639/8666 (8.5)	41/771 (6.7)	0.38	12/128(18.3)	0.057	27/641 (7.0)	0.51	7/140(4.9)	0.20
Radiography	1390/11 015 (13.2)	140/1153 (11.9)	0.52	27/159(17.7)	0.30	102/964(9.8)	0.040	30/160(15.9)	0.46
Referral to other physician	2546/13 359 (18.4)	267/1400 (22.6)	0.46	53/191 (33.6)	0.017	207/1152 (17.7)	0.81	55/204 (25.9)	0.146

CT = computed tomography; MRI = magnetic resonance imaging; NHAMCS = National Hospital Ambulatory Medical Care Survey.

^*Values are sample numerator/sample denominator (percentages). Proportions may not match percentages because the percentages represent national estimates based on population weighting. Boldface values indicate comparisons with P < 0.05.

Appendix Table 5.

Multivariable-Adjusted Proportions of Utilization, by Clinician Type, in Physician Offices (NAMCS)^*

Variable	Physician (Reference) (n = 12 170)	Nurse Practitioner Only (n = 70)	P Value	Nurse Practitioner and Physician (n = 103)	P Value	Physician Assistant Only (n = 82)	P Value	Physician Assistant and Physician (n = 218)	P Value
Antibiotics	49.2	46.8	0.78	27.3	0.034	56.5	0.51	59.1	0.101
CT/MRI	5.7	0	NA	7.7	0.66	2.2	0.140	2.2	0.163
Radiography	9.8	10.1	0.95	6.5	0.34	12.6	0.56	7.6	0.52
Referral to other physician	7.4	8.2	0.83	8.5	0.74	6.8	0.88	0	0.62

CT = computed tomography; MRI = magnetic resonance imaging; NA = not available; NAMCS = National Ambulatory Medical Care Survey.

^*Values are percentages. Proportions may not match percentages because the percentages represent national estimates based on population weighting. Boldface values indicate comparisons with P < 0.05.

Appendix Table 6.

Multivariable-Adjusted Proportions of Utilization, by Clinician Type, in Hospital Clinics (NHAMCS)^*

Variable	Physician (Reference) (n = 13 359)	Nurse Practitioner Only (n = 1400)	P Value	Nurse Practitioner and Physician (n = 191)	P Value	Physician Assistant Only (n = 1152)	P Value	Physician Assistant and Physician (n = 204)	P Value
Antibiotics	51.9	52.7	0.88	48.1	0.56	52.8	0.83	47.7	0.70
CT/MRI	7.3	5.4	0.25	13.7	0.056	6.6	0.68	3.9	0.182
Radiography	12.2	12.0	0.91	13.5	0.68	10.6	0.31	15.0	0.45
Referral to other physician	14.8	19.4	0.22	23.5	0.044	16.6	0.38	19.5	0.25

CT = computed tomography; MRI = magnetic resonance imaging; NHAMCS = National Hospital Ambulatory Medical Care Survey.

^*Values are percentages. Proportions may not match percentages because the percentages represent national estimates based on population weighting. Boldface values indicate comparisons with P < 0.05.

Appendix Table 7.

Unadjusted Sensitivity Analysis of Proportions of Service Use Among Various Collaborative Practice Arrangements^*

Variable	Physicians Always Alone (Reference)	Nurse Practitioners				Physician Assistants				Nurse Practitioner Visits Alone or Shared	P Value	Physician Assistant Visits Alone or Shared	P Value	Practices With Too Few Visits to Be Categorized
		Always Alone	P Value	Always Shared	P Value	Always Alone	P Value	Always Shared	P Value
Antibiotics
Physician offices	1489/3019(49.1)	5/13(40.6)	0.50	18/53 (30.1)	0.093	22/34 (71.6)	0.020	63/130 (47.4)	0.78	0(0)	NA	0(0)	NA	489/1016(50.4)
Hospital clinics	203/454 (54.0)	590/1191 (54.4)	0.93	27/64 (43.8)	0.37	685/1367(52.7)	0.74	81/178 (46.7)	0.29	130/257 (54.6)	0.93	51/113(42.5)	0.182	1131/2399 (51.2)
CT/MRI
Physician offices	373/6078 (6.0)	0/25 (0)	NA	3/76 (3.2)	0.28	4/38 (9.9)	0.30	12/184 (6.8)	0.69	0(0)	NA	0(0)	NA	131/2038 (6.7)
Hospital clinics	43/789 (11.9)	140/2203 (7.1)	0.20	11/247 (2.8)	0.015	167/2161 (7.9)	0.27	20/255 (8.3)	0.46	33/332 (12.9)	0.85	14/203 (8.1)	0.54	298/4156 (8.4)
Radiography
Physician offices	723/7518(10.2)	2/28 (5.0)	0.34	9/92 (7.5)	0.41	6/60 (11.4)	0.71	28/267 (9.5)	0.75	0(0)	NA	0(0)	NA	264/2527 (10.2)
Hospital clinics	125/1029 (14.4)	343/2809 (13.2)	0.70	21/252 (9.4)	0.188	346/2909(11.8)	0.28	52/367(12.1)	0.57	96/493 (17.0)	0.62	34/236(15.3)	0.83	672/5356(12.6)
Referral to other physician
Physician offices	789/9052 (8.2)	1/37 (1.7)	0.104	7/128 (3.8)	0.113	6/72 (5.9)	0.52	30/314(8.6)	0.86	0(0)	NA	0(0)	NA	277/3040 (8.3)
Hospital clinics	206/1241 (16.2)	625/3385 (16.3)	0.98	50/311 (10.0)	0.28	625/3514(18.9)	0.56	86/433(13.3)	0.60	119/584 (27.5)	0.25	107/313(24.4)	0.34	1310/6525(20.5)

CT = computed tomography; MRI = magnetic resonance imaging; NA = not available; NAMCS = National Ambulatory Medical Care Survey; NHAMCS = National Hospital Ambulatory Medical Care Survey.

^*Values are sample numerator/sample denominator (percentages). Boldface values indicate comparisons with P < 0.05. Of note, categories are created on the basis of whether there are at least 4 observations occurring with each physician (NAMCS) or hospital (NHAMCS) clinic. The cutoff of 4 was chosen because it represents the median number of advanced practice clinician providers in each provider setting. For example, if a physician or hospital has >4 visits with nurse practitioners seeing patients alone (and 0 patients in shared visits), then this practice and its visits would fall under the first column (Nurse Practitioners Always Alone). Or if a practice has >4 nurse practitioner alone visits and >4 nurse practitioner shared visits, then we conclude that this practice allows for discretionary sharing. Also, percentages may differ from numerator/denominator proportion because of sample weighting. Finally, mixed-practice settings in this table occur only in the hospital clinic setting (NHAMCS data).